Toybrick

标题: TB-RK3399ProX Android8.1用rk_ssd_demo这个demo跑自己的模型时总是... [打印本页]

作者: fu-ai 时间: 2021-5-25 15:21
标题: TB-RK3399ProX Android8.1用rk_ssd_demo这个demo跑自己的模型时总是...
本帖最后由 fu-ai 于 2021-5-25 15:39 编辑

转换模型时的log信息如下  ↓ ↓ ↓

--> Config model
done
--> Loading model
W:tensorflow:From /home/toybrick/.local/lib/python3.7/site-packages/rknn/api/rknn.py:75: extract_sub_graph (from tensorflow.python.framework.graph_util_impl) is deprecated and will be removed in a future version.
Instructions for updating:
Use `tf.compat.v1.graph_util.extract_sub_graph`
W Network may not support batch > 1 !
W Network may not support batch > 1 !
W Network may not support batch > 1 !
W Network may not support batch > 1 !
W Network may not support batch > 1 !
W Network may not support batch > 1 !
W Network may not support batch > 1 !
W Network may not support batch > 1 !
W Network may not support batch > 1 !
W Network may not support batch > 1 !
W Network may not support batch > 1 !
W Network may not support batch > 1 !
/home/toybrick/.local/lib/python3.7/site-packages/rknn/api/rknn.py:75: FutureWarning: Possible nested set at position 7
  std_values=std_values, size_with_batch=size_with_batch)
done
--> Building model
W The target_platform is not set in config, using default target platform rk1808.
W:tensorflow:From /home/toybrick/.local/lib/python3.7/site-packages/tensorflow_core/python/framework/func_graph.py:915: py_func (from tensorflow.python.ops.script_ops) is deprecated and will be removed in a future version.
Instructions for updating:
tf.py_func is deprecated in TF V2. Instead, there are two
options available in V2.
- tf.py_function takes a python function which manipulates tf eager
tensors instead of numpy arrays. It's easy to convert a tf eager tensor to
an ndarray (just call tensor.numpy()) but having access to eager tensors
means `tf.py_function`s can use accelerators such as GPUs as well as
being differentiable using a gradient tape.
- tf.numpy_function maintains the semantics of the deprecated tf.py_func
(it is not differentiable, and manipulates numpy arrays). It drops the
stateful argument making all functions stateful.

W:tensorflow:From /home/toybrick/.local/lib/python3.7/site-packages/tensorflow_core/python/ops/resource_variable_ops.py:1630: calling BaseResourceVariable.__init__ (from tensorflow.python.ops.resource_variable_ops) with constraint is deprecated and will be removed in a future version.
Instructions for updating:
If using Keras pass *_constraint arguments to layers.
done
--> Export RKNN model
done
--> Init runtime environment
I NPUTransfer: Starting NPU Transfer Client, Transfer version 2.1.0 (b5861e7@2020-11-23T11:50:51)
D RKNNAPI: ==============================================
D RKNNAPI: RKNN VERSION:
D RKNNAPI: API: 1.6.0 (79320de build: 2020-12-29 10:57:09)
D RKNNAPI: DRV: 1.6.0 (159d2d3 build: 2021-01-12 15:23:09)
D RKNNAPI: ==============================================
done
--> Running model
D RKNNAPI: __can_use_fixed_point: use_fixed_point = 1.

done
--> Evaluate model performance
W When performing performance evaluation, inputs can be set to None to use fake inputs.
========================================================================
                           Performance
========================================================================
Total Time(us): 11979
FPS: 83.48
========================================================================

done

Android Studio 的 log信息如下  ↓ ↓ ↓

2021-01-06 13:06:23.981 2405-2405/? I/zygote64: Late-enabling -Xcheck:jni
2021-01-06 13:06:24.352 2405-2405/com.rockchip.gpadc.ssddemo D/ssd: Create /data/user/0/com.rockchip.gpadc.ssddemo/cache/ssd.rknn
2021-01-06 13:06:24.466 2405-2410/com.rockchip.gpadc.ssddemo I/zygote64: Do partial code cache collection, code=29KB, data=20KB
2021-01-06 13:06:24.466 2405-2410/com.rockchip.gpadc.ssddemo I/zygote64: After code cache collection, code=20KB, data=17KB
2021-01-06 13:06:24.466 2405-2410/com.rockchip.gpadc.ssddemo I/zygote64: Increasing code cache capacity to 128KB
2021-01-06 13:06:24.498 2405-2405/com.rockchip.gpadc.ssddemo D/ssd: Loaded box priors!
2021-01-06 13:06:24.499 2405-2405/com.rockchip.gpadc.ssddemo D/ssd: Loaded label!
2021-01-06 13:06:24.606 2405-2405/com.rockchip.gpadc.ssddemo V/ssd: Camera Supported Preview Size = 1280x720
2021-01-06 13:06:24.606 2405-2405/com.rockchip.gpadc.ssddemo V/ssd: Camera Supported Preview Size = 640x480
2021-01-06 13:06:24.623 2405-2444/com.rockchip.gpadc.ssddemo D/OpenGLRenderer: HWUI GL Pipeline
2021-01-06 13:06:24.660 2405-2444/com.rockchip.gpadc.ssddemo I/mali_so: [File] : hardware/arm/maliT760/driver/product/base/src/mali_base_kbase.c; [Line] : 947; [Func] : base_context_deal_with_version_affairs_rk_ext;
arm_release_ver of this mali_so is 'r18p0-01rel0', rk_so_ver is '11@0'.
2021-01-06 13:06:24.660 2405-2444/com.rockchip.gpadc.ssddemo I/mali_so: [File] : hardware/arm/maliT760/driver/product/base/src/mali_base_kbase.c; [Line] : 914; [Func] : determine_policy_against_patch_for_sf_memory_leak;
cmdline : com.rockchip.gpadc.ssddemo, we should GO THROUGH patch_for_sf_memory_leak.
2021-01-06 13:06:24.660 2405-2444/com.rockchip.gpadc.ssddemo D/mali_so: [File] : hardware/arm/maliT760/driver/product/base/src/mali_base_kbase.c; [Line] : 955; [Func] : base_context_deal_with_version_affairs_rk_ext;
current process is NOT sf, to bail out.
2021-01-06 13:06:24.672 2405-2444/com.rockchip.gpadc.ssddemo I/com.rockchip.gpadc.ssddemo: android::hardware::configstore::V1_0::ISurfaceFlingerConfigs::hasWideColorDisplay retrieved: 0
2021-01-06 13:06:24.672 2405-2444/com.rockchip.gpadc.ssddemo I/OpenGLRenderer: Initialized EGL, version 1.4
2021-01-06 13:06:24.672 2405-2444/com.rockchip.gpadc.ssddemo D/OpenGLRenderer: Swap behavior 2
2021-01-06 13:06:24.679 2405-2444/com.rockchip.gpadc.ssddemo D/mali_winsys: EGLint new_window_surface(egl_winsys_display *, void *, EGLSurface, EGLConfig, egl_winsys_surface **, egl_color_buffer_format *, EGLBoolean) returns 0x3000
2021-01-06 13:06:24.696 2405-2430/com.rockchip.gpadc.ssddemo D/mali_winsys: EGLint new_window_surface(egl_winsys_display *, void *, EGLSurface, EGLConfig, egl_winsys_surface **, egl_color_buffer_format *, EGLBoolean) returns 0x3000
2021-01-06 13:06:24.802 2405-2458/com.rockchip.gpadc.ssddemo I/rkssd4j: try rknn_init!
2021-01-06 13:06:24.823 2405-2405/com.rockchip.gpadc.ssddemo I/Thread-3: type=1400 audit(0.0:120): avc: denied { write } for name="transfer_proxy" dev="tmpfs" ino=9963 scontext=u:r:untrusted_app:s0:c512,c768 tcontext=u

bject_r:socket_device:s0 tclass=sock_file permissive=1
2021-01-06 13:06:24.823 2405-2405/com.rockchip.gpadc.ssddemo I/Thread-3: type=1400 audit(0.0:121): avc: denied { connectto } for path="/dev/socket/transfer_proxy" scontext=u:r:untrusted_app:s0:c512,c768 tcontext=u:r:npu_transfer_proxy:s0 tclass=unix_stream_socket permissive=1
2021-01-06 13:06:27.723 2405-2410/com.rockchip.gpadc.ssddemo I/zygote64: Do partial code cache collection, code=49KB, data=55KB
2021-01-06 13:06:27.724 2405-2410/com.rockchip.gpadc.ssddemo I/zygote64: After code cache collection, code=43KB, data=53KB
2021-01-06 13:06:27.724 2405-2410/com.rockchip.gpadc.ssddemo I/zygote64: Increasing code cache capacity to 256KB
2021-01-06 13:06:27.724 2405-2410/com.rockchip.gpadc.ssddemo I/zygote64: Compiler allocated 6MB to compile void android.view.ViewRootImpl.performTraversals()
2021-01-06 13:06:33.444 2405-2410/com.rockchip.gpadc.ssddemo I/zygote64: Do full code cache collection, code=124KB, data=90KB
2021-01-06 13:06:33.445 2405-2410/com.rockchip.gpadc.ssddemo I/zygote64: After code cache collection, code=94KB, data=63KB
2021-01-06 13:06:35.605 2405-2458/com.rockchip.gpadc.ssddemo D/RKNNAPI: ==============================================
2021-01-06 13:06:35.605 2405-2458/com.rockchip.gpadc.ssddemo D/RKNNAPI: RKNN VERSION:
2021-01-06 13:06:35.605 2405-2458/com.rockchip.gpadc.ssddemo D/RKNNAPI: API: 1.6.0 (159d2d3 build: 2021-01-22 11:15:48)
2021-01-06 13:06:35.605 2405-2458/com.rockchip.gpadc.ssddemo D/RKNNAPI: DRV: 1.5.0 (83d70a9 build: 2020-09-30 16:25:49)
2021-01-06 13:06:35.605 2405-2458/com.rockchip.gpadc.ssddemo D/RKNNAPI: ==============================================
2021-01-06 13:06:35.612 2405-2458/com.rockchip.gpadc.ssddemo I/rkssd4j: rknn_init success!
2021-01-06 13:06:35.614 2405-2458/com.rockchip.gpadc.ssddemo D/RKNNAPI: __can_use_fixed_point: use_fixed_point = 1.
2021-01-06 13:06:35.617 2405-2458/com.rockchip.gpadc.ssddemo E/RKNNAPI: rknn_outputs_get,  outputs[0].size = 15336! expect 30672!
2021-01-06 13:06:35.617 2405-2458/com.rockchip.gpadc.ssddemo E/rkssd4j: rknn_outputs_get fail! ret=-5
2021-01-06 13:06:35.621 2405-2458/com.rockchip.gpadc.ssddemo E/RKNNAPI: rknn_outputs_get,  outputs[0].size = 15336! expect 30672!
2021-01-06 13:06:35.622 2405-2458/com.rockchip.gpadc.ssddemo E/rkssd4j: rknn_outputs_get fail! ret=-5
2021-01-06 13:06:35.630 2405-2458/com.rockchip.gpadc.ssddemo E/RKNNAPI: rknn_outputs_get,  outputs[0].size = 15336! expect 30672!
2021-01-06 13:06:35.630 2405-2458/com.rockchip.gpadc.ssddemo E/rkssd4j: rknn_outputs_get fail! ret=-5
2021-01-06 13:06:35.648 2405-2405/com.rockchip.gpadc.ssddemo D/ssd: something may wrong! validCount=1917
2021-01-06 13:06:35.661 2405-2405/com.rockchip.gpadc.ssddemo D/ssd: something may wrong! validCount=1917
2021-01-06 13:06:35.663 2405-2458/com.rockchip.gpadc.ssddemo W/RKNNAPI: rknn_run,  is about to block until rknn_outputs_get is called!
2021-01-06 13:06:40.776 2405-2410/com.rockchip.gpadc.ssddemo I/zygote64: Do partial code cache collection, code=124KB, data=83KB
2021-01-06 13:06:40.777 2405-2410/com.rockchip.gpadc.ssddemo I/zygote64: After code cache collection, code=120KB, data=81KB
2021-01-06 13:06:40.777 2405-2410/com.rockchip.gpadc.ssddemo I/zygote64: Increasing code cache capacity to 512KB
2021-01-06 13:06:54.158 2405-2405/com.rockchip.gpadc.ssddemo I/ssd: stopped camera

请问大佬，这个问题怎么解决？？
推理图如图

作者: jefferyzhang 时间: 2021-5-25 16:16
如果只是DRV差异那就简单了，直接升级下Android的rknn的driver即可

作者: fu-ai 时间: 2021-5-25 16:18
不是，我也跑了一下你们的rk_ssd_demo，只是模型不同

作者: fu-ai 时间: 2021-5-25 16:23
我跑你们的rk_ssd_demo，用你们的模型可以显示检测框和fps，但是换成我自己的模型、coco_labels_list.txt 和 box_priors.txt 后虽然可以打开app，但是fps显示0和无检测框。

作者: 395876134 时间: 2021-5-25 16:23
http://t.rock-chips.com/forum.ph ... tid=3087&extra=

作者: fu-ai 时间: 2021-5-25 16:27
照理说如果是DRV差异，那么你们官方的模型应该也不会出现检测框？？还是说我这边把模型、coco_labels_list.txt 和 box_priors.txt这三个东西发给你那边看看？？

作者: jefferyzhang 时间: 2021-5-25 16:31

fu-ai 发表于 2021-5-25 16:27
照理说如果是DRV差异，那么你们官方的模型应该也不会出现检测框？？还是说我这边把模型、coco_labels_list ...

转换模型用的rknntoolkit版本必须和drv是匹配的。我们的模型并不一定是1.6 rknntoolkit转换的。不用纠结这个。
你只要确定你的模型linux下1.6可用，android下1.5不可用，那就是drv版本问题了

作者: fu-ai 时间: 2021-5-25 16:34
如果照你这么说，那就是drv的问题了，那我该怎么操作，在Debian10还是Android上？

作者: jefferyzhang 时间: 2021-5-25 16:42

fu-ai 发表于 2021-5-25 16:34
如果照你这么说，那就是drv的问题了，那我该怎么操作，在Debian10还是Android上？ ...

http://t.rock-chips.com/forum.ph ... &extra=page%3D1

直接在android上更新一把试试

作者: fu-ai 时间: 2021-5-25 16:45
我没弄过Android底层，不知道这个怎么操作 adb root && adb remount && adb push npu_fw /vendor/etc/ && adb reboot ，请大佬点拨点拨

作者: fu-ai 时间: 2021-5-25 17:22

jefferyzhang 发表于 2021-5-25 16:42
http://t.rock-chips.com/forum.php?mod=viewthread&tid=3087&extra=page%3D1

直接在android上更新一把 ...

大佬，请问这是要在双系统下操作么

作者: fu-ai 时间: 2021-5-25 19:18

jefferyzhang 发表于 2021-5-25 16:42
http://t.rock-chips.com/forum.php?mod=viewthread&tid=3087&extra=page%3D1

直接在android上更新一把 ...

大佬，更新了还是不行，log如下 ↓ ↓ ↓

2021-01-06 14:47:00.030 2272-2272/? I/zygote64: Late-enabling -Xcheck:jni
2021-01-06 14:47:00.375 2272-2277/com.rockchip.gpadc.ssddemo I/zygote64: Do partial code cache collection, code=30KB, data=18KB
2021-01-06 14:47:00.375 2272-2277/com.rockchip.gpadc.ssddemo I/zygote64: After code cache collection, code=21KB, data=16KB
2021-01-06 14:47:00.375 2272-2277/com.rockchip.gpadc.ssddemo I/zygote64: Increasing code cache capacity to 128KB
2021-01-06 14:47:00.394 2272-2272/com.rockchip.gpadc.ssddemo D/ssd: Loaded box priors!
2021-01-06 14:47:00.395 2272-2272/com.rockchip.gpadc.ssddemo D/ssd: Loaded label!
2021-01-06 14:47:00.512 2272-2272/com.rockchip.gpadc.ssddemo V/ssd: Camera Supported Preview Size = 1280x720
2021-01-06 14:47:00.512 2272-2272/com.rockchip.gpadc.ssddemo V/ssd: Camera Supported Preview Size = 640x480
2021-01-06 14:47:00.538 2272-2312/com.rockchip.gpadc.ssddemo D/OpenGLRenderer: HWUI GL Pipeline
2021-01-06 14:47:00.574 2272-2312/com.rockchip.gpadc.ssddemo I/mali_so: [File] : hardware/arm/maliT760/driver/product/base/src/mali_base_kbase.c; [Line] : 947; [Func] : base_context_deal_with_version_affairs_rk_ext;
arm_release_ver of this mali_so is 'r18p0-01rel0', rk_so_ver is '11@0'.
2021-01-06 14:47:00.575 2272-2312/com.rockchip.gpadc.ssddemo I/mali_so: [File] : hardware/arm/maliT760/driver/product/base/src/mali_base_kbase.c; [Line] : 914; [Func] : determine_policy_against_patch_for_sf_memory_leak;
cmdline : com.rockchip.gpadc.ssddemo, we should GO THROUGH patch_for_sf_memory_leak.
2021-01-06 14:47:00.575 2272-2312/com.rockchip.gpadc.ssddemo D/mali_so: [File] : hardware/arm/maliT760/driver/product/base/src/mali_base_kbase.c; [Line] : 955; [Func] : base_context_deal_with_version_affairs_rk_ext;
current process is NOT sf, to bail out.
2021-01-06 14:47:00.585 2272-2312/com.rockchip.gpadc.ssddemo I/com.rockchip.gpadc.ssddemo: android::hardware::configstore::V1_0::ISurfaceFlingerConfigs::hasWideColorDisplay retrieved: 0
2021-01-06 14:47:00.586 2272-2312/com.rockchip.gpadc.ssddemo I/OpenGLRenderer: Initialized EGL, version 1.4
2021-01-06 14:47:00.586 2272-2312/com.rockchip.gpadc.ssddemo D/OpenGLRenderer: Swap behavior 2
2021-01-06 14:47:00.591 2272-2312/com.rockchip.gpadc.ssddemo D/mali_winsys: EGLint new_window_surface(egl_winsys_display *, void *, EGLSurface, EGLConfig, egl_winsys_surface **, egl_color_buffer_format *, EGLBoolean) returns 0x3000
2021-01-06 14:47:00.604 2272-2298/com.rockchip.gpadc.ssddemo D/mali_winsys: EGLint new_window_surface(egl_winsys_display *, void *, EGLSurface, EGLConfig, egl_winsys_surface **, egl_color_buffer_format *, EGLBoolean) returns 0x3000
2021-01-06 14:47:00.648 2272-2325/com.rockchip.gpadc.ssddemo I/rkssd4j: try rknn_init!
2021-01-06 14:47:00.663 2272-2272/com.rockchip.gpadc.ssddemo I/Thread-3: type=1400 audit(0.0:122): avc: denied { connectto } for path="/dev/socket/transfer_proxy" scontext=u:r:untrusted_app:s0:c512,c768 tcontext=u:r:npu_transfer_proxy:s0 tclass=unix_stream_socket permissive=1
2021-01-06 14:47:03.690 2272-2277/com.rockchip.gpadc.ssddemo I/zygote64: Do partial code cache collection, code=47KB, data=55KB
2021-01-06 14:47:03.691 2272-2277/com.rockchip.gpadc.ssddemo I/zygote64: After code cache collection, code=42KB, data=53KB
2021-01-06 14:47:03.691 2272-2277/com.rockchip.gpadc.ssddemo I/zygote64: Increasing code cache capacity to 256KB
2021-01-06 14:47:03.692 2272-2277/com.rockchip.gpadc.ssddemo I/zygote64: Compiler allocated 6MB to compile void android.view.ViewRootImpl.performTraversals()
2021-01-06 14:47:06.759 2272-2277/com.rockchip.gpadc.ssddemo I/zygote64: Do full code cache collection, code=118KB, data=92KB
2021-01-06 14:47:06.760 2272-2277/com.rockchip.gpadc.ssddemo I/zygote64: After code cache collection, code=91KB, data=65KB
2021-01-06 14:47:12.044 2272-2325/com.rockchip.gpadc.ssddemo D/RKNNAPI: ==============================================
2021-01-06 14:47:12.044 2272-2325/com.rockchip.gpadc.ssddemo D/RKNNAPI: RKNN VERSION:
2021-01-06 14:47:12.044 2272-2325/com.rockchip.gpadc.ssddemo D/RKNNAPI: API: 1.6.0 (159d2d3 build: 2021-01-22 11:15:48)
2021-01-06 14:47:12.044 2272-2325/com.rockchip.gpadc.ssddemo D/RKNNAPI: DRV: 1.6.0 (159d2d3 build: 2021-01-12 15:23:09)
2021-01-06 14:47:12.044 2272-2325/com.rockchip.gpadc.ssddemo D/RKNNAPI: ==============================================
2021-01-06 14:47:12.051 2272-2325/com.rockchip.gpadc.ssddemo I/rkssd4j: rknn_init success!
2021-01-06 14:47:12.053 2272-2325/com.rockchip.gpadc.ssddemo D/RKNNAPI: __can_use_fixed_point: use_fixed_point = 1.
2021-01-06 14:47:12.055 2272-2325/com.rockchip.gpadc.ssddemo E/RKNNAPI: rknn_outputs_get,  outputs[0].size = 15336! expect 30672!
2021-01-06 14:47:12.055 2272-2325/com.rockchip.gpadc.ssddemo E/rkssd4j: rknn_outputs_get fail! ret=-5
2021-01-06 14:47:12.058 2272-2325/com.rockchip.gpadc.ssddemo E/RKNNAPI: rknn_outputs_get,  outputs[0].size = 15336! expect 30672!
2021-01-06 14:47:12.058 2272-2325/com.rockchip.gpadc.ssddemo E/rkssd4j: rknn_outputs_get fail! ret=-5
2021-01-06 14:47:12.061 2272-2325/com.rockchip.gpadc.ssddemo E/RKNNAPI: rknn_outputs_get,  outputs[0].size = 15336! expect 30672!
2021-01-06 14:47:12.061 2272-2325/com.rockchip.gpadc.ssddemo E/rkssd4j: rknn_outputs_get fail! ret=-5
2021-01-06 14:47:12.098 2272-2272/com.rockchip.gpadc.ssddemo D/ssd: something may wrong! validCount=1917
2021-01-06 14:47:12.105 2272-2325/com.rockchip.gpadc.ssddemo W/RKNNAPI: rknn_run,  is about to block until rknn_outputs_get is called!
2021-01-06 14:47:12.111 2272-2277/com.rockchip.gpadc.ssddemo I/zygote64: Do partial code cache collection, code=123KB, data=88KB
2021-01-06 14:47:12.111 2272-2277/com.rockchip.gpadc.ssddemo I/zygote64: After code cache collection, code=123KB, data=88KB
2021-01-06 14:47:12.111 2272-2277/com.rockchip.gpadc.ssddemo I/zygote64: Increasing code cache capacity to 512KB
2021-01-06 14:47:13.214 2272-2272/com.rockchip.gpadc.ssddemo I/ssd: stopped camera

作者: jefferyzhang 时间: 2021-5-26 08:47
你到底在仿真里跑通过没有，得到的结果是否正确的。你这里输出纬度都不一样，仿真怎么跑过的？
E/RKNNAPI: rknn_outputs_get, outputs[0].size = 15336! expect 30672!

作者: fu-ai 时间: 2021-5-26 10:15

jefferyzhang 发表于 2021-5-26 08:47
你到底在仿真里跑通过没有，得到的结果是否正确的。你这里输出纬度都不一样，仿真怎么跑过的？
E/RKNNAPI: ...

我在板子上跑通了test.py，并出图了，不知道这个是不是你说的仿真

作者: fu-ai 时间: 2021-5-26 10:19
之前我跑test.py的时候，也出现过类似这个问题，不过那时是1917! expect 3834!的样子，后来把test.py里的类别数由1改成2（我的模型只有一个类别）的时候，就正常跑通并出图

作者: jefferyzhang 时间: 2021-5-26 10:21

fu-ai 发表于 2021-5-26 10:19
之前我跑test.py的时候，也出现过类似这个问题，不过那时是1917! expect 3834!的样子，后来把test.py里的类 ...

拜托你先了解下你的ssd输出数量和维度在搞吧，出图就叫跑通了我也是无语了，python不会出现内存溢出，c++这么搞肯定会出问题了。

作者: fu-ai 时间: 2021-5-26 10:25

jefferyzhang 发表于 2021-5-26 10:21
拜托你先了解下你的ssd输出数量和维度在搞吧，出图就叫跑通了我也是无语了，python不会出现内存溢出，c++ ...

那我怎么看输出数量和维度？

作者: fu-ai 时间: 2021-5-26 10:28
这是我用的test.py，需要更改么

import numpy as np

import re
import math
import random
import cv2

from rknn.api import RKNN

INPUT_SIZE = 300

NUM_RESULTS = 1917
NUM_CLASSES = 2# num+1

Y_SCALE = 10.0
X_SCALE = 10.0
H_SCALE = 5.0
W_SCALE = 5.0

def expit(x):
return 1. / (1. + math.exp(-x))

def unexpit(y):
return -1.0 * math.log((1.0 / y) - 1.0);

def CalculateOverlap(xmin0, ymin0, xmax0, ymax0, xmin1, ymin1, xmax1, ymax1):
w = max(0.0, min(xmax0, xmax1) - max(xmin0, xmin1))
h = max(0.0, min(ymax0, ymax1) - max(ymin0, ymin1))
i = w * h
u = (xmax0 - xmin0) * (ymax0 - ymin0) + (xmax1 - xmin1) * (ymax1 - ymin1) - i

if u <= 0.0:
      return 0.0

return i / u

def load_box_priors():
box_priors_ = []
fp = open('./box_priors.txt', 'r')
ls = fp.readlines()
for s in ls:
      aList = re.findall('([-+]?\d+(\.\d*)?|\.\d+)([eE][-+]?\d+)?', s)
      for ss in aList:
         aNum = float((ss[0]+ss[2]))
         box_priors_.append(aNum)
fp.close()

box_priors = np.array(box_priors_)
box_priors = box_priors.reshape(4, NUM_RESULTS)

return box_priors

if __name__ == '__main__':

# Create RKNN object
rknn = RKNN()

# Config for Model Input PreProcess
print('--> Config model')
rknn.config(mean_values=[[127.5, 127.5, 127.5]], std_values=[[127.5, 127.5, 127.5]], reorder_channel='0 1 2')
print('done')

# Load TensorFlow Model
print('--> Loading model')
ret = rknn.load_tensorflow(tf_pb='./tflite_graph.pb',
                           inputs=['normalized_input_image_tensor'],
                           outputs=['concat', 'concat_1'],
                           input_size_list=[[INPUT_SIZE, INPUT_SIZE, 3]])
if ret != 0:
      print('Load model failed!')
      exit(ret)
print('done')

# Build Model
print('--> Building model')
ret = rknn.build(do_quantization=True, dataset='./dataset.txt')
if ret != 0:
      print('Build model failed!')
      exit(ret)
print('done')

# Export RKNN Model
print('--> Export RKNN model')
rknn.export_rknn('./ssd.rknn')
if ret != 0:
      print('Export RKNN model failed!')
      exit(ret)
print('done')

# Direct Load RKNN Model
# rknn.load_rknn('./ssd.rknn')

# Set inputs
orig_img = cv2.imread('./horizontal.bmp')
img = cv2.cvtColor(orig_img, cv2.COLOR_BGR2RGB)
img = cv2.resize(img, (INPUT_SIZE, INPUT_SIZE), interpolation=cv2.INTER_CUBIC)

# init runtime environment
print('--> Init runtime environment')
ret = rknn.init_runtime()
if ret != 0:
      print('Init runtime environment failed')
      exit(ret)
print('done')

# Inference
print('--> Running model')
outputs = rknn.inference(inputs=[img])
print('done')

predictions = outputs[0].reshape((1, NUM_RESULTS, 4))
outputClasses = outputs[1].reshape((1, NUM_RESULTS, NUM_CLASSES))
candidateBox = np.zeros([2, NUM_RESULTS], dtype=int)
vaildCnt = 0

box_priors = load_box_priors()

# Post Process
# got valid candidate box
for i in range(0, NUM_RESULTS):
      topClassScore = -1000
      topClassScoreIndex = -1

      # Skip the first catch-all class.
      for j in range(1, NUM_CLASSES):
         score = expit(outputClasses[0][i][j]);

         if score > topClassScore:
            topClassScoreIndex = j
            topClassScore = score

      if topClassScore > 0.4:
         candidateBox[0][vaildCnt] = i
         candidateBox[1][vaildCnt] = topClassScoreIndex
         vaildCnt += 1

# calc position
for i in range(0, vaildCnt):
      if candidateBox[0][i] == -1:
         continue

      n = candidateBox[0][i]
      ycenter = predictions[0][n][0] / Y_SCALE * box_priors[2][n] + box_priors[0][n]
      xcenter = predictions[0][n][1] / X_SCALE * box_priors[3][n] + box_priors[1][n]
      h = math.exp(predictions[0][n][2] / H_SCALE) * box_priors[2][n]
      w = math.exp(predictions[0][n][3] / W_SCALE) * box_priors[3][n]

      ymin = ycenter - h / 2.
      xmin = xcenter - w / 2.
      ymax = ycenter + h / 2.
      xmax = xcenter + w / 2.

      predictions[0][n][0] = ymin
      predictions[0][n][1] = xmin
      predictions[0][n][2] = ymax
      predictions[0][n][3] = xmax

# NMS
for i in range(0, vaildCnt):
      if candidateBox[0][i] == -1:
         continue

      n = candidateBox[0][i]
      xmin0 = predictions[0][n][1]
      ymin0 = predictions[0][n][0]
      xmax0 = predictions[0][n][3]
      ymax0 = predictions[0][n][2]

      for j in range(i+1, vaildCnt):
         m = candidateBox[0][j]

         if m == -1:
            continue

         xmin1 = predictions[0][m][1]
         ymin1 = predictions[0][m][0]
         xmax1 = predictions[0][m][3]
         ymax1 = predictions[0][m][2]

         iou = CalculateOverlap(xmin0, ymin0, xmax0, ymax0, xmin1, ymin1, xmax1, ymax1)

         if iou >= 0.45:
            candidateBox[0][j] = -1

# Draw result
for i in range(0, vaildCnt):
      if candidateBox[0][i] == -1:
         continue

      n = candidateBox[0][i]

      xmin = max(0.0, min(1.0, predictions[0][n][1])) * INPUT_SIZE
      ymin = max(0.0, min(1.0, predictions[0][n][0])) * INPUT_SIZE
      xmax = max(0.0, min(1.0, predictions[0][n][3])) * INPUT_SIZE
      ymax = max(0.0, min(1.0, predictions[0][n][2])) * INPUT_SIZE

      # print("%d @ (%d, %d) (%d, %d) score=%f" % (topClassScoreIndex, xmin, ymin, xmax, ymax, topClassScore))
      cv2.rectangle(orig_img, (int(xmin), int(ymin)), (int(xmax), int(ymax)),
                  (random.random()*255, random.random()*255, random.random()*255), 3)

cv2.imwrite("out.jpg", orig_img)

# Evaluate Perf on Simulator
print('--> Evaluate model performance')
rknn.eval_perf(inputs=[img], is_print=True)
print('done')

# Release RKNN Context
rknn.release()

作者: fu-ai 时间: 2021-5-26 11:11
版主，是不是我这里要修改？
predictions = outputs[0].reshape((1, NUM_RESULTS, 4))
outputClasses = outputs[1].reshape((1, NUM_RESULTS, NUM_CLASSES))
candidateBox = np.zeros([2, NUM_RESULTS], dtype=int)

作者: fu-ai 时间: 2021-5-26 11:39
版主，你在吗？

作者: fu-ai 时间: 2021-5-26 17:25
提示: 该帖被管理员或版主屏蔽

欢迎光临 Toybrick (https://t.rock-chips.com/)