Toybrick

Toybrick»开源社区 › 技术论坛 › 人工智能 › rknn-toolkit & rknn-api for Toybrick
楼主: jefferyzhang

rknn-toolkit & rknn-api for Toybrick

 

Lubagov

注册会员

积分
132
QQ
楼主
发表于 2021-3-2 03:13:59 | 显示全部楼层
I understand that you are not ASUS, but I just don't know where to ask questions. Although I do not exclude that you will ban me without answering:

In version 1.4 from your GitHub there is an example with SSD ssd.rknn. It works well with API 1.4.0, and version 1.3.2 of the NPU firmware, which is from ASUS.

1. Should it work, with firmware versions 1.4, 1.5, 1.6, (I found them in your repositories, and your firmwares). If I'm just flashing the NPU?

2. Do I need to convert the model again for the target API version and NPU firmware? Let's say ssd.rknn from API 1.4 if I'm using firmware 1.5 or 1.6. Do I need to convert the model using the appropriate RKNN-Toolkit? Or is it backward compatible?

3. Is there a possibility that your firmware for NPU, which is posted on Github, is not compatible with ASUS Tinker Edge R. After all, in principle, this is the same CPU, NPU?

4. Interestingly, if you look, the contents of `rootfs` from Debain, then there` librknn_api.so` has version 1.4. However, the NPU firmware is version 1.6. this is normal? I just don't see a newer version of lubrknn_api.so anywhere. Is it compatible? I saw in the answers that the API and driver versions should be the same.

So far, I was able to download this model, only in these versions, both on Android and Linux (librknn_api.so from your repositories / firmware, NPU firmware (driver) from ASUS):
D RKNNAPI: ============================================
D RKNNAPI: RKNN VERSION:
D RKNNAPI: API: 1.4.0 (bbe0dfc build: 2020-09-14 14:06:05)
D RKNNAPI: DRV: 1.3.2 (f113d64 build: 2020-04-23 09:35:26)
D RKNNAPI: ============================================
Otherwise, I have errors.

I really hope for an answer, thanks!
回复

使用道具 举报

Lubagov

注册会员

积分
132
QQ
沙发
发表于 2021-3-2 19:50:42 | 显示全部楼层
本帖最后由 Lubagov 于 2021-3-2 20:44 编辑

I was glad to receive an answer, thank you very much!

It seems ASUS is not going to update the API at all. What I have is version 1.3.0 released a year ago in Debain 9, and 1.3.2 in Android.
Even though the firmware was updated in December.

I continue to play with this Debian rootfs, and if use form Debian 10 (NPU FW from ASUS, no other seems to work with it):

If i compile app over with preinstalled libs:
  1. gcc ./main.c -L. -lrknn_api -lpthread
复制代码

  2. D RKNNAPI: ==============================================
  3. D RKNNAPI: RKNN VERSION:
  4. D RKNNAPI:   API: 1.4.0 (bbe0dfc build: 2020-09-14 14:06:05)
  5. D RKNNAPI:   DRV: 1.3.2 (f113d64 build: 2020-04-23 09:35:26)
  6. D RKNNAPI: ==============================================
复制代码
and when i check:

  2. $ strings /usr/lib/aarch64-linux-gnu/librknn_api.so | grep build:
  3. (bbe0dfc build: 2020-09-14 14:06:05
复制代码

When i use Python:

  2. D RKNNAPI: ==============================================
  3. D RKNNAPI: RKNN VERSION:
  4. D RKNNAPI:   API: 1.6.0 (79320de build: 2020-12-29 10:57:09)
  5. D RKNNAPI:   DRV: 1.3.2 (f113d64 build: 2020-04-23 09:35:26)
  6. D RKNNAPI: ==============================================
复制代码

Have small fix it for it can update/download packages fine.
It seems you didn't really want it to be used this image on other boards.
But I am think packages have been updated correctly from your repository.


By the way, the guide in the first post seems a little outdated:
If I execute this command, it becomes very bad:


  2. pip3 install --user -U tensorflow == 1.14.0
复制代码

Version is already preinstalled, 2.0.0 and it works, version 1.14.0 does not work.

But it is not solve my problem, in any case, i have same problem when call `rknn_init`, with any other firmware expect 1.3.2 from ASUS:

in c:

  2. toybrick@debian10:~/test_npu/comp$ ./a.out
  3. E RKNNAPI: rknn_init,  recv(MsgLoadAck) fail, -9(ERROR_PIPE) != 368!
  4. rknn_init fail! ret=-3
  5. Segmentation fault
复制代码

in Python:

  2. toybrick@debian10:~/test_npu$ python3 testapi.py
  3. --> Loading RKNN model
  4. done
  5. I NPUTransfer: Starting NPU Transfer Client, Transfer version 2.1.0 (b5861e7@2020-11-23T11:50:51)
  6. E RKNNAPI: rknn_init,  recv(MsgLoadAck) fail, -9(ERROR_PIPE) != 368!
  7. E Catch exception when init runtime!
  8. E Traceback (most recent call last):
  9. E   File "rknn/api/rknn_base.py", line 844, in rknn.api.rknn_base.RKNNBase.init_runtime
  10. E   File "rknn/api/rknn_runtime.py", line 340, in rknn.api.rknn_runtime.RKNNRuntime.build_graph
  11. E Exception: RKNN init failed. error code: RKNN_ERR_DEVICE_UNAVAILABLE
  12. --> Running model
  13. W We need do some initial work, it will increase call time of inference for the first time.
  14. I NPUTransfer: Starting NPU Transfer Client, Transfer version 2.1.0 (b5861e7@2020-11-23T11:50:51)
  15. E NPUTransfer: Transfer interface open failed!, ret = -1
  16. E RKNNAPI: rknn_init,  driver open fail!  ret = -1(ERROR_IO)!
  17. E Catch exception when init model!
  18. E Traceback (most recent call last):
  19. E   File "rknn/api/rknn_base.py", line 931, in rknn.api.rknn_base.RKNNBase.inference
  20. E   File "rknn/api/rknn_runtime.py", line 340, in rknn.api.rknn_runtime.RKNNRuntime.build_graph
  21. E Exception: RKNN init failed. error code: RKNN_ERR_DEVICE_UNAVAILABLE
  22. done
复制代码dmesg output in NPU:

  2. [  873.529384] android_work: sent uevent USB_STATE=DISCONNECTED
  3. [  873.530761] configfs-gadget gadget: unbind function 'Function FS Gadget'/ffffffc0052c1e38
  4. [  873.530845] configfs-gadget gadget: unbind function 'Function FS Gadget'/ffffffc0052c1c38
  5. [  873.531441] read descriptors
  6. [  873.531466] read strings
  7. [  874.545684] configfs-gadget fd000000.dwc3: failed to start rockchip: -19
  8. [  874.573016] read descriptors
  9. [  874.573057] read strings
  10. [  874.579310] android_work: did not send uevent (0 0           (null))
  11. [  874.706836] android_work: did not send uevent (0 0           (null))
  12. [  874.788465] android_work: sent uevent USB_STATE=CONNECTED
  13. [  874.819649] configfs-gadget gadget: super-speed config #1: b
  14. [  874.819883] android_work: sent uevent USB_STATE=CONFIGURED
复制代码
回复

使用道具 举报

Lubagov

注册会员

积分
132
QQ
板凳
发表于 2021-3-4 21:34:23 | 显示全部楼层
本帖最后由 Lubagov 于 2021-3-5 00:57 编辑

Unpacked from the NPU firmware, boot.img into parts via `imgrepackrk`. And also unpacked the 1.6 firmware from Debain 10 from Toybrick. Replaced the ramdisk in the ASUS firmware with the ramdisk from the Toybrick firmware. Then, I packed it back. those. The kernel and resources are from ASUS, and the ramdisk is from Toybrick. And it works at least `rknn_init` fine call.Better of cose build it from source somehow, and make it working.. Only not clear how to build kernel, working on ASUS...

I NPUTransfer: Starting NPU Transfer Client, Transfer version 2.1.0 (b5861e7@2020-11-23T11:50:51)
D RKNNAPI: ==============================================
D RKNNAPI: RKNN VERSION:
D RKNNAPI:   API: 1.6.0 (79320de build: 2020-12-29 10:57:09)
D RKNNAPI:   DRV: 1.6.0 (159d2d3 build: 2021-01-12 15:23:09)
D RKNNAPI: ==============================================

/=============
I am trying to run the YOLO5 example. I understand correctly that `/dev/card/dri0` should be on the NPU device, but only with the new kernel?
Or, is it on RK1808 but not on RK3399Pro?


  2. # ./run_rk180x.sh
  3. Read model/test2.jpg ...
  4. img width = 1280, img height = 720
  5. Loading mode...
  6. failed to open /dev/dri/card0
  7. sdk version: librknn_runtime version 1.6.0 (096015c build: 2021-01-12 15:25:30 base: 1126) driver version: 6.4.3.5.293908
  8. model input num: 1, output num: 3
  9. index=0 name=images_165 n_dims=4 dims=[1 3 640 640] n_elems=1228800 size=1228800 fmt=0 type=3 qnt_type=2 fl=0 zp=0 scale=0.003922
  10. index=0 name=Conv_Conv_159/out0_0 n_dims=4 dims=[1 255 80 80] n_elems=1632000 size=1632000 fmt=0 type=3 qnt_type=2 fl=-69 zp=187 scale=0.127843
  11. index=1 name=Conv_Conv_160/out0_1 n_dims=4 dims=[1 255 40 40] n_elems=408000 size=408000 fmt=0 type=3 qnt_type=2 fl=-74 zp=182 scale=0.113217
  12. index=2 name=Conv_Conv_161/out0_2 n_dims=4 dims=[1 255 20 20] n_elems=102000 size=102000 fmt=0 type=3 qnt_type=2 fl=-84 zp=172 scale=0.103272
  13. model is NCHW input fmt
  14. model input height=640, width=640, channel=3
  15. failed to open /dev/dri/card0
  16. ./run_rk180x.sh: line 4:   221 Segmentation fault      ./rknn_yolov5_demo model/${chip_dir}/yolov5s_relu_rk180x_out_opt.rknn model/test2.jpg
  17. /userdata/rknn_yolov5_demo # root@debian10:/home/toybrick/yolo5/rknn_yolov5_demo/install#
复制代码
I will be very grateful for the answer where /dev/dri/card0 comes from. I don't have it, on any firmware. This is ok for rk3399pro?




回复

使用道具 举报

Lubagov

注册会员

积分
132
QQ
地板
发表于 2021-3-8 18:44:35 | 显示全部楼层
本帖最后由 Lubagov 于 2021-3-9 23:58 编辑

I just try to run this sample, step by step:
https://github.com/rockchip-linu ... es/rknn_yolov5_demo
And on NPU itself have not /dev/dri/card0 (if i open it over `adb shell` to get NPU shell i don't have /dev/dri/*).
SSD sample on last firmware work, but also write about card0 is not here... YOLO5 sample direct work with DRM, and ofcose not work... On 1.3.2 FW, SSD work without any messages. But dri interface also has not....

Seems it was appended to 1.6.0 kernel? And maybe also it dependent new API rknn_inputs_map/rknn_outputs_map, with direct memory access... Why do we need a DRM interface in new firmware? Which is naturally not in the old kernel, becouse kernel i put from old ASUS FW(becouse other kernels not work at all, with that ASUS)... I am right?
======
I ordered TB-RK1808 and TB-RK3399ProD, hopefully I will have them at the end of the month. I will compare and try ...
ASUS is preferred because it is easier to buy in large quantities, and cheaper (180EURO when buying locally). But it makes no sense, to use something where the firmware is not updated....


回复

使用道具 举报

Lubagov

注册会员

积分
132
QQ
5#
发表于 2021-3-17 04:53:10 | 显示全部楼层
本帖最后由 Lubagov 于 2021-3-17 04:55 编辑

Why not update the version of the library, I mean /usr/lib/aarch64-linux-gnu/librknn_api.so C library to verdion 1.6? Why 1.6, have only in python in Debain? I think these questions are related to this.But this library has in GitHub https://github.com/rockchip-linu ... /rknn_api/arm/lib64
回复

使用道具 举报

Lubagov

注册会员

积分
132
QQ
6#
发表于 2021-4-2 21:43:39 | 显示全部楼层
本帖最后由 Lubagov 于 2021-4-2 22:35 编辑

Hello!
I receive my toybrick board today, TB-RK3399ProD_33101_V1.2. Interested it was already flash on Debian 10, from box. And already with 1.6 RKNN API.
I want on this run YOLO5, and i get same problem like with ASUS:
it is after
  1. adb shell
复制代码, i trying to run direcly on NPU:
  1. /userdata/rknn_yolov5_demo # ./run_rk180x.sh
  2. Read model/test2.jpg ...
  3. img width = 1280, img height = 720
  4. Loading mode...
  5. failed to open /dev/dri/card0
  6. sdk version: librknn_runtime version 1.6.0 (096015c build: 2021-01-12 15:25:30 base: 1126) driver version: 6.4.3.5.293908
  7. model input num: 1, output num: 3
  8. index=0 name=images_165 n_dims=4 dims=[1 3 640 640] n_elems=1228800 size=1228800 fmt=0 type=3 qnt_type=2 fl=0 zp=0 scale=0.003922
  9. index=0 name=Conv_Conv_159/out0_0 n_dims=4 dims=[1 255 80 80] n_elems=1632000 size=1632000 fmt=0 type=3 qnt_type=2 fl=-69 zp=187 scale=0.127843
  10. index=1 name=Conv_Conv_160/out0_1 n_dims=4 dims=[1 255 40 40] n_elems=408000 size=408000 fmt=0 type=3 qnt_type=2 fl=-74 zp=182 scale=0.113217
  11. index=2 name=Conv_Conv_161/out0_2 n_dims=4 dims=[1 255 20 20] n_elems=102000 size=102000 fmt=0 type=3 qnt_type=2 fl=-84 zp=172 scale=0.103272
  12. model is NCHW input fmt
  13. model input height=640, width=640, channel=3
  14. failed to open /dev/dri/card0
  15. ./run_rk180x.sh: line 4:   202 Segmentation fault      ./rknn_yolov5_demo model/${chip_dir}/yolov5s_relu_rk180x_out_opt.rknn model/test2.jpg
  16. /userdata/rknn_yolov5_demo #
复制代码

  1. /userdata/rknn_yolov5_demo # ./run_rv1109_rv1126.sh
  2. Read model/test2.jpg ...
  3. img width = 1280, img height = 720
  4. Loading mode...
  5. failed to open /dev/dri/card0
  6. Create RKNN model fail, error=-13
  7. rknn_init error ret=-13
复制代码

So similar NPU have not /dev/dri/card0, it have only CPU... While i don't recive Toybrick stick, after several days should recive it...
What i do wrong? /dev/dri/ should be on RK3399Pro NPU?

SSD Demo working, but in any case write about /dev/dri/card0 is not here.

And if just llok code, at all nothing ctitical with libdrm, it use https://github.com/rockchip-linux/linux-rga for resize/convert format image, and for this open DRM...
But why it has not here?

回复

使用道具 举报

Lubagov

注册会员

积分
132
QQ
7#
发表于 2021-4-15 17:39:49 | 显示全部楼层
本帖最后由 Lubagov 于 2021-4-15 18:48 编辑

Hello! i recive Toybrick USB yesterday.
I cant find `rga_ctx->rga_handle = dlopen("/usr/lib/librga.so", RTLD_LAZY);` i can found `/usr/lib64/librockchip_rga.so` but here has not `c_RkRgaInit`
i ofcose can find https://github.com/rockchip-linux/linux-rga but really not think, that i should build from sources... And USB stick use Fedora? Debian only in PCIe?

UPD:
I build rga from sources... but, i modify CMakeLists.txt and append here /core/RgaApi.cpp, and now it found functions, like c_RkRgaInit and start working...
回复

使用道具 举报

返回列表
高级模式
B Color Image Link Quote Code Smilies
您需要登录后才可以回帖 登录 | 立即注册

本版积分规则

产品中心 购买渠道 开源社区 Wiki教程 资料下载 关于Toybrick

瑞芯微电子股份有限公司所有       闽ICP备19006074号-1      闽公网安备 35010202001061号

快速回复 返回顶部 返回列表