rknn-toolkit & rknn-api for Toybrick

pytorch模型转换报错
在转换openpose的模型时也报错
环境：Ubuntu 18.04  x86，torch 1.6，toolkit 1.6，python 3.6，npu：toybrick 1808，报错信息：
*************************
all device(s) with ntb mode:
TM018084201100261
*************************
([], ['TM018084201100261'])
**************************************************
./kinetics-st_gcn.pt ********************
E Catch exception when loading pytorch model: ./kinetics-st_gcn.pt!
E Traceback (most recent call last):
E   File "rknn/api/rknn_base.py", line 339, in rknn.api.rknn_base.RKNNBase.load_pytorch
E   File "rknn/base/RKNNlib/RK_nn.py", line 146, in rknn.base.RKNNlib.RK_nn.RKnn.load_pytorch
E   File "rknn/base/RKNNlib/app/importer/import_pytorch.py", line 128, in rknn.base.RKNNlib.app.importer.import_pytorch.ImportPytorch.run
E   File "rknn/base/RKNNlib/converter/convert_pytorch_new.py", line 2243, in rknn.base.RKNNlib.converter.convert_pytorch_new.convert_pytorch.load
E   File ".../lib/python3.6/site-packages/torch/jit/__init__.py", line 275, in load
E     cpp_module = torch._C.import_ir_module(cu, f, map_location, _extra_files)
E RuntimeError: [enforce fail at inline_container.cc:144] . PytorchStreamReader failed reading zip archive: failed finding central directory
-1
劳驾指点一下

I was glad to receive an answer, thank you very much!

It seems ASUS is not going to update the API at all. What I have is version 1.3.0 released a year ago in Debain 9, and 1.3.2 in Android.
Even though the firmware was updated in December.

I continue to play with this Debian rootfs, and if use form Debian 10 (NPU FW from ASUS, no other seems to work with it):

If i compile app over with preinstalled libs:

1. gcc ./main.c -L. -lrknn_api -lpthread

复制代码

1. 
   
2. D RKNNAPI: ==============================================
   
3. D RKNNAPI: RKNN VERSION:
   
4. D RKNNAPI:   API: 1.4.0 (bbe0dfc build: 2020-09-14 14:06:05)
   
5. D RKNNAPI:   DRV: 1.3.2 (f113d64 build: 2020-04-23 09:35:26)
   
6. D RKNNAPI: ==============================================
   

复制代码
and when i check:

1. 
   
2. $ strings /usr/lib/aarch64-linux-gnu/librknn_api.so | grep build:
   
3. (bbe0dfc build: 2020-09-14 14:06:05
   

复制代码

When i use Python:

1. 
   
2. D RKNNAPI: ==============================================
   
3. D RKNNAPI: RKNN VERSION:
   
4. D RKNNAPI:   API: 1.6.0 (79320de build: 2020-12-29 10:57:09)
   
5. D RKNNAPI:   DRV: 1.3.2 (f113d64 build: 2020-04-23 09:35:26)
   
6. D RKNNAPI: ==============================================
   

复制代码

Have small fix it for it can update/download packages fine.
It seems you didn't really want it to be used this image on other boards.
But I am think packages have been updated correctly from your repository.


By the way, the guide in the first post seems a little outdated:
If I execute this command, it becomes very bad:

1. 
   
2. pip3 install --user -U tensorflow == 1.14.0
   

复制代码

Version is already preinstalled, 2.0.0 and it works, version 1.14.0 does not work.

But it is not solve my problem, in any case, i have same problem when call `rknn_init`, with any other firmware expect 1.3.2 from ASUS:

in c:

1. 
   
2. toybrick@debian10:~/test_npu/comp$ ./a.out
   
3. E RKNNAPI: rknn_init,  recv(MsgLoadAck) fail, -9(ERROR_PIPE) != 368!
   
4. rknn_init fail! ret=-3
   
5. Segmentation fault
   

复制代码

in Python:

1. 
   
2. toybrick@debian10:~/test_npu$ python3 testapi.py
   
3. --> Loading RKNN model
   
4. done
   
5. I NPUTransfer: Starting NPU Transfer Client, Transfer version 2.1.0 (b5861e7@2020-11-23T11:50:51)
   
6. E RKNNAPI: rknn_init,  recv(MsgLoadAck) fail, -9(ERROR_PIPE) != 368!
   
7. E Catch exception when init runtime!
   
8. E Traceback (most recent call last):
   
9. E   File "rknn/api/rknn_base.py", line 844, in rknn.api.rknn_base.RKNNBase.init_runtime
   
10. E   File "rknn/api/rknn_runtime.py", line 340, in rknn.api.rknn_runtime.RKNNRuntime.build_graph
    
11. E Exception: RKNN init failed. error code: RKNN_ERR_DEVICE_UNAVAILABLE
    
12. --> Running model
    
13. W We need do some initial work, it will increase call time of inference for the first time.
    
14. I NPUTransfer: Starting NPU Transfer Client, Transfer version 2.1.0 (b5861e7@2020-11-23T11:50:51)
    
15. E NPUTransfer: Transfer interface open failed!, ret = -1
    
16. E RKNNAPI: rknn_init,  driver open fail!  ret = -1(ERROR_IO)!
    
17. E Catch exception when init model!
    
18. E Traceback (most recent call last):
    
19. E   File "rknn/api/rknn_base.py", line 931, in rknn.api.rknn_base.RKNNBase.inference
    
20. E   File "rknn/api/rknn_runtime.py", line 340, in rknn.api.rknn_runtime.RKNNRuntime.build_graph
    
21. E Exception: RKNN init failed. error code: RKNN_ERR_DEVICE_UNAVAILABLE
    
22. done
    

复制代码dmesg output in NPU:

1. 
   
2. [  873.529384] android_work: sent uevent USB_STATE=DISCONNECTED
   
3. [  873.530761] configfs-gadget gadget: unbind function 'Function FS Gadget'/ffffffc0052c1e38
   
4. [  873.530845] configfs-gadget gadget: unbind function 'Function FS Gadget'/ffffffc0052c1c38
   
5. [  873.531441] read descriptors
   
6. [  873.531466] read strings
   
7. [  874.545684] configfs-gadget fd000000.dwc3: failed to start rockchip: -19
   
8. [  874.573016] read descriptors
   
9. [  874.573057] read strings
   
10. [  874.579310] android_work: did not send uevent (0 0           (null))
    
11. [  874.706836] android_work: did not send uevent (0 0           (null))
    
12. [  874.788465] android_work: sent uevent USB_STATE=CONNECTED
    
13. [  874.819649] configfs-gadget gadget: super-speed config #1: b
    
14. [  874.819883] android_work: sent uevent USB_STATE=CONFIGURED
    

复制代码

Unpacked from the NPU firmware, boot.img into parts via `imgrepackrk`. And also unpacked the 1.6 firmware from Debain 10 from Toybrick. Replaced the ramdisk in the ASUS firmware with the ramdisk from the Toybrick firmware. Then, I packed it back. those. The kernel and resources are from ASUS, and the ramdisk is from Toybrick. And it works at least `rknn_init` fine call.Better of cose build it from source somehow, and make it working.. Only not clear how to build kernel, working on ASUS...

I NPUTransfer: Starting NPU Transfer Client, Transfer version 2.1.0 (b5861e7@2020-11-23T11:50:51)
D RKNNAPI: ==============================================
D RKNNAPI: RKNN VERSION:
D RKNNAPI:   API: 1.6.0 (79320de build: 2020-12-29 10:57:09)
D RKNNAPI:   DRV: 1.6.0 (159d2d3 build: 2021-01-12 15:23:09)
D RKNNAPI: ==============================================

/=============
I am trying to run the YOLO5 example. I understand correctly that `/dev/card/dri0` should be on the NPU device, but only with the new kernel?
Or, is it on RK1808 but not on RK3399Pro?

1. 
   
2. # ./run_rk180x.sh
   
3. Read model/test2.jpg ...
   
4. img width = 1280, img height = 720
   
5. Loading mode...
   
6. failed to open /dev/dri/card0
   
7. sdk version: librknn_runtime version 1.6.0 (096015c build: 2021-01-12 15:25:30 base: 1126) driver version: 6.4.3.5.293908
   
8. model input num: 1, output num: 3
   
9. index=0 name=images_165 n_dims=4 dims=[1 3 640 640] n_elems=1228800 size=1228800 fmt=0 type=3 qnt_type=2 fl=0 zp=0 scale=0.003922
   
10. index=0 name=Conv_Conv_159/out0_0 n_dims=4 dims=[1 255 80 80] n_elems=1632000 size=1632000 fmt=0 type=3 qnt_type=2 fl=-69 zp=187 scale=0.127843
    
11. index=1 name=Conv_Conv_160/out0_1 n_dims=4 dims=[1 255 40 40] n_elems=408000 size=408000 fmt=0 type=3 qnt_type=2 fl=-74 zp=182 scale=0.113217
    
12. index=2 name=Conv_Conv_161/out0_2 n_dims=4 dims=[1 255 20 20] n_elems=102000 size=102000 fmt=0 type=3 qnt_type=2 fl=-84 zp=172 scale=0.103272
    
13. model is NCHW input fmt
    
14. model input height=640, width=640, channel=3
    
15. failed to open /dev/dri/card0
    
16. ./run_rk180x.sh: line 4:   221 Segmentation fault      ./rknn_yolov5_demo model/${chip_dir}/yolov5s_relu_rk180x_out_opt.rknn model/test2.jpg
    
17. /userdata/rknn_yolov5_demo # root@debian10:/home/toybrick/yolo5/rknn_yolov5_demo/install#
    

复制代码
I will be very grateful for the answer where /dev/dri/card0 comes from. I don't have it, on any firmware. This is ok for rk3399pro?

这官方源里torch1.6 的命名有问题吧：torch-1.6.0a0+b31f58d-cp37-cp37m-linux_aarch64.whl ，也没有torchvision

Lubagov 发表于 2021-3-4 21:34
Unpacked from the NPU firmware, boot.img into parts via `imgrepackrk`. And also unpacked the 1.6 fir ...

/dev/dri/card0  is used for drm.
Try rknn-toolkit-lite ?

I just try to run this sample, step by step:
https://github.com/rockchip-linu ... es/rknn_yolov5_demo
And on NPU itself have not /dev/dri/card0 (if i open it over `adb shell` to get NPU shell i don't have /dev/dri/*).
SSD sample on last firmware work, but also write about card0 is not here... YOLO5 sample direct work with DRM, and ofcose not work... On 1.3.2 FW, SSD work without any messages. But dri interface also has not....

Seems it was appended to 1.6.0 kernel? And maybe also it dependent new API rknn_inputs_map/rknn_outputs_map, with direct memory access... Why do we need a DRM interface in new firmware? Which is naturally not in the old kernel, becouse kernel i put from old ASUS FW(becouse other kernels not work at all, with that ASUS)... I am right?
======
I ordered TB-RK1808 and TB-RK3399ProD, hopefully I will have them at the end of the month. I will compare and try ...
ASUS is preferred because it is easier to buy in large quantities, and cheaper (180EURO when buying locally). But it makes no sense, to use something where the firmware is not updated....

jefferyzhang 发表于 2019-10-30 12:13
Added

OneDrive的链接打开还是1.4版本的，请更新

jefferyzhang 发表于 2019-10-30 12:13
Added

上面OneDrive的链接还是1.4的呢, 请更新!

请问下如何在容器中访问3399pro的npu资源呢？我们想把AI算法打包到容器中，不过不清楚如何配置共享宿主机NPU资源。这块可以指点下吗？

请问现在1.6版本的API-SDK有下载了吗？http://repo.rock-chips.com/rk1808/rknn-api/ 这个网页似乎是1.4的。另外一个问题是不知道3288作为安卓上位机，能和rk1808被动模式一起用吗？好像看到SDK对安卓的要求是8.1或以上，但是RK3288的安卓应该是6或者7.1的吧，所以能一起用吗？