Toybrick

标题: FusedBatchNormV3 support [打印本页]
作者: kaishi    时间: 2020-3-24 11:59
标题: FusedBatchNormV3 support

My RKNN SDK versio is 1.3.0.
When I convert tensorflow lanenet.pb to rknn, I encounter the following error below which is due to FusedBatchNormV3 operator.
Does SDK support this?

f system memory.
E Unknow layer "fusedbatchnormv3"
E Try match FusedBatchNormV3 lanenet_model/vgg_backend/instance_seg/pix_bn/FusedBatchNormV3 failed, catch exception!
E Catch exception when loading tensorflow model: ./lanenet.pb!
E Traceback (most recent call last):
E   File "rknn/base/RKNNlib/converter/convert_tf.py", line 643, in rknn.base.RKNNlib.converter.convert_tf.convert_tf.match_paragraph_and_param
E   File "rknn/base/RKNNlib/RKNNnet.py", line 189, in rknn.base.RKNNlib.RKNNnet.RKNNNet.new_layer
E   File "/home/test/anaconda3/envs/directron2/lib/python3.6/site-packages/rknn/base/RKNNlib/RKNNlog.py", line 327, in e
E     raise ValueError(msg)
E ValueError: Unknow layer "fusedbatchnormv3"
E During handling of the above exception, another exception occurred:
E Traceback (most recent call last):
E   File "rknn/api/rknn_base.py", line 303, in rknn.api.rknn_base.RKNNBase.load_tensorflow
E   File "rknn/base/RKNNlib/converter/convert_tf.py", line 742, in rknn.base.RKNNlib.converter.convert_tf.convert_tf.match_paragraph_and_param
E   File "/home/test/anaconda3/envs/directron2/lib/python3.6/site-packages/rknn/base/RKNNlib/RKNNlog.py", line 327, in e
E     raise ValueError(msg)
E ValueError: Try match FusedBatchNormV3 lanenet_model/vgg_backend/instance_seg/pix_bn/FusedBatchNormV3 failed, catch exception!
done
--> Building model
Traceback (most recent call last):
  File "./lanenet.py", line 86, in <module>
    rknn.build(do_quantization=True, dataset='./dataset.txt')
  File "/home/test/anaconda3/envs/directron2/lib/python3.6/site-packages/rknn/api/rknn.py", line 220, in build
    inputs = self.rknn_base.net.get_input_layers()
AttributeError: 'NoneType' object has no attribute 'get_input_layers'


作者: jefferyzhang    时间: 2020-3-24 12:09
1. tensorflow MUST is 1.13.x
2. keras.backend.set_learning_phase(0) before create tf session !
作者: jefferyzhang    时间: 2020-3-24 12:11
本帖最后由 jefferyzhang 于 2020-3-24 12:22 编辑

BTW, you are so lucky, because I have tried lanenet last year...
https://github.com/rknn-demos/lanenet-lane-detection
作者: kaishi    时间: 2020-3-24 12:53
jefferyzhang 发表于 2020-3-24 12:11
BTW, you are so lucky, because I have tried lanenet last year...
https://github.com/rknn-demos/lanen ...

I use your script to do ckpt to pb:
python rknn/save_pb.py
and  then convert rknn using
python rknn/pb_to_rknn.py -i lanenet.pb  -o lanenet.rknn

But I got the same error
My tensorflow version is '1.15.0'. Could this be the sisue?


作者: kaishi    时间: 2020-3-24 13:39
I install tensorflow=1.13.0 and it works now.
However, it is quite slow on rk3399pro. The rknn_perf shows 0.24fps
Is there a reason why quantize is not on during pb_to_rknn conversion?
作者: jefferyzhang    时间: 2020-3-24 16:04
kaishi 发表于 2020-3-24 13:39
I install tensorflow=1.13.0 and it works now.
However, it is quite slow on rk3399pro. The rknn_perf  ...

Yes, it is a perf without quantization.
You must set do_quantization=True, and run again.
作者: kaishi    时间: 2020-3-25 02:37
I enable quantization and speed is 3fps now. Is there a way to speed up to realtime?
作者: cfan    时间: 2020-12-23 11:49
jefferyzhang 发表于 2020-3-24 16:04
Yes, it is a perf without quantization.
You must set do_quantization=True, and run again.

我在你的代码中看到有这么一句话，
# Workaround RKNN 1.1.0 bug
想问问是遇到了什么样的bug。
我遇到了一个sortmax层相关的bug，
其输出的结果在rknn和在直接pycharm调用pb模型上输出的结果有较大差异
作者: jefferyzhang    时间: 2020-12-23 11:57
cfan 发表于 2020-12-23 11:49
我在你的代码中看到有这么一句话，
# Workaround RKNN 1.1.0 bug
想问问是遇到了什么样的bug。

这个bug早在1.2.0就解决了。。。
作者: cfan    时间: 2020-12-24 13:53
jefferyzhang 发表于 2020-12-23 11:57
这个bug早在1.2.0就解决了。。。

再问一下，我把模型用一张图片进行量化之后速度达到7帧左右，还有什么办法能提高速度吗
作者: jefferyzhang    时间: 2020-12-24 14:13
cfan 发表于 2020-12-24 13:53
再问一下，我把模型用一张图片进行量化之后速度达到7帧左右，还有什么办法能提高速度吗
...

可以进置顶帖qq群里跟他们讨论下。
作者: cfan    时间: 2020-12-28 15:25
jefferyzhang 发表于 2020-12-24 14:13
可以进置顶帖qq群里跟他们讨论下。

问了，没人理，，，我用一张图像量化FPS是6.65，用两百张图像反而成了6.63。用的是python接口，连接1808评价得到的。
  1.     # Config for Model Input PreProcess
  2.     rknn.config(channel_mean_value='127.5 127.5 127.5 127.5', reorder_channel='0 1 2',batch_size = 8)
  3.     #rknn.config(channel_mean_value='0 0 0 1', reorder_channel='0 1 2')

  5.     # Load TensorFlow Model
  6.     print('--> Loading model')
  7.     rknn.load_tensorflow(tf_pb='./model.pb',
  8.                          inputs=['lanenet/input_tensor'],
  9.                          outputs=['lanenet_model/vgg_frontend/vgg16_decode_module/binary_seg_decode/binary_final_logits/binary_final_logits'],
  10.                          input_size_list=[[256, 512, 3]])
  11.     print('done')

  13.     # Build Model
  14.     print('--> Building model')
  15.     rknn.build(do_quantization=True, dataset='./dataset.txt')
  16.     print('done')

  18.     print('--> Begin evaluate model performance')
  19.     perf_results = rknn.eval_perf(inputs=[img])                #评估模型性能。 返回模型在硬件上运行的总时间
  20.     print('done')
复制代码



欢迎光临 Toybrick (https://t.rock-chips.com/) Powered by Discuz! X3.3