Toybrick

标题: jasper pytorch模型直接加载或转onnx或转tf模型均不成功 [打印本页]
作者: xsky    时间: 2020-3-14 17:28
标题: jasper pytorch模型直接加载或转onnx或转tf模型均不成功
本帖最后由 xsky 于 2020-3-14 17:30 编辑

rknn  v1.3.0
pytorch v1.2
tensorflow  v1.14onnx  v1.4.1
onnx-tf  v1.3.0  

代码及模型
.pt是原始模型
链接：https://pan.baidu.com/s/10xjD5cuDKNkN_c-AWlxksg
提取码：1okw

1. .pt模型是可以正确运行的;
   但rknn加载pytorch模型提示错误:
  1. E Catch exception when loading pytorch model: toml_fp32_acoustic.pt!
  2. E Traceback (most recent call last):
  3. E File "rknn\api\rknn_base.py", line 567, in rknn.api.rknn_base.RKNNBase.load_pytorch
  4. E File "rknn\base\RKNNlib\app\importer\import_pytorch.py", line 95, in rknn.base.RKNNlib.app.importer.import_pytorch.ImportPytorch.run
  5. E File "rknn\base\RKNNlib\converter\convert_pytorch.py", line 517, in rknn.base.RKNNlib.converter.convert_pytorch.convert_pytorch.__init__
  6. E File "rknn\base\RKNNlib\converter\convert_pytorch.py", line 601, in rknn.base.RKNNlib.converter.convert_pytorch.convert_pytorch.model_simplify
  7. E File "rknn\base\RKNNlib\converter\convert_pytorch.py", line 104, in rknn.base.RKNNlib.converter.convert_pytorch.torch_inference_engine.shape_pick
  8. ……   类似重复n次
  9. E File "rknn\base\RKNNlib\converter\convert_pytorch.py", line 104, in rknn.base.RKNNlib.converter.convert_pytorch.torch_inference_engine.shape_pick
  10. E File "rknn\base\RKNNlib\converter\convert_pytorch.py", line 139, in rknn.base.RKNNlib.converter.convert_pytorch.torch_inference_engine.__ir_shape_inference
  11. E File "rknn\base\RKNNlib\converter\convert_pytorch.py", line 251, in rknn.base.RKNNlib.converter.convert_pytorch.torch_inference_engine.convolution_shape
  12. E File "rknn\base\RKNNlib\converter\convert_pytorch.py", line 242, in rknn.base.RKNNlib.converter.convert_pytorch.torch_inference_engine._pad_pick
  13. E IndexError: list index out of range
  14. Traceback (most recent call last):
复制代码
2. .onnx模型也可以正确运行,直接使用onnx-tf运行结果与.pt结果一致  (onnx-tf需要更新为v1.3.0)
  onnx转为.rknn 未量化, 可运行,但结果错误输出为全0
3.  通过onnx转为tf模型然后再转为.rknn, 运行提示模型错误失败:
  1. --> Init runtime environment
  2. rknn run target self
  3. I npu_transfer_proxy pid: 748, status: sleeping
  4. D NPUTransfer: efuseid = 3399:4632a3468cf39f3b
  5. I NPUTransfer: Starting NPU Transfer Client, Transfer version 1.9.8 (cab3961@2019-12-12T09:54:26)
  6. D NPUTransfer: Transfer spec = local:transfer_proxy
  7. D NPUTransfer: Transfer interface successfully opened, fd = 4
  8. E RKNNAPI: rknn_init, msg_load_ack fail, ack = 1(ACK_FAIL), expect 0(ACK_SUCC)!
  9. E RKNNAPI: ==============================================
  10. E RKNNAPI: RKNN VERSION:
  11. E RKNNAPI: API: 1.3.0 (c5654ea build: 2019-12-25 12:40:55)
  12. E RKNNAPI: DRV: 1.3.0 (c4f8c23 build: 2019-11-25 10:39:29)
  13. E RKNNAPI: ==============================================
  14. D NPUTransfer: Transfer client closed, fd = 4
  15. E Catch exception when init runtime!
  16. E Traceback (most recent call last):
  17. E File "rknn/api/rknn_base.py", line 988, in rknn.api.rknn_base.RKNNBase.init_runtime
  18. E File "rknn/api/rknn_runtime.py", line 320, in rknn.api.rknn_runtime.RKNNRuntime.build_graph
  19. E Exception: RKNN init failed. error code: RKNN_ERR_MODEL_INVALID
  20. Init runtime environment failed
复制代码


代码:

  2. import platform
  3. import os
  4. import torch
  5. import numpy as np

  7. from rknn.api import RKNN
  8. from timeit import default_timer as timer

  10. import onnx
  11. from onnx_tf.backend import prepare

  13. def rm(file):
  14.     if os.path.exists(file):
  15.         os.remove(file)

  17. if __name__ == '__main__':

  19.     rknn = RKNN(verbose=True)
  20.     rknn.config(batch_size=1,
  21.                 epochs=1)  # asymmetric_quantized-u8,  quantized_dtype='dynamic_fixed_point-16' , channel_mean_value='0 0 0 1', reorder_channel='0 1 2',
  22.     system = platform.system()

  24.     seq_len = 64
  25.     isize = 672
  26.     pt_file = 'toml_fp32_acoustic.pt'
  27.     m = torch.jit.load(pt_file)
  28.     m.eval()
  29.     #x = torch.rand((1, seq_len, isize))
  30.     x = torch.linspace(-1, 1, steps=(isize * seq_len))
  31.     x = x.view(1, 1, seq_len, -1)
  32.     input_size_list = [[1, 1, seq_len, isize]]
  33.     #x1 = torch.full((1, hsize), fill_value=0.2, dtype=torch.float)
  34.     ix = x.numpy()
  35.     t0 = timer()
  36.     out = m.forward(x)
  37.     t1 = timer()
  38.     print("torch out, time:", t1 - t0)
  39.     print(out.detach().numpy())

  41.     use_pytorch = False
  42.     if use_pytorch:
  43.         rknn.load_pytorch(model=pt_file, input_size_list=input_size_list)
  44.     else:
  45.         onnx_file = pt_file + '.onnx'
  46.         #onnx_file = 'toml_fp32_acoustic.pt_op10.onnx'
  47.         tf_model = onnx.load(onnx_file)
  48.         tf_rep = prepare(tf_model)

  50.         t0 = timer()
  51.         tf_out = tf_rep.run(ix)
  52.         t1 = timer()
  53.         tf_out = np.array(tf_out)
  54.         print("onnx out:", ix.shape, ', ', tf_out.shape, " time:", t1 - t0)
  55.         print(tf_out)
  56.         rknn.load_onnx(model=onnx_file)

  58.         # onnx转为tf .pb并加载
  59.         # pb_file = onnx_file + '.pb'
  60.         # rm(pb_file)
  61.         # tf_rep.export_graph(pb_file)
  62.         # input_size_list = [[1, seq_len, isize]]
  63.         # rknn.load_tensorflow(tf_pb=pb_file,
  64.         #                      inputs=['input'],
  65.         #                      outputs=['transpose_326'],
  66.         #                      input_size_list=input_size_list)

  68.     is_quant = False
  69.     if is_quant:
  70.         rknn_file = pt_file + '_quant.rknn'
  71.     else:
  72.         rknn_file = pt_file + '.rknn'

  74.     print('--> Building model')
  75.     ret = rknn.build(do_quantization=is_quant, dataset='./dataset.txt')
  76.     if ret != 0:
  77.         print('rknn.build failed!')
  78.         exit(ret)
  79.     print('build done')
  80.     rm(rknn_file)
  81.     rknn.export_rknn(rknn_file)

  83.     print('--> Init runtime environment')
  84.     if system == 'Windows':
  85.         print("rknn run target rk1808")
  86.         ret = rknn.init_runtime(target='rk1808', target_sub_class='AICS')
  87.     else:
  88.         print("rknn run target self")
  89.         ret = rknn.init_runtime()
  90.     if ret != 0:
  91.         print('Init runtime environment failed')
  92.         exit(ret)
  93.     print('done')

  95.     print('input:')
  96.     print(ix)

  98.     t0 = timer()
  99.     torch_out = m.forward(x)
  100.     t1 = timer()
  101.     print("torch [out]:", x.shape, ', ', torch_out.shape, " time:", t1 - t0)
  102.     print(torch_out.detach().numpy())

  104.     t0 = timer()
  105.     rknn_out = rknn.inference(inputs=[ix], data_type='float32', data_format='nchw')   # , inputs_pass_through=[1]
  106.     t1 = timer()

  108.     if rknn_out is None:
  109.         print('rknn inference failed')
  110.         exit(-100)
  111.     y = np.array(rknn_out[0])
  112.     print("rknn out:", ix.shape, ', ', y.shape, " time:", t1 - t0)
  113.     print(y)

  115.     rknn.release()
复制代码




作者: jefferyzhang    时间: 2020-3-17 08:55
已上报问题
作者: xsky    时间: 2020-3-21 18:09
jefferyzhang 发表于 2020-3-17 08:55
已上报问题

麻烦能不能帮问一下这个有什么头绪吗，现在着急评估项目，谢谢啊

拆开算子试了一下
.load_pytorch应该是对conv1d不支持

onnx可以正确运行conv1d
但
conv1d → relu → bn  load_onnx运行结果正确
conv1d → bn → relu：pytorch/onnx输出shape为(1, 64, 672)但rknn输出shape为(1, 672, 64)
主楼中的模型就是类似这样的结构重复多次，这里输出shape会被改变，再往后传输的时候会错乱吧



  2. import platform
  3. import os
  4. import torch
  5. import numpy as np
  6. import copy

  8. from rknn.api import RKNN
  9. from timeit import default_timer as timer

  11. import onnx
  12. from onnx_tf.backend import prepare

  14. def rm(file):
  15.     if os.path.exists(file):
  16.         os.remove(file)


  19. class MyConv1d(torch.nn.Module):
  20.     def __init__(self, seq_len, isize):
  21.         super(MyConv1d, self).__init__()

  23.         self._conv1d = torch.nn.Conv1d(seq_len, seq_len, kernel_size=11, stride=1, padding=5, dilation=1, groups=1, bias=False)
  24.         self._seq_len = seq_len
  25.         self._isize = isize
  26.         self._bn = torch.nn.BatchNorm1d(seq_len, eps=1e-3, momentum=0.1)
  27.         self._relu = torch.nn.ReLU()

  29.     def forward(self, x):
  30.         x = x.view(1, self._seq_len, self._isize)
  31.         x = self._conv1d(x)
  32.         x = self._bn(x)
  33.         x = self._relu(x)
  34.         return x


  37. if __name__ == '__main__':

  39.     rknn = RKNN(verbose=True)
  40.     rknn.config(batch_size=1,
  41.                 epochs=1)  # asymmetric_quantized-u8,  quantized_dtype='dynamic_fixed_point-16' , channel_mean_value='0 0 0 1', reorder_channel='0 1 2',
  42.     system = platform.system()

  44.     seq_len = 64
  45.     isize = 672
  46.     pt_file = 'conv1d.pt'

  48.     x = torch.rand((1, seq_len, isize))
  49.     #x = torch.linspace(-1, 1, steps=(isize * seq_len))
  50.     x = x.view(1, 1, seq_len, -1)
  51.     input_size_list = [[1, seq_len, isize]]
  52.     #x1 = torch.full((1, hsize), fill_value=0.2, dtype=torch.float)
  53.     ix = x.numpy()
  54.     pt_m = MyConv1d(seq_len, isize)
  55.     pt_m.eval()
  56.     t0 = timer()
  57.     out = pt_m.forward(x)
  58.     t1 = timer()
  59.     print("torch out, ", out.shape, "time:", t1 - t0)
  60.     print(out.detach().numpy())

  62.     jit_m = torch.jit.trace(pt_m, x)
  63.     rm(pt_file)
  64.     jit_m.save(pt_file)

  66.     jit_m = torch.jit.load(pt_file)
  67.     jit_m.eval()

  69.     t0 = timer()
  70.     out = jit_m.forward(x)
  71.     t1 = timer()
  72.     print("torch out, ", out.shape, "time:", t1 - t0)
  73.     print(out.detach().numpy())

  75.     use_pytorch = False
  76.     if use_pytorch:
  77.         rknn.load_pytorch(model=pt_file, input_size_list=input_size_list)
  78.     else:
  79.         onnx_file = pt_file + '.onnx'
  80.         rm(onnx_file)
  81.         torch.onnx.export(pt_m, x, onnx_file,
  82.                           export_params=True, verbose=False,
  83.                           input_names=['input'],
  84.                           output_names=['output'],
  85.                           opset_version=10
  86.                           )
  87.         tf_model = onnx.load(onnx_file)
  88.         tf_rep = prepare(tf_model)

  90.         tf_ix = ix
  91.         t0 = timer()
  92.         tf_out = tf_rep.run(tf_ix)
  93.         t1 = timer()
  94.         tf_out = np.array(tf_out)
  95.         print("onnx out:", tf_ix.shape, ', ', tf_out.shape, " time:", t1 - t0)
  96.         print(tf_out)
  97.         rknn.load_onnx(model=onnx_file)

  99.         # # onnx转为tf .pb并加载
  100.         # pb_file = onnx_file + '.pb'
  101.         # rm(pb_file)
  102.         # tf_rep.export_graph(pb_file)
  103.         # input_size_list = [[1, seq_len, isize]]
  104.         # rknn.load_tensorflow(tf_pb=pb_file,
  105.         #                      inputs=['input'],
  106.         #                      outputs=['transpose_191'],
  107.         #                      input_size_list=input_size_list)

  109.     is_quant = False
  110.     if is_quant:
  111.         rknn_file = pt_file + '_quant.rknn'
  112.     else:
  113.         rknn_file = pt_file + '.rknn'

  115.     print('--> Building model')
  116.     ret = rknn.build(do_quantization=is_quant, dataset='./dataset.txt')
  117.     if ret != 0:
  118.         print('rknn.build failed!')
  119.         exit(ret)
  120.     print('build done')
  121.     rm(rknn_file)
  122.     rknn.export_rknn(rknn_file)

  124.     print('--> Init runtime environment')
  125.     if system == 'Windows':
  126.         print("rknn run target rk1808")
  127.         ret = rknn.init_runtime(target='rk1808', target_sub_class='AICS')
  128.     else:
  129.         print("rknn run target self")
  130.         ret = rknn.init_runtime()
  131.     if ret != 0:
  132.         print('Init runtime environment failed')
  133.         exit(ret)
  134.     print('done')

  136.     print('input:')
  137.     print(ix)

  139.     pt_m_fp16 = pt_m.cuda()
  140.     pt_m_fp16.half()
  141.     pt_m_fp16.eval()
  142.     t0 = timer()
  143.     pt_y = pt_m_fp16.forward(x.cuda().half())
  144.     t1 = timer()
  145.     print("torch out, cuda:fp16", pt_y.shape, "time:", t1 - t0)
  146.     pt_y = pt_y.cpu().float().detach().numpy()
  147.     print(pt_y)
  148.     # t0 = timer()
  149.     # torch_out = m.forward(x)
  150.     # t1 = timer()
  151.     # print("torch [out]:", x.shape, ', ', torch_out.shape, " time:", t1 - t0)
  152.     # print(torch_out.detach().numpy())

  154.     t0 = timer()
  155.     rknn_out = rknn.inference(inputs=[ix], data_type='float32', data_format='nchw')   # , inputs_pass_through=[1]
  156.     t1 = timer()

  158.     if rknn_out is None:
  159.         print('rknn inference failed')
  160.         exit(-100)
  161.     y = np.array(rknn_out[0])
  162.     print("rknn out:", ix.shape, ', ', y.shape, " time:", t1 - t0)
  163.     print(y)

  165.     if y.shape[1] != pt_y.shape[1] or  y.shape[2] != pt_y.shape[2]:
  166.         print("******* shape err: expect ", pt_y.shape, " but rknn out ", y.shape, " reshape:")
  167. #        y = y.reshape(pt_y.shape[0], pt_y.shape[1], -1)
  168. #        print(y)

  170.     diff = np.abs(y - pt_y)
  171.     cnt = y.shape[0] * y.shape[1] * y.shape[2]
  172.     print("diff:")
  173.     print(diff)
  174.     d = diff[np.where(diff > 0.05)]
  175.     print("warning values(diff > 0.05):", d.shape, " %:", 100.0 * np.array(d.shape, dtype=np.float) // cnt)
  176.     print(d)
  177.     d = diff[np.where(diff > 0.001)]
  178.     print("warning values(diff > 0.001):", d.shape, " %:", 100.0 * np.array(d.shape, dtype=np.float) // cnt)
  179.     print(d)
  180.     d = diff[np.where(diff > 0.0001)]
  181.     print("warning values(diff > 0.0001):", d.shape,  " %:", 100.0 * np.array(d.shape, dtype=np.float) // cnt)
  182.     print(d)

  184.     rknn.release()
复制代码

作者: jefferyzhang    时间: 2020-3-21 22:45
NPU部门回复：
目前pytorch只支持NCHW这种4维输入的卷积，这个模型的卷积输入是3维，暂不支持。



欢迎光临 Toybrick (https://t.rock-chips.com/) Powered by Discuz! X3.3