Toybrick

标题: jasper pytorch模型直接加载或转onnx或转tf模型均不成功 [打印本页]

作者: xsky    时间: 2020-3-14 17:28
标题: jasper pytorch模型直接加载或转onnx或转tf模型均不成功
本帖最后由 xsky 于 2020-3-14 17:30 编辑

rknn  v1.3.0
pytorch v1.2
tensorflow  v1.14onnx  v1.4.1
onnx-tf  v1.3.0  

代码及模型
.pt是原始模型
链接:https://pan.baidu.com/s/10xjD5cuDKNkN_c-AWlxksg
提取码:1okw

1. .pt模型是可以正确运行的;
   但rknn加载pytorch模型提示错误:
  1. E Catch exception when loading pytorch model: toml_fp32_acoustic.pt!
  2. E Traceback (most recent call last):
  3. E File "rknn\api\rknn_base.py", line 567, in rknn.api.rknn_base.RKNNBase.load_pytorch
  4. E File "rknn\base\RKNNlib\app\importer\import_pytorch.py", line 95, in rknn.base.RKNNlib.app.importer.import_pytorch.ImportPytorch.run
  5. E File "rknn\base\RKNNlib\converter\convert_pytorch.py", line 517, in rknn.base.RKNNlib.converter.convert_pytorch.convert_pytorch.__init__
  6. E File "rknn\base\RKNNlib\converter\convert_pytorch.py", line 601, in rknn.base.RKNNlib.converter.convert_pytorch.convert_pytorch.model_simplify
  7. E File "rknn\base\RKNNlib\converter\convert_pytorch.py", line 104, in rknn.base.RKNNlib.converter.convert_pytorch.torch_inference_engine.shape_pick
  8. ……   类似重复n次
  9. E File "rknn\base\RKNNlib\converter\convert_pytorch.py", line 104, in rknn.base.RKNNlib.converter.convert_pytorch.torch_inference_engine.shape_pick
  10. E File "rknn\base\RKNNlib\converter\convert_pytorch.py", line 139, in rknn.base.RKNNlib.converter.convert_pytorch.torch_inference_engine.__ir_shape_inference
  11. E File "rknn\base\RKNNlib\converter\convert_pytorch.py", line 251, in rknn.base.RKNNlib.converter.convert_pytorch.torch_inference_engine.convolution_shape
  12. E File "rknn\base\RKNNlib\converter\convert_pytorch.py", line 242, in rknn.base.RKNNlib.converter.convert_pytorch.torch_inference_engine._pad_pick
  13. E IndexError: list index out of range
  14. Traceback (most recent call last):
复制代码

2. .onnx模型也可以正确运行,直接使用onnx-tf运行结果与.pt结果一致  (onnx-tf需要更新为v1.3.0)
  onnx转为.rknn 未量化, 可运行,但结果错误输出为全0
3.  通过onnx转为tf模型然后再转为.rknn, 运行提示模型错误失败:
  1. --> Init runtime environment
  2. rknn run target self
  3. I npu_transfer_proxy pid: 748, status: sleeping
  4. D NPUTransfer: efuseid = 3399:4632a3468cf39f3b
  5. I NPUTransfer: Starting NPU Transfer Client, Transfer version 1.9.8 (cab3961@2019-12-12T09:54:26)
  6. D NPUTransfer: Transfer spec = local:transfer_proxy
  7. D NPUTransfer: Transfer interface successfully opened, fd = 4
  8. E RKNNAPI: rknn_init, msg_load_ack fail, ack = 1(ACK_FAIL), expect 0(ACK_SUCC)!
  9. E RKNNAPI: ==============================================
  10. E RKNNAPI: RKNN VERSION:
  11. E RKNNAPI: API: 1.3.0 (c5654ea build: 2019-12-25 12:40:55)
  12. E RKNNAPI: DRV: 1.3.0 (c4f8c23 build: 2019-11-25 10:39:29)
  13. E RKNNAPI: ==============================================
  14. D NPUTransfer: Transfer client closed, fd = 4
  15. E Catch exception when init runtime!
  16. E Traceback (most recent call last):
  17. E File "rknn/api/rknn_base.py", line 988, in rknn.api.rknn_base.RKNNBase.init_runtime
  18. E File "rknn/api/rknn_runtime.py", line 320, in rknn.api.rknn_runtime.RKNNRuntime.build_graph
  19. E Exception: RKNN init failed. error code: RKNN_ERR_MODEL_INVALID
  20. Init runtime environment failed
复制代码



代码:

  1. import platform
  2. import os
  3. import torch
  4. import numpy as np

  5. from rknn.api import RKNN
  6. from timeit import default_timer as timer

  7. import onnx
  8. from onnx_tf.backend import prepare

  9. def rm(file):
  10.     if os.path.exists(file):
  11.         os.remove(file)

  12. if __name__ == '__main__':

  13.     rknn = RKNN(verbose=True)
  14.     rknn.config(batch_size=1,
  15.                 epochs=1)  # asymmetric_quantized-u8,  quantized_dtype='dynamic_fixed_point-16' , channel_mean_value='0 0 0 1', reorder_channel='0 1 2',
  16.     system = platform.system()

  17.     seq_len = 64
  18.     isize = 672
  19.     pt_file = 'toml_fp32_acoustic.pt'
  20.     m = torch.jit.load(pt_file)
  21.     m.eval()
  22.     #x = torch.rand((1, seq_len, isize))
  23.     x = torch.linspace(-1, 1, steps=(isize * seq_len))
  24.     x = x.view(1, 1, seq_len, -1)
  25.     input_size_list = [[1, 1, seq_len, isize]]
  26.     #x1 = torch.full((1, hsize), fill_value=0.2, dtype=torch.float)
  27.     ix = x.numpy()
  28.     t0 = timer()
  29.     out = m.forward(x)
  30.     t1 = timer()
  31.     print("torch out, time:", t1 - t0)
  32.     print(out.detach().numpy())

  33.     use_pytorch = False
  34.     if use_pytorch:
  35.         rknn.load_pytorch(model=pt_file, input_size_list=input_size_list)
  36.     else:
  37.         onnx_file = pt_file + '.onnx'
  38.         #onnx_file = 'toml_fp32_acoustic.pt_op10.onnx'
  39.         tf_model = onnx.load(onnx_file)
  40.         tf_rep = prepare(tf_model)

  41.         t0 = timer()
  42.         tf_out = tf_rep.run(ix)
  43.         t1 = timer()
  44.         tf_out = np.array(tf_out)
  45.         print("onnx out:", ix.shape, ', ', tf_out.shape, " time:", t1 - t0)
  46.         print(tf_out)
  47.         rknn.load_onnx(model=onnx_file)

  48.         # onnx转为tf .pb并加载
  49.         # pb_file = onnx_file + '.pb'
  50.         # rm(pb_file)
  51.         # tf_rep.export_graph(pb_file)
  52.         # input_size_list = [[1, seq_len, isize]]
  53.         # rknn.load_tensorflow(tf_pb=pb_file,
  54.         #                      inputs=['input'],
  55.         #                      outputs=['transpose_326'],
  56.         #                      input_size_list=input_size_list)

  57.     is_quant = False
  58.     if is_quant:
  59.         rknn_file = pt_file + '_quant.rknn'
  60.     else:
  61.         rknn_file = pt_file + '.rknn'

  62.     print('--> Building model')
  63.     ret = rknn.build(do_quantization=is_quant, dataset='./dataset.txt')
  64.     if ret != 0:
  65.         print('rknn.build failed!')
  66.         exit(ret)
  67.     print('build done')
  68.     rm(rknn_file)
  69.     rknn.export_rknn(rknn_file)

  70.     print('--> Init runtime environment')
  71.     if system == 'Windows':
  72.         print("rknn run target rk1808")
  73.         ret = rknn.init_runtime(target='rk1808', target_sub_class='AICS')
  74.     else:
  75.         print("rknn run target self")
  76.         ret = rknn.init_runtime()
  77.     if ret != 0:
  78.         print('Init runtime environment failed')
  79.         exit(ret)
  80.     print('done')

  81.     print('input:')
  82.     print(ix)

  83.     t0 = timer()
  84.     torch_out = m.forward(x)
  85.     t1 = timer()
  86.     print("torch [out]:", x.shape, ', ', torch_out.shape, " time:", t1 - t0)
  87.     print(torch_out.detach().numpy())

  88.     t0 = timer()
  89.     rknn_out = rknn.inference(inputs=[ix], data_type='float32', data_format='nchw')   # , inputs_pass_through=[1]
  90.     t1 = timer()

  91.     if rknn_out is None:
  92.         print('rknn inference failed')
  93.         exit(-100)
  94.     y = np.array(rknn_out[0])
  95.     print("rknn out:", ix.shape, ', ', y.shape, " time:", t1 - t0)
  96.     print(y)

  97.     rknn.release()
复制代码






作者: jefferyzhang    时间: 2020-3-17 08:55
已上报问题
作者: xsky    时间: 2020-3-21 18:09
jefferyzhang 发表于 2020-3-17 08:55
已上报问题

麻烦能不能帮问一下这个有什么头绪吗,现在着急评估项目,谢谢啊

拆开算子试了一下
.load_pytorch应该是对conv1d不支持

onnx可以正确运行conv1d

conv1d → relu → bn  load_onnx运行结果正确
conv1d → bn → relu:pytorch/onnx输出shape为(1, 64, 672)但rknn输出shape为(1, 672, 64)
主楼中的模型就是类似这样的结构重复多次,这里输出shape会被改变,再往后传输的时候会错乱吧



  1. import platform
  2. import os
  3. import torch
  4. import numpy as np
  5. import copy

  6. from rknn.api import RKNN
  7. from timeit import default_timer as timer

  8. import onnx
  9. from onnx_tf.backend import prepare

  10. def rm(file):
  11.     if os.path.exists(file):
  12.         os.remove(file)


  13. class MyConv1d(torch.nn.Module):
  14.     def __init__(self, seq_len, isize):
  15.         super(MyConv1d, self).__init__()

  16.         self._conv1d = torch.nn.Conv1d(seq_len, seq_len, kernel_size=11, stride=1, padding=5, dilation=1, groups=1, bias=False)
  17.         self._seq_len = seq_len
  18.         self._isize = isize
  19.         self._bn = torch.nn.BatchNorm1d(seq_len, eps=1e-3, momentum=0.1)
  20.         self._relu = torch.nn.ReLU()

  21.     def forward(self, x):
  22.         x = x.view(1, self._seq_len, self._isize)
  23.         x = self._conv1d(x)
  24.         x = self._bn(x)
  25.         x = self._relu(x)
  26.         return x


  27. if __name__ == '__main__':

  28.     rknn = RKNN(verbose=True)
  29.     rknn.config(batch_size=1,
  30.                 epochs=1)  # asymmetric_quantized-u8,  quantized_dtype='dynamic_fixed_point-16' , channel_mean_value='0 0 0 1', reorder_channel='0 1 2',
  31.     system = platform.system()

  32.     seq_len = 64
  33.     isize = 672
  34.     pt_file = 'conv1d.pt'

  35.     x = torch.rand((1, seq_len, isize))
  36.     #x = torch.linspace(-1, 1, steps=(isize * seq_len))
  37.     x = x.view(1, 1, seq_len, -1)
  38.     input_size_list = [[1, seq_len, isize]]
  39.     #x1 = torch.full((1, hsize), fill_value=0.2, dtype=torch.float)
  40.     ix = x.numpy()
  41.     pt_m = MyConv1d(seq_len, isize)
  42.     pt_m.eval()
  43.     t0 = timer()
  44.     out = pt_m.forward(x)
  45.     t1 = timer()
  46.     print("torch out, ", out.shape, "time:", t1 - t0)
  47.     print(out.detach().numpy())

  48.     jit_m = torch.jit.trace(pt_m, x)
  49.     rm(pt_file)
  50.     jit_m.save(pt_file)

  51.     jit_m = torch.jit.load(pt_file)
  52.     jit_m.eval()

  53.     t0 = timer()
  54.     out = jit_m.forward(x)
  55.     t1 = timer()
  56.     print("torch out, ", out.shape, "time:", t1 - t0)
  57.     print(out.detach().numpy())

  58.     use_pytorch = False
  59.     if use_pytorch:
  60.         rknn.load_pytorch(model=pt_file, input_size_list=input_size_list)
  61.     else:
  62.         onnx_file = pt_file + '.onnx'
  63.         rm(onnx_file)
  64.         torch.onnx.export(pt_m, x, onnx_file,
  65.                           export_params=True, verbose=False,
  66.                           input_names=['input'],
  67.                           output_names=['output'],
  68.                           opset_version=10
  69.                           )
  70.         tf_model = onnx.load(onnx_file)
  71.         tf_rep = prepare(tf_model)

  72.         tf_ix = ix
  73.         t0 = timer()
  74.         tf_out = tf_rep.run(tf_ix)
  75.         t1 = timer()
  76.         tf_out = np.array(tf_out)
  77.         print("onnx out:", tf_ix.shape, ', ', tf_out.shape, " time:", t1 - t0)
  78.         print(tf_out)
  79.         rknn.load_onnx(model=onnx_file)

  80.         # # onnx转为tf .pb并加载
  81.         # pb_file = onnx_file + '.pb'
  82.         # rm(pb_file)
  83.         # tf_rep.export_graph(pb_file)
  84.         # input_size_list = [[1, seq_len, isize]]
  85.         # rknn.load_tensorflow(tf_pb=pb_file,
  86.         #                      inputs=['input'],
  87.         #                      outputs=['transpose_191'],
  88.         #                      input_size_list=input_size_list)

  89.     is_quant = False
  90.     if is_quant:
  91.         rknn_file = pt_file + '_quant.rknn'
  92.     else:
  93.         rknn_file = pt_file + '.rknn'

  94.     print('--> Building model')
  95.     ret = rknn.build(do_quantization=is_quant, dataset='./dataset.txt')
  96.     if ret != 0:
  97.         print('rknn.build failed!')
  98.         exit(ret)
  99.     print('build done')
  100.     rm(rknn_file)
  101.     rknn.export_rknn(rknn_file)

  102.     print('--> Init runtime environment')
  103.     if system == 'Windows':
  104.         print("rknn run target rk1808")
  105.         ret = rknn.init_runtime(target='rk1808', target_sub_class='AICS')
  106.     else:
  107.         print("rknn run target self")
  108.         ret = rknn.init_runtime()
  109.     if ret != 0:
  110.         print('Init runtime environment failed')
  111.         exit(ret)
  112.     print('done')

  113.     print('input:')
  114.     print(ix)

  115.     pt_m_fp16 = pt_m.cuda()
  116.     pt_m_fp16.half()
  117.     pt_m_fp16.eval()
  118.     t0 = timer()
  119.     pt_y = pt_m_fp16.forward(x.cuda().half())
  120.     t1 = timer()
  121.     print("torch out, cuda:fp16", pt_y.shape, "time:", t1 - t0)
  122.     pt_y = pt_y.cpu().float().detach().numpy()
  123.     print(pt_y)
  124.     # t0 = timer()
  125.     # torch_out = m.forward(x)
  126.     # t1 = timer()
  127.     # print("torch [out]:", x.shape, ', ', torch_out.shape, " time:", t1 - t0)
  128.     # print(torch_out.detach().numpy())

  129.     t0 = timer()
  130.     rknn_out = rknn.inference(inputs=[ix], data_type='float32', data_format='nchw')   # , inputs_pass_through=[1]
  131.     t1 = timer()

  132.     if rknn_out is None:
  133.         print('rknn inference failed')
  134.         exit(-100)
  135.     y = np.array(rknn_out[0])
  136.     print("rknn out:", ix.shape, ', ', y.shape, " time:", t1 - t0)
  137.     print(y)

  138.     if y.shape[1] != pt_y.shape[1] or  y.shape[2] != pt_y.shape[2]:
  139.         print("******* shape err: expect ", pt_y.shape, " but rknn out ", y.shape, " reshape:")
  140. #        y = y.reshape(pt_y.shape[0], pt_y.shape[1], -1)
  141. #        print(y)

  142.     diff = np.abs(y - pt_y)
  143.     cnt = y.shape[0] * y.shape[1] * y.shape[2]
  144.     print("diff:")
  145.     print(diff)
  146.     d = diff[np.where(diff > 0.05)]
  147.     print("warning values(diff > 0.05):", d.shape, " %:", 100.0 * np.array(d.shape, dtype=np.float) // cnt)
  148.     print(d)
  149.     d = diff[np.where(diff > 0.001)]
  150.     print("warning values(diff > 0.001):", d.shape, " %:", 100.0 * np.array(d.shape, dtype=np.float) // cnt)
  151.     print(d)
  152.     d = diff[np.where(diff > 0.0001)]
  153.     print("warning values(diff > 0.0001):", d.shape,  " %:", 100.0 * np.array(d.shape, dtype=np.float) // cnt)
  154.     print(d)

  155.     rknn.release()
复制代码



作者: jefferyzhang    时间: 2020-3-21 22:45
NPU部门回复:
目前pytorch只支持NCHW这种4维输入的卷积,这个模型的卷积输入是3维,暂不支持。




欢迎光临 Toybrick (https://t.rock-chips.com/) Powered by Discuz! X3.3