Toybrick

标题: 加载简单pytorch模型失败 [打印本页]
作者: pfwhnudhwq    时间: 2020-7-15 11:59
标题: 加载简单pytorch模型失败
import torch
import torch.nn as nn
import torch.nn.functional as F
from rknn.api import RKNN


class siamrpn(nn.Module):
    def __init__(self):
        super(siamrpn, self).__init__()

    def forward(self, x, w):
        return F.conv2d(x, w)


if __name__ == "__main__":
    net = siamrpn()
    net.eval()

    torch_model = 'siamrpn.pt'
    rknn_model = 'siamrpn.rknn'

    # export pt
    trace_model = torch.jit.trace(
        net, (torch.Tensor(1, 3, 224, 224), torch.Tensor(1, 3, 4, 4)))
    print(trace_model.code)
    trace_model.save(torch_model)

    # export rknn
    rknn = RKNN()

    # pre-process config
    print('--> config model')
    rknn.config(target_platform='rk3399pro')
    print('done')

    # Load pytorch model
    print('--> Loading model')
    ret = rknn.load_pytorch(model=torch_model, input_size_list=[
                            [3, 224, 224], [3, 4, 4]])
    if ret != 0:
        print('Load pytorch model failed!')
        exit(ret)
    print('done')



报错：
WARNING: Token 'COMMENT' defined, but not used
WARNING: There is 1 unused token
E Try match aten::_convolution_at_17ut0 failed, catch exception!
E Catch exception when loading pytorch model: siamrpn.pt!
E Traceback (most recent call last):
E   File "rknn/base/RKNNlib/converter/convert_pytorch.py", line 1191, in rknn.base.RKNNlib.converter.convert_pytorch.convert_pytorch.match_paragraph_and_param
E   File "rknn/base/RKNNlib/converter/convert_pytorch.py", line 1111, in rknn.base.RKNNlib.converter.convert_pytorch.convert_pytorch._torch_build_acu_layer
E   File "rknn/base/RKNNlib/converter/convert_pytorch.py", line 1085, in rknn.base.RKNNlib.converter.convert_pytorch.convert_pytorch._torch_acu_param_assign
E   File "rknn/base/RKNNlib/converter/convert_pytorch.py", line 1077, in rknn.base.RKNNlib.converter.convert_pytorch.convert_pytorch._torch_acu_blob_assign
E   File "rknn/base/RKNNlib/converter/convert_pytorch.py", line 1070, in rknn.base.RKNNlib.converter.convert_pytorch.convert_pytorch._torch_parase_execute
E   File "<string>", line 1, in <module>
E   File "rknn/base/RKNNlib/converter/convert_pytorch.py", line 893, in rknn.base.RKNNlib.converter.convert_pytorch.convert_pytorch.tensor_to_numpy
E   File "<__array_function__ internals>", line 6, in transpose
E   File "/home/xcq/.local/lib/python3.6/site-packages/numpy/core/fromnumeric.py", line 653, in transpose
E     return _wrapfunc(a, 'transpose', axes)
E   File "/home/xcq/.local/lib/python3.6/site-packages/numpy/core/fromnumeric.py", line 55, in _wrapfunc
E     return _wrapit(obj, method, *args, **kwds)
E   File "/home/xcq/.local/lib/python3.6/site-packages/numpy/core/fromnumeric.py", line 44, in _wrapit
E     result = getattr(asarray(obj), method)(*args, **kwds)
E ValueError: axes don't match array
E During handling of the above exception, another exception occurred:
E Traceback (most recent call last):
E   File "rknn/api/rknn_base.py", line 611, in rknn.api.rknn_base.RKNNBase.load_pytorch
E   File "rknn/base/RKNNlib/app/importer/import_pytorch.py", line 104, in rknn.base.RKNNlib.app.importer.import_pytorch.ImportPytorch.run
E   File "rknn/base/RKNNlib/converter/convert_pytorch.py", line 1202, in rknn.base.RKNNlib.converter.convert_pytorch.convert_pytorch.match_paragraph_and_param
E   File "rknn/api/rknn_log.py", line 312, in rknn.api.rknn_log.RKNNLog.e
E ValueError: Try match aten::_convolution_at_17ut0 failed, catch exception!
Load pytorch model failed!

环境：rk3399pro,debian10,rknn-toolkit1.3.2,pytorch1.2


作者: pfwhnudhwq    时间: 2020-7-15 12:01
  1. import torch
  2. import torch.nn as nn
  3. import torch.nn.functional as F
  4. from rknn.api import RKNN


  7. class siamrpn(nn.Module):
  8.     def __init__(self):
  9.         super(siamrpn, self).__init__()

  11.     def forward(self, x, w):
  12.         return F.conv2d(x, w)


  15. if __name__ == "__main__":
  16.     net = siamrpn()
  17.     net.eval()

  19.     torch_model = 'siamrpn.pt'
  20.     rknn_model = 'siamrpn.rknn'

  22.     # export pt
  23.     trace_model = torch.jit.trace(
  24.         net, (torch.Tensor(1, 3, 224, 224), torch.Tensor(1, 3, 4, 4)))
  25.     print(trace_model.code)
  26.     trace_model.save(torch_model)

  28.     # export rknn
  29.     rknn = RKNN(verbose=True)

  31.     # pre-process config
  32.     print('--> config model')
  33.     rknn.config(target_platform='rk3399pro')
  34.     print('done')

  36.     # Load pytorch model
  37.     print('--> Loading model')
  38.     ret = rknn.load_pytorch(model=torch_model, input_size_list=[
  39.                             [3, 224, 224], [3, 4, 4]])
  40.     if ret != 0:
  41.         print('Load pytorch model failed!')
  42.         exit(ret)
  43.     print('done')
复制代码

这是源码。。。。
作者: pfwhnudhwq    时间: 2020-7-15 12:08
请问：
1.不能使用pytorch的函数接口吗？就是代码里的F.conv2d(nn.Conv2d可以正确加载）
2.有提供3399pro,debian10的pytorch.whl吗，想装一个CPU版的。
作者: jefferyzhang    时间: 2020-7-16 09:08
pfwhnudhwq 发表于 2020-7-15 12:08
请问：
1.不能使用pytorch的函数接口吗？就是代码里的F.conv2d(nn.Conv2d可以正确加载）
2.有提供3399pro,d ...
1.能不能用接口函数请参看文档，貌似按其他客户经验确实很多这种F.xxx函数无法正确转换。
2.只要是toybrick的板子的debian10系统，直接pip3 install 就可以安装预编译好的pytorch
作者: pfwhnudhwq    时间: 2020-7-16 11:38
jefferyzhang 发表于 2020-7-16 09:08
1.能不能用接口函数请参看文档，貌似按其他客户经验确实很多这种F.xxx函数无法正确转换。
2.只要是toybrick ...

谢谢,能否帮我看看下面这段代码,用rknn.load_pytorch和pytorch的inference结果差这么多
  1.     model = 'pt/feature.pt'
  2.     input_size_list = [[3, 127, 127]]
  3.     inp_z = np.ones(shape=[127, 127, 3], dtype=np.uint8)*255
  4.     # rknn inference
  5.     rknn = RKNN()
  6.     rknn.config(reorder_channel='0 1 2')
  7.     ret = rknn.load_pytorch(model=model, input_size_list=input_size_list)
  8.     ret = rknn.build(do_quantization=False)
  9.     ret = rknn.init_runtime()
  10.     outputs = rknn.inference(inputs=[inp_z])
  11.     rknn.release()
  12.     # pytorch inference
  13.     inp_z = np.ones([3, 127, 127], dtype=np.uint8)
  14.     inp_z = torch.Tensor(inp_z).unsqueeze(0)
  15.     module = torch.jit.load(model)
  16.     out = module.forward(inp_z)
  17.     out = out.data.cpu().numpy()
  18.     # difference between rknn and pytorch
  19.     print(np.sum(np.abs(out-outputs)))

  21. WARNING: Token 'COMMENT' defined, but not used
  22. WARNING: There is 1 unused token
  23. W The target_platform is not set in config, using default target platform rk1808.
  24. W [set_chip_platform_env:187]evaluate model on RK1808
  25. 1384.8684
复制代码
作者: jefferyzhang    时间: 2020-7-16 15:04
pfwhnudhwq 发表于 2020-7-16 11:38
谢谢,能否帮我看看下面这段代码,用rknn.load_pytorch和pytorch的inference结果差这么多
...

这个需要debug的。。。
调试思路大概可以这样：
1. 先确认你API、DRV是不是最新的版本（1.3.2-1.3.3），每个版本都解决了很多op的bug。如果drv不匹配，很容易出现结果出错
2. 非量化非预编译是否正常，量化后是否正常 （这两个走的处理器不一样，不排除某个有bug）
3. Dump每一层输出，检查哪层计算错了

然后就可以报bug给我们，我们会提交给NPU部门处理。



欢迎光临 Toybrick (https://t.rock-chips.com/) Powered by Discuz! X3.3