Toybrick

rk3588ai模型转换问题

ZeroK

新手上路

积分
9
楼主
发表于 2024-4-23 14:26:52    查看: 2811|回复: 0 | [复制链接]    打印 | 只看该作者
本帖最后由 ZeroK 于 2024-4-23 14:31 编辑

我有个onnx模型的输入输出是这样的,怎么将这个模型转换成rknn格式,在转换的时候std_values  mean_values dynamic_input应该如何配置
模型的netron分析如下
https://netron.app/?url=https:// ... .opt2.onnx.prototxt


模型下载链接如下
https://storage.googleapis.com/ailia-models/whisper/decoder_tiny_fix_kv_cache.opt2.onnx



四个输入如下

name: tokens
tensor: int64[tokens_dynamic_axes_1,tokens_dynamic_axes_2]
name: audio_features
tensor: float32[audio_features_dynamic_axes_1,1500,384]



name: kv_cache
tensor: float32[8,kv_cache_dynamic_axes_1,451,384]



name: offset
tensor: int64

两个输出如下



name: logits
tensor: float32[Castlogits_dim_0,Castlogits_dim_1,51865]



name: output_kv_cache
tensor: float32[8,ScatterNDoutput_kv_cache_dim_1,451,384]


尝试转换代码如下
    # Create RKNN object
    rknn = RKNN(verbose=False)

    dynamic_input = [
            [[1,2],[1,1500,384],[8,1,451,384],[]],
            [[2,3],[2,1500,384],[8,2,451,384],[]],   
            [[3,4],[3,1500,384],[8,3,451,384],[]],


    inputs=[[2,4,4,4],[1 for _ in range(1500)],[8,3,66],[]]

    # Pre-process config
    print('--> Config model')
    rknn.config(mean_values=inputs, std_values=inputs, target_platform="rk3588",dynamic_input=dynamic_input)
    print('done')

    # Load model
    print('--> Loading model')
    ret = rknn.load_onnx(model='./decoder_tiny_fix_kv_cache.opt2.onnx')
    if ret != 0:
        print('Load model failed!')
        exit(ret)
    print('done')

    # Build model
    print('--> Building model')
    ret = rknn.build(do_quantization= False)
    if ret != 0:
        print('Build model failed!')
        exit(ret)
    print('done')

    # Export rknn model
    print('--> Export rknn model')
    ret = rknn.export_rknn('./encoder_tiny.rknn')
    if ret != 0:
        print('Export rknn model failed!')
        exit(ret)
    print('done')

    # Release
    rknn.release()


转换结果
I rknn-toolkit2 version: 2.0.0b0+9bab5682
--> Config model
W config: Please make sure the model can be dynamic when enable 'config.dynamic_input'!
I The 'dynamic_input' function has been enabled, the MaxShape is dynamic_input[2] = [[3, 4], [3, 1500, 384], [8, 3, 451, 384], []]!
          The following functions are subject to the MaxShape:
            1. The quantified dataset needs to be configured according to MaxShape
            2. The eval_perf or eval_memory return the results of MaxShape
done
--> Loading model
I It is recommended onnx opset 19, but your onnx model opset is 11!
I Loading :   0%|                                                           | 0/337 [00:00<?, ?I Loading :  30%|██████████████▍                                 | 101/337 [00:00<00:00, 533.10I Loading : 100%|███████████████████████████████████████████████| 337/337 [00:00<00:00, 1772.95it/s]
W load_onnx: Input dtype is 'int64', the mean_values for input 0 are ignored!
W load_onnx: Input dtype is 'int64', the std_values for input 0 are ignored!
done
--> Building model
E build: Catch exception when building RKNN model!
E build: Traceback (most recent call last):
E build:   File "rknn/api/rknn_base.py", line 1977, in rknn.api.rknn_base.RKNNBase.build
E build:   File "rknn/api/graph_optimizer.py", line 874, in rknn.api.graph_optimizer.GraphOptimizer.fold_constant
E build:   File "rknn/api/load_checker.py", line 91, in rknn.api.load_checker.create_random_data
E build: ValueError: operands could not be broadcast together with shapes (8,2,451,384) (1,3,1,1)
W If you can't handle this error, please try updating to the latest version of the toolkit2 and runtime from:
  https://console.zbox.filez.com/l/I00fc3 (Pwd: rknn)  Path: RKNPU2_SDK / 2.X.X / develop /
  If the error still exists in the latest version, please collect the corresponding error logs and the model,
  convert script, and input data that can reproduce the problem, and then submit an issue on:
  https://redmine.rock-chips.com (Please consult our sales or FAE for the redmine account)
Build model failed!




回复

使用道具 举报

您需要登录后才可以回帖 登录 | 立即注册

本版积分规则

产品中心 购买渠道 开源社区 Wiki教程 资料下载 关于Toybrick


快速回复 返回顶部 返回列表