rk3588ai模型转换问题

[复制链接] · 发表于 2024-4-23 14:26:52

本帖最后由 ZeroK 于 2024-4-23 14:31 编辑

我有个onnx模型的输入输出是这样的，怎么将这个模型转换成rknn格式，在转换的时候std_values  mean_values dynamic_input应该如何配置
模型的netron分析如下
https://netron.app/?url=https:// ... .opt2.onnx.prototxt

模型下载链接如下
https://storage.googleapis.com/ailia-models/whisper/decoder_tiny_fix_kv_cache.opt2.onnx

四个输入如下

name: tokens
tensor: int64[tokens_dynamic_axes_1,tokens_dynamic_axes_2]
name: audio_features
tensor: float32[audio_features_dynamic_axes_1,1500,384]

name: kv_cache
tensor: float32[8,kv_cache_dynamic_axes_1,451,384]

name: offset
tensor: int64

两个输出如下

name: logits
tensor: float32[Castlogits_dim_0,Castlogits_dim_1,51865]

name: output_kv_cache
tensor: float32[8,ScatterNDoutput_kv_cache_dim_1,451,384]

尝试转换代码如下
# Create RKNN object
rknn = RKNN(verbose=False)

dynamic_input = [
         [[1,2],[1,1500,384],[8,1,451,384],[]],
         [[2,3],[2,1500,384],[8,2,451,384],[]],
         [[3,4],[3,1500,384],[8,3,451,384],[]],

inputs=[[2,4,4,4],[1 for _ in range(1500)],[8,3,66],[]]

# Pre-process config
print('--> Config model')
rknn.config(mean_values=inputs, std_values=inputs, target_platform="rk3588",dynamic_input=dynamic_input)
print('done')

# Load model
print('--> Loading model')
ret = rknn.load_onnx(model='./decoder_tiny_fix_kv_cache.opt2.onnx')
if ret != 0:
      print('Load model failed!')
      exit(ret)
print('done')

# Build model
print('--> Building model')
ret = rknn.build(do_quantization= False)
if ret != 0:
      print('Build model failed!')
      exit(ret)
print('done')

# Export rknn model
print('--> Export rknn model')
ret = rknn.export_rknn('./encoder_tiny.rknn')
if ret != 0:
      print('Export rknn model failed!')
      exit(ret)
print('done')

# Release
rknn.release()

转换结果
I rknn-toolkit2 version: 2.0.0b0+9bab5682
--> Config model
W config: Please make sure the model can be dynamic when enable 'config.dynamic_input'!
I The 'dynamic_input' function has been enabled, the MaxShape is dynamic_input[2] = [[3, 4], [3, 1500, 384], [8, 3, 451, 384], []]!
      The following functions are subject to the MaxShape:
         1. The quantified dataset needs to be configured according to MaxShape
         2. The eval_perf or eval_memory return the results of MaxShape
done
--> Loading model
I It is recommended onnx opset 19, but your onnx model opset is 11!
I Loading : 0%|                                                          | 0/337 [00:00<?, ?I Loading :  30%|██████████████▍                               | 101/337 [00:00<00:00, 533.10I Loading : 100%|███████████████████████████████████████████████| 337/337 [00:00<00:00, 1772.95it/s]
W load_onnx: Input dtype is 'int64', the mean_values for input 0 are ignored!
W load_onnx: Input dtype is 'int64', the std_values for input 0 are ignored!
done
--> Building model
E build: Catch exception when building RKNN model!
E build: Traceback (most recent call last):
E build: File "rknn/api/rknn_base.py", line 1977, in rknn.api.rknn_base.RKNNBase.build
E build: File "rknn/api/graph_optimizer.py", line 874, in rknn.api.graph_optimizer.GraphOptimizer.fold_constant
E build: File "rknn/api/load_checker.py", line 91, in rknn.api.load_checker.create_random_data
E build: ValueError: operands could not be broadcast together with shapes (8,2,451,384) (1,3,1,1)
W If you can't handle this error, please try updating to the latest version of the toolkit2 and runtime from:
  https://console.zbox.filez.com/l/I00fc3 (Pwd: rknn)  Path: RKNPU2_SDK / 2.X.X / develop /
  If the error still exists in the latest version, please collect the corresponding error logs and the model,
  convert script, and input data that can reproduce the problem, and then submit an issue on:
  https://redmine.rock-chips.com (Please consult our sales or FAE for the redmine account)
Build model failed!