|
因为需要再RK3588部署一个单目标跟踪项目,众所周知,跟踪项目通常为两个模型,一个特征提取模型,一个特征融合模型
特征提取模型输入为cv Mat,输出为tensor,具体如下:
- [Instance branch]: INPUT_ATTR
- [TENSOR INDEX: 0]
- [TENSOR NAME: input]
- [TENSOR N_DIMS: 4]
- [TENSOR DIMS: [1, 255, 255, 3, ]]
- [TENSOR N_ELEMS: 195075]
- [TENSOR SIZE: 195075]
- [TENSOR W_STRIDE: 256]
- [TENSOR SIZE_WITH_STRIDE: 195840]
- [TENSOR FORMAT: NHWC]
- [TENSOR TYPE: INT8]
- [TENSOR QNT_TYPE: AFFINE]
- [TENSOR ZP: -128]
- [TENSOR SCALE: 1]
- [Instance branch]: OUTPUT_ATTR
- [TENSOR INDEX: 0]
- [TENSOR NAME: output]
- [TENSOR N_DIMS: 4]
- [TENSOR DIMS: [1, 48, 16, 16, ]]
- [TENSOR N_ELEMS: 12288]
- [TENSOR SIZE: 12288]
- [TENSOR W_STRIDE: 0]
- [TENSOR SIZE_WITH_STRIDE: 12288]
- [TENSOR FORMAT: NCHW]
- [TENSOR TYPE: INT8]
- [TENSOR QNT_TYPE: AFFINE]
- [TENSOR ZP: 3]
- [TENSOR SCALE: 0.14404]
特征融合模型输入为tensor, 输出为tensor,具体如下:
- [Correlation branch]: INPUT_ATTR
- [TENSOR INDEX: 0]
- [TENSOR NAME: input1]
- [TENSOR N_DIMS: 4]
- [TENSOR DIMS: [1, 8, 8, 48, ]]
- [TENSOR N_ELEMS: 3072]
- [TENSOR SIZE: 3072]
- [TENSOR W_STRIDE: 8]
- [TENSOR SIZE_WITH_STRIDE: 3072]
- [TENSOR FORMAT: NHWC]
- [TENSOR TYPE: INT8]
- [TENSOR QNT_TYPE: AFFINE]
- [TENSOR ZP: -22]
- [TENSOR SCALE: 0.124435]
- [Correlation branch]: INPUT_ATTR
- [TENSOR INDEX: 1]
- [TENSOR NAME: input2]
- [TENSOR N_DIMS: 4]
- [TENSOR DIMS: [1, 16, 16, 48, ]]
- [TENSOR N_ELEMS: 12288]
- [TENSOR SIZE: 12288]
- [TENSOR W_STRIDE: 16]
- [TENSOR SIZE_WITH_STRIDE: 12288]
- [TENSOR FORMAT: NHWC]
- [TENSOR TYPE: INT8]
- [TENSOR QNT_TYPE: AFFINE]
- [TENSOR ZP: -4]
- [TENSOR SCALE: 0.189816]
- [Correlation branch]: OUTPUT_ATTR
- [TENSOR INDEX: 0]
- [TENSOR NAME: output1]
- [TENSOR N_DIMS: 4]
- [TENSOR DIMS: [1, 2, 16, 16, ]]
- [TENSOR N_ELEMS: 512]
- [TENSOR SIZE: 512]
- [TENSOR W_STRIDE: 0]
- [TENSOR SIZE_WITH_STRIDE: 4096]
- [TENSOR FORMAT: NCHW]
- [TENSOR TYPE: INT8]
- [TENSOR QNT_TYPE: AFFINE]
- [TENSOR ZP: -5]
- [TENSOR SCALE: 0.0481221]
- [Correlation branch]: OUTPUT_ATTR
- [TENSOR INDEX: 1]
- [TENSOR NAME: output2]
- [TENSOR N_DIMS: 4]
- [TENSOR DIMS: [1, 4, 16, 16, ]]
- [TENSOR N_ELEMS: 1024]
- [TENSOR SIZE: 1024]
- [TENSOR W_STRIDE: 0]
- [TENSOR SIZE_WITH_STRIDE: 4096]
- [TENSOR FORMAT: NCHW]
- [TENSOR TYPE: INT8]
- [TENSOR QNT_TYPE: AFFINE]
- [TENSOR ZP: -128]
- [TENSOR SCALE: 0.315689]
当前我采用的编程思路为特征提取模型采用零拷贝API
因为特征融合网络输入通道不为1,3,4,所以只能采用通用API
问题在于:
特征提取网络输出格式为NCHW,数据类型为INT8,特征融合网络输入NHWC,数据类型为INT8
我针对特征提取网络输出做了如下设置:
- attr_output_z.fmt = RKNN_TENSOR_NCHW;
- attr_output_z.type = RKNN_TENSOR_FLOAT32;
- ret = rknn_set_io_mem(ctx_z, mem_input_z, &attr_input_z);
对于特征融合网络如下设置:
- rknn_input input;
- for (int i = 0; i < io_num.n_input; i++) {
- input.index = i;
- input.type = RKNN_TENSOR_FLOAT32;
- input.size = attr_inputs[i].size * sizeof(float);
- input.fmt = RKNN_TENSOR_NCHW;
- input.pass_through = 0;
- inputs.push_back(input);
- }
两个模型之间的数据赋值,即特征提取网络结果作为特征融合模型输入代码如下:
- inputs[0].buf = mem_output_z->virt_addr;
报错信息如下:
- E RKNN: [02:51:54.114] Meet unsupported src layout for normalize: 2
- E RKNN: [02:51:54.114] rknn_inputs_set, normalize error(-1) index=0
- E RKNN: [02:51:54.114] Meet unsupported src layout for normalize: 2
- E RKNN: [02:51:54.114] rknn_inputs_set, normalize error(-1) index=1
请问是思路有问题,比如统一使用通用API?还是需要手动将NCHW的输入转换为NHWC,在进行操作?
|
|