Toybrick

标题: 请问现在RK3399 Pro支持caffe SSD模型吗 [打印本页]

作者: RK3399 时间: 2019-4-12 09:47
标题: 请问现在RK3399 Pro支持caffe SSD模型吗
caffe ssd 模型转换出现问题

模型转换（有3个输出节点，去除DetectionOutput层）：
D Process mbox_loc_70 ...
D RKNN output shape(concat): (0 17680)
D Process output_73 ...
D RKNN output shape(output): (0 17680)
D Process conv3_3_norm_mbox_conf_new_42 ...
D RKNN output shape(convolution): (0 30 40 6)
D Process conv3_3_norm_mbox_conf_perm_43 ...
D RKNN output shape(permute): (0 30 40 6)
D Process conv3_3_norm_mbox_conf_flat_44 ...
D RKNN output shape(flatten): (0 7200)
D Process conv4_3_norm_mbox_conf_50 ...
D RKNN output shape(convolution): (0 15 20 4)
D Process conv4_3_norm_mbox_conf_perm_51 ...
D RKNN output shape(permute): (0 15 20 4)
D Process conv4_3_norm_mbox_conf_flat_52 ...
D RKNN output shape(flatten): (0 1200)
D Process conv5_3_norm_mbox_conf_58 ...
D RKNN output shape(convolution): (0 8 10 4)
D Process conv5_3_norm_mbox_conf_perm_59 ...
D RKNN output shape(permute): (0 8 10 4)
D Process conv5_3_norm_mbox_conf_flat_60 ...
D RKNN output shape(flatten): (0 320)
D Process conv6_3_norm_mbox_conf_66 ...
D RKNN output shape(convolution): (0 4 5 6)
D Process conv6_3_norm_mbox_conf_perm_67 ...
D RKNN output shape(permute): (0 4 5 6)
D Process conv6_3_norm_mbox_conf_flat_68 ...
D RKNN output shape(flatten): (0 120)
D Process mbox_conf_71 ...
D RKNN output shape(concat): (0 8840)
D Process output_74 ...
D RKNN output shape(output): (0 8840)
D Process conv3_3_norm_mbox_priorbox_45 ...
D RKNN output shape(priorbox): (0 14400 1 2)
D Process conv4_3_norm_mbox_priorbox_53 ...
D RKNN output shape(priorbox): (0 2400 1 2)
D Process conv5_3_norm_mbox_priorbox_61 ...
D RKNN output shape(priorbox): (0 640 1 2)
D Process conv6_3_norm_mbox_priorbox_69 ...
D RKNN output shape(priorbox): (0 240 1 2)
D Process mbox_priorbox_72 ...
D RKNN output shape(concat): (0 17680 1 2)
D Process output_75 ...
D RKNN output shape(output): (0 17680 1 2)
I Build SSDNet complete.

运行（只输出了两个节点）：
D [rknn_init:749] Input Tensors:
D [printRKNNTensor:662] index=0 name= n_dims=4 dims=[1 3 240 320] n_elems=230400 size=460800 fmt=NCHW type=FP16 qnt_type=NONE fl=-86 zp=2081219754 scale=0.000000
D [rknn_init:762] Output Tensors:
D [printRKNNTensor:662] index=0 name= n_dims=2 dims=[0 0 1 17680] n_elems=17680 size=35360 fmt=NCHW type=FP16 qnt_type=NONE fl=-86 zp=2081219754 scale=0.000000
D [printRKNNTensor:662] index=1 name= n_dims=2 dims=[0 0 1 8840] n_elems=8840 size=17680 fmt=NCHW type=FP16 qnt_type=NONE fl=-86 zp=2081219754 scale=0.000000
D [rknn_inputs_set:964] 0 pass_through=0
D [rknn_inputs_set:982] 0 input.type=3
D [__input_copy:533] src->type=3 src->fmt=1 input_reorder=1
D [__input_copy:573] UINT8->FLOAT16
I [RK_nn_ConvertTensorToData:495]Create 35360 data.
I [RK_nn_ConvertTensorToData:495]Create 17680 data.
out: 2 [array([[-0.1295166 , 1.0966797 , -2.53125 , ..., -0.48291016,
-0.51171875, -0.5683594 ]], dtype=float32), array([[ 1.296875 , -0.6689453 , 1.7861328 , ..., -0.8808594 ,
0.6894531 , -0.81884766]], dtype=float32)]

模型只运行了两个输出节点（SSD去除了DetectionOutput层），但第三个mbox_priorbox_72后的输出节点output_75并没有输出，请问是什么原因？是不支持PriorBox层吗？还是框架本身存在bug?

作者: RK3399 时间: 2019-4-12 16:04
运行时间output_num是2，是该框架不支持高于2的输出节点个数吗？本人看来RKNN的库是支持PriorBox的
不管是python下还是Android下运行都无法获取第三个节点的数据，说明模型转换时就丢弃了第三个节点，或者根本是不支持的？

W verbose file path is invalid, debug info will not dump to file.
I [rknn_CreateRKNN:3132]rknn_CreateRKNN

I [rknn_CreateRKNN:3202]NET_NODE_NUM=49

I [rknn_CreateRKNN:3211]NET_NORM_TENSOR_NUM=3

I [rknn_CreateRKNN:3220]NET_CONST_TENSOR_NUM=52

I [rknn_CreateRKNN:3229]NET_VIRTUAL_TENSOR_NUM =49

I [rknn_CreateRKNN:3240]input_num =1

I [rknn_CreateRKNN:3248]output_num =2

D [setup_nodes:2408]setup node:0
I [setup_nodes:2412]name=conv1_1_1_relu1_1_2
I [setup_nodes:2420]uid=2
I [str2op:117]op=RK_NN_OP_CONV_RELU, id=3
I [setup_nodes:2428]op=3

作者: jefferyzhang 时间: 2019-4-12 16:08
rknn支持多输出，请参看我们配套给出的ssd demo

作者: RK3399 时间: 2019-4-12 16:50
你们给出的demo中仍然只有两个输出节点，而且没有基于caffe模型的，如下，第三个节点数据使用的是文本文件静态导入的prior box，我不太清楚这样做的原因？

# Create RKNN object
rknn = RKNN()

# Config for Model Input PreProcess
rknn.config(channel_mean_value='128 128 128 128', reorder_channel='0 1 2')

# Load TensorFlow Model
print('--> Loading model')
rknn.load_tensorflow(tf_pb='./ssd_mobilenet_v1_coco_2017_11_17.pb',
                     inputs=['FeatureExtractor/MobilenetV1/MobilenetV1/Conv2d_0/BatchNorm/batchnorm/mul_1'],
                     outputs=['concat', 'concat_1'],
                     input_size_list=[[INPUT_SIZE, INPUT_SIZE, 3]])
print('done')

# Build Model
print('--> Building model')
rknn.build(do_quantization=True, dataset='./dataset.txt')
print('done')

# Export RKNN Model
rknn.export_rknn('./ssd_mobilenet_v1_coco.rknn')

# Direct Load RKNN Model
# rknn.load_rknn('./ssd_mobilenet_v1_coco.rknn')

# Set inputs
orig_img = cv2.imread('./road.bmp')
img = cv2.cvtColor(orig_img, cv2.COLOR_BGR2RGB)
img = cv2.resize(img, (INPUT_SIZE, INPUT_SIZE), interpolation=cv2.INTER_CUBIC)

# init runtime environment
print('--> Init runtime environment')
ret = rknn.init_runtime()
if ret != 0:
      print('Init runtime environment failed')
      exit(ret)
print('done')

# Inference
print('--> Running model')
outputs = rknn.inference(inputs=[img])
print('done')
print('inference result: ', outputs)

predictions = outputs[1].reshape((1, NUM_RESULTS, 4))
outputClasses = outputs[0].reshape((1, NUM_RESULTS, NUM_CLASSES))

box_priors = load_box_priors()

作者: RK3399 时间: 2019-4-12 17:00
本帖最后由 RK3399 于 2019-4-19 16:53 编辑

已经解决了，主要是RKNN的支持原因

作者: RK3399 时间: 2019-4-18 10:55
本帖最后由 RK3399 于 2019-4-19 16:51 编辑

解决了，参考 http://t.rock-chips.com/forum.ph ... =305&extra=page%3D1
我写了一个仓库，可以参考[url]https://github.com/Pinnh/NPU_CaffeSSD[/url]

欢迎光临 Toybrick (https://t.rock-chips.com/)