3399pro来做语音识别，pb模型转rknn后效果非常不好

bobby_jiang 发表于 2019-5-21 10:30
你现在用的rknn-toolkit是什么版本？
另外，你把原始模型和rknn模型，还有代码都贴上来。 ...

rknn-toolkit版本是0.9.9
原始pb模型和rknn模型放在百度云盘里
链接：https://pan.baidu.com/s/1doU0uLZXVBMv9EZ6C_r4rA
提取码：rlsq


keras模型代码

1. input_data = Input(name='the_input', shape=(320000, 1), dtype='float32')
   
2. 
   
3. input_wavImage = Reshape((1600, 200, 1), name='Reshape')(input_data)
   
4. 
   
5. layer_h1 = Conv2D(32, (3,3), use_bias=False, activation='relu', padding='same', kernel_initializer='he_normal')(input_wavImage) # 卷积层
   
6. layer_h1 = Dropout(0.05)(layer_h1)
   
7. layer_h2 = Conv2D(32, (3,3), use_bias=True, activation='relu', padding='same', kernel_initializer='he_normal')(layer_h1) # 卷积层
   
8. layer_h3 = MaxPooling2D(pool_size=2, strides=None, padding="valid")(layer_h2) # 池化层
   
9. #layer_h3 = Dropout(0.2)(layer_h2) # 随机中断部分神经网络连接，防止过拟合
   
10. layer_h3 = Dropout(0.05)(layer_h3)
    
11. layer_h4 = Conv2D(64, (3,3), use_bias=True, activation='relu', padding='same', kernel_initializer='he_normal')(layer_h3) # 卷积层
    
12. layer_h4 = Dropout(0.1)(layer_h4)
    
13. layer_h5 = Conv2D(64, (3,3), use_bias=True, activation='relu', padding='same', kernel_initializer='he_normal')(layer_h4) # 卷积层
    
14. layer_h6 = MaxPooling2D(pool_size=2, strides=None, padding="valid")(layer_h5) # 池化层
    
15. 
    
16. layer_h6 = Dropout(0.1)(layer_h6)
    
17. layer_h7 = Conv2D(128, (3,3), use_bias=True, activation='relu', padding='same', kernel_initializer='he_normal')(layer_h6) # 卷积层
    
18. layer_h7 = Dropout(0.15)(layer_h7)
    
19. layer_h8 = Conv2D(128, (3,3), use_bias=True, activation='relu', padding='same', kernel_initializer='he_normal')(layer_h7) # 卷积层
    
20. layer_h9 = MaxPooling2D(pool_size=2, strides=None, padding="valid")(layer_h8) # 池化层
    
21. 
    
22. layer_h9 = Dropout(0.15)(layer_h9)
    
23. layer_h10 = Conv2D(128, (3,3), use_bias=True, activation='relu', padding='same', kernel_initializer='he_normal')(layer_h9) # 卷积层
    
24. layer_h10 = Dropout(0.2)(layer_h10)
    
25. layer_h11 = Conv2D(128, (3,3), use_bias=True, activation='relu', padding='same', kernel_initializer='he_normal')(layer_h10) # 卷积层
    
26. layer_h12 = MaxPooling2D(pool_size=1, strides=None, padding="valid")(layer_h11) # 池化层
    
27. 
    
28. layer_h12 = Dropout(0.2)(layer_h12)
    
29. layer_h13 = Conv2D(128, (3,3), use_bias=True, activation='relu', padding='same', kernel_initializer='he_normal')(layer_h12) # 卷积层
    
30. layer_h13 = Dropout(0.2)(layer_h13)
    
31. layer_h14 = Conv2D(128, (3,3), use_bias=True, activation='relu', padding='same', kernel_initializer='he_normal')(layer_h13) # 卷积层
    
32. layer_h15 = MaxPooling2D(pool_size=1, strides=None, padding="valid")(layer_h14) # 池化层
    
33. 
    
34. #test=Model(inputs = input_data, outputs = layer_h12)
    
35. #test.summary()
    
36. 
    
37. layer_h16 = Reshape((200, 3200))(layer_h15) #Reshape层
    
38. #layer_h5 = LSTM(256, activation='relu', use_bias=True, return_sequences=True)(layer_h4) # LSTM层
    
39. #layer_h6 = Dropout(0.2)(layer_h5) # 随机中断部分神经网络连接，防止过拟合
    
40. layer_h16 = Dropout(0.3)(layer_h16)
    
41. layer_h17 = Dense(128, activation="relu", use_bias=True, kernel_initializer='he_normal')(layer_h16) # 全连接层
    
42. layer_h17 = Dropout(0.3)(layer_h17)
    
43. layer_h18 = Dense(1422, use_bias=True, kernel_initializer='he_normal')(layer_h17) # 全连接层

复制代码

rknn模型转换代码，且log中没有error或warning

1. from rknn.api import RKNN
   
2. 
   
3. rknn = RKNN(verbose=True, verbose_file='./build.log')
   
4. rknn.config()
   
5. 
   
6. print('--> Loading model')
   
7. rknn.load_tensorflow(tf_pb='./model.pb',
   
8.                      inputs=['the_input'],
   
9.                      outputs=['Activation0/truediv'],
   
10.                      input_size_list=[[320000, 1]])
    
11. print('done')
    
12. 
    
13. # Build Model
    
14. print('--> Building model')
    
15. rknn.build(do_quantization=False)
    
16. print('done')
    
17. 
    
18. rknn.export_rknn('./model.rknn')
    
19. 
    
20. rknn.release()

复制代码

rknn模型inference代码

1. rknn = RKNN()
   
2. rknn.load_rknn('./model.rknn')
   
3. ret = rknn.init_runtime(target='rk3399pro')
   
4. if ret != 0:
   
5.     print('Init runtime environment failed')
   
6.     exit(ret)
   
7. print('done')
   
8. 
   
9. # x_in.size  = (1, 320000, 1)
   
10. x_in, in_len = load_wav_input(filename)
    
11. 
    
12. out = rknn.inference(inputs=[x_in], data_type='float32')

复制代码

同样的环境原来模型没用Reshape直接输入(1600,200,1)可以inference，但最后softmax后np.sum(out)应该=200，pb模型是这样，但rknn模型np.sum(out)是几千完全不对，因此想输入一维向量在模型中reshape，但出之前的错误

protossw512 发表于 2019-5-21 06:05
对，因为它转换的时候和inference的时候长宽两个维度的顺序反了，所以可以正产转换，但是没办法读取来做i ...

可是照你这样说的话模型中就不支持长宽不同的输入reshape操作了？