Toybrick

标题: TB-RK3399ProD 3G(2G+1G)转换模型后推理报找不到NPU [打印本页]

作者: ooo 时间: 2022-7-31 16:40
标题: TB-RK3399ProD 3G(2G+1G)转换模型后推理报找不到NPU
以下是整个运行流程（请官方技术解答一下）：

toybrick@debian10:~/tensorflow/ssd_mobilenet_v1$ python3 test.py--> Config modeldone--> Loading modelW:tensorflow:From /home/toybrick/.local/lib/python3.7/site-packages/rknn/api/rknn.py:75: extract_sub_graph (from tensorflow.python.framework.graph_util_impl) is deprecated and will be removed in a future version.Instructions for updating:Use `tf.compat.v1.graph_util.extract_sub_graph`W Network may not support batch > 1 !W Network may not support batch > 1 !W Network may not support batch > 1 !W Network may not support batch > 1 !W Network may not support batch > 1 !W Network may not support batch > 1 !W Network may not support batch > 1 !W Network may not support batch > 1 !W Network may not support batch > 1 !W Network may not support batch > 1 !W Network may not support batch > 1 !W Network may not support batch > 1 !/home/toybrick/.local/lib/python3.7/site-packages/rknn/api/rknn.py:75: FutureWarning: Possible nested set at position 7 std_values=std_values, size_with_batch=size_with_batch)done--> Building modelW The target_platform is not set in config, using default target platform rk1808.W:tensorflow:From /home/toybrick/.local/lib/python3.7/site-packages/tensorflow_core/python/framework/func_graph.py:915: py_func (from tensorflow.python.ops.script_ops) is deprecated and will be removed in a future version.Instructions for updating:tf.py_func is deprecated in TF V2. Instead, there are two options available in V2. - tf.py_function takes a python function which manipulates tf eager tensors instead of numpy arrays. It's easy to convert a tf eager tensor to an ndarray (just call tensor.numpy()) but having access to eager tensors means `tf.py_function`s can use accelerators such as GPUs as well as being differentiable using a gradient tape. - tf.numpy_function maintains the semantics of the deprecated tf.py_func (it is not differentiable, and manipulates numpy arrays). It drops the stateful argument making all functions stateful. W:tensorflow:From /home/toybrick/.local/lib/python3.7/site-packages/tensorflow_core/python/ops/resource_variable_ops.py:1630: calling BaseResourceVariable.__init__ (from tensorflow.python.ops.resource_variable_ops) with constraint is deprecated and will be removed in a future version.Instructions for updating:If using Keras pass *_constraint arguments to layers.done--> Export RKNN modeldone--> Init runtime environmentE Only support ntb mode on ARM64 platform. But can not find device with ntb mode.E Catch exception when init runtime!E Traceback (most recent call last):E File "rknn/api/rknn_base.py", line 815, in rknn.api.rknn_base.RKNNBase.init_runtimeE File "rknn/api/rknn_runtime.py", line 170, in rknn.api.rknn_runtime.RKNNRuntime.__init__E File "rknn/api/rknn_platform_utils.py", line 307, in rknn.api.rknn_platform_utils.start_ntp_or_adbE Exception: Init runtime environment failed!Init runtime environment failed

作者: ooo 时间: 2022-7-31 16:44
本帖最后由 ooo 于 2022-7-31 16:46 编辑

上面格式不行，下面重新上传

作者: ooo 时间: 2022-7-31 16:45
toybrick@debian10:~/tensorflow/ssd_mobilenet_v1$ python3 test.py
--> Config model
done
--> Loading model
W:tensorflow:From /home/toybrick/.local/lib/python3.7/site-packages/rknn/api/rknn.py:75: extract_sub_graph (from tensorflow.python.framework.graph_util_impl) is deprecated and will be removed in a future version.
Instructions for updating:
Use `tf.compat.v1.graph_util.extract_sub_graph`
W Network may not support batch > 1 !
W Network may not support batch > 1 !
W Network may not support batch > 1 !
W Network may not support batch > 1 !
W Network may not support batch > 1 !
W Network may not support batch > 1 !
W Network may not support batch > 1 !
W Network may not support batch > 1 !
W Network may not support batch > 1 !
W Network may not support batch > 1 !
W Network may not support batch > 1 !
W Network may not support batch > 1 !
/home/toybrick/.local/lib/python3.7/site-packages/rknn/api/rknn.py:75: FutureWarning: Possible nested set at position 7
std_values=std_values, size_with_batch=size_with_batch)
done
--> Building model
W The target_platform is not set in config, using default target platform rk1808.
W:tensorflow:From /home/toybrick/.local/lib/python3.7/site-packages/tensorflow_core/python/framework/func_graph.py:915: py_func (from tensorflow.python.ops.script_ops) is deprecated and will be removed in a future version.
Instructions for updating:
tf.py_func is deprecated in TF V2. Instead, there are two
options available in V2.
- tf.py_function takes a python function which manipulates tf eager
tensors instead of numpy arrays. It's easy to convert a tf eager tensor to
an ndarray (just call tensor.numpy()) but having access to eager tensors
means `tf.py_function`s can use accelerators such as GPUs as well as
being differentiable using a gradient tape.
- tf.numpy_function maintains the semantics of the deprecated tf.py_func
(it is not differentiable, and manipulates numpy arrays). It drops the
stateful argument making all functions stateful.

W:tensorflow:From /home/toybrick/.local/lib/python3.7/site-packages/tensorflow_core/python/ops/resource_variable_ops.py:1630: calling BaseResourceVariable.__init__ (from tensorflow.python.ops.resource_variable_ops) with constraint is deprecated and will be removed in a future version.
Instructions for updating:
If using Keras pass *_constraint arguments to layers.
done
--> Export RKNN model
done
--> Init runtime environment
E Only support ntb mode on ARM64 platform. But can not find device with ntb mode.
E Catch exception when init runtime!
E Traceback (most recent call last):
E File "rknn/api/rknn_base.py", line 815, in rknn.api.rknn_base.RKNNBase.init_runtime
E File "rknn/api/rknn_runtime.py", line 170, in rknn.api.rknn_runtime.RKNNRuntime.__init__
E File "rknn/api/rknn_platform_utils.py", line 307, in rknn.api.rknn_platform_utils.start_ntp_or_adb
E Exception: Init runtime environment failed!
Init runtime environment failed

作者: ooo 时间: 2022-7-31 16:50
补充：试过烧录当前固件下载页面的Linux固件（TB-RK3399ProD: 百度网盘 OneDrive），还是如此

作者: ooo 时间: 2022-7-31 16:53
补充：Android8.1下跑RK的例程，没有检测框（估计是没用到npu），官方技术解答一下

作者: ooo 时间: 2022-7-31 17:21
请官方技术解答一下，在线等

作者: jefferyzhang 时间: 2022-7-31 23:05
看过去是Init runtime时候target写错了吧

作者: ooo 时间: 2022-7-31 23:09
import numpy as np

import re
import math
import random
import cv2

from rknn.api import RKNN

INPUT_SIZE = 300

NUM_RESULTS = 1917
NUM_CLASSES = 2

Y_SCALE = 10.0
X_SCALE = 10.0
H_SCALE = 5.0
W_SCALE = 5.0

def expit(x):
return 1. / (1. + math.exp(-x))

def unexpit(y):
return -1.0 * math.log((1.0 / y) - 1.0);

def CalculateOverlap(xmin0, ymin0, xmax0, ymax0, xmin1, ymin1, xmax1, ymax1):
w = max(0.0, min(xmax0, xmax1) - max(xmin0, xmin1))
h = max(0.0, min(ymax0, ymax1) - max(ymin0, ymin1))
i = w * h
u = (xmax0 - xmin0) * (ymax0 - ymin0) + (xmax1 - xmin1) * (ymax1 - ymin1) - i

if u <= 0.0:
      return 0.0

return i / u

def load_box_priors():
box_priors_ = []
fp = open('./box_priors.txt', 'r')
ls = fp.readlines()
for s in ls:
      aList = re.findall('([-+]?\d+(\.\d*)?|\.\d+)([eE][-+]?\d+)?', s)
      for ss in aList:
         aNum = float((ss[0]+ss[2]))
         box_priors_.append(aNum)
fp.close()

box_priors = np.array(box_priors_)
box_priors = box_priors.reshape(4, NUM_RESULTS)

return box_priors

if __name__ == '__main__':

# Create RKNN object
rknn = RKNN()

# Config for Model Input PreProcess
print('--> Config model')
rknn.config(mean_values=[[127.5, 127.5, 127.5]], std_values=[[127.5, 127.5, 127.5]], reorder_channel='0 1 2')
print('done')

# Load TensorFlow Model
print('--> Loading model')
ret = rknn.load_tensorflow(tf_pb='./tflite_graph.pb',
                           inputs=['normalized_input_image_tensor'],
                           outputs=['concat', 'concat_1'],
                           input_size_list=[[INPUT_SIZE, INPUT_SIZE, 3]])
if ret != 0:
      print('Load model failed!')
      exit(ret)
print('done')

# Build Model
print('--> Building model')
ret = rknn.build(do_quantization=True, dataset='./dataset.txt')
if ret != 0:
      print('Build model failed!')
      exit(ret)
print('done')

# Export RKNN Model
print('--> Export RKNN model')
rknn.export_rknn('./ssd.rknn')
if ret != 0:
      print('Export RKNN model failed!')
      exit(ret)
print('done')

# Direct Load RKNN Model
# rknn.load_rknn('./ssd.rknn')

# Set inputs
orig_img = cv2.imread('./horizontal.bmp')
img = cv2.cvtColor(orig_img, cv2.COLOR_BGR2RGB)
img = cv2.resize(img, (INPUT_SIZE, INPUT_SIZE), interpolation=cv2.INTER_CUBIC)

# init runtime environment
print('--> Init runtime environment')
ret = rknn.init_runtime()
if ret != 0:
      print('Init runtime environment failed')
      exit(ret)
print('done')

# Inference
print('--> Running model')
outputs = rknn.inference(inputs=[img])
print('done')

predictions = outputs[0].reshape((1, NUM_RESULTS, 4))
outputClasses = outputs[1].reshape((1, NUM_RESULTS, NUM_CLASSES))
candidateBox = np.zeros([2, NUM_RESULTS], dtype=int)
vaildCnt = 0

box_priors = load_box_priors()

# Post Process
# got valid candidate box
for i in range(0, NUM_RESULTS):
      topClassScore = -1000
      topClassScoreIndex = -1

      # Skip the first catch-all class.
      for j in range(1, NUM_CLASSES):
         score = expit(outputClasses[0][i][j]);

         if score > topClassScore:
            topClassScoreIndex = j
            topClassScore = score

      if topClassScore > 0.4:
         candidateBox[0][vaildCnt] = i
         candidateBox[1][vaildCnt] = topClassScoreIndex
         vaildCnt += 1

# calc position
for i in range(0, vaildCnt):
      if candidateBox[0][i] == -1:
         continue

      n = candidateBox[0][i]
      ycenter = predictions[0][n][0] / Y_SCALE * box_priors[2][n] + box_priors[0][n]
      xcenter = predictions[0][n][1] / X_SCALE * box_priors[3][n] + box_priors[1][n]
      h = math.exp(predictions[0][n][2] / H_SCALE) * box_priors[2][n]
      w = math.exp(predictions[0][n][3] / W_SCALE) * box_priors[3][n]

      ymin = ycenter - h / 2.
      xmin = xcenter - w / 2.
      ymax = ycenter + h / 2.
      xmax = xcenter + w / 2.

      predictions[0][n][0] = ymin
      predictions[0][n][1] = xmin
      predictions[0][n][2] = ymax
      predictions[0][n][3] = xmax

# NMS
for i in range(0, vaildCnt):
      if candidateBox[0][i] == -1:
         continue

      n = candidateBox[0][i]
      xmin0 = predictions[0][n][1]
      ymin0 = predictions[0][n][0]
      xmax0 = predictions[0][n][3]
      ymax0 = predictions[0][n][2]

      for j in range(i+1, vaildCnt):
         m = candidateBox[0][j]

         if m == -1:
            continue

         xmin1 = predictions[0][m][1]
         ymin1 = predictions[0][m][0]
         xmax1 = predictions[0][m][3]
         ymax1 = predictions[0][m][2]

         iou = CalculateOverlap(xmin0, ymin0, xmax0, ymax0, xmin1, ymin1, xmax1, ymax1)

         if iou >= 0.45:
            candidateBox[0][j] = -1

# Draw result
for i in range(0, vaildCnt):
      if candidateBox[0][i] == -1:
         continue

      n = candidateBox[0][i]

      xmin = max(0.0, min(1.0, predictions[0][n][1])) * INPUT_SIZE
      ymin = max(0.0, min(1.0, predictions[0][n][0])) * INPUT_SIZE
      xmax = max(0.0, min(1.0, predictions[0][n][3])) * INPUT_SIZE
      ymax = max(0.0, min(1.0, predictions[0][n][2])) * INPUT_SIZE

      # print("%d @ (%d, %d) (%d, %d) score=%f" % (topClassScoreIndex, xmin, ymin, xmax, ymax, topClassScore))
      cv2.rectangle(orig_img, (int(xmin), int(ymin)), (int(xmax), int(ymax)),
                  (random.random()*255, random.random()*255, random.random()*255), 3)

cv2.imwrite("out.jpg", orig_img)

# Evaluate Perf on Simulator
print('--> Evaluate model performance')
rknn.eval_perf(inputs=[img], is_print=True)
print('done')

# Release RKNN Context
rknn.release()

作者: ooo 时间: 2022-7-31 23:17

jefferyzhang 发表于 2022-7-31 23:05
看过去是Init runtime时候target写错了吧

>>> from rknn.api import RKNN
>>> rknn = RKNN()
>>> rknn.list_devices()
*************************
None devices connected.
*************************
([], [])
>>>

作者: ooo 时间: 2022-7-31 23:26
补充：我这有两个TB-RK3399ProD 3G(2G+1G),但是另外一个却可以正常转换模型后推理,同样的文件,同样的操作

作者: jefferyzhang 时间: 2022-8-1 08:53

ooo 发表于 2022-7-31 23:26
补充：我这有两个TB-RK3399ProD 3G(2G+1G),但是另外一个却可以正常转换模型后推理,同样的文件,同样的操作 ...

lsusb看下有没有2207的设备，还有固件是否是V3.0+的？

作者: ooo 时间: 2022-8-1 09:27

jefferyzhang 发表于 2022-8-1 08:53
lsusb看下有没有2207的设备，还有固件是否是V3.0+的？

toybrick@debian10:~$ lsusb
Bus 004 Device 004: ID 05e3:0616 Genesys Logic, Inc. hub
Bus 004 Device 001: ID 1d6b:0003 Linux Foundation 3.0 root hub
Bus 003 Device 012: ID 0000:3825
Bus 003 Device 011: ID 1c4f:0002 SiGma Micro Keyboard TRACER Gamma Ivory
Bus 003 Device 010: ID 05e3:0610 Genesys Logic, Inc. 4-port hub
Bus 003 Device 001: ID 1d6b:0002 Linux Foundation 2.0 root hub
Bus 007 Device 001: ID 1d6b:0001 Linux Foundation 1.1 root hub
Bus 005 Device 001: ID 1d6b:0002 Linux Foundation 2.0 root hub
Bus 006 Device 001: ID 1d6b:0001 Linux Foundation 1.1 root hub
Bus 002 Device 002: ID 0bda:d723 Realtek Semiconductor Corp.
Bus 002 Device 001: ID 1d6b:0002 Linux Foundation 2.0 root hub
toybrick@debian10:~$

作者: ooo 时间: 2022-8-1 09:28

jefferyzhang 发表于 2022-8-1 08:53
lsusb看下有没有2207的设备，还有固件是否是V3.0+的？

TB-RK3399ProD_Release_V3.1.rar

作者: jefferyzhang 时间: 2022-8-1 09:34
你这个看过去都没NPU，你还是找淘宝客服保修吧

作者: ooo 时间: 2022-8-1 09:35

jefferyzhang 发表于 2022-8-1 09:34
你这个看过去都没NPU，你还是找淘宝客服保修吧

这个是NPU坏了?

作者: jefferyzhang 时间: 2022-8-1 09:40

ooo 发表于 2022-8-1 09:35
这个是NPU坏了?

不知道啊，看不出来，如果你烧了完整板子固件还是这样，你就直接找淘宝客服问问。

作者: ooo 时间: 2022-8-1 09:42

jefferyzhang 发表于 2022-8-1 09:40
不知道啊，看不出来，如果你烧了完整板子固件还是这样，你就直接找淘宝客服问问。
...

这个保修费用大概多少?

作者: ooo 时间: 2022-8-1 09:45

jefferyzhang 发表于 2022-8-1 09:40
不知道啊，看不出来，如果你烧了完整板子固件还是这样，你就直接找淘宝客服问问。
...

已经不在保修期了,这个维修费用大概多少?

作者: jefferyzhang 时间: 2022-8-1 09:53

ooo 发表于 2022-8-1 09:45
已经不在保修期了,这个维修费用大概多少?

我哪知道

我只是个打工的，还只是个软件 ....
你直接问淘宝客服，他们肯定要先检测下哪里坏了才知道多少钱吧。

作者: ooo 时间: 2022-8-1 10:11

jefferyzhang 发表于 2022-8-1 09:53
我哪知道我只是个打工的，还只是个软件 ....
你直接问淘宝客服，他们肯定要先检测下哪里坏了才 ...

对了,如果过了保修期,可以寄给你们维修么?

作者: jefferyzhang 时间: 2022-8-1 10:47

ooo 发表于 2022-8-1 10:11
对了,如果过了保修期,可以寄给你们维修么?

都是要走淘宝客服，寄给工厂修的。
过保也是可以修的，具体都要问客服，不是走我们这里的

作者: ooo 时间: 2022-8-2 12:08

jefferyzhang 发表于 2022-8-1 10:47
都是要走淘宝客服，寄给工厂修的。
过保也是可以修的，具体都要问客服，不是走我们这里的 ...

深圳市嵌维科技科技也是淘宝店么，好像是瑞芯微旗下的一个子公司，也不可以直接寄给你们检修？

作者: jefferyzhang 时间: 2022-8-2 12:15

ooo 发表于 2022-8-2 12:08
深圳市嵌维科技科技也是淘宝店么，好像是瑞芯微旗下的一个子公司，也不可以直接寄给你们检修？ ...

没有听说过这家，RK哪来子公司我都没听过。你还是联系下他们客服怎么修。
RK不是电子厂，不负责维修硬件的，不然你联系下RK官方旗舰店的淘宝客服看下怎么操作。

作者: ooo 时间: 2022-8-2 12:24
好吧，我找找看

作者: ooo 时间: 2022-8-12 03:14

jefferyzhang 发表于 2022-8-1 08:53
lsusb看下有没有2207的设备，还有固件是否是V3.0+的？

你好,前几天使用v3.0固件,偶然间正常,但是重新再烧录一次的时候,又出现这个问题,请问一下固件3.0是不是有问题?

作者: ooo 时间: 2022-8-12 03:23
现在是板子有问题还是固件问题?

作者: ooo 时间: 2022-8-12 03:24
这个板子已经找不到是哪个淘宝店铺卖的,像我这种情况怎么办?

欢迎光临 Toybrick (https://t.rock-chips.com/)