Toybrick

楼主: fu-ai

TB-RK3399ProX Android8.1用rk_ssd_demo这个demo跑自己的模型时总是...

fu-ai

新手上路

积分
45
11#
 楼主| 发表于 2021-5-25 17:22:20 | 只看该作者
jefferyzhang 发表于 2021-5-25 16:42
http://t.rock-chips.com/forum.php?mod=viewthread&tid=3087&extra=page%3D1

直接在android上更新一把 ...

大佬,请问这是要在双系统下操作么
回复

使用道具 举报

fu-ai

新手上路

积分
45
12#
 楼主| 发表于 2021-5-25 19:18:42 | 只看该作者
jefferyzhang 发表于 2021-5-25 16:42
http://t.rock-chips.com/forum.php?mod=viewthread&tid=3087&extra=page%3D1

直接在android上更新一把 ...

大佬,更新了还是不行,log如下 ↓ ↓ ↓

2021-01-06 14:47:00.030 2272-2272/? I/zygote64: Late-enabling -Xcheck:jni
2021-01-06 14:47:00.375 2272-2277/com.rockchip.gpadc.ssddemo I/zygote64: Do partial code cache collection, code=30KB, data=18KB
2021-01-06 14:47:00.375 2272-2277/com.rockchip.gpadc.ssddemo I/zygote64: After code cache collection, code=21KB, data=16KB
2021-01-06 14:47:00.375 2272-2277/com.rockchip.gpadc.ssddemo I/zygote64: Increasing code cache capacity to 128KB
2021-01-06 14:47:00.394 2272-2272/com.rockchip.gpadc.ssddemo D/ssd: Loaded box priors!
2021-01-06 14:47:00.395 2272-2272/com.rockchip.gpadc.ssddemo D/ssd: Loaded label!
2021-01-06 14:47:00.512 2272-2272/com.rockchip.gpadc.ssddemo V/ssd: Camera Supported Preview Size = 1280x720
2021-01-06 14:47:00.512 2272-2272/com.rockchip.gpadc.ssddemo V/ssd: Camera Supported Preview Size = 640x480
2021-01-06 14:47:00.538 2272-2312/com.rockchip.gpadc.ssddemo D/OpenGLRenderer: HWUI GL Pipeline
2021-01-06 14:47:00.574 2272-2312/com.rockchip.gpadc.ssddemo I/mali_so: [File] : hardware/arm/maliT760/driver/product/base/src/mali_base_kbase.c; [Line] : 947; [Func] : base_context_deal_with_version_affairs_rk_ext;
    arm_release_ver of this mali_so is 'r18p0-01rel0', rk_so_ver is '11@0'.
2021-01-06 14:47:00.575 2272-2312/com.rockchip.gpadc.ssddemo I/mali_so: [File] : hardware/arm/maliT760/driver/product/base/src/mali_base_kbase.c; [Line] : 914; [Func] : determine_policy_against_patch_for_sf_memory_leak;
    cmdline : com.rockchip.gpadc.ssddemo, we should GO THROUGH patch_for_sf_memory_leak.
2021-01-06 14:47:00.575 2272-2312/com.rockchip.gpadc.ssddemo D/mali_so: [File] : hardware/arm/maliT760/driver/product/base/src/mali_base_kbase.c; [Line] : 955; [Func] : base_context_deal_with_version_affairs_rk_ext;
    current process is NOT sf, to bail out.
2021-01-06 14:47:00.585 2272-2312/com.rockchip.gpadc.ssddemo I/com.rockchip.gpadc.ssddemo: android::hardware::configstore::V1_0::ISurfaceFlingerConfigs::hasWideColorDisplay retrieved: 0
2021-01-06 14:47:00.586 2272-2312/com.rockchip.gpadc.ssddemo I/OpenGLRenderer: Initialized EGL, version 1.4
2021-01-06 14:47:00.586 2272-2312/com.rockchip.gpadc.ssddemo D/OpenGLRenderer: Swap behavior 2
2021-01-06 14:47:00.591 2272-2312/com.rockchip.gpadc.ssddemo D/mali_winsys: EGLint new_window_surface(egl_winsys_display *, void *, EGLSurface, EGLConfig, egl_winsys_surface **, egl_color_buffer_format *, EGLBoolean) returns 0x3000
2021-01-06 14:47:00.604 2272-2298/com.rockchip.gpadc.ssddemo D/mali_winsys: EGLint new_window_surface(egl_winsys_display *, void *, EGLSurface, EGLConfig, egl_winsys_surface **, egl_color_buffer_format *, EGLBoolean) returns 0x3000
2021-01-06 14:47:00.648 2272-2325/com.rockchip.gpadc.ssddemo I/rkssd4j: try rknn_init!
2021-01-06 14:47:00.663 2272-2272/com.rockchip.gpadc.ssddemo I/Thread-3: type=1400 audit(0.0:122): avc: denied { connectto } for path="/dev/socket/transfer_proxy" scontext=u:r:untrusted_app:s0:c512,c768 tcontext=u:r:npu_transfer_proxy:s0 tclass=unix_stream_socket permissive=1
2021-01-06 14:47:03.690 2272-2277/com.rockchip.gpadc.ssddemo I/zygote64: Do partial code cache collection, code=47KB, data=55KB
2021-01-06 14:47:03.691 2272-2277/com.rockchip.gpadc.ssddemo I/zygote64: After code cache collection, code=42KB, data=53KB
2021-01-06 14:47:03.691 2272-2277/com.rockchip.gpadc.ssddemo I/zygote64: Increasing code cache capacity to 256KB
2021-01-06 14:47:03.692 2272-2277/com.rockchip.gpadc.ssddemo I/zygote64: Compiler allocated 6MB to compile void android.view.ViewRootImpl.performTraversals()
2021-01-06 14:47:06.759 2272-2277/com.rockchip.gpadc.ssddemo I/zygote64: Do full code cache collection, code=118KB, data=92KB
2021-01-06 14:47:06.760 2272-2277/com.rockchip.gpadc.ssddemo I/zygote64: After code cache collection, code=91KB, data=65KB
2021-01-06 14:47:12.044 2272-2325/com.rockchip.gpadc.ssddemo D/RKNNAPI: ==============================================
2021-01-06 14:47:12.044 2272-2325/com.rockchip.gpadc.ssddemo D/RKNNAPI: RKNN VERSION:
2021-01-06 14:47:12.044 2272-2325/com.rockchip.gpadc.ssddemo D/RKNNAPI:   API: 1.6.0 (159d2d3 build: 2021-01-22 11:15:48)
2021-01-06 14:47:12.044 2272-2325/com.rockchip.gpadc.ssddemo D/RKNNAPI:   DRV: 1.6.0 (159d2d3 build: 2021-01-12 15:23:09)
2021-01-06 14:47:12.044 2272-2325/com.rockchip.gpadc.ssddemo D/RKNNAPI: ==============================================
2021-01-06 14:47:12.051 2272-2325/com.rockchip.gpadc.ssddemo I/rkssd4j: rknn_init success!
2021-01-06 14:47:12.053 2272-2325/com.rockchip.gpadc.ssddemo D/RKNNAPI: __can_use_fixed_point: use_fixed_point = 1.
2021-01-06 14:47:12.055 2272-2325/com.rockchip.gpadc.ssddemo E/RKNNAPI: rknn_outputs_get,  outputs[0].size = 15336! expect 30672!
2021-01-06 14:47:12.055 2272-2325/com.rockchip.gpadc.ssddemo E/rkssd4j: rknn_outputs_get fail! ret=-5
2021-01-06 14:47:12.058 2272-2325/com.rockchip.gpadc.ssddemo E/RKNNAPI: rknn_outputs_get,  outputs[0].size = 15336! expect 30672!
2021-01-06 14:47:12.058 2272-2325/com.rockchip.gpadc.ssddemo E/rkssd4j: rknn_outputs_get fail! ret=-5
2021-01-06 14:47:12.061 2272-2325/com.rockchip.gpadc.ssddemo E/RKNNAPI: rknn_outputs_get,  outputs[0].size = 15336! expect 30672!
2021-01-06 14:47:12.061 2272-2325/com.rockchip.gpadc.ssddemo E/rkssd4j: rknn_outputs_get fail! ret=-5
2021-01-06 14:47:12.098 2272-2272/com.rockchip.gpadc.ssddemo D/ssd: something may wrong! validCount=1917
2021-01-06 14:47:12.105 2272-2325/com.rockchip.gpadc.ssddemo W/RKNNAPI: rknn_run,  is about to block until rknn_outputs_get is called!
2021-01-06 14:47:12.111 2272-2277/com.rockchip.gpadc.ssddemo I/zygote64: Do partial code cache collection, code=123KB, data=88KB
2021-01-06 14:47:12.111 2272-2277/com.rockchip.gpadc.ssddemo I/zygote64: After code cache collection, code=123KB, data=88KB
2021-01-06 14:47:12.111 2272-2277/com.rockchip.gpadc.ssddemo I/zygote64: Increasing code cache capacity to 512KB
2021-01-06 14:47:13.214 2272-2272/com.rockchip.gpadc.ssddemo I/ssd: stopped camera
回复

使用道具 举报

jefferyzhang

版主

积分
13572
13#
发表于 2021-5-26 08:47:09 | 只看该作者
你到底在仿真里跑通过没有,得到的结果是否正确的。你这里输出纬度都不一样,仿真怎么跑过的?
E/RKNNAPI: rknn_outputs_get,  outputs[0].size = 15336! expect 30672!
回复

使用道具 举报

fu-ai

新手上路

积分
45
14#
 楼主| 发表于 2021-5-26 10:15:05 | 只看该作者
jefferyzhang 发表于 2021-5-26 08:47
你到底在仿真里跑通过没有,得到的结果是否正确的。你这里输出纬度都不一样,仿真怎么跑过的?
E/RKNNAPI:  ...

我在板子上跑通了test.py,并出图了,不知道这个是不是你说的仿真
回复

使用道具 举报

fu-ai

新手上路

积分
45
15#
 楼主| 发表于 2021-5-26 10:19:28 | 只看该作者
之前我跑test.py的时候,也出现过类似这个问题,不过那时是1917! expect 3834!的样子,后来把test.py里的类别数由1改成2(我的模型只有一个类别)的时候,就正常跑通并出图
回复

使用道具 举报

jefferyzhang

版主

积分
13572
16#
发表于 2021-5-26 10:21:16 | 只看该作者
fu-ai 发表于 2021-5-26 10:19
之前我跑test.py的时候,也出现过类似这个问题,不过那时是1917! expect 3834!的样子,后来把test.py里的类 ...

拜托你先了解下你的ssd输出数量和维度在搞吧,出图就叫跑通了我也是无语了,python不会出现内存溢出,c++这么搞肯定会出问题了。
回复

使用道具 举报

fu-ai

新手上路

积分
45
17#
 楼主| 发表于 2021-5-26 10:25:49 | 只看该作者
jefferyzhang 发表于 2021-5-26 10:21
拜托你先了解下你的ssd输出数量和维度在搞吧,出图就叫跑通了我也是无语了,python不会出现内存溢出,c++ ...

那我怎么看输出数量和维度?
回复

使用道具 举报

fu-ai

新手上路

积分
45
18#
 楼主| 发表于 2021-5-26 10:28:40 | 只看该作者
这是我用的test.py,需要更改么

import numpy as np

import re
import math
import random
import cv2

from rknn.api import RKNN

INPUT_SIZE = 300

NUM_RESULTS = 1917
NUM_CLASSES = 2# num+1

Y_SCALE = 10.0
X_SCALE = 10.0
H_SCALE = 5.0
W_SCALE = 5.0


def expit(x):
    return 1. / (1. + math.exp(-x))


def unexpit(y):
    return -1.0 * math.log((1.0 / y) - 1.0);


def CalculateOverlap(xmin0, ymin0, xmax0, ymax0, xmin1, ymin1, xmax1, ymax1):
    w = max(0.0, min(xmax0, xmax1) - max(xmin0, xmin1))
    h = max(0.0, min(ymax0, ymax1) - max(ymin0, ymin1))
    i = w * h
    u = (xmax0 - xmin0) * (ymax0 - ymin0) + (xmax1 - xmin1) * (ymax1 - ymin1) - i

    if u <= 0.0:
        return 0.0

    return i / u


def load_box_priors():
    box_priors_ = []
    fp = open('./box_priors.txt', 'r')
    ls = fp.readlines()
    for s in ls:
        aList = re.findall('([-+]?\d+(\.\d*)?|\.\d+)([eE][-+]?\d+)?', s)
        for ss in aList:
            aNum = float((ss[0]+ss[2]))
            box_priors_.append(aNum)
    fp.close()

    box_priors = np.array(box_priors_)
    box_priors = box_priors.reshape(4, NUM_RESULTS)

    return box_priors


if __name__ == '__main__':

    # Create RKNN object
    rknn = RKNN()

    # Config for Model Input PreProcess
    print('--> Config model')
    rknn.config(mean_values=[[127.5, 127.5, 127.5]], std_values=[[127.5, 127.5, 127.5]], reorder_channel='0 1 2')
    print('done')

    # Load TensorFlow Model
    print('--> Loading model')
    ret = rknn.load_tensorflow(tf_pb='./tflite_graph.pb',
                               inputs=['normalized_input_image_tensor'],
                               outputs=['concat', 'concat_1'],
                               input_size_list=[[INPUT_SIZE, INPUT_SIZE, 3]])
    if ret != 0:
        print('Load model failed!')
        exit(ret)
    print('done')

    # Build Model
    print('--> Building model')
    ret = rknn.build(do_quantization=True, dataset='./dataset.txt')
    if ret != 0:
        print('Build model failed!')
        exit(ret)
    print('done')

    # Export RKNN Model
    print('--> Export RKNN model')
    rknn.export_rknn('./ssd.rknn')
    if ret != 0:
        print('Export RKNN model failed!')
        exit(ret)
    print('done')

    # Direct Load RKNN Model
    # rknn.load_rknn('./ssd.rknn')

    # Set inputs
    orig_img = cv2.imread('./horizontal.bmp')
    img = cv2.cvtColor(orig_img, cv2.COLOR_BGR2RGB)
    img = cv2.resize(img, (INPUT_SIZE, INPUT_SIZE), interpolation=cv2.INTER_CUBIC)

    # init runtime environment
    print('--> Init runtime environment')
    ret = rknn.init_runtime()
    if ret != 0:
        print('Init runtime environment failed')
        exit(ret)
    print('done')

    # Inference
    print('--> Running model')
    outputs = rknn.inference(inputs=[img])
    print('done')

    predictions = outputs[0].reshape((1, NUM_RESULTS, 4))
    outputClasses = outputs[1].reshape((1, NUM_RESULTS, NUM_CLASSES))
    candidateBox = np.zeros([2, NUM_RESULTS], dtype=int)
    vaildCnt = 0

    box_priors = load_box_priors()

    # Post Process
    # got valid candidate box
    for i in range(0, NUM_RESULTS):
        topClassScore = -1000
        topClassScoreIndex = -1

        # Skip the first catch-all class.
        for j in range(1, NUM_CLASSES):
            score = expit(outputClasses[0][i][j]);

            if score > topClassScore:
                topClassScoreIndex = j
                topClassScore = score

        if topClassScore > 0.4:
            candidateBox[0][vaildCnt] = i
            candidateBox[1][vaildCnt] = topClassScoreIndex
            vaildCnt += 1

    # calc position
    for i in range(0, vaildCnt):
        if candidateBox[0][i] == -1:
            continue

        n = candidateBox[0][i]
        ycenter = predictions[0][n][0] / Y_SCALE * box_priors[2][n] + box_priors[0][n]
        xcenter = predictions[0][n][1] / X_SCALE * box_priors[3][n] + box_priors[1][n]
        h = math.exp(predictions[0][n][2] / H_SCALE) * box_priors[2][n]
        w = math.exp(predictions[0][n][3] / W_SCALE) * box_priors[3][n]

        ymin = ycenter - h / 2.
        xmin = xcenter - w / 2.
        ymax = ycenter + h / 2.
        xmax = xcenter + w / 2.

        predictions[0][n][0] = ymin
        predictions[0][n][1] = xmin
        predictions[0][n][2] = ymax
        predictions[0][n][3] = xmax

    # NMS
    for i in range(0, vaildCnt):
        if candidateBox[0][i] == -1:
            continue

        n = candidateBox[0][i]
        xmin0 = predictions[0][n][1]
        ymin0 = predictions[0][n][0]
        xmax0 = predictions[0][n][3]
        ymax0 = predictions[0][n][2]

        for j in range(i+1, vaildCnt):
            m = candidateBox[0][j]

            if m == -1:
                continue

            xmin1 = predictions[0][m][1]
            ymin1 = predictions[0][m][0]
            xmax1 = predictions[0][m][3]
            ymax1 = predictions[0][m][2]

            iou = CalculateOverlap(xmin0, ymin0, xmax0, ymax0, xmin1, ymin1, xmax1, ymax1)

            if iou >= 0.45:
                candidateBox[0][j] = -1

    # Draw result
    for i in range(0, vaildCnt):
        if candidateBox[0][i] == -1:
            continue

        n = candidateBox[0][i]

        xmin = max(0.0, min(1.0, predictions[0][n][1])) * INPUT_SIZE
        ymin = max(0.0, min(1.0, predictions[0][n][0])) * INPUT_SIZE
        xmax = max(0.0, min(1.0, predictions[0][n][3])) * INPUT_SIZE
        ymax = max(0.0, min(1.0, predictions[0][n][2])) * INPUT_SIZE

        # print("%d @ (%d, %d) (%d, %d) score=%f" % (topClassScoreIndex, xmin, ymin, xmax, ymax, topClassScore))
        cv2.rectangle(orig_img, (int(xmin), int(ymin)), (int(xmax), int(ymax)),
                      (random.random()*255, random.random()*255, random.random()*255), 3)

    cv2.imwrite("out.jpg", orig_img)

    # Evaluate Perf on Simulator
    print('--> Evaluate model performance')
    rknn.eval_perf(inputs=[img], is_print=True)
    print('done')

    # Release RKNN Context
    rknn.release()
回复

使用道具 举报

fu-ai

新手上路

积分
45
19#
 楼主| 发表于 2021-5-26 11:11:09 | 只看该作者
版主,是不是我这里要修改?
predictions = outputs[0].reshape((1, NUM_RESULTS, 4))
    outputClasses = outputs[1].reshape((1, NUM_RESULTS, NUM_CLASSES))
    candidateBox = np.zeros([2, NUM_RESULTS], dtype=int)
回复

使用道具 举报

fu-ai

新手上路

积分
45
20#
 楼主| 发表于 2021-5-26 11:39:17 | 只看该作者
版主,你在吗?
回复

使用道具 举报

您需要登录后才可以回帖 登录 | 立即注册

本版积分规则

产品中心 购买渠道 开源社区 Wiki教程 资料下载 关于Toybrick


快速回复 返回顶部 返回列表