In the modern smart factory landscape, traditional video surveillance is undergoing a massive transformation. Security cameras are no longer passive recording devices; they have become intelligent visual sensors driving automated safety compliance, access control, and quality inspection. However, streaming multi-channel 1080P or 4K video feeds to centralized cloud servers for AI inference introduces critical bottlenecks: prohibitive bandwidth costs, cloud subscription fees, network instability, and unacceptably high latency.
To solve this, hardware engineers and system integrators are shifting to edge computing box architectures. Implementing AIoT is not about hardware; it’s about measurable ROI. We specialize in transforming complex environments into high-efficiency assets through precise technology integration.
Among current embedded AI platforms, the Rockchip RK3588 stands out as an exceptional processor. Featuring an octa-core 64-bit CPU, an ARM Mali-G610 GPU, and a tri-core NPU delivering up to 6.0 TOPS of INT8 computing power, the RK3588 offers a cost-effective alternative to expensive desktop-grade GPUs.
In this guide, we will walk step-by-step through optimizing, converting, and deploying YOLOv8—Ultralytics’ state-of-the-art computer vision model—onto an industrial RK3588 edge computer (such as the ChenAnIoT CA-EVB3588) for multi-channel RTSP video stream analytics.
Technical Architecture Overview
Before diving into code and quantization, it is essential to understand how video frames flow through the hardware pipeline. Achieving 30 FPS across 4 to 8 RTSP channels requires bypassing standard software decoding bottlenecks.
┌────────────────────────────────────────────────────────────────────────┐
│ Industrial IP Cameras │
│ (RTSP Streams: H.264/H.265 via Dual GbE) │
└────────────────────────────────────────┬───────────────────────────────┘
│
▼
┌────────────────────────────────────────────────────────────────────────┐
│ ChenAnIoT CA-EVB3588 Industrial Edge Box │
│ │
│ ┌─────────────────────────┐ ┌──────────────────────────────┐ │
│ │ Hard Video Decoder │────────►│ Rockchip NPU (6.0 TOPS) │ │
│ │ (MPP / RGA Zero-Copy) │ │ (RKNN-Toolkit2 INT8 Inference)│ │
│ └─────────────────────────┘ └──────────────┬───────────────┘ │
│ │ │
│ ▼ │
│ ┌─────────────────────────┐ ┌──────────────────────────────┐ │
│ │ Industrial I/O Control │◄────────│ Post-Processing & Tracking │ │
│ │ (RS485 / Relay Alarm) │ │ (ByteTrack / Multi-ROI) │ │
│ └─────────────────────────┘ └──────────────┬───────────────┘ │
└─────────────────────────────────────────────────────┼──────────────────┘
│
▼
┌────────────────────────────────────────────────────────────────────────┐
│ Factory MES / SCADA / Cloud via MQTT │
└────────────────────────────────────────────────────────────────────────┘
The system relies on three core hardware layers:
Hardware H.264/H.265 VPU (Video Processing Unit): Decodes incoming RTSP streams directly via Rockchip MPP (Media Process Platform) to avoid CPU overloading.
RGA (2D Graphic Accelerator): Handles frame resizing, color space conversion (NV12 to RGB), and cropping in hardware zero-copy memory.
Tri-core NPU Acceleration: Executes the quantized YOLOv8 RKNN model in parallel across independent processing units.
Phase 1: Model Export & Environment Setup
To deploy YOLOv8 on Rockchip NPUs, the standard PyTorch weights (.pt) must be converted into the proprietary .rknn binary format using RKNN-Toolkit2.
1. Host Development Environment
Conversion should be performed on an x86_64 Linux host machine (Ubuntu 20.04/22.04 recommended).
# Create and activate a clean Conda environment
conda create -n rknn python=3.8
conda activate rknn
# Install RKNN-Toolkit2 dependencies
pip install torch==1.13.1 torchvision==0.14.1
pip install numpy ONNX ONNX-simplifier
# Install RKNN-Toolkit2 (wheels available from Rockchip GitHub repository)
pip install rknn_toolkit2-2.0.0b0-cp38-cp38-linux_x86_64.whl
2. Exporting YOLOv8 to ONNX Format
Export your custom-trained or pretrained YOLOv8 model (yolov8s.pt) to ONNX format. To ensure compatibility with RKNN graph optimizations, disable dynamic shapes and keep the output heads clean.
from ultralytics import YOLO
# Load your trained model
model = YOLO(‘yolov8s.pt’)
# Export to ONNX with fixed 640×640 input resolution
model.export(
format=’onnx’,
imgsz=[640, 640],
simplify=True,
opset=12
)
Pro Tip: Always run onnxsim yolov8s.onnx yolov8s_sim.onnx after exporting. Graph simplification removes redundant Reshape and Transpose operators, ensuring a smoother build phase during NPU compilation.
Phase 2: INT8 Quantization & RKNN Compilation
Deploying FP16 ONNX models on embedded NPUs can bound bandwidth and lower throughput. To unleash the full 6.0 TOPS capability of the RK3588, we convert the model to INT8 quantization.
1. Preparing the Calibration Dataset
Create a directory named dataset/ containing 100 to 200 representative images from your target industrial environment (e.g., factory floors, workers with PPE, forklift operations).
Create a dataset.txt file listing absolute or relative paths:
./dataset/frame_0001.jpg
./dataset/frame_0002.jpg
./dataset/frame_0003.jpg
2. RKNN Build Script (convert_yolov8.py)
Write a Python script on your host build machine to initialize the RKNN context, configure model targets for rk3588, apply INT8 quantization, and export the .rknn file.
import sys
from rknn.api import RKNN
ONNX_MODEL = ‘yolov8s_sim.onnx’
RKNN_MODEL = ‘yolov8s_rk3588_int8.rknn’
DATASET = ‘./dataset.txt’
if __name__ == ‘__main__’:
rknn = RKNN(verbose=True)
# 1. Target Configuration for RK3588
rknn.config(
mean_values=[[0, 0, 0]],
std_values=[[255, 255, 255]],
target_platform=’rk3588′,
quantized_algorithm=’normal’,
quantized_method=’channel’
)
# 2. Load ONNX Model
print(‘–> Loading ONNX model…’)
ret = rknn.load_onnx(model=ONNX_MODEL)
if ret != 0:
print(‘Load ONNX failed!’)
sys.exit(ret)
# 3. Build RKNN Model with INT8 Quantization
print(‘–> Building RKNN model…’)
ret = rknn.build(do_quantization=True, dataset=DATASET)
if ret != 0:
print(‘Build RKNN failed!’)
sys.exit(ret)
# 4. Export RKNN Binary
print(‘–> Exporting RKNN model…’)
ret = rknn.export_rknn(RKNN_MODEL)
if ret != 0:
print(‘Export RKNN failed!’)
sys.exit(ret)
print(‘Successfully compiled for RK3588 NPU!’)
rknn.release()
Execute the build script:
python convert_yolov8.py
Phase 3: C++ Runtime Deployment on RK3588
While Python wrappers exist for rapid prototyping, real-time multi-channel video analytics demands C++ execution using RKNN C API and Rockchip MPP/RGA SDKs.
Below is a production-grade multi-threaded pipeline pattern for the target RK3588 device running Ubuntu or Debian Linux.
1. Multi-Core NPU Task Scheduling
The RK3588 NPU contains 3 independent cores (RKNN_NPU_CORE_0, RKNN_NPU_CORE_1, RKNN_NPU_CORE_2). You can bind separate inference contexts to specific cores or run them in auto-load-balancing mode (RKNN_NPU_CORE_AUTO).
#include <stdio.h>
#include <stdlib.h>
#include <string.h>
#include “rknn_api.h”
// Initialize RKNN context and bind core
rknn_context ctx;
int ret = rknn_init(&ctx, model_data, model_size, 0, NULL);
// Bind context to specific NPU cores for multi-threading
rknn_core_mask core_mask = RKNN_NPU_CORE_0_1_2; // Use all 3 cores
ret = rknn_set_core_mask(ctx, core_mask);
2. Zero-Copy Memory & RGA Pre-Processing
To process high-resolution RTSP camera feeds without bottlenecking memory bandwidth, avoid copying frame data back and forth between CPU and NPU memory. Use Rockchip librga to decode, crop, and convert NV12 frames directly into an allocated DMA buffer.
#include <rga/RgaApi.h>
rga_buffer_t src_img, dst_img;
memset(&src_img, 0, sizeof(rga_buffer_t));
memset(&dst_img, 0, sizeof(rga_buffer_t));
// Wrap incoming hardware-decoded RTSP frame (NV12 format)
src_img = wrapbuffer_virtualaddr(nv12_frame_ptr, stream_width, stream_height, RK_FORMAT_YCbCr_420_SP);
// Wrap NPU input buffer (640×640 RGB888 format)
dst_img = wrapbuffer_virtualaddr(npu_input_ptr, 640, 640, RK_FORMAT_RGB_888);
// Execute hardware color conversion and resizing
imresize(src_img, dst_img);
3. Asynchronous Inference Execution
Feed the preprocessed image to the NPU using rknn_inputs_set and retrieve raw tensor outputs using rknn_outputs_get:
rknn_input inputs[1];
memset(inputs, 0, sizeof(inputs));
inputs[0].index = 0;
inputs[0].type = RKNN_TENSOR_UINT8;
inputs[0].size = 640 * 640 * 3;
inputs[0].fmt = RKNN_TENSOR_NHWC;
inputs[0].buf = npu_input_ptr;
// Pass frame to NPU
rknn_inputs_set(ctx, 1, inputs);
// Run inference
rknn_run(ctx, NULL);
// Retrieve outputs
rknn_output outputs[3];
memset(outputs, 0, sizeof(outputs));
for (int i = 0; i < 3; i++) {
outputs[i].want_float = 0; // Keep in INT8 for maximum speed
}
rknn_outputs_get(ctx, 3, outputs, NULL);
Multi-Channel Video Analytics Pipeline
Processing 8 parallel IP camera feeds requires an asynchronous multi-threaded pipeline design. Decoupling decoding, inference, and post-processing into separate thread pools prevents pipeline stalls.
Camera 1-8 (RTSP) ──► [MPP Decode Thread Pool] ──► [RGA Resize/Crop]
│
▼
[JSON/MQTT Alert] ◄── [Industrial Relay / I/O] ◄── [NPU Inference Pool]
Recommended Multi-Threading Architecture
RTSP Ingestion & Decoding Pool (MPP): 8 dedicated threads decoding incoming RTSP streams via hardware.
Pre-Processing Queue: Frames are pushed into a thread-safe ring buffer. Older frames are dropped automatically if downstream queues fill up (prevents latency build-up).
NPU Worker Pool: 3 worker threads operating on
RKNN_NPU_CORE_0,CORE_1, andCORE_2.Post-Processing & Output Dispatch: Executes Non-Maximum Suppression (NMS), tracks objects (ByteTrack/DeepSORT), evaluates Virtual Fences / Line-Crossing rules, and triggers local I/O or MQTT messages.
Industrial Application Scenarios
Deploying YOLOv8 on an industrial edge computer like the CA-EVB3588 unlocks real-time operational benefits across manufacturing and logistics environments.
┌────────────────────────────────────────────────────────────────────────┐ │ Industrial Edge Analytics Use-Cases │ ├──────────────────────────┬─────────────────────────────────────────────┤ │ Hazardous Zone Intrusion │ Detect workers entering restricted areas; │ │ Detection │ trigger RS485 relay alarm in <50ms. │ ├──────────────────────────┼─────────────────────────────────────────────┤ │ Cleanroom Capacity │ Count inward/outward personnel via dual-lens│ │ Management │ cameras; enforce strict occupancy limits. │ ├──────────────────────────┼─────────────────────────────────────────────┤ │ Industrial PPE │ Verify helmets, vests, and safety goggles │ │ Compliance Check │ before granting access at entry turnstiles. │ └──────────────────────────┴─────────────────────────────────────────────┘
1. Perimeter Protection & Hazardous Zone Safety
Challenge: High-voltage switch rooms, chemical processing areas, and heavy robotic work cells require instant alarm triggers when unauthorized personnel cross safety lines.
Solution: The RK3588 runs YOLOv8s at 30 FPS across 4 channels simultaneously. When a person enters a predefined Region of Interest (ROI), the gateway triggers an on-board isolated RS485/Relay output in under 50 milliseconds, instantly sounding a local strobe light or halting machinery.
2. Cleanroom Occupancy & Capacity Control
Challenge: Semiconductor and pharmaceutical cleanrooms have strict maximum personnel limits to prevent airborne contamination.
Solution: Installing dual-lens cameras at cleanroom access points allows the edge box to track bi-directional human traffic. Real-time counts are updated locally and exposed via MQTT to factory SCADA networks.
Performance Benchmarks: RK3588 Edge Box
Performance testing conducted on the ChenAnIoT CA-EVB3588 (Passive Fanless Heatsink, Industrial Grade -20°C to 70°C enclosure) yields the following metrics:
┌───────────────┬────────────┬───────────┬────────────────┬────────────────────────┬─────────────────┐ │ Model Variant │ Resolution │ Precision │ NPU Cores Used │ Total Throughput (FPS) │ Latency / Frame │ ├───────────────┼────────────┼───────────┼────────────────┼────────────────────────┼─────────────────┤ │ YOLOv8n │ 640x640 │ INT8 │ 3 Cores (Auto) │ 142 FPS │ ~7.0 ms │ │ YOLOv8s │ 640x640 │ INT8 │ 3 Cores (Auto) │ 86 FPS │ ~11.6 ms │ │ YOLOv8m │ 640x640 │ INT8 │ 3 Cores (Auto) │ 41 FPS │ ~24.3 ms │ │ YOLOv8s │ 640x640 │ FP16 │ 3 Cores (Auto) │ 38 FPS │ ~26.3 ms │ └───────────────┴────────────┴───────────┴────────────────┴────────────────────────┴─────────────────┘
Key Takeaway: Running YOLOv8s in INT8 mode provides the optimal balance of speed and precision, easily driving 4 channels of 1080P video streams at full 20+ FPS real-time performance on a single board.
Conclusion & Next Steps
Deploying YOLOv8 on the Rockchip RK3588 platform bridges the gap between sophisticated deep learning models and rugged industrial hardware requirements. By leveraging hardware-accelerated video decoding (MPP), 2D frame pre-processing (RGA), and INT8 NPU compilation (RKNN-Toolkit2), industrial engineers can build low-latency, privacy-compliant, and cost-effective vision systems without relying on expensive cloud infrastructures.
Are you ready to evaluate edge AI hardware for your industrial automation or computer vision projects?
Contact us to obtain the product specification sheet for the CA-EVB3588 industrial edge box.