yolonas-cpp

An CPP library for object detection with full processing using OpenCV DNN (for ONNX) on YOLO-NAS model.
Showcase, demo and quckstart YouTube video

Introduction

As normal world of AI today becomed reality, we use it as much as we can. Everyone knows that today to run an AI model, or anything similar to that, you need intermediate processing power that many people cannot afford it. As we, humans, we always choose the easiest path to do something. That is similar here. If you search online for some machine vision thing, you will probably 99% of time get something written in Python. Currently, Python is the easiest path, but in inference time not the fastest. If you understand what Python Interpreter is, you will understand how is it slow to do some complex tasks. There comes CPP. It almost runs everything today under the hood, and it's incredibly FAST. But ONLY KNOWLEDGABLE PEOPLE knows how to use it. So, here comes this libary. You can in TWO LINES of code run machine detection. It runs using OpenCV, and it can use YOLO-NAS, that model which is really good, with fastest detection (inference) time, and it CAN detect extremly small objects. Here is it's efficiency comparing to other models:

Detection

Library uses OpenCV and it's DNN to run the model. Model is under ONNX format. It's also able to run with CUDA (not tested yet).

Requirements

• GCC
• CMake
• OpenCV (installed as CMake library) with dnn submodule enabled
• CUDA compatible GPU + nvcc (optionally, if you want to use GPU inference)

Attention!

I got issues when I used precompiled version of OpenCV, libopencv-dev, installed with apt package manager. Then, I was not able to start a inference at all. To fix this issue, I compiled and installed OpenCV using this gist. Please use it to avoid any issues.

Setup

via Docker

To setup this library and interact with it via Docker, you can simply pull it and run by using commands:

docker pull ukicomputers/yolonas-cpp
docker run ukicomputers/yolonas-cpp

By default, container will use default model and picture for detection. Currently, Docker container DOES NOT support GPU accelerated, and video inference. These features are comming soon. You can interact with it by setting enviroment variables when running it:

docker run -v /path/to/write/out/imgs:/output -e model="path/to/model" -e metadata="path/to/metadata" -e source="path/to/source" ukicomputers/yolonas-cpp

It is important that if you want to get images of detections, you need to make a bridge from folder on your device, exactly to folder on container located on /output. You may also want to create bridges for models, etc located on your local device. You can also find image on Docker Hub.

Local setup

To install this library on your computer, firstly clone this repository:

git clone https://github.com/ukicomputers/yolonas-cpp.git

Then change directory to it, and execute install.bash with superuser permission, installing as CMake libary:

cd yolonas-cpp
sudo bash install.bash

After that library will be installed.

Aditionally, if you want to download YOLO-NAS S COCO model to use directly from example, you can just execute this bash script:

bash download_models.bash

Download size is about ~47MB (maybe more, as the models are constantly updated).

Quick usage

CPP code (not full code, just minimal example):

// include
#include <ukicomputers/YoloNAS.hpp>
//              modelpath           metadata   Labels            CUDA
YoloNAS net("./yolo_nas_s.onnx", "./metadata", {"lbl1", "lbl2"}, false);
//          img  draw  score threshold
net.predict(img, true, 0.5);

Include library in CMake:

find_package(YoloNAS REQUIRED)
target_link_libraries(${PROJECT_NAME} ukicomputers::YoloNAS)

Also required to include OpenCV:

find_package(OpenCV REQUIRED)
include_directories(${OpenCV_INCLUDE_DIRS})
target_link_libraries(${PROJECT_NAME} ${OpenCV_LIBS})

Full library usage

YoloNAS class

Acceptable arguments:

YoloNAS::YoloNAS(string netPath, string config, vector<string> lbls, bool cuda = false);

Initializes YoloNAS class.

• modelpath (std::string),
• Metadata file (std::string),
• labels in a vector (std::vector<std::string>)
• CUDA support (bool, default false),

YoloNAS net(modelPath, metadata, CUDA, labels);

Function predict

Acceptable arguments:

vector<YoloNAS::detectionInfo> YoloNAS::predict(cv::Mat &img, bool applyOverlayOnImage = true, float scoreThresh = -1.00)

Predicts (detects) objects from image.

• image (cv::Mat)
• visualy display detection (bool, writing on given image, default true)
• score thereshold (float, if thereshold is not given, library will use thereshold from metadata, otherwise, it will use from given argument, values are entered from 0.1-1.0 not 1-100, may vary for if different model is used)

net.predict(image, overlayOnImage, scoreThreshold);

Returns detectionInfo struct(s) in vector for each detection.
Accesible elements from detectionInfo struct:

struct detectionInfo
{
    int x, y, w, h;
    float score;
    string label;
};

Score is returned as float value from 0.1-1.0. May vary if different model is used.

Demo

Demo is located in folder demo from downloaded repository. To use it out-of-box, you can download example models by executing download_models.bash. To compile and run it, execute build.bash from demo folder.

Custom model & metadata

To use your own model, and run it also inside library, use metadata.py script, link here. To use it, in metadata.py, first few variables needs to be changed according to your model (model path, model type, number of classes). IMPORTANT: metadata.py DOES NOT ACCEPT .onnx FILE FORMAT! It only accepts the standard YOLO .pt format.

Script will convert your model to ONNX, and return required metadata file, that can be later used in inference.

TODO

• fix required Docker things
• make detection visualisation look cooler

License & contributions

Please add my name to top of a document to use library. It helps for me to get reputation.

// YOLO-NAS CPP library written by Uglješa Lukešević (github.com/ukicomputers)

Also, feel free to open new issues, and contribute it by opening pull request.

References

• https://github.com/Deci-AI/super-gradients
• https://github.com/Hyuto/yolo-nas-onnx

ukicomputers.