License Plate Recognition with od-bridge GoDoc Sourcegraph Go Report Card GitHub tag

Table of Contents

About

This is a gRPC server which accepts image and can make license plate recognition (using YOLOv3 or YOLOv4 neural network).

Neural networks were trained on dataset of russian license plates. But you can train it on another dataset - read about process here https://github.com/AlexeyAB/darknet#how-to-train-to-detect-your-custom-objects

Server tries to find license plate at first. Then it does OCR (if it's possible) and provides output which can be represented later as follows:

Sample plate #1 Sample plate #2

Images has been taken near my house

Darknet architecture for finding license plates - Yolo V3

Darknet architecture for doing OCR stuff - Yolo V4

gRPC server accepts this data struct according proto3 specification

// Essential information to process
message LPRRequest{
    // Bytes representation of image (PNG)
    bytes image = 1;
    // Optional information about image. Could be usefull if client-side already knows where license plate should be located (due some object detections technique)
    BBox bbox = 2;
}

// Reference information about detection rectangle
message BBox{
    int32 x_left = 1;
    int32 y_top = 2;
    int32 height = 3;
    int32 width = 4;
}

The gRPC server response is:

// Response from server
message LPRResponse{
    // Set of found license plates with corresponding information
    repeated LPRInfo license_plates = 1;
    // Number of seconds has taken to proccess license plate detections and OCR
    float elapsed = 2;
    // Optional message from server
    string message = 3;
    // Optional warning message from server. If it is not empty you probably should investiage such behavior
    string warning = 4;
    // Optional error message from server. If it is not empty you should investiage the error
    string error = 5;
}

// Information about single license plate
message LPRInfo {
    // License plate location
    BBox bbox = 1;
    // License plate OCR bounding bboxes. Bounding bboxes are sorted by horizontal line
    // Warning: those coordinates are relative to license plate bounding box, not the parent image!
    repeated BBox ocr_bboxes = 2;
    // License plate text
    string text = 3;
}

Full gRPC documentation is here: HTML or Markdown

Requirements (server-side only)

The gRPC client does not require any native dependencies.

od-bridge (Rust inference backend)

The server uses od-bridge for ONNX model inference via CGO. See the od-bridge installation guide for build and install instructions.

# 1. Clone and build
git clone https://github.com/LdDl/od-bridge.git
cd od-bridge
cargo build --release                  # CPU only
# cargo build --release --features cuda  # with CUDA

# 2. Install shared library and header
sudo mkdir -p /usr/local/include/od-bridge
sudo cp od_bridge.h /usr/local/include/od-bridge/
sudo cp target/release/libod_bridge.so /usr/local/lib/

# 3. If built with --features cuda, also install ORT provider libraries:
# sudo cp target/release/libonnxruntime_providers_cuda.so /usr/local/lib/
# sudo cp target/release/libonnxruntime_providers_shared.so /usr/local/lib/

sudo ldconfig

Verify the installation:

ldconfig -p | grep od_bridge

See the od-bridge README for details.

Neural network weights

Models must be in ONNX format. If the pre-trained weights suit your needs, you can download them from the latest release:

mkdir -p data && cd data
curl -sLO https://github.com/LdDl/license_plate_recognition/releases/download/latest/license_plates.onnx
curl -sLO https://github.com/LdDl/license_plate_recognition/releases/download/latest/license_plates.names
curl -sLO https://github.com/LdDl/license_plate_recognition/releases/download/latest/ocr_plates.onnx
curl -sLO https://github.com/LdDl/license_plate_recognition/releases/download/latest/ocr_plates.names

If you want to convert darknet .cfg + .weights to ONNX yourself, download the source weights and use darknet2onnx:

mkdir -p data && cd data
curl -sLO https://github.com/LdDl/license_plate_recognition/releases/download/latest/license_plates_inference.cfg
curl -sLO https://github.com/LdDl/license_plate_recognition/releases/download/latest/license_plates_100000.weights
curl -sLO https://github.com/LdDl/license_plate_recognition/releases/download/latest/ocr_plates_inference.cfg
curl -sLO https://github.com/LdDl/license_plate_recognition/releases/download/latest/ocr_plates_140000.weights

darknet2onnx --cfg license_plates_inference.cfg --weights license_plates_100000.weights --output license_plates.onnx --format yolov8
darknet2onnx --cfg ocr_plates_inference.cfg --weights ocr_plates_140000.weights --output ocr_plates.onnx --format yolov8

Installation

Just pull source code:

go get github.com/LdDl/license_plate_recognition

Usage

Start server

Test Client-Server

Notice: server should be started