Honir - Machine learning inference FPGA Compute Platform

Defined in Task

Task 4.4: Realtime sensor/data processing

Short description

This tool allows to perform machine learning / deep learning algorithm inference in real time on images data source.

This tool is a piece of software design that have to be loaded on a board containing an FPGA (Xilinx VU9P for example) and a QSFP+ network interface.

Honir is an IP core responsible of performing machine learning inference. Thanks to a loaded quantized machine learning model converted with keras2tool (produced in task 5.3), this module consumes images coming from Lodur tool (produced in task 4.4 too) and produces the tensor results in a stream enhanced with metadata.

The advantages of this tool are:

Real time computing thanks to FPGA platform that allows high parallelism computing

Performance / Watt better than usual machine learning platform (CPUs, GPUs)

Industrial components thanks to FPGA and electronic boards build for constraints environments

Example of usage

A machine learning model was generated with the VHDL implementation of the layers. This machine learning model correspond to a demonstration that is used for Scortex demonstration purposes that allows defect detection on parts fundable in the market, like bricks parts, electrical switch parts and some handles.

This topology allows the inference of up to 100 images per seconds with a resolution of 1920x1200. That represent a huge amount of compute that is done in an efficient way with less than 10 milliseconds of latency

Images are provided in a convenient format by the Lodur tool (“frame grabber”, produced in task 4.4). Inference is ran on images by Honir and the results produced allow to draw a red box around the defect and also provide a confidence of detection in the source image. The following image shows a zoom on the part and the localization of the defect.

To ease the use of Honir, we included it in our demonstrator to allow live stream of images and live detections.

This image represents the Scortex interface that shows a part moving under the camera with a Machine learning detection at 100 FPS.


At a system level:

IN : video stream (provided by Lodur)

OUT : prediction matrix / tensor

At a user level:

IN : video stream

OUT : results of machine learning inference in a matrix or shown in live on images

Subordinates and platform dependencies

This module takes input images that are sent by the frame grabber Lodur (see decicated component).

This module is configured based on a keras2RTL tool conversion. Indeed, the user first design a keras model, then use the keras2RLTconvertor, and then can use Honir for inference with this model.


Honir is in development and remains the property of Scortex. For now, it will be used and integrated by Scortex exclusively.




To be considered in particular for the following COGNITWIN pilots

The platform will be considered as a way to run the tracking system for the Saarstahl use case. But it could be extended to any other pilots running deep learning on images