Lodur

Name

Lodur - Frame grabber FPGA Compute Platform

Defined in Task

Task 4.4: Realtime sensor/data processing

Short description

Lodur is a frame grabber. It is responsible for connecting a camera to an FPGA platform (Honir).

This tool is a piece of software design that has to be loaded on a board containing an FPGA (Xilinx VU9P for example) and a QSFP+ network interface.

Lodur is an IP core responsible of grabbing images in gigEvision standard usually supported by industrial cameras. It is also responsible to send the received stream into another stream understandable by the machine learning inference engine Honir (done in task 4.4 too).

Lodur performs data health checks while receiving the images such as ensuring that the image is complete, or that the format is respected. It can also perform a pre-processing step on the mage if required, such as pixel format conversion from YUV to RGB.

The advantages of this tool are:

  • Support of gigEvision frame grabbing on FPGA
  • Support pixel format conversion YUV to RGB
  • Possibility to support 10gigEvision protocol (10x gigEvision) in the next versions
  • Possibility to support multi camera stream inputs in the next versions
  • Smooth integration with Honir

    Example of usage

    Above: An industrial camera takes picture and send it over the gigEvision link. The gigEvision stream rely on UDP packet transfer. It means that the image is cut in a number of packets (sub part of the image) that are sent in sequence to Lodur.

    Lodur receive the package, perform health checks on the image, perform pre-processing if activated and cut the image in pixels and then send it to the next module: Honir (inference engine).

    Lodur is able to receive images from a camera at maximum camera streaming speed: 1gpbs. We tested it with a computer simulating the camera with a higher throughput and it is able to support up to 1.8 gbps. This means it is possible to support more powerful cameras that stream at more than 1 gpbs.

    Interfaces

    At a system level:

    IN: gigEvision video stream

    OUT: pixel stream

    Subordinates and platform dependencies

    This module can work in standalone. It is, however, necessary for Honir (inference engine) to work properly.

    License

    In development, remains the property of Scortex. Will be used by Scortex exclusively

    TRL

    TRL5

    References

    https://www.visiononline.org/vision-standards-details.cfm?type=5

    To be considered in particular for the following COGNITWIN pilots

    The Honir platform will be considered as a way to run the tracking system for the Saarstahl use case. In which case, Lodur will be used to maintain the best FPS (frame per second) performances possible. But it could be extended to any other pilots running deep learning on images.