Keras2RTL - Machine learning to FPGA conversion

Defined in Task

Task 5.3: Deep learning performance

Short description

This tool allows to fill the gap between machine / deep learning development environments and FPGA development environments.

Machine learning development environments are typically based on the python programming language and libraries such as keras and tensorflow (or pytorch) relying on Nvidia low level computing library named CUDA.

Meanwhile, FPGA developments are based on VHDL programming language and integrated development environments (IDE) provided by the company that provides the FPGA component. For a Xilinx Virtex Ultrascale+ VU9P, the IDE is named Vivado.

Keras2RTL takes as input a tf.keras model that is a file with .h5 extension that contains the topology of the machine learning neural network and the weights. At Scortex, the keras model is generated using the bonzai library (see other 5.3 components).

Keras2RTL convert the topology from the .h5 file and generate the VHDL configuration files. These files will be used by Vivado to generate Honir.

Example of usage

The overall usage flow is:

  • Train a tf.keras / keras model (typically using bonzai).
  • Quantize the model (or train it in a quantized way immediately).
  • Convert it using keras2RTL.

    Use this model to run inference on images using the Honir component.

    As a demonstrator, Scortex trained a keras model on data from one of its demonstrators. The model allows defect detection on common goods parts, such as lego bricks, electrical switch and door handles.

    This deep learning model was quantized using the bonzai tool (see other component) and then processed by Keras2RTL process manually. The VHDL files produced was used for the configuration of inference engine (Honir: in task 4.4).

    In a more general manner, the tool should be used to allow automatic and fast conversion from a keras machine learning model to a VHDL config file that can be used for Honir creation.


    At a user level (a command line tool)

    IN : keras .h5 file (topology + weights)

    OUT : VHDL config file (for Honir tool build in T4.4)

    Subordinates and platform dependencies

    This module can work in standalone. It is, however, necessary for Honir (inference engine) to work properly. Today the tool will support only keras models.


    In development, remains the property of Scortex. Will be used by Scortex exclusively




    To be considered in particular for the following COGNITWIN pilots

    The Honir platform will be considered as a way to run the tracking system for the Saarstahl use case. In which case, keras2TL will be used to generate Honir configuration. But it could be extended to any other pilots running deep learning on images