External Libraries
Hummingbird
Concrete ML allows the conversion of an ONNX inference to NumPy inference (note that NumPy is always the entry point to run models in FHE with Concrete ML).
Hummingbird exposes a convert
function that can be imported as follows from the hummingbird.ml
package:
This function can be used to convert a machine learning model to an ONNX as follows:
In theory, the resulting onnx_model
could be used directly within Concrete ML's get_equivalent_numpy_forward
method (as long as all operators present in the ONNX model are implemented in NumPy) and get the NumPy inference.
In practice, there are some steps needed to clean the ONNX output and make the graph compatible with Concrete ML, such as applying quantization where needed or deleting/replacing non-FHE friendly ONNX operators (such as Softmax and ArgMax).
skorch
This wrapper implements Torch training boilerplate code, lessening the work required of the user. It is possible to add hooks during the training phase, for example once an epoch is finished.
Brevitas
While Brevitas provides many types of quantization, for Concrete ML, a custom "mixed integer" quantization applies. This "mixed integer" quantization is much simpler than the "integer only" mode of Brevitas. The "mixed integer" network design is defined as:
all weights and activations of convolutional, linear and pooling layers must be quantized (e.g., using Brevitas layers,
QuantConv2D
,QuantAvgPool2D
,QuantLinear
)
For "mixed integer" quantization to work, the first layer of a Brevitas nn.Module
must be a QuantIdentity
layer. However, you can then use functions such as torch.sigmoid
on the result of such a quantizing operation.
For examples of such a "mixed integer" network design, please see the Quantization Aware Training examples:
Last updated