1 of 75

0.6 What is Concrete ML?

⭐️ Star the repo on Github | 🗣 Community support forum | 📁 Contribute to the project

Concrete-ML is an open-source, privacy-preserving, machine learning inference framework based on fully homomorphic encryption (FHE). It enables data scientists without any prior knowledge of cryptography to automatically turn machine learning models into their FHE equivalent, using familiar APIs from Scikit-learn and PyTorch (see how it looks for linear models, tree-based models, and neural networks).

Fully Homomorphic Encryption (FHE) is an encryption technique that allows computing directly on encrypted data, without needing to decrypt it. With FHE, you can build private-by-design applications without compromising on features. You can learn more about FHE in this introduction or by joining the FHE.org community.

Example usage

Here is a simple example of classification on encrypted data using logistic regression. More examples can be found here.

from sklearn.datasets import make_classification
from sklearn.model_selection import train_test_split
from concrete.ml.sklearn import LogisticRegression

# Lets create a synthetic data-set
x, y = make_classification(n_samples=100, class_sep=2, n_features=30, random_state=42)

# Split the data-set into a train and test set
X_train, X_test, y_train, y_test = train_test_split(
    x, y, test_size=0.2, random_state=42
)

# Now we train in the clear and quantize the weights
model = LogisticRegression(n_bits=8)
model.fit(X_train, y_train)

# We can simulate the predictions in the clear
y_pred_clear = model.predict(X_test)

# We then compile on a representative set
model.compile(X_train)

# Finally we run the inference on encrypted inputs
y_pred_fhe = model.predict(X_test, execute_in_fhe=True)

print("In clear  :", y_pred_clear)
print("In FHE    :", y_pred_fhe)
print(f"Similarity: {int((y_pred_fhe == y_pred_clear).mean()*100)}%")

# Output:
    # In clear  : [0 0 0 0 1 0 1 0 1 1 0 0 1 0 0 1 1 1 0 0]
    # In FHE    : [0 0 0 0 1 0 1 0 1 1 0 0 1 0 0 1 1 1 0 0]
    # Similarity: 100%

This example shows the typical flow of a Concrete-ML model:

The model is trained on unencrypted (plaintext) data using scikit-learn. As FHE operates over integers, Concrete-ML quantizes the model to use only integers during inference.
The quantized model is compiled to a FHE equivalent. Under the hood, the model is first converted to a Concrete-Numpy program, then compiled.
Inference can then be done on encrypted data. The above example shows encrypted inference in the model-development phase. Alternatively, during deployment in a client/server setting, the data is encrypted by the client, processed securely by the server, and then decrypted by the client.

Current limitations

To make a model work with FHE, the only constraint is to make it run within the supported precision limitations of Concrete-ML (currently 16-bit integers). Thus, machine learning models are required to be quantized, which sometimes leads to a loss of accuracy versus the original model, which operates on plaintext.

Additionally, Concrete-ML currently only supports FHE inference. On the other hand, training has to be done on unencrypted data, producing a model which is then converted to a FHE equivalent that can perform encrypted inference, i.e. prediction over encrypted data.

Finally, in Concrete-ML there is currently no support for pre-processing model inputs and post-processing model outputs. These processing stages may involve text-to-numerical feature transformation, dimensionality reduction, KNN or clustering, featurization, normalization, and the mixing of results of ensemble models.

All of these issues are currently being addressed and significant improvements are expected to be released in the coming months.

Concrete stack

Concrete-ML is built on top of Zama's Concrete framework. It uses Concrete-Numpy, which itself uses the Concrete-Compiler and the Concrete-Library. To use these libraries directly, refer to the Concrete-Numpy and Concrete-Framework documentations.

Online demos and tutorials

Various tutorials are available for the built-in models and for deep learning. In addition, several standalone demos for use-cases can be found in the Demos and Tutorials section.

If you have built awesome projects using Concrete-ML, feel free to let us know and we'll link to your work!

Additional resources

Looking for support? Ask our team!

Support forum: https://community.zama.ai (we answer in less than 24 hours).
Live discussion on the FHE.org Discord server: https://discord.fhe.org (inside the #concrete channel).
Do you have a question about Zama? You can write us on Twitter or send us an email at: [email protected]

Getting Started

Installation

Please note that not all hardware/OS combinations are supported. Determine your platform, OS version, and Python version before referencing the table below.

Depending on your OS, Concrete-ML may be installed with Docker or with pip:

OS / HW

Available on Docker

Available on pip

Linux

Yes

Windows

Yes

Not currently

Windows Subsystem for Linux

Yes

macOS (Intel)

Yes

macOS (Apple Silicon, ie M1, M2 etc)

Yes

Not currently

Also, only some versions of python are supported: in the current release, these are 3.7 (Linux only), 3.8, and 3.9. Moreover, the Concrete-ML Python package requires glibc >= 2.28. On Linux, you can check your glibc version by running ldd --version.

Concrete-ML can be installed on Kaggle (see question on community for more details), but not on Google Colab (see question on community for more details).

Most of these limits are shared with the rest of the Concrete stack (namely Concrete-Numpy and Concrete-Compiler). Support for more platforms will be added in the future.

Using PyPi

Requirements

Installing Concrete-ML using PyPi requires a Linux-based OS or macOS running on an x86 CPU. For Apple Silicon, Docker is the only currently supported option (see below).

Installing on Windows can be done using Docker or WSL. On WSL, Concrete-ML will work as long as the package is not installed in the /mnt/c/ directory, which corresponds to the host OS filesystem.

Installation

To install Concrete-ML from PyPi, run the following:

pip install -U pip wheel setuptools
pip install concrete-ml

This will automatically install all dependencies, notably Concrete-Numpy.

Using Docker

Concrete-ML can be installed using Docker by either pulling the latest image or a specific version:

docker pull zamafhe/concrete-ml:latest
# or
docker pull zamafhe/concrete-ml:v0.4.0

The image can be used with Docker volumes, see the Docker documentation here.

The image can then be used via the following command:

# Without local volume:
docker run --rm -it -p 8888:8888 zamafhe/concrete-ml

# With local volume to save notebooks on host:
docker run --rm -it -p 8888:8888 -v /host/path:/data zamafhe/concrete-ml

This will launch a Concrete-ML enabled Jupyter server in Docker that can be accessed directly from a browser.

Alternatively, a shell can be lauched in Docker, with or without volumes:

docker run --rm -it zamafhe/concrete-ml /bin/bash

Key Concepts

Concrete-ML is built on top of Concrete-Numpy, which enables Numpy programs to be converted into FHE circuits.

Lifecycle of a Concrete-ML model

I. Model development

training: A model is trained using plaintext, non-encrypted, training data.
quantization: The model is converted into an integer equivalent using quantization. Concrete-ML performs this step either during training (Quantization Aware Training) or after training (Post-training Quantization), depending on model type. Quantization converts inputs, model weights, and all intermediate values of the inference computation to integers. More information is available here.
simulation using the Virtual Library: Testing FHE models on very large data-sets can take a long time. Furthermore, not all models are compatible with FHE constraints out-of-the-box. Simulation using the Virtual Library allows you to execute a model that was quantized, to measure the accuracy it would have in FHE, but also to determine the modifications required to make it FHE compatible. Simulation is described in more detail here.
compilation: Once the model is quantized, simulation can confirm it has good accuracy in FHE. The model then needs to be compiled using Concrete's FHE compiler to produce an equivalent FHE circuit. This circuit is represented as an MLIR program consisting of low level cryptographic operations. You can read more about FHE compilation here, MLIR here, and about the low-level Concrete library here.
inference: The compiled model can then be executed on encrypted data, once the proper keys have been generated. The model can also be deployed to a server and used to run private inference on encrypted inputs.

You can see some examples of the model development workflow here.

II. Model deployment

client/server deployment: In a client/server setting, the model can be exported in a way that:
- allows the client to generate keys, encrypt, and decrypt.
- provides a compiled model that can run on the server to perform inference on encrypted data.
key generation: The data owner (client) needs to generate a pair of private keys (to encrypt/decrypt their data and results) and a public evaluation key (for the model's FHE evaluation on the server).

You can see an example of the model deployment workflow here.

Cryptography concepts

Concrete-ML and Concrete-Numpy are tools that hide away the details of the underlying cryptography scheme, called TFHE. However, some cryptography concepts are still useful when using these two toolkits:

encryption/decryption: These operations transform plaintext, i.e. human-readable information, into ciphertext, i.e. data that contains a form of the original plaintext that is unreadable by a human or computer without the proper key to decrypt it. Encryption takes plaintext and an encryption key and produces ciphertext, while decryption is the inverse operation.
encrypted inference: FHE allows a third party to execute (i.e. run inference or predict) a machine learning model on encrypted data (a ciphertext). The result of the inference is also encrypted and can only be read by the person who receives the decryption key.
keys: A key is a series of bits used within an encryption algorithm for encrypting data so that the corresponding ciphertext appears random.
key generation: Cryptographic keys need to be generated using random number generators. Their size may be large and key generation may take a long time. However, keys only need to be generated once for each model used by a client.
guaranteed correctness of encrypted computations: To achieve security, TFHE, the underlying encryption scheme, adds random noise as ciphertexts. This can induce errors during processing of encrypted data, depending on noise parameters. By default, Concrete-ML uses parameters that ensure the correctness of the encrypted computation, so there is no need to account for noise parametrization. Therefore, the results on encrypted data will be the same as the results of simulation on clear data.

While Concrete-ML users only need to understand the cryptography concepts above, for a deeper understanding of the cryptography behind the Concrete stack, please see the whitepaper on TFHE and Programmable Boostrapping or this series of blogs.

Model accuracy considerations under FHE constraints

To respect FHE constraints, all numerical programs that include non-linear operations over encrypted data must have all inputs, constants, and intermediate values represented with integers of a maximum of 16 bits.

Thus, Concrete-ML quantizes the input data and model outputs in the same way as weights and activations. The main levers to control accumulator bit-width are the number of bits used for the inputs, weights, and activations of the model. These parameters are crucial to comply with the constraint on accumulator bit-widths. Please refer to the quantization documentation for more details about how to develop models with quantization in Concrete-ML.

However, these methods may cause a reduction in the accuracy of the model since its representative power is diminished. Most importantly, carefully choosing a quantization approach can alleviate accuracy loss, all the while allowing compilation to FHE. Concrete-ML offers built-in models that already include quantization algorithms, and users only need to configure some of their parameters, such as the number of bits, discussed above. See the advanced quantization guide for information about configuring these parameters for various models.

Additional specific methods can help to make models compatible with FHE constraints. For instance, dimensionality reduction can reduce the number of input features and, thus, the maximum accumulator bit-width reached within a circuit. Similarly, sparsity-inducing training methods, such as pruning, deactivate some features during inference, which also helps. For now, dimensionality reduction is considered as a pre-processing step, while pruning is used in the built-in neural networks.

The configuration of model quantization parameters is illustrated in the advanced examples for Linear and Logistic Regressions and dimensionality reduction is shown in the Poisson regression example.

Inference in the Cloud

Concrete-ML models can be easily deployed in a client/server setting, enabling the creation of privacy-preserving services in the cloud.

As seen in the , a Concrete-ML model, once compiled to FHE, generates machine code that performs the inference on private data. Furthermore, secret encryption keys are needed so that the user can securely encrypt their data and decrypt the inference result. An evaluation key is also needed for the server to securely process the user's encrypted data.

Keys are generated by the user once for each service they use, based on the model the service provides and its cryptographic parameters.

The overall communications protocol to enable cloud deployment of machine learning services can be summarized in the following diagram:

The steps detailed above are as follows:

The model developer deploys the compiled machine learning model to the server. This model includes the cryptographic parameters. The server is now ready to provide private inference.
The client requests the cryptographic parameters (also called "client specs"). Once it receives them from the server, the secret and evaluation keys are generated.
The client sends the evaluation key to the server. The server is now ready to accept requests from this client. The client sends their encrypted data.
The server uses the evaluation key to securely run inference on the user's data and sends back the encrypted result.
The client now decrypts the result and can send back new requests.

For more information on how to implement this basic secure inference protocol, refer to the and to the .

Demos and Tutorials

This section lists several demos that apply Concrete-ML to some popular machine learning problems. They show how to build ML models that perform well under FHE constraints, and then how to perform the conversion to FHE.

Simpler tutorials that discuss only model usage and compilation are also available for the built-in models and for deep learning.

Built-in Models

Tree-based Models

Concrete-ML provides several of the most popular classification and regression tree models that can be found in :

Concrete-ML

scikit-learn

In addition to support for scikit-learn, Concrete-ML also supports 's XGBClassifier:

Concrete-ML

XGboost

Example

Here's an example of how to use this model in FHE on a popular data-set using some of scikit-learn's pre-processing tools. A more complete example can be found in the .

In a similar example, the decision boundaries of the Concrete-ML model can be plotted, and, then, compared to the results of the classical XGBoost model executed in the clear. A 6-bit model is shown in order to illustrate the impact of quantization on classification. Similar plots can be found in the .

Quantization parameters

This graph above shows that, when using a sufficiently high bit-width, quantization has little impact on the decision boundaries of the Concrete-ML FHE decision tree models. As the quantization is done individually on each input feature, the impact of quantization is strongly reduced, and, thus, FHE tree-based models reach similar accuracy as their floating point equivalents. Using 6 bits for quantization makes the Concrete-ML model reach or exceed the floating point accuracy. The number of bits for quantization can be adjusted through the n_bits parameter.

When n_bits is set low, the quantization process may sometimes create some artifacts that could lead to a decrease in performance, but the execution speed in FHE decreases. In this way, it is possible to adjust the accuracy/speed trade-off, and some accuracy can be recovered by increasing the n_estimators.

The following graph shows that using 5-6 bits of quantization is usually sufficient to reach the performance of a non-quantized XGBoost model on floating point data. The metrics plotted are accuracy and F1-score on the spambase data-set.

Neural Networks

Concrete-ML provides simple built-in neural networks models with a scikit-learn interface through the NeuralNetClassifier and NeuralNetRegressor classes.

Concrete-ML

Scikit-learn

The neural network models are implemented with , which provides a scikit-learn-like interface to Torch models (more ).

The Concrete-ML models are multi-layer, fully-connected networks with customizable activation functions and a number of neurons in each layer. This approach is similar to what is available in scikit-learn using the MLPClassifier/MLPRegressor classes. The built-in models train easily with a single call to .fit(), which will automatically quantize the weights and activations. These models use Quantization Aware Training, allowing good performance for low precision (down to 2-3 bit) weights and activations.

While NeuralNetClassifier and NeuralNetClassifier provide scikit-learn-like models, their architecture is somewhat restricted in order to make training easy and robust. If you need more advanced models, you can convert custom neural networks as described in the .

Good quantization parameter values are critical to make models . Weights and activations should be quantized to low precision (e.g. 2-4 bits). Furthermore, the sparsity of the network can be tuned , to avoid accumulator overflow.

Example usage

To create an instance of a Fully Connected Neural Network (FCNN), you need to instantiate one of the NeuralNetClassifier and NeuralNetRegressor classes and configure a number of parameters that are passed to their constructor. Note that some parameters need to be prefixed by module__, while others don't. Basically, the parameters that are related to the model, i.e. the underlying nn.Module, must have the prefix. The parameters that are related to training options do not require the prefix.

The shows the behavior of built-in neural networks on several synthetic data-sets.

The figure above shows, on the right, the Concrete-ML neural network, trained with Quantization Aware Training, in a FHE-compatible configuration. The figure compares this network to the floating-point equivalent, trained with scikit-learn.

Architecture parameters

module__n_layers: number of layers in the FCNN, must be at least 1. Note that this is the total number of layers. For a single, hidden layer NN model, set module__n_layers=2
module__n_outputs: number of outputs (classes or targets)
module__input_dim: dimensionality of the input
module__activation_function: can be one of the Torch activations (e.g. nn.ReLU, see the full list )

Quantization parameters

n_w_bits (default 3): number of bits for weights
n_a_bits (default 3): number of bits for activations and inputs
n_accum_bits (default 8): maximum accumulator bit-width that is desired. The implementation will attempt to keep accumulators under this bit-width through , i.e. setting some weights to zero

Training parameters (from skorch)

max_epochs: The number of epochs to train the network (default 10)
verbose: Whether to log loss/metrics during training (default: False)
lr: Learning rate (default 0.001)

Other parameters from skorch are in the .

Advanced parameters

module__n_hidden_neurons_multiplier: The number of hidden neurons will be automatically set proportional to the dimensionality of the input (i.e. the value for module__input_dim). This parameter controls the proportionality factor and is set to 4 by default. This value gives good accuracy while avoiding accumulator overflow. See the and sections for more info.

Network input/output

When you have training data in the form of a NumPy array, and targets in a NumPy 1D array, you can set:

Class weights

You can give weights to each class to use in training. Note that this must be supported by the underlying PyTorch loss function.

Overflow errors

The n_hidden_neurons_multiplier parameter influences training accuracy as it controls the number of non-zero neurons that are allowed in each layer. Increasing n_hidden_neurons_multiplier improves accuracy, but should take into account precision limitations to avoid overflow in the accumulator. The default value is a good compromise that avoids overflow in most cases, but you may want to change the value of this parameter to reduce the breadth of the network if you have overflow errors. A value of 1 should be completely safe with respect to overflow.

Pandas

Concrete-ML provides partial support for Pandas, with most available models (linear and tree-based models) usable on Pandas dataframes just as they would be used with NumPy arrays.

The table below summarizes current compatibility:

Methods

Support Pandas dataframe

Example

The following example considers a LogisticRegression model on a simple classification problem. A more advanced example can be found in the , which considers a XGBClassifier.

Built-in Model Examples

These examples illustrate the basic usage of built-in Concrete-ML models. For more examples showing how to train high-accuracy models on more complex data-sets, see the section.

FHE constraints considerations

In Concrete-ML, built-in linear models are exact equivalents to their scikit-learn counterparts. Indeed, since they do not apply any non-linearity during inference, these models are very fast (~1ms FHE inference time) and can use high precision integers (between 20-25 bits).

Tree-based models apply non-linear functions that enable comparisons of inputs and trained thresholds. Thus, they are limited with respect to the number of bits used to represent the inputs. But as these examples show, in practice 5-6 bits are sufficient to exactly reproduce the behavior of their scikit-learn counterpart models.

As shown in the examples below, built-in neural networks can be configured to work with user-specified accumulator sizes, which allow the user to adjust the speed/accuracy tradeoff.

It is recommended to use to configure the speed/accuracy trade-off for tree-based models and neural networks, using grid-search or your own heuristics.

List of examples

1. Linear and logistic regression

These examples show how to use the built-in linear models on synthetic data, which allows for easy visualization of the decision boundaries or trend lines. Executing these 1D and 2D models in FHE takes around 1 millisecond.

2. Generalized linear models

These two examples show generalized linear models (GLM) on the real-world data-set. As the non-linear, inverse-link functions are computed, these models do not use , and are, thus, very fast (~1ms execution time).

3. Decision tree

Using the data-set, this example shows how to train a classifier that detects spam, based on features extracted from email messages. A grid-search is performed over decision-tree hyper-parameters to find the best ones.

4. XGBoost and Random Forest classifier

This example shows how to train tree-ensemble models (either XGBoost or Random Forest), first on a synthetic data-set, and then on the data-set. Grid-search is used to find the best number of trees in the ensemble.

5. XGBoost regression

Privacy-preserving prediction of house prices is shown in this example, using the data-set. Using 50 trees in the ensemble, with 5 bits of precision for the input features, the FHE regressor obtains an score of 0.90 and an execution time of 7-8 seconds.

6. Fully connected neural network

Two different configurations of the built-in, fully-connected neural networks are shown. First, a small bit-width accumulator network is trained on and compared to a Pytorch floating point network. Second, a larger accumulator (>8 bits) is demonstrated on .

7. Comparison of classifiers

Based on three different synthetic data-sets, all the built-in classifiers are demonstrated in this notebook, showing accuracies, inference times, accumulator bit-widths, and decision boundaries.

Deep Learning

Deep Learning Examples

These examples illustrate the basic usage of Concrete-ML to build various types of neural networks. They use simple data-sets, focusing on the syntax and usage of Concrete-ML. For examples showing how to train high-accuracy models on more complex data-sets, see the Demos and Tutorials section.

FHE constraints considerations

The examples listed here make use of simulation (using the Virtual Library) to perform evaluation over large test sets. Since FHE execution can be slow, only a few FHE executions can be performed. The correctness guarantees of Concrete-ML ensure that accuracy measured with simulation is the same that will be obtained during FHE execution.

Some examples constrain accumulators to 7-8 bits, which can be sufficient for simple data-sets. Up to 16-bit accumulators can be used, but this introduces a slowdown of 4-5x compared to 8-bit accumulators.

List of Examples

1. Step-by-step guide to building a custom NN

Quantization aware training example

Shows how to use Quantization Aware Training and pruning when starting out from a classical PyTorch network. This example uses a simple data-set and a small NN, which achieves good accuracy with low accumulator size.

2. Custom convolutional NN on the Digits data-set

Convolutional Neural Network

Following the Step-by-step guide, this notebook implements a Quantization Aware Training convolutional neural network on the MNIST data-set. It uses 3-bit weights and activations, giving a 7-bit accumulator.

Advanced topics

Quantization

Quantization is the process of constraining an input from a continuous or otherwise large set of values (such as real numbers) to a discrete set (such as integers).

This means that some accuracy in the representation is lost (e.g. a simple approach is to eliminate least-significant bits). However, in many cases in machine learning, it is possible to adapt the models to give meaningful results while using these smaller data types. This significantly reduces the number of bits necessary for intermediary results during the execution of these machine learning models.

Since FHE is currently limited to 16-bit integers, it is necessary to quantize models to make them compatible. As a general rule, the smaller the bit-width of integer values used in models, the better the FHE performance. This trade-off should be taken into account when designing models, especially neural networks.

Overview of quantization in Concrete ML

Quantization implemented in Concrete-ML is applied in two ways:

Built-in models apply quantization internally and the user only needs to configure some quantization parameters. This approach requires little work by the user but may not be a one-size-fits-all solution for all types of models. The final quantized model is FHE-friendly and ready to predict over encrypted data. In this setting, Post-Training Quantization (PTQ) is for linear models, data quantization is used for tree-based models and, finally, Quantization Aware Training (QAT) is included in the built-in neural network models.
For custom neural networks with more complex topology, obtaining FHE-compatible models with good accuracy requires QAT. Concrete-ML offers the possibility for the user to perform quantization before compiling to FHE. This can be achieved through a third-party library that offers QAT tools, such as for PyTorch. In this approach, the user is responsible for implementing a full-integer model, respecting FHE constraints. Please refer to the for tips on designing FHE neural networks.

While Concrete-ML quantizes machine learning models, the data the client has is often in floating point. The Concrete-ML models provide APIs to quantize inputs and de-quantize outputs.

Please note that the floating point input is quantized in the clear, i.e. it is converted to integers before being encrypted. Moreover, the model's output are also integers and are decrypted before de-quantization.

Basics of quantization

Let be the range of a value to quantize where is the minimum and is the maximum. To quantize a range of floating point values (in ) to integer values (in ), the first step is to choose the data type that is going to be used. Many ML models work with weights and activations represented as 8-bit integers, so this will be the value used in this example. Knowing the number of bits that can be used for a value in the range , the scale can be computed :

where is the number of bits (). For the sake of example, let's take .

In practice, the quantization scale is then . This means the gap between consecutive representable values cannot be smaller than , which, in turn, means there can be a substantial loss of precision. Every interval of length will be represented by a value within the range .

The other important parameter from this quantization schema is the zero point value. This essentially brings the 0 floating point value to a specific integer. If the quantization scheme is asymmetric (quantized values are not centered in 0), the resulting will be in .

When using quantized values in a matrix multiplication or convolution, the equations for computing the result become more complex. The IntelLabs Distiller documentation provides a more of the maths used to quantize values and how to keep computations consistent.

Configuring model quantization parameters

Built-in models provide a simple interface for configuring quantization parameters, most notably the number of bits used for inputs, model weights, intermediary values, and output values.

For , the quantization is done post-training. Thus, the model is trained in floating point, and then, the best integer weight representations are found, depending on the distribution of inputs and weights. For these models, the user can select the value of the n_bits parameter.

For linear models, n_bits is used to quantize both model inputs and weights. Depending on the number of features, you can use a single integer value for the n_bits parameter (e.g. a value between 2 and 7). When the number of features is high, the n_bits parameter should be decreased if you encounter compilation errors. It is also possible to quantize inputs and weights with different numbers of bits by passing a dictionary to n_bits containing the op_inputs and op_weights keys.

For , the training and test data is quantized. The maximum accumulator bit-width for a model trained with n_bits=n for this type of model is known beforehand: it will need n+1 bits. Through experimentation, it was determined that in many cases a value of 5 or 6 bits gives the same accuracy as training in floating point and values above n=7 do not increase model performance (but they induce a strong slowdown).

Tree-based models can directly control the accumulator bit-width used. However, if 6 or 7 bits are not sufficient to obtain good accuracy on your data-set, one option is to use an ensemble model (RandomForest or XGBoost) and increase the number of trees in the ensemble. This, however, will have a detrimental impact on FHE execution speed.

For built-in , several linear layers are used. Thus, the outputs of a layer are used as inputs to a new layer. Built-in neural networks use Quantization Aware Training. The parameters controlling the maximum accumulator bit-width are the number of weights and activation bits ( module__n_w_bits, module__n_a_bits ), but also the pruning factor. This factor is determined automatically by specifying a desired accumulator bit-width module__n_accum_bits and, optionally, a multiplier factor, module__n_hidden_neurons_multiplier.

Note that for built-in neural networks, the maximum accumulator bit-width cannot be precisely controlled. To use many input features and a high number of bits is beneficial for model accuracy, but it can conflict with the 16-bit accumulator constraint. Finding the best quantization parameters to maximize accuracy, while keeping the accumulator size down, can only be accomplished through experimentation.

Quantizing model inputs and outputs

The models implemented in Concrete-ML provide features to let the user quantize the input data and de-quantize the output data.

In a client/server setting, the client is responsible for quantizing inputs before sending them, encrypted, to the server. Further, the client must de-quantize the encrypted integer results received from the server. See the section for more details.

Here is a simple example showing how to perform inference, starting from float values and ending up with float values. Note that the FHE engine that is compiled for the ML models does not support data batching.

Resources

IntelLabs distiller explanation of quantization:

Pruning

Pruning is a method to reduce neural network complexity, usually applied in order to reduce the computation cost or memory size. Pruning is used in Concrete-ML to control the size of accumulators in neural networks, thus making them FHE-compatible. See for an explanation of accumulator bit-width constraints.

Overview of pruning in Concrete ML

Pruning is used in Concrete-ML for two types of neural networks:

Built-in include a pruning mechanism that can be parameterized by the user. The pruning type is based on L1-norm. To comply with FHE constraints, Concrete-ML uses unstructured pruning, as the aim is not to eliminate neurons or convolutional filters completely, but to decrease their accumulator bit-width.
Custom neural networks, to work well under FHE constraints, should include pruning. When implemented with PyTorch, you can use the (e.g.L1-Unstructured) to good effect.

Basics of pruning

In neural networks, a neuron computes a linear combination of inputs and learned weights, then applies an activation function.

The neuron computes:

When building a full neural network, each layer will contain multiple neurons, which are connected to the inputs or to the neuron outputs of a previous layer.

For every neuron shown in each layer of the figure above, the linear combinations of inputs and learned weights are computed. Depending on the values of the inputs and weights, the sum - which for Concrete-ML neural networks is computed with integers - can take a range of different values.

To respect the bit-width constraint of the FHE , the values of the accumulator must remain small to be representable using a maximum of 16 bits. In other words, the values must be between 0 and .

Pruning a neural network entails fixing some of the weights to be zero during training. This is advantageous to meet FHE constraints, as irrespective of the distribution of , multiplying these input values by 0 does not increase the accumulator value.

Fixing some of the weights to 0 makes the network graph look more similar to the following:

While pruning weights can reduce the prediction performance of the neural network, studies show that a high level of pruning (above 50%) can often be applied. See here how Concrete-ML uses pruning in .

Pruning in practice

In the formula above, in the worst case, the maximum number of the input and weights that can make the result exceed $n$ bits is given by:

Here, is the maximum precision allowed.

For example, if and with , the worst case is where all inputs and weights are equal to their maximal value . In this case, there can be at most elements in the multi-sums.

In practice, the distribution of the weights of a neural network is Gaussian, with many weights either 0 or having a small value. This enables exceeding the worst-case number of active neurons without having to risk overflowing the bit-width. In built-in neural networks, the parameter n_hidden_neurons_multiplier is multiplied with to determine the total number of non-zero weights that should be kept in a neuron.

Compilation

Compilation of a model produces machine code that executes the model on encrypted data. In some cases, notably in the client/server setting, the compilation can be done by the server when loading the model for serving.

As FHE execution is much slower than execution on non-encrypted data, Concrete-ML has a simulation mode, using an execution mode named the Virtual Library. Since, by default, the cryptographic parameters are chosen such that the results obtained in FHE are the same as those on clear data, the Virtual Library allows you to benchmark models quickly during development.

Compilation

Concrete-ML implements machine model inference using Concrete-Numpy as a backend. In order to execute in FHE, a numerical program written in Concrete-Numpy needs to be compiled. This functionality is described here, and Concrete-ML hides away most of the complexity of this step, completing the entire compilation process itself.

From the perspective of the Concrete-ML user, the compilation process performed by Concrete-Numpy can be broken up into 3 steps:

tracing the Numpy program and creating a Concrete-Numpy op-graph
checking the op-graph for FHE compatability
producing machine code for the op-graph (this step automatically determines cryptographic parameters)

Additionally, the client/server API packages the result of the last step in a way that allows the deployment of the encrypted circuit to a server, as well as key generation, encryption, and decryption on the client side.

Simulation with the Virtual Library

The first step in the list above takes a Python function implemented using the Concrete-Numpy supported operation set and transforms it into an executable operation graph.

The result of this single step of the compilation pipeline allows the:

execution of the op-graph, which includes TLUs, on clear non-encrypted data. This is, of course, not secure, but it is much faster than executing in FHE. This mode is useful for debugging, i.e. to find the appropriate hyper-parameters. This mode is called the Virtual Library (which is referred as Virtual Circuits in Concrete-Numpy).
verification of the maximum bit-width of the op-graph, to determine FHE compatibility, without actually compiling the circuit to machine code.

Enabling Virtual Library execution requires the definition of a compilation Configuration. As simulation does not execute in FHE, this can be considered unsafe:

    COMPIL_CONFIG_VL = Configuration(
        dump_artifacts_on_unexpected_failures=False,
        enable_unsafe_features=True,  # This is for our tests in Virtual Library only
    )

Next, the following code uses the simulation mode for built-in models:

    clf.compile(
        X_train,
        use_virtual_lib=True,
        configuration=COMPIL_CONFIG_VL,
    )

And finally, for custom models, it is possible to enable simulation using the following syntax:

    quantized_numpy_module = compile_torch_model(
        torch_model,  # our model
        X_train,  # a representative input-set to be used for both quantization and compilation
        n_bits={"net_inputs": 5, "op_inputs": 3, "op_weights": 3, "net_outputs": 5},
        import_qat=is_qat,  # signal to the conversion function whether the network is QAT
        use_virtual_lib=True,
        configuration=COMPIL_CONFIG_VL,
    )

Obtaining the simulated predictions of the models using the Virtual Library has the same syntax as execution in FHE:

    Z = clf.predict_proba(X, execute_in_fhe=True)

Moreover, the maximum accumulator bit-width is determined as follows:

    bit_width = clf.quantized_module_.forward_fhe.graph.maximum_integer_bit_width()

A simple Concrete-Numpy example

While Concrete-ML hides away all the Concrete-Numpy code that performs model inference, it can be useful to understand how Concrete-Numpy code works. Here is a toy example for a simple linear regression model on integers. Note that this is just an example to illustrate compilation concepts. Generally, it is recommended to use the built-in models, which provide linear regression out of the box.

import numpy
from concrete.numpy import compiler

# Let's assume Quantization has been applied and we are left with integers only.
# This is essentially the work of Concrete-ML

# Some parameters (weight and bias) for our model taking a single feature
w = [2]
b = 2

# The function that implements our model
@compiler({"x": "encrypted"})
def linear_model(x):
    return w @ x + b

# A representative input-set is needed to compile the function
# (used for tracing)
n_bits_input = 2
inputset = numpy.arange(0, 2**n_bits_input).reshape(-1, 1)
circuit = linear_model.compile(inputset)

# Use the API to get the maximum bit-width in the circuit
max_bit_width = circuit.graph.maximum_integer_bit_width()
print("Max bit_width = ", max_bit_width)
# Max bit_width =  4

# Test our FHE inference
circuit.encrypt_run_decrypt(numpy.array([3]))
# 8

# Print the graph of the circuit
print(circuit)
# %0 = 2                     # ClearScalar<uint2>
# %1 = [2]                   # ClearTensor<uint2, shape=(1,)>
# %2 = x                     # EncryptedTensor<uint2, shape=(1,)>
# %3 = matmul(%1, %2)        # EncryptedScalar<uint3>
# %4 = add(%3, %0)           # EncryptedScalar<uint4>
# return %4

Production Deployment

Concrete-ML provides functionality to deploy FHE machine learning models in a client/server setting. The deployment workflow and model serving pattern is as follows:

Deployment

The diagram above shows the steps that a developer goes through to prepare a model for encrypted inference in a client/server setting. The training of the model and its compilation to FHE are performed on a development machine. Three different files are created when saving the model:

client.zip contains client.specs.json which lists the secure cryptographic parameters needed for the client to generate private and evaluation keys.
serialized_processing.json describes the pre-processing and post-processing required by the machine learning model, such as quantization parameters to quantize the input and de-quantize the output. It should be deployed in the same way as client.zip.
server.zip contains the compiled model. This file is sufficient to run the model on a server. The compiled model is machine-architecture specific (i.e. a model compiled on x86 cannot run on ARM).

The compiled model (server.zip) is deployed to a server and the cryptographic parameters (client.zip), along with the model metadata (serialized_processing.json), are shared with the clients. In some settings, such as a phone application, the client.zip can be directly deployed on the client device and the server does not need to host it.

Serving

The client-side deployment of a secured inference machine learning model follows the schema above. First, the client obtains the cryptographic parameters (stored in client.zip) and generates a private encryption/decryption key as well as a set of public evaluation keys. The public evaluation keys are then sent to the server, while the secret key remains on the client.

The private data is then encrypted by the client as described in serialized_processing.json, and it is then sent to the server. Server-side, the FHE model inference is run on encrypted inputs using the public evaluation keys.

The encrypted result is then returned by the server to the client, which decrypts it using its private key. Finally, the client performs any necessary post-processing of the decrypted result as specified in serialized_processing.json.

The server-side implementation of a Concrete-ML model follows the diagram above. The public evaluation keys sent by clients are stored. They are then retrieved for the client that is querying the service and used to evaluate the machine learning model stored in server.zip. Finally, the server sends the encrypted result of the computation back to the client.

Example notebook

For a complete example, see .

Developer Guide

Workflow

Set Up the Project

Concrete-ML is a Python library, so Python should be installed to develop Concrete-ML. v3.8 and v3.9 are the only supported versions. Concrete-ML also uses Poetry and Make.

First of all, you need to git clone the project:

Some tests require files tracked by git-lfs to be downloaded. To do so, please follow the instructions on , then run git lfs pull.

Automatic installation

A simple way to have everything installed is to use the development Docker (see the guide). On Linux and macOS, you have to run the script in ./script/make_utils/setup_os_deps.sh. Specify the --linux-install-python flag if you want to install python3.8 as well on apt-enabled Linux distributions. The script should install everything you need for Docker and bare OS development (you can first review the content of the file to check what it will do).

For Windows users, the setup_os_deps.sh script does not install dependencies because of how many different installation methods there are due to the lack of a single package manager.

The first step is to (as some of the dev tools depend on it), then . In addition to installing Python, you are still going to need the following software available on path on Windows, as some of the basic dev tools depend on them:

git
jq
make

Development on Windows only works with the Docker environment. Follow .

Manual installation

Python

To manually install Python, you can follow guide (alternatively, you can google how to install Python 3.8 (or 3.9)).

Poetry

Poetry is used as the package manager. It drastically simplifies dependency and environment management. You can follow official guide to install it.

As there is no concrete-compiler package for Windows, only the dev dependencies can be installed. This requires Poetry >= 1.2.

make

The dev tools use make to launch various commands.

On Linux, you can install make from your distribution's preferred package manager.

On macOS, you can install a more recent version of make via brew:

It is possible to install gmake as make. Check this for more info.

On Windows, check .

In the following sections, be sure to use the proper make tool for your system: make, gmake, or other.

Cloning the repository

To get the source code of Concrete-ML, clone the code repository using the link for your favorite communication protocol (ssh or https).

Setting up environment on your host OS

We are going to make use of virtual environments. This helps to keep the project isolated from other Python projects in the system. The following commands will create a new virtual environment under the project directory and install dependencies to it.

The following command will not work on Windows if you don't have Poetry >= 1.2.

Activating the environment

Finally, activate the newly created environment using the following command:

macOS or Linux

Windows

Setting up environment on Docker

Docker automatically creates and sources a venv in ~/dev_venv/

The venv persists thanks to volumes. It also creates a volume for ~/.cache to speedup later reinstallations. You can check which Docker volumes exist with:

You can still run all make commands inside Docker (to update the venv, for example). Be mindful of the current venv being used (the name in parentheses at the beginning of your command prompt).

Leaving the environment

After your work is done, you can simply run the following command to leave the environment:

Syncing environment with the latest changes

From time to time, new dependencies will be added to the project or the old ones will be removed. The command below will make sure the project has the proper environment, so run it regularly!

Troubleshooting your environment

in your OS

If you are having issues, consider using the dev Docker exclusively (unless you are working on OS-specific bug fixes or features).

Here are the steps you can take on your OS to try and fix issues:

At this point, you should consider using Docker as nobody will have the exact same setup as you. If, however, you need to develop on your OS directly, you can .

in Docker

Here are the steps you can take in your Docker to try and fix issues:

If the problem persists at this point, you should ask for help. We're here and ready to assist!

Set Up Docker

Before you start this section, you must install Docker by following official guide.

Building the image

Once you have access to this repository and the dev environment is installed on your host OS (via make setup_env once ), you should be able to launch the commands to build the dev Docker image with make docker_build.

Once you do that, you can get inside the Docker environment using the following command:

After you finish your work, you can leave Docker by using the exit command or by pressing CTRL + D.

Documentation

Using GitBook

Documentation with GitBook is done mainly by pushing content on GitHub. GitBook then pulls the docs from the repository and publishes. In most cases, GitBook is just a mirror of what is available in GitHub.

There are, however, some use-cases where documentation can be modified directly in GitBook (and, then, push the modifications to GitHub), for example when the documentation is modified by a person outside of Zama. In this case, a GitHub branch is created, and a GitHub space is associated to it: modifications are done in this space and automatically pushed to the branch. Once the modifications have been completed, one can simply create a pull-request, to finally merge modifications on the main branch.

Using Sphinx

Documentation can alternatively be built using Sphinx:

The documentation contains both files written by hand by developers (the .md files) and files automatically created by parsing the source files.

Then to open it, go to docs/_build/html/index.html or use the follwing command:

To build and open the docs at the same time, use:

Support and Issues

Concrete-ML is a constant work-in-progress, and thus may contain bugs or suboptimal APIs.

Before opening an issue or asking for support, please read this documentation to understand common issues and limitations of Concrete-ML. You can also check the outstanding issues on github.

Furthermore, undefined behavior may occur if the input-set, which is internally used by the compilation core to set bit-widths of some intermediate data, is not sufficiently representative of the future user inputs. With all the inputs in the input-set, it appears that intermediate data can be represented as an n-bit integer. But, for a particular computation, this same intermediate data needs additional bits to be represented. The FHE execution for this computation will result in an incorrect output, as typically occurs in integer overflows in classical programs.

If you didn't find an answer, you can ask a question on the Zama forum or in the FHE.org Discord.

Submitting an issue

When submitting an issue (here), ideally include as much information as possible. In addition to the Python script, the following information is useful:

the reproducibility rate you see on your side
any insight you might have on the bug
any workaround you have been able to find

If you would like to contribute to a project and send pull requests, take a look at the contributor guide.

Contributing

There are three ways to contribute to Concrete-ML:

You can open issues to report bugs and typos and to suggest ideas.
You can ask to become an official contributor by emailing . Only approved contributors can send pull requests (PR), so please make sure to get in touch before you do.
You can also provide new tutorials or use-cases, showing what can be done with the library. The more examples we have, the better and clearer it is for the other users.

1. Creating a new branch

To create your branch, you have to use the issue ID somewhere in the branch name:

e.g.

2. Before committing

2.1 Conformance

Each commit to Concrete-ML should conform to the standards of the project. You can let the development tools fix some issues automatically with the following command:

Conformance can be checked using the following command:

2.2 Testing

Your code must be well documented, containing tests and not breaking other tests:

You need to make sure you get 100% code coverage. The make pytest command checks that by default and will fail with a coverage report at the end should some lines of your code not be executed during testing.

If your coverage is below 100%, you should write more tests and then create the pull request. If you ignore this warning and create the PR, GitHub actions will fail and your PR will not be merged.

There may be cases where covering your code is not possible (an exception that cannot be triggered in normal execution circumstances). In those cases, you may be allowed to disable coverage for some specific lines. This should be the exception rather than the rule, and reviewers will ask why some lines are not covered. If it appears they can be covered, then the PR won't be accepted in that state.

3. Committing

Concrete-ML uses a consistent commit naming scheme, and you are expected to follow it as well (the CI will make sure you do). The accepted format can be printed to your terminal by running:

e.g.

Just a reminder that commit messages are checked in the comformance step and are rejected if they don't follow the rules. To learn more about conventional commits, check page.

4. Rebasing

You should rebase on top of the main branch before you create your pull request. Merge commits are not allowed, so rebasing on main before pushing gives you the best chance of to avoid rewriting parts of your PR later if conflicts arise with other PRs being merged. After you commit changes to your new branch, you can use the following commands to rebase:

You can learn more about rebasing .

Inner Workings

Importing ONNX

Internally, Concrete-ML uses ONNX operators as intermediate representation (or IR) for manipulating machine learning models produced through export for PyTorch, Hummingbird, and skorch.

As ONNX is becoming the standard exchange format for neural networks, this allows Concrete-ML to be flexible while also making model representation manipulation easy. In addition, it allows for straight-forward mapping to NumPy operators, supported by Concrete-Numpy to use Concrete stack's FHE-conversion capabilities.

Torch to NumPy conversion using ONNX

The diagram below gives an overview of the steps involved in the conversion of an ONNX graph to a FHE-compatible format (i.e. a format that can be compiled to FHE through Concrete-Numpy).

All Concrete-ML built-in models follow the same pattern for FHE conversion:

The models are trained with sklearn or PyTorch.
All models have a PyTorch implementation for inference. This implementation is provided either by a third-party tool such as Hummingbird or implemented directly in Concrete-ML.
The PyTorch model is exported to ONNX. For more information on the use of ONNX in Concrete-ML, see here.
The Concrete-ML ONNX parser checks that all the operations in the ONNX graph are supported and assigns reference NumPy operations to them. This step produces a NumpyModule.
Quantization is performed on the NumpyModule, producing a QuantizedModule. Two steps are performed: calibration and assignment of equivalent QuantizedOp objects to each ONNX operation. The QuantizedModule class is the quantized counterpart of the NumpyModule.
Once the QuantizedModule is built, Concrete-Numpy is used to trace the ._forward() function of the QuantizedModule.

Moreover, by passing a user provided nn.Module to step 2 of the above process, Concrete-ML supports custom user models. See the associated FHE-friendly model documentation for instructions about working with such models.

Once an ONNX model is imported, it is converted to a NumpyModule, then to a QuantizedModule and, finally, to a FHE circuit. However, as the diagram shows, it is perfectly possible to stop at the NumpyModule level if you just want to run the PyTorch model as NumPy code without doing quantization.

Note that the NumpyModule interpreter currently supports the following ONNX operators.

Inspecting the ONNX models

In order to better understand how Concrete-ML works under the hood, it is possible to access each model in their ONNX format and then either print it or visualize it by importing the associated file in Netron. For example, with LogisticRegression:

import onnx
from sklearn.datasets import make_classification
from sklearn.model_selection import train_test_split

from concrete.ml.sklearn import LogisticRegression

# Create the data for classification
x, y = make_classification(n_samples=250, class_sep=2, n_features=30, random_state=42)

# Retrieve train and test sets
X_train, X_test, y_train, y_test = train_test_split(
    x, y, test_size=0.4, random_state=42
)

# Fix the number of bits to used for quantization
model = LogisticRegression(n_bits=8)

# Fit the model
model.fit(X_train, y_train)

# Access to the model
onnx_model = model.onnx_model

# Print the model
print(onnx.helper.printable_graph(onnx_model.graph))

# Save the model
onnx.save(onnx_model, "tmp.onnx")

# And then visualize it with Netron

Quantization Tools

Quantizing data

Concrete-ML has support for quantized ML models and also provides quantization tools for Quantization Aware Training and Post-Training Quantization. The core of this functionality is the conversion of floating point values to integers and back. This is done using QuantizedArray in concrete.ml.quantization.

The QuantizedArray class takes several arguments that determine how float values are quantized:

n_bits define the precision of the quantization
values are floating point values that will be converted to integers
is_signed determines if the quantized integer values should allow negative values
is_symmetric determines if the range of floating point values to be quantized should be taken as symmetric around zero

See also the UniformQuantizer reference for more information:

from concrete.ml.quantization import QuantizedArray
import numpy
numpy.random.seed(0)
A = numpy.random.uniform(-2, 2, 10)
print("A = ", A)
# array([ 0.19525402,  0.86075747,  0.4110535,  0.17953273, -0.3053808,
#         0.58357645, -0.24965115,  1.567092 ,  1.85465104, -0.46623392])
q_A = QuantizedArray(7, A)
print("q_A.qvalues = ", q_A.qvalues)
# array([ 37,          73,          48,         36,          9,
#         58,          12,          112,        127,         0])
# the quantized integers values from A.
print("q_A.quantizer.scale = ", q_A.quantizer.scale)
# 0.018274684777173276, the scale S.
print("q_A.quantizer.zero_point = ", q_A.quantizer.zero_point)
# 26, the zero point Z.
print("q_A.dequant() = ", q_A.dequant())
# array([ 0.20102153,  0.85891018,  0.40204307,  0.18274685, -0.31066964,
#         0.58478991, -0.25584559,  1.57162289,  1.84574316, -0.4751418 ])
# Dequantized values.

It is also possible to use symmetric quantization, where the integer values are centered around 0:

q_A = QuantizedArray(3, A)
print("Unsigned: q_A.qvalues = ", q_A.qvalues)
print("q_A.quantizer.zero_point = ", q_A.quantizer.zero_point)
# Unsigned: q_A.qvalues =  [2 4 2 2 0 3 0 6 7 0]
# q_A.quantizer.zero_point =  1

q_A = QuantizedArray(3, A, is_signed=True, is_symmetric=True)
print("Signed Symmetric: q_A.qvalues = ", q_A.qvalues)
print("q_A.quantizer.zero_point = ", q_A.quantizer.zero_point)
# Signed Symmetric: q_A.qvalues =  [ 0  1  1  0  0  1  0  3  3 -1]
# q_A.quantizer.zero_point =  0

In the following example, showing the de-quantization of model outputs, the QuantizedArray class is used in a different way. Here it uses pre-quantized integer values and has the scale and zero-point set explicitly. Once the QuantizedArray is constructed, calling dequant() will compute the floating point values corresponding to the integer values qvalues, which are the output of the forward_fhe.encrypt_run_decrypt(..) call.

import numpy
from concrete.ml.quantization.quantizers import QuantizationOptions

q_values = [0, 0, 1, 2, 3, -1]
QuantizedArray(
        q_A.quantizer.n_bits,
        q_values,
        value_is_float=False,
        options=q_A.quantizer.quant_options,
        stats=q_A.quantizer.quant_stats,
        params=q_A.quantizer.quant_params,
).dequant()

Quantized modules

Machine learning models are implemented with a diverse set of operations, such as convolution, linear transformations, activation functions, and element-wise operations. When working with quantized values, these operations cannot be carried out in an equivalent way to floating point values. With quantization, it is necessary to re-scale the input and output values of each operation to fit in the quantization domain.

In Concrete-ML, the quantized equivalent of a scikit-learn model or a PyTorch nn.Module is the QuantizedModule. Note that only inference is implemented in the QuantizedModule, and it is built through a conversion of the inference function of the corresponding scikit-learn or PyTorch module.

Built-in neural networks expose the quantized_module member, while a QuantizedModule is also the result of the compilation of custom models through compile_torch_model and compile_brevitas_qat_model.

The quantized versions of floating point model operations are stored in the QuantizedModule. The ONNX_OPS_TO_QUANTIZED_IMPL dictionary maps ONNX floating point operators (e.g. Gemm) to their quantized equivalent (e.g. QuantizedGemm). For more information on implementing these operations, please see the FHE-compatible op-graph section.

The computation graph is taken from the corresponding floating point ONNX graph exported from scikit-learn using HummingBird, or from the ONNX graph exported by PyTorch. Calibration is used to obtain quantized parameters for the operations in the QuantizedModule. Parameters are also determined for the quantization of inputs during model deployment.

Calibration is the process of determining the typical distributions of values encountered for the intermediate values of a model during inference.

To perform calibration, an interpreter goes through the ONNX graph in topological order and stores the intermediate results as it goes. The statistics of these values determine quantization parameters.

That QuantizedModule generates the Concrete-Numpy function that is compiled to FHE. The compilation will succeed if the intermediate values conform to the 16-bits precision limit of the Concrete stack. See the compilation section for details.

Resources

Lei Mao's blog on quantization: Quantization for Neural Networks
Google paper on neural network quantization and integer-only inference: Quantization and Training of Neural Networks for Efficient Integer-Arithmetic-Only Inference

External Libraries

Hummingbird

Hummingbird is a third-party, open-source library that converts machine learning models into tensor computations, and it can export these models to ONNX. The list of supported models can be found in the Hummingbird documentation.

Concrete-ML allows the conversion of an ONNX inference to NumPy inference (note that NumPy is always the entry point to run models in FHE with Concrete-ML).

Hummingbird exposes a convert function that can be imported as follows from the hummingbird.ml package:

# Disable Hummingbird warnings for pytest.
import warnings
warnings.filterwarnings("ignore")
from hummingbird.ml import convert

This function can be used to convert a machine learning model to an ONNX as follows:

from sklearn.datasets import make_classification
from sklearn.linear_model import LogisticRegression

# Instantiate the logistic regression from sklearn
model = LogisticRegression()

# Create synthetic data
X, y = make_classification(
    n_samples=100, n_features=20, n_classes=2
)

# Fit the model
model.fit(X, y)

# Convert the model to ONNX
onnx_model = convert(model, backend="onnx", test_input=X).model

In theory, the resulting onnx_model could be used directly within Concrete-ML's get_equivalent_numpy_forward method (as long as all operators present in the ONNX model are implemented in NumPy) and get the NumPy inference.

In practice, there are some steps needed to clean the ONNX output and make the graph compatible with Concrete-ML, such as applying quantization where needed or deleting/replacing non-FHE friendly ONNX operators (such as Softmax and ArgMax).

Skorch

Concrete-ML uses Skorch to implement multi-layer, fully-connected PyTorch neural networks in a way that is compatible with the scikit-learn API.

This wrapper implements Torch training boilerplate code, lessening the work required of the user. It is possible to add hooks during the training phase, for example once an epoch is finished.

Skorch allows the user to easily create a classifier or regressor around a neural network (NN), implemented in Torch as a nn.Module, which is used by Concrete-ML to provide a fully-connected, multi-layer NN with a configurable number of layers and optional pruning (see pruning and the neural network documentation for more information).

Under the hood, Concrete-ML uses a Skorch wrapper around a single PyTorch module, SparseQuantNeuralNetImpl. More information can be found in the API guide.

class SparseQuantNeuralNetImpl(nn.Module):
    """Sparse Quantized Neural Network classifier.

Brevitas

Brevitas is a quantization aware learning toolkit built on top of PyTorch. It provides quantization layers that are one-to-one equivalents to PyTorch layers, but also contain operations that perform the quantization during training.

While Brevitas provides many types of quantization, for Concrete-ML, a custom "mixed integer" quantization applies. This "mixed integer" quantization is much simpler than the "integer only" mode of Brevitas. The "mixed integer" network design is defined as:

all weights and activations of convolutional, linear and pooling layers must be quantized (e.g. using Brevitas layers, QuantConv2D, QuantAvgPool2D, QuantLinear)
PyTorch floating-point versions of univariate functions can be used. E.g. torch.relu, nn.BatchNormalization2D, torch.max (encrypted vs. constant), torch.add, torch.exp. See the PyTorch supported layers page for a full list.

The "mixed integer" mode used in Concrete-ML neural networks is based on the "integer only" Brevitas quantization that makes both weights and activations representable as integers during training. However, through the use of lookup tables in Concrete-ML, floating point univariate PyTorch functions are supported.

For "mixed integer" quantization to work, the first layer of a Brevitas nn.Module must be a QuantIdentity layer. However, you can then use functions such as torch.sigmoid on the result of such a quantizing operation.

import torch.nn as nn

class QATnetwork(nn.Module):
    def __init__(self):
        super(QATnetwork, self).__init__()
        self.quant_inp = qnn.QuantIdentity(
            bit_width=4, return_quant_tensor=True)
        # ...

    def forward(self, x):
        out = self.quant_inp(x)
        return torch.sigmoid(out)
        # ...

For examples of such a "mixed integer" network design, please see the Quantization Aware Training examples:

or go to the MNIST use-case example.

You can also refer to the SparseQuantNeuralNetImpl class, which is the basis of the built-in NeuralNetworkClassifier.

concrete.ml.common.check_inputs.md

module `concrete.ml.common.check_inputs`

Check and conversion tools.

Utils that are used to check (including convert) some data types which are compatible with scikit-learn to numpy types.

function `check_array_and_assert`

sklearn.utils.check_array with an assert.

Equivalent of sklearn.utils.check_array, with a final assert that the type is one which is supported by Concrete-ML.

Args:

X (object): Input object to check / convert

Returns: The converted and validated array

function `check_X_y_and_assert`

sklearn.utils.check_X_y with an assert.

Equivalent of sklearn.utils.check_X_y, with a final assert that the type is one which is supported by Concrete-ML.

Args:

X (ndarray, list, sparse matrix): Input data
y (ndarray, list, sparse matrix): Labels
*args: The arguments to pass to check_X_y
**kwargs: The keyword arguments to pass to check_X_y

Returns: The converted and validated arrays

concrete.ml.common.debugging.custom_assert.md

module `concrete.ml.common.debugging.custom_assert`

Provide some variants of assert.

function `assert_true`

Provide a custom assert to check that the condition is True.

Args:

condition (bool): the condition. If False, raise AssertionError
on_error_msg (str): optional message for precising the error, in case of error
error_type (Type[Exception]): the type of error to raise, if condition is not fulfilled. Default to AssertionError

function `assert_false`

Provide a custom assert to check that the condition is False.

Args:

condition (bool): the condition. If True, raise AssertionError
on_error_msg (str): optional message for precising the error, in case of error
error_type (Type[Exception]): the type of error to raise, if condition is not fulfilled. Default to AssertionError

function `assert_not_reached`

Provide a custom assert to check that a piece of code is never reached.

Args:

on_error_msg (str): message for precising the error
error_type (Type[Exception]): the type of error to raise, if condition is not fulfilled. Default to AssertionError

concrete.ml.common.debugging.md

module `concrete.ml.common.debugging`

Module for debugging.

Global Variables

custom_assert

concrete.ml.common.md

module `concrete.ml.common`

Module for shared data structures and code.

Global Variables

debugging
check_inputs
utils

concrete.ml.common.utils.md

module `concrete.ml.common.utils`

Utils that can be re-used by other pieces of code in the module.

Global Variables

MAX_BITWIDTH_BACKWARD_COMPATIBLE

function `replace_invalid_arg_name_chars`

replace_invalid_arg_name_chars(arg_name: str) → str

Sanitize arg_name, replacing invalid chars by _.

This does not check that the starting character of arg_name is valid.

Args:

arg_name (str): the arg name to sanitize.

Returns:

str: the sanitized arg name, with only chars in _VALID_ARG_CHARS.

function `generate_proxy_function`

generate_proxy_function(
    function_to_proxy: Callable,
    desired_functions_arg_names: Iterable[str]
) → Tuple[Callable, Dict[str, str]]

Generate a proxy function for a function accepting only *args type arguments.

This returns a runtime compiled function with the sanitized argument names passed in desired_functions_arg_names as the arguments to the function.

Args:

function_to_proxy (Callable): the function defined like def f(*args) for which to return a function like f_proxy(arg_1, arg_2) for any number of arguments.
desired_functions_arg_names (Iterable[str]): the argument names to use, these names are sanitized and the mapping between the original argument name to the sanitized one is returned in a dictionary. Only the sanitized names will work for a call to the proxy function.

Returns:

Tuple[Callable, Dict[str, str]]: the proxy function and the mapping of the original arg name to the new and sanitized arg names.

function `get_onnx_opset_version`

get_onnx_opset_version(onnx_model: ModelProto) → int

Return the ONNX opset_version.

Args:

onnx_model (onnx.ModelProto): the model.

Returns:

int: the version of the model

function `manage_parameters_for_pbs_errors`

manage_parameters_for_pbs_errors(
    p_error: Optional[float] = None,
    global_p_error: Optional[float] = None
)

Return (p_error, global_p_error) that we want to give to Concrete-Numpy and the compiler.

The returned (p_error, global_p_error) depends on user's parameters and the way we want to manage defaults in Concrete-ML, which may be different from the way defaults are managed in Concrete-Numpy

Principle: - if none are set, we set global_p_error to a default value of our choice - if both are set, we raise an error - if one is set, we use it and forward it to Concrete-Numpy and the compiler

Note that global_p_error is currently not simulated by the VL, i.e., taken as 0.

Args:

p_error (Optional[float]): probability of error of a single PBS.
global_p_error (Optional[float]): probability of error of the full circuit.

Returns:

(p_error, global_p_error): parameters to give to the compiler

Raises:

ValueError: if the two parameters are set (this is not as in Concrete-Numpy)

function `check_there_is_no_p_error_options_in_configuration`

check_there_is_no_p_error_options_in_configuration(configuration)

Check the user did not set p_error or global_p_error in configuration.

It would be dangerous, since we set them in direct arguments in our calls to Concrete-Numpy.

Args:

configuration: Configuration object to use during compilation

concrete.ml.deployment.md

module `concrete.ml.deployment`

Module for deployment of the FHE model.

Global Variables

fhe_client_server

concrete.ml.onnx.convert.md

module `concrete.ml.onnx.convert`

ONNX conversion related code.

Global Variables

IMPLEMENTED_ONNX_OPS
OPSET_VERSION_FOR_ONNX_EXPORT

function `get_equivalent_numpy_forward_and_onnx_model`

get_equivalent_numpy_forward_and_onnx_model(
    torch_module: Module,
    dummy_input: Union[Tensor, Tuple[Tensor, ]],
    output_onnx_file: Optional[Path, str] = None
) → Tuple[Callable[, Tuple[ndarray, ]], GraphProto]

Get the numpy equivalent forward of the provided torch Module.

Args:

torch_module (torch.nn.Module): the torch Module for which to get the equivalent numpy forward.
dummy_input (Union[torch.Tensor, Tuple[torch.Tensor, ...]]): dummy inputs for ONNX export.
output_onnx_file (Optional[Union[Path, str]]): Path to save the ONNX file to. Will use a temp file if not provided. Defaults to None.

Returns:

Tuple[Callable[..., Tuple[numpy.ndarray, ...]], onnx.GraphProto]: The function that will execute the equivalent numpy code to the passed torch_module and the generated ONNX model.

function `get_equivalent_numpy_forward`

get_equivalent_numpy_forward(
    onnx_model: ModelProto,
    check_model: bool = True
) → Callable[, Tuple[ndarray, ]]

Get the numpy equivalent forward of the provided ONNX model.

Args:

onnx_model (onnx.ModelProto): the ONNX model for which to get the equivalent numpy forward.
check_model (bool): set to True to run the onnx checker on the model. Defaults to True.

Raises:

ValueError: Raised if there is an unsupported ONNX operator required to convert the torch model to numpy.

Returns:

Callable[..., Tuple[numpy.ndarray, ...]]: The function that will execute the equivalent numpy function.

concrete.ml.onnx.md

module `concrete.ml.onnx`

ONNX module.

Global Variables

onnx_impl_utils
ops_impl
onnx_utils
convert
onnx_model_manipulations

concrete.ml.onnx.onnx_impl_utils.md

module `concrete.ml.onnx.onnx_impl_utils`

Utility functions for onnx operator implementations.

function `numpy_onnx_pad`

Pad a tensor according to ONNX spec, using an optional custom pad value.

Args:

x (numpy.ndarray): input tensor to pad
pads (List[int]): padding values according to ONNX spec
pad_value (Optional[Union[float, int]]): value used to fill in padding, default 0
int_only (bool): set to True to generate integer only code with Concrete-Numpy

Returns:

res (numpy.ndarray): the input tensor with padding applied

function `compute_conv_output_dims`

Compute the output shape of a pool or conv operation.

See https://pytorch.org/docs/stable/generated/torch.nn.AvgPool2d.html for details on the computation of the output shape.

Args:

input_shape (Tuple[int, ...]): shape of the input to be padded as N x C x H x W
kernel_shape (Tuple[int, ...]): shape of the conv or pool kernel, as Kh x Kw (or n-d)
pads (Tuple[int, ...]): padding values following ONNX spec: dim1_start, dim2_start, .. dimN_start, dim1_end, dim2_end, ... dimN_end where in the 2-d case dim1 is H, dim2 is W
strides (Tuple[int, ...]): strides for each dimension
ceil_mode (int): set to 1 to use the ceil function to compute the output shape, as described in the PyTorch doc

Returns:

res (Tuple[int, ...]): shape of the output of a conv or pool operator with given parameters

function `compute_onnx_pool_padding`

Compute any additional padding needed to compute pooling layers.

The ONNX standard uses ceil_mode=1 to match tensorflow style pooling output computation. In this setting, the kernel can be placed at a valid position even though it contains values outside of the input shape including padding. The ceil_mode parameter controls whether this mode is enabled. If the mode is not enabled, the output shape follows PyTorch rules.

Args:

input_shape (Tuple[int, ...]): shape of the input to be padded as N x C x H x W
kernel_shape (Tuple[int, ...]): shape of the conv or pool kernel, as Kh x Kw (or n-d)
pads (Tuple[int, ...]): padding values following ONNX spec: dim1_start, dim2_start, .. dimN_start, dim1_end, dim2_end, ... dimN_end where in the 2-d case dim1 is H, dim2 is W
strides (Tuple[int, ...]): strides for each dimension
ceil_mode (int): set to 1 to use the ceil function to compute the output shape, as described in the PyTorch doc

Returns:

res (Tuple[int, ...]): shape of the output of a conv or pool operator with given parameters

function `onnx_avgpool_compute_norm_const`

Compute the average pooling normalization constant.

This constant can be a tensor of the same shape as the input or a scalar.

Args:

input_shape (Tuple[int, ...]): shape of the input to be padded as N x C x H x W
kernel_shape (Tuple[int, ...]): shape of the conv or pool kernel, as Kh x Kw (or n-d)
pads (Tuple[int, ...]): padding values following ONNX spec: dim1_start, dim2_start, .. dimN_start, dim1_end, dim2_end, ... dimN_end where in the 2-d case dim1 is H, dim2 is W
strides (Tuple[int, ...]): strides for each dimension
ceil_mode (int): set to 1 to use the ceil function to compute the output shape, as described in the PyTorch doc

Returns:

res (float): tensor or scalar, corresponding to normalization factors to apply for the average pool computation for each valid kernel position

concrete.ml.onnx.onnx_utils.md

module `concrete.ml.onnx.onnx_utils`

Utils to interpret an ONNX model with numpy.

Global Variables

ATTR_TYPES
ATTR_GETTERS
ONNX_OPS_TO_NUMPY_IMPL
ONNX_COMPARISON_OPS_TO_NUMPY_IMPL_FLOAT
ONNX_COMPARISON_OPS_TO_NUMPY_IMPL_BOOL
ONNX_OPS_TO_NUMPY_IMPL_BOOL
IMPLEMENTED_ONNX_OPS

function `get_attribute`

get_attribute(attribute: AttributeProto) → Any

Get the attribute from an ONNX AttributeProto.

Args:

attribute (onnx.AttributeProto): The attribute to retrieve the value from.

Returns:

Any: The stored attribute value.

function `get_op_type`

get_op_type(node)

Construct the qualified type name of the ONNX operator.

Args:

node (Any): ONNX graph node

Returns:

result (str): qualified name

function `execute_onnx_with_numpy`

execute_onnx_with_numpy(graph: GraphProto, *inputs: ndarray) → Tuple[ndarray, ]

Execute the provided ONNX graph on the given inputs.

Args:

graph (onnx.GraphProto): The ONNX graph to execute.
*inputs: The inputs of the graph.

Returns:

Tuple[numpy.ndarray]: The result of the graph's execution.

function `remove_initializer_from_input`

remove_initializer_from_input(model: ModelProto)

Remove initializers from model inputs.

In some cases, ONNX initializers may appear, erroneously, as graph inputs. This function searches all model inputs and removes those that are initializers.

Args:

model (onnx.ModelProto): the model to clean

Returns:

onnx.ModelProto: the cleaned model

concrete.ml.pytest.md

module `concrete.ml.pytest`

Module which is used to contain common functions for pytest.

Global Variables

torch_models
utils

concrete.ml.pytest.utils.md

module `concrete.ml.pytest.utils`

Common functions or lists for test files, which can't be put in fixtures.

Global Variables

regressor_models
classifier_models
classifiers
regressors

function `sanitize_test_and_train_datasets`

Sanitize datasets depending on the model type.

Args:

model: the model
x: the first output of load_data, i.e., the inputs
y: the second output of load_data, i.e., the labels

Returns: Tuple containing sanitized (model_params, x, y, x_train, y_train, x_test)

concrete.ml.quantization.md

module `concrete.ml.quantization`

Modules for quantization.

Global Variables

quantizers
base_quantized_op
quantized_module
quantized_ops
post_training

concrete.ml.quantization.quantized_module.md

module `concrete.ml.quantization.quantized_module`

QuantizedModule API.

class `QuantizedModule`

Inference for a quantized model.

method `init`

property fhe_circuit

Get the FHE circuit.

Returns:

Circuit: the FHE circuit

property is_compiled

Return the compiled status of the module.

Returns:

bool: the compiled status of the module.

property onnx_model

Get the ONNX model.

.. # noqa: DAR201

Returns:

_onnx_model (onnx.ModelProto): the ONNX model

property post_processing_params

Get the post-processing parameters.

Returns:

Dict[str, Any]: the post-processing parameters

method `compile`

Compile the forward function of the module.

Args:

q_inputs (Union[Tuple[numpy.ndarray, ...], numpy.ndarray]): Needed for tracing and building the boundaries.
configuration (Optional[Configuration]): Configuration object to use during compilation
compilation_artifacts (Optional[DebugArtifacts]): Artifacts object to fill during
show_mlir (bool): if set, the MLIR produced by the converter and which is going to be sent to the compiler backend is shown on the screen, e.g., for debugging or demo. Defaults to False.
use_virtual_lib (bool): set to use the so called virtual lib simulating FHE computation. Defaults to False.
p_error (Optional[float]): probability of error of a single PBS.
global_p_error (Optional[float]): probability of error of the full circuit. Not simulated by the VL, i.e., taken as 0
verbose_compilation (bool): whether to show compilation information

Returns:

Circuit: the compiled Circuit.

method `dequantize_output`

Take the last layer q_out and use its dequant function.

Args:

qvalues (numpy.ndarray): Quantized values of the last layer.

Returns:

numpy.ndarray: Dequantized values of the last layer.

method `forward`

Forward pass with numpy function only.

Args:

*qvalues (numpy.ndarray): numpy.array containing the quantized values.
debug (bool): In debug mode, returns quantized intermediary values of the computation. This is useful when a model's intermediary values in Concrete-ML need to be compared with the intermediary values obtained in pytorch/onnx. When set, the second return value is a dictionary containing ONNX operation names as keys and, as values, their input QuantizedArray or ndarray. The use can thus extract the quantized or float values of quantized inputs.

Returns:

(numpy.ndarray): Predictions of the quantized model

method `forward_and_dequant`

Forward pass with numpy function only plus dequantization.

Args:

*q_x (numpy.ndarray): numpy.ndarray containing the quantized input values. Requires the input dtype to be int64.

Returns:

(numpy.ndarray): Predictions of the quantized model

method `post_processing`

Post-processing of the quantized output.

Args:

qvalues (numpy.ndarray): numpy.ndarray containing the quantized input values.

Returns:

(numpy.ndarray): Predictions of the quantized model

method `quantize_input`

Take the inputs in fp32 and quantize it using the learned quantization parameters.

Args:

*values (numpy.ndarray): Floating point values.

Returns:

Union[numpy.ndarray, Tuple[numpy.ndarray, ...]]: Quantized (numpy.int64) values.

method `set_inputs_quantization_parameters`

Set the quantization parameters for the module's inputs.

Args:

*input_q_params (UniformQuantizer): The quantizer(s) for the module.

concrete.ml.sklearn.rf.md

module `concrete.ml.sklearn.rf`

Implements RandomForest models.

class `RandomForestClassifier`

Implements the RandomForest classifier.

method `init`

Initialize the RandomForestClassifier.

noqa: DAR101

property onnx_model

Get the ONNX model.

.. # noqa: DAR201

Returns:

onnx.ModelProto: the ONNX model

class `RandomForestRegressor`

Implements the RandomForest regressor.

method `init`

Initialize the RandomForestRegressor.

noqa: DAR101

property onnx_model

Get the ONNX model.

.. # noqa: DAR201

Returns:

onnx.ModelProto: the ONNX model

concrete.ml.sklearn.svm.md

module `concrete.ml.sklearn.svm`

Implement Support Vector Machine.

class `LinearSVR`

A Regression Support Vector Machine (SVM).

Parameters:

n_bits (int, Dict[str, int]): Number of bits to quantize the model. If an int is passed for n_bits, the value will be used for quantizing inputs and weights. If a dict is passed, then it should contain "op_inputs" and "op_weights" as keys with corresponding number of quantization bits so that: - op_inputs : number of bits to quantize the input values - op_weights: number of bits to quantize the learned parameters Default to 8.

For more details on LinearSVR please refer to the scikit-learn documentation: https://scikit-learn.org/stable/modules/generated/sklearn.svm.LinearSVR.html

method `init`

__init__(
    n_bits=8,
    epsilon=0.0,
    tol=0.0001,
    C=1.0,
    loss='epsilon_insensitive',
    fit_intercept=True,
    intercept_scaling=1.0,
    dual=True,
    verbose=0,
    random_state=None,
    max_iter=1000
)

class `LinearSVC`

A Classification Support Vector Machine (SVM).

Parameters:

n_bits (int, Dict[str, int]): Number of bits to quantize the model. If an int is passed for n_bits, the value will be used for quantizing inputs and weights. If a dict is passed, then it should contain "op_inputs" and "op_weights" as keys with corresponding number of quantization bits so that: - op_inputs : number of bits to quantize the input values - op_weights: number of bits to quantize the learned parameters Default to 8.

For more details on LinearSVC please refer to the scikit-learn documentation: https://scikit-learn.org/stable/modules/generated/sklearn.svm.LinearSVC.html

method `init`

__init__(
    n_bits=8,
    penalty='l2',
    loss='squared_hinge',
    dual=True,
    tol=0.0001,
    C=1.0,
    multi_class='ovr',
    fit_intercept=True,
    intercept_scaling=1,
    class_weight=None,
    verbose=0,
    random_state=None,
    max_iter=1000
)

concrete.ml.sklearn.torch_modules.md

module `concrete.ml.sklearn.torch_modules`

Implement torch module.

concrete.ml.sklearn.tree.md

module `concrete.ml.sklearn.tree`

Implement the sklearn tree models.

class `DecisionTreeClassifier`

Implements the sklearn DecisionTreeClassifier.

method `init`

__init__(
    criterion='gini',
    splitter='best',
    max_depth=None,
    min_samples_split=2,
    min_samples_leaf=1,
    min_weight_fraction_leaf=0.0,
    max_features=None,
    random_state=None,
    max_leaf_nodes=None,
    min_impurity_decrease=0.0,
    class_weight=None,
    ccp_alpha: float = 0.0,
    n_bits: int = 6
)

Initialize the DecisionTreeClassifier.

noqa: DAR101

property onnx_model

Get the ONNX model.

.. # noqa: DAR201

Returns:

onnx.ModelProto: the ONNX model

class `DecisionTreeRegressor`

Implements the sklearn DecisionTreeClassifier.

method `init`

__init__(
    criterion='squared_error',
    splitter='best',
    max_depth=None,
    min_samples_split=2,
    min_samples_leaf=1,
    min_weight_fraction_leaf=0.0,
    max_features=None,
    random_state=None,
    max_leaf_nodes=None,
    min_impurity_decrease=0.0,
    ccp_alpha=0.0,
    n_bits: int = 6
)

Initialize the DecisionTreeRegressor.

noqa: DAR101

property onnx_model

Get the ONNX model.

.. # noqa: DAR201

Returns:

onnx.ModelProto: the ONNX model

concrete.ml.sklearn.tree_to_numpy.md

module `concrete.ml.sklearn.tree_to_numpy`

Implements the conversion of a tree model to a numpy function.

Global Variables

MAX_BITWIDTH_BACKWARD_COMPATIBLE
OPSET_VERSION_FOR_ONNX_EXPORT
EXPECTED_NUMBER_OF_OUTPUTS_PER_TASK

function `tree_to_numpy`

Convert the tree inference to a numpy functions using Hummingbird.

Args:

model (onnx.ModelProto): The model to convert.
x (numpy.ndarray): The input data.
framework (str): The framework from which the onnx_model is generated.
(options: 'xgboost', 'sklearn')
task (Task): The task the model is solving
output_n_bits (int): The number of bits of the output.

Returns:

Tuple[Callable, List[QuantizedArray], onnx.ModelProto]: A tuple with a function that takes a numpy array and returns a numpy array, QuantizedArray object to quantize and dequantize the output of the tree, and the ONNX model.

class `Task`

Task enumerate.

concrete.ml.torch.md

module `concrete.ml.torch`

Modules for torch to numpy conversion.

Global Variables

numpy_module

concrete.ml.torch.numpy_module.md

module `concrete.ml.torch.numpy_module`

A torch to numpy module.

Global Variables

OPSET_VERSION_FOR_ONNX_EXPORT

class `NumpyModule`

General interface to transform a torch.nn.Module to numpy module.

Args:

torch_model (Union[nn.Module, onnx.ModelProto]): A fully trained, torch model along with its parameters or the onnx graph of the model.
dummy_input (Union[torch.Tensor, Tuple[torch.Tensor, ...]]): Sample tensors for all the module inputs, used in the ONNX export to get a simple to manipulate nn representation.
debug_onnx_output_file_path: (Optional[Union[Path, str]], optional): An optional path to indicate where to save the ONNX file exported by torch for debug. Defaults to None.

method `init`

__init__(
    model: Union[Module, ModelProto],
    dummy_input: Optional[Tensor, Tuple[Tensor, ]] = None,
    debug_onnx_output_file_path: Optional[Path, str] = None
)

property onnx_model

Get the ONNX model.

.. # noqa: DAR201

Returns:

_onnx_model (onnx.ModelProto): the ONNX model

method `forward`

forward(*args: ndarray) → Union[ndarray, Tuple[ndarray, ]]

Apply a forward pass on args with the equivalent numpy function only.

Args:

*args: the inputs of the forward function

Returns:

Union[numpy.ndarray, Tuple[numpy.ndarray, ...]]: result of the forward on the given inputs

concrete.ml.version.md

module `concrete.ml.version`

File to manage the version of the package.

concrete.ml.sklearn.protocols.md

module `concrete.ml.sklearn.protocols`

Protocols.

Protocols are used to mix type hinting with duck-typing. Indeed we don't always want to have an abstract parent class between all objects. We are more interested in the behavior of such objects. Implementing a Protocol is a way to specify the behavior of objects.

To read more about Protocol please read: https://peps.python.org/pep-0544

class `Quantizer`

Quantizer Protocol.

To use to type hint a quantizer.

method `dequant`

dequant(X: 'ndarray') → ndarray

Dequantize some values.

Args:

X (numpy.ndarray): Values to dequantize

.. # noqa: DAR202

Returns:

numpy.ndarray: Dequantized values

method `quant`

quant(values: 'ndarray') → ndarray

Quantize some values.

Args:

values (numpy.ndarray): Values to quantize

.. # noqa: DAR202

Returns:

numpy.ndarray: The quantized values

class `ConcreteBaseEstimatorProtocol`

A Concrete Estimator Protocol.

property onnx_model

onnx_model.

.. # noqa: DAR202

Results: onnx.ModelProto

property quantize_input

Quantize input function.

method `compile`

compile(
    X: 'ndarray',
    configuration: 'Optional[Configuration]',
    compilation_artifacts: 'Optional[DebugArtifacts]',
    show_mlir: 'bool',
    use_virtual_lib: 'bool',
    p_error: 'float',
    global_p_error: 'float',
    verbose_compilation: 'bool'
) → Circuit

Compiles a model to a FHE Circuit.

Args:

X (numpy.ndarray): the dequantized dataset
configuration (Optional[Configuration]): the options for compilation
compilation_artifacts (Optional[DebugArtifacts]): artifacts object to fill during compilation
show_mlir (bool): whether or not to show MLIR during the compilation
use_virtual_lib (bool): whether to compile using the virtual library that allows higher bitwidths
p_error (float): probability of error of a single PBS
global_p_error (float): probability of error of the full circuit. Not simulated by the VL, i.e., taken as 0
verbose_compilation (bool): whether to show compilation information

.. # noqa: DAR202

Returns:

Circuit: the compiled Circuit.

method `fit`

fit(X: 'ndarray', y: 'ndarray', **fit_params) → ConcreteBaseEstimatorProtocol

Initialize and fit the module.

Args:

X : training data By default, you should be able to pass: * numpy arrays * torch tensors * pandas DataFrame or Series
y (numpy.ndarray): labels associated with training data
**fit_params: additional parameters that can be used during training

.. # noqa: DAR202

Returns:

ConcreteBaseEstimatorProtocol: the trained estimator

method `fit_benchmark`

fit_benchmark(
    X: 'ndarray',
    y: 'ndarray',
    *args,
    **kwargs
) → Tuple[ConcreteBaseEstimatorProtocol, BaseEstimator]

Fit the quantized estimator and return reference estimator.

This function returns both the quantized estimator (itself), but also a wrapper around the non-quantized trained NN. This is useful in order to compare performance between the quantized and fp32 versions of the classifier

Args:

X : training data By default, you should be able to pass: * numpy arrays * torch tensors * pandas DataFrame or Series
y (numpy.ndarray): labels associated with training data
*args: The arguments to pass to the underlying model.
**kwargs: The keyword arguments to pass to the underlying model.

.. # noqa: DAR202

Returns:

self: self fitted
model: underlying estimator

method `post_processing`

post_processing(y_preds: 'ndarray') → ndarray

Post-process models predictions.

Args:

y_preds (numpy.ndarray): predicted values by model (clear-quantized)

.. # noqa: DAR202

Returns:

numpy.ndarray: the post-processed predictions

class `ConcreteBaseClassifierProtocol`

Concrete classifier protocol.

property onnx_model

onnx_model.

.. # noqa: DAR202

Results: onnx.ModelProto

property quantize_input

Quantize input function.

method `compile`

compile(
    X: 'ndarray',
    configuration: 'Optional[Configuration]',
    compilation_artifacts: 'Optional[DebugArtifacts]',
    show_mlir: 'bool',
    use_virtual_lib: 'bool',
    p_error: 'float',
    global_p_error: 'float',
    verbose_compilation: 'bool'
) → Circuit

Compiles a model to a FHE Circuit.

Args:

X (numpy.ndarray): the dequantized dataset
configuration (Optional[Configuration]): the options for compilation
compilation_artifacts (Optional[DebugArtifacts]): artifacts object to fill during compilation
show_mlir (bool): whether or not to show MLIR during the compilation
use_virtual_lib (bool): whether to compile using the virtual library that allows higher bitwidths
p_error (float): probability of error of a single PBS
global_p_error (float): probability of error of the full circuit. Not simulated by the VL, i.e., taken as 0
verbose_compilation (bool): whether to show compilation information

.. # noqa: DAR202

Returns:

Circuit: the compiled Circuit.

method `fit`

fit(X: 'ndarray', y: 'ndarray', **fit_params) → ConcreteBaseEstimatorProtocol

Initialize and fit the module.

Args:

X : training data By default, you should be able to pass: * numpy arrays * torch tensors * pandas DataFrame or Series
y (numpy.ndarray): labels associated with training data
**fit_params: additional parameters that can be used during training

.. # noqa: DAR202

Returns:

ConcreteBaseEstimatorProtocol: the trained estimator

method `fit_benchmark`

fit_benchmark(
    X: 'ndarray',
    y: 'ndarray',
    *args,
    **kwargs
) → Tuple[ConcreteBaseEstimatorProtocol, BaseEstimator]

Fit the quantized estimator and return reference estimator.

Args:

X : training data By default, you should be able to pass: * numpy arrays * torch tensors * pandas DataFrame or Series
y (numpy.ndarray): labels associated with training data
*args: The arguments to pass to the underlying model.
**kwargs: The keyword arguments to pass to the underlying model.

.. # noqa: DAR202

Returns:

self: self fitted
model: underlying estimator

method `post_processing`

post_processing(y_preds: 'ndarray') → ndarray

Post-process models predictions.

Args:

y_preds (numpy.ndarray): predicted values by model (clear-quantized)

.. # noqa: DAR202

Returns:

numpy.ndarray: the post-processed predictions

method `predict`

predict(X: 'ndarray', execute_in_fhe: 'bool') → ndarray

Predicts for each sample the class with highest probability.

Args:

X (numpy.ndarray): Features
execute_in_fhe (bool): Whether the inference should be done in fhe or not.

.. # noqa: DAR202

Returns: numpy.ndarray

method `predict_proba`

predict_proba(X: 'ndarray', execute_in_fhe: 'bool') → ndarray

Predicts for each sample the probability of each class.

Args:

X (numpy.ndarray): Features
execute_in_fhe (bool): Whether the inference should be done in fhe or not.

.. # noqa: DAR202

Returns: numpy.ndarray

class `ConcreteBaseRegressorProtocol`

Concrete regressor protocol.

property onnx_model

onnx_model.

.. # noqa: DAR202

Results: onnx.ModelProto

property quantize_input

Quantize input function.

method `compile`

compile(
    X: 'ndarray',
    configuration: 'Optional[Configuration]',
    compilation_artifacts: 'Optional[DebugArtifacts]',
    show_mlir: 'bool',
    use_virtual_lib: 'bool',
    p_error: 'float',
    global_p_error: 'float',
    verbose_compilation: 'bool'
) → Circuit

Compiles a model to a FHE Circuit.

Args:

X (numpy.ndarray): the dequantized dataset
configuration (Optional[Configuration]): the options for compilation
compilation_artifacts (Optional[DebugArtifacts]): artifacts object to fill during compilation
show_mlir (bool): whether or not to show MLIR during the compilation
use_virtual_lib (bool): whether to compile using the virtual library that allows higher bitwidths
p_error (float): probability of error of a single PBS
global_p_error (float): probability of error of the full circuit. Not simulated by the VL, i.e., taken as 0
verbose_compilation (bool): whether to show compilation information

.. # noqa: DAR202

Returns:

Circuit: the compiled Circuit.

method `fit`

fit(X: 'ndarray', y: 'ndarray', **fit_params) → ConcreteBaseEstimatorProtocol

Initialize and fit the module.

Args:

X : training data By default, you should be able to pass: * numpy arrays * torch tensors * pandas DataFrame or Series
y (numpy.ndarray): labels associated with training data
**fit_params: additional parameters that can be used during training

.. # noqa: DAR202

Returns:

ConcreteBaseEstimatorProtocol: the trained estimator

method `fit_benchmark`

fit_benchmark(
    X: 'ndarray',
    y: 'ndarray',
    *args,
    **kwargs
) → Tuple[ConcreteBaseEstimatorProtocol, BaseEstimator]

Fit the quantized estimator and return reference estimator.

Args:

X : training data By default, you should be able to pass: * numpy arrays * torch tensors * pandas DataFrame or Series
y (numpy.ndarray): labels associated with training data
*args: The arguments to pass to the underlying model.
**kwargs: The keyword arguments to pass to the underlying model.

.. # noqa: DAR202

Returns:

self: self fitted
model: underlying estimator

method `post_processing`

post_processing(y_preds: 'ndarray') → ndarray

Post-process models predictions.

Args:

y_preds (numpy.ndarray): predicted values by model (clear-quantized)

.. # noqa: DAR202

Returns:

numpy.ndarray: the post-processed predictions

method `predict`

predict(X: 'ndarray', execute_in_fhe: 'bool') → ndarray

Predicts for each sample the expected value.

Args:

X (numpy.ndarray): Features
execute_in_fhe (bool): Whether the inference should be done in fhe or not.

.. # noqa: DAR202

Returns: numpy.ndarray

concrete.ml.sklearn.qnn.md

module `concrete.ml.sklearn.qnn`

Scikit-learn interface for concrete quantized neural networks.

Global Variables

MAX_BITWIDTH_BACKWARD_COMPATIBLE

class `SparseQuantNeuralNetImpl`

Sparse Quantized Neural Network classifier.

This class implements an MLP that is compatible with FHE constraints. The weights and activations are quantized to low bitwidth and pruning is used to ensure accumulators do not surpass an user-provided accumulator bit-width. The number of classes and number of layers are specified by the user, as well as the breadth of the network

method `init`

__init__(
    input_dim,
    n_layers,
    n_outputs,
    n_hidden_neurons_multiplier=4,
    n_w_bits=3,
    n_a_bits=3,
    n_accum_bits=8,
    activation_function=<class 'torch.nn.modules.activation.ReLU'>,
    quant_narrow=False,
    quant_signed=True
)

Sparse Quantized Neural Network constructor.

Args:

input_dim: Number of dimensions of the input data
n_layers: Number of linear layers for this network
n_outputs: Number of output classes or regression targets
n_w_bits: Number of weight bits
n_a_bits: Number of activation and input bits
n_accum_bits: Maximal allowed bitwidth of intermediate accumulators
n_hidden_neurons_multiplier: A factor that is multiplied by the maximal number of active (non-zero weight) neurons for every layer. The maximal number of neurons in the worst case scenario is: 2^n_max-1 max_active_neurons(n_max, n_w, n_a) = floor(---------------------) (2^n_w-1)*(2^n_a-1) ) The worst case scenario for the bitwidth of the accumulator is when all weights and activations are maximum simultaneously. We set, for each layer, the total number of neurons to be: n_hidden_neurons_multiplier * max_active_neurons(n_accum_bits, n_w_bits, n_a_bits) Through experiments, for typical distributions of weights and activations, the default value for n_hidden_neurons_multiplier, 4, is safe to avoid overflow.
activation_function: a torch class that is used to construct activation functions in the network (e.g. torch.ReLU, torch.SELU, torch.Sigmoid, etc)
quant_narrow : whether this network should use narrow range quantized integer values
quant_signed : whether to use signed quantized integer values

Raises:

ValueError: if the parameters have invalid values or the computed accumulator bitwidth is zero

method `enable_pruning`

enable_pruning()

Enable pruning in the network. Pruning must be made permanent to recover pruned weights.

Raises:

ValueError: if the quantization parameters are invalid

method `forward`

forward(x)

Forward pass.

Args:

x (torch.Tensor): network input

Returns:

x (torch.Tensor): network prediction

method `make_pruning_permanent`

make_pruning_permanent()

Make the learned pruning permanent in the network.

method `max_active_neurons`

max_active_neurons()

Compute the maximum number of active (non-zero weight) neurons.

The computation is done using the quantization parameters passed to the constructor. Warning: With the current quantization algorithm (asymmetric) the value returned by this function is not guaranteed to ensure FHE compatibility. For some weight distributions, weights that are 0 (which are pruned weights) will not be quantized to 0. Therefore the total number of active quantized neurons will not be equal to max_active_neurons.

Returns:

n (int): maximum number of active neurons

method `on_train_end`

on_train_end()

Call back when training is finished, can be useful to remove training hooks.

class `QuantizedSkorchEstimatorMixin`

Mixin class that adds quantization features to Skorch NN estimators.

property base_estimator_type

Get the sklearn estimator that should be trained by the child class.

property base_module_to_compile

Get the module that should be compiled to FHE. In our case this is a torch nn.Module.

Returns:

module (nn.Module): the instantiated torch module

property fhe_circuit

Get the FHE circuit.

Returns:

Circuit: the FHE circuit

property input_quantizers

Get the input quantizers.

Returns:

List[Quantizer]: the input quantizers

property n_bits_quant

Return the number of quantization bits.

This is stored by the torch.nn.module instance and thus cannot be retrieved until this instance is created.

Returns:

n_bits (int): the number of bits to quantize the network

Raises:

ValueError: with skorch estimators, the module_ is not instantiated until .fit() is called. Thus this estimator needs to be .fit() before we get the quantization number of bits. If it is not trained we raise an exception

property onnx_model

Get the ONNX model.

.. # noqa: DAR201

Returns:

_onnx_model_ (onnx.ModelProto): the ONNX model

property output_quantizers

Get the input quantizers.

Returns:

List[QuantizedArray]: the input quantizers

property quantize_input

Get the input quantization function.

Returns:

Callable : function that quantizes the input

method `get_params_for_benchmark`

get_params_for_benchmark()

Get parameters for benchmark when cloning a skorch wrapped NN.

We must remove all parameters related to the module. Skorch takes either a class or a class instance for the module parameter. We want to pass our trained model, a class instance. But for this to work, we need to remove all module related constructor params. If not, skorch will instantiate a new class instance of the same type as the passed module see skorch net.py NeuralNet::initialize_instance

Returns:

params (dict): parameters to create an equivalent fp32 sklearn estimator for benchmark

method `infer`

infer(x, **fit_params)

Perform a single inference step on a batch of data.

This method is specific to Skorch estimators.

Args:

x (torch.Tensor): A batch of the input data, produced by a Dataset
**fit_params (dict) : Additional parameters passed to the forward method of the module and to the self.train_split call.

Returns: A torch tensor with the inference results for each item in the input

method `on_train_end`

on_train_end(net, X=None, y=None, **kwargs)

Call back when training is finished by the skorch wrapper.

Check if the underlying neural net has a callback for this event and, if so, call it.

Args:

net: estimator for which training has ended (equal to self)
X: data
y: targets
kwargs: other arguments

class `FixedTypeSkorchNeuralNet`

A mixin with a helpful modification to a skorch estimator that fixes the module type.

method `get_params`

get_params(deep=True, **kwargs)

Get parameters for this estimator.

Args:

deep (bool): If True, will return the parameters for this estimator and contained subobjects that are estimators.
**kwargs: any additional parameters to pass to the sklearn BaseEstimator class

Returns:

params : dict, Parameter names mapped to their values.

class `NeuralNetClassifier`

Scikit-learn interface for quantized FHE compatible neural networks.

This class wraps a quantized NN implemented using our Torch tools as a scikit-learn Estimator. It uses the skorch package to handle training and scikit-learn compatibility, and adds quantization and compilation functionality. The neural network implemented by this class is a multi layer fully connected network trained with Quantization Aware Training (QAT).

The datatypes that are allowed for prediction by this wrapper are more restricted than standard scikit-learn estimators as this class needs to predict in FHE and network inference executor is the NumpyModule.

method `init`

__init__(
    *args,
    criterion=<class 'torch.nn.modules.loss.CrossEntropyLoss'>,
    classes=None,
    optimizer=<class 'torch.optim.adam.Adam'>,
    **kwargs
)

property base_estimator_type

property base_module_to_compile

Get the module that should be compiled to FHE. In our case this is a torch nn.Module.

Returns:

module (nn.Module): the instantiated torch module

property classes_

property fhe_circuit

Get the FHE circuit.

Returns:

Circuit: the FHE circuit

property history

property input_quantizers

Get the input quantizers.

Returns:

List[Quantizer]: the input quantizers

property n_bits_quant

Return the number of quantization bits.

This is stored by the torch.nn.module instance and thus cannot be retrieved until this instance is created.

Returns:

n_bits (int): the number of bits to quantize the network

Raises:

ValueError: with skorch estimators, the module_ is not instantiated until .fit() is called. Thus this estimator needs to be .fit() before we get the quantization number of bits. If it is not trained we raise an exception

property onnx_model

Get the ONNX model.

.. # noqa: DAR201

Returns:

_onnx_model_ (onnx.ModelProto): the ONNX model

property output_quantizers

Get the input quantizers.

Returns:

List[QuantizedArray]: the input quantizers

property quantize_input

Get the input quantization function.

Returns:

Callable : function that quantizes the input

method `fit`

fit(X, y, **fit_params)

method `get_params`

get_params(deep=True, **kwargs)

Get parameters for this estimator.

Args:

deep (bool): If True, will return the parameters for this estimator and contained subobjects that are estimators.
**kwargs: any additional parameters to pass to the sklearn BaseEstimator class

Returns:

params : dict, Parameter names mapped to their values.

method `get_params_for_benchmark`

get_params_for_benchmark()

Get parameters for benchmark when cloning a skorch wrapped NN.

Returns:

params (dict): parameters to create an equivalent fp32 sklearn estimator for benchmark

method `infer`

infer(x, **fit_params)

Perform a single inference step on a batch of data.

This method is specific to Skorch estimators.

Args:

x (torch.Tensor): A batch of the input data, produced by a Dataset
**fit_params (dict) : Additional parameters passed to the forward method of the module and to the self.train_split call.

Returns: A torch tensor with the inference results for each item in the input

method `on_train_end`

on_train_end(net, X=None, y=None, **kwargs)

Call back when training is finished by the skorch wrapper.

Check if the underlying neural net has a callback for this event and, if so, call it.

Args:

net: estimator for which training has ended (equal to self)
X: data
y: targets
kwargs: other arguments

method `predict`

predict(X, execute_in_fhe=False)

Predict on user provided data.

Predicts using the quantized clear or FHE classifier

Args:

X : input data, a numpy array of raw values (non quantized)
execute_in_fhe : whether to execute the inference in FHE or in the clear

Returns:

y_pred : numpy ndarray with predictions

class `NeuralNetRegressor`

Scikit-learn interface for quantized FHE compatible neural networks.

method `init`

__init__(*args, optimizer=<class 'torch.optim.adam.Adam'>, **kwargs)

property base_estimator_type

property base_module_to_compile

Get the module that should be compiled to FHE. In our case this is a torch nn.Module.

Returns:

module (nn.Module): the instantiated torch module

property fhe_circuit

Get the FHE circuit.

Returns:

Circuit: the FHE circuit

property history

property input_quantizers

Get the input quantizers.

Returns:

List[Quantizer]: the input quantizers

property n_bits_quant

Return the number of quantization bits.

This is stored by the torch.nn.module instance and thus cannot be retrieved until this instance is created.

Returns:

n_bits (int): the number of bits to quantize the network

Raises:

ValueError: with skorch estimators, the module_ is not instantiated until .fit() is called. Thus this estimator needs to be .fit() before we get the quantization number of bits. If it is not trained we raise an exception

property onnx_model

Get the ONNX model.

.. # noqa: DAR201

Returns:

_onnx_model_ (onnx.ModelProto): the ONNX model

property output_quantizers

Get the input quantizers.

Returns:

List[QuantizedArray]: the input quantizers

property quantize_input

Get the input quantization function.

Returns:

Callable : function that quantizes the input

method `fit`

fit(X, y, **fit_params)

method `get_params`

get_params(deep=True, **kwargs)

Get parameters for this estimator.

Args:

deep (bool): If True, will return the parameters for this estimator and contained subobjects that are estimators.
**kwargs: any additional parameters to pass to the sklearn BaseEstimator class

Returns:

params : dict, Parameter names mapped to their values.

method `get_params_for_benchmark`

get_params_for_benchmark()

Get parameters for benchmark when cloning a skorch wrapped NN.

Returns:

params (dict): parameters to create an equivalent fp32 sklearn estimator for benchmark

method `infer`

infer(x, **fit_params)

Perform a single inference step on a batch of data.

This method is specific to Skorch estimators.

Args:

x (torch.Tensor): A batch of the input data, produced by a Dataset
**fit_params (dict) : Additional parameters passed to the forward method of the module and to the self.train_split call.

Returns: A torch tensor with the inference results for each item in the input

method `on_train_end`

on_train_end(net, X=None, y=None, **kwargs)

Call back when training is finished by the skorch wrapper.

Check if the underlying neural net has a callback for this event and, if so, call it.

Args:

net: estimator for which training has ended (equal to self)
X: data
y: targets
kwargs: other arguments

concrete.ml.pytest.torch_models.md

module `concrete.ml.pytest.torch_models`

Torch modules for our pytests.

class `FCSmall`

Torch model for the tests.

method `init`

__init__(input_output, activation_function)

method `forward`

forward(x)

Forward pass.

Args:

x: the input of the NN

Returns: the output of the NN

class `FC`

Torch model for the tests.

method `init`

__init__(activation_function, input_output=3072)

method `forward`

forward(x)

Forward pass.

Args:

x: the input of the NN

Returns: the output of the NN

class `CNN`

Torch CNN model for the tests.

method `init`

__init__(input_output, activation_function)

method `forward`

forward(x)

Forward pass.

Args:

x: the input of the NN

Returns: the output of the NN

class `CNNMaxPool`

Torch CNN model for the tests with a max pool.

method `init`

__init__(input_output, activation_function)

method `forward`

forward(x)

Forward pass.

Args:

x: the input of the NN

Returns: the output of the NN

class `CNNOther`

Torch CNN model for the tests.

method `init`

__init__(input_output, activation_function)

method `forward`

forward(x)

Forward pass.

Args:

x: the input of the NN

Returns: the output of the NN

class `CNNInvalid`

Torch CNN model for the tests.

method `init`

__init__(activation_function, padding, groups, gather_slice)

method `forward`

forward(x)

Forward pass.

Args:

x: the input of the NN

Returns: the output of the NN

class `CNNGrouped`

Torch CNN model with grouped convolution for compile torch tests.

method `init`

__init__(input_output, activation_function, groups)

method `forward`

forward(x)

Forward pass.

Args:

x: the input of the NN

Returns: the output of the NN

class `NetWithLoops`

Torch model, where we reuse some elements in a loop.

Torch model, where we reuse some elements in a loop in the forward and don't expect the user to define these elements in a particular order.

method `init`

__init__(activation_function, input_output, n_fc_layers)

method `forward`

forward(x)

Forward pass.

Args:

x: the input of the NN

Returns: the output of the NN

class `MultiInputNN`

Torch model to test multiple inputs forward.

method `init`

__init__(input_output, activation_function)

method `forward`

forward(x, y)

Forward pass.

Args:

x: the first input of the NN
y: the second input of the NN

Returns: the output of the NN

class `BranchingModule`

Torch model with some branching and skip connections.

method `init`

__init__(input_output, activation_function)

method `forward`

forward(x)

Forward pass.

Args:

x: the input of the NN

Returns: the output of the NN

class `BranchingGemmModule`

Torch model with some branching and skip connections.

method `init`

__init__(input_output, activation_function)

method `forward`

forward(x)

Forward pass.

Args:

x: the input of the NN

Returns: the output of the NN

class `UnivariateModule`

Torch model that calls univariate and shape functions of torch.

method `init`

__init__(input_output, activation_function)

method `forward`

forward(x)

Forward pass.

Args:

x: the input of the NN

Returns: the output of the NN

class `StepActivationModule`

Torch model implements a step function that needs Greater, Cast and Where.

method `init`

__init__(input_output, activation_function)

method `forward`

forward(x)

Forward pass with a quantizer built into the computation graph.

Args:

x: the input of the NN

Returns: the output of the NN

class `NetWithConcatUnsqueeze`

Torch model to test the concat and unsqueeze operators.

method `init`

__init__(activation_function, input_output, n_fc_layers)

method `forward`

forward(x)

Forward pass.

Args:

x: the input of the NN

Returns: the output of the NN

class `MultiOpOnSingleInputConvNN`

Network that applies two quantized operations on a single input.

method `init`

__init__()

method `forward`

forward(x)

Forward pass.

Args:

x: the input of the NN

Returns: the output of the NN

class `FCSeq`

Torch model that should generate MatMul->Add ONNX patterns.

This network generates additions with a constant scalar

method `init`

__init__(input_output, act)

method `forward`

forward(x)

Forward pass.

Args:

x: the input of the NN

Returns: the output of the NN

class `FCSeqAddBiasVec`

Torch model that should generate MatMul->Add ONNX patterns.

This network tests the addition with a constant vector

method `init`

__init__(input_output, act)

method `forward`

forward(x)

Forward pass.

Args:

x: the input of the NN

Returns: the output of the NN

class `TinyCNN`

A very small CNN.

method `init`

__init__(n_classes, act) → None

Create the tiny CNN with two conv layers.

Args:

n_classes: number of classes
act: the activation

method `forward`

forward(x)

Forward the two layers with the chosen activation function.

Args:

x: the input of the NN

Returns: the output of the NN

class `TinyQATCNN`

A very small QAT CNN to classify the sklearn digits dataset.

This class also allows pruning to a maximum of 10 active neurons, which should help keep the accumulator bit-width low.

method `init`

__init__(n_classes, n_bits, n_active, signed, narrow) → None

Construct the CNN with a configurable number of classes.

Args:

n_classes (int): number of outputs of the neural net
n_bits (int): number of weight and activation bits for quantization
n_active (int): number of active (non-zero weight) neurons to keep
signed (bool): whether quantized integer values are signed
narrow (bool): whether the range of quantized integer values is narrow/symmetric

method `forward`

forward(x)

Run inference on the tiny CNN, apply the decision layer on the reshaped conv output.

Args:

x: the input to the NN

Returns: the output of the NN

method `test_torch`

test_torch(test_loader)

Test the network: measure accuracy on the test set.

Args:

test_loader: the test loader

Returns:

res: the number of correctly classified test examples

method `toggle_pruning`

toggle_pruning(enable)

Enable or remove pruning.

Args:

enable: if we enable the pruning or not

class `SimpleQAT`

Torch model implements a step function that needs Greater, Cast and Where.

method `init`

__init__(input_output, activation_function, n_bits=2, disable_bit_check=False)

method `forward`

forward(x)

Forward pass with a quantizer built into the computation graph.

Args:

x: the input of the NN

Returns: the output of the NN

class `QATTestModule`

Torch model that implements a simple non-uniform quantizer.

method `init`

__init__(activation_function)

method `forward`

forward(x)

Forward pass with a quantizer built into the computation graph.

Args:

x: the input of the NN

Returns: the output of the NN

class `SingleMixNet`

Torch model that with a single conv layer that produces the output, e.g. a blur filter.

method `init`

__init__(use_conv, use_qat, inp_size, n_bits)

method `forward`

forward(x)

Execute the single convolution.

Args:

x: the input of the NN

Returns: the output of the NN

class `TorchSum`

Torch model to test the ReduceSum ONNX operator in a leveled circuit.

method `init`

__init__(dim=(0,), keepdim=True)

Initialize the module.

Args:

dim (Tuple[int]): The axis along which the sum should be executed
keepdim (bool): If the output should keep the same dimension as the input or not

method `forward`

forward(x)

Forward pass.

Args:

x (torch.tensor): The input of the model

Returns:

torch_sum (torch.tensor): The sum of the input's tensor elements along the given axis

class `TorchSumMod`

Torch model to test the ReduceSum ONNX operator in a circuit containing a PBS.

method `init`

__init__(dim=(0,), keepdim=True)

Initialize the module.

Args:

dim (Tuple[int]): The axis along which the sum should be executed
keepdim (bool): If the output should keep the same dimension as the input or not

method `forward`

forward(x)

Forward pass.

Args:

x (torch.tensor): The input of the model

Returns:

torch_sum (torch.tensor): The sum of the input's tensor elements along the given axis

API

Modules

concrete.ml.common: Module for shared data structures and code.
concrete.ml.common.check_inputs: Check and conversion tools.
concrete.ml.common.debugging: Module for debugging.
concrete.ml.common.debugging.custom_assert: Provide some variants of assert.
concrete.ml.common.utils: Utils that can be re-used by other pieces of code in the module.
concrete.ml.deployment: Module for deployment of the FHE model.
concrete.ml.deployment.fhe_client_server: APIs for FHE deployment.
concrete.ml.onnx: ONNX module.
concrete.ml.onnx.convert: ONNX conversion related code.
concrete.ml.onnx.onnx_impl_utils: Utility functions for onnx operator implementations.
concrete.ml.onnx.onnx_model_manipulations: Some code to manipulate models.
concrete.ml.onnx.onnx_utils: Utils to interpret an ONNX model with numpy.
concrete.ml.onnx.ops_impl: ONNX ops implementation in python + numpy.
concrete.ml.pytest: Module which is used to contain common functions for pytest.
concrete.ml.pytest.torch_models: Torch modules for our pytests.
concrete.ml.pytest.utils: Common functions or lists for test files, which can't be put in fixtures.
concrete.ml.quantization: Modules for quantization.
concrete.ml.quantization.base_quantized_op: Base Quantized Op class that implements quantization for a float numpy op.
concrete.ml.quantization.post_training: Post Training Quantization methods.
concrete.ml.quantization.quantized_module: QuantizedModule API.
concrete.ml.quantization.quantized_ops: Quantized versions of the ONNX operators for post training quantization.
concrete.ml.quantization.quantizers: Quantization utilities for a numpy array/tensor.
concrete.ml.sklearn: Import sklearn models.
concrete.ml.sklearn.base: Module that contains base classes for our libraries estimators.
concrete.ml.sklearn.glm: Implement sklearn's Generalized Linear Models (GLM).
concrete.ml.sklearn.linear_model: Implement sklearn linear model.
concrete.ml.sklearn.protocols: Protocols.
concrete.ml.sklearn.qnn: Scikit-learn interface for concrete quantized neural networks.
concrete.ml.sklearn.rf: Implements RandomForest models.
concrete.ml.sklearn.svm: Implement Support Vector Machine.
concrete.ml.sklearn.torch_modules: Implement torch module.
concrete.ml.sklearn.tree: Implement the sklearn tree models.
concrete.ml.sklearn.tree_to_numpy: Implements the conversion of a tree model to a numpy function.
concrete.ml.sklearn.xgb: Implements XGBoost models.
concrete.ml.torch: Modules for torch to numpy conversion.
concrete.ml.torch.compile: torch compilation function.
concrete.ml.torch.numpy_module: A torch to numpy module.
concrete.ml.version: File to manage the version of the package.

Classes

fhe_client_server.FHEModelClient: Client API to encrypt and decrypt FHE data.
fhe_client_server.FHEModelDev: Dev API to save the model and then load and run the FHE circuit.
fhe_client_server.FHEModelServer: Server API to load and run the FHE circuit.
ops_impl.ONNXMixedFunction: A mixed quantized-raw valued onnx function.
torch_models.BranchingGemmModule: Torch model with some branching and skip connections.
torch_models.BranchingModule: Torch model with some branching and skip connections.
torch_models.CNN: Torch CNN model for the tests.
torch_models.CNNGrouped: Torch CNN model with grouped convolution for compile torch tests.
torch_models.CNNInvalid: Torch CNN model for the tests.
torch_models.CNNMaxPool: Torch CNN model for the tests with a max pool.
torch_models.CNNOther: Torch CNN model for the tests.
torch_models.FC: Torch model for the tests.
torch_models.FCSeq: Torch model that should generate MatMul->Add ONNX patterns.
torch_models.FCSeqAddBiasVec: Torch model that should generate MatMul->Add ONNX patterns.
torch_models.FCSmall: Torch model for the tests.
torch_models.MultiInputNN: Torch model to test multiple inputs forward.
torch_models.MultiOpOnSingleInputConvNN: Network that applies two quantized operations on a single input.
torch_models.NetWithConcatUnsqueeze: Torch model to test the concat and unsqueeze operators.
torch_models.NetWithLoops: Torch model, where we reuse some elements in a loop.
torch_models.QATTestModule: Torch model that implements a simple non-uniform quantizer.
torch_models.SimpleQAT: Torch model implements a step function that needs Greater, Cast and Where.
torch_models.SingleMixNet: Torch model that with a single conv layer that produces the output, e.g. a blur filter.
torch_models.StepActivationModule: Torch model implements a step function that needs Greater, Cast and Where.
torch_models.TinyCNN: A very small CNN.
torch_models.TinyQATCNN: A very small QAT CNN to classify the sklearn digits dataset.
torch_models.TorchSum: Torch model to test the ReduceSum ONNX operator in a leveled circuit.
torch_models.TorchSumMod: Torch model to test the ReduceSum ONNX operator in a circuit containing a PBS.
torch_models.UnivariateModule: Torch model that calls univariate and shape functions of torch.
base_quantized_op.QuantizedMixingOp: An operator that mixes (adds or multiplies) together encrypted inputs.
base_quantized_op.QuantizedOp: Base class for quantized ONNX ops implemented in numpy.
base_quantized_op.QuantizedOpUnivariateOfEncrypted: An univariate operator of an encrypted value.
post_training.ONNXConverter: Base ONNX to Concrete ML computation graph conversion class.
post_training.PostTrainingAffineQuantization: Post-training Affine Quantization.
post_training.PostTrainingQATImporter: Converter of Quantization Aware Training networks.
quantized_module.QuantizedModule: Inference for a quantized model.
quantized_ops.QuantizedAbs: Quantized Abs op.
quantized_ops.QuantizedAdd: Quantized Addition operator.
quantized_ops.QuantizedAvgPool: Quantized Average Pooling op.
quantized_ops.QuantizedBatchNormalization: Quantized Batch normalization with encrypted input and in-the-clear normalization params.
quantized_ops.QuantizedBrevitasQuant: Brevitas uniform quantization with encrypted input.
quantized_ops.QuantizedCast: Cast the input to the required data type.
quantized_ops.QuantizedCelu: Quantized Celu op.
quantized_ops.QuantizedClip: Quantized clip op.
quantized_ops.QuantizedConcat: Concatenate operator.
quantized_ops.QuantizedConv: Quantized Conv op.
quantized_ops.QuantizedDiv: Div operator /.
quantized_ops.QuantizedElu: Quantized Elu op.
quantized_ops.QuantizedErf: Quantized erf op.
quantized_ops.QuantizedExp: Quantized Exp op.
quantized_ops.QuantizedFlatten: Quantized flatten for encrypted inputs.
quantized_ops.QuantizedFloor: Quantized Floor op.
quantized_ops.QuantizedGemm: Quantized Gemm op.
quantized_ops.QuantizedGreater: Comparison operator >.
quantized_ops.QuantizedGreaterOrEqual: Comparison operator >=.
quantized_ops.QuantizedHardSigmoid: Quantized HardSigmoid op.
quantized_ops.QuantizedHardSwish: Quantized Hardswish op.
quantized_ops.QuantizedIdentity: Quantized Identity op.
quantized_ops.QuantizedLeakyRelu: Quantized LeakyRelu op.
quantized_ops.QuantizedLess: Comparison operator <.
quantized_ops.QuantizedLessOrEqual: Comparison operator <=.
quantized_ops.QuantizedLog: Quantized Log op.
quantized_ops.QuantizedMatMul: Quantized MatMul op.
quantized_ops.QuantizedMax: Quantized Max op.
quantized_ops.QuantizedMaxPool: Quantized Max Pooling op.
quantized_ops.QuantizedMin: Quantized Min op.
quantized_ops.QuantizedMul: Multiplication operator.
quantized_ops.QuantizedNeg: Quantized Neg op.
quantized_ops.QuantizedNot: Quantized Not op.
quantized_ops.QuantizedOr: Or operator ||.
quantized_ops.QuantizedPRelu: Quantized PRelu op.
quantized_ops.QuantizedPad: Quantized Padding op.
quantized_ops.QuantizedPow: Quantized pow op.
quantized_ops.QuantizedReduceSum: ReduceSum with encrypted input.
quantized_ops.QuantizedRelu: Quantized Relu op.
quantized_ops.QuantizedReshape: Quantized Reshape op.
quantized_ops.QuantizedRound: Quantized round op.
quantized_ops.QuantizedSelu: Quantized Selu op.
quantized_ops.QuantizedSigmoid: Quantized sigmoid op.
quantized_ops.QuantizedSign: Quantized Neg op.
quantized_ops.QuantizedSoftplus: Quantized Softplus op.
quantized_ops.QuantizedSub: Subtraction operator.
quantized_ops.QuantizedTanh: Quantized Tanh op.
quantized_ops.QuantizedTranspose: Transpose operator for quantized inputs.
quantized_ops.QuantizedUnsqueeze: Unsqueeze operator.
quantized_ops.QuantizedWhere: Where operator on quantized arrays.
quantizers.MinMaxQuantizationStats: Calibration set statistics.
quantizers.QuantizationOptions: Options for quantization.
quantizers.QuantizedArray: Abstraction of quantized array.
quantizers.UniformQuantizationParameters: Quantization parameters for uniform quantization.
quantizers.UniformQuantizer: Uniform quantizer.
base.BaseTreeClassifierMixin: Mixin class for tree-based classifiers.
base.BaseTreeEstimatorMixin: Mixin class for tree-based estimators.
base.BaseTreeRegressorMixin: Mixin class for tree-based regressors.
base.QuantizedTorchEstimatorMixin: Mixin that provides quantization for a torch module and follows the Estimator API.
base.SklearnLinearClassifierMixin: A Mixin class for sklearn linear classifiers with FHE.
base.SklearnLinearModelMixin: A Mixin class for sklearn linear models with FHE.
glm.GammaRegressor: A Gamma regression model with FHE.
glm.PoissonRegressor: A Poisson regression model with FHE.
glm.TweedieRegressor: A Tweedie regression model with FHE.
linear_model.ElasticNet: An ElasticNet regression model with FHE.
linear_model.Lasso: A Lasso regression model with FHE.
linear_model.LinearRegression: A linear regression model with FHE.
linear_model.LogisticRegression: A logistic regression model with FHE.
linear_model.Ridge: A Ridge regression model with FHE.
protocols.ConcreteBaseClassifierProtocol: Concrete classifier protocol.
protocols.ConcreteBaseEstimatorProtocol: A Concrete Estimator Protocol.
protocols.ConcreteBaseRegressorProtocol: Concrete regressor protocol.
protocols.Quantizer: Quantizer Protocol.
qnn.FixedTypeSkorchNeuralNet: A mixin with a helpful modification to a skorch estimator that fixes the module type.
qnn.NeuralNetClassifier: Scikit-learn interface for quantized FHE compatible neural networks.
qnn.NeuralNetRegressor: Scikit-learn interface for quantized FHE compatible neural networks.
qnn.QuantizedSkorchEstimatorMixin: Mixin class that adds quantization features to Skorch NN estimators.
qnn.SparseQuantNeuralNetImpl: Sparse Quantized Neural Network classifier.
rf.RandomForestClassifier: Implements the RandomForest classifier.
rf.RandomForestRegressor: Implements the RandomForest regressor.
svm.LinearSVC: A Classification Support Vector Machine (SVM).
svm.LinearSVR: A Regression Support Vector Machine (SVM).
tree.DecisionTreeClassifier: Implements the sklearn DecisionTreeClassifier.
tree.DecisionTreeRegressor: Implements the sklearn DecisionTreeClassifier.
tree_to_numpy.Task: Task enumerate.
xgb.XGBClassifier: Implements the XGBoost classifier.
xgb.XGBRegressor: Implements the XGBoost regressor.
numpy_module.NumpyModule: General interface to transform a torch.nn.Module to numpy module.

Functions

check_inputs.check_X_y_and_assert: sklearn.utils.check_X_y with an assert.
check_inputs.check_array_and_assert: sklearn.utils.check_array with an assert.
custom_assert.assert_false: Provide a custom assert to check that the condition is False.
custom_assert.assert_not_reached: Provide a custom assert to check that a piece of code is never reached.
custom_assert.assert_true: Provide a custom assert to check that the condition is True.
utils.check_there_is_no_p_error_options_in_configuration: Check the user did not set p_error or global_p_error in configuration.
utils.generate_proxy_function: Generate a proxy function for a function accepting only *args type arguments.
utils.get_onnx_opset_version: Return the ONNX opset_version.
utils.manage_parameters_for_pbs_errors: Return (p_error, global_p_error) that we want to give to Concrete-Numpy and the compiler.
utils.replace_invalid_arg_name_chars: Sanitize arg_name, replacing invalid chars by _.
convert.get_equivalent_numpy_forward: Get the numpy equivalent forward of the provided ONNX model.
convert.get_equivalent_numpy_forward_and_onnx_model: Get the numpy equivalent forward of the provided torch Module.
onnx_impl_utils.compute_conv_output_dims: Compute the output shape of a pool or conv operation.
onnx_impl_utils.compute_onnx_pool_padding: Compute any additional padding needed to compute pooling layers.
onnx_impl_utils.numpy_onnx_pad: Pad a tensor according to ONNX spec, using an optional custom pad value.
onnx_impl_utils.onnx_avgpool_compute_norm_const: Compute the average pooling normalization constant.
onnx_model_manipulations.clean_graph_after_node_name: Clean the graph of the onnx model by removing nodes after the given node name.
onnx_model_manipulations.clean_graph_after_node_op_type: Clean the graph of the onnx model by removing nodes after the given node type.
onnx_model_manipulations.keep_following_outputs_discard_others: Keep the outputs given in outputs_to_keep and remove the others from the model.
onnx_model_manipulations.remove_identity_nodes: Remove identity nodes from a model.
onnx_model_manipulations.remove_node_types: Remove unnecessary nodes from the ONNX graph.
onnx_model_manipulations.remove_unused_constant_nodes: Remove unused Constant nodes in the provided onnx model.
onnx_model_manipulations.simplify_onnx_model: Simplify an ONNX model, removes unused Constant nodes and Identity nodes.
onnx_utils.execute_onnx_with_numpy: Execute the provided ONNX graph on the given inputs.
onnx_utils.get_attribute: Get the attribute from an ONNX AttributeProto.
onnx_utils.get_op_type: Construct the qualified type name of the ONNX operator.
onnx_utils.remove_initializer_from_input: Remove initializers from model inputs.
ops_impl.cast_to_float: Cast values to floating points.
ops_impl.numpy_abs: Compute abs in numpy according to ONNX spec.
ops_impl.numpy_acos: Compute acos in numpy according to ONNX spec.
ops_impl.numpy_acosh: Compute acosh in numpy according to ONNX spec.
ops_impl.numpy_add: Compute add in numpy according to ONNX spec.
ops_impl.numpy_asin: Compute asin in numpy according to ONNX spec.
ops_impl.numpy_asinh: Compute sinh in numpy according to ONNX spec.
ops_impl.numpy_atan: Compute atan in numpy according to ONNX spec.
ops_impl.numpy_atanh: Compute atanh in numpy according to ONNX spec.
ops_impl.numpy_avgpool: Compute Average Pooling using Torch.
ops_impl.numpy_batchnorm: Compute the batch normalization of the input tensor.
ops_impl.numpy_cast: Execute ONNX cast in Numpy.
ops_impl.numpy_celu: Compute celu in numpy according to ONNX spec.
ops_impl.numpy_concatenate: Apply concatenate in numpy according to ONNX spec.
ops_impl.numpy_constant: Return the constant passed as a kwarg.
ops_impl.numpy_cos: Compute cos in numpy according to ONNX spec.
ops_impl.numpy_cosh: Compute cosh in numpy according to ONNX spec.
ops_impl.numpy_div: Compute div in numpy according to ONNX spec.
ops_impl.numpy_elu: Compute elu in numpy according to ONNX spec.
ops_impl.numpy_equal: Compute equal in numpy according to ONNX spec.
ops_impl.numpy_erf: Compute erf in numpy according to ONNX spec.
ops_impl.numpy_exp: Compute exponential in numpy according to ONNX spec.
ops_impl.numpy_flatten: Flatten a tensor into a 2d array.
ops_impl.numpy_floor: Compute Floor in numpy according to ONNX spec.
ops_impl.numpy_greater: Compute greater in numpy according to ONNX spec.
ops_impl.numpy_greater_float: Compute greater in numpy according to ONNX spec and cast outputs to floats.
ops_impl.numpy_greater_or_equal: Compute greater or equal in numpy according to ONNX spec.
ops_impl.numpy_greater_or_equal_float: Compute greater or equal in numpy according to ONNX specs and cast outputs to floats.
ops_impl.numpy_hardsigmoid: Compute hardsigmoid in numpy according to ONNX spec.
ops_impl.numpy_hardswish: Compute hardswish in numpy according to ONNX spec.
ops_impl.numpy_identity: Compute identity in numpy according to ONNX spec.
ops_impl.numpy_leakyrelu: Compute leakyrelu in numpy according to ONNX spec.
ops_impl.numpy_less: Compute less in numpy according to ONNX spec.
ops_impl.numpy_less_float: Compute less in numpy according to ONNX spec and cast outputs to floats.
ops_impl.numpy_less_or_equal: Compute less or equal in numpy according to ONNX spec.
ops_impl.numpy_less_or_equal_float: Compute less or equal in numpy according to ONNX spec and cast outputs to floats.
ops_impl.numpy_log: Compute log in numpy according to ONNX spec.
ops_impl.numpy_matmul: Compute matmul in numpy according to ONNX spec.
ops_impl.numpy_max: Compute Max in numpy according to ONNX spec.
ops_impl.numpy_maxpool: Compute Max Pooling using Torch.
ops_impl.numpy_min: Compute Min in numpy according to ONNX spec.
ops_impl.numpy_mul: Compute mul in numpy according to ONNX spec.
ops_impl.numpy_neg: Compute Negative in numpy according to ONNX spec.
ops_impl.numpy_not: Compute not in numpy according to ONNX spec.
ops_impl.numpy_not_float: Compute not in numpy according to ONNX spec and cast outputs to floats.
ops_impl.numpy_or: Compute or in numpy according to ONNX spec.
ops_impl.numpy_or_float: Compute or in numpy according to ONNX spec and cast outputs to floats.
ops_impl.numpy_pow: Compute pow in numpy according to ONNX spec.
ops_impl.numpy_relu: Compute relu in numpy according to ONNX spec.
ops_impl.numpy_round: Compute round in numpy according to ONNX spec.
ops_impl.numpy_selu: Compute selu in numpy according to ONNX spec.
ops_impl.numpy_sigmoid: Compute sigmoid in numpy according to ONNX spec.
ops_impl.numpy_sign: Compute Sign in numpy according to ONNX spec.
ops_impl.numpy_sin: Compute sin in numpy according to ONNX spec.
ops_impl.numpy_sinh: Compute sinh in numpy according to ONNX spec.
ops_impl.numpy_softmax: Compute softmax in numpy according to ONNX spec.
ops_impl.numpy_softplus: Compute softplus in numpy according to ONNX spec.
ops_impl.numpy_sub: Compute sub in numpy according to ONNX spec.
ops_impl.numpy_tan: Compute tan in numpy according to ONNX spec.
ops_impl.numpy_tanh: Compute tanh in numpy according to ONNX spec.
ops_impl.numpy_thresholdedrelu: Compute thresholdedrelu in numpy according to ONNX spec.
ops_impl.numpy_transpose: Transpose in numpy according to ONNX spec.
ops_impl.numpy_where: Compute the equivalent of numpy.where.
ops_impl.numpy_where_body: Compute the equivalent of numpy.where.
ops_impl.onnx_func_raw_args: Decorate a numpy onnx function to flag the raw/non quantized inputs.
utils.sanitize_test_and_train_datasets: Sanitize datasets depending on the model type.
post_training.get_n_bits_dict: Convert the n_bits parameter into a proper dictionary.
quantizers.fill_from_kwargs: Fill a parameter set structure from kwargs parameters.
base.get_sklearn_linear_models: Return the list of available linear models in Concrete-ML.
base.get_sklearn_models: Return the list of available models in Concrete-ML.
base.get_sklearn_neural_net_models: Return the list of available neural net models in Concrete-ML.
base.get_sklearn_tree_models: Return the list of available tree models in Concrete-ML.
tree_to_numpy.tree_to_numpy: Convert the tree inference to a numpy functions using Hummingbird.
compile.compile_brevitas_qat_model: Compile a Brevitas Quantization Aware Training model.
compile.compile_onnx_model: Compile a torch module into a FHE equivalent.
compile.compile_torch_model: Compile a torch module into a FHE equivalent.
compile.convert_torch_tensor_or_numpy_array_to_numpy_array: Convert a torch tensor or a numpy array to a numpy array.

concrete.ml.sklearn.base.md

module `concrete.ml.sklearn.base`

Module that contains base classes for our libraries estimators.

Global Variables

OPSET_VERSION_FOR_ONNX_EXPORT

function `get_sklearn_models`

get_sklearn_models()

Return the list of available models in Concrete-ML.

Returns: the lists of models in Concrete-ML

function `get_sklearn_linear_models`

get_sklearn_linear_models(
    classifier: bool = True,
    regressor: bool = True,
    str_in_class_name: str = None
)

Return the list of available linear models in Concrete-ML.

Args:

classifier (bool): whether you want classifiers or not
regressor (bool): whether you want regressors or not
str_in_class_name (str): if not None, only return models with this as a substring in the class name

Returns: the lists of linear models in Concrete-ML

function `get_sklearn_tree_models`

get_sklearn_tree_models(
    classifier: bool = True,
    regressor: bool = True,
    str_in_class_name: str = None
)

Return the list of available tree models in Concrete-ML.

Args:

classifier (bool): whether you want classifiers or not
regressor (bool): whether you want regressors or not
str_in_class_name (str): if not None, only return models with this as a substring in the class name

Returns: the lists of tree models in Concrete-ML

function `get_sklearn_neural_net_models`

get_sklearn_neural_net_models(
    classifier: bool = True,
    regressor: bool = True,
    str_in_class_name: str = None
)

Return the list of available neural net models in Concrete-ML.

Args:

classifier (bool): whether you want classifiers or not
regressor (bool): whether you want regressors or not
str_in_class_name (str): if not None, only return models with this as a substring in the class name

Returns: the lists of neural net models in Concrete-ML

class `QuantizedTorchEstimatorMixin`

Mixin that provides quantization for a torch module and follows the Estimator API.

This class should be mixed in with another that provides the full Estimator API. This class only provides modifiers for .fit() (with quantization) and .predict() (optionally in FHE)

method `init`

__init__()

property base_estimator_type

Get the sklearn estimator that should be trained by the child class.

property base_module_to_compile

Get the Torch module that should be compiled to FHE.

property fhe_circuit

Get the FHE circuit.

Returns:

Circuit: the FHE circuit

property input_quantizers

Get the input quantizers.

Returns:

List[Quantizer]: the input quantizers

property n_bits_quant

Get the number of quantization bits.

property onnx_model

Get the ONNX model.

.. # noqa: DAR201

Returns:

_onnx_model_ (onnx.ModelProto): the ONNX model

property output_quantizers

Get the input quantizers.

Returns:

List[QuantizedArray]: the input quantizers

property quantize_input

Get the input quantization function.

Returns:

Callable : function that quantizes the input

method `compile`

compile(
    X: ndarray,
    configuration: Optional[Configuration] = None,
    compilation_artifacts: Optional[DebugArtifacts] = None,
    show_mlir: bool = False,
    use_virtual_lib: bool = False,
    p_error: Optional[float] = None,
    global_p_error: Optional[float] = None,
    verbose_compilation: bool = False
) → Circuit

Compile the model.

Args:

X (numpy.ndarray): the dequantized dataset
configuration (Optional[Configuration]): the options for compilation
compilation_artifacts (Optional[DebugArtifacts]): artifacts object to fill during compilation
show_mlir (bool): whether or not to show MLIR during the compilation
use_virtual_lib (bool): whether to compile using the virtual library that allows higher bitwidths
p_error (Optional[float]): probability of error of a single PBS
global_p_error (Optional[float]): probability of error of the full circuit. Not simulated by the VL, i.e., taken as 0
verbose_compilation (bool): whether to show compilation information

Returns:

Circuit: the compiled Circuit.

Raises:

ValueError: if called before the model is trained

method `fit`

fit(X, y, **fit_params)

Initialize and fit the module.

If the module was already initialized, by calling fit, the module will be re-initialized (unless warm_start is True). In addition to the torch training step, this method performs quantization of the trained torch model.

Args:

X : training data By default, you should be able to pass: * numpy arrays * torch tensors * pandas DataFrame or Series
y (numpy.ndarray): labels associated with training data
**fit_params: additional parameters that can be used during training, these are passed to the torch training interface

Returns:

self: the trained quantized estimator

method `fit_benchmark`

fit_benchmark(X: ndarray, y: ndarray, *args, **kwargs) → Tuple[Any, Any]

Fit the quantized estimator as well as its equivalent float estimator.

This function returns both the quantized estimator (itself) as well as its non-quantized (float) equivalent, which are both trained separately. This is useful in order to compare performances between quantized and fp32 versions.

Args:

X : The training data By default, you should be able to pass: * numpy arrays * torch tensors * pandas DataFrame or Series
y (numpy.ndarray): The labels associated with the training data
*args: The arguments to pass to the sklearn linear model.
**kwargs: The keyword arguments to pass to the sklearn linear model.

Returns:

self: The trained quantized estimator
fp32_model: The trained float equivalent estimator

method `get_params_for_benchmark`

get_params_for_benchmark()

Get the parameters to instantiate the sklearn estimator trained by the child class.

Returns:

params (dict): dictionary with parameters that will initialize a new Estimator

method `post_processing`

post_processing(y_preds: ndarray) → ndarray

Post-processing the output.

Args:

y_preds (numpy.ndarray): the output to post-process

Raises:

ValueError: if unknown post-processing function

Returns:

numpy.ndarray: the post-processed output

method `predict`

predict(X, execute_in_fhe=False)

Predict on user provided data.

Predicts using the quantized clear or FHE classifier

Args:

X : input data, a numpy array of raw values (non quantized)
execute_in_fhe : whether to execute the inference in FHE or in the clear

Returns:

y_pred : numpy ndarray with predictions

method `predict_proba`

predict_proba(X, execute_in_fhe=False)

Predict on user provided data, returning probabilities.

Predicts using the quantized clear or FHE classifier

Args:

X : input data, a numpy array of raw values (non quantized)
execute_in_fhe : whether to execute the inference in FHE or in the clear

Returns:

y_pred : numpy ndarray with probabilities (if applicable)

Raises:

ValueError: if the estimator was not yet trained or compiled

class `BaseTreeEstimatorMixin`

Mixin class for tree-based estimators.

A place to share methods that are used on all tree-based estimators.

method `init`

__init__(n_bits: int)

Initialize the TreeBasedEstimatorMixin.

Args:

n_bits (int): number of bits used for quantization

property onnx_model

Get the ONNX model.

.. # noqa: DAR201

Returns:

onnx.ModelProto: the ONNX model

method `compile`

compile(
    X: ndarray,
    configuration: Optional[Configuration] = None,
    compilation_artifacts: Optional[DebugArtifacts] = None,
    show_mlir: bool = False,
    use_virtual_lib: bool = False,
    p_error: Optional[float] = None,
    global_p_error: Optional[float] = None,
    verbose_compilation: bool = False
) → Circuit

Compile the model.

Args:

X (numpy.ndarray): the dequantized dataset
configuration (Optional[Configuration]): the options for compilation
compilation_artifacts (Optional[DebugArtifacts]): artifacts object to fill during compilation
show_mlir (bool): whether or not to show MLIR during the compilation
use_virtual_lib (bool): set to True to use the so called virtual lib simulating FHE computation. Defaults to False
p_error (Optional[float]): probability of error of a single PBS
global_p_error (Optional[float]): probability of error of the full circuit. Not simulated by the VL, i.e., taken as 0
verbose_compilation (bool): whether to show compilation information

Returns:

Circuit: the compiled Circuit.

method `dequantize_output`

dequantize_output(y_preds: ndarray)

Dequantize the integer predictions.

Args:

y_preds (numpy.ndarray): the predictions

Returns: the dequantized predictions

method `fit_benchmark`

fit_benchmark(
    X: ndarray,
    y: ndarray,
    *args,
    random_state: Optional[int] = None,
    **kwargs
) → Tuple[Any, Any]

Fit the sklearn tree-based model and the FHE tree-based model.

Args:

X (numpy.ndarray): The input data.
y (numpy.ndarray): The target data. random_state (Optional[Union[int, numpy.random.RandomState, None]]): The random state. Defaults to None.
*args: args for super().fit
**kwargs: kwargs for super().fit

Returns: Tuple[ConcreteEstimators, SklearnEstimators]: The FHE and sklearn tree-based models.

method `quantize_input`

quantize_input(X: ndarray)

Quantize the input.

Args:

X (numpy.ndarray): the input

Returns: the quantized input

class `BaseTreeRegressorMixin`

Mixin class for tree-based regressors.

A place to share methods that are used on all tree-based regressors.

method `init`

__init__(n_bits: int)

Initialize the TreeBasedEstimatorMixin.

Args:

n_bits (int): number of bits used for quantization

property onnx_model

Get the ONNX model.

.. # noqa: DAR201

Returns:

onnx.ModelProto: the ONNX model

method `compile`

compile(
    X: ndarray,
    configuration: Optional[Configuration] = None,
    compilation_artifacts: Optional[DebugArtifacts] = None,
    show_mlir: bool = False,
    use_virtual_lib: bool = False,
    p_error: Optional[float] = None,
    global_p_error: Optional[float] = None,
    verbose_compilation: bool = False
) → Circuit

Compile the model.

Args:

X (numpy.ndarray): the dequantized dataset
configuration (Optional[Configuration]): the options for compilation
compilation_artifacts (Optional[DebugArtifacts]): artifacts object to fill during compilation
show_mlir (bool): whether or not to show MLIR during the compilation
use_virtual_lib (bool): set to True to use the so called virtual lib simulating FHE computation. Defaults to False
p_error (Optional[float]): probability of error of a single PBS
global_p_error (Optional[float]): probability of error of the full circuit. Not simulated by the VL, i.e., taken as 0
verbose_compilation (bool): whether to show compilation information

Returns:

Circuit: the compiled Circuit.

method `dequantize_output`

dequantize_output(y_preds: ndarray)

Dequantize the integer predictions.

Args:

y_preds (numpy.ndarray): the predictions

Returns: the dequantized predictions

method `fit`

fit(X, y: ndarray, **kwargs) → Any

Fit the tree-based estimator.

Args:

X : training data By default, you should be able to pass: * numpy arrays * torch tensors * pandas DataFrame or Series
y (numpy.ndarray): The target data.
**kwargs: args for super().fit

Returns:

Any: The fitted model.

method `fit_benchmark`

fit_benchmark(
    X: ndarray,
    y: ndarray,
    *args,
    random_state: Optional[int] = None,
    **kwargs
) → Tuple[Any, Any]

Fit the sklearn tree-based model and the FHE tree-based model.

Args:

X (numpy.ndarray): The input data.
y (numpy.ndarray): The target data. random_state (Optional[Union[int, numpy.random.RandomState, None]]): The random state. Defaults to None.
*args: args for super().fit
**kwargs: kwargs for super().fit

Returns: Tuple[ConcreteEstimators, SklearnEstimators]: The FHE and sklearn tree-based models.

method `post_processing`

post_processing(y_preds: ndarray) → ndarray

Apply post-processing to the predictions.

Args:

y_preds (numpy.ndarray): The predictions.

Returns:

numpy.ndarray: The post-processed predictions.

method `predict`

predict(X: ndarray, execute_in_fhe: bool = False) → ndarray

Predict the probability.

Args:

X (numpy.ndarray): The input data.
execute_in_fhe (bool): Whether to execute in FHE. Defaults to False.

Returns:

numpy.ndarray: The predicted probabilities.

method `quantize_input`

quantize_input(X: ndarray)

Quantize the input.

Args:

X (numpy.ndarray): the input

Returns: the quantized input

class `BaseTreeClassifierMixin`

Mixin class for tree-based classifiers.

A place to share methods that are used on all tree-based classifiers.

method `init`

__init__(n_bits: int)

Initialize the TreeBasedEstimatorMixin.

Args:

n_bits (int): number of bits used for quantization

property onnx_model

Get the ONNX model.

.. # noqa: DAR201

Returns:

onnx.ModelProto: the ONNX model

method `compile`

compile(
    X: ndarray,
    configuration: Optional[Configuration] = None,
    compilation_artifacts: Optional[DebugArtifacts] = None,
    show_mlir: bool = False,
    use_virtual_lib: bool = False,
    p_error: Optional[float] = None,
    global_p_error: Optional[float] = None,
    verbose_compilation: bool = False
) → Circuit

Compile the model.

Args:

X (numpy.ndarray): the dequantized dataset
configuration (Optional[Configuration]): the options for compilation
compilation_artifacts (Optional[DebugArtifacts]): artifacts object to fill during compilation
show_mlir (bool): whether or not to show MLIR during the compilation
use_virtual_lib (bool): set to True to use the so called virtual lib simulating FHE computation. Defaults to False
p_error (Optional[float]): probability of error of a single PBS
global_p_error (Optional[float]): probability of error of the full circuit. Not simulated by the VL, i.e., taken as 0
verbose_compilation (bool): whether to show compilation information

Returns:

Circuit: the compiled Circuit.

method `dequantize_output`

dequantize_output(y_preds: ndarray)

Dequantize the integer predictions.

Args:

y_preds (numpy.ndarray): the predictions

Returns: the dequantized predictions

method `fit`

fit(X, y: ndarray, **kwargs) → Any

Fit the tree-based estimator.

Args:

X : training data By default, you should be able to pass: * numpy arrays * torch tensors * pandas DataFrame or Series
y (numpy.ndarray): The target data.
**kwargs: args for super().fit

Returns:

Any: The fitted model.

method `fit_benchmark`

fit_benchmark(
    X: ndarray,
    y: ndarray,
    *args,
    random_state: Optional[int] = None,
    **kwargs
) → Tuple[Any, Any]

Fit the sklearn tree-based model and the FHE tree-based model.

Args:

X (numpy.ndarray): The input data.
y (numpy.ndarray): The target data. random_state (Optional[Union[int, numpy.random.RandomState, None]]): The random state. Defaults to None.
*args: args for super().fit
**kwargs: kwargs for super().fit

Returns: Tuple[ConcreteEstimators, SklearnEstimators]: The FHE and sklearn tree-based models.

method `post_processing`

post_processing(y_preds: ndarray) → ndarray

Apply post-processing to the predictions.

Args:

y_preds (numpy.ndarray): The predictions.

Returns:

numpy.ndarray: The post-processed predictions.

method `predict`

predict(X: ndarray, execute_in_fhe: bool = False) → ndarray

Predict the class with highest probability.

Args:

X (numpy.ndarray): The input data.
execute_in_fhe (bool): Whether to execute in FHE. Defaults to False.

Returns:

numpy.ndarray: The predicted target values.

method `predict_proba`

predict_proba(X: ndarray, execute_in_fhe: bool = False) → ndarray

Predict the probability.

Args:

X (numpy.ndarray): The input data.
execute_in_fhe (bool): Whether to execute in FHE. Defaults to False.

Returns:

numpy.ndarray: The predicted probabilities.

method `quantize_input`

quantize_input(X: ndarray)

Quantize the input.

Args:

X (numpy.ndarray): the input

Returns: the quantized input

class `SklearnLinearModelMixin`

A Mixin class for sklearn linear models with FHE.

method `init`

__init__(*args, n_bits: Union[int, Dict[str, int]] = 8, **kwargs)

Initialize the FHE linear model.

Args:

n_bits (int, Dict[str, int]): Number of bits to quantize the model. If an int is passed for n_bits, the value will be used for quantizing inputs and weights. If a dict is passed, then it should contain "op_inputs" and "op_weights" as keys with corresponding number of quantization bits so that: - op_inputs : number of bits to quantize the input values - op_weights: number of bits to quantize the learned parameters Default to 8.
*args: The arguments to pass to the sklearn linear model.
**kwargs: The keyword arguments to pass to the sklearn linear model.

method `clean_graph`

clean_graph()

Clean the graph of the onnx model.

This will remove the Cast node in the model's onnx.graph since they have no use in quantized or FHE models.

method `compile`

compile(
    X: ndarray,
    configuration: Optional[Configuration] = None,
    compilation_artifacts: Optional[DebugArtifacts] = None,
    show_mlir: bool = False,
    use_virtual_lib: bool = False,
    p_error: Optional[float] = None,
    global_p_error: Optional[float] = None,
    verbose_compilation: bool = False
) → Circuit

Compile the FHE linear model.

Args:

X (numpy.ndarray): The input data.
configuration (Optional[Configuration]): Configuration object to use during compilation
compilation_artifacts (Optional[DebugArtifacts]): Artifacts object to fill during compilation
show_mlir (bool): If set, the MLIR produced by the converter and which is going to be sent to the compiler backend is shown on the screen, e.g., for debugging or demo. Defaults to False.
use_virtual_lib (bool): Whether to compile using the virtual library that allows higher bitwidths with simulated FHE computation. Defaults to False
p_error (Optional[float]): Probability of error of a single PBS
global_p_error (Optional[float]): probability of error of the full circuit. Not simulated by the VL, i.e., taken as 0
verbose_compilation (bool): whether to show compilation information

Returns:

Circuit: The compiled Circuit.

method `dequantize_output`

dequantize_output(q_y_preds: ndarray) → ndarray

Dequantize the output.

Args:

q_y_preds (numpy.ndarray): The quantized output to dequantize

Returns:

numpy.ndarray: The dequantized output

method `fit`

fit(X, y: ndarray, *args, **kwargs) → Any

Fit the FHE linear model.

Args:

X : Training data By default, you should be able to pass: * numpy arrays * torch tensors * pandas DataFrame or Series
y (numpy.ndarray): The target data.
*args: The arguments to pass to the sklearn linear model.
**kwargs: The keyword arguments to pass to the sklearn linear model.

Returns: Any

method `fit_benchmark`

fit_benchmark(
    X: ndarray,
    y: ndarray,
    *args,
    random_state: Optional[int] = None,
    **kwargs
) → Tuple[Any, Any]

Fit the sklearn linear model and the FHE linear model.

Args:

X (numpy.ndarray): The input data.
y (numpy.ndarray): The target data. random_state (Optional[Union[int, numpy.random.RandomState, None]]): The random state. Defaults to None.
*args: The arguments to pass to the sklearn linear model. or not (False). Default to False.
*args: Arguments for super().fit
**kwargs: Keyword arguments for super().fit

Returns: Tuple[SklearnLinearModelMixin, sklearn.linear_model.LinearRegression]: The FHE and sklearn LinearRegression.

method `post_processing`

post_processing(y_preds: ndarray) → ndarray

Post-processing the quantized output.

For linear models, post-processing only considers a dequantization step.

Args:

y_preds (numpy.ndarray): The quantized outputs to post-process

Returns:

numpy.ndarray: The post-processed output

method `predict`

predict(X: ndarray, execute_in_fhe: bool = False) → ndarray

Predict on user data.

Predict on user data using either the quantized clear model, implemented with tensors, or, if execute_in_fhe is set, using the compiled FHE circuit

Args:

X (numpy.ndarray): The input data
execute_in_fhe (bool): Whether to execute the inference in FHE

Returns:

numpy.ndarray: The prediction as ordinals

method `quantize_input`

quantize_input(X: ndarray)

Quantize the input.

Args:

X (numpy.ndarray): The input to quantize

Returns:

numpy.ndarray: The quantized input

class `SklearnLinearClassifierMixin`

A Mixin class for sklearn linear classifiers with FHE.

method `init`

__init__(*args, n_bits: Union[int, Dict[str, int]] = 8, **kwargs)

Initialize the FHE linear model.

Args:

n_bits (int, Dict[str, int]): Number of bits to quantize the model. If an int is passed for n_bits, the value will be used for quantizing inputs and weights. If a dict is passed, then it should contain "op_inputs" and "op_weights" as keys with corresponding number of quantization bits so that: - op_inputs : number of bits to quantize the input values - op_weights: number of bits to quantize the learned parameters Default to 8.
*args: The arguments to pass to the sklearn linear model.
**kwargs: The keyword arguments to pass to the sklearn linear model.

method `clean_graph`

clean_graph()

Clean the graph of the onnx model.

Any operators following gemm, including the sigmoid, softmax and argmax operators, are removed from the graph. They will be executed in clear in the post-processing method.

method `compile`

compile(
    X: ndarray,
    configuration: Optional[Configuration] = None,
    compilation_artifacts: Optional[DebugArtifacts] = None,
    show_mlir: bool = False,
    use_virtual_lib: bool = False,
    p_error: Optional[float] = None,
    global_p_error: Optional[float] = None,
    verbose_compilation: bool = False
) → Circuit

Compile the FHE linear model.

Args:

X (numpy.ndarray): The input data.
configuration (Optional[Configuration]): Configuration object to use during compilation
compilation_artifacts (Optional[DebugArtifacts]): Artifacts object to fill during compilation
show_mlir (bool): If set, the MLIR produced by the converter and which is going to be sent to the compiler backend is shown on the screen, e.g., for debugging or demo. Defaults to False.
use_virtual_lib (bool): Whether to compile using the virtual library that allows higher bitwidths with simulated FHE computation. Defaults to False
p_error (Optional[float]): Probability of error of a single PBS
global_p_error (Optional[float]): probability of error of the full circuit. Not simulated by the VL, i.e., taken as 0
verbose_compilation (bool): whether to show compilation information

Returns:

Circuit: The compiled Circuit.

method `decision_function`

decision_function(X: ndarray, execute_in_fhe: bool = False) → ndarray

Predict confidence scores for samples.

Args:

X (numpy.ndarray): Samples to predict.
execute_in_fhe (bool): If True, the inference will be executed in FHE. Default to False.

Returns:

numpy.ndarray: Confidence scores for samples.

method `dequantize_output`

dequantize_output(q_y_preds: ndarray) → ndarray

Dequantize the output.

Args:

q_y_preds (numpy.ndarray): The quantized output to dequantize

Returns:

numpy.ndarray: The dequantized output

method `fit`

fit(X, y: ndarray, *args, **kwargs) → Any

Fit the FHE linear model.

Args:

X : Training data By default, you should be able to pass: * numpy arrays * torch tensors * pandas DataFrame or Series
y (numpy.ndarray): The target data.
*args: The arguments to pass to the sklearn linear model.
**kwargs: The keyword arguments to pass to the sklearn linear model.

Returns: Any

method `fit_benchmark`

fit_benchmark(
    X: ndarray,
    y: ndarray,
    *args,
    random_state: Optional[int] = None,
    **kwargs
) → Tuple[Any, Any]

Fit the sklearn linear model and the FHE linear model.

Args:

X (numpy.ndarray): The input data.
y (numpy.ndarray): The target data. random_state (Optional[Union[int, numpy.random.RandomState, None]]): The random state. Defaults to None.
*args: The arguments to pass to the sklearn linear model. or not (False). Default to False.
*args: Arguments for super().fit
**kwargs: Keyword arguments for super().fit

Returns: Tuple[SklearnLinearModelMixin, sklearn.linear_model.LinearRegression]: The FHE and sklearn LinearRegression.

method `post_processing`

post_processing(y_preds: ndarray, already_dequantized: bool = False)

Post-processing the predictions.

This step may include a dequantization of the inputs if not done previously, in particular within the client-server workflow.

Args:

y_preds (numpy.ndarray): The predictions to post-process.
already_dequantized (bool): Whether the inputs were already dequantized or not. Default to False.

Returns:

numpy.ndarray: The post-processed predictions.

method `predict`

predict(X: ndarray, execute_in_fhe: bool = False) → ndarray

Predict on user data.

Predict on user data using either the quantized clear model, implemented with tensors, or, if execute_in_fhe is set, using the compiled FHE circuit.

Args:

X (numpy.ndarray): Samples to predict.
execute_in_fhe (bool): If True, the inference will be executed in FHE. Default to False.

Returns:

numpy.ndarray: The prediction as ordinals.

method `predict_proba`

predict_proba(X: ndarray, execute_in_fhe: bool = False) → ndarray

Predict class probabilities for samples.

Args:

X (numpy.ndarray): Samples to predict.
execute_in_fhe (bool): If True, the inference will be executed in FHE. Default to False.

Returns:

numpy.ndarray: Class probabilities for samples.

method `quantize_input`

quantize_input(X: ndarray)

Quantize the input.

Args:

X (numpy.ndarray): The input to quantize

Returns:

numpy.ndarray: The quantized input

concrete.ml.quantization.quantized_ops.md

module `concrete.ml.quantization.quantized_ops`

Quantized versions of the ONNX operators for post training quantization.

class `QuantizedSigmoid`

Quantized sigmoid op.

property int_input_names

Get the names of encrypted integer tensors that are used by this op.

Returns:

List[str]: the names of the tensors

class `QuantizedHardSigmoid`

Quantized HardSigmoid op.

property int_input_names

Get the names of encrypted integer tensors that are used by this op.

Returns:

List[str]: the names of the tensors

class `QuantizedRelu`

Quantized Relu op.

property int_input_names

Get the names of encrypted integer tensors that are used by this op.

Returns:

List[str]: the names of the tensors

class `QuantizedPRelu`

Quantized PRelu op.

property int_input_names

Get the names of encrypted integer tensors that are used by this op.

Returns:

List[str]: the names of the tensors

class `QuantizedLeakyRelu`

Quantized LeakyRelu op.

property int_input_names

Get the names of encrypted integer tensors that are used by this op.

Returns:

List[str]: the names of the tensors

class `QuantizedHardSwish`

Quantized Hardswish op.

property int_input_names

Get the names of encrypted integer tensors that are used by this op.

Returns:

List[str]: the names of the tensors

class `QuantizedElu`

Quantized Elu op.

property int_input_names

Get the names of encrypted integer tensors that are used by this op.

Returns:

List[str]: the names of the tensors

class `QuantizedSelu`

Quantized Selu op.

property int_input_names

Get the names of encrypted integer tensors that are used by this op.

Returns:

List[str]: the names of the tensors

class `QuantizedCelu`

Quantized Celu op.

property int_input_names

Get the names of encrypted integer tensors that are used by this op.

Returns:

List[str]: the names of the tensors

class `QuantizedClip`

Quantized clip op.

property int_input_names

Get the names of encrypted integer tensors that are used by this op.

Returns:

List[str]: the names of the tensors

class `QuantizedRound`

Quantized round op.

property int_input_names

Get the names of encrypted integer tensors that are used by this op.

Returns:

List[str]: the names of the tensors

class `QuantizedPow`

Quantized pow op.

Only works for a float constant power. This operation will be fused to a (potentially larger) TLU.

property int_input_names

Get the names of encrypted integer tensors that are used by this op.

Returns:

List[str]: the names of the tensors

class `QuantizedGemm`

Quantized Gemm op.

method `init`

__init__(
    n_bits_output: int,
    int_input_names: Set[str] = None,
    constant_inputs: Optional[Dict[str, Any], Dict[int, Any]] = None,
    input_quant_opts: QuantizationOptions = None,
    **attrs
) → None

property int_input_names

Get the names of encrypted integer tensors that are used by this op.

Returns:

List[str]: the names of the tensors

method `q_impl`

q_impl(*q_inputs: QuantizedArray, **attrs) → QuantizedArray

class `QuantizedMatMul`

Quantized MatMul op.

method `init`

__init__(
    n_bits_output: int,
    int_input_names: Set[str] = None,
    constant_inputs: Optional[Dict[str, Any], Dict[int, Any]] = None,
    input_quant_opts: QuantizationOptions = None,
    **attrs
) → None

property int_input_names

Get the names of encrypted integer tensors that are used by this op.

Returns:

List[str]: the names of the tensors

method `q_impl`

q_impl(*q_inputs: QuantizedArray, **attrs) → QuantizedArray

class `QuantizedAdd`

Quantized Addition operator.

Can add either two variables (both encrypted) or a variable and a constant

property int_input_names

Get the names of encrypted integer tensors that are used by this op.

Returns:

List[str]: the names of the tensors

method `can_fuse`

can_fuse() → bool

Determine if this op can be fused.

Add operation can be computed in float and fused if it operates over inputs produced by a single integer tensor. For example the expression x + x * 1.75, where x is an encrypted tensor, can be computed with a single TLU.

Returns:

bool: Whether the number of integer input tensors allows computing this op as a TLU

method `q_impl`

q_impl(*q_inputs: QuantizedArray, **attrs) → QuantizedArray

class `QuantizedTanh`

Quantized Tanh op.

property int_input_names

Get the names of encrypted integer tensors that are used by this op.

Returns:

List[str]: the names of the tensors

class `QuantizedSoftplus`

Quantized Softplus op.

property int_input_names

Get the names of encrypted integer tensors that are used by this op.

Returns:

List[str]: the names of the tensors

class `QuantizedExp`

Quantized Exp op.

property int_input_names

Get the names of encrypted integer tensors that are used by this op.

Returns:

List[str]: the names of the tensors

class `QuantizedLog`

Quantized Log op.

property int_input_names

Get the names of encrypted integer tensors that are used by this op.

Returns:

List[str]: the names of the tensors

class `QuantizedAbs`

Quantized Abs op.

property int_input_names

Get the names of encrypted integer tensors that are used by this op.

Returns:

List[str]: the names of the tensors

class `QuantizedIdentity`

Quantized Identity op.

property int_input_names

Get the names of encrypted integer tensors that are used by this op.

Returns:

List[str]: the names of the tensors

method `q_impl`

q_impl(*q_inputs: QuantizedArray, **attrs) → QuantizedArray

class `QuantizedReshape`

Quantized Reshape op.

property int_input_names

Get the names of encrypted integer tensors that are used by this op.

Returns:

List[str]: the names of the tensors

method `q_impl`

q_impl(*q_inputs: QuantizedArray, **attrs) → QuantizedArray

Reshape the input integer encrypted tensor.

Args:

q_inputs: an encrypted integer tensor at index 0 and one constant shape at index 1
attrs: additional optional reshape options

Returns:

result (QuantizedArray): reshaped encrypted integer tensor

class `QuantizedConv`

Quantized Conv op.

method `init`

__init__(
    n_bits_output: int,
    int_input_names: Set[str] = None,
    constant_inputs: Optional[Dict[str, Any], Dict[int, Any]] = None,
    input_quant_opts: QuantizationOptions = None,
    **attrs
) → None

Construct the quantized convolution operator and retrieve parameters.

Args:

n_bits_output: number of bits for the quantization of the outputs of this operator
int_input_names: names of integer tensors that are taken as input for this operation
constant_inputs: the weights and activations
input_quant_opts: options for the input quantizer
attrs: convolution options
dilations (Tuple[int]): dilation of the kernel. Default to 1 on all dimensions.
group (int): number of convolution groups. Default to 1.
kernel_shape (Tuple[int]): shape of the kernel. Should have 2 elements for 2d conv
pads (Tuple[int]): padding in ONNX format (begin, end) on each axis
strides (Tuple[int]): stride of the convolution on each axis

property int_input_names

Get the names of encrypted integer tensors that are used by this op.

Returns:

List[str]: the names of the tensors

method `q_impl`

q_impl(*q_inputs: QuantizedArray, **attrs) → QuantizedArray

Compute the quantized convolution between two quantized tensors.

Allows an optional quantized bias.

Args:

q_inputs: input tuple, contains
x (numpy.ndarray): input data. Shape is N x C x H x W for 2d
w (numpy.ndarray): weights tensor. Shape is (O x I x Kh x Kw) for 2d
b (numpy.ndarray, Optional): bias tensor, Shape is (O,)
attrs: convolution options handled in constructor

Returns:

res (QuantizedArray): result of the quantized integer convolution

class `QuantizedAvgPool`

Quantized Average Pooling op.

method `init`

__init__(
    n_bits_output: int,
    int_input_names: Set[str] = None,
    constant_inputs: Optional[Dict[str, Any], Dict[int, Any]] = None,
    input_quant_opts: QuantizationOptions = None,
    **attrs
) → None

property int_input_names

Get the names of encrypted integer tensors that are used by this op.

Returns:

List[str]: the names of the tensors

method `q_impl`

q_impl(*q_inputs: QuantizedArray, **attrs) → QuantizedArray

class `QuantizedMaxPool`

Quantized Max Pooling op.

method `init`

__init__(
    n_bits_output: int,
    int_input_names: Set[str] = None,
    constant_inputs: Optional[Dict[str, Any], Dict[int, Any]] = None,
    input_quant_opts: QuantizationOptions = None,
    **attrs
) → None

property int_input_names

Get the names of encrypted integer tensors that are used by this op.

Returns:

List[str]: the names of the tensors

method `can_fuse`

can_fuse() → bool

Determine if this op can be fused.

Max Pooling operation can not be fused since it must be performed over integer tensors and it combines different elements of the input tensors.

Returns:

bool: False, this operation can not be fused as it adds different encrypted integers

method `q_impl`

q_impl(*q_inputs: QuantizedArray, **attrs) → QuantizedArray

class `QuantizedPad`

Quantized Padding op.

method `init`

__init__(
    n_bits_output: int,
    int_input_names: Set[str] = None,
    constant_inputs: Optional[Dict[str, Any], Dict[int, Any]] = None,
    input_quant_opts: QuantizationOptions = None,
    **attrs
) → None

property int_input_names

Get the names of encrypted integer tensors that are used by this op.

Returns:

List[str]: the names of the tensors

method `can_fuse`

can_fuse() → bool

Determine if this op can be fused.

Pad operation cannot be fused since it must be performed over integer tensors.

Returns:

bool: False, this operation cannot be fused as it is manipulates integer tensors

class `QuantizedWhere`

Where operator on quantized arrays.

Supports only constants for the results produced on the True/False branches.

method `init`

__init__(
    n_bits_output: int,
    int_input_names: Set[str] = None,
    constant_inputs: Optional[Dict[str, Any], Dict[int, Any]] = None,
    input_quant_opts: QuantizationOptions = None,
    **attrs
) → None

property int_input_names

Get the names of encrypted integer tensors that are used by this op.

Returns:

List[str]: the names of the tensors

class `QuantizedCast`

Cast the input to the required data type.

In FHE we only support a limited number of output types. Booleans are cast to integers.

property int_input_names

Get the names of encrypted integer tensors that are used by this op.

Returns:

List[str]: the names of the tensors

class `QuantizedGreater`

Comparison operator >.

Only supports comparison with a constant.

method `init`

__init__(
    n_bits_output: int,
    int_input_names: Set[str] = None,
    constant_inputs: Optional[Dict[str, Any], Dict[int, Any]] = None,
    input_quant_opts: QuantizationOptions = None,
    **attrs
) → None

property int_input_names

Get the names of encrypted integer tensors that are used by this op.

Returns:

List[str]: the names of the tensors

class `QuantizedGreaterOrEqual`

Comparison operator >=.

Only supports comparison with a constant.

method `init`

__init__(
    n_bits_output: int,
    int_input_names: Set[str] = None,
    constant_inputs: Optional[Dict[str, Any], Dict[int, Any]] = None,
    input_quant_opts: QuantizationOptions = None,
    **attrs
) → None

property int_input_names

Get the names of encrypted integer tensors that are used by this op.

Returns:

List[str]: the names of the tensors

class `QuantizedLess`

Comparison operator <.

Only supports comparison with a constant.

method `init`

__init__(
    n_bits_output: int,
    int_input_names: Set[str] = None,
    constant_inputs: Optional[Dict[str, Any], Dict[int, Any]] = None,
    input_quant_opts: QuantizationOptions = None,
    **attrs
) → None

property int_input_names

Get the names of encrypted integer tensors that are used by this op.

Returns:

List[str]: the names of the tensors

class `QuantizedLessOrEqual`

Comparison operator <=.

Only supports comparison with a constant.

method `init`

__init__(
    n_bits_output: int,
    int_input_names: Set[str] = None,
    constant_inputs: Optional[Dict[str, Any], Dict[int, Any]] = None,
    input_quant_opts: QuantizationOptions = None,
    **attrs
) → None

property int_input_names

Get the names of encrypted integer tensors that are used by this op.

Returns:

List[str]: the names of the tensors

class `QuantizedOr`

Or operator ||.

This operation is not really working as a quantized operation. It just works when things got fused, as in e.g. Act(x) = x || (x + 42))

property int_input_names

Get the names of encrypted integer tensors that are used by this op.

Returns:

List[str]: the names of the tensors

class `QuantizedDiv`

Div operator /.

This operation is not really working as a quantized operation. It just works when things got fused, as in e.g. Act(x) = 1000 / (x + 42))

property int_input_names

Get the names of encrypted integer tensors that are used by this op.

Returns:

List[str]: the names of the tensors

class `QuantizedMul`

Multiplication operator.

Only multiplies an encrypted tensor with a float constant for now. This operation will be fused to a (potentially larger) TLU.

property int_input_names

Get the names of encrypted integer tensors that are used by this op.

Returns:

List[str]: the names of the tensors

class `QuantizedSub`

Subtraction operator.

This works the same as addition on both encrypted - encrypted and on encrypted - constant.

property int_input_names

Get the names of encrypted integer tensors that are used by this op.

Returns:

List[str]: the names of the tensors

method `can_fuse`

can_fuse() → bool

Determine if this op can be fused.

Returns:

bool: Whether the number of integer input tensors allows computing this op as a TLU

method `q_impl`

q_impl(*q_inputs: QuantizedArray, **attrs) → QuantizedArray

class `QuantizedBatchNormalization`

Quantized Batch normalization with encrypted input and in-the-clear normalization params.

property int_input_names

Get the names of encrypted integer tensors that are used by this op.

Returns:

List[str]: the names of the tensors

class `QuantizedFlatten`

Quantized flatten for encrypted inputs.

property int_input_names

Get the names of encrypted integer tensors that are used by this op.

Returns:

List[str]: the names of the tensors

method `can_fuse`

can_fuse() → bool

Determine if this op can be fused.

Flatten operation cannot be fused since it must be performed over integer tensors.

Returns:

bool: False, this operation cannot be fused as it is manipulates integer tensors.

method `q_impl`

q_impl(*q_inputs: QuantizedArray, **attrs) → QuantizedArray

Flatten the input integer encrypted tensor.

Args:

q_inputs: an encrypted integer tensor at index 0
attrs: contains axis attribute

Returns:

result (QuantizedArray): reshaped encrypted integer tensor

class `QuantizedReduceSum`

ReduceSum with encrypted input.

method `init`

__init__(
    n_bits_output: int,
    int_input_names: Set[str] = None,
    constant_inputs: Optional[Dict[str, Any], Dict[int, Any]] = None,
    input_quant_opts: Optional[QuantizationOptions] = None,
    **attrs
) → None

Construct the quantized ReduceSum operator and retrieve parameters.

Args:

n_bits_output (int): Number of bits for the operator's quantization of outputs.
int_input_names (Optional[Set[str]]): Names of input integer tensors. Default to None.
constant_inputs (Optional[Dict]): Input constant tensor.
axes (Optional[numpy.ndarray]): Array of integers along which to reduce. The default is to reduce over all the dimensions of the input tensor if 'noop_with_empty_axes' is false, else act as an Identity op when 'noop_with_empty_axes' is true. Accepted range is [-r, r-1] where r = rank(data). Default to None.
input_quant_opts (Optional[QuantizationOptions]): Options for the input quantizer. Default to None.
attrs (dict): RecuseSum options.
keepdims (int): Keep the reduced dimension or not, 1 means keeping the input dimension, 0 will reduce it along the given axis. Default to 1.
noop_with_empty_axes (int): Defines behavior if 'axes' is empty or set to None. Default behavior with 0 is to reduce all axes. When axes is empty and this attribute is set to true 1, input tensor will not be reduced, and the output tensor would be equivalent to input tensor. Default to 0.

property int_input_names

Get the names of encrypted integer tensors that are used by this op.

Returns:

List[str]: the names of the tensors

method `calibrate`

calibrate(*inputs: ndarray) → ndarray

Create corresponding QuantizedArray for the output of the activation function.

Args:

*inputs (numpy.ndarray): Calibration sample inputs.

Returns:

numpy.ndarray: The output values for the provided calibration samples.

method `q_impl`

q_impl(*q_inputs: QuantizedArray, **attrs) → QuantizedArray

Sum the encrypted tensor's values along the given axes.

Args:

q_inputs (QuantizedArray): An encrypted integer tensor at index 0.
attrs (Dict): Options are handled in constructor.

Returns:

(QuantizedArray): The sum of all values along the given axes.

class `QuantizedErf`

Quantized erf op.

property int_input_names

Get the names of encrypted integer tensors that are used by this op.

Returns:

List[str]: the names of the tensors

class `QuantizedNot`

Quantized Not op.

property int_input_names

Get the names of encrypted integer tensors that are used by this op.

Returns:

List[str]: the names of the tensors

class `QuantizedBrevitasQuant`

Brevitas uniform quantization with encrypted input.

method `init`

__init__(
    n_bits_output: int,
    int_input_names: Set[str] = None,
    constant_inputs: Optional[Dict[str, Any], Dict[int, Any]] = None,
    input_quant_opts: Optional[QuantizationOptions] = None,
    **attrs
) → None

Construct the Brevitas quantization operator.

Args:

n_bits_output (int): Number of bits for the operator's quantization of outputs. Not used, will be overridden by the bit_width in ONNX
int_input_names (Optional[Set[str]]): Names of input integer tensors. Default to None.
constant_inputs (Optional[Dict]): Input constant tensor.
scale (float): Quantizer scale
zero_point (float): Quantizer zero-point
bit_width (int): Number of bits of the integer representation
input_quant_opts (Optional[QuantizationOptions]): Options for the input quantizer. Default to None. attrs (dict):
rounding_mode (str): Rounding mode (default and only accepted option is "ROUND")
signed (int): Whether this op quantizes to signed integers (default 1),
narrow (int): Whether this op quantizes to a narrow range of integers e.g. [-2n_bits-1 .. 2n_bits-1] (default 0),

property int_input_names

Get the names of encrypted integer tensors that are used by this op.

Returns:

List[str]: the names of the tensors

method `calibrate`

calibrate(*inputs: ndarray) → ndarray

Create corresponding QuantizedArray for the output of Quantization function.

Args:

*inputs (numpy.ndarray): Calibration sample inputs.

Returns:

numpy.ndarray: the output values for the provided calibration samples.

method `q_impl`

q_impl(*q_inputs: QuantizedArray, **attrs) → QuantizedArray

Quantize values.

Args:

q_inputs: an encrypted integer tensor at index 0 and one constant shape at index 1
attrs: additional optional reshape options

Returns:

result (QuantizedArray): reshaped encrypted integer tensor

class `QuantizedTranspose`

Transpose operator for quantized inputs.

This operator performs quantization, transposes the encrypted data, then dequantizes again.

property int_input_names

Get the names of encrypted integer tensors that are used by this op.

Returns:

List[str]: the names of the tensors

method `q_impl`

q_impl(*q_inputs: QuantizedArray, **attrs) → QuantizedArray

Reshape the input integer encrypted tensor.

Args:

q_inputs: an encrypted integer tensor at index 0 and one constant shape at index 1
attrs: additional optional reshape options

Returns:

result (QuantizedArray): reshaped encrypted integer tensor

class `QuantizedFloor`

Quantized Floor op.

property int_input_names

Get the names of encrypted integer tensors that are used by this op.

Returns:

List[str]: the names of the tensors

class `QuantizedMax`

Quantized Max op.

property int_input_names

Get the names of encrypted integer tensors that are used by this op.

Returns:

List[str]: the names of the tensors

class `QuantizedMin`

Quantized Min op.

property int_input_names

Get the names of encrypted integer tensors that are used by this op.

Returns:

List[str]: the names of the tensors

class `QuantizedNeg`

Quantized Neg op.

property int_input_names

Get the names of encrypted integer tensors that are used by this op.

Returns:

List[str]: the names of the tensors

class `QuantizedSign`

Quantized Neg op.

property int_input_names

Get the names of encrypted integer tensors that are used by this op.

Returns:

List[str]: the names of the tensors

class `QuantizedUnsqueeze`

Unsqueeze operator.

property int_input_names

Get the names of encrypted integer tensors that are used by this op.

Returns:

List[str]: the names of the tensors

method `q_impl`

q_impl(*q_inputs: QuantizedArray, **attrs) → QuantizedArray

Unsqueeze the input tensors on a given axis.

Args:

q_inputs: an encrypted integer tensor
attrs: additional optional unsqueeze options

Returns:

result (QuantizedArray): unsqueezed encrypted integer tensor

class `QuantizedConcat`

Concatenate operator.

property int_input_names

Get the names of encrypted integer tensors that are used by this op.

Returns:

List[str]: the names of the tensors

method `q_impl`

q_impl(*q_inputs: QuantizedArray, **attrs) → QuantizedArray

Concatenate the input tensors on a giver axis.

Args:

q_inputs: an encrypted integer tensor
attrs: additional optional concatenate options

Returns:

result (QuantizedArray): concatenated encrypted integer tensor

concrete.ml.onnx.ops_impl.md

module `concrete.ml.onnx.ops_impl`

ONNX ops implementation in python + numpy.

function `cast_to_float`

cast_to_float(inputs)

Cast values to floating points.

Args:

inputs (Tuple[numpy.ndarray]): The values to consider.

Returns:

Tuple[numpy.ndarray]: The float values.

function `onnx_func_raw_args`

onnx_func_raw_args(*args)

Decorate a numpy onnx function to flag the raw/non quantized inputs.

Args:

*args (tuple[Any]): function argument names

Returns:

result (ONNXMixedFunction): wrapped numpy function with a list of mixed arguments

function `numpy_where_body`

numpy_where_body(c: ndarray, t: ndarray, f: Union[ndarray, int]) → ndarray

Compute the equivalent of numpy.where.

This function is not mapped to any ONNX operator (as opposed to numpy_where). It is usable by functions which are mapped to ONNX operators, e.g. numpy_div or numpy_where.

Args:

c (numpy.ndarray): Condition operand.
t (numpy.ndarray): True operand.
f (numpy.ndarray): False operand.

Returns:

numpy.ndarray: numpy.where(c, t, f)

function `numpy_where`

numpy_where(c: ndarray, t: ndarray, f: ndarray) → Tuple[ndarray]

Compute the equivalent of numpy.where.

Args:

c (numpy.ndarray): Condition operand.
t (numpy.ndarray): True operand.
f (numpy.ndarray): False operand.

Returns:

numpy.ndarray: numpy.where(c, t, f)

function `numpy_add`

numpy_add(a: ndarray, b: ndarray) → Tuple[ndarray]

Compute add in numpy according to ONNX spec.

See https://github.com/onnx/onnx/blob/main/docs/Changelog.md#Add-13

Args:

a (numpy.ndarray): First operand.
b (numpy.ndarray): Second operand.

Returns:

Tuple[numpy.ndarray]: Result, has same element type as two inputs

function `numpy_constant`

numpy_constant(**kwargs)

Return the constant passed as a kwarg.

See https://github.com/onnx/onnx/blob/main/docs/Changelog.md#Constant-13

Args:

**kwargs: keyword arguments

Returns:

Any: The stored constant.

function `numpy_matmul`

numpy_matmul(a: ndarray, b: ndarray) → Tuple[ndarray]

Compute matmul in numpy according to ONNX spec.

See https://github.com/onnx/onnx/blob/main/docs/Changelog.md#MatMul-13

Args:

a (numpy.ndarray): N-dimensional matrix A
b (numpy.ndarray): N-dimensional matrix B

Returns:

Tuple[numpy.ndarray]: Matrix multiply results from A * B

function `numpy_relu`

numpy_relu(x: ndarray) → Tuple[ndarray]

Compute relu in numpy according to ONNX spec.

See https://github.com/onnx/onnx/blob/main/docs/Changelog.md#Relu-14

Args:

x (numpy.ndarray): Input tensor

Returns:

Tuple[numpy.ndarray]: Output tensor

function `numpy_sigmoid`

numpy_sigmoid(x: ndarray) → Tuple[ndarray]

Compute sigmoid in numpy according to ONNX spec.

See https://github.com/onnx/onnx/blob/main/docs/Changelog.md#Sigmoid-13

Args:

x (numpy.ndarray): Input tensor

Returns:

Tuple[numpy.ndarray]: Output tensor

function `numpy_softmax`

numpy_softmax(x, axis=1, keepdims=True)

Compute softmax in numpy according to ONNX spec.

Softmax is currently not supported in FHE.

See https://github.com/onnx/onnx/blob/main/docs/Changelog.md#softmax-13

Args:

x (numpy.ndarray): Input tensor
axis (None, int, tuple of int): Axis or axes along which a softmax's sum is performed. If None, it will sum all of the elements of the input array. If axis is negative it counts from the last to the first axis. Default to 1.
keepdims (bool): If True, the axes which are reduced along the sum are left in the result as dimensions with size one. Default to True.

Returns:

Tuple[numpy.ndarray]: Output tensor

function `numpy_cos`

numpy_cos(x: ndarray) → Tuple[ndarray]

Compute cos in numpy according to ONNX spec.

See https://github.com/onnx/onnx/blob/main/docs/Changelog.md#Cos-7

Args:

x (numpy.ndarray): Input tensor

Returns:

Tuple[numpy.ndarray]: Output tensor

function `numpy_cosh`

numpy_cosh(x: ndarray) → Tuple[ndarray]

Compute cosh in numpy according to ONNX spec.

See https://github.com/onnx/onnx/blob/main/docs/Changelog.md#Cosh-9

Args:

x (numpy.ndarray): Input tensor

Returns:

Tuple[numpy.ndarray]: Output tensor

function `numpy_sin`

numpy_sin(x: ndarray) → Tuple[ndarray]

Compute sin in numpy according to ONNX spec.

See https://github.com/onnx/onnx/blob/main/docs/Changelog.md#Sin-7

Args:

x (numpy.ndarray): Input tensor

Returns:

Tuple[numpy.ndarray]: Output tensor

function `numpy_sinh`

numpy_sinh(x: ndarray) → Tuple[ndarray]

Compute sinh in numpy according to ONNX spec.

See https://github.com/onnx/onnx/blob/main/docs/Changelog.md#Sinh-9

Args:

x (numpy.ndarray): Input tensor

Returns:

Tuple[numpy.ndarray]: Output tensor

function `numpy_tan`

numpy_tan(x: ndarray) → Tuple[ndarray]

Compute tan in numpy according to ONNX spec.

See https://github.com/onnx/onnx/blob/main/docs/Changelog.md#Tan-7

Args:

x (numpy.ndarray): Input tensor

Returns:

Tuple[numpy.ndarray]: Output tensor

function `numpy_tanh`

numpy_tanh(x: ndarray) → Tuple[ndarray]

Compute tanh in numpy according to ONNX spec.

See https://github.com/onnx/onnx/blob/main/docs/Changelog.md#Tanh-13

Args:

x (numpy.ndarray): Input tensor

Returns:

Tuple[numpy.ndarray]: Output tensor

function `numpy_acos`

numpy_acos(x: ndarray) → Tuple[ndarray]

Compute acos in numpy according to ONNX spec.

See https://github.com/onnx/onnx/blob/main/docs/Changelog.md#Acos-7

Args:

x (numpy.ndarray): Input tensor

Returns:

Tuple[numpy.ndarray]: Output tensor

function `numpy_acosh`

numpy_acosh(x: ndarray) → Tuple[ndarray]

Compute acosh in numpy according to ONNX spec.

See https://github.com/onnx/onnx/blob/main/docs/Changelog.md#Acosh-9

Args:

x (numpy.ndarray): Input tensor

Returns:

Tuple[numpy.ndarray]: Output tensor

function `numpy_asin`

numpy_asin(x: ndarray) → Tuple[ndarray]

Compute asin in numpy according to ONNX spec.

See https://github.com/onnx/onnx/blob/main/docs/Changelog.md#Asin-7

Args:

x (numpy.ndarray): Input tensor

Returns:

Tuple[numpy.ndarray]: Output tensor

function `numpy_asinh`

numpy_asinh(x: ndarray) → Tuple[ndarray]

Compute sinh in numpy according to ONNX spec.

See https://github.com/onnx/onnx/blob/main/docs/Changelog.md#Asinh-9

Args:

x (numpy.ndarray): Input tensor

Returns:

Tuple[numpy.ndarray]: Output tensor

function `numpy_atan`

numpy_atan(x: ndarray) → Tuple[ndarray]

Compute atan in numpy according to ONNX spec.

See https://github.com/onnx/onnx/blob/main/docs/Changelog.md#Atan-7

Args:

x (numpy.ndarray): Input tensor

Returns:

Tuple[numpy.ndarray]: Output tensor

function `numpy_atanh`

numpy_atanh(x: ndarray) → Tuple[ndarray]

Compute atanh in numpy according to ONNX spec.

See https://github.com/onnx/onnx/blob/main/docs/Changelog.md#Atanh-9

Args:

x (numpy.ndarray): Input tensor

Returns:

Tuple[numpy.ndarray]: Output tensor

function `numpy_elu`

numpy_elu(x: ndarray, alpha: float = 1) → Tuple[ndarray]

Compute elu in numpy according to ONNX spec.

See https://github.com/onnx/onnx/blob/main/docs/Changelog.md#Elu-6

Args:

x (numpy.ndarray): Input tensor
alpha (float): Coefficient

Returns:

Tuple[numpy.ndarray]: Output tensor

function `numpy_selu`

numpy_selu(
    x: ndarray,
    alpha: float = 1.6732632423543772,
    gamma: float = 1.0507009873554805
) → Tuple[ndarray]

Compute selu in numpy according to ONNX spec.

See https://github.com/onnx/onnx/blob/main/docs/Changelog.md#Selu-6

Args:

x (numpy.ndarray): Input tensor
alpha (float): Coefficient
gamma (float): Coefficient

Returns:

Tuple[numpy.ndarray]: Output tensor

function `numpy_celu`

numpy_celu(x: ndarray, alpha: float = 1) → Tuple[ndarray]

Compute celu in numpy according to ONNX spec.

See https://github.com/onnx/onnx/blob/main/docs/Changelog.md#Celu-12

Args:

x (numpy.ndarray): Input tensor
alpha (float): Coefficient

Returns:

Tuple[numpy.ndarray]: Output tensor

function `numpy_leakyrelu`

numpy_leakyrelu(x: ndarray, alpha: float = 0.01) → Tuple[ndarray]

Compute leakyrelu in numpy according to ONNX spec.

See https://github.com/onnx/onnx/blob/main/docs/Changelog.md#LeakyRelu-6

Args:

x (numpy.ndarray): Input tensor
alpha (float): Coefficient

Returns:

Tuple[numpy.ndarray]: Output tensor

function `numpy_thresholdedrelu`

numpy_thresholdedrelu(x: ndarray, alpha: float = 1) → Tuple[ndarray]

Compute thresholdedrelu in numpy according to ONNX spec.

See https://github.com/onnx/onnx/blob/main/docs/Changelog.md#ThresholdedRelu-10

Args:

x (numpy.ndarray): Input tensor
alpha (float): Coefficient

Returns:

Tuple[numpy.ndarray]: Output tensor

function `numpy_hardsigmoid`

numpy_hardsigmoid(
    x: ndarray,
    alpha: float = 0.2,
    beta: float = 0.5
) → Tuple[ndarray]

Compute hardsigmoid in numpy according to ONNX spec.

See https://github.com/onnx/onnx/blob/main/docs/Changelog.md#HardSigmoid-6

Args:

x (numpy.ndarray): Input tensor
alpha (float): Coefficient
beta (float): Coefficient

Returns:

Tuple[numpy.ndarray]: Output tensor

function `numpy_softplus`

numpy_softplus(x: ndarray) → Tuple[ndarray]

Compute softplus in numpy according to ONNX spec.

See https://github.com/onnx/onnx/blob/main/docs/Changelog.md#Softplus-1

Args:

x (numpy.ndarray): Input tensor

Returns:

Tuple[numpy.ndarray]: Output tensor

function `numpy_abs`

numpy_abs(x: ndarray) → Tuple[ndarray]

Compute abs in numpy according to ONNX spec.

See https://github.com/onnx/onnx/blob/main/docs/Changelog.md#Abs-13

Args:

x (numpy.ndarray): Input tensor

Returns:

Tuple[numpy.ndarray]: Output tensor

function `numpy_div`

numpy_div(a: ndarray, b: ndarray) → Tuple[ndarray]

Compute div in numpy according to ONNX spec.

See https://github.com/onnx/onnx/blob/main/docs/Changelog.md#Div-14

Args:

a (numpy.ndarray): Input tensor
b (numpy.ndarray): Input tensor

Returns:

Tuple[numpy.ndarray]: Output tensor

function `numpy_mul`

numpy_mul(a: ndarray, b: ndarray) → Tuple[ndarray]

Compute mul in numpy according to ONNX spec.

See https://github.com/onnx/onnx/blob/main/docs/Changelog.md#Mul-14

Args:

a (numpy.ndarray): Input tensor
b (numpy.ndarray): Input tensor

Returns:

Tuple[numpy.ndarray]: Output tensor

function `numpy_sub`

numpy_sub(a: ndarray, b: ndarray) → Tuple[ndarray]

Compute sub in numpy according to ONNX spec.

See https://github.com/onnx/onnx/blob/main/docs/Changelog.md#Sub-14

Args:

a (numpy.ndarray): Input tensor
b (numpy.ndarray): Input tensor

Returns:

Tuple[numpy.ndarray]: Output tensor

function `numpy_log`

numpy_log(x: ndarray) → Tuple[ndarray]

Compute log in numpy according to ONNX spec.

See https://github.com/onnx/onnx/blob/main/docs/Changelog.md#Log-13

Args:

x (numpy.ndarray): Input tensor

Returns:

Tuple[numpy.ndarray]: Output tensor

function `numpy_erf`

numpy_erf(x: ndarray) → Tuple[ndarray]

Compute erf in numpy according to ONNX spec.

See https://github.com/onnx/onnx/blob/main/docs/Changelog.md#Erf-13

Args:

x (numpy.ndarray): Input tensor

Returns:

Tuple[numpy.ndarray]: Output tensor

function `numpy_hardswish`

numpy_hardswish(x: ndarray) → Tuple[ndarray]

Compute hardswish in numpy according to ONNX spec.

See https://github.com/onnx/onnx/blob/main/docs/Changelog.md#hardswish-14

Args:

x (numpy.ndarray): Input tensor

Returns:

Tuple[numpy.ndarray]: Output tensor

function `numpy_exp`

numpy_exp(x: ndarray) → Tuple[ndarray]

Compute exponential in numpy according to ONNX spec.

See https://github.com/onnx/onnx/blob/main/docs/Changelog.md#Exp-13

Args:

x (numpy.ndarray): Input tensor

Returns:

Tuple[numpy.ndarray]: The exponential of the input tensor computed element-wise

function `numpy_equal`

numpy_equal(x: ndarray, y: ndarray) → Tuple[ndarray]

Compute equal in numpy according to ONNX spec.

See https://github.com/onnx/onnx/blob/main/docs/Changelog.md#Equal-11

Args:

x (numpy.ndarray): Input tensor
y (numpy.ndarray): Input tensor

Returns:

Tuple[numpy.ndarray]: Output tensor

function `numpy_not`

numpy_not(x: ndarray) → Tuple[ndarray]

Compute not in numpy according to ONNX spec.

See https://github.com/onnx/onnx/blob/main/docs/Changelog.md#Not-1

Args:

x (numpy.ndarray): Input tensor

Returns:

Tuple[numpy.ndarray]: Output tensor

function `numpy_not_float`

numpy_not_float(x: ndarray) → Tuple[ndarray]

Compute not in numpy according to ONNX spec and cast outputs to floats.

See https://github.com/onnx/onnx/blob/main/docs/Changelog.md#Not-1

Args:

x (numpy.ndarray): Input tensor

Returns:

Tuple[numpy.ndarray]: Output tensor

function `numpy_greater`

numpy_greater(x: ndarray, y: ndarray) → Tuple[ndarray]

Compute greater in numpy according to ONNX spec.

See https://github.com/onnx/onnx/blob/main/docs/Changelog.md#Greater-13

Args:

x (numpy.ndarray): Input tensor
y (numpy.ndarray): Input tensor

Returns:

Tuple[numpy.ndarray]: Output tensor

function `numpy_greater_float`

numpy_greater_float(x: ndarray, y: ndarray) → Tuple[ndarray]

Compute greater in numpy according to ONNX spec and cast outputs to floats.

See https://github.com/onnx/onnx/blob/main/docs/Changelog.md#Greater-13

Args:

x (numpy.ndarray): Input tensor
y (numpy.ndarray): Input tensor

Returns:

Tuple[numpy.ndarray]: Output tensor

function `numpy_greater_or_equal`

numpy_greater_or_equal(x: ndarray, y: ndarray) → Tuple[ndarray]

Compute greater or equal in numpy according to ONNX spec.

See https://github.com/onnx/onnx/blob/main/docs/Changelog.md#GreaterOrEqual-12

Args:

x (numpy.ndarray): Input tensor
y (numpy.ndarray): Input tensor

Returns:

Tuple[numpy.ndarray]: Output tensor

function `numpy_greater_or_equal_float`

numpy_greater_or_equal_float(x: ndarray, y: ndarray) → Tuple[ndarray]

Compute greater or equal in numpy according to ONNX specs and cast outputs to floats.

See https://github.com/onnx/onnx/blob/main/docs/Changelog.md#GreaterOrEqual-12

Args:

x (numpy.ndarray): Input tensor
y (numpy.ndarray): Input tensor

Returns:

Tuple[numpy.ndarray]: Output tensor

function `numpy_less`

numpy_less(x: ndarray, y: ndarray) → Tuple[ndarray]

Compute less in numpy according to ONNX spec.

See https://github.com/onnx/onnx/blob/main/docs/Changelog.md#Less-13

Args:

x (numpy.ndarray): Input tensor
y (numpy.ndarray): Input tensor

Returns:

Tuple[numpy.ndarray]: Output tensor

function `numpy_less_float`

numpy_less_float(x: ndarray, y: ndarray) → Tuple[ndarray]

Compute less in numpy according to ONNX spec and cast outputs to floats.

See https://github.com/onnx/onnx/blob/main/docs/Changelog.md#Less-13

Args:

x (numpy.ndarray): Input tensor
y (numpy.ndarray): Input tensor

Returns:

Tuple[numpy.ndarray]: Output tensor

function `numpy_less_or_equal`

numpy_less_or_equal(x: ndarray, y: ndarray) → Tuple[ndarray]

Compute less or equal in numpy according to ONNX spec.

See https://github.com/onnx/onnx/blob/main/docs/Changelog.md#LessOrEqual-12

Args:

x (numpy.ndarray): Input tensor
y (numpy.ndarray): Input tensor

Returns:

Tuple[numpy.ndarray]: Output tensor

function `numpy_less_or_equal_float`

numpy_less_or_equal_float(x: ndarray, y: ndarray) → Tuple[ndarray]

Compute less or equal in numpy according to ONNX spec and cast outputs to floats.

See https://github.com/onnx/onnx/blob/main/docs/Changelog.md#LessOrEqual-12

Args:

x (numpy.ndarray): Input tensor
y (numpy.ndarray): Input tensor

Returns:

Tuple[numpy.ndarray]: Output tensor

function `numpy_identity`

numpy_identity(x: ndarray) → Tuple[ndarray]

Compute identity in numpy according to ONNX spec.

See https://github.com/onnx/onnx/blob/main/docs/Changelog.md#Identity-14

Args:

x (numpy.ndarray): Input tensor

Returns:

Tuple[numpy.ndarray]: Output tensor

function `numpy_transpose`

numpy_transpose(x: ndarray, perm=None) → Tuple[ndarray]

Transpose in numpy according to ONNX spec.

See https://github.com/onnx/onnx/blob/main/docs/Changelog.md#Transpose-13

Args:

x (numpy.ndarray): Input tensor
perm (numpy.ndarray): Permutation of the axes

Returns:

Tuple[numpy.ndarray]: Output tensor

function `numpy_avgpool`

numpy_avgpool(
    x: ndarray,
    ceil_mode: int,
    kernel_shape: Tuple[int, ],
    pads: Tuple[int, ] = None,
    strides: Tuple[int, ] = None
) → Tuple[ndarray]

Compute Average Pooling using Torch.

Currently supports 2d average pooling with torch semantics. This function is ONNX compatible.

See: https://github.com/onnx/onnx/blob/main/docs/Operators.md#AveragePool

Args:

x (numpy.ndarray): input data (many dtypes are supported). Shape is N x C x H x W for 2d
ceil_mode (int): ONNX rounding parameter, expected 0 (torch style dimension computation)
kernel_shape (Tuple[int, ...]): shape of the kernel. Should have 2 elements for 2d conv
pads (Tuple[int, ...]): padding in ONNX format (begin, end) on each axis
strides (Tuple[int, ...]): stride of the convolution on each axis

Returns:

res (numpy.ndarray): a tensor of size (N x InChannels x OutHeight x OutWidth).
See https: //pytorch.org/docs/stable/generated/torch.nn.AvgPool2d.html

Raises:

AssertionError: if the pooling arguments are wrong

function `numpy_maxpool`

numpy_maxpool(
    x: ndarray,
    kernel_shape: Tuple[int, ],
    strides: Tuple[int, ] = None,
    auto_pad: str = 'NOTSET',
    pads: Tuple[int, ] = None,
    dilations: Optional[Tuple[int, ], List[int]] = None,
    ceil_mode: int = 0,
    storage_order: int = 0
) → Tuple[ndarray]

Compute Max Pooling using Torch.

Currently supports 2d max pooling with torch semantics. This function is ONNX compatible.

See: https://github.com/onnx/onnx/blob/main/docs/Operators.md#MaxPool

Args:

x (numpy.ndarray): the input
kernel_shape (Union[Tuple[int, ...], List[int]]): shape of the kernel
strides (Optional[Union[Tuple[int, ...], List[int]]]): stride along each spatial axis set to 1 along each spatial axis if not set
auto_pad (str): padding strategy, default = "NOTSET"
pads (Optional[Union[Tuple[int, ...], List[int]]]): padding for the beginning and ending along each spatial axis (D1_begin, D2_begin, ..., D1_end, D2_end, ...) set to 0 along each spatial axis if not set
dilations (Optional[Union[Tuple[int, ...], List[int]]]): dilation along each spatial axis set to 1 along each spatial axis if not set
ceil_mode (int): ceiling mode, default = 1
storage_order (int): storage order, 0 for row major, 1 for column major, default = 0

Returns:

res (numpy.ndarray): a tensor of size (N x InChannels x OutHeight x OutWidth).
See https: //pytorch.org/docs/stable/generated/torch.nn.AvgPool2d.html

function `numpy_cast`

numpy_cast(data: ndarray, to: int) → Tuple[ndarray]

Execute ONNX cast in Numpy.

Supports only booleans for now, which are converted to integers.

See: https://github.com/onnx/onnx/blob/main/docs/Operators.md#Cast

Args:

data (numpy.ndarray): Input encrypted tensor
to (int): integer value of the onnx.TensorProto DataType enum

Returns:

result (numpy.ndarray): a tensor with the required data type

function `numpy_batchnorm`

numpy_batchnorm(
    x: ndarray,
    scale: ndarray,
    bias: ndarray,
    input_mean: ndarray,
    input_var: ndarray,
    epsilon=1e-05,
    momentum=0.9,
    training_mode=0
) → Tuple[ndarray]

Compute the batch normalization of the input tensor.

This can be expressed as:

Y = (X - input_mean) / sqrt(input_var + epsilon) * scale + B

See https://github.com/onnx/onnx/blob/main/docs/Changelog.md#BatchNormalization-14

Args:

x (numpy.ndarray): tensor to normalize, dimensions are in the form of (N,C,D1,D2,...,Dn), where N is the batch size, C is the number of channels.
scale (numpy.ndarray): scale tensor of shape (C,)
bias (numpy.ndarray): bias tensor of shape (C,)
input_mean (numpy.ndarray): mean values to use for each input channel, shape (C,)
input_var (numpy.ndarray): variance values to use for each input channel, shape (C,)
epsilon (float): avoids division by zero
momentum (float): momentum used during training of the mean/variance, not used in inference
training_mode (int): if the model was exported in training mode this is set to 1, else 0

Returns:

numpy.ndarray: Normalized tensor

function `numpy_flatten`

numpy_flatten(x: ndarray, axis: int = 1) → Tuple[ndarray]

Flatten a tensor into a 2d array.

See https://github.com/onnx/onnx/blob/main/docs/Changelog.md#Flatten-13.

Args:

x (numpy.ndarray): tensor to flatten
axis (int): axis after which all dimensions will be flattened (axis=0 gives a 1D output)

Returns:

result: flattened tensor

function `numpy_or`

numpy_or(a: ndarray, b: ndarray) → Tuple[ndarray]

Compute or in numpy according to ONNX spec.

See https://github.com/onnx/onnx/blob/main/docs/Changelog.md#Or-7

Args:

a (numpy.ndarray): Input tensor
b (numpy.ndarray): Input tensor

Returns:

Tuple[numpy.ndarray]: Output tensor

function `numpy_or_float`

numpy_or_float(a: ndarray, b: ndarray) → Tuple[ndarray]

Compute or in numpy according to ONNX spec and cast outputs to floats.

See https://github.com/onnx/onnx/blob/main/docs/Changelog.md#Or-7

Args:

a (numpy.ndarray): Input tensor
b (numpy.ndarray): Input tensor

Returns:

Tuple[numpy.ndarray]: Output tensor

function `numpy_round`

numpy_round(a: ndarray) → Tuple[ndarray]

Compute round in numpy according to ONNX spec.

See https://github.com/onnx/onnx/blob/main/docs/Changelog.md#Round-11 Remark that ONNX Round operator is actually a rint, since the number of decimals is forced to be 0

Args:

a (numpy.ndarray): Input tensor whose elements to be rounded.

Returns:

Tuple[numpy.ndarray]: Output tensor with rounded input elements.

function `numpy_pow`

numpy_pow(a: ndarray, b: ndarray) → Tuple[ndarray]

Compute pow in numpy according to ONNX spec.

See https://github.com/onnx/onnx/blob/main/docs/Changelog.md#Pow-13

Args:

a (numpy.ndarray): Input tensor whose elements to be raised.
b (numpy.ndarray): The power to which we want to raise.

Returns:

Tuple[numpy.ndarray]: Output tensor.

function `numpy_floor`

numpy_floor(x: ndarray) → Tuple[ndarray]

Compute Floor in numpy according to ONNX spec.

See https://github.com/onnx/onnx/blob/main/docs/Changelog.md#Floor-1

Args:

x (numpy.ndarray): Input tensor

Returns:

Tuple[numpy.ndarray]: Output tensor

function `numpy_max`

numpy_max(a: ndarray, b: ndarray) → Tuple[ndarray]

Compute Max in numpy according to ONNX spec.

Computes the max between the first input and a float constant.

See https://github.com/onnx/onnx/blob/main/docs/Changelog.md#Max-1

Args:

a (numpy.ndarray): Input tensor
b (numpy.ndarray): Constant tensor to compare to the first input

Returns:

Tuple[numpy.ndarray]: Output tensor

function `numpy_min`

numpy_min(a: ndarray, b: ndarray) → Tuple[ndarray]

Compute Min in numpy according to ONNX spec.

Computes the minimum between the first input and a float constant.

See https://github.com/onnx/onnx/blob/main/docs/Changelog.md#Max-1

Args:

a (numpy.ndarray): Input tensor
b (numpy.ndarray): Constant tensor to compare to the first input

Returns:

Tuple[numpy.ndarray]: Output tensor

function `numpy_sign`

numpy_sign(x: ndarray) → Tuple[ndarray]

Compute Sign in numpy according to ONNX spec.

See https://github.com/onnx/onnx/blob/main/docs/Changelog.md#Sign-9

Args:

x (numpy.ndarray): Input tensor

Returns:

Tuple[numpy.ndarray]: Output tensor

function `numpy_neg`

numpy_neg(x: ndarray) → Tuple[ndarray]

Compute Negative in numpy according to ONNX spec.

See https://github.com/onnx/onnx/blob/main/docs/Changelog.md#Sign-9

Args:

x (numpy.ndarray): Input tensor

Returns:

Tuple[numpy.ndarray]: Output tensor

function `numpy_concatenate`

numpy_concatenate(*x: ndarray, axis: int) → Tuple[ndarray]

Apply concatenate in numpy according to ONNX spec.

See https://github.com/onnx/onnx/blob/main/docs/Changelog.md#concat-13

Args:

*x (numpy.ndarray): Input tensors to be concatenated.
axis (int): Which axis to concat on.

Returns:

Tuple[numpy.ndarray]: Output tensor.

class `ONNXMixedFunction`

A mixed quantized-raw valued onnx function.

ONNX functions will take inputs which can be either quantized or float. Some functions only take quantized inputs, but some functions take both types. For mixed functions we need to tag the parameters that do not need quantization. Thus quantized ops can know which inputs are not QuantizedArray and we avoid unnecessary wrapping of float values as QuantizedArrays.

method `init`

__init__(function, non_quant_params: Set[str])

Create the mixed function and raw parameter list.

Args:

function (Any): function to be decorated
non_quant_params: Set[str]: set of parameters that will not be quantized (stored as numpy.ndarray)