Concrete-ML models can be easily deployed in a client/server setting, enabling the creation of privacy-preserving services in the cloud.

As seen in the concepts section, a Concrete-ML model, once compiled to FHE, generates machine code that performs the inference on private data. Furthermore, secret encryption keys are needed so that the user can securely encrypt their data and decrypt the inference result. An evaluation key is also needed for the server to securely process the user's encrypted data.

Keys are generated by the user once for each service they use, based on the model the service provides and its cryptographic parameters.

The overall communications protocol to enable cloud deployment of machine learning services can be summarized in the following diagram:

The steps detailed above are as follows:

1. The model developer deploys the compiled machine learning model to the server. This model includes the cryptographic parameters. The server is now ready to provide private inference.

2. The client requests the cryptographic parameters (also called "client specs"). Once it gets them from the server, the secret and evaluation keys are generated.

3. The client sends the evaluation key to the server. The server is now ready to accept requests from this client. The client sends their encrypted data.

4. The server uses the evaluation key to securely run inference on the user's data and sends back the encrypted result.

5. The client now decrypts the result and can send back new requests.

For more information on how to implement this basic secure inference protocol, refer to the Production Deployment section and to the client/server example.

Concrete-ML is built on top of Concrete-Numpy, which enables Numpy programs to be converted into FHE circuits.

Lifecycle of a Concrete-ML model

I. Model Development

1. Training. A model is trained using plaintext, non-encrypted, training data.

2. Quantization. The model is converted into an integer equivalent using quantization. Concrete-ML performs this step either during training (Quantization-Aware Training) or after training (Post-Training Quantization), depending on model type. Quantization converts inputs, model weights and all intermediate values of the inference computation to integers. More information is available here.

3. Simulation using the Virtual Library. Testing FHE models on very large datasets can take a long time. Furthermore, not all models are compatible with FHE constraints out-of-the-box. Simulation using the Virtual Library allows you to execute a model that was quantized, to measure the accuracy it would have in FHE, but also to determine the modifications required to make it FHE compatible. Simulation is described in more details here.

4. Compilation. Once the model is quantized, simulation can confirm it has good accuracy in FHE. The model then needs to be compiled using Concrete's FHE compiler to produce an equivalent FHE circuit. This circuit is represented as an MLIR program consisting of low level cryptographic operations. You can read more about FHE compilation here, MLIR here and about the low-level Concrete library here.

5. Inference. The compiled model can then be executed on encrypted data, once the proper keys have been generated. The model can also be deployed to a server and used to run private inference on encrypted inputs.

You can see some examples of the model development workflow here.

II. Model deployment

1. Client/Server deployment. In a client/server setting, the model can be exported in a way that:

   • allows the client to generate keys, encrypt and decrypt.

   • provides a compiled model that can run on the server to perform inference on encrypted data

2. Key generation. The data owner (client) needs to generate a pair of private keys (to encrypt/decrypt their data and results) and a public evaluation key (for the model's FHE evaluation on the server).

You can see an example of the model deployment workflow here.

Cryptography concepts

Concrete-ML and Concrete-Numpy are tools that hide away the details of the underlying cryptography scheme, called TFHE. However, some cryptography concepts are still useful when using these two toolkits:

1. Encryption/Decryption. These operations transform plaintext, i.e. human-readable information, into ciphertext, i.e. data that contains a form of the original plaintext that is unreadable by a human or computer without the proper key to decrypt it. Encryption takes plaintext and an encryption key and produces ciphertext, while decryption is the inverse operation.

2. Encrypted inference. FHE allows a third party to execute (i.e. run inference or predict) a machine learning model on encrypted data (a ciphertext). The result of the inference is also encrypted and can only be read by the person who gets the decryption key.

3. Keys. A key is a series of bits used within an encryption algorithm for encrypting data so that the corresponding ciphertext appears random.

4. Key generation. Cryptographic keys need to be generated using random number generators. Their size may be large and key generation may take a long time. However, keys only need to be generated once for each model a client uses.

5. Guaranteed correctness of encrypted computations. To achieve security, TFHE, the underlying encryption scheme, adds random noise as ciphertexts. This can induce errors during processing of encrypted data, depending on noise parameters. By default, Concrete-ML uses parameters that ensure the correctness of the encrypted computation, so you do not need to take into account the noise parametrization. Therefore, results on encrypted data will be the same as the results of simulation on clear data.

While Concrete-ML users only need to understand the cryptography concepts above, for a deeper understanding of the cryptography behind the Concrete stack, please see the whitepaper on TFHE and Programmable Boostrapping or this series of blogs.

Model accuracy considerations under FHE constraints

To respect FHE constraints, all numerical programs over encrypted data must have all inputs, constants and intermediate values represented with integers of a maximum of 8 bits.

Thus, Concrete-ML quantizes the input data and model outputs in the same way as weights and activations. The main levers to control accumulator bit-width are the numbers of bits used for the inputs, weights and activations of the model. These parameters are crucial to comply with the constraint on accumulator bit-widths. Please refer to the quantization documentation for more details about how to develop models with quantization in Concrete-ML.

However, these methods may cause a reduction in the accuracy of the model since its representative power is diminished. Most importantly, carefully choosing a quantization approach can alleviate accuracy loss, all the while allowing compilation to FHE. Concrete-ML offers built-in models that already include quantization algorithms, and users only need to configure some of their parameters, such as the number of bits, discussed above. See the advanced quantization guide for information about configuring these parameters for various models.

Additional specific methods can help to make models compatible with FHE constraints. For instance, dimensionality reduction can reduce the number of input features and, thus, the maximum accumulator bit-width reached within a circuit. Similarly, sparsity-inducing training methods, such as pruning, de-activate some features during inference, which also helps. For now, dimensionality reduction is considered as a pre-processing step, while pruning is used in the built-in neural networks.

The configuration of model quantization parameters is illustrated in the advanced examples for Linear and Logistic Regressions and dimensionality reduction is shown in the Poisson regression example.

Concrete-ML provides several of the most popular linear models for regression or classification that can be found in Scikit-learn:

Using these models in FHE is extremely similar to what can be done with scikit-learn's API, making it easy for data scientists who are used to this framework to get started with Concrete-ML.

Models are also compatible with some of scikit-learn's main workflows, such as Pipeline() or GridSearch().

Example

Here's an example of how to use this model in FHE on a simple data-set below. A more complete example can be found in the LogisticRegression notebook.

import numpy
from tqdm import tqdm
from sklearn.datasets import make_classification
from sklearn.model_selection import train_test_split

from concrete.ml.sklearn import LogisticRegression

# Create the data for classification
X, y = make_classification(
    n_features=2,
    n_redundant=0,
    n_informative=2,
    random_state=2,
    n_clusters_per_class=1,
    n_samples=100,
)

# Retrieve train and test sets
X_train, X_test, y_train, y_test = train_test_split(X, y, test_size=0.4, random_state=42)

# Instantiate the model
model = LogisticRegression(n_bits=2)

# Fit the model
model.fit(X_train, y_train)

# Evaluate the model on the test set in clear
y_pred_clear = model.predict(X_test)

# Compile the model
model.compile(X_train)

# Perform the inference in FHE
# Note that here the encryption and decryption is done behind the scene.
# It is recommended to run this with a very small batch of
# examples first (e.g. N_TEST_FHE = 3)
N_TEST_FHE = 3
y_pred_fhe = numpy.array([
  model.predict([sample], execute_in_fhe=True)[0]
  for sample in tqdm(X_test[:N_TEST_FHE])
])

# Assert that FHE predictions are the same as the clear predictions
print(f"{(y_pred_fhe == y_pred_clear[:N_TEST_FHE]).sum()} "
      f"examples over {N_TEST_FHE} have a FHE inference equal to the clear inference.")

# Output:
#  3 examples over 3 have a FHE inference equal to the clear inference

We can then plot the decision boundary of the classifier and then compare those results with a scikit-learn model executed in clear. The complete code can be found in the LogisticRegression notebook.

We can clearly observe the impact of quantization over the decision boundaries in the FHE model, separating the initial lines into broken lines with steps. However, this does not change the overall score as both models output the same accuracy (90%).

In fact, the quantization process may sometimes create some artifacts that could lead to a decrease in performance. Still, the impact of those artifacts is often minor when considering linear models as FHE models reach similar scores as their equivalent clear ones.

The following table summarizes the various examples in this section, along with their accuracies.

A * means that FHE accuracy was calculated on a subset of the validation set.

Concrete-ML models

• LinearRegression.ipynb

• LogisticRegression.ipynb

• PoissonRegression.ipynb

• DecisionTreeClassifier.ipynb

• XGBClassifier.ipynb

• GLMComparison.ipynb

• XGBRegressor.ipynb

Comparison of classifiers

• ClassifierComparison.ipynb

Kaggle competition

• KaggleTitanic.ipynb

Depending on your OS, Concrete-ML may be installed with Docker or with pip:

Also, only some versions of python are supported: in the current release, these are 3.8 and 3.9. Please note that, at the time of this Concrete-ML version release, Kaggle or Google Colab use Python 3.7 which is a deprecated version and is not supported by Concrete-ML.

Most of these limits are shared with the rest of the Concrete stack (namely Concrete-Numpy and Concrete-Compiler). Support for more platforms will be added in the future.

Using PyPi

Requirements

Installing Concrete-ML using PyPi requires a Linux-based OS or macOS running on an x86 CPU. For Apple Silicon, Docker is the only currently supported option (see below).

Installing on Windows can be done using Docker or WSL. On WSL, Concrete-ML will work as long as the package is not installed in the /mnt/c/ directory, which corresponds to the host OS filesystem.

Installation

To install Concrete-ML from PyPi, run the following:

pip install -U pip wheel setuptools
pip install concrete-ml

This will automatically install all dependencies, notably Concrete-Numpy.

Using Docker

Concrete-ML can be installed using Docker by either pulling the latest image or a specific version:

docker pull zamafhe/concrete-ml:latest
# or
docker pull zamafhe/concrete-ml:v0.4.0

The image can be used with Docker volumes, see the Docker documentation here.

The image can then be used via the following command:

# Without local volume:
docker run --rm -it -p 8888:8888 zamafhe/concrete-ml

# With local volume to save notebooks on host:
docker run --rm -it -p 8888:8888 -v /host/path:/data zamafhe/concrete-ml

This will launch a Concrete-ML enabled Jupyter server in Docker that can be accessed directly from a browser.

Alternatively, a shell can be lauched in Docker, with or without volumes:

docker run --rm -it zamafhe/concrete-ml /bin/bash