Concrete ML
WebsiteLibrariesProducts & ServicesDevelopersSupport
0.5
0.5
  • What is Concrete ML?
  • Getting Started
    • Installation
    • Key Concepts
    • Inference in the Cloud
  • Built-in Models
    • Linear Models
    • Tree-based Models
    • Neural Networks
    • Pandas
    • Built-in Model Examples
  • Deep Learning
    • Using Torch
    • Using ONNX
    • Step-by-Step Guide
    • Deep Learning Examples
    • Debugging Models
  • Advanced topics
    • Quantization
    • Pruning
    • Compilation
    • Production Deployment
    • Advanced Features
  • Developer Guide
    • Workflow
      • Set Up the Project
      • Set Up Docker
      • Documentation
      • Support and Issues
      • Contributing
    • Inner workings
      • Importing ONNX
      • Quantization tools
      • FHE Op-graph design
      • External Libraries
    • API
      • concrete.ml.common
      • concrete.ml.common.check_inputs
      • concrete.ml.common.debugging
      • concrete.ml.common.debugging.custom_assert
      • concrete.ml.common.utils
      • concrete.ml.deployment
      • concrete.ml.deployment.fhe_client_server
      • concrete.ml.onnx
      • concrete.ml.onnx.convert
      • concrete.ml.onnx.onnx_model_manipulations
      • concrete.ml.onnx.onnx_utils
      • concrete.ml.onnx.ops_impl
      • concrete.ml.quantization
      • concrete.ml.quantization.base_quantized_op
      • concrete.ml.quantization.post_training
      • concrete.ml.quantization.quantized_module
      • concrete.ml.quantization.quantized_ops
      • concrete.ml.quantization.quantizers
      • concrete.ml.sklearn
      • concrete.ml.sklearn.base
      • concrete.ml.sklearn.glm
      • concrete.ml.sklearn.linear_model
      • concrete.ml.sklearn.protocols
      • concrete.ml.sklearn.qnn
      • concrete.ml.sklearn.rf
      • concrete.ml.sklearn.svm
      • concrete.ml.sklearn.torch_module
      • concrete.ml.sklearn.tree
      • concrete.ml.sklearn.tree_to_numpy
      • concrete.ml.sklearn.xgb
      • concrete.ml.torch
      • concrete.ml.torch.compile
      • concrete.ml.torch.numpy_module
      • concrete.ml.version
Powered by GitBook

Libraries

  • TFHE-rs
  • Concrete
  • Concrete ML
  • fhEVM

Developers

  • Blog
  • Documentation
  • Github
  • FHE resources

Company

  • About
  • Introduction to FHE
  • Media
  • Careers
On this page
  • Deployment
  • Serving
  • Example notebook

Was this helpful?

Export as PDF
  1. Advanced topics

Production Deployment

PreviousCompilationNextAdvanced Features

Last updated 2 years ago

Was this helpful?

Concrete-ML provides functionality to deploy FHE machine learning models in a client/server setting. The deployment workflow and model serving pattern is as follows:

Deployment

The training of the model and its compilation to FHE are performed on a development machine. Three different files are created when saving the model:

  • client.json contains the secure cryptographic parameters needed for the client to generate private and evaluation keys.

  • server.json contains the compiled model. This file is sufficient to run the model on a server.

  • serialized_processing.json contains the metadata about pre- and post-processing, such as quantization parameters to quantize the input and de-quantize the output.

The compiled model (server.zip) is deployed to a server and the cryptographic parameters (client.zip) along with the model meta data (serialized_processing.json) are shared with the clients.

Serving

The client obtains the cryptographic parameters (using client.zip) and generates a private encryption/decryption key as well as a set of public evaluation keys. The public evaluation keys are then sent to the server, while the secret key remains on the client.

The private data is then encrypted using serialized_processing.json by the client and sent to the server. Server-side, the FHE model inference is run on the encrypted inputs using the public evaluation keys.

The encrypted result is then returned by the server to the client, which decrypts it using its private key. Finally, the client performs any necessary post-processing of the decrypted result using serialized_processing.json.

Example notebook

For a complete example, see

this notebook