Concrete ML
WebsiteLibrariesProducts & ServicesDevelopersSupport
1.9
1.9
  • Welcome
  • Get Started
    • What is Concrete ML?
    • Installation
    • Key concepts
    • Inference in the cloud
  • Built-in Models
    • Linear models
    • Tree-based models
    • Neural networks
    • Nearest neighbors
    • Encrypted dataframe
    • Encrypted training
  • LLMs
    • Inference
    • Encrypted fine-tuning
  • Deep Learning
    • Using Torch
    • Using ONNX
    • Step-by-step guide
    • Debugging models
    • Optimizing inference
  • Guides
    • Prediction with FHE
    • Production deployment
    • Hybrid models
    • Serialization
    • GPU acceleration
  • Tutorials
    • See all tutorials
    • Built-in model examples
    • Deep learning examples
  • References
    • API
  • Explanations
    • Security and correctness
    • Quantization
    • Pruning
    • Compilation
    • Advanced features
    • Project architecture
      • Importing ONNX
      • Quantization tools
      • FHE Op-graph design
      • External libraries
  • Developers
    • Set up the project
    • Set up Docker
    • Documentation
    • Support and issues
    • Contributing
    • Support new ONNX node
    • Release note
    • Feature request
    • Bug report
Powered by GitBook

Libraries

  • TFHE-rs
  • Concrete
  • Concrete ML
  • fhEVM

Developers

  • Blog
  • Documentation
  • Github
  • FHE resources

Company

  • About
  • Introduction to FHE
  • Media
  • Careers
On this page

Was this helpful?

Export as PDF
  1. Get Started

Inference in the cloud

PreviousKey conceptsNextLinear models

Last updated 1 month ago

Was this helpful?

This document illustrate how Concrete ML model and DataFrames are deployed in client/server setting when creating privacy-preserving services in the cloud.

Once compiled to FHE, a Concrete ML model or DataFrame generates machine code that execute prediction, training or pre-processing on encrypted data. During this process, Concrete ML generates and .

Communication protocols

The overall communications protocol to enable cloud deployment of machine learning services can be summarized in the following diagram:

The steps detailed above are:

  1. Model Deployment: The model developer deploys the compiled machine learning model to the server. This model includes the cryptographic parameters. The server is now ready to provide private inference. Cryptographic parameters and compiled programs for DataFrames are included directly in Concrete ML.

  2. Client request: The client requests the cryptographic parameters (client specs). Once the client receives them from the server, the secret and evaluation keys are generated.

  3. Key exchanges: The client sends the evaluation key to the server. The server is now ready to accept requests from this client. The client sends their encrypted data. Serialized DataFrames include client evaluation keys.

  4. Private inference: The server uses the evaluation key to securely run prediction, training and pre-processing on the user's data and sends back the encrypted result.

  5. Decryption: The client now decrypts the result and can send back new requests.

For more information on how to implement this basic secure inference protocol, refer to the and to the . For information on training on encrypted data, see .

Production Deployment section
client/server example
the corresponding section
the private encryption keys
the pubic evaluation keys