Concrete ML
WebsiteLibrariesProducts & ServicesDevelopersSupport
1.9
1.9
  • Welcome
  • Get Started
    • What is Concrete ML?
    • Installation
    • Key concepts
    • Inference in the cloud
  • Built-in Models
    • Linear models
    • Tree-based models
    • Neural networks
    • Nearest neighbors
    • Encrypted dataframe
    • Encrypted training
  • LLMs
    • Inference
    • Encrypted fine-tuning
  • Deep Learning
    • Using Torch
    • Using ONNX
    • Step-by-step guide
    • Debugging models
    • Optimizing inference
  • Guides
    • Prediction with FHE
    • Production deployment
    • Hybrid models
    • Serialization
    • GPU acceleration
  • Tutorials
    • See all tutorials
    • Built-in model examples
    • Deep learning examples
  • References
    • API
  • Explanations
    • Security and correctness
    • Quantization
    • Pruning
    • Compilation
    • Advanced features
    • Project architecture
      • Importing ONNX
      • Quantization tools
      • FHE Op-graph design
      • External libraries
  • Developers
    • Set up the project
    • Set up Docker
    • Documentation
    • Support and issues
    • Contributing
    • Support new ONNX node
    • Release note
    • Feature request
    • Bug report
Powered by GitBook

Libraries

  • TFHE-rs
  • Concrete
  • Concrete ML
  • fhEVM

Developers

  • Blog
  • Documentation
  • Github
  • FHE resources

Company

  • About
  • Introduction to FHE
  • Media
  • Careers
On this page
  • Supported models for encrypted inference
  • Supported models for encrypted training
  • Ciphertext format compatibility
  • Quantization parameters
  • Pre-trained models
  • Example
  • Model accuracy
  • Loading a pre-trained model

Was this helpful?

Export as PDF
  1. Built-in Models

Linear models

PreviousInference in the cloudNextTree-based models

Last updated 1 month ago

Was this helpful?

This page explains Concrete ML linear models for both classification and regression. These models are based on linear models.

Supported models for encrypted inference

The following models are supported for training on clear data and predicting on encrypted data. Their API is similar to the one of . These models are also compatible with some of scikit-learn's main workflows, such as Pipeline() and GridSearch().

Concrete ML
scikit-learn

Supported models for encrypted training

In addition to predicting on encrypted data, the following models support training on encrypted data.

| | |

Ciphertext format compatibility

Quantization parameters

The n_bits parameter controls the bit-width of the inputs and weights of the linear models. Linear models do not use table lookups and thus allows weight and inputs to be high precision integers.

For models with input dimensions up to 300, the parameter n_bits can be set to 8 or more. When the input dimensions are larger, n_bits must be reduced to 6-7. In many cases, quantized models can preserve all performance metrics compared to the non-quantized float models from scikit-learn when n_bits is down to 6. You should validate accuracy on held-out test sets and adjust n_bits accordingly.

Pre-trained models

Example

from sklearn.datasets import make_classification
from sklearn.model_selection import train_test_split

from concrete.ml.sklearn import LogisticRegression

# Create the data for classification:
X, y = make_classification(
    n_features=30,
    n_redundant=0,
    n_informative=2,
    random_state=2,
    n_clusters_per_class=1,
    n_samples=250,
)

# Retrieve train and test sets:
X_train, X_test, y_train, y_test = train_test_split(X, y, test_size=0.4, random_state=42)

# Instantiate the model:
model = LogisticRegression(n_bits=8)

# Fit the model:
model.fit(X_train, y_train)

# Evaluate the model on the test set in clear:
y_pred_clear = model.predict(X_test)

# Compile the model:
model.compile(X_train)

# Perform the inference in FHE:
y_pred_fhe = model.predict(X_test, fhe="execute")

# Assert that FHE predictions are the same as the clear predictions:
print(
    f"{(y_pred_fhe == y_pred_clear).sum()} examples over {len(y_pred_fhe)} "
    "have an FHE inference equal to the clear inference."
)

# Output:
#  100 examples over 100 have an FHE inference equal to the clear inference

Model accuracy

Loading a pre-trained model

An alternative to the example above is to train a scikit-learn model in a separate step and then to convert it to Concrete ML.

from sklearn.linear_model import LogisticRegression as SKlearnLogisticRegression

# Instantiate the model:
model = SKlearnLogisticRegression()

# Fit the model:
model.fit(X_train, y_train)

cml_model = LogisticRegression.from_sklearn_model(model, X_train, n_bits=8)

# Compile the model:
cml_model.compile(X_train)

# Perform the inference in FHE:
y_pred_fhe = cml_model.predict(X_test, fhe="execute")

Zama 5-Question Developer Survey

These models only support Concrete ciphertexts. See documentation for more details.

For optimal results, you can use standard or min-max normalization to achieve a similar distribution of individual features. When there are many one-hot features, consider as a pre-processing stage.

For a more detailed comparison of the impact of such pre-processing, please refer to .

You can convert an already trained scikit-learn linear model to a Concrete ML one by using the method. See .

The following example shows how to train a LogisticRegression model on a simple data-set and then use FHE to perform inference on encrypted data. You can find a more complete example in the .

The figure below compares the decision boundary of the FHE classifier and a scikit-learn model executed in clear. You can find the complete code in the .

The overall accuracy scores are identical (93%) between the scikit-learn model (executed in the clear) and the Concrete ML one (executed in FHE). In fact, quantization has little impact on the decision boundaries, as linear models can use large precision numbers when quantizing inputs and weights in Concrete ML. Additionally, as the linear models do not use , the FHE computations are always exact, irrespective of the . This ensures that the FHE predictions are always identical to the quantized clear ones.

We want to hear from you! Take 1 minute to share your thoughts and help us enhance our documentation and libraries. 👉 to participate.

scikit-learn
scikit-learn
SGDClassifier
SGDClassifier
Principal Component Analysis
the logistic regression notebook
LogisticRegression notebook
LogisticRegression notebook
Click here
from_sklearn_model
the following example
LinearRegression
LinearRegression
LogisticRegression
LogisticRegression
LinearSVC
LinearSVC
LinearSVR
LinearSVR
PoissonRegressor
PoissonRegressor
TweedieRegressor
TweedieRegressor
GammaRegressor
GammaRegressor
Lasso
Lasso
Ridge
Ridge
ElasticNet
ElasticNet
SGDRegressor
SGDRegressor
the ciphertexts format
Programmable Boostrapping
PBS error tolerance setting
Sklearn model decision boundaries
FHE model decision boundaries