Concrete ML
WebsiteLibrariesProducts & ServicesDevelopersSupport
1.2
1.2
  • What is Concrete ML?
  • Getting Started
    • Installation
    • Key Concepts
    • Inference in the Cloud
    • Demos and Tutorials
  • Built-in Models
    • Linear Models
    • Tree-based Models
    • Neural Networks
    • Nearest Neighbors
    • Pandas
    • Built-in Model Examples
  • Deep Learning
    • Using Torch
    • Using ONNX
    • Step-by-step Guide
    • Deep Learning Examples
    • Debugging Models
    • Optimizing Inference
  • Deployment
    • Prediction with FHE
    • Hybrid models
    • Production Deployment
    • Serialization
  • Advanced topics
    • Quantization
    • Pruning
    • Compilation
    • Advanced Features
  • Developer Guide
    • Workflow
      • Set Up the Project
      • Set Up Docker
      • Documentation
      • Support and Issues
      • Contributing
    • Inner Workings
      • Importing ONNX
      • Quantization Tools
      • FHE Op-graph Design
      • External Libraries
    • API
      • concrete.ml.common.check_inputs.md
      • concrete.ml.common.debugging.custom_assert.md
      • concrete.ml.common.debugging.md
      • concrete.ml.common.md
      • concrete.ml.common.serialization.decoder.md
      • concrete.ml.common.serialization.dumpers.md
      • concrete.ml.common.serialization.encoder.md
      • concrete.ml.common.serialization.loaders.md
      • concrete.ml.common.serialization.md
      • concrete.ml.common.utils.md
      • concrete.ml.deployment.deploy_to_aws.md
      • concrete.ml.deployment.deploy_to_docker.md
      • concrete.ml.deployment.fhe_client_server.md
      • concrete.ml.deployment.md
      • concrete.ml.deployment.server.md
      • concrete.ml.deployment.utils.md
      • concrete.ml.onnx.convert.md
      • concrete.ml.onnx.md
      • concrete.ml.onnx.onnx_impl_utils.md
      • concrete.ml.onnx.onnx_model_manipulations.md
      • concrete.ml.onnx.onnx_utils.md
      • concrete.ml.onnx.ops_impl.md
      • concrete.ml.pytest.md
      • concrete.ml.pytest.torch_models.md
      • concrete.ml.pytest.utils.md
      • concrete.ml.quantization.base_quantized_op.md
      • concrete.ml.quantization.md
      • concrete.ml.quantization.post_training.md
      • concrete.ml.quantization.quantized_module.md
      • concrete.ml.quantization.quantized_module_passes.md
      • concrete.ml.quantization.quantized_ops.md
      • concrete.ml.quantization.quantizers.md
      • concrete.ml.search_parameters.md
      • concrete.ml.search_parameters.p_error_search.md
      • concrete.ml.sklearn.base.md
      • concrete.ml.sklearn.glm.md
      • concrete.ml.sklearn.linear_model.md
      • concrete.ml.sklearn.md
      • concrete.ml.sklearn.neighbors.md
      • concrete.ml.sklearn.qnn.md
      • concrete.ml.sklearn.qnn_module.md
      • concrete.ml.sklearn.rf.md
      • concrete.ml.sklearn.svm.md
      • concrete.ml.sklearn.tree.md
      • concrete.ml.sklearn.tree_to_numpy.md
      • concrete.ml.sklearn.xgb.md
      • concrete.ml.torch.compile.md
      • concrete.ml.torch.hybrid_model.md
      • concrete.ml.torch.md
      • concrete.ml.torch.numpy_module.md
      • concrete.ml.version.md
Powered by GitBook

Libraries

  • TFHE-rs
  • Concrete
  • Concrete ML
  • fhEVM

Developers

  • Blog
  • Documentation
  • Github
  • FHE resources

Company

  • About
  • Introduction to FHE
  • Media
  • Careers
On this page
  • module concrete.ml.torch.hybrid_model
  • Global Variables
  • function tuple_to_underscore_str
  • function underscore_str_to_tuple
  • function convert_conv1d_to_linear
  • class HybridFHEMode
  • class RemoteModule
  • class HybridFHEModel
  • class LoggerStub
  • class HybridFHEModelServer

Was this helpful?

Export as PDF
  1. Developer Guide
  2. API

concrete.ml.torch.hybrid_model.md

Previousconcrete.ml.torch.compile.mdNextconcrete.ml.torch.md

Last updated 1 year ago

Was this helpful?

module concrete.ml.torch.hybrid_model

Implement the conversion of a torch model to a hybrid fhe/torch inference.

Global Variables

  • MAX_BITWIDTH_BACKWARD_COMPATIBLE


function tuple_to_underscore_str

tuple_to_underscore_str(tup: Tuple) → str

Convert a tuple to a string representation.

Args:

  • tup (Tuple): a tuple to change into string representation

Returns:

  • str: a string representing the tuple


function underscore_str_to_tuple

underscore_str_to_tuple(tup: str) → Tuple

Convert a a string representation of a tuple to a tuple.

Args:

  • tup (str): a string representing the tuple

Returns:

  • Tuple: a tuple to change into string representation


function convert_conv1d_to_linear

convert_conv1d_to_linear(layer_or_module)

Convert all Conv1D layers in a module or a Conv1D layer itself to nn.Linear.

Args:

  • layer_or_module (nn.Module or Conv1D): The module which will be recursively searched for Conv1D layers, or a Conv1D layer itself.

Returns:

  • nn.Module or nn.Linear: The updated module with Conv1D layers converted to Linear layers, or the Conv1D layer converted to a Linear layer.


class HybridFHEMode

Simple enum for different modes of execution of HybridModel.


class RemoteModule

A wrapper class for the modules to be evaluated remotely with FHE.

method __init__

__init__(
    module: Optional[Module] = None,
    server_remote_address: Optional[str] = None,
    module_name: Optional[str] = None,
    model_name: Optional[str] = None,
    verbose: int = 0
)

method forward

forward(x: Tensor) → Union[Tensor, QuantTensor]

Forward pass of the remote module.

To change the behavior of this forward function one must change the fhe_local_mode attribute. Choices are:

  • disable: forward using torch module

  • remote: forward with fhe client-server

  • simulate: forward with local fhe simulation

  • calibrate: forward for calibration

Args:

  • x (torch.Tensor): The input tensor.

Returns:

  • (torch.Tensor): The output tensor.

Raises:

  • ValueError: if local_fhe_mode is not supported


method init_fhe_client

init_fhe_client(
    path_to_client: Optional[Path] = None,
    path_to_keys: Optional[Path] = None
)

Set the clients keys.

Args:

  • path_to_client (str): Path where the client.zip is located.

  • path_to_keys (str): Path where keys are located.

Raises:

  • ValueError: if anything goes wrong with the server.


method remote_call

remote_call(x: Tensor) → Tensor

Call the remote server to get the private module inference.

Args:

  • x (torch.Tensor): The input tensor.

Returns:

  • torch.Tensor: The result of the FHE computation


class HybridFHEModel

Convert a model to a hybrid model.

This is done by converting targeted modules by RemoteModules. This will modify the model in place.

Args:

  • model (nn.Module): The model to modify (in-place modification)

  • module_names (Union[str, List[str]]): The module name(s) to replace with FHE server.

  • server_remote_address): The remote address of the FHE server

  • model_name (str): Model name identifier

  • verbose (int): If logs should be printed when interacting with FHE server

method __init__

__init__(
    model: Module,
    module_names: Union[str, List[str]],
    server_remote_address=None,
    model_name: str = 'model',
    verbose: int = 0
)

method compile_model

compile_model(
    x: Tensor,
    n_bits: Union[int, Dict[str, int]] = 8,
    rounding_threshold_bits: Optional[int] = None,
    p_error: Optional[float] = None,
    configuration: Optional[Configuration] = None
)

Compiles the specific layers to FHE.

Args:

  • x (torch.Tensor): The input tensor for the model. This is used to run the model once for calibration.

  • n_bits (int): The bit precision for quantization during FHE model compilation. Default is 8.

  • rounding_threshold_bits (int): The number of bits to use for rounding threshold during FHE model compilation. Default is 8.

  • p_error (float): Error allowed for each table look-up in the circuit.

  • configuration (Configuration): A concrete Configuration object specifying the FHE encryption parameters. If not specified, a default configuration is used.


method init_client

init_client(
    path_to_clients: Optional[Path] = None,
    path_to_keys: Optional[Path] = None
)

Initialize client for all remote modules.

Args:

  • path_to_clients (Optional[Path]): Path to the client.zip files.

  • path_to_keys (Optional[Path]): Path to the keys folder.


method publish_to_hub

publish_to_hub()

Allow the user to push the model and FHE required files to HF Hub.


method save_and_clear_private_info

save_and_clear_private_info(path: Path, via_mlir=False)

Save the PyTorch model to the provided path and also saves the corresponding FHE circuit.

Args:

  • path (Path): The directory where the model and the FHE circuit will be saved.

  • via_mlir (bool): if fhe circuits should be serialized using via_mlir option useful for cross-platform (compile on one architecture and run on another)


method set_fhe_mode

set_fhe_mode(hybrid_fhe_mode: Union[str, HybridFHEMode])

Set Hybrid FHE mode for all remote modules.

Args:

  • hybrid_fhe_mode (Union[str, HybridFHEMode]): Hybrid FHE mode to set to all remote modules.


class LoggerStub

Placeholder type for a typical logger like the one from loguru.


method info

info(msg: str)

Placholder function for logger.info.

Args:

  • msg (str): the message to output


class HybridFHEModelServer

Hybrid FHE Model Server.

This is a class object to server FHE models serialized using HybridFHEModel.

method __init__

__init__(key_path: Path, model_dir: Path, logger: Optional[LoggerStub])

method add_key

add_key(key: bytes, model_name: str, module_name: str, input_shape: str)

Add public key.

Arguments:

  • key (bytes): public key

  • model_name (str): model name

  • module_name (str): name of the module in the model

  • input_shape (str): input shape of said module

Returns: Dict[str, str] - uid: uid a personal uid


method check_inputs

check_inputs(
    model_name: str,
    module_name: Optional[str],
    input_shape: Optional[str]
)

Check that the given configuration exist in the compiled models folder.

Args:

  • model_name (str): name of the model

  • module_name (Optional[str]): name of the module in the model

  • input_shape (Optional[str]): input shape of the module

Raises:

  • ValueError: if the given configuration does not exist.


method compute

compute(
    model_input: bytes,
    uid: str,
    model_name: str,
    module_name: str,
    input_shape: str
)

Compute the circuit over encrypted input.

Arguments:

  • model_input (bytes): input of the circuit

  • uid (str): uid of the public key to use

  • model_name (str): model name

  • module_name (str): name of the module in the model

  • input_shape (str): input shape of said module

Returns:

  • bytes: the result of the circuit


method dump_key

dump_key(key_bytes: bytes, uid: Union[UUID, str]) → None

Dump a public key to a stream.

Args:

  • key_bytes (bytes): stream to dump the public serialized key to

  • uid (Union[str, uuid.UUID]): uid of the public key to dump


method get_circuit

get_circuit(model_name, module_name, input_shape)

Get circuit based on model name, module name and input shape.

Args:

  • model_name (str): name of the model

  • module_name (str): name of the module in the model

  • input_shape (str): input shape of the module

Returns:

  • FHEModelServer: a fhe model server of the given module of the given model for the given shape


method get_client

get_client(model_name: str, module_name: str, input_shape: str)

Get client.

Args:

  • model_name (str): name of the model

  • module_name (str): name of the module in the model

  • input_shape (str): input shape of the module

Returns:

  • Path: the path to the correct client

Raises:

  • ValueError: if client couldn't be found


method list_modules

list_modules(model_name: str)

List all modules in a model.

Args:

  • model_name (str): name of the model

Returns: Dict[str, Dict[str, Dict]]


method list_shapes

list_shapes(model_name: str, module_name: str)

List all modules in a model.

Args:

  • model_name (str): name of the model

  • module_name (str): name of the module in the model

Returns: Dict[str, Dict]


method load_key

load_key(uid: Union[str, UUID]) → bytes

Load a public key from the key path in the file system.

Args:

  • uid (Union[str, uuid.UUID]): uid of the public key to load

Returns:

  • bytes: the bytes of the public key