concrete.ml.torch.hybrid_model.md

module `concrete.ml.torch.hybrid_model`

Implement the conversion of a torch model to a hybrid fhe/torch inference.

Global Variables

MAX_BITWIDTH_BACKWARD_COMPATIBLE

function `tuple_to_underscore_str`

tuple_to_underscore_str(tup: Tuple) → str

Convert a tuple to a string representation.

Args:

tup (Tuple): a tuple to change into string representation

Returns:

str: a string representing the tuple

function `underscore_str_to_tuple`

underscore_str_to_tuple(tup: str) → Tuple

Convert a a string representation of a tuple to a tuple.

Args:

tup (str): a string representing the tuple

Returns:

Tuple: a tuple to change into string representation

function `convert_conv1d_to_linear`

convert_conv1d_to_linear(layer_or_module)

Convert all Conv1D layers in a module or a Conv1D layer itself to nn.Linear.

Args:

layer_or_module (nn.Module or Conv1D): The module which will be recursively searched for Conv1D layers, or a Conv1D layer itself.

Returns:

nn.Module or nn.Linear: The updated module with Conv1D layers converted to Linear layers, or the Conv1D layer converted to a Linear layer.

class `HybridFHEMode`

Simple enum for different modes of execution of HybridModel.

class `RemoteModule`

A wrapper class for the modules to be evaluated remotely with FHE.

method `init`

__init__(
    module: Optional[Module] = None,
    server_remote_address: Optional[str] = None,
    module_name: Optional[str] = None,
    model_name: Optional[str] = None,
    verbose: int = 0
)

method `forward`

forward(x: Tensor) → Union[Tensor, QuantTensor]

Forward pass of the remote module.

To change the behavior of this forward function one must change the fhe_local_mode attribute. Choices are:

disable: forward using torch module
remote: forward with fhe client-server
simulate: forward with local fhe simulation
calibrate: forward for calibration

Args:

x (torch.Tensor): The input tensor.

Returns:

(torch.Tensor): The output tensor.

Raises:

ValueError: if local_fhe_mode is not supported

method `init_fhe_client`

init_fhe_client(
    path_to_client: Optional[Path] = None,
    path_to_keys: Optional[Path] = None
)

Set the clients keys.

Args:

path_to_client (str): Path where the client.zip is located.
path_to_keys (str): Path where keys are located.

Raises:

ValueError: if anything goes wrong with the server.

method `remote_call`

remote_call(x: Tensor) → Tensor

Call the remote server to get the private module inference.

Args:

x (torch.Tensor): The input tensor.

Returns:

torch.Tensor: The result of the FHE computation

class `HybridFHEModel`

Convert a model to a hybrid model.

This is done by converting targeted modules by RemoteModules. This will modify the model in place.

Args:

model (nn.Module): The model to modify (in-place modification)
module_names (Union[str, List[str]]): The module name(s) to replace with FHE server.
server_remote_address): The remote address of the FHE server
model_name (str): Model name identifier
verbose (int): If logs should be printed when interacting with FHE server

method `init`

__init__(
    model: Module,
    module_names: Union[str, List[str]],
    server_remote_address=None,
    model_name: str = 'model',
    verbose: int = 0
)

method `compile_model`

compile_model(
    x: Tensor,
    n_bits: Union[int, Dict[str, int]] = 8,
    rounding_threshold_bits: Optional[int] = None,
    p_error: Optional[float] = None,
    configuration: Optional[Configuration] = None
)

Compiles the specific layers to FHE.

Args:

x (torch.Tensor): The input tensor for the model. This is used to run the model once for calibration.
n_bits (int): The bit precision for quantization during FHE model compilation. Default is 8.
rounding_threshold_bits (int): The number of bits to use for rounding threshold during FHE model compilation. Default is 8.
p_error (float): Error allowed for each table look-up in the circuit.
configuration (Configuration): A concrete Configuration object specifying the FHE encryption parameters. If not specified, a default configuration is used.

method `init_client`

init_client(
    path_to_clients: Optional[Path] = None,
    path_to_keys: Optional[Path] = None
)

Initialize client for all remote modules.

Args:

path_to_clients (Optional[Path]): Path to the client.zip files.
path_to_keys (Optional[Path]): Path to the keys folder.

method `publish_to_hub`

publish_to_hub()

Allow the user to push the model and FHE required files to HF Hub.

method `save_and_clear_private_info`

save_and_clear_private_info(path: Path, via_mlir=False)

Save the PyTorch model to the provided path and also saves the corresponding FHE circuit.

Args:

path (Path): The directory where the model and the FHE circuit will be saved.
via_mlir (bool): if fhe circuits should be serialized using via_mlir option useful for cross-platform (compile on one architecture and run on another)

method `set_fhe_mode`

set_fhe_mode(hybrid_fhe_mode: Union[str, HybridFHEMode])

Set Hybrid FHE mode for all remote modules.

Args:

hybrid_fhe_mode (Union[str, HybridFHEMode]): Hybrid FHE mode to set to all remote modules.

class `LoggerStub`

Placeholder type for a typical logger like the one from loguru.

method `info`

info(msg: str)

Placholder function for logger.info.

Args:

msg (str): the message to output

class `HybridFHEModelServer`

Hybrid FHE Model Server.

This is a class object to server FHE models serialized using HybridFHEModel.

method `init`

__init__(key_path: Path, model_dir: Path, logger: Optional[LoggerStub])

method `add_key`

add_key(key: bytes, model_name: str, module_name: str, input_shape: str)

Add public key.

Arguments:

key (bytes): public key
model_name (str): model name
module_name (str): name of the module in the model
input_shape (str): input shape of said module

Returns: Dict[str, str] - uid: uid a personal uid

method `check_inputs`

check_inputs(
    model_name: str,
    module_name: Optional[str],
    input_shape: Optional[str]
)

Check that the given configuration exist in the compiled models folder.

Args:

model_name (str): name of the model
module_name (Optional[str]): name of the module in the model
input_shape (Optional[str]): input shape of the module

Raises:

ValueError: if the given configuration does not exist.

method `compute`

compute(
    model_input: bytes,
    uid: str,
    model_name: str,
    module_name: str,
    input_shape: str
)

Compute the circuit over encrypted input.

Arguments:

model_input (bytes): input of the circuit
uid (str): uid of the public key to use
model_name (str): model name
module_name (str): name of the module in the model
input_shape (str): input shape of said module

Returns:

bytes: the result of the circuit

method `dump_key`

dump_key(key_bytes: bytes, uid: Union[UUID, str]) → None

Dump a public key to a stream.

Args:

key_bytes (bytes): stream to dump the public serialized key to
uid (Union[str, uuid.UUID]): uid of the public key to dump

method `get_circuit`

get_circuit(model_name, module_name, input_shape)

Get circuit based on model name, module name and input shape.

Args:

model_name (str): name of the model
module_name (str): name of the module in the model
input_shape (str): input shape of the module

Returns:

FHEModelServer: a fhe model server of the given module of the given model for the given shape

method `get_client`

get_client(model_name: str, module_name: str, input_shape: str)

Get client.

Args:

model_name (str): name of the model
module_name (str): name of the module in the model
input_shape (str): input shape of the module

Returns:

Path: the path to the correct client

Raises:

ValueError: if client couldn't be found

method `list_modules`

list_modules(model_name: str)

List all modules in a model.

Args:

model_name (str): name of the model

Returns: Dict[str, Dict[str, Dict]]

method `list_shapes`

list_shapes(model_name: str, module_name: str)

List all modules in a model.

Args:

model_name (str): name of the model
module_name (str): name of the module in the model

Returns: Dict[str, Dict]

method `load_key`

load_key(uid: Union[str, UUID]) → bytes

Load a public key from the key path in the file system.

Args:

uid (Union[str, uuid.UUID]): uid of the public key to load

Returns:

bytes: the bytes of the public key

Previousconcrete.ml.torch.compile.md Nextconcrete.ml.torch.md

Last updated 6 months ago

module concrete.ml.torch.hybrid_model

Global Variables

function tuple_to_underscore_str

function underscore_str_to_tuple

function convert_conv1d_to_linear

class HybridFHEMode

class RemoteModule

method __init__

method forward

method init_fhe_client

method remote_call

class HybridFHEModel

method __init__

method compile_model

method init_client

method publish_to_hub

method save_and_clear_private_info

method set_fhe_mode

class LoggerStub

method info

class HybridFHEModelServer

method __init__

method add_key

method check_inputs

method compute

method dump_key

method get_circuit

method get_client

method list_modules

method list_shapes

method load_key

module `concrete.ml.torch.hybrid_model`

function `tuple_to_underscore_str`

function `underscore_str_to_tuple`

function `convert_conv1d_to_linear`

class `HybridFHEMode`

class `RemoteModule`

method `init`

method `forward`

method `init_fhe_client`

method `remote_call`

class `HybridFHEModel`

method `init`

method `compile_model`

method `init_client`

method `publish_to_hub`

method `save_and_clear_private_info`

method `set_fhe_mode`

class `LoggerStub`

method `info`

class `HybridFHEModelServer`

method `init`

method `add_key`

method `check_inputs`

method `compute`

method `dump_key`

method `get_circuit`

method `get_client`

method `list_modules`

method `list_shapes`

method `load_key`