concrete.ml.torch.compile.md
Last updated
Was this helpful?
Last updated
Was this helpful?
concrete.ml.torch.compiletorch compilation function.
MAX_BITWIDTH_BACKWARD_COMPATIBLE
OPSET_VERSION_FOR_ONNX_EXPORT
convert_torch_tensor_or_numpy_array_to_numpy_arrayConvert a torch tensor or a numpy array to a numpy array.
Args:
torch_tensor_or_numpy_array (Tensor): the value that is either a torch tensor or a numpy array.
Returns:
numpy.ndarray: the value converted to a numpy array.
build_quantized_moduleBuild a quantized module from a Torch or ONNX model.
Take a model in torch or ONNX, turn it to numpy, quantize its inputs / weights / outputs and retrieve the associated quantized module.
Args:
model (Union[torch.nn.Module, onnx.ModelProto]): The model to quantize, either in torch or in ONNX.
torch_inputset (Dataset): the calibration input-set, can contain either torch tensors or numpy.ndarray
import_qat (bool): Flag to signal that the network being imported contains quantizers in in its computation graph and that Concrete ML should not re-quantize it
n_bits: the number of bits for the quantization
rounding_threshold_bits (int): if not None, every accumulators in the model are rounded down to the given bits of precision
Returns:
QuantizedModule: The resulting QuantizedModule.
compile_torch_modelCompile a torch module into an FHE equivalent.
Take a model in torch, turn it to numpy, quantize its inputs / weights / outputs and finally compile it with Concrete
Args:
torch_model (torch.nn.Module): the model to quantize
torch_inputset (Dataset): the calibration input-set, can contain either torch tensors or numpy.ndarray.
import_qat (bool): Set to True to import a network that contains quantizers and was trained using quantization aware training
configuration (Configuration): Configuration object to use during compilation
artifacts (DebugArtifacts): Artifacts object to fill during compilation
show_mlir (bool): if set, the MLIR produced by the converter and which is going to be sent to the compiler backend is shown on the screen, e.g., for debugging or demo
n_bits: the number of bits for the quantization
rounding_threshold_bits (int): if not None, every accumulators in the model are rounded down to the given bits of precision
p_error (Optional[float]): probability of error of a single PBS
global_p_error (Optional[float]): probability of error of the full circuit. In FHE simulation global_p_error is set to 0
verbose (bool): whether to show compilation information
Returns:
QuantizedModule: The resulting compiled QuantizedModule.
compile_onnx_modelCompile a torch module into an FHE equivalent.
Take a model in torch, turn it to numpy, quantize its inputs / weights / outputs and finally compile it with Concrete-Python
Args:
onnx_model (onnx.ModelProto): the model to quantize
torch_inputset (Dataset): the calibration input-set, can contain either torch tensors or numpy.ndarray.
import_qat (bool): Flag to signal that the network being imported contains quantizers in in its computation graph and that Concrete ML should not re-quantize it.
configuration (Configuration): Configuration object to use during compilation
artifacts (DebugArtifacts): Artifacts object to fill during compilation
show_mlir (bool): if set, the MLIR produced by the converter and which is going to be sent to the compiler backend is shown on the screen, e.g., for debugging or demo
n_bits: the number of bits for the quantization
rounding_threshold_bits (int): if not None, every accumulators in the model are rounded down to the given bits of precision
p_error (Optional[float]): probability of error of a single PBS
global_p_error (Optional[float]): probability of error of the full circuit. In FHE simulation global_p_error is set to 0
verbose (bool): whether to show compilation information
Returns:
QuantizedModule: The resulting compiled QuantizedModule.
compile_brevitas_qat_modelCompile a Brevitas Quantization Aware Training model.
The torch_model parameter is a subclass of torch.nn.Module that uses quantized operations from brevitas.qnn. The model is trained before calling this function. This function compiles the trained model to FHE.
Args:
torch_model (torch.nn.Module): the model to quantize
torch_inputset (Dataset): the calibration input-set, can contain either torch tensors or numpy.ndarray.
n_bits (Optional[Union[int, dict]): the number of bits for the quantization. By default, for most models, a value of None should be given, which instructs Concrete ML to use the bit-widths configured using Brevitas quantization options. For some networks, that perform a non-linear operation on an input on an output, if None is given, a default value of 8 bits is used for the input/output quantization. For such models the user can also specify a dictionary with model_inputs/model_outputs keys to override the 8-bit default or a single integer for both values.
configuration (Configuration): Configuration object to use during compilation
artifacts (DebugArtifacts): Artifacts object to fill during compilation
show_mlir (bool): if set, the MLIR produced by the converter and which is going to be sent to the compiler backend is shown on the screen, e.g., for debugging or demo
rounding_threshold_bits (int): if not None, every accumulators in the model are rounded down to the given bits of precision
p_error (Optional[float]): probability of error of a single PBS
global_p_error (Optional[float]): probability of error of the full circuit. In FHE simulation global_p_error is set to 0
output_onnx_file (str): temporary file to store ONNX model. If None a temporary file is generated
verbose (bool): whether to show compilation information
Returns:
QuantizedModule: The resulting compiled QuantizedModule.