QuantizedModule API.

class QuantizedModule

Inference for a quantized model.

method __init__

    ordered_module_input_names: Iterable[str] = None,
    ordered_module_output_names: Iterable[str] = None,
    quant_layers_dict: Dict[str, Tuple[Tuple[str, ], QuantizedOp]] = None

property fhe_circuit

Get the FHE circuit.


  • Circuit: the FHE circuit

property is_compiled

Return the compiled status of the module.


  • bool: the compiled status of the module.

property onnx_model

Get the ONNX model.

.. # noqa: DAR201


  • _onnx_model (onnx.ModelProto): the ONNX model

property post_processing_params

Get the post-processing parameters.


  • Dict[str, Any]: the post-processing parameters

method compile

    q_inputs: Union[Tuple[ndarray, ], ndarray],
    configuration: Optional[Configuration] = None,
    compilation_artifacts: Optional[DebugArtifacts] = None,
    show_mlir: bool = False,
    use_virtual_lib: bool = False,
    p_error: Optional[float] = None,
    global_p_error: Optional[float] = None,
    verbose_compilation: bool = False
) → Circuit

Compile the forward function of the module.


  • q_inputs (Union[Tuple[numpy.ndarray, ...], numpy.ndarray]): Needed for tracing and building the boundaries.

  • configuration (Optional[Configuration]): Configuration object to use during compilation

  • compilation_artifacts (Optional[DebugArtifacts]): Artifacts object to fill during

  • show_mlir (bool): if set, the MLIR produced by the converter and which is going to be sent to the compiler backend is shown on the screen, e.g., for debugging or demo. Defaults to False.

  • use_virtual_lib (bool): set to use the so called virtual lib simulating FHE computation. Defaults to False.

  • p_error (Optional[float]): probability of error of a single PBS.

  • global_p_error (Optional[float]): probability of error of the full circuit. Not simulated by the VL, i.e., taken as 0

  • verbose_compilation (bool): whether to show compilation information


  • Circuit: the compiled Circuit.

method dequantize_output

dequantize_output(qvalues: ndarray) → ndarray

Take the last layer q_out and use its dequant function.


  • qvalues (numpy.ndarray): Quantized values of the last layer.


  • numpy.ndarray: Dequantized values of the last layer.

method forward

    *qvalues: ndarray,
    debug: bool = False
) → Union[ndarray, Tuple[ndarray, Dict[str, ndarray]]]

Forward pass with numpy function only.


  • *qvalues (numpy.ndarray): numpy.array containing the quantized values.

  • debug (bool): In debug mode, returns quantized intermediary values of the computation. This is useful when a model's intermediary values in Concrete-ML need to be compared with the intermediary values obtained in pytorch/onnx. When set, the second return value is a dictionary containing ONNX operation names as keys and, as values, their input QuantizedArray or ndarray. The use can thus extract the quantized or float values of quantized inputs.


  • (numpy.ndarray): Predictions of the quantized model

method forward_and_dequant

forward_and_dequant(*q_x: ndarray) → ndarray

Forward pass with numpy function only plus dequantization.


  • *q_x (numpy.ndarray): numpy.ndarray containing the quantized input values. Requires the input dtype to be int64.


  • (numpy.ndarray): Predictions of the quantized model

method post_processing

post_processing(qvalues: ndarray) → ndarray

Post-processing of the quantized output.


  • qvalues (numpy.ndarray): numpy.ndarray containing the quantized input values.


  • (numpy.ndarray): Predictions of the quantized model

method quantize_input

quantize_input(*values: ndarray) → Union[ndarray, Tuple[ndarray, ]]

Take the inputs in fp32 and quantize it using the learned quantization parameters.


  • *values (numpy.ndarray): Floating point values.


  • Union[numpy.ndarray, Tuple[numpy.ndarray, ...]]: Quantized (numpy.int64) values.

method set_inputs_quantization_parameters

set_inputs_quantization_parameters(*input_q_params: UniformQuantizer)

Set the quantization parameters for the module's inputs.


  • *input_q_params (UniformQuantizer): The quantizer(s) for the module.

Last updated