concrete.ml.quantization.base_quantized_op.md

module concrete.ml.quantization.base_quantized_op

Base Quantized Op class that implements quantization for a float numpy op.

Global Variables

• ONNX_OPS_TO_NUMPY_IMPL

• ALL_QUANTIZED_OPS

• ONNX_OPS_TO_QUANTIZED_IMPL

• DEFAULT_MODEL_BITS

---

class QuantizedOp

Base class for quantized ONNX ops implemented in numpy.

Args:

• n_bits_output (int): The number of bits to use for the quantization of the output

• int_input_names (Set[str]): The set of names of integer tensors that are inputs to this op

• constant_inputs (Optional[Union[Dict[str, Any], Dict[int, Any]]]): The constant tensors that are inputs to this op

• input_quant_opts (QuantizationOptions): Input quantizer options, determine the quantization that is applied to input tensors (that are not constants)

method __init__

__init__(
    n_bits_output: int,
    int_input_names: Optional[Set[str]] = None,
    constant_inputs: Optional[Dict[str, Any], Dict[int, Any]] = None,
    input_quant_opts: Optional[QuantizationOptions] = None,
    **attrs
) → None

---

property int_input_names

Get the names of encrypted integer tensors that are used by this op.

Returns:

• List[str]: the names of the tensors

---

method calibrate

calibrate(*inputs: ndarray) → ndarray

Create corresponding QuantizedArray for the output of the activation function.

Args:

• *inputs (numpy.ndarray): Calibration sample inputs.

Returns:

• numpy.ndarray: the output values for the provided calibration samples.

---

method call_impl

call_impl(*inputs: Optional[ndarray, QuantizedArray], **attrs) → ndarray

Call self.impl to centralize mypy bug workaround.

Args:

• *inputs (numpy.ndarray): real valued inputs.

• **attrs: the QuantizedOp attributes.

Returns:

• numpy.ndarray: return value of self.impl

---

method can_fuse

can_fuse() → bool

Determine if the operator impedes graph fusion.

This function shall be overloaded by inheriting classes to test self._int_input_names, to determine whether the operation can be fused to a TLU or not. For example an operation that takes inputs produced by a unique integer tensor can be fused to a TLU. Example: f(x) = x * (x + 1) can be fused. A function that does f(x) = x * (x @ w + 1) can't be fused.

Returns:

• bool: whether this instance of the QuantizedOp produces Concrete Numpy code that can be fused to TLUs

---

classmethod must_quantize_input

must_quantize_input(input_name_or_idx: int) → bool

Determine if an input must be quantized.

Quantized ops and numpy onnx ops take inputs and attributes. Inputs can be either constant or variable (encrypted). Note that this does not handle attributes, which are handled by QuantizedOp classes separately in their constructor.

Args:

• input_name_or_idx (int): Index of the input to check.

Returns:

• result (bool): Whether the input must be quantized (must be a QuantizedArray) or if it stays as a raw numpy.array read from ONNX.

---

classmethod op_type

op_type()

Get the type of this operation.

Returns:

• op_type (str): The type of this operation, in the ONNX referential

---

method prepare_output

prepare_output(qoutput_activation: ndarray) → QuantizedArray

Quantize the output of the activation function.

The calibrate method needs to be called with sample data before using this function.

Args:

• qoutput_activation (numpy.ndarray): Output of the activation function.

Returns:

• QuantizedArray: Quantized output.

---

method q_impl

q_impl(*q_inputs: QuantizedArray, **attrs) → QuantizedArray

Execute the quantized forward.

Args:

• *q_inputs (QuantizedArray): Quantized inputs.

• **attrs: the QuantizedOp attributes.

Returns:

• QuantizedArray: The returned quantized value.

---

class QuantizedOpUnivariateOfEncrypted

An univariate operator of an encrypted value.

This operation is not really operating as a quantized operation. It is useful when the computations get fused into a TLU, as in e.g. Act(x) = x || (x + 42)).

method __init__

__init__(
    n_bits_output: int,
    int_input_names: Optional[Set[str]] = None,
    constant_inputs: Optional[Dict[str, Any], Dict[int, Any]] = None,
    input_quant_opts: Optional[QuantizationOptions] = None,
    **attrs
) → None

---

property int_input_names

Get the names of encrypted integer tensors that are used by this op.

Returns:

• List[str]: the names of the tensors

---

method calibrate

calibrate(*inputs: ndarray) → ndarray

Create corresponding QuantizedArray for the output of the activation function.

Args:

• *inputs (numpy.ndarray): Calibration sample inputs.

Returns:

• numpy.ndarray: the output values for the provided calibration samples.

---

method call_impl

call_impl(*inputs: Optional[ndarray, QuantizedArray], **attrs) → ndarray

Call self.impl to centralize mypy bug workaround.

Args:

• *inputs (numpy.ndarray): real valued inputs.

• **attrs: the QuantizedOp attributes.

Returns:

• numpy.ndarray: return value of self.impl

---

method can_fuse

can_fuse() → bool

Determine if this op can be fused.

This operation can be fused and computed in float when a single integer tensor generates both the operands. For example in the formula: f(x) = x || (x + 1) where x is an integer tensor.

Returns:

• bool: Can fuse

---

classmethod must_quantize_input

must_quantize_input(input_name_or_idx: int) → bool

Determine if an input must be quantized.

Quantized ops and numpy onnx ops take inputs and attributes. Inputs can be either constant or variable (encrypted). Note that this does not handle attributes, which are handled by QuantizedOp classes separately in their constructor.

Args:

• input_name_or_idx (int): Index of the input to check.

Returns:

• result (bool): Whether the input must be quantized (must be a QuantizedArray) or if it stays as a raw numpy.array read from ONNX.

---

classmethod op_type

op_type()

Get the type of this operation.

Returns:

• op_type (str): The type of this operation, in the ONNX referential

---

method prepare_output

prepare_output(qoutput_activation: ndarray) → QuantizedArray

Quantize the output of the activation function.

The calibrate method needs to be called with sample data before using this function.

Args:

• qoutput_activation (numpy.ndarray): Output of the activation function.

Returns:

• QuantizedArray: Quantized output.

---

method q_impl

q_impl(*q_inputs: QuantizedArray, **attrs) → QuantizedArray

Execute the quantized forward.

Args:

• *q_inputs (QuantizedArray): Quantized inputs.

• **attrs: the QuantizedOp attributes.

Returns:

• QuantizedArray: The returned quantized value.

---

class QuantizedMixingOp

An operator that mixes (adds or multiplies) together encrypted inputs.

Mixing operators cannot be fused to TLUs.

method __init__

__init__(
    n_bits_output: int,
    int_input_names: Optional[Set[str]] = None,
    constant_inputs: Optional[Dict[str, Any], Dict[int, Any]] = None,
    input_quant_opts: Optional[QuantizationOptions] = None,
    **attrs
) → None

---

property int_input_names

Get the names of encrypted integer tensors that are used by this op.

Returns:

• List[str]: the names of the tensors

---

method calibrate

calibrate(*inputs: ndarray) → ndarray

Create corresponding QuantizedArray for the output of the activation function.

Args:

• *inputs (numpy.ndarray): Calibration sample inputs.

Returns:

• numpy.ndarray: the output values for the provided calibration samples.

---

method call_impl

call_impl(*inputs: Optional[ndarray, QuantizedArray], **attrs) → ndarray

Call self.impl to centralize mypy bug workaround.

Args:

• *inputs (numpy.ndarray): real valued inputs.

• **attrs: the QuantizedOp attributes.

Returns:

• numpy.ndarray: return value of self.impl

---

method can_fuse

can_fuse() → bool

Determine if this op can be fused.

Mixing operations cannot be fused since it must be performed over integer tensors and it combines different encrypted elements of the input tensors. Mixing operations are Conv, MatMul, etc.

Returns:

• bool: False, this operation cannot be fused as it adds different encrypted integers

---

method make_output_quant_parameters

make_output_quant_parameters(
    q_values: Union[ndarray, Any],
    scale: float64,
    zero_point: Union[int, float, ndarray]
) → QuantizedArray

Build a quantized array from quantized integer results of the op and quantization params.

Args:

• q_values (Union[numpy.ndarray, Any]): the quantized integer values to wrap in the QuantizedArray

• scale (float): the pre-computed scale of the quantized values

• zero_point (Union[int, float, numpy.ndarray]): the pre-computed zero_point of the q_values

Returns:

• QuantizedArray: the quantized array that will be passed to the QuantizedModule output.

---

classmethod must_quantize_input

must_quantize_input(input_name_or_idx: int) → bool

Determine if an input must be quantized.

Quantized ops and numpy onnx ops take inputs and attributes. Inputs can be either constant or variable (encrypted). Note that this does not handle attributes, which are handled by QuantizedOp classes separately in their constructor.

Args:

• input_name_or_idx (int): Index of the input to check.

Returns:

• result (bool): Whether the input must be quantized (must be a QuantizedArray) or if it stays as a raw numpy.array read from ONNX.

---

classmethod op_type

op_type()

Get the type of this operation.

Returns:

• op_type (str): The type of this operation, in the ONNX referential

---

method prepare_output

prepare_output(qoutput_activation: ndarray) → QuantizedArray

Quantize the output of the activation function.

The calibrate method needs to be called with sample data before using this function.

Args:

• qoutput_activation (numpy.ndarray): Output of the activation function.

Returns:

• QuantizedArray: Quantized output.

---

method q_impl

q_impl(*q_inputs: QuantizedArray, **attrs) → QuantizedArray

Execute the quantized forward.

Args:

• *q_inputs (QuantizedArray): Quantized inputs.

• **attrs: the QuantizedOp attributes.

Returns:

• QuantizedArray: The returned quantized value.