torch.ao.quantization.observer.MinMaxObserver#

class torch.ao.quantization.observer.MinMaxObserver(dtype=torch.quint8, qscheme=torch.per_tensor_affine, reduce_range=False, quant_min=None, quant_max=None, factory_kwargs=None, memoryless=False)[源代码]#

Observer module for computing the quantization parameters based on the running min and max values.

This observer uses the tensor min/max statistics to compute the quantization parameters. The module records the running minimum and maximum of incoming tensors, and uses this statistic to compute the quantization parameters.

参数

  • dtype – Quantized data type

  • qscheme – Quantization scheme to be used

  • reduce_range – Reduces the range of the quantized data type by 1 bit

  • quant_min – Minimum quantization value. If unspecified, it will follow the 8-bit setup.

  • quant_max – Maximum quantization value. If unspecified, it will follow the 8-bit setup.

  • memoryless – Boolean that controls whether observer removes old data when a new input is seen. This is most useful for simulating dynamic quantization, especially during QAT.

Given running min/max as \(x_\text{min}\) and \(x_\text{max}\), scale \(s\) and zero point \(z\) are computed as:

The running minimum/maximum \(x_\text{min/max}\) is computed as:

\[\begin{split}\begin{array}{ll} x_\text{min} &= \begin{cases} \min(X) & \text{if~}x_\text{min} = \text{None} \\ \min\left(x_\text{min}, \min(X)\right) & \text{otherwise} \end{cases}\\ x_\text{max} &= \begin{cases} \max(X) & \text{if~}x_\text{max} = \text{None} \\ \max\left(x_\text{max}, \max(X)\right) & \text{otherwise} \end{cases}\\ \end{array}\end{split}\]

where \(X\) is the observed tensor.

The scale \(s\) and zero point \(z\) are then computed as:

\[\begin{split}\begin{aligned} \text{if Symmetric:}&\\ &s = 2 \max(|x_\text{min}|, x_\text{max}) / \left( Q_\text{max} - Q_\text{min} \right) \\ &z = \begin{cases} 0 & \text{if dtype is qint8} \\ 128 & \text{otherwise} \end{cases}\\ \text{Otherwise:}&\\ &s = \left( x_\text{max} - x_\text{min} \right ) / \left( Q_\text{max} - Q_\text{min} \right ) \\ &z = Q_\text{min} - \text{round}(x_\text{min} / s) \end{aligned}\end{split}\]

where \(Q_\text{min}\) and \(Q_\text{max}\) are the minimum and maximum of the quantized data type.

警告

dtype can only take torch.qint8 or torch.quint8.

备注

If the running minimum equals to the running maximum, the scale and zero_point are set to 1.0 and 0.

上一页

torch.ao.quantization.observer._ObserverBase

下一页

torch.ao.quantization.observer.MovingAverageMinMaxObserver