FTVMQnnCanonicalize

FTVMQnnCanonicalize#

源码:tvm/include/tvm/relay/qnn/transform.h

/*!
 * \brief Legalizes a QNN expr. Contains specifically two types of Legalizations. First,
 * converts/Lowers an expression containing QNN ops to an expression containing only core Relay ops.
 * Each QNN op is lowered to a sequence of exisiting Relay ops. This is a target-independent pass.
 * One can register the lowering/transformation function for this op using FTVMQnnCanonicalize
 * attr_name for FTVMLegalize op attribute. Second, as opposed to Relay Legalize, this one legalizes
 * only QNN ops. One can register a transformation/legalization function for an op by using the
 * FTVMQnnLegalize attr_name for FTVMLegalize op attribute. The isolation of QNN and Relay Legalize
 * gives us separation of concerns, leading to a better software practice. The legalization can be
 * configured to happen per target.
 *
 * \return The pass.
 */
TVM_DLL Pass Legalize();

这段代码是一个名为 Legalize 的函数,它的作用是将 QNN 表达式合法化。具体来说,它包含两种类型的合法化:

  1. 将包含 QNN 算子的表达式转换为仅包含核心 Relay 算子的表达式。每个 QNN 算子都会被转换为一系列现有的 Relay 算子。这是一个与目标 target 无关的传递。可以使用 FTVMQnnCanonicalize 属性名称为 FTVMLegalize 算子属性注册 transformation/legalization 函数。

  2. 与 Relay Legalize 不同,这个函数只对 QNN 算子进行合法化。可以通过使用 FTVMQnnLegalize 属性名称为 FTVMLegalize 算子属性注册一个算子的 transformation/legalization 函数。QNN 和 Relay Legalize 的隔离使我们能够更好地分离关注点,从而得到更好的软件实践。合法化可以针对每个目标(target)进行配置。

函数返回 Pass 对象。

CanonicalizeOps()#

源码:tvm/python/tvm/relay/qnn/transform.py

import testing
from tvm.relay.qnn.transform import CanonicalizeOps

CanonicalizeOps?
Signature: CanonicalizeOps()
Docstring:
Converts/Lowers an expression containing QNN ops to an expression containing only core
(non-Dialect) Relay ops. Each QNN op is lowered to a sequence of existing Relay ops. This is a
target-independent pass. One can register the lowering/transformation function for this op using
FTVMQnnCanonicalize attr_name for FTVMLegalize op attribute.  An example of this transformation
is below

Examples
________

.. code-block:: python

    # Original expression
    qnn_expr = relay.qnn.op.requantize(y,
                                       input_scale=1,
                                       input_zero_point=0,
                                       output_scale=1,
                                       output_zero_point=0,
                                       out_dtype='int8')

    # We want to utilize all the existing Relay infrastructure. So, instead of supporting this
    # QNN requantize op, we convert it into a sequence of existing Relay operators.
    mod = tvm.IRModule.from_expr(qnn_expr)
    mod = relay.qnn.transform.CanonicalizeOps()(mod)
    relay_expr = mod['main']
    print(relay_expr)

    def @main(%quantized_data: Tensor[(200), int32]) -> Tensor[(200), int8] {
      %0 = cast(%quantized_data, dtype="int64") /* ty=Tensor[(200), int64] */;
      %1 = multiply(%0, 2 /* ty=int64 */) /* ty=Tensor[(200), int64] */;
      %2 = multiply(%1, 1073741824 /* ty=int64 */) /* ty=Tensor[(200), int64] */;
      %3 = add(%2, 1073741824 /* ty=int64 */) /* ty=Tensor[(200), int64] */;
      %4 = right_shift(%3, 31 /* ty=int64 */) /* ty=Tensor[(200), int64] */;
      %5 = add(0 /* ty=int64 */, %4) /* ty=Tensor[(200), int64] */;
      %6 = clip(%5, a_min=-128f, a_max=127f) /* ty=Tensor[(200), int64] */;
      cast(%6, dtype="int8") /* ty=Tensor[(200), int8] */
    }

Returns
-------
ret : tvm.transform.Pass
    The registered pass that canonicalizes QNN ops to Relay ops.
File:      /media/pc/data/lxw/ai/tvm/python/tvm/relay/qnn/transform.py
Type:      function

这段代码定义了一个名为 CanonicalizeOps 的函数,它的作用是将包含 QNN 算子的表达式转换为仅包含核心(非 Dialect)Relay 算子的表达式。每个 QNN 算子都会被转换为一系列现有的 Relay 算子。这是一个与目标无关的传递。

函数返回 tvm.transform.Pass 对象,该对象将 QNN 算子规范化为 Relay 算子。

可以使用 FTVMQnnCanonicalize 属性名称为 FTVMLegalize 算子属性注册 lowering/transformation 函数。

from tvm.relay.qnn.op import register_qnn_canonicalize

register_qnn_canonicalize?
Signature: register_qnn_canonicalize(op_name, legal_op=None, level=10)
Docstring:
Register canonicalization function for a QNN op.

This transforms QNN ops to mainline Relay components.

Parameters
----------
op_name : str
    The name of the operator

legal_op: function (Attrs, List[Expr], List[relay.Type]) -> Expr
    The function for transforming an expr to another expr.

level : int
    The priority level
File:      /media/pc/data/lxw/ai/tvm/python/tvm/relay/qnn/op/op.py
Type:      function