tvm.relay.frontend#

Frontends for constructing Relay programs.

Contains the model importers currently defined for Relay.

Classes:

ChangeDatatype(src, dst)

Mutator for changing the datatype of Relay programs.

Functions:

`from_caffe`(init_net, predict_net, ...)	Convert from caffe model into compatible relay Function.
`from_caffe2`(init_net, predict_net[, shape, ...])	Load caffe2 graph which contains init_net and predict_net into Relay Function.
`from_coreml`(model[, shape])	Convert from coreml model into Relay Function.
`from_darknet`(net[, shape, dtype])	Convert from Darknet's model into compatible relay Function.
`from_keras`(model[, shape, layout])	Convert keras model to relay Function.
`from_mxnet`(symbol[, shape, dtype, ...])	Convert from MXNet"s model into compatible relay Function.
`from_oneflow`(graph, model_dir_path)	Convert a OneFlow model into an equivalent Relay Function.
`from_onnx`(model[, shape, dtype, opset, ...])	Convert a ONNX model into an equivalent Relay Function.
`from_paddle`(program_or_layer[, shape_dict, ...])	Convert a PaddlePaddle model into an equivalent Relay Function.
`from_pytorch`(script_module, input_infos[, ...])	Load PyTorch model in the form of a scripted PyTorch model and convert into relay.
`from_tensorflow`(graph[, layout, shape, ...])	Load tensorflow graph which is a python tensorflow graph object into relay.
`from_tflite`(model[, shape_dict, dtype_dict, ...])	Convert from tflite model into compatible relay Function.
`quantize_conv_bias_mkldnn_from_var`(bias_var, ...)	Quantized conv2d bias

class tvm.relay.frontend.ChangeDatatype(src, dst)[源代码]#

Mutator for changing the datatype of Relay programs.

This pass should be useful for users of the Bring Your Own Datatypes framework. TODO(@gussmith23 @hypercubestart) Add link to documentation when it exists

Example:

from tvm.relay.testing.inception_v3 import get_workload
mod, params = get_workload()

def change_dtype(mod, params, src, dst):
    mod = ChangeDatatype(src, dst)(mod)
    params = dict((p, tvm.nd.array(params[p].numpy().astype(dst))) for p in params)
    return mod, params

mod, params = change_dtype(mod, params, "float32", "custom[posites2]32")

Parameters#

srcString: The source datatype name, e.g. “float” or “posites2” (but not “float32” or “custom[posites2]32”).
dstString: The destination datatype name, in the same format.

Returns#

modtvm.IRModule: Module where all nodes of dtype src have been changed to have dtype dst.

tvm.relay.frontend.from_caffe(init_net, predict_net, shape_dict, dtype_dict)[源代码]#

Convert from caffe model into compatible relay Function.

Parameters#

init_netcaffe_pb2.NetParameter: caffemodel
predict_netcaffe_pb2.NetParameter: caffe prototxt
shape_dictdict of str to int list/tuple: Input shapes of the model.
dtype_dictdict of str to str: Input types of the model.

Returns#

modtvm.IRModule: The relay module for compilation.
paramsdict of str to tvm.NDArray: The parameter dict to be used by relay

tvm.relay.frontend.from_caffe2(init_net, predict_net, shape=None, dtype='float32')[源代码]#

Load caffe2 graph which contains init_net and predict_net into Relay Function.

Parameters#

init_netprotobuf object: Caffe2 NetDef containing the weights
predict_netprotobuf object: Caffe2 NetDef containing the graph
shapedict of str to tuple: The input shape to the graph
dtypestr or dict of str to str: The input types to the graph

Returns#

modtvm.IRModule: The module that optimizations will be performed on.
paramsdict of str to tvm.nd.NDArray: Dict of converted parameters stored in tvm.nd.NDArray format

tvm.relay.frontend.from_coreml(model, shape=None)[源代码]#

Convert from coreml model into Relay Function.

Parameters#

model:: coremltools.models.MLModel of a NeuralNetworkClassifier
shapedict of str to int list/tuple, optional: The input shapes

Returns#

modtvm.IRModule: The relay module for compilation.
paramsdict of str to tvm.nd.NDArray: The parameter dict to be used by Relay.

tvm.relay.frontend.from_darknet(net, shape=None, dtype='float32')[源代码]#

Convert from Darknet’s model into compatible relay Function.

Parameters#

netDarknet net parameter: Darknet net structure.
shapedict of str to tuple, optional: The input shape to the graph
dtypestr or dict of str to str: The input types to the graph

Returns#

modtvm.IRModule: The relay module for compilation.
paramsdict of str to tvm.nd.NDArray: The parameter dict to be used by relay

tvm.relay.frontend.from_keras(model, shape=None, layout='NCHW')[源代码]#

Convert keras model to relay Function.

Parameters#

modelkeras.engine.training.Model or tensorflow.keras.models.Model: The keras model to be converted.
shape: dict of str to int list/tuple: Input shapes of the model, optional
layout: str: One of ‘NWC’, ‘NCHW’, ‘NHWC’, ‘NDHWC’ indicates how data should be arranged in the output model. Default layout is ‘NCHW’ as it in general performs better across TVM.

Returns#

modtvm.IRModule: The relay module for compilation.
paramsdict of str to tvm.nd.NDArray: The parameter dict to be used by Relay.

tvm.relay.frontend.from_mxnet(symbol, shape=None, dtype='float32', arg_params=None, aux_params=None)[源代码]#

Convert from MXNet”s model into compatible relay Function.

Parameters#

symbolmxnet.Symbol or mxnet.gluon.HybridBlock: MXNet symbol.
shapedict of str to tuple, optional: The input shape to the graph
dtypestr or dict of str to str: The input types to the graph
arg_paramsdict of str to mx.NDArray: The argument parameters in mxnet
aux_paramsdict of str to mx.NDArray: The auxiliary parameters in mxnet

Returns#

modtvm.IRModule: The relay module for compilation
paramsdict of str to tvm.nd.NDArray: The parameter dict to be used by nnvm

tvm.relay.frontend.from_oneflow(graph, model_dir_path)[源代码]#

Convert a OneFlow model into an equivalent Relay Function.

At present, there are two ways to run models in deep learning framework Dynamic Graph and Static Graph, which are also called Eager Mode and Graph Mode in OneFlow.

In general, dynamic graphs are easier to use and static graphs have better performance. OneFlow offers nn.Graph, so that users can use the eager-like programming style to build static graphs and train the models.

We utilize the intermediate representation of nn.Graph to convert the OneFlow model to Reley.

Parameters#

nodesdict, keys: node.name, value: node: contain the graph
model_dir_path: str: The path of weight

Returns#

modtvm.IRModule: The returned relay module
paramsdict: A dict of name: tvm.nd.array pairs, used as pretrained weights

tvm.relay.frontend.from_onnx(model, shape=None, dtype='float32', opset=None, freeze_params=True, convert_config=None, export_node_renamed_model_path=None)[源代码]#

Convert a ONNX model into an equivalent Relay Function.

ONNX graphs are represented as Python Protobuf objects. The companion parameters will be handled automatically. However, the input names from onnx graph is vague, mixing inputs and network weights/bias such as “1”, “2”… For convenience, we rename the real input names to “input_0”, “input_1”… And renaming parameters to “param_0”, “param_1”…

By default, ONNX defines models in terms of dynamic shapes. The ONNX importer retains that dynamism upon import, and the compiler attempts to convert the model into a static shapes at compile time. If this fails, there may still be dynamic operations in the model. Not all TVM kernels currently support dynamic shapes, please file an issue on discuss.tvm.apache.org if you hit an error with dynamic kernels.

Parameters#

modelprotobuf object

ONNX ModelProto after ONNX v1.1.0

shapedict of str to tuple, optional

The input shape to the graph

dtypestr or dict of str to str

The input types to the graph

opsetint, optional

Override to autodetected opset. This can be helpful for some testing.

freeze_params: bool

If this parameter is true, the importer will take any provided onnx input values (weights, shapes, etc) and embed them into the relay model as Constants instead of variables. This allows more aggressive optimizations at compile time and helps in making models static if certain inputs represent attributes relay would traditionally consider compile-time constants.

convert_configOptional[Dict[str, Any]]

Default config:

use_nt_batch_matmulbool = True: True to convert qualified onnx matmul to nn.batch_matmul strict to NT format (transpose_a=False, transpose_b=True).

export_node_renamed_model_pathstr, optional

Export the node renamed onnx model to the path. Some models do not contain names in their nodes. During the conversion, if names of nodes are empty, new names will be assigned based on their op types. The exported model can be the reference to spans.

Returns#

modtvm.IRModule: The relay module for compilation
paramsdict of str to tvm.nd.NDArray: The parameter dict to be used by relay

tvm.relay.frontend.from_paddle(program_or_layer, shape_dict=None, scope=None)[源代码]#

Convert a PaddlePaddle model into an equivalent Relay Function. PaddlePaddle Program/TranslatedLayer represent the computation graph of PaddlePaddle model, and PaddlePaddle scope stores all the weights of PaddlePaddle model.

Parameters#

program_or_layerobject of paddle.static.Program or paddle.jit.TranslatedLayer: Loaded model by paddle.static.load_inference_model or paddle.jit.load
shape_dictdict of str to tuple/list, optional: The input shape of model
scopeobject of paddle.static.Scope, optional: The scope that saves all the weights of model, use paddle.static.global_scope by default

Returns#

modtvm.IRModule: The relay module for compilation

params : dict of str to tvm.nd.NDArray

tvm.relay.frontend.from_pytorch(script_module, input_infos, custom_convert_map=None, default_dtype='float32', use_parser_friendly_name=False, keep_quantized_weight=False, export_renamed_c_graph_path=None, preserve_pytorch_scopes=False)[源代码]#

Load PyTorch model in the form of a scripted PyTorch model and convert into relay. The companion parameters will be handled automatically.

Parameters#

script_moduleTopLevelTracedModule object: TorchScripted PyTorch graph Note: We currently only support traces (ie: torch.jit.trace(model, input))
input_infosList of tuples: Can be (input name, input shape) or (input name, (input shape, input types)) Graph level input shape and type list The same input names need to be used for deployment, so choose easy to remember names (such as: input0, input1) e.g. [(‘input0’, (1, 2)), (‘input1’, (3, 4))] or [(‘input0’, ((1, 2), ‘int’)), (‘input1’, ((3, 4), ‘float’))]
custom_convert_mapDictionary of str to Relay op: A custom op conversion map in the same format as _convert_map above
default_typestr: The default dtype to use when type information is not provided by PyTorch.
use_parser_friendly_namebool: When True, replace ‘.’ with `_’ in a original parameter name. The Relay text parser treats a variable name followed by a period as a tuple element access, so a variable name like “dense.weight” cannot be parsed correctly. Use this option when you want to run the AnnotateSpans pass on the imported module.
keep_quantized_weightbool: Return quantized weights and bias, rather than float ones. PyTorch stores quantized weights in a custom format, so we cannot directly access 8 bit weights as Numpy arrays. We use a PyTorch function to unpack quantized weights into float32 arrays and quantization parameters. By default, we return float32 weights and rely on the QNN lowering and the Relay constant folding pass to quantize weights at compile time. In BYOC use cases, however, we cannot apply the constant folding pass on a QNN graph. If keep_quantized_weight is True, we quantize weights in the frontend using a function that is equivalent to qnn.op.quantize(…) operating on Numpy arrays.
export_renamed_c_graph_pathstr, optional: Export the renamed torch._C.Graph to the path. During the conversion, variable names in torch._C.Graph will be assigned based on their op types. The exported text file can be the reference to spans.
preserve_pytorch_scopesbool: When naming the nodes in the Relay graph, use the “scope name” from the Pytorch model. If false, a default namer is used that does not preserve the Pytorch scope names.

Returns#

modtvm.IRModule: The module that optimizations will be performed on.
paramsdict of str to tvm.runtime.NDArray: Dict of converted parameters stored in tvm.runtime.ndarray format

tvm.relay.frontend.from_tensorflow(graph, layout='NHWC', shape=None, outputs=None, convert_config=None)[源代码]#

Load tensorflow graph which is a python tensorflow graph object into relay. The companion parameters will be handled automatically.

Parameters#

graphGraphDef object

Tensorflow GraphDef

layouttarget layout to be used (Optional)

NCHW only supported now to enable NHWC models on GPU.

shapeDictionary of input dimensions (Optional)

Graph level input shape dictionary.

outputsList of output tensor names (Optional)

if not specified then the last node is assumed as graph output.

convert_configOptional[Dict[str, Any]]

Default config:

use_densebool = True: Ture to convert tf.matmul to nn.dense, else to nn.matmul. The nn.dense op requires the data tensor to be non-transposed and weight tensor to be transposed, may insert extra transpose to the original graph.
use_nt_batch_matmulbool = True: True to convert tf.batch_matmul to nn.batch_matmul strict to NT format (transpose_a=False, transpose_b=True).

Returns#

modtvm.IRModule: The module that optimizations will be performed on.
paramsdict of str to tvm.nd.NDArray: Dict of converted parameters stored in tvm.nd.NDArray format

tvm.relay.frontend.from_tflite(model, shape_dict=None, dtype_dict=None, op_converter=<class 'tvm.relay.frontend.tflite.OperatorConverter'>)[源代码]#

Convert from tflite model into compatible relay Function.

Parameters#

model:: tflite.Model or tflite.Model.Model (depending on tflite version)
shape_dictdict of str to int list/tuple: Input shapes of the model.
dtype_dictdict of str to str: Input types of the model.

Returns#

modtvm.IRModule: The relay module for compilation.
paramsdict of str to tvm.nd.NDArray: The parameter dict to be used by relay

tvm.relay.frontend.quantize_conv_bias_mkldnn_from_var(bias_var, bias_scale)[源代码]#: Quantized conv2d bias

tvm.relay.frontend

目录

tvm.relay.frontend#

Parameters#

Returns#

Parameters#

Returns#

Parameters#

Returns#

Parameters#

Returns#

Parameters#

Returns#

Parameters#

Returns#

Parameters#

Returns#

Parameters#

Returns#

Parameters#

Returns#

Parameters#

Returns#

Parameters#

Returns#

Parameters#

Returns#

Parameters#

Returns#