tvm.runtime

TVM runtime namespace.

Classes:

DataType(type_str)	TVM datatype structure
DataTypeCode()	DataType code in DLTensor.
Device(device_type, device_id)	TVM device strucure.
Module(handle)	Runtime Module.
Object()	Base class for all tvm's runtime objects.
ObjectGeneric()	Base class for all classes that can be converted to object.
ObjectPath()	Path to an object from some root object.
ObjectPathPair()	Pair of ObjectPaths, one for each object being tested for structural equality.
PackedFunc(handle, is_global)	The PackedFunc object used in TVM.
Report(calls, device_metrics, configuration)	A container for information gathered during a profiling run.
Scriptable()	A base class that enables the script() and show() method.
ShapeTuple(shape)	TVM runtime ShapeTuple object. Parameters ---------- shape : list[int] The shape list used to construct the object.
String(content)	TVM runtime.String object, represented as a python str.

Functions:

cl([dev_id])	Construct a OpenCL device
const(value[, dtype, span])	construct a constant
convert(value[, span])	Convert value to TVM object or function.
convert_to_object(value[, span])	Convert a Python value to corresponding object type.
cpu([dev_id])	Construct a CPU device
cuda([dev_id])	Construct a CUDA GPU device
device(dev_type[, dev_id])	Construct a TVM device with given device type and id.
enabled(target)	Whether module runtime is enabled for target
ext_dev([dev_id])	Construct a extension device
gpu([dev_id])	Construct a CUDA GPU device
load_module(path[, fmt])	Load module from file.
load_param_dict(param_bytes)	Load parameter dictionary from binary bytes.
load_param_dict_from_file(path)	Load parameter dictionary from file.
load_static_library(path, func_names)	Load the .o library at path which implements functions with func_names.
metal([dev_id])	Construct a metal device
mtl([dev_id])	Construct a metal device
num_threads()	Get the number of threads in use by the TVM runtime.
opencl([dev_id])	Construct a OpenCL device
rocm([dev_id])	Construct a ROCM device
save_param_dict(params)	Save parameter dictionary to binary bytes.
save_param_dict_to_file(params, path)	Save parameter dictionary to file.
system_lib([symbol_prefix])	Get system-wide library module singleton.
vpi([dev_id])	Construct a VPI simulated device
vulkan([dev_id])	Construct a Vulkan device

class tvm.runtime.DataType(type_str)

TVM datatype structure

Methods:

itemsize()

Get the number of bytes of a single element of this data type.

itemsize()

Get the number of bytes of a single element of this data type. When the number of lanes is greater than 1, the itemsize is the size of the vector type.

Returns

itemsizeint

The number of bytes of a single element of this data type

class tvm.runtime.DataTypeCode

DataType code in DLTensor.

class tvm.runtime.Device(device_type, device_id)

TVM device strucure.

Typically constructed using convenience function tvm.runtime.device().

Exposes uniform interface to device-specific APIs such as CUDA or OpenCL. Some properties may return None depending on whether an API exposes that particular property.

NOTE! The integer values in MASK2STR and STR2MASK must correspond to the values provided by the DLDeviceType and TVMDeviceExtType enums.

Methods:

_GetDeviceAttr(device_type, device_id, attr_id)	Internal helper function to invoke runtime.GetDeviceAttr
create_raw_stream()	Create a new runtime stream at the context.
free_raw_stream(stream)	Free a created stream handle.
set_raw_stream(stream)	Set a created stream handle.
sync([stream])	Synchronize until jobs finished at the context.
texture_spatial_limit()	Returns limits for textures by spatial dimensions

Attributes:

api_version	Returns version number of the SDK used to compile TVM.
compute_version	Get compute version number as string.
device_name	Return the vendor-specific name of device.
driver_version	Returns version number of the driver
exist	Whether this device exists.
l2_cache_size_bytes	Return the size of the device L2 cache in bytes
max_clock_rate	Return the max clock frequency of device (kHz).
max_shared_memory_per_block	Total amount of shared memory per block in bytes.
max_thread_dimensions	Return the maximum size of each thread axis
max_threads_per_block	Maximum number of threads on each block.
multi_processor_count	Return the number of compute units in the device.
total_global_memory	Return size of the total global memory.
warp_size	Number of threads that execute concurrently.

_GetDeviceAttr(device_type, device_id, attr_id)

Internal helper function to invoke runtime.GetDeviceAttr

create_raw_stream()

Create a new runtime stream at the context.

User should free the stream after use.

Returns

streamTVMStreamHandle

The created runtime stream.

free_raw_stream(stream)

Free a created stream handle.

Parameters

streamTVMStreamHandle

The stream which should to be released.

set_raw_stream(stream)

Set a created stream handle.

Parameters

streamTVMStreamHandle

The stream which should to be set to the device.

sync(stream=None)

Synchronize until jobs finished at the context.

Parameters

streamTVMStreamHandle

Jobs in this stream should be finished.

texture_spatial_limit()

Returns limits for textures by spatial dimensions

Returns

limitint or None

Maximum size of the texture by spatial dimensions

property api_version

Returns version number of the SDK used to compile TVM.

For example, CUDA_VERSION for cuda or VK_HEADER_VERSION for Vulkan.

Returns device value for cuda, rocm, opencl, and vulkan. Returns remote device value for RPC devices. Returns None for all other devices.

Returns

versionint or None

The version of the SDK

property compute_version

Get compute version number as string.

Returns maximum API version (e.g. CUDA/OpenCL/Vulkan) supported by the device.

Returns device value for cuda, rocm, opencl, and vulkan. Returns remote device value for RPC devices. Returns None for all other devices.

Returns

versionstr or None

The version string in major.minor format.

property device_name

Return the vendor-specific name of device.

Returns device value for cuda, rocm, opencl, and vulkan. Returns remote device value for RPC devices. Returns None for all other devices.

Returns

device_namestr or None

The name of the device.

property driver_version

Returns version number of the driver

Returns driver vendor’s internal version number. (e.g. “450.408.256” for nvidia-driver-450)

Returns device value for opencl and vulkan. Returns remote device value for RPC devices. Returns None for all other devices.

Returns

versionstr or None

The version string in major.minor.patch format.

property exist

Whether this device exists.

Returns True if TVM has support for the device, if the physical device is present, and the device is accessible through appropriate drivers (e.g. cuda/vulkan).

Returns

existbool

True if the device exists

property l2_cache_size_bytes

Return the size of the device L2 cache in bytes

Supported devices include CUDA/ROCM/OpenCL.

Returns

l2_cache_size_bytesint or None

The size of the device L2 cache in bytes returned by device runtime API. Return None if the device does not support this feature.

Note

The value returned by opencl’s API is smaller than actual device L2 cache size.

property max_clock_rate

Return the max clock frequency of device (kHz).

Returns device value for cuda, rocm, and opencl. Returns remote device value for RPC devices. Returns None for all other devices.

Returns

max_clock_rateint or None

The maximum clock frequency of the device (kHz)

property max_shared_memory_per_block

Total amount of shared memory per block in bytes.

Returns device value for cuda, rocm, opencl, and vulkan. Returns remote device value for RPC devices. Returns None for all other devices.

Returns

max_shared_memory_per_blockint or None

Total amount of shared memory per block in bytes

property max_thread_dimensions

Return the maximum size of each thread axis

Returns device value for cuda, rocm, opencl, and vulkan. Returns remote device value for RPC devices. Returns None for all other devices.

Returns

dims: List of int, or None

The maximum length of threadIdx.x, threadIdx.y, threadIdx.z

property max_threads_per_block

Maximum number of threads on each block.

Returns device value for cuda, metal, rocm, opencl, and vulkan devices. Returns remote device value for RPC devices. Returns None for all other devices.

Returns

max_threads_per_blockint or None

The number of threads on each block

property multi_processor_count

Return the number of compute units in the device.

Returns device value for cuda, rocm, and opencl. Returns remote device value for RPC devices. Returns None for all other devices.

Returns

multi_processor_countint or None

Thee number of compute units in the device

property total_global_memory

Return size of the total global memory.

Supported devices include CUDA/ROCm/Metal/OpenCL.

Returns

total_global_memoryint or None

Return the global memory available on device in bytes. Return None if the device does not support this feature.

property warp_size

Number of threads that execute concurrently.

Returns device value for cuda, rocm, and vulkan. Returns 1 for metal and opencl devices, regardless of the physical device. Returns remote device value for RPC devices. Returns None for all other devices.

Returns

warp_sizeint or None

Number of threads that execute concurrently

class tvm.runtime.Module(handle)

Runtime Module.

Methods:

_collect_from_import_tree(filter_func)	Helper function to collect modules from the tree matching a filter_func, then return it.
clear_imports()	Remove all imports of the module.
export_library(file_name, *[, fcompile, ...])	Export the module and all imported modules into a single device library.
get_function(name[, query_imports])	Get function from the module.
get_property_mask()	Get the runtime module property mask.
get_source([fmt])	Get source code from module, if available.
implements_function(name[, query_imports])	Returns True if the module has a definition for the global function with name.
import_module(module)	Add module to the import list of current one.
save(file_name[, fmt])	Save the module to file.
time_evaluator(func_name, dev[, number, ...])	Get an evaluator that measures time cost of running function.

Attributes:

entry_func	Get the entry function
format	Get the format of the module.
imported_modules	Get imported modules
is_binary_serializable	Returns true if module is 'binary serializable', ie can be serialzed into binary
is_dso_exportable	Returns true if module is 'DSO exportable', ie can be included in result of export_library by the external compiler directly.
is_runnable	Returns true if module is 'runnable'.
type_key	Get type key of the module.

_collect_from_import_tree(filter_func)

Helper function to collect modules from the tree matching a filter_func, then return it.

Parameters

filter_funcCallable[[Module], bool]

A function which is invoked for each Module discovered in the import tree (including self).

Returns

list[Module] :

A list of matching Module.

clear_imports()

Remove all imports of the module.

export_library(file_name, *, fcompile=None, fpack_imports=None, addons=None, workspace_dir=None, **kwargs)

Export the module and all imported modules into a single device library.

This function only works on host LLVM modules, other runtime::Module subclasses will work with this API but they must support implement the save and load mechanisms of modules completely including saving from streams and files. This will pack your non-shared library module into a single shared library which can later be loaded by TVM.

Parameters

file_namestr

The name of the shared library.

fcompilefunction(target, file_list, kwargs), optional

The compilation function to use create the final library object during export.

For example, when fcompile=_cc.create_shared, or when it is not supplied but module is “llvm,” this is used to link all produced artifacts into a final dynamic library.

This behavior is controlled by the type of object exported. If fcompile has attribute object_format, will compile host library to that format. Otherwise, will use default format “o”.

fpack_imports: function(mod: runtime.Module, is_system_lib: bool, symbol_prefix: str,

workspace_dir: str) -> str

Function used to pack imported modules from mod into a file suitable for passing to fcompile as an input file. The result can be a C source, or an .o object file, or any other file that the fcompile function can handle. The function returns the name of the created file.

If not provided, the imported modules will be serialized either via packing to an LLVM module, or to a C source file.

workspace_dirstr, optional

The path of the directory used to create the intermediate artifacts when exporting the module. If this is not provided a temporary dir will be created.

kwargsdict, optional

Additional arguments passed to fcompile

Returns

result of fcompile()unknown, optional

If the compilation function returns an artifact it would be returned via export_library, if any.

get_function(name, query_imports=False)

Get function from the module.

Parameters

namestr

The name of the function

query_importsbool

Whether also query modules imported by this module.

Returns

ftvm.runtime.PackedFunc

The result function.

get_property_mask()

Get the runtime module property mask. The mapping is stated in ModulePropertyMask.

Returns

maskint

Bitmask of runtime module property

get_source(fmt='')

Get source code from module, if available.

Parameters

fmtstr, optional

The specified format.

Returns

sourcestr

The result source code.

implements_function(name, query_imports=False)

Returns True if the module has a definition for the global function with name. Note that has_function(name) does not imply get_function(name) is non-null since the module may be, eg, a CSourceModule which cannot supply a packed-func implementation of the function without further compilation. However, get_function(name) non null should always imply has_function(name).

Parameters

namestr

The name of the function

query_importsbool

Whether to also query modules imported by this module.

Returns

bBool

True if module (or one of its imports) has a definition for name.

import_module(module)

Add module to the import list of current one.

Parameters

moduletvm.runtime.Module

The other module.

save(file_name, fmt='')

Save the module to file.

This do not save the dependent device modules. See also export_shared

Parameters

file_namestr

The name of the file.

fmtstr

The format of the file.

See Also

runtime.Module.export_library : export the module to shared library.

time_evaluator(func_name, dev, number=10, repeat=1, min_repeat_ms=0, limit_zero_time_iterations=100, cooldown_interval_ms=0, repeats_to_cooldown=1, cache_flush_bytes=0, f_preproc='')

Get an evaluator that measures time cost of running function.

Parameters

func_name: str

The name of the function in the module.

dev: Device

The device we should run this function on.

number: int

The number of times to run this function for taking average. We call these runs as one repeat of measurement.

repeat: int, optional

The number of times to repeat the measurement. In total, the function will be invoked (1 + number x repeat) times, where the first one is warm up and will be discarded. The returned result contains repeat costs, each of which is an average of number costs.

min_repeat_ms: int, optional

The minimum duration of one repeat in milliseconds. By default, one repeat contains number runs. If this parameter is set, the parameters number will be dynamically adjusted to meet the minimum duration requirement of one repeat. i.e., When the run time of one repeat falls below this time, the number parameter will be automatically increased.

limit_zero_time_iterations: int, optional

The maximum number of repeats when measured time is equal to 0. It helps to avoid hanging during measurements.

cooldown_interval_ms: int, optional

The cooldown interval in milliseconds between the number of repeats defined by repeats_to_cooldown.

repeats_to_cooldown: int, optional

The number of repeats before the cooldown is activated.

cache_flush_bytes: int, optional

The number of bytes to flush from the cache before each repeat.

f_preproc: str, optional

The preprocess function name we want to execute before executing the time evaluator.

Note

The function will be invoked (1 + number x repeat) times, with the first call discarded in case there is lazy initialization.

Returns

ftimerfunction

The function that takes same argument as func and returns a BenchmarkResult. The ProfileResult reports repeat time costs in seconds.

property entry_func

Get the entry function

Returns

ftvm.runtime.PackedFunc

The entry function if exist

property format

Get the format of the module.

property imported_modules

Get imported modules

Returns

moduleslist of Module

The module

property is_binary_serializable

Returns true if module is ‘binary serializable’, ie can be serialzed into binary

stream and loaded back to the runtime module.

Returns

bBool

True if the module is binary serializable.

property is_dso_exportable

Returns true if module is ‘DSO exportable’, ie can be included in result of export_library by the external compiler directly.

Returns

bBool

True if the module is DSO exportable.

property is_runnable

Returns true if module is ‘runnable’. ie can be executed without any extra compilation/linking steps.

Returns

bBool

True if the module is runnable.

property type_key

Get type key of the module.

class tvm.runtime.Object

Base class for all tvm’s runtime objects.

Methods:

_move()

Create an RValue reference to the object and mark the object as moved.

_move()

Create an RValue reference to the object and mark the object as moved.

This is a advanced developer API that can be useful when passing an unique reference to an Object that you no longer needed to a function.

A unique reference can trigger copy on write optimization that avoids copy when we transform an object.

Note

All the reference of the object becomes invalid after it is moved. Be very careful when using this feature.

Examples

x = tvm.tir.Var("x", "int32")
x0 = x
some_packed_func(x._move())
# both x0 and x will points to None after the function call.

Returns

rvalue : The rvalue reference.

class tvm.runtime.ObjectGeneric

Base class for all classes that can be converted to object.

Methods:

asobject()

Convert value to object

asobject()

Convert value to object

class tvm.runtime.ObjectPath

Path to an object from some root object.

class tvm.runtime.ObjectPathPair

Pair of ObjectPaths, one for each object being tested for structural equality.

class tvm.runtime.PackedFunc(handle, is_global)

The PackedFunc object used in TVM.

Function plays an key role to bridge front and backend in TVM. Function provide a type-erased interface, you can call function with positional arguments.

The compiled module returns Function. TVM backend also registers and exposes its API as Functions.

The following are list of common usage scenario of tvm.runtime.PackedFunc.

• Automatic exposure of C++ API into python

• To call PackedFunc from python side

• To call python callbacks to inspect results in generated code

• Bring python hook into C++ backend

See Also

tvm.register_func: How to register global function. tvm.get_global_func: How to get global function.

class tvm.runtime.Report(calls, device_metrics, configuration)

A container for information gathered during a profiling run.

Attributes

callsArray[Dict[str, Object]]

Per-call profiling metrics (function name, runtime, device, …).

device_metricsDict[Device, Dict[str, Object]]

Per-device metrics collected over the entire run.

Methods:

__init__(calls, device_metrics, configuration)	Construct a profiling report from a list of metrics and per-device metrics.
csv()	Convert this profiling report into CSV format.
from_json(s)	Deserialize a report from JSON.
json()	Convert this profiling report into JSON format.
table([sort, aggregate, col_sums])	Generate a human-readable table

__init__(calls, device_metrics, configuration)

Construct a profiling report from a list of metrics and per-device metrics.

Parameters

callsSequence[Dict[str, Object]]

Per function call metrics.

device_metricsDict[str, Dict[str, Object]]

Per device metrics.

configurationDict[str, Object]

Configuration of TVM for this profiling run. Includes number of threads, executor.

参数:

• calls (Sequence[Dict[str, Object]])

• device_metrics (Dict[str, Dict[str, Object]])

• configuration (Dict[str, Object])

csv()

Convert this profiling report into CSV format.

This only includes calls and not overall metrics.

Returns

csvstr

calls in CSV format.

classmethod from_json(s)

Deserialize a report from JSON.

Parameters

sstr

Report serialize via json().

Returns

reportReport

The deserialized report.

json()

Convert this profiling report into JSON format.

Example output:

Returns

jsonstr

Formatted JSON

table(sort=True, aggregate=True, col_sums=True)

Generate a human-readable table

Parameters

sort : bool

If aggregate is true, whether to sort call frames by descending duration. If aggregate is False, whether to sort frames by order of appearancei n the program.

aggregate : bool

Whether to join multiple calls to the same op into a single line.

col_sums : bool

Whether to include the sum of each column.

Returns

table : str

A human-readable table

参数:

• calls (Sequence[Dict[str, Object]])

• device_metrics (Dict[str, Dict[str, Object]])

• configuration (Dict[str, Object])

class tvm.runtime.Scriptable

A base class that enables the script() and show() method.

Methods:

script(*[, name, show_meta, ir_prefix, ...])	Print TVM IR into TVMScript text format
show([style, black_format, name, show_meta, ...])	A sugar for print highlighted TVM script.

script(*, name=None, show_meta=False, ir_prefix='I', tir_prefix='T', relax_prefix='R', module_alias='cls', buffer_dtype='float32', int_dtype='int32', float_dtype='void', verbose_expr=False, indent_spaces=4, print_line_numbers=False, num_context_lines=-1, syntax_sugar=True, show_object_address=False, show_all_struct_info=True, path_to_underline=None, path_to_annotate=None, obj_to_underline=None, obj_to_annotate=None)

Print TVM IR into TVMScript text format

Parameters

nameOptional[str] = None

The name of the object

show_metabool = False

Whether to print the meta data of the object

ir_prefixstr = “I”

The prefix of AST nodes from tvm.ir

tir_prefixstr = “T”

The prefix of AST nodes from tvm.tir

relax_prefixstr = “R”

The prefix of AST nodes from tvm.relax

module_aliasstr = “cls”

The alias of the current module at cross-function call, Directly use module name if it’s empty.

buffer_dtypestr = “float32”

The default data type of buffer

int_dtypestr = “int32”

The default data type of integer

float_dtypestr = “void”

The default data type of float

verbose_exprbool = False

Whether to print the detailed definition of each variable in the expression

indent_spacesint = 4

The number of spaces for indentation

print_line_numbersbool = False

Whether to print line numbers

num_context_linesint = -1

The number of lines of context to print before and after the line to underline.

syntax_sugar: bool = True

Whether to output with syntax sugar, set false for complete printing.

show_object_address: bool = False

Whether to include the object’s address as part of the TVMScript name

show_all_struct_info: bool = True

If True (default), annotate all variable bindings with the struct info of that variable. If False, only add annotations where required for unambiguous round-trip of Relax -> TVMScript -> Relax.

path_to_underlineOptional[List[ObjectPath]] = None

Object path to be underlined

path_to_annotateOptional[Dict[ObjectPath, str]] = None

Object path to be annotated

obj_to_underlineOptional[List[Object]] = None

Object to be underlined

obj_to_annotateOptional[Dict[Object, str]] = None

Object to be annotated

Returns

scriptstr

The TVM Script of the given TVM IR

参数:

• name (str | None)

• show_meta (bool)

• ir_prefix (str)

• tir_prefix (str)

• relax_prefix (str)

• module_alias (str)

• buffer_dtype (str)

• int_dtype (str)

• float_dtype (str)

• verbose_expr (bool)

• indent_spaces (int)

• print_line_numbers (bool)

• num_context_lines (int)

• syntax_sugar (bool)

• show_object_address (bool)

• show_all_struct_info (bool)

• path_to_underline (List[ObjectPath] | None)

• path_to_annotate (Dict[ObjectPath, str] | None)

• obj_to_underline (List[Object] | None)

• obj_to_annotate (Dict[Object, str] | None)

返回类型:

str

show(style=None, black_format=None, *, name=None, show_meta=False, ir_prefix='I', tir_prefix='T', relax_prefix='R', module_alias='cls', buffer_dtype='float32', int_dtype='int32', float_dtype='void', verbose_expr=False, indent_spaces=4, print_line_numbers=False, num_context_lines=-1, syntax_sugar=True, show_object_address=False, show_all_struct_info=True, path_to_underline=None, path_to_annotate=None, obj_to_underline=None, obj_to_annotate=None)

A sugar for print highlighted TVM script.

Parameters

stylestr, optional

Pygmentize printing style, auto-detected if None. See tvm.script.highlight.cprint for more details.

black_format: Optional[bool]

If true, use the formatter Black to format the TVMScript. If false, do not apply the auto-formatter.

If None (default), determine the behavior based on the environment variable “TVM_BLACK_FORMAT”. If this environment variable is unset, set to the empty string, or set to the integer zero, black auto-formatting will be disabled. If the environment variable is set to a non-zero integer, black auto-formatting will be enabled.

Note that the “TVM_BLACK_FORMAT” environment variable only applies to the .show() method, and not the underlying .script() method. The .show() method is intended for human-readable output based on individual user preferences, while the .script() method is intended to provided a consistent output regardless of environment.

nameOptional[str] = None

The name of the object

show_metabool = False

Whether to print the meta data of the object

ir_prefixstr = “I”

The prefix of AST nodes from tvm.ir

tir_prefixstr = “T”

The prefix of AST nodes from tvm.tir

relax_prefixstr = “R”

The prefix of AST nodes from tvm.relax

module_aliasstr = “cls”

The alias of the current module at cross-function call, Directly use module name if it’s empty.

buffer_dtypestr = “float32”

The default data type of buffer

int_dtypestr = “int32”

The default data type of integer

float_dtypestr = “void”

The default data type of float

verbose_exprbool = False

Whether to print the detailed definition of each variable in the expression

indent_spacesint = 4

The number of spaces for indentation

print_line_numbersbool = False

Whether to print line numbers

num_context_linesint = -1

The number of lines of context to print before and after the line to underline.

syntax_sugar: bool = True

Whether to output with syntax sugar, set false for complete printing.

show_object_address: bool = False

Whether to include the object’s address as part of the TVMScript name

show_all_struct_info: bool = True

If True (default), annotate all variable bindings with the struct info of that variable. If False, only add annotations where required for unambiguous round-trip of Relax -> TVMScript -> Relax.

path_to_underlineOptional[List[ObjectPath]] = None

Object path to be underlined

path_to_annotateOptional[Dict[ObjectPath, str]] = None

Object path to be annotated

obj_to_underlineOptional[List[Object]] = None

Object to be underlined

obj_to_annotateOptional[Dict[Object, str]] = None

Object to be annotated

参数:

• style (str | None)

• black_format (bool | None)

• name (str | None)

• show_meta (bool)

• ir_prefix (str)

• tir_prefix (str)

• relax_prefix (str)

• module_alias (str)

• buffer_dtype (str)

• int_dtype (str)

• float_dtype (str)

• verbose_expr (bool)

• indent_spaces (int)

• print_line_numbers (bool)

• num_context_lines (int)

• syntax_sugar (bool)

• show_object_address (bool)

• show_all_struct_info (bool)

• path_to_underline (List[ObjectPath] | None)

• path_to_annotate (Dict[ObjectPath, str] | None)

• obj_to_underline (List[Object] | None)

• obj_to_annotate (Dict[Object, str] | None)

返回类型:

None

class tvm.runtime.ShapeTuple(shape)

TVM runtime ShapeTuple object. Parameters ———- shape : list[int]

The shape list used to construct the object.

class tvm.runtime.String(content)

TVM runtime.String object, represented as a python str.

Parameters

contentstr

The content string used to construct the object.

Methods:

__from_tvm_object__(obj)	Construct from a given tvm object.
__new__(cls, content)	Construct from string content.

__from_tvm_object__(obj)

Construct from a given tvm object.

static __new__(cls, content)

Construct from string content.

tvm.runtime.cl(dev_id=0)

Construct a OpenCL device

Parameters

dev_idint, optional

The integer device id

Returns

devDevice

The created device

tvm.runtime.const(value, dtype=None, span=None)

construct a constant

Parameters

valuenumber

The content of the constant number.

dtypestr or None, optional

The data type.

spanOptional[Span]

The location of the constant value in the source.

Returns

const_val: tvm.Expr

The result expression.

tvm.runtime.convert(value, span=None)

Convert value to TVM object or function.

Parameters

value : python value

spanOptional[Span]

The location of this statement in the source code.

Returns

tvm_valObject or Function

Converted value in TVM

Note

This function is redirected to convert_to_object as it is widely used in the codebase. We can choose one to keep and discard the other one later.

tvm.runtime.convert_to_object(value, span=None)

Convert a Python value to corresponding object type.

Parameters

valuestr

The value to be inspected.

spanOptional[Span]

The location of this itervar in the source code.

Returns

objObject

The corresponding object value.

tvm.runtime.cpu(dev_id=0)

Construct a CPU device

Parameters

dev_idint, optional

The integer device id

Returns

devDevice

The created device

tvm.runtime.cuda(dev_id=0)

Construct a CUDA GPU device

Parameters

dev_idint, optional

The integer device id

Returns

devDevice

The created device

tvm.runtime.device(dev_type, dev_id=0)

Construct a TVM device with given device type and id.

Parameters

dev_type: int or str

The device type mask or name of the device.

dev_idint, optional

The integer device id

Returns

dev: tvm.runtime.Device

The corresponding device.

Examples

Device can be used to create reflection of device by string representation of the device type.

assert tvm.device("cpu", 1) == tvm.cpu(1)
assert tvm.device("cuda", 0) == tvm.cuda(0)

tvm.runtime.enabled(target)

Whether module runtime is enabled for target

Parameters

targetstr

The target device type.

Returns

enabledbool

Whether runtime is enabled.

Examples

The following code checks if gpu is enabled.

>>> tvm.runtime.enabled("gpu")

tvm.runtime.ext_dev(dev_id=0)

Construct a extension device

Parameters

dev_idint, optional

The integer device id

Returns

devDevice

The created device

Note

This API is reserved for quick testing of new device by plugin device API as ext_dev.

tvm.runtime.gpu(dev_id=0)

Construct a CUDA GPU device

deprecated:: 0.9.0 Use tvm.cuda() instead.

Parameters

dev_idint, optional

The integer device id

Returns

devDevice

The created device

tvm.runtime.load_module(path, fmt='')

Load module from file.

Parameters

pathstr

The path to the module file.

fmtstr, optional

The format of the file, if not specified it will be inferred from suffix of the file.

Returns

moduleruntime.Module

The loaded module

Note

This function will automatically call cc.create_shared if the path is in format .o or .tar

tvm.runtime.load_param_dict(param_bytes)

Load parameter dictionary from binary bytes.

Parameters

param_bytes: bytearray

Serialized parameters.

Returns

paramsdict of str to NDArray

The parameter dictionary.

tvm.runtime.load_param_dict_from_file(path)

Load parameter dictionary from file.

Parameters

path: str

The path to the parameter file to load from.

Returns

paramsdict of str to NDArray

The parameter dictionary.

tvm.runtime.load_static_library(path, func_names)

Load the .o library at path which implements functions with func_names. Unlike the generic load_module the result will remain as a static_library and will not be relinked on-the-fly into a .so library.

tvm.runtime.metal(dev_id=0)

Construct a metal device

Parameters

dev_idint, optional

The integer device id

Returns

devDevice

The created device

tvm.runtime.mtl(dev_id=0)

Construct a metal device

Parameters

dev_idint, optional

The integer device id

Returns

devDevice

The created device

tvm.runtime.num_threads()

Get the number of threads in use by the TVM runtime.

Returns

int

Number of threads in use.

返回类型:

int

tvm.runtime.opencl(dev_id=0)

Construct a OpenCL device

Parameters

dev_idint, optional

The integer device id

Returns

devDevice

The created device

tvm.runtime.rocm(dev_id=0)

Construct a ROCM device

Parameters

dev_idint, optional

The integer device id

Returns

devDevice

The created device

tvm.runtime.save_param_dict(params)

Save parameter dictionary to binary bytes.

The result binary bytes can be loaded by the GraphModule with API “load_params”.

Parameters

paramsdict of str to NDArray

The parameter dictionary.

Returns

param_bytes: bytearray

Serialized parameters.

Examples

# set up the parameter dict
params = {"param0": arr0, "param1": arr1}
# save the parameters as byte array
param_bytes = tvm.runtime.save_param_dict(params)
# We can serialize the param_bytes and load it back later.
# Pass in byte array to module to directly set parameters
tvm.runtime.load_param_dict(param_bytes)

tvm.runtime.save_param_dict_to_file(params, path)

Save parameter dictionary to file.

Parameters

paramsdict of str to NDArray

The parameter dictionary.

path: str

The path to the parameter file.

tvm.runtime.system_lib(symbol_prefix='')

Get system-wide library module singleton.

System lib is a global module that contains self register functions in startup. Unlike normal dso modules which need to be loaded explicitly. It is useful in environments where dynamic loading api like dlopen is banned.

To build system lib function, simply specify target option `llvm --system-lib` The system lib will be available as long as the result code is linked by the program.

The system lib is intended to be linked and loaded during the entire life-cyle of the program. If you want dynamic loading features, use dso modules instead.

Parameters

symbol_prefix: Optional[str]

Optional symbol prefix that can be used for search. When we lookup a symbol symbol_prefix + name will first be searched, then the name without symbol_prefix.

Returns

moduleruntime.Module

The system-wide library module.

tvm.runtime.vpi(dev_id=0)

Construct a VPI simulated device

Parameters

dev_idint, optional

The integer device id

Returns

devDevice

The created device

tvm.runtime.vulkan(dev_id=0)

Construct a Vulkan device

Parameters

dev_idint, optional

The integer device id

Returns

devDevice

The created device