Language Reference

This document provides references to embedded languages and IRs in the TVM stack.

Introduction to Relay

Relay is a functional, differentiable programming language designed to be an expressive intermediate representation for machine learning systems. Relay supports algebraic data types, closures, control flow, and recursion, allowing it to directly represent more complex models than computation graph-based IRs can. Relay also includes a form of dependent typing using type relations in order to handle shape analysis for operators with complex requirements on argument shapes.

Relay is extensible by design and makes it easy for machine learning researchers and practitioners to develop new large-scale program transformations and optimizations.

The below pages describe the grammar, type system, algebraic data types, and operators in Relay, respectively.

• Expressions in Relay
  • Dataflow and Control Fragments
  • Variables
  • Functions
  • Operators
  • ADT Constructors
  • Call
  • Module and Global Functions
  • Constant
  • Tuples
  • Let Bindings
  • Graph Bindings
  • If-Then-Else
  • ADT Matching
  • TempExprs
• Relay's Type System
  • Type
  • Tensor Type
  • Tuple Type
  • Type Parameter
  • Type Constraint
  • Function Type
  • Type Relation
  • Incomplete Type
  • Algebraic Data Types
• Algebraic Data Types in Relay
  • Defining and Matching on an ADT
  • Type-Checking ADTs and Polymorphism
  • Recursion with ADTs
  • Pattern Matching in Match Expressions
  • Common ADT Uses
  • Implementing Neural Nets Using ADTs
• Relay Core Tensor Operators
  • Overview of Operators
• Pattern Matching in Relay
  • Pattern Examples
  • Matching Fuzzy Patterns
  • Pattern Language Design
  • Applications

Hybrid Script

The below page describes the TVM hybrid script front-end, which uses software emulation to support some constructs not officially supported in TVM.

• Hybrid Frontend Language Reference
  • Overview
  • Features