ONNX
By ONNX (Linux Foundation project, originated at Microsoft and Facebook)
ONNX, short for Open Neural Network Exchange, is an open standard format for representing machine learning models so they can move between different frameworks and run on different hardware and inference engines. A model trained in one…
Definition
ONNX, short for Open Neural Network Exchange, is an open standard format for representing machine learning models so they can move between different frameworks and run on different hardware and inference engines. A model trained in one framework can be exported to ONNX and then loaded by any tool that supports the format, decoupling how a model is trained from how it is deployed.
Overview
Machine learning teams often train a model in one framework, such as PyTorch, but need to deploy it in an environment optimized for a different runtime, such as a mobile device, an embedded system, or a specialized inference server. Without a common format, this requires either running the original training framework everywhere, which can be heavy or unsupported on certain hardware, or manually reimplementing the model in the target environment, which is error-prone and duplicative. ONNX exists to remove that friction by giving the ecosystem a shared target to convert into and out of. At its core, ONNX defines a computation graph representation: a model is described as a directed graph of standard operators, such as convolutions, matrix multiplications, and activation functions, along with the learned weights attached to that graph. Exporting a model to ONNX means translating a framework's internal representation into this common graph format, typically via converter tools provided by the source framework. Once in ONNX form, any runtime that implements the ONNX operator set can load and execute the model without needing the original training framework installed, which is what makes the format portable across tools. ONNX is a format and specification, not a runtime itself, which distinguishes it from tools like ONNX Runtime or TensorRT that actually execute ONNX models; it plays a similar interoperability role to formats in other domains, giving the ML ecosystem something analogous to a shared file format that many independently built tools agree to read and write. It sits upstream of the serving layer, feeding into runtimes and inference servers rather than replacing them. In practice, ONNX is used to move models between a training framework and a production-optimized runtime, to enable a model to run on hardware or platforms the original training framework does not directly support, and to give tool builders and hardware vendors a single target format to optimize for, rather than needing custom support for every source framework and every downstream deployment target. Limitations include imperfect operator coverage: not every operation available in a source framework has a direct ONNX equivalent, so highly custom model architectures or newly introduced layers can fail to export cleanly or require custom operator implementations. Conversion can also occasionally introduce subtle numerical differences from the original model, so validating exported models against the source is standard practice before deploying them, particularly for models where small numerical drift could affect downstream decisions.
Specification
- Defines an open standard graph format for representing ML models
- Enables exporting models from frameworks like PyTorch or TensorFlow
- Decouples model training framework from deployment runtime
- Is implemented by many independent inference engines and hardware vendors
- Represents models as a graph of standard mathematical operators
- Functions as a specification rather than an execution runtime itself
- Is maintained as an open, community-governed standard