ONNX Model Interchange Cheat Sheet

Export, inspect, and run models across frameworks using the Open Neural Network Exchange format and the ONNX Runtime.

2 PagesIntermediateMar 3, 2026

Export a PyTorch Model to ONNX

Trace a model and write it to the ONNX format with dynamic batch axes.

python

import torchmodel.eval()dummy_input = torch.randn(1, 3, 224, 224)torch.onnx.export(    model,    dummy_input,    "model.onnx",    input_names=["input"],    output_names=["output"],    dynamic_axes={"input": {0: "batch_size"}, "output": {0: "batch_size"}},    opset_version=18,)

Export a scikit-learn Model

Convert a trained sklearn pipeline into ONNX using skl2onnx.

python

from skl2onnx import to_onnxfrom skl2onnx.common.data_types import FloatTensorTypeinitial_type = [("input", FloatTensorType([None, X_train.shape[1]]))]onnx_model = to_onnx(clf, initial_types=initial_type)with open("clf.onnx", "wb") as f:    f.write(onnx_model.SerializeToString())

Run Inference with ONNX Runtime

Load an .onnx file and run a prediction with the ONNX Runtime session API.

python

import onnxruntime as ortimport numpy as npsess = ort.InferenceSession("model.onnx", providers=["CUDAExecutionProvider", "CPUExecutionProvider"])input_name = sess.get_inputs()[0].nameoutput_name = sess.get_outputs()[0].namex = np.random.randn(1, 3, 224, 224).astype(np.float32)result = sess.run([output_name], {input_name: x})print(result[0].shape)

Inspect and Optimize a Graph

Validate the model, print its graph, and apply graph-level optimizations before deployment.

bash

# validate the model structurepython -c "import onnx; m = onnx.load('model.onnx'); onnx.checker.check_model(m); print('valid')"# human-readable graph dumppython -c "import onnx; print(onnx.helper.printable_graph(onnx.load('model.onnx').graph))"# quantize to int8 for faster CPU inferencepython -m onnxruntime.quantization.preprocess --input model.onnx --output model-pre.onnxpython -c "from onnxruntime.quantization import quantize_dynamic, QuantType; \quantize_dynamic('model-pre.onnx', 'model-int8.onnx', weight_type=QuantType.QInt8)"

Execution Providers

Hardware backends ONNX Runtime can target via the providers list, tried in order.

CPUExecutionProvider- default fallback, runs on any machine
CUDAExecutionProvider- NVIDIA GPU inference via CUDA/cuDNN
TensorrtExecutionProvider- NVIDIA TensorRT for lower-latency GPU inference
CoreMLExecutionProvider- Apple Silicon/Neural Engine acceleration
OpenVINOExecutionProvider- Intel CPU/iGPU/VPU acceleration
DmlExecutionProvider- DirectML backend for Windows GPUs

Pro Tip

Always set dynamic_axes for the batch dimension on export — a model hardcoded to batch size 1 will silently fail or require re-export the moment you need to batch requests in production.

Was this cheat sheet helpful?

Optuna Hyperparameter Optimization Cheat Sheet

LangChain Cheat Sheet

Explore Topics

#ONNXModelInterchange#ONNXModelInterchangeCheatSheet#DataScience#Intermediate#Export#PyTorch#Model#ONNX#Networking#MachineLearning#CheatSheet#SkillVeris

Sri Hayavadhana Info-Tech

Professional Web Designing Services