# Load and Run an ONNX Model

onnx/models is a repository for storing the pre-trained ONNX models. Every ONNX backend should support running these models out of the box. After downloading and extracting the tarball of each model, there should be:

• A protobuf file model.onnx which is the serialized ONNX model.
• Several sets of sample inputs and outputs files (test_data_*.npz), they are numpy serialized archive.

In this tutorial, you’ll learn how to use a backend to load and run a ONNX model.

## Example: Using TensorFlow backend

First, install ONNX TensorFlow backend by following the instructions here.

Next, we load the necessary R and Python libraries (via reticulate):

library(onnx)
library(reticulate)
np <- import("numpy", convert = FALSE)
backend <- import("onnx_tf.backend")

We can then use the loaded numpy Python library to define a helper function to load testing sample from numpy serialized archive.

load_npz_samples <- function(npz_path) {
sample <- np$load(normalizePath(npz_path), encoding = 'bytes') list( inputs = sample$items()[[0]][[1]][[0]],
outputs = sample$items()[[1]][[1]] ) } Finally, we can load the ONNX model and the testing samples, and then run the model using ONNX TensorFlow backend: # Specify paths to ONNX model and testing samples onnx_model_dir <- "~/Downloads/resnet50" model_pb_path <- file.path(onnx_model_dir, "model.onnx") npz_path <- file.path(onnx_model_dir, "test_data_0.npz") # Load ONNX model model <- load_from_file(model_pb_path) # Load testing sample from numpy serialized archive samples <- load_npz_samples(npz_path) inputs <- samples$inputs
expected_outputs <- samples$outputs # Run the model with an onnx backend actual_outputs <- backend$run_model(model, inputs)

We can also use numpy to verify the result:

np$testing$assert_almost_equal(expected_outputs, actual_outputs, decimal = 6)

That’s it! Isn’t it easy? Next you can go ahead and try out different ONNX models as well as different ONNX backends, e.g. PyTorch, MXNet, Caffe2, CNTK, Chainer, etc.