How to deploy a model on Nvidia AGX Xavier without RPC

Questions
1

I can compile and run a model on Jetson AGX Xavier using this tutorial

https://tvm.apache.org/docs/how_to/deploy_models/deploy_model_on_nano.html#sphx-glr-how-to-deploy-models-deploy-model-on-nano-py

Now I am trying to run the compiled model without RPC. I am exporting the library using:

with tvm.transform.PassContext(opt_level=3):
    lib = relay.build(func, target=target, params=params)

lib.export_library('./net.tar')

Then I copy this net.tar file to target. I am using following code at the target to try to run the model:

import tvm
from tvm.contrib.download import download_testdata
from PIL import Image
import numpy as np

img_url = "https://github.com/dmlc/mxnet.js/blob/main/data/cat.png?raw=true"
img_name = "cat.png"
img_path = download_testdata(img_url, img_name, module="data")
image = Image.open(img_path).resize((224, 224))

def transform_image(image):
    image = np.array(image) - np.array([123.0, 117.0, 104.0])
    image /= np.array([58.395, 57.12, 57.375])
    image = image.transpose((2, 0, 1))
    image = image[np.newaxis, :]
    return image

x = transform_image(image)

synset_url = "".join(
    [
        "https://gist.githubusercontent.com/zhreshold/",
        "4d0b62f3d01426887599d4f7ede23ee5/raw/",
        "596b27d23537e5a1b5751d2b0481ef172f58b539/",
        "imagenet1000_clsid_to_human.txt",
    ]
)
synset_name = "imagenet1000_clsid_to_human.txt"
synset_path = download_testdata(synset_url, synset_name, module="data")
with open(synset_path) as f:
    synset = eval(f.read())


lib = tvm.runtime.load_module('net.tar')
dev = tvm.runtime.cuda(0)
module = tvm.runtime.Module(lib["default"](dev))

But in the end, I cannot use module. Any advice or hints to correct directions would be great!

Thanks

2

Hi,

I successfully run the compiled model with/without RPC on Jetson AGX Xavier. Are you still working on it? I can help you.