Run TVMC package with VTA

[ShadAhammed]

Hi, I am a new user of Apache TVM and VTA. I am currently having some confusion regarding TVMC package and VTA. I have compiled my ANN model and got output of a tar file consist of mod.so/json/params. I am trying to do inference on PYNQ using VTA. I guess it’s not possible using tvmc.run as there is not option to chose vta as device.

Can someone please give me some direction how after having the TVMCPackage I can run it in FPGA using VTA? I am using RPC servers. Thank you.

[leandron]

Hi @ShadAhammed, thanks for posting. You’re correct that vta is not a target device supported by tvmc run at the moment.

I don’t think there is a big limiting factor in TVM that prevents that to be supported, I guess the reason on why this is not in place it mostly because nobody implemented/tested that specific use case yet.

@ShadAhammed

Can someone please give me some direction how after having the TVMCPackage I can run it in FPGA using VTA?

Regarding your questions, I think @areusch can perhaps give some pointers on where to look for help on that.

[areusch]

hi @ShadAhammed , you’re right we don’t support this in tvmc at the moment. I think what I can point you to now is the VTA tutorial, in particular see tune_and_evaluate here.

[ShadAhammed]

Hi @areusch, firstly thanks a lot for referring this tutorial. I actually have gone through all those tutorials on how to synchronise my specific classification based neural network with the examples. My neural network is based of data, not image. I have created the TVMC package and run inteference with test data on host computer with tvmc.run. My compiled module is created with ‘tar’ extension and it has mod.so, mod.params and mod.so files. The next job was to probably use the vta.build function as in your referred tutorial.

my_vadd = vta.build( … )

But I don’t have the parameters to use in this function. Compiled module from the tvmc.compile is a tvmc package which I don’t know how to use for the next steps. If you have any further idea/suggestion/direction, kindly share.

[vegaluis]

Hey @ShadAhammed , we have recently ported VTA to Sabana

Run VTA in the cloud (FPGA) with a single click

Hello there, We have recently ported VTA (Chisel backend) to our platform here at Sabana, which will allow anybody with a Sabana free account to run it without the pain of setting up physical FPGA boards or installing complex toolchains. The source code (hardware and tests) are available here. We also created a Replit template for those of you who want to deploy and run VTA on a FPGA directly from a web browser. Let us know what you think or if you would like to try it out on a discord commun…

I would be interested on knowing more about your use case. Happy to chat, let me know!

[ShadAhammed]

Hi @vegaluis , thank you for responding. Will be happy to share our knowledge. I am trying to work now on the graph execution procedure for vta. Cannot do it with my normal classification network. Unfortunately have no use cases in the community forum too to verify.

[vegaluis]

@ShadAhammed Maybe it will be great to chat and see what you are building? I can help you out with VTA related stuff…Feel free to message me on our discord and we can do a zoom call.