Using tvms for RISC-V

Questions
#1

Hello,

Overview about my architecture:

I am using a Nexys a7-100t board where a RISC-V is set up and where a Debian Linux is running on (https://github.com/eugene-tarassov/vivado-risc-v)

Problem

What I want to do is to compile the pretrained ResNet-50 v2 model on this RISC-V architecture. For compilation I am using the tvmc tool and as target I am using the LLVM for RISC-V. In your community I already found a thread about it, but I got the same error:

tvmc compile --target "llvm -march=rv64imv -mcpu=generic-rv64" --output resnet50-v2-7-tvm.tar resnet50-v2-7.onnx

It gives me following error

LLVM ERROR: CPU 'generic' is not supported. Use generic-rv64

My question is if anyone has already built it successfully with this approach, or whether this is possible at all with LLVM and RISC-V? Or is this a completely wrong approach and I should use something a different approach like using microTVM?

Thank you in advance

#2

cc @fPecc @alter-xp who might have some ideas here.

#3

In my code, I used this target
target = tvm.target.Target( "llvm -mtriple=riscv64-unknown-linux-gnu -mcpu=generic-rv64 -mabi=lp64d -mattr=+64bit,+m,+a,+f,+d,+c" )

image
image782×187 34 KB

but when I tried to use this target and your command in tvmc, I failed too. Maybe you can try to use this “target” in Python to build the model directly, instead of tvmc, and configure -mabi compilation instructions according to your hardware.

#4

Thank you very much. I will try this out and will inform you if it’s working.

#5

Hi @joschi2804 ,

Sadly I can’t give any feedback on this, because my current use case is using the C target to generate C code for an embedded OS running on RISC-V.

But let us know how the proposal from @alter-xp works!

#6

Hi @alter-xp @joschi2804 @fPecc ,

I am also interested on running a model on a RISCV (PicoRV32) implementation running on an FPGA on Replit. Do you know what would be best to start with? as in, is there an example somewhere of running a simple model based on a simple operation (say CONV2D)? before going all the way through ResNet?

#7

Hi, I tried the proposal of @alter-xp but I got the same issue. I basically did the step-by-step-guide but used a target you proposed.

My next attempt was to update LLVM - maybe there was an issue. So I rebuilded LLVM with version 15.0.0 (previously I was using 14.0.6), but now I am getting a different error:

/TVM_PYTHON_ENV/lib/python3.8/site-packages/tvm-0.10.dev531+g5dcf62288-py3.8-linux-x86_64.egg/tvm/driver/build_module.py:266: UserWarning: target_host parameter is going to be deprecated. Please pass in tvm.target.Target(target, host=target_host) instead. 
warnings.warn(
[20:58:21] /TVM_INSTALL/src/target/llvm/codegen_llvm.cc:167: Warning: Set native vector bits to be 128 for riscv64
LLVM ERROR: Relocation type not implemented yet!

It crashes in that function and it seems that it doesn’t matter if I define a target or not.

mod, params = relay.frontend.from_onnx(onnx_model, shape_dict)

Do you think this could be more a LLVM issue or TVM?

BR

#8

@joschi2804 is that trying to use “RISCV vector instructions”?

#9

Hi,

I think I found my issue. I thought that I need to use a cross compiled LLVM (for RISC-V) instead of the standard one. So if I am using now the standard LLVM I am able to build it with the target that was proposed by @alter-xp.

Thanks for your support!

BR

#10

Awesome,

Quick question @joschi2804 — how do you run that shared library (.so)? Do you install tvm on the board? or is there a way to call a function in that library?

#11

Yes my plan is to run the tvm runtime on the board. I will post it into this thread when I have some updates on that.

#12

Short update on those who are interested:

The first step after compiling the model was to export the model as a library

lib.export_library("model.so", cc="/opt/riscv/bin/riscv64-unknown-linux-gnu-g++")

Next step was to set up the runtime environment for riscv. I followed basically this straight forward instruction. After the successful build of the runtime library I wrote a small C++ program that includes the tvm_runtime library and reads the model I’ve created before within python. I used the apps/howto_deploy example as reference. In principle I only had to adapt the makefile to my needs (Adapting the TVM root path, setting the riscv compiler) and compile it.

In the end I uploaded the compiled binary and the tvm_runtime library to my device, added the library to LD_LIBRARY_PATH and then executed the binary.

#13

hey can you provide me steps so that I can also port tvm on my riscv fpga

#14

Hi, there is a fix now for jit: https://github.com/apache/tvm/pull/15964