How to add new backend(like VTA) into TVM

very confused, wonder to know how to add a new backend into TVM…

could you be more specific about what backend you are targeting? I don’t think there is a one-size fits all approach but we can try to draw comparisons between the design you are targeting and VTA

We have designed new hardware backend using FPGA and a related compiler, we use it to do deep learning inference work and it works very well, so we want to try our compiler on TVM, but I have no idea now~~

I see - I think that a good point to start is to have a software-hardware interface specification of your FPGA hardware to start with, or an API specification of your compiler. Something like this page (https://docs.tvm.ai/vta/dev/hardware.html#microarchitectural-overview) that should explain in a nutshell how your accelerator is programmed.

Then the question is: how do you call into this API, or build your instruction stream from scratch? Perhaps you may be able to call directly into this API from TVM IR, for which you’ll have to implement your own IR passes, like we did in here: https://github.com/dmlc/tvm/blob/master/vta/python/vta/ir_pass.py (note that these were calls into the VTA runtime API).
Or in the second case of building an instruction stream from scratch, you can preserve the same runtime API we provide, but implement your own runtime instead that assembles instructions for your accelerator on the fly. This would be equivalent of implementing your own substitute to this: https://github.com/dmlc/tvm/blob/master/vta/src/runtime.cc

Is there any information you can reveal on your FPGA design?

1 Like

Thank you very much for yr advice.

I’m sorry I can’t reveal any details of our FPGA design according to our company’s confidential regulations.

I still don’t know how to connect TVM and our compiler, I think the reason is my question is not very good, I will give my questions more specifically and with some related code posted here in the next.

Yes, please think of the accelerator abstractions and how one would typically program it. Providing a programming example would help understand the types of challenges you are facing. You could also draw comparisons with mainstream designs (e.g. Google TPU, MIT’s Eyeriss etc.) which should help understand what kind of programming challenges you are facing.

Hello, did you tell me how tvm makes nvidia gpu run through, now I want to add mlu to TVM