I am working on BYOC for a new NPU and I went through the device/target interactions documentation and I wonder whether it is up to date with the current unity work or not.
Is the device/target interactions documentation still valid for unity?
When creating a target for my new device, do I have to provide a FTVMRelayToTIR?
Also I was hoping that I could keep high level relax operations to pass to my codegen, is it possible?
Hi, @slai-nick.
Yes, I believe this is still valid. This component is relax/relay-independent so should be compatible.
No. You don’t have to define any Relay side component.
Yes. Once you define your own runtime and codegen, we have a mechanism to offload operators by using a pass sequence. You may refer: https://github.com/apache/tvm/blob/unity/tests/python/relax/test_codegen_tensorrt.py#L98
Although we don’t currently have the example of NPU integration yet in Unity, it should be possible. What you need to do is to implement BYOC codegen and runtime. (Example of TensorRT runtime, TensorRT codegen). It is worth noting that runtime part is relay/relax-independent, so you can reuse infra if you have one for Relay. For codgen, you may need to handle some difference in AST and operator sets, but so far this haven’t been very difficult. Hope this helps.