While the TVM supports “device” specific compilation, the runtime side seems to lack a mechanism to support device specific implementations. For example, one vendor can have multiple devices for Vulkan target. Those devices may have different runtime implementations due to the hardware difference. Can we extend the DLContext structure to allow device info to be passed from the compiler to the runtime?
Thanks @samwyi it would be great to provide an example about what you mean by device discovery. For example, we might be able to perform some formal setup to select the device before start using one.
DLContext is specifically limited to the common terminology(in terms of device id and device type) so that it is easier to exchange among frameworks(since different framework might refer to the same device)
Adding an example would be good. I believe that what @samyi is specifically referring to are GPUs that have a shared memory with CPU for instance. For instance if your vulkan GPU is integrated with the CPU, you can perform some tricks, e.g. eliminate the staged copy when moving data from CPU to GPU.
Yes, the problem I met is as @thierry mentioned, I want to have a specific Vulkan runtime implementation for a specific device, but I didn’t find a way to pass the device information from the compiler to the runtime other than modifying DLContext. Maybe there’s a better way to do this?
hmm, one potential thing that we might be able to is to do a setup step before using any vulkan device, to allow us to initialize the vulkan runtime with the desired device and feature. This is also related to the opencl runtime where we have ways to select the right opencl runtime