Hi everyone: Recently I am profiling the execution compiled by tvm. It’ s a pytorch model, the target is cuda. when I profiled the python script by Nvidia Nsight system
There are a lot of kernels like tvmgen_default_*. I wonder how these kernel were created, and for the kernel tvmgen_default_fused_nn_einsum_2_kernel0, does it do some computation
When TVM reads a model, it represents it in relay. There, initially each operator looks like a separate function, and invocation of an operator looks like a function call. Then, a fusion pass will partition the graph, where each partition is a group of such calls. Each such group will become a single function later on. This is where names like tvmgen_default_fused_add_multiply_erf_multiply_add_multiply come from. Then, once the relay has been translated into TIR, some loop nests in it may be marked as targeted for an accelerator, like an NVIDIA GPU. Each such loop nest will be extracted into its own function, with _kernelN appended to the name. These are typically pure computations, without any function calls, or control flow.
In your code, you can try doing something like print(lib_einsum.ir_mod) to see the TIR module, but it may have the kernels already extracted.
Hi，thanks for your reply.
I am sorry that I haven’t reply in time.
Now I get the cuda source code about my network.
and it look like this
Yes, that’s pretty much it.
