Hello. I am a newbie in TVM and I am very interested in using TVM for CNN training. I have made a trivial example of using TVM for training LeNet-5 on MNIST with SGD and it works well in general. My code is here:
But I have several newbee questions/problems about process:
- I use topi as main instrument and topi.te.gradient to get weight gradients. Can I use relay to obtain weight gradients automatically? I can’t find anything useful in documentation.
- Building with llvm is very slow - about 10 minutes even for network with 5 layers. The similar network without tvm.te.gradient builds in 10 seconds. Is it ok?
- Without autotuning training cycle(1 iteration) is about 20 times slower than Pytorch on CPU (0.6 sec vs 0.03 sec.). It seems strange. I use llvm with -mcpu=core-avx2. With similar settings without gradients tvm works well - faster than torch.
I will be grateful for any help.