@KireinaHoro Yes, OpenOCD is unfortunately very slow. The way I’ve sped things up in my personal usage has been to run AutoTVM with 8 microcontrollers, but I’d like to have a solution that doesn’t require “brute forcing” it with devices.
Do you have the check_correctness
option set? If so, it could be this line that’s forcing a copy.
But if you don’t have check_correctness
set, it looks like this else block will trigger, which will also perform a device copy.
As a short-term hack, you could comment out both of those blocks. There might still be other device copies that I’m unaware of.
In the long-term, it might be worth it to have code on the device that generates random tensors, rather than the host generating them and transmitting them over OpenOCD. Or we could keep the same input/output tensors on the device for the entire tuning run—not recommended if the operator being tuned is sensitive to the values of its inputs.
Besides, I’m wondering if the AutoTVM implementation for uTVM in https://github.com/apache/incubator-tvm/pull/4274 is complete
As far as I know, the implementation of AutoTVM for µTVM in mainline TVM is complete. It’s worked in my own experiments, but I’m not an expert in the AutoTVM side of the codebase, so it’s possible I’ve missed some functionality. There are definitely some ergonomic changes that need to be made, some of which I’ve already implemented in my fork.
One big problem is that each RPC server has a fixed µTVM device config, so if you want to change the memory profile of an operator (maybe it takes up more of the text
section, but doesn’t require as large of a workspace
section), you need to restart the server with a new device config that reflects that memory layout. My current solution to that problem is to watch the file system for changes to dev_config.json
and automatically restart the server, but it would be great if we could dynamically change the memory layout.
Anyways, let me know if you have any more questions. Good luck on your deadline! I’m excited to see your upcoming PR.