When running tune_relay_vta.py, I get the following error:
Extract tasks… Extracted 10 conv2d tasks: (1, 14, 14, 256, 512, 1, 1, 0, 0, 2, 2) (1, 28, 28, 128, 256, 1, 1, 0, 0, 2, 2) (1, 56, 56, 64, 128, 1, 1, 0, 0, 2, 2) (1, 56, 56, 64, 64, 3, 3, 1, 1, 1, 1) (1, 28, 28, 128, 128, 3, 3, 1, 1, 1, 1) (1, 56, 56, 64, 128, 3, 3, 1, 1, 2, 2) (1, 14, 14, 256, 256, 3, 3, 1, 1, 1, 1) (1, 28, 28, 128, 256, 3, 3, 1, 1, 2, 2) (1, 7, 7, 512, 512, 3, 3, 1, 1, 1, 1) (1, 14, 14, 256, 512, 3, 3, 1, 1, 2, 2) Tuning… [Task 1/10] Current/Best: 0.00/ 0.00 GFLOPS | Progress: (0/480) | 0.00 s Traceback (most recent call last): File “tune_relay_vta.py”, line 452, in tune_and_evaluate(tuning_option) File “tune_relay_vta.py”, line 408, in tune_and_evaluate tune_tasks(tasks, **tuning_opt) File “tune_relay_vta.py”, line 291, in tune_tasks autotvm.callback.log_to_file(tmp_log_file), File “/home/jennifersowash/Achronix/tvm/python/tvm/autotvm/tuner/tuner.py”, line 112, in tune measure_batch = create_measure_batch(self.task, measure_option) File “/home/jennifersowash/Achronix/tvm/python/tvm/autotvm/measure/measure.py”, line 257, in create_measure_batch attach_objects = runner.set_task(task) File “/home/jennifersowash/Achronix/tvm/python/tvm/autotvm/measure/measure_methods.py”, line 252, in set_task "Cannot get remote devices from the tracker. " RuntimeError: Cannot get remote devices from the tracker. Please check the status of tracker by ‘python -m tvm.exec.query_rpc_tracker --port [THE PORT YOU USE]’ and make sure you have free devices on the queue status.
Querying 0.0.0.0:9197 returns:
Server List
server-address key
[ip_address]:41476 server:pynq
Queue Status
key total free pending
pynq 1 0 1
I’m using the latest version of the repo on main branch since I read an earlier question where this issue was supposedly fixed. I see in the Pynq tutorial, you mention “The RPC server should be listening on port 9091 . If not, an earlier process might have terminated unexpectedly and it’s recommended in this case to just reboot the Pynq, and re-run the RPC server.” My RPCServer is bound to port 9095: INFO:RPCServer:bind to 0.0.0.0:9095. Could this be a possible issue? I also tried increasing the timeout on this line: https://github.com/apache/incubator-tvm/blob/cffb4fba03ea582417e2630bd163bca773756af6/python/tvm/autotvm/measure/measure_methods.py#L534 like one previous answer suggested. Do you know what could be going wrong, or have any possible solutions?