[AutoTVM Solved] I connected the PYNQ, but cannot get remote devices from the tracker

[denis]

Going through the tutorial tune_relay_vta, I got the problem when trying to autotune the CNN on PYNQ.

Then I got an ERROR:

RuntimeError: Cannot get remote devices from the tracker. 

I can confirm the pynq is registered, by querying the rpc_tracker.

I spent several days to search the problem, finding some solutions, like
[AutoTVM] Cannot get remote devices from tracker
[SOLVED] [AutoTVM] RuntimeError: Cannot get remote devices from the tracker
[VTA] Workaround for Autotuning with One PYNQ Z1 Board
VTA autotuning from tutorial fails with one PYNQ, but succeeds with two PYNQs ,
but those solutions don’t work for me at all.
By the way, I was also confused, what’s the difference between start_rpc_server.sh and start_rpc_server_to_tracker.sh in $TVM_PATH/apps/vta_rpc/, which script should I use when auto-tuning, or I just need to run python -m tvm.exec.rpc_server --tracker=[HOST_IP]:9190 --key=pynq? (I tried all of them)
I fell so helpless and don’t know what to do.
Any help will be appreciated.

[denis]

FIrst, I ran python -m tvm.exec.rpc_tracker --host=0.0.0.0 --port=9190 on the Host-side.

Second, I ran python3 -m tvm.exec.rpc_server --tracker=192.168.2.100:9190 --key=pynq on the PYNQ-side.

Then, I ran python -m tvm.exec.query_rpc_tracker --host=0.0.0.0 --port=9190 on the Host-side.

Finally, I ran python tune_relay_vta.py on the Host-side.
And I got the ERROR.

On the Host-side:

On the PYNQ-side:

Is there something wrong ?

[denis]

@denis

FIrst, I ran python -m tvm.exec.rpc_tracker --host=0.0.0.0 --port=9190 on the Host-side.

Second, I ran python3 -m tvm.exec.rpc_server --tracker=192.168.2.100:9190 --key=pynq on the PYNQ-side.

Then, I ran python -m tvm.exec.query_rpc_tracker --host=0.0.0.0 --port=9190 on the Host-side.

One more question, my Host IP is 192.168.2.100, should I run python -m tvm.exec.rpc_tracker --host=192.168.2.100 instead of host=0.0.0.0?

[denis]

Another more question, what’s the difference between tvm.exec.rpc_server and vta.exec.rpc_server, there are two .py files called rpc_server.py in directory $TVM_PATH/tvm/python/tvm/exec and $TVM_PATH/tvm/vta/python/vta/exec

Which rpc_server.py should I use to register the pynq to the tracker?
That means, should I run
python3 -m tvm.exec.rpc_server --tracker=192.168.2.100:9190 --key=pynq
or
python3 -m vta.exec.rpc_server --tracker=192.168.2.100:9190 --key=pynq
on the pynq-side?

[denis]

@denis

there are two .py files called rpc_server.py in directory $TVM_PATH/tvm/python/tvm/exec and $TVM_PATH/tvm/vta/python/vta/exec

I use the rpc_server.py in $TVM_PATH/tvm/vta/python/vta/exec, and start RPC with commands:

# (Host-side) start a RPC tracker
python -m tvm.exec.rpc_tracker --host=192.168.2.100 --port=9190

# (PYNQ-side) Register devices
# add the following command to start_rpc_server_to_tracker.sh
# python3 -m vta.exec.rpc_server --tracker=192.168.2.100:9190 --key=pynq
sudo apps/vta_rpc/start_rpc_server_to_tracker.sh

# (Host-side) confirm devices
python -m tvm.exec.query_rpc_tracker --host=192.168.2.100 --port=9190

And now I can run:

python tune_relay_vta.py

If I run python -m tvm.exec.rpc_tracker --host=192.168.2.100 --port=9190, which defined in original start_rpc_server_to_tracker.sh it will call rpc_tracker in directory $TVM_PATH/tvm/python/tvm/exec and then failed.
I run python -m vta.exec.rpc_tracker --host=192.168.2.100 --port=9190 and it works.

I use random tuner, and it takes about 2 hours to tune the ResNet18 with one PYNQ board.
Finally, I got the log file vta.resnet18_v1.log.

The content in log file:

Have a nice weekend!