[Android][RPC] Requesting remote session from RPC Tracker failed

Hi, I’m trying to deploy Qualcomm Snapdragon 865 and following the guide from android_rpc. Everything is progressed on the docker image provided.

After installing RPC APK on the board, I started tracker from the host.

$ python3 -m tvm.exec.rpc_tracker --host=0.0.0.0 --port=9190
INFO:root:If you are running ROCM/Metal, fork will cause compiler internal error. Try to launch with arg ```--no-fork```
INFO:RPCTracker:bind to 0.0.0.0:9190

I enabled RPC from the android application putting host:0.0.0.0, port=9190, key=android.

Then, I execute query_rpc_tracker after adding reverse connection, I get this message:

$ adb reverse tcp:9190 tcp:9190
9190
$ python3 -m tvm.exec.query_rpc_tracker --host=0.0.0.0 --port=9190
Tracker address 0.0.0.0:9190

Server List
----------------------------
server-address	key
----------------------------
127.0.0.1:34913	server:android
----------------------------

Queue Status
-------------------------------
key       total  free  pending
-------------------------------
android   1      1     0      
-------------------------------

So I think at least the host is recognizing the board. But, if I test android_rpc_test.py,

$ python3 /workspace/apps/android_rpc/tests/android_rpc_test.py

I get nothing except for a blank cursor, for like 30 minutes (I didn’t modify the test code), showing that something is running, then I see an error as below:

Traceback (most recent call last):
  File "android_rpc_test.py", line 129, in <module>
    test_rpc_module()
  File "android_rpc_test.py", line 57, in test_rpc_module
    remote = tracker.request(key)
  File "/workspace/python/tvm/rpc/client.py", line 401, in request
    "Cannot request %s after %d retry, last_error:%s" % (key, max_retry, str(last_err))
RuntimeError: Cannot request android after 5 retry, last_error:Traceback (most recent call last):
  [bt] (4) /workspace/build/libtvm.so(TVMFuncCall+0x61) [0x7fc7d1306bf1]
  [bt] (3) /workspace/build/libtvm.so(+0x127f052) [0x7fc7d136c052]
  [bt] (2) /workspace/build/libtvm.so(tvm::runtime::RPCClientConnect(std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> >, int, std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> >, tvm::runtime::TVMArgs)+0xdd) [0x7fc7d136b85d]
  [bt] (1) /workspace/build/libtvm.so(tvm::runtime::RPCConnect(std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> >, int, std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> >, tvm::runtime::TVMArgs)+0x28db) [0x7fc7d136a92b]
  [bt] (0) /workspace/build/libtvm.so(dmlc::LogMessageFatal::~LogMessageFatal()+0x67) [0x7fc7d0611767]
  File "/workspace/src/runtime/rpc/rpc_socket_impl.cc", line 73
TVMError: 
---------------------------------------------------------------
An internal invariant was violated during the execution of TVM.
Please read TVM's error reporting guidelines.
More details can be found here: https://discuss.tvm.ai/t/error-reporting/7793.
---------------------------------------------------------------
  Check failed: sock.Connect(addr) == false: Connect to 127.0.0.1:5001 failed

I traced the code “android_rpc_test.py” line by line and found out that the following line was causing a stall:

It seems like the host has sent the request but did not get an ACK from the board. Or could I have done something wrong?

Thanks