Hi. I’m trying to run open sourced quantization method and deployment code with TVM here(https://github.com/zkkli/I-ViT). I’ve got good mAP of ViT-base by training with QAT. Then, I’ve tried thousand times to run autotuning with “evaluate_latency.py” but never succeeded. it always failed after some iteration with error like " Get 64 programs to measure: …E…E… ******************************.E…T.Exception in thread Thread-1 (_listen_loop): Traceback (most recent call last): File “/home/user/miniconda3/envs/tvm-build-venv/lib/python3.11/threading.py”, line 1045, in _bootstrap_inner self.run() File “/home/user/miniconda3/envs/tvm-build-venv/lib/python3.11/threading.py”, line 982, in run self._target(*self._args, **self._kwargs) File “/home/user/miniconda3/envs/tvm-build-venv/lib/python3.11/site-packages/tvm/rpc/server.py”, line 279, in _listen_loop _serving(conn, addr, opts, load_library) File “/home/user/miniconda3/envs/tvm-build-venv/lib/python3.11/site-packages/tvm/rpc/server.py”, line 168, in _serving raise RuntimeError( RuntimeError: Child process 752420 exited unsuccessfully with error code -11
^C*******ETraceback (most recent call last): File “/home/user/workspace/I-ViT2/TVM_benchmark/evaluate_latency.py”, line 128, in main() File “/home/user/workspace/I-ViT2/TVM_benchmark/evaluate_latency.py”, line 74, in main tuner.tune(tune_option) File “/home/user/miniconda3/envs/tvm-build-venv/lib/python3.11/site-packages/tvm/auto_scheduler/task_scheduler.py”, line 357, in tune self._tune_task(idx) File “/home/user/miniconda3/envs/tvm-build-venv/lib/python3.11/site-packages/tvm/auto_scheduler/task_scheduler.py”, line 458, in _tune_task measure_inputs, measure_results = self.search_policies[task_idx].continue_search_one_round( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File “/home/user/miniconda3/envs/tvm-build-venv/lib/python3.11/site-packages/tvm/auto_scheduler/search_policy.py”, line 119, in continue_search_one_round return _ffi_api.SearchPolicyContinueSearchOneRound(self, num_measure, measurer) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File “tvm/_ffi/_cython/./packed_func.pxi”, line 339, in tvm._ffi._cy3.core.PackedFuncBase.call File “tvm/_ffi/_cython/./packed_func.pxi”, line 270, in tvm._ffi._cy3.core.FuncCall File “tvm/_ffi/_cython/./packed_func.pxi”, line 259, in tvm._ffi._cy3.core.FuncCall3 File “tvm/_ffi/_cython/./base.pxi”, line 185, in tvm._ffi._cy3.core.CHECK_CALL File “/home/user/miniconda3/envs/tvm-build-venv/lib/python3.11/site-packages/tvm/_ffi/base.py”, line 420, in raise_last_ffi_error c_err_msg = py_str(_LIB.TVMGetLastError()) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File “/home/user/miniconda3/envs/tvm-build-venv/lib/python3.11/site-packages/tvm/_ffi/base.py”, line 53, in py_str = lambda x: x.decode(“utf-8”) ^^^^^^^^ AttributeError: ‘NoneType’ object has no attribute ‘decode’"
or
"Begin tuning… Get devices for measurement successfully!
------------------------------ [ Task Scheduler ]
| ID | Task Description | Latency (ms) | Speed (GFLOPS) | Trials |
| 0 | vm_mod_fused_nn_dense | - | - | 0 | | 1 | vm_mod_fused_nn_conv2d_add | - | - | 0 | | 2 | vm_mod_fused_nn_batch_matmul_1 | - | - | 0 | | 3 | vm_mod_fused_max | - | - | 0 | | 4 | vm_mod_fused_nn_softmax | - | - | 0 | | 5 | vm_mod_fused_nn_dense_add_cast_multiply | - | - | 0 | | 6 | vm_mod_fused_nn_batch_matmul | - | - | 0 | | 7 | vm_mod_fused_nn_dense_3 | - | - | 0 | | 8 | vm_mod_fused_nn_dense_2 | - | - | 0 | | 9 | vm_mod_fused_nn_dense_1 | - | - | 0 | | 10 | vm_mod_fused_max_1 | - | - | 0 | | 11 | vm_mod_fused_sum | - | - | 0 | | 12 | vm_mod_fused_multiply_cast_sum | - | - | 0 | | 13 | vm_mod_fused_mean | - | - | 0 |
Estimated total latency: - ms Trials: 0 Used time : 0 s Next ID: 0
------------------------------ [ Search ]
Generate Sketches #s: 1 Sample Initial Population #s: 166 fail_ct: 1882 Time elapsed: 1.71 GA Iter: 0 Max score: 0.9964 Min score: 0.2186 #Pop: 128 #M+: 0 #M-: 0 GA Iter: 4 Max score: 0.9999 Min score: 0.9788 #Pop: 128 #M+: 1390 #M-: 0 EvolutionarySearch #s: 128 Time elapsed: 20.46
------------------------------ [ Measure ]
Get 64 programs to measure: .E.E.E.E…E…E…E.E.E…E…E… *****************************************************Time elapsed for measurement: 76.92 s
------------------------------ [ Train cost model ]
Traceback (most recent call last):
File “/home/jseob/workspace/I-ViT/TVM_benchmark/evaluate_latency.py”, line 94, in
main()
File “/home/jseob/workspace/I-ViT/TVM_benchmark/evaluate_latency.py”, line 72, in main
tuner.tune(tune_option)
File “/home/jseob/miniconda3/envs/tvm-build-venv/lib/python3.11/site-packages/tvm/auto_scheduler/task_scheduler.py”, line 357, in tune
self._tune_task(idx)
File “/home/jseob/miniconda3/envs/tvm-build-venv/lib/python3.11/site-packages/tvm/auto_scheduler/task_scheduler.py”, line 458, in _tune_task
measure_inputs, measure_results = self.search_policies[task_idx].continue_search_one_round(
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File “/home/jseob/miniconda3/envs/tvm-build-venv/lib/python3.11/site-packages/tvm/auto_scheduler/search_policy.py”, line 119, in continue_search_one_round
return _ffi_api.SearchPolicyContinueSearchOneRound(self, num_measure, measurer)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File “tvm/_ffi/_cython/./packed_func.pxi”, line 339, in tvm._ffi._cy3.core.PackedFuncBase.call
File “tvm/_ffi/_cython/./packed_func.pxi”, line 270, in tvm._ffi._cy3.core.FuncCall
File “tvm/_ffi/_cython/./packed_func.pxi”, line 259, in tvm._ffi._cy3.core.FuncCall3
File “tvm/_ffi/_cython/./base.pxi”, line 185, in tvm._ffi._cy3.core.CHECK_CALL
File “/home/jseob/miniconda3/envs/tvm-build-venv/lib/python3.11/site-packages/tvm/_ffi/base.py”, line 481, in raise_last_ffi_error
raise py_err
File “tvm/_ffi/_cython/./packed_func.pxi”, line 56, in tvm._ffi._cy3.core.tvm_callback
File “/home/jseob/miniconda3/envs/tvm-build-venv/lib/python3.11/site-packages/tvm/auto_scheduler/cost_model/cost_model.py”, line 93, in update_func
self.update(inputs, results)
File “/home/jseob/miniconda3/envs/tvm-build-venv/lib/python3.11/site-packages/tvm/auto_scheduler/cost_model/xgb_model.py”, line 199, in update
dtrain = pack_sum_xgbmatrix(
^^^^^^^^^^^^^^^^^^^
File “/home/jseob/miniconda3/envs/tvm-build-venv/lib/python3.11/site-packages/tvm/auto_scheduler/cost_model/xgb_model.py”, line 419, in pack_sum_xgbmatrix
ret = xgb.DMatrix(np.array(x_flatten), y_flatten)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File “/home/jseob/miniconda3/envs/tvm-build-venv/lib/python3.11/site-packages/xgboost/core.py”, line 434, in init
handle, feature_names, feature_types = dispatch_data_backend(
^^^^^^^^^^^^^^^^^^^^^^
File “/home/jseob/miniconda3/envs/tvm-build-venv/lib/python3.11/site-packages/xgboost/data.py”, line 494, in dispatch_data_backend
return _from_numpy_array(data, missing, threads, feature_names,
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File “/home/jseob/miniconda3/envs/tvm-build-venv/lib/python3.11/site-packages/xgboost/data.py”, line 135, in _from_numpy_array
flatten = _transform_np_array(data)
^^^^^^^^^^^^^^^^^^^^^^^^^
File “/home/jseob/miniconda3/envs/tvm-build-venv/lib/python3.11/site-packages/xgboost/data.py”, line 115, in _transform_np_array
flatten = np.array(data.reshape(data.size), copy=False,
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
ValueError: Unable to avoid copy while creating an array as requested.
If using np.array(obj, copy=False) replace it with np.asarray(obj) to allow a copy when needed (no behavior change in NumPy 1.x).
For more details, see NumPy 2.0 migration guide — NumPy v2.2.dev0 Manual.
Exception ignored in: <function LocalRPCMeasureContext.del at 0x7f3599160400>
Traceback (most recent call last):
File “/home/jseob/miniconda3/envs/tvm-build-venv/lib/python3.11/site-packages/tvm/auto_scheduler/measure.py”, line 588, in del
File “/home/jseob/miniconda3/envs/tvm-build-venv/lib/python3.11/site-packages/tvm/rpc/tracker.py”, line 496, in terminate
File “/home/jseob/miniconda3/envs/tvm-build-venv/lib/python3.11/site-packages/tvm/contrib/popen_pool.py”, line 145, in kill
File “/home/jseob/miniconda3/envs/tvm-build-venv/lib/python3.11/site-packages/tvm/contrib/popen_pool.py”, line 43, in kill_child_processes
ImportError: sys.meta_path is None, Python is likely shutting down
Exception ignored in: <function Tracker.del at 0x7f35906c93a0>
Traceback (most recent call last):
File “/home/jseob/miniconda3/envs/tvm-build-venv/lib/python3.11/site-packages/tvm/rpc/tracker.py”, line 501, in del
File “/home/jseob/miniconda3/envs/tvm-build-venv/lib/python3.11/site-packages/tvm/rpc/tracker.py”, line 492, in terminate
File “/home/jseob/miniconda3/envs/tvm-build-venv/lib/python3.11/site-packages/tvm/rpc/tracker.py”, line 480, in _stop_tracker
ConnectionRefusedError: [Errno 111] Connection refused"
how to solve it?
Plus, I never see the correct result of same inference result between pytorch and TVM when I run the code “evaluate_accuracy.py” I tried to debug the code, but I got stuck after the patch embedding part, “layer.requantize”. the pre-defined input, output scale is strange like 74 or something like that(bigger than 1!). the printed requantized values are all -127 or 128. so all previous, various patch embedding values are lost.
Please help me~!