I used the same code to run. The first time I use llvm to autotune, it runs perfectly.
target = tvm.target.Target("llvm -mcpu=core-avx2")
when I switch to cuda, it suddenly break down when run this code. tuner.tune(tune_option)
target = tvm.target.cuda()
when using target="cuda"，the error indicates “Target has been reduced to 12 due to too many failures or duplications”
Generate Sketches #s: 2
Sample Iter: 5 #Pop: 10 #Target: 50 fail_ct: 10230 Time elapsed: 2.17
#Target has been reduced to 25 due to too many failures or duplications
Sample Iter: 10 #Pop: 10 #Target: 25 fail_ct: 20470 Time elapsed: 4.33
#Target has been reduced to 12 due to too many failures or duplications
Sample Iter: 15 #Pop: 10 #Target: 12 fail_ct: 30710 Time elapsed: 6.50
#Target has been reduced to 6 due to too many failures or duplications
Sample Initial Population #s: 10 fail_ct: 32758 Time elapsed: 6.95
GA Iter: 0 Max score: 0.8634 Min score: 0.0405 #Pop: 10 #M+: 0 #M-: 0
GA Iter: 4 Max score: 0.8634 Min score: 0.0405 #Pop: 10 #M+: 588 #M-: 5381
EvolutionarySearch #s: 10 Time elapsed: 1.50
----------------------------------------------------------------------
------------------------------ [ Measure ]
----------------------------------------------------------------------
Get 10 programs to measure:
..........*E
when using target = tvm.target.cuda(),it raises error when measuring programs
Generate Sketches #s: 1
Sample Initial Population #s: 145 fail_ct: 1903 Time elapsed: 1.78
GA Iter: 0 Max score: 0.9943 Min score: 0.8152 #Pop: 28 #M+: 0 #M-: 0
GA Iter: 4 Max score: 0.9998 Min score: 0.9972 #Pop: 28 #M+: 1392 #M-: 0
EvolutionarySearch #s: 28 Time elapsed: 15.88
----------------------------------------------------------------------
------------------------------ [ Measure ]
----------------------------------------------------------------------
Get 14 programs to measure:
..............*ETraceback (most recent call last):
File "/home/elle/anaconda3/envs/Lu1/lib/python3.9/runpy.py", line 197, in _run_module_as_main
return _run_code(code, main_globals, None,
File "/home/elle/anaconda3/envs/Lu1/lib/python3.9/runpy.py", line 87, in _run_code
exec(code, run_globals)
File "/home/elle/.vscode/extensions/ms-python.python-2021.12.1559732655/pythonFiles/lib/python/debugpy/__main__.py", line 45, in <module>
cli.main()
File "/home/elle/.vscode/extensions/ms-python.python-2021.12.1559732655/pythonFiles/lib/python/debugpy/../debugpy/server/cli.py", line 444, in main
run()
File "/home/elle/.vscode/extensions/ms-python.python-2021.12.1559732655/pythonFiles/lib/python/debugpy/../debugpy/server/cli.py", line 285, in run_file
runpy.run_path(target_as_str, run_name=compat.force_str("__main__"))
File "/home/elle/anaconda3/envs/Lu1/lib/python3.9/runpy.py", line 268, in run_path
return _run_module_code(code, init_globals, run_name,
File "/home/elle/anaconda3/envs/Lu1/lib/python3.9/runpy.py", line 97, in _run_module_code
_run_code(code, mod_globals, init_globals,
File "/home/elle/anaconda3/envs/Lu1/lib/python3.9/runpy.py", line 87, in _run_code
exec(code, run_globals)
File "/home/elle/bing/proj/code/tvm1-Image-Adaptive-3DLUT/use-LUT-model-and-TVM-to-tune.py", line 203, in <module>
run_tuning()
File "/home/elle/bing/proj/code/tvm1-Image-Adaptive-3DLUT/use-LUT-model-and-TVM-to-tune.py", line 201, in run_tuning
tuner.tune(tune_option)
File "/home/elle/bing/proj/tvm/python/tvm/auto_scheduler/task_scheduler.py", line 360, in tune
self._tune_task(idx)
File "/home/elle/bing/proj/tvm/python/tvm/auto_scheduler/task_scheduler.py", line 455, in _tune_task
measure_inputs, measure_results = self.search_policies[task_idx].continue_search_one_round(
File "/home/elle/bing/proj/tvm/python/tvm/auto_scheduler/search_policy.py", line 119, in continue_search_one_round
return _ffi_api.SearchPolicyContinueSearchOneRound(self, num_measure, measurer)
File "/home/elle/bing/proj/tvm/python/tvm/_ffi/_ctypes/packed_func.py", line 237, in __call__
raise get_last_ffi_error()
ValueError: Traceback (most recent call last):
[bt] (7) /home/elle/bing/proj/tvm/build/libtvm.so(TVMFuncCall+0x57) [0x7f0a9467a457]
[bt] (6) /home/elle/bing/proj/tvm/build/libtvm.so(+0x2f39b92) [0x7f0a93132b92]
[bt] (5) /home/elle/bing/proj/tvm/build/libtvm.so(tvm::auto_scheduler::SketchPolicyNode::ContinueSearchOneRound(int, tvm::auto_scheduler::ProgramMeasurer)+0x3a0) [0x7f0a93141ab0]
[bt] (4) /home/elle/bing/proj/tvm/build/libtvm.so(tvm::auto_scheduler::ProgramMeasurerNode::Measure(tvm::auto_scheduler::SearchTask const&, tvm::auto_scheduler::SearchPolicy const&, tvm::runtime::Array<tvm::auto_scheduler::MeasureInput, void> const&, int)+0x483) [0x7f0a931037d3]
[bt] (3) /home/elle/bing/proj/tvm/build/libtvm.so(tvm::auto_scheduler::ProgramMeasurerNode::SilentMeasure(tvm::auto_scheduler::SearchTask const&, tvm::runtime::Array<tvm::auto_scheduler::MeasureInput, void> const&, tvm::runtime::Array<tvm::auto_scheduler::MeasureResult, void>*)+0x101) [0x7f0a93101d21]
[bt] (2) /home/elle/bing/proj/tvm/build/libtvm.so(tvm::auto_scheduler::LocalRunnerNode::Run(tvm::runtime::Array<tvm::auto_scheduler::MeasureInput, void> const&, tvm::runtime::Array<tvm::auto_scheduler::BuildResult, void> const&, int)+0x19a) [0x7f0a93102eaa]
[bt] (1) /home/elle/bing/proj/tvm/build/libtvm.so(+0x2be9122) [0x7f0a92de2122]
[bt] (0) /home/elle/bing/proj/tvm/build/libtvm.so(tvm::runtime::Backtrace[abi:cxx11]()+0x2c) [0x7f0a9469833c]
File "/home/elle/bing/proj/tvm/python/tvm/_ffi/_ctypes/packed_func.py", line 81, in cfun
rv = local_pyfunc(*pyargs)
File "/home/elle/bing/proj/tvm/python/tvm/auto_scheduler/measure.py", line 1026, in local_run
res = call_func_with_timeout(
File "/home/elle/bing/proj/tvm/python/tvm/auto_scheduler/utils.py", line 293, in call_func_with_timeout
worker.send(func, args, kwargs, timeout)
File "/home/elle/bing/proj/tvm/python/tvm/contrib/popen_pool.py", line 244, in send
self._writer.write(struct.pack("<i", len(data)))
ValueError: write to closed file