Does TVM support multithreaded inference? That is to have each thread load a precompiled .so into a module. Given the per thread modules, inference by the set_input, run, and get_output pattern. Thanks!
Does TVM support multithreaded inference? That is to have each thread load a precompiled .so into a module. Given the per thread modules, inference by the set_input, run, and get_output pattern. Thanks!