Hello all,
I have tried following the C++ Deployment TVM Sample, which works perfectly for x86/.so compiled model binaries, but this is not the case for .tar binaries for arm.
Compilation of the C++ sample works fine and is tested to without any issues on x86, but arm is giving me this error:
pi@proteus:$ ./main -i ../test/airplane.jpg
terminate called after throwing an instance of 'tvm::runtime::InternalError'
what(): [17:54:11] runtime/../../../../../tvm/src/runtime/module.cc:87:
---------------------------------------------------------------
An internal invariant was violated during the execution of TVM.
Please read TVM's error reporting guidelines.
More details can be found here: https://discuss.tvm.ai/t/error-reporting/7793.
---------------------------------------------------------------
Check failed: (f != nullptr) is false: Loader of (runtime.module.loadfile_tar) is not presented.
Stack trace:
[bt] (0) ./main(+0x165a4) [0x5590ae65a4]
[bt] (1) ./main(+0x102ec) [0x5590ae02ec]
[bt] (2) ./main(+0x23120) [0x5590af3120]
[bt] (3) ./main(+0x10634) [0x5590ae0634]
[bt] (4) ./main(+0xa550) [0x5590ada550]
[bt] (5) /lib/aarch64-linux-gnu/libc.so.6(__libc_start_main+0xe4) [0x7f9906bd24]
[bt] (6) ./main(+0xdb18) [0x5590addb18]
Aborted
I am feeding in the model through LoadFromFile:
int dtype_code = kDLFloat;
int dtype_bits = 32;
int dtype_lanes = 1;
int device_type = kDLCPU;
int device_id = 0;
int in_ndim = 4;
int64_t in_shape[4] = {1, 3, 224, 224};
int in_size = (1 * 3 * 224 * 224);
// Initialize Runtime
LOG(INFO) << "Loading Binary...";
DLContext ctx{kDLCPU, 0};
tvm::runtime::Module mod_factory = tvm::runtime::Module::LoadFromFile(model_path);
LOG(INFO) << "Creating Runtime...";
gettimeofday(&t0, 0);
tvm::runtime::Module gmod = mod_factory.GetFunction("default")(ctx);
tvm::runtime::PackedFunc set_input = gmod.GetFunction("set_input");
tvm::runtime::PackedFunc get_output = gmod.GetFunction("get_output");
tvm::runtime::PackedFunc run = gmod.GetFunction("run");
gettimeofday(&t1, 0);
set_input(input_name, tvm_input);
Digging into /tvm/src/runtime/module.cc, I noticed there is no implementation for .tar:
Module Module::LoadFromFile(const std::string& file_name, const std::string& format) {
std::string fmt = GetFileFormat(file_name, format);
ICHECK(fmt.length() != 0) << "Cannot deduce format of file " << file_name;
if (fmt == "dll" || fmt == "dylib" || fmt == "dso") {
fmt = "so";
}
std::string load_f_name = "runtime.module.loadfile_" + fmt;
const PackedFunc* f = Registry::Get(load_f_name);
ICHECK(f != nullptr) << "Loader of " << format << "(" << load_f_name << ") is not presented.";
Module m = (*f)(file_name, format);
return m;
}
Are there any current fixes around this issue?
