Relay VM C++ runtime error

hsuan · September 2, 2020, 1:04pm

I am testing VM runtime with it’s c++ API, it worked with this commit a while ago.

But after I update to the lasted master branch, it breaks(I changed my code a bit due to the recent changes in vm apis.), I got the following error:

terminate called after throwing an instance of ‘dmlc::Error’ what(): [12:25:59] /home/developer/LocalLib/tvm/src/runtime/vm/executable.cc:460: Check failed: header == kTVMVMBytecodeMagic: Invalid VM file format in the header section.
Stack trace:
  [bt] (0) /home/developer/LocalLib/tvm_runtime/lib/libtvm_runtime.so(+0x964f2) [0x7f0129afe4f2]
  [bt] (1) /home/developer/LocalLib/tvm_runtime/lib/libtvm_runtime.so(tvm::runtime::vm::LoadHeader(dmlc::Stream*)+0xdf) [0x7f0129afec3f]
  [bt] (2) /home/developer/LocalLib/tvm_runtime/lib/libtvm_runtime.so(tvm::runtime::vm::Executable::Load(std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const&, tvm::runtime::Module)+0x135) [0x7f0129b062a5]

Here is my code to load the bytecode:

std::string vm_bytecode; std::ifstream bytecode_file(bytecode_path, std::ios::binary); vm_bytecode.assign(std::istreambuf_iterator(bytecode_file), std::istreambuf_iterator()); m_exec_mod = tvm::runtime::vm::Executable::Load(vm_bytecode, m_lib);

I guess the error is related to how I read the bytecode file, but I can’t find any examples of the recent changes, anyone know how to fix this?

zhiics · September 2, 2020, 4:08pm

I managed to use C++ APIs to load the artifacts recently. Can you verify the loaded vm_bytecode is valid? We have a utility function to load it here:

github.com

apache/incubator-tvm/blob/master/src/runtime/file_util.cc#L115



std::string GetMetaFilePath(const std::string& file_name) {
  size_t pos = file_name.find_last_of(".");
  if (pos != std::string::npos) {
    return file_name.substr(0, pos) + ".tvm_meta.json";
  } else {
    return file_name + ".tvm_meta.json";
  }
}

void LoadBinaryFromFile(const std::string& file_name, std::string* data) {
  std::ifstream fs(file_name, std::ios::in | std::ios::binary);
  CHECK(!fs.fail()) << "Cannot open " << file_name;
  // get its size:
  fs.seekg(0, std::ios::end);
  size_t size = static_cast<size_t>(fs.tellg());
  fs.seekg(0, std::ios::beg);
  data->resize(size);
  fs.read(&(*data)[0], size);
}

hsuan · September 2, 2020, 5:13pm

Hi Thanks, it turned out that the bytecode it self is invalid, the bytecode file was written as string which is the root of my issue.

Now the c++ apis works for me, thank you very much. the utile you pointed out is very useful.

wangxiang · January 20, 2021, 2:42am

Hi, i am also testing VM runtime with it’s c++ API, could you please share your code? Thankyou very much!