Time evaluator for VM

Hi, what is the correct way to use time evaluator with VM runtime? Or is it supported at all? I quickly tried the following

    vm_exec = relay.vm.compile(mod, target=target, params=params)
    vm = VirtualMachine(vm_exec, ctx)
    ftimer = vm.module.time_evaluator("run", ctx, number=1, repeat=50)

but got the error

tvm._ffi.base.TVMError: Traceback (most recent call last):
  [bt] (4) /mnt/2e797a66-fd2b-44fc-a3ba-24d7d65f2780/projects/dev/tvm/build/libtvm.so(TVMFuncCall+0x69) [0x7f1b88d013f9]
  [bt] (3) /mnt/2e797a66-fd2b-44fc-a3ba-24d7d65f2780/projects/dev/tvm/build/libtvm.so(+0x11e0302) [0x7f1b88d6b302]
  [bt] (2) /mnt/2e797a66-fd2b-44fc-a3ba-24d7d65f2780/projects/dev/tvm/build/libtvm.so(tvm::runtime::ModuleNode::GetFunction(std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const&, bool)+0x4d) [0x7f1b88d1efcd]
  [bt] (1) /mnt/2e797a66-fd2b-44fc-a3ba-24d7d65f2780/projects/dev/tvm/build/libtvm.so(tvm::runtime::vm::VirtualMachine::GetFunction(std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const&, tvm::runtime::ObjectPtr<tvm::runtime::Object> const&)+0x279) [0x7f1b88d4d169]
  [bt] (0) /mnt/2e797a66-fd2b-44fc-a3ba-24d7d65f2780/projects/dev/tvm/build/libtvm.so(+0x11bd5bb) [0x7f1b88d485bb]
  File "/home/masa/projects/dev/tvm/src/runtime/vm/vm.cc", line 173
TVMError: Unknown packed function: run

cc @zhiics @kevinthesun

vm.module.time_evaluator("invoke", ctx, number=1, repeat=10)

ok, how do you run the timer? It looks like I need to pass arguments. If I do

    ftimer = vm.module.time_evaluator("invoke", ctx, number=1, repeat=50)
    prof_res = np.array(ftimer().results) * 1e3

I got

TVMError: Check failed: i < num_args (0 vs. 0) : not enough argument passed, 0 passed but request arg[0].

If I pass a numpy array, I got

    prof_res = np.array(ftimer(data.numpy()).results) * 1e3
  File "/home/masa/projects/dev/tvm/python/tvm/runtime/module.py", line 225, in evaluator
    blob = feval(*args)
  File "/home/masa/projects/dev/tvm/python/tvm/_ffi/_ctypes/packed_func.py", line 223, in __call__
    values, tcodes, num_args = _make_tvm_args(args, temp_args)
  File "/home/masa/projects/dev/tvm/python/tvm/_ffi/_ctypes/packed_func.py", line 188, in _make_tvm_args
    raise TypeError("Don't know how to handle type %s" % type(arg))
TypeError: Don't know how to handle type <class 'numpy.ndarray'>

If I pass TVM ndarray, I got

TVMError: Check failed: type_code_ == kTVMStr (13 vs. 11) : expected str but get NDArrayContainer

I think you can initialize the VM and set up inputs first first (or use run with args). Then you can do:

vm.module.time_evaluator("invoke", ctx, number=1, repeat=10)

here is an example:

1 Like

Thanks, good to know there is an example. The following seems to work. Passing a numpy array won’t work, need to wrap with tvm ndarray.

ftimer("main", tvm.nd.array(np_arr))

The following also works, if the argument is set beforehand

ftimer("main")