[SOLVED] Using contrib libraries with Android

1

I’d like to run some operations that live in contrib on android using TVM4J but can’t figure out how to get everything properly linked. As an example, when I try to run argsort over RPC I get the error:

TVMError: Except caught from RPC call: [12:30:32] /home/jwfromm/tvm/apps/android_rpc/app/src/main/jni/../../../../../../include/../src/runtime/module_util.cc:73: Check failed: ret == 0 (-1 vs. 0) : TVMError: Check failed: f != nullptr: Cannot find function tvm.contrib.sort.argsort in the imported modules or global registry

The same code works fine when run locally so it’s not an issue of TVM being built without these functions. My best guess on how to solve this is to explicitly tell either TVM4J or the android application itself to rebuild contrib sources but I’m not experienced enough with android to track down where this change can be made. Can anyone help point me in the right direction?

2

@yzhliu, it looks like you wrote nearly all of the JVM code so this might be right up your alley. I’d love to hear your thoughts on getting this to work.

3

argsort should be built by default. how did you invoke? does python rpc work well?

4

The android RPC server works great, I have no problem running examples that use standard tvm functions like conv2d. Here’s a snippet showing the code that yielded the error above.

target = 'llvm -target=arm64-linux-android'

x = relay.var('x', shape=[100], dtype='float32')
s = relay.argsort(x)
mod = relay.Function([x], s)
params = {}

with relay.build_config(opt_level=3):
    graph, lib, params = relay.build(mod, target=target, params=params)

lib.export_library('net.so', ndk.create_shared)
tracker = rpc.connect_tracker('0.0.0.0', 9190)
remote = tracker.request('android', priority=0, session_timeout=60)

remote.upload('net.so')
rlib = remote.load_module('net.so')
module = runtime.create(graph, rlib, remote.cpu(0))

x_np = np.random.normal(size=[100]).astype('float32')
module.set_input('x', x_np)
module.set_input(**params)
module.run()

To be clear, the above code works fine when argsort is replaced by some other function such as relay.pow. I built JVM and the android_rpc app exactly according to the guide at https://github.com/dmlc/tvm/tree/master/apps/android_rpc.

Is the tvm runtime rebuilt for JVM or is it somehow just linking in the library built during the normal TVM build process? If it’s rebuilding, USE_SORT may be off by default however I can’t locate what to change to fix this as I’m not very experienced with android build systems.

5

Not sure if this is useful but I added:

#include "../src/contrib/sort/sort.cc"

to apps/android_rpc/app/src/main/jni/tvm_runtime.h and rebuilt the rpc app. This changes the error from the one posted above to:

TVMError: Except caught from RPC call: [11:22:36] /home/jwfromm/tvm/apps/android_rpc/app/src/main/jni/../../../../../../include/../src/runtime/module_util.cc:73: Check failed: ret == 0 (-1 vs. 0) : TVMError: Check failed: f: Function _datatype_get_type_registered not found

So it seems like it can at least find the proper symbols now. However, I’m still not sure what’s missing.

6

You actually find a bug in the code that requires us to hit the customized data type codepath by default, which we should not. cc @gussmith23

7

cc @jwfromm can you also send a PR to add sort directly to the android runtime?

8

Your commit does fix the issue, thanks Tianqi! I’d be happy to send a PR to add sort to android, and will push it soon.

1 Like
9

Thanks for finding this bug all!