Build relay op for ARM and serialize to shared object

Development
#1

Hi,

I would like to use TVM to:

  1. (offline) build a relay op for ARM
  2. (offline) export the module to .so
  3. (runtime) load the lib back to LLVM module and call the op using TVM runtime, on some DLTensor or similar

This code works for X86:

    v0 = relay.var('v0', dtype="float32", shape=[])
    v1 = relay.var('v1', dtype="float32", shape=[])
    add = v0+v1

    mod = tvm.IRModule.from_expr(add)

    x_data = tvm.nd.array(np.array(10, dtype="float32"))
    y_data = tvm.nd.array(np.array(3, dtype="float32"))
    res_data = tvm.nd.array(np.zeros((), dtype="float32"))
    _, lib, _ = relay.build(mod, target="llvm")
    # _, lib, _ = relay.build(mod, target="llvm -device=arm_cpu -mtriple=aarch64-linux-gnu")
    lib.export_library("./arm_poc.so")

   
    loaded_lib = tvm.runtime.load_module("./arm_poc.so")
    print(res_data)
    loaded_lib["tvmgen_default_fused_add"](x_data, y_data,res_data)
    print(res_data)

I can see a shared object generated from the export_library() call and can later reload it, retrieve the packed function for the add op and call it. However, for ARM, the export_library() fails. Note that this is cross compilation. edit: I had to save as tar (static objects lib)

this is the error I get:

Compilation error:
/usr/bin/ld: /tmp/tmpl9gqeezg/lib0.o: Relocations in generic ELF (EM: 183)
/usr/bin/ld: /tmp/tmpl9gqeezg/lib0.o: Relocations in generic ELF (EM: 183)
/usr/bin/ld: /tmp/tmpl9gqeezg/lib0.o: Relocations in generic ELF (EM: 183)
/usr/bin/ld: /tmp/tmpl9gqeezg/lib0.o: Relocations in generic ELF (EM: 183)
/usr/bin/ld: /tmp/tmpl9gqeezg/lib0.o: Relocations in generic ELF (EM: 183)
/usr/bin/ld: /tmp/tmpl9gqeezg/lib0.o: error adding symbols: file in wrong format
collect2: error: ld returned 1 exit status

Command line: g++ -shared -fPIC -o ./arm_poc.so /tmp/tmpl9gqeezg/lib0.o

obviously I do not expect the execution to work since I am on an X86 machine but the serialization I do. so my question is how to serialize a shared object for ARM (cross compiling from X86)

A little background for those interested: basically I am trying to achieve sort of “fallback to ARM” for some ops where the ARM acts as an accelerator and NOT THE HOST. I have some sort of unique HOST, unable to execute any op so I need the ARM as fallback. In that case, the TVM runtime can execute on ARM as an accelerator. I cannot use ARM as HOST for other reasons though.

Thanks

edit: turns that I had to export to ‘.tar’ as static lib so now I can see my ELF object inside which looks fine. How can I reload it to runtime::module though? I get a linker error on X86 but this is probably because the ELF is for ARM. I cannot quickly check but is this supposed to work on an ARM machine:?

 _, lib, _ = relay.build(mod, target="llvm -mtriple=aarch64-linux-gnu")                
lib.export_library("./arm_poc.tar")
loaded_lib = tvm.runtime.load_module("./arm_poc.tar")
loaded_lib["tvmgen_default_fused_add"](x_data, y_data,res_data)