Is there any guide for adding a new backend to microTVM?

leoluopy · February 27, 2023, 9:12am

now i have a Cadence dsp [a new hardware] , and i am developing a Soc system. i want to integrate the dsp backend into microTVM ， is there any guide for me ?

leoluopy · March 2, 2023, 6:43am

anyone know some suggestion to this question ?

areusch · March 2, 2023, 11:48pm

hi @leoluopy, could you tell us a little bit more about your system? is there any link to the particular dsp you’re trying to use? i’m not sure if you’re just trying to use specific asm with your CPU (i.e. SIMD-type instructions such as ARM v8-M DSP extensions), or whether you’re trying to control a peripheral accelerator (similar to UMA work done here).

leoluopy · March 3, 2023, 1:12am

well , thanks for responding . @areusch

now i am using Cadence DSP IP , the version is vision Q7 . and we are using a RISC-V cpu to control this DSP . while inferencing the DNN network , some operator can not be inferenced by NPU then we have to use DSP to accelerate it . so my questiong comes , in order to use microTVM to tune the operator to accelerate it while inferencing at DSP , so how to add my DSP backend to microTVM .
the DSP vendor gives us both the function libraries and assembly DSP instructions . any suggestions ?

areusch · March 3, 2023, 6:35pm

got it. I believe this is very much an NPU controlled over APB, so that means you’re probably looking for UMA. cc @PhilippvK @MJKlaiber @r.stahl @cgerum

leoluopy · March 4, 2023, 1:56am

yes , i just refered to the example of UMA . UMA demonstrates how to add a new conv2d backed to the framework . and it is using the x86 default gcc compiler , how can i change compiler to DSA compiler like xt-clang , risc-v compiler and so on ?

MJKlaiber · March 7, 2023, 12:58pm

@leoluopy, essentially UMA uses the C target. From that point on you can use any compiler you like. Have a look at compile_and_run in tvm.testing.aot

github.com

apache/tvm/blob/main/python/tvm/testing/aot.py#L874




        return True


    if test_dir is None:
        tmpdir = utils.tempdir()
        return run_and_check_body(os.path.join(tmpdir.path, "test"))
    else:
        return run_and_check_body(test_dir)




def compile_and_run(
    models: Union[List[AOTTestModel], AOTTestModel],
    runner: AOTTestRunner,
    interface_api: str,
    use_unpacked_api: bool,
    debug_calculated_workspaces: bool = False,
    workspace_byte_alignment: int = 8,
    constant_byte_alignment: int = 8,
    enable_op_fusion: bool = True,
    data_linkage: AOTDataLinkage = None,
    use_runtime_executor: bool = True,

areusch · March 7, 2023, 5:29pm

@leoluopy

@MJKlaiber is correct, for UMA, TVM typically generates C code. This is typically collected into a Model Library Format tarball, which can then be passed along to a downstream compiler.

You can emit assembly instructions and various other compiler-specific code via pragma import_c (TE-based example).

You can also leverage the LLVM backend, which can be used to directly generate LLVM IR. We don’t presently have an example of this with UMA, and it’s not as tested with RISC-V. However, there is a corresponding import_llvm pragma you can use there.