Is there a way to use a model with static input but dynamic output?

Questions
#1

I am trying to run LightGlue. I can fix the input shape to be static but due to the nature of the model the output cannot be fixed. This is causing issues with relay.build where it requires static output.

I have also tried

executor = relay.build_module.create_executor(
        "graph", mod, tvm.cpu(0), target, params
    )

which also runs into the same issues of course

What I did think would work is

vm_exec = relay.vm.compile(mod, target=target, params=params)

But I get errors with that too

InternalError: Check failed: undefined.size() == 0 (1 vs. 0) : In PrimFunc vm_mod_fused_multiply_sum variables [ax0_ax1_fused_ax2_fused_ax3_fused_ax4_fused_inner] are used, but are not passed in as API arguments

To clarify, the conversion to relay module from onnx runs without any hiccups. I am just not able to build it to be able to run on my OpenCL target.

Are there known fixes for this? Performance is not really a concern at this point, I just need it working. If I have a functional relay module, would it be possible to just interpret it directly on my target?

#2

I managed to get it to compile by changing the target to “opencl” from “opencl -device=adreno”, but do I do after I get the vm_exec?

#3

Not sure if relay can do, but relax should work :slight_smile:

#4

I tried relax but the converter from onnx frontend is missing too many op translations unlike relay

#5

Please tell which ops are missing if possible. As far as I know, relax have already supported most of the operators that relay supports