[relay] fails build on stride parameter for hardware target

Troubleshooting
#1

I am following the guide at 3. microTVM AOT

The code executes fine with a custom model when the target is set to cpu/default x86, but when trying to compile for a hardware such as an STM target, the build seems to fail on some stride layer:

Traceback (most recent call last):
  File "tflite_aot.py", line 63, in <module>
    relay_mod, target=TARGET, params=params, runtime=RUNTIME, executor=EXECUTOR
  File "/home/arseni/repositories/tvm/python/tvm/relay/build_module.py", line 372, in build
    mod_name=mod_name,
  File "/home/arseni/repos
...100 lines of error
  0: tvm::tir::BinderAddAssert(tvm::arith::Analyzer*, tvm::PrimExpr, std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const&, std::vector<tvm::tir::Stmt, std::allocator<tvm::tir::Stmt> >*)
  File "/home/arseni/repositories/tvm/src/tir/transforms/arg_binder.cc", line 40
TVMError: Bind have an unmet assertion: T.bool(False),  on argument data.strides[1]

Code for context:

INPUT_NAME = "serving_default_images_0"
input_shape = (1,192,192,3)
input_dtype = "int8"

relay_mod, params = relay.frontend.from_tflite(
    tflite_model, shape_dict={INPUT_NAME: input_shape}, dtype_dict={INPUT_NAME: input_dtype}
)

sample = get_sample_input(input_tensor_name=INPUT_NAME, type=input_dtype, nchw=False)

# Use the C runtime (crt) and enable static linking by setting system-lib to True
RUNTIME = Runtime("crt", {"system-lib": True})

# Simulate a microcontroller on the host machine. Uses the main() from `src/runtime/crt/host/main.cc`.
# To use physical hardware, replace "host" with something matching your hardware.
TARGET = tvm.micro.testing.get_target("crt")

# Use the AOT executor rather than graph or vm executors. Don't use unpacked API or C calling style.
EXECUTOR = Executor("aot")

if use_physical_hw:
    BOARD = os.getenv("TVM_MICRO_BOARD", default="stm32f746g_disco")
    SERIAL = os.getenv("TVM_MICRO_SERIAL", default=None)
    TARGET = tvm.micro.testing.get_target("zephyr", BOARD)

with tvm.transform.PassContext(opt_level=3, config={"tir.disable_vectorize": True}):
    module = tvm.relay.build(
        relay_mod, target=TARGET, params=params, runtime=RUNTIME, executor=EXECUTOR
    )

Is there anything inherent to HW that needs to be changed that is not being accounted for in the graph? Or is my config wrong?

#2

Hey,

I am using TVM for one of my STM boards, here is the code I use for that, maybe it’s helpful for you:

mod, params = relay.frontend.from_tflite(
    tflite_model,
    shape_dict={
        "serving_default_x:0": (batch_size, input_width),
    },
    dtype_dict={
        "serving_default_x:0": input_dtype,
    },
)


# Use the C runtime (crt) 
RUNTIME = Runtime("crt", {"system-lib": False})

# Simulate a microcontroller on the host machine. Uses the main() from `src/runtime/crt/host/main.cc`.
# To use physical hardware, replace "host" with something matching your hardware.
TARGET = tvm.target.target.stm32("stm32F7xx")

# Use the AOT executor rather than graph or vm executors.
EXECUTOR = Executor("aot", options={"interface-api": "c", "unpacked-api": 1})

config = {"tir.disable_vectorize": True}
config["tir.usmp.enable"] = True 

with tvm.transform.PassContext(opt_level=3, config=config):
    if USE_CMSIS_NN:
        mod = cmsisnn.partition_for_cmsisnn(mod, params, mcpu=TARGET.mcpu)
    lowered = tvm.relay.build(
        mod, target=TARGET, params=params, runtime=RUNTIME, executor=EXECUTOR
    )
    #tasks = tvm.autotvm.task.extract_from_program(mod["main"], {}, TARGET)
1 Like
#3

This makes my code build. Thank you. I tried adding lines one-by-one and the change was at:

with tvm.transform.PassContext(opt_level=3, config=config):
    if USE_CMSIS_NN:
        mod = cmsisnn.partition_for_cmsisnn(mod, params, mcpu=TARGET.mcpu)
    lowered = tvm.relay.build(
        mod, target=TARGET, params=params, runtime=RUNTIME, executor=EXECUTOR
    )

And then passing the lowered build into generate the project. It seems like perhaps the default TFLITE relay builder has some edge case for the stride parameter that CMSIS is able to handle.