AssertionError: assert expr.struct_info.shape is not None

Troubleshooting
#1 
File "/Users/jiuzhao/Workspace/AIWokrspace/mlc-llm/python/mlc_llm/model/qwen2_vl/qwen2_vl_model.py", line 128, in apply_rotary_emb_torch
output = x * cos
         ~~^~~~~
  File "/Users/jiuzhao/miniconda3/envs/mlc_base/lib/python3.11/site-packages/tvm/relax/frontend/nn/_tensor_op.py", line 60, in __mul__
return _op().multiply(self, other)
       ^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Users/jiuzhao/miniconda3/envs/mlc_base/lib/python3.11/site-packages/tvm/relax/frontend/nn/op.py", line 161, in multiply
return wrap_nested(_op.multiply(a._expr, b._expr), name)
       ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Users/jiuzhao/miniconda3/envs/mlc_base/lib/python3.11/site-packages/tvm/relax/frontend/nn/core.py", line 596, in wrap_nested
return Tensor(_expr=expr)
       ^^^^^^^^^^^^^^^^^^
  File "/Users/jiuzhao/miniconda3/envs/mlc_base/lib/python3.11/site-packages/tvm/relax/frontend/nn/core.py", line 116, in __init__
_check_tensor(_expr)
  File "/Users/jiuzhao/miniconda3/envs/mlc_base/lib/python3.11/site-packages/tvm/relax/frontend/nn/core.py", line 111, in _check_tensor
assert expr.struct_info.shape is not None
       ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
AssertionError

The two tensor shape is:

cos=Tensor([1, resized_height // T.int64(28) * (resized_width // T.int64(28)) * T.int64(4), 16, 80], "float32")
x=Tensor([1, resized_height // T.int64(14) * (resized_width // T.int64(14)), 16, 80], "float32")

Is this a problem with dynamic tensor shape?

#2

Note that x // 2 * 2 is not equal to x. The same

[1, resized_height // T.int64(28) * (resized_width // T.int64(28)) * T.int64(4), 16, 80]

is not euqal to

resized_height // T.int64(14) * (resized_width // T.int64(14))
#3

Thanks for your reply, in my case, the resized_height and resized_width is divisible by 28, so resized_height // T.int64(28) * T.int64(2) is equal to resized_height // T.int64(14).

#4

if you have extra knowledge, you can try use match_cast to convert to the shape you desire and it will turn into a runtime check

1 Like
#5

Really thanks, it’s worked. After all that issues fixed, the model can be exported successfully, but when compile the model, I got the following error, and I didn’t have any clue how to figure out which code is causing the problem:

[2024-10-10 20:33:57] INFO pipeline.py:54: Lowering to TVM TIR kernels
Traceback (most recent call last):
  File "/Users/jiuzhao/miniconda3/envs/mlc_base/bin/mlc_llm", line 33, in <module>
    sys.exit(load_entry_point('mlc-llm', 'console_scripts', 'mlc_llm')())
             ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Users/jiuzhao/Workspace/AIWokrspace/mlc-llm/python/mlc_llm/__main__.py", line 33, in main
    cli.main(sys.argv[2:])
  File "/Users/jiuzhao/Workspace/AIWokrspace/mlc-llm/python/mlc_llm/cli/compile.py", line 129, in main
    compile(
  File "/Users/jiuzhao/Workspace/AIWokrspace/mlc-llm/python/mlc_llm/interface/compile.py", line 243, in compile
    _compile(args, model_config)
  File "/Users/jiuzhao/Workspace/AIWokrspace/mlc-llm/python/mlc_llm/interface/compile.py", line 188, in _compile
    args.build_func(
  File "/Users/jiuzhao/Workspace/AIWokrspace/mlc-llm/python/mlc_llm/support/auto_target.py", line 311, in build
    relax.build(
  File "/Users/jiuzhao/miniconda3/envs/mlc_base/lib/python3.11/site-packages/tvm/relax/vm_build.py", line 335, in build
    mod = pipeline(mod)
          ^^^^^^^^^^^^^
  File "/Users/jiuzhao/miniconda3/envs/mlc_base/lib/python3.11/site-packages/tvm/ir/transform.py", line 238, in __call__
    return _ffi_transform_api.RunPass(self, mod)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "tvm/_ffi/_cython/./packed_func.pxi", line 339, in tvm._ffi._cy3.core.PackedFuncBase.__call__
  File "tvm/_ffi/_cython/./packed_func.pxi", line 270, in tvm._ffi._cy3.core.FuncCall
  File "tvm/_ffi/_cython/./packed_func.pxi", line 259, in tvm._ffi._cy3.core.FuncCall3
  File "tvm/_ffi/_cython/./base.pxi", line 185, in tvm._ffi._cy3.core.CHECK_CALL
  File "/Users/jiuzhao/miniconda3/envs/mlc_base/lib/python3.11/site-packages/tvm/_ffi/base.py", line 481, in raise_last_ffi_error
    raise py_err
  File "tvm/_ffi/_cython/./packed_func.pxi", line 56, in tvm._ffi._cy3.core.tvm_callback
  File "/Users/jiuzhao/Workspace/AIWokrspace/mlc-llm/python/mlc_llm/compiler_pass/pipeline.py", line 187, in _pipeline
    mod = seq(mod)
          ^^^^^^^^
  File "/Users/jiuzhao/miniconda3/envs/mlc_base/lib/python3.11/site-packages/tvm/ir/transform.py", line 238, in __call__
    return _ffi_transform_api.RunPass(self, mod)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "tvm/_ffi/_cython/./packed_func.pxi", line 339, in tvm._ffi._cy3.core.PackedFuncBase.__call__
  File "tvm/_ffi/_cython/./packed_func.pxi", line 270, in tvm._ffi._cy3.core.FuncCall
  File "tvm/_ffi/_cython/./packed_func.pxi", line 259, in tvm._ffi._cy3.core.FuncCall3
  File "tvm/_ffi/_cython/./base.pxi", line 185, in tvm._ffi._cy3.core.CHECK_CALL
  File "/Users/jiuzhao/miniconda3/envs/mlc_base/lib/python3.11/site-packages/tvm/_ffi/base.py", line 481, in raise_last_ffi_error
    raise py_err
tvm._ffi.base.TVMError: Traceback (most recent call last):
  File "/Users/catalyst/Workspace/mlc-ai-package-self-runner/_work/package/package/tvm/include/tvm/tir/expr_functor.h", line 154
TVMError: Do not have a default for tir.Call