[frontend][ONNX] Fail to compile an ONNX model at opt_level=3

Troubleshooting
#1

Description

When I compile an ONNX model at opt_level=3, the compilation will fail with error:

DiagnosticError: Traceback (most recent call last):
  [bt] (8) 9   libtvm.dylib                        0x0000000117405f57 void std::__1::__invoke_void_return_wrapper<void>::__call<void tvm::runtime::TypedPackedFunc<tvm::IRModule (tvm::IRModule, tvm::transform::PassContext)>::AssignTypedLambda<tvm::relay::transform::InferType()::$_1>(tvm::relay::transform::InferType()::$_1)::'lambda'(tvm::runtime::TVMArgs const&, tvm::runtime::TVMRetValue*)&, tvm::runtime::TVMArgs, tvm::runtime::TVMRetValue*>(tvm::relay::transform::InferType()::$_1&&...) + 71
  [bt] (7) 8   libtvm.dylib                        0x0000000117405fc7 decltype(std::__1::forward<tvm::relay::transform::InferType()::$_1>(fp)(std::__1::forward<void tvm::runtime::TypedPackedFunc<tvm::IRModule (tvm::IRModule, tvm::transform::PassContext)>::AssignTypedLambda<tvm::relay::transform::InferType()::$_1>(tvm::relay::transform::InferType()::$_1)::'lambda'(tvm::runtime::TVMArgs const&, tvm::runtime::TVMRetValue*)&>(fp0)...)) std::__1::__invoke<void tvm::runtime::TypedPackedFunc<tvm::IRModule (tvm::IRModule, tvm::transform::PassContext)>::AssignTypedLambda<tvm::relay::transform::InferType()::$_1>(tvm::relay::transform::InferType()::$_1)::'lambda'(tvm::runtime::TVMArgs const&, tvm::runtime::TVMRetValue*)&, tvm::runtime::TVMArgs, tvm::runtime::TVMRetValue*>(tvm::relay::transform::InferType()::$_1&&, void tvm::runtime::TypedPackedFunc<tvm::IRModule (tvm::IRModule, tvm::transform::PassContext)>::AssignTypedLambda<tvm::relay::transform::InferType()::$_1>(tvm::relay::transform::InferType()::$_1)::'lambda'(tvm::runtime::TVMArgs const&, tvm::runtime::TVMRetValue*)&...) + 71
  [bt] (6) 7   libtvm.dylib                        0x00000001174066b4 void tvm::runtime::TypedPackedFunc<tvm::IRModule (tvm::IRModule, tvm::transform::PassContext)>::AssignTypedLambda<tvm::relay::transform::InferType()::$_1>(tvm::relay::transform::InferType()::$_1)::'lambda'(tvm::runtime::TVMArgs const&, tvm::runtime::TVMRetValue*)::operator()(tvm::runtime::TVMArgs const&, tvm::runtime::TVMRetValue*) const + 1748
  [bt] (5) 6   libtvm.dylib                        0x0000000117406bc7 tvm::relay::transform::InferType()::$_1::operator()(tvm::IRModule, tvm::transform::PassContext const&) const + 1063
  [bt] (4) 5   libtvm.dylib                        0x0000000115055643 tvm::DiagnosticContext::Render() + 467
  [bt] (3) 4   libtvm.dylib                        0x0000000114a3bd75 tvm::runtime::detail::LogFatal::~LogFatal() + 21
  [bt] (2) 3   libtvm.dylib                        0x0000000114a3dffd tvm::runtime::detail::LogFatal::~LogFatal() + 29
  [bt] (1) 2   libtvm.dylib                        0x0000000114a3e0ac tvm::runtime::detail::LogFatal::Entry::Finalize() + 156
  [bt] (0) 1   libtvm.dylib                        0x0000000117b10365 tvm::runtime::Backtrace() + 37
  File "/tvm/src/ir/diagnostic.cc", line 105
DiagnosticError: one or more error diagnostics were emitted, please check diagnostic render for output.

However, I can compile the model at opt_level=2

Code to reproduce

import tvm
import onnx

model_path = "efficientnet-lite4-11.onnx"
#opt_level = 2 is ok
opt_level = 3
target = "llvm"

onnx_model = onnx.load(model_path)
mod, params = tvm.relay.frontend.from_onnx(onnx_model)
with tvm.transform.PassContext(opt_level=opt_level):
    lib  = tvm.relay.build(mod, target=target, params=params)

Notice that at opt_level = 3, if you first apply some transformations to the relay graph:

seq = tvm.transform.Sequential([
    tvm.relay.transform.SimplifyInference(),
    tvm.relay.transform.FuseOps()
])
mod = seq(mod)

Then everything will be OK.

Model Download

Google drive: efficientnet-lite4-11.onnx - Google Drive

Environment

tvm version: 0.8.dev0

ONNX version: 1.8.1

OS: macOS 10.15.7

#2

@kevinthesun @haichen @merrymercy

I did some debugging, this is a type error is coming in when the compiler attempts to InferType after running AlterOpLayout. I’m honestly not sure why we have a shape problem if we run AlterOpLayout on the flat graph, but we don’t have an issue if we run the pass after partitioning. Any ideas or could one of you take a look?

#3

This model compiles without the workaround if we use a gpu target or add disabled_pass={"AlterOpLayout"} to PassContext. This also suggests this is AlterOpLayout related issue.

#4

@yuheng This issue was fixed in [Relay] Dense alter layout fixed for packed input by masahi · Pull Request #8669 · apache/tvm · GitHub, thanks for reporting!

2 Likes