Error when quantizing MobilenetV2 model from MxNet

Troubleshooting
#1

Hi all, I’m getting the following rhs error while using Relay to quantize MobileNetV2 from the MxNet Gluon model zoo. The code below is mainly from the test script below found in the tvm github: https://github.com/apache/incubator-tvm/blob/main/tests/python/nightly/quantization/test_quantization_accuracy.py

The error is coming from the Expr Conv2dRealize() function in realize.cc. Does anyone have any ideas as for a solution to this error? Or would it be better to simply try quantizing MobileNetV2 through PyTorch instead of MxNet?

The code and error is listed below. Thanks.

model_name="mobilenetv2_1.0"
target = "llvm"
ctx = tvm.cpu(0)
batch_size = 1 
qconfig = qtz.qconfig(
    skip_conv_layers=[0],
    nbit_input=8,
    nbit_weight=8,
    global_scale=4.0,
    dtype_input="int8",
    dtype_weight="int8",
    dtype_activation="int32",
    debug_enabled_ops=None,
    )

gluon_model = gluon.model_zoo.vision.get_model(model_name, pretrained=True)
data_shape = (batch_size, 3, img_size, img_size)
mod, params = relay.frontend.from_mxnet(gluon_model, {"data": data_shape})

with tvm.transform.PassContext(opt_level=3):
    qfunc = relay.quantize.prerequisite_optimize(net, params=params)
logging.debug("original")
logging.debug(qfunc.astext(show_meta_data=False))

with qconfig:
    logging.debug("current quantize config")
    logging.debug(qtz.current_qconfig())
    qfunc = qtz.quantize(qfunc)
    logging.debug("after quantize")
    logging.debug(qfunc.astext(show_meta_data=False))

with tvm.transform.PassContext(opt_level=3):
    graph, lib, params = relay.build(qfunc, target)
   
m = tvm.contrib.graph_runtime.create(graph, lib, ctx)
m.set_input(**params)

# Testing Quantization For Now

TVMError below

Traceback (most recent call last):
  File "/tvm/EE464H/quantize.py", line 43, in <module>
    qfunc = qtz.quantize(qfunc)
  File "/home/tjackson/.local/lib/python3.6/site-packages/tvm-0.8.dev0-py3.6-linux-x86_64.egg/tvm/relay/quantize/quantize.py", line 371, in quantize
    mod = quantize_seq(mod)
  File "/home/tjackson/.local/lib/python3.6/site-packages/tvm-0.8.dev0-py3.6-linux-x86_64.egg/tvm/ir/transform.py", line 127, in __call__
    return _ffi_transform_api.RunPass(self, mod)
  File "/home/tjackson/.local/lib/python3.6/site-packages/tvm-0.8.dev0-py3.6-linux-x86_64.egg/tvm/_ffi/_ctypes/packed_func.py", line 237, in __call__
    raise get_last_ffi_error()
tvm._ffi.base.TVMError: Traceback (most recent call last):
  [bt] (8) /home//.local/lib/python3.6/site-packages/tvm-0.8.dev0-py3.6-linux-x86_64.egg/tvm/libtvm.so(tvm::relay::MixedModeMutator::DispatchVisitExpr(tvm::RelayExpr const&)+0x1d) [0x7fcd9d7db59d]
  [bt] (7) /home/tjackson/.local/lib/python3.6/site-packages/tvm-0.8.dev0-py3.6-linux-x86_64.egg/tvm/libtvm.so(tvm::relay::ExprMutator::VisitExpr(tvm::RelayExpr const&)+0x8e) [0x7fcd9d7db52e]
  [bt] (6) /home/tjackson/.local/lib/python3.6/site-packages/tvm-0.8.dev0-py3.6-linux-x86_64.egg/tvm/libtvm.so(tvm::relay::ExprFunctor<tvm::RelayExpr (tvm::RelayExpr const&)>::VisitExpr(tvm::RelayExpr const&)+0x82) [0x7fcd9d7e08b2]
  [bt] (5) /home/tjackson/.local/lib/python3.6/site-packages/tvm-0.8.dev0-py3.6-linux-x86_64.egg/tvm/libtvm.so(tvm::relay::ExprFunctor<tvm::RelayExpr (tvm::RelayExpr const&)>::InitVTable()::{lambda(tvm::runtime::ObjectRef const&, tvm::relay::ExprFunctor<tvm::RelayExpr (tvm::RelayExpr const&)>*)#6}::_FUN(tvm::runtime::ObjectRef const&, tvm::relay::ExprFunctor<tvm::RelayExpr (tvm::RelayExpr const&)>*)+0x27) [0x7fcd9d679f97]
  [bt] (4) /home/tjackson/.local/lib/python3.6/site-packages/tvm-0.8.dev0-py3.6-linux-x86_64.egg/tvm/libtvm.so(tvm::relay::MixedModeMutator::VisitExpr_(tvm::relay::CallNode const*)+0x96) [0x7fcd9d5ff116]
  [bt] (3) /home/tjackson/.local/lib/python3.6/site-packages/tvm-0.8.dev0-py3.6-linux-x86_64.egg/tvm/libtvm.so(tvm::relay::ForwardRewriter::Rewrite_(tvm::relay::CallNode const*, tvm::RelayExpr const&)+0x13af) [0x7fcd9d650c3f]
  [bt] (2) /home/tjackson/.local/lib/python3.6/site-packages/tvm-0.8.dev0-py3.6-linux-x86_64.egg/tvm/libtvm.so(tvm::runtime::TypedPackedFunc<tvm::RelayExpr (tvm::relay::Call const&, tvm::runtime::Array<tvm::RelayExpr, void> const&, tvm::runtime::ObjectRef const&)>::AssignTypedLambda<tvm::RelayExpr (*)(tvm::relay::Call const&, tvm::runtime::Array<tvm::RelayExpr, void> const&, tvm::runtime::ObjectRef const&)>(tvm::RelayExpr (*)(tvm::relay::Call const&, tvm::runtime::Array<tvm::RelayExpr, void> const&, tvm::runtime::ObjectRef const&))::{lambda(tvm::runtime::TVMArgs const&, tvm::runtime::TVMRetValue*)#1}::operator()(tvm::runtime::TVMArgs const&, tvm::runtime::TVMRetValue*) const+0x24b) [0x7fcd9d5bf9bb]
  [bt] (1) /home/tjackson/.local/lib/python3.6/site-packages/tvm-0.8.dev0-py3.6-linux-x86_64.egg/tvm/libtvm.so(tvm::relay::quantize::Conv2dRealize(tvm::relay::Call const&, tvm::runtime::Array<tvm::RelayExpr, void> const&, tvm::runtime::ObjectRef const&)+0x829) [0x7fcd9d7163b9]
  [bt] (0) /home/sachacon/.local/lib/python3.6/site-packages/tvm-0.8.dev0-py3.6-linux-x86_64.egg/tvm/libtvm.so(+0x1641a12) [0x7fcd9d713a12]
  File "/home/tjackson/tvm/src/relay/quantize/realize.cc", line 215
TVMError: Check failed: rhs:
1 Like
#2

Were you able to resolve this issue? Please let me know, in case you have a workaround.

#3

The same question with you

#4

I had the same question, and it seems the test_quantization_accuracy.py hasn’t been updated for a long time. Maybe I can fix it carefully when I have time :slight_smile:

1 Like