Hi all, I’m getting the following rhs error while using Relay to quantize MobileNetV2 from the MxNet Gluon model zoo. The code below is mainly from the test script below found in the tvm github: https://github.com/apache/incubator-tvm/blob/main/tests/python/nightly/quantization/test_quantization_accuracy.py
The error is coming from the Expr Conv2dRealize() function in realize.cc. Does anyone have any ideas as for a solution to this error? Or would it be better to simply try quantizing MobileNetV2 through PyTorch instead of MxNet?
The code and error is listed below. Thanks.
model_name="mobilenetv2_1.0"
target = "llvm"
ctx = tvm.cpu(0)
batch_size = 1
qconfig = qtz.qconfig(
skip_conv_layers=[0],
nbit_input=8,
nbit_weight=8,
global_scale=4.0,
dtype_input="int8",
dtype_weight="int8",
dtype_activation="int32",
debug_enabled_ops=None,
)
gluon_model = gluon.model_zoo.vision.get_model(model_name, pretrained=True)
data_shape = (batch_size, 3, img_size, img_size)
mod, params = relay.frontend.from_mxnet(gluon_model, {"data": data_shape})
with tvm.transform.PassContext(opt_level=3):
qfunc = relay.quantize.prerequisite_optimize(net, params=params)
logging.debug("original")
logging.debug(qfunc.astext(show_meta_data=False))
with qconfig:
logging.debug("current quantize config")
logging.debug(qtz.current_qconfig())
qfunc = qtz.quantize(qfunc)
logging.debug("after quantize")
logging.debug(qfunc.astext(show_meta_data=False))
with tvm.transform.PassContext(opt_level=3):
graph, lib, params = relay.build(qfunc, target)
m = tvm.contrib.graph_runtime.create(graph, lib, ctx)
m.set_input(**params)
# Testing Quantization For Now
TVMError below
Traceback (most recent call last):
File "/tvm/EE464H/quantize.py", line 43, in <module>
qfunc = qtz.quantize(qfunc)
File "/home/tjackson/.local/lib/python3.6/site-packages/tvm-0.8.dev0-py3.6-linux-x86_64.egg/tvm/relay/quantize/quantize.py", line 371, in quantize
mod = quantize_seq(mod)
File "/home/tjackson/.local/lib/python3.6/site-packages/tvm-0.8.dev0-py3.6-linux-x86_64.egg/tvm/ir/transform.py", line 127, in __call__
return _ffi_transform_api.RunPass(self, mod)
File "/home/tjackson/.local/lib/python3.6/site-packages/tvm-0.8.dev0-py3.6-linux-x86_64.egg/tvm/_ffi/_ctypes/packed_func.py", line 237, in __call__
raise get_last_ffi_error()
tvm._ffi.base.TVMError: Traceback (most recent call last):
[bt] (8) /home//.local/lib/python3.6/site-packages/tvm-0.8.dev0-py3.6-linux-x86_64.egg/tvm/libtvm.so(tvm::relay::MixedModeMutator::DispatchVisitExpr(tvm::RelayExpr const&)+0x1d) [0x7fcd9d7db59d]
[bt] (7) /home/tjackson/.local/lib/python3.6/site-packages/tvm-0.8.dev0-py3.6-linux-x86_64.egg/tvm/libtvm.so(tvm::relay::ExprMutator::VisitExpr(tvm::RelayExpr const&)+0x8e) [0x7fcd9d7db52e]
[bt] (6) /home/tjackson/.local/lib/python3.6/site-packages/tvm-0.8.dev0-py3.6-linux-x86_64.egg/tvm/libtvm.so(tvm::relay::ExprFunctor<tvm::RelayExpr (tvm::RelayExpr const&)>::VisitExpr(tvm::RelayExpr const&)+0x82) [0x7fcd9d7e08b2]
[bt] (5) /home/tjackson/.local/lib/python3.6/site-packages/tvm-0.8.dev0-py3.6-linux-x86_64.egg/tvm/libtvm.so(tvm::relay::ExprFunctor<tvm::RelayExpr (tvm::RelayExpr const&)>::InitVTable()::{lambda(tvm::runtime::ObjectRef const&, tvm::relay::ExprFunctor<tvm::RelayExpr (tvm::RelayExpr const&)>*)#6}::_FUN(tvm::runtime::ObjectRef const&, tvm::relay::ExprFunctor<tvm::RelayExpr (tvm::RelayExpr const&)>*)+0x27) [0x7fcd9d679f97]
[bt] (4) /home/tjackson/.local/lib/python3.6/site-packages/tvm-0.8.dev0-py3.6-linux-x86_64.egg/tvm/libtvm.so(tvm::relay::MixedModeMutator::VisitExpr_(tvm::relay::CallNode const*)+0x96) [0x7fcd9d5ff116]
[bt] (3) /home/tjackson/.local/lib/python3.6/site-packages/tvm-0.8.dev0-py3.6-linux-x86_64.egg/tvm/libtvm.so(tvm::relay::ForwardRewriter::Rewrite_(tvm::relay::CallNode const*, tvm::RelayExpr const&)+0x13af) [0x7fcd9d650c3f]
[bt] (2) /home/tjackson/.local/lib/python3.6/site-packages/tvm-0.8.dev0-py3.6-linux-x86_64.egg/tvm/libtvm.so(tvm::runtime::TypedPackedFunc<tvm::RelayExpr (tvm::relay::Call const&, tvm::runtime::Array<tvm::RelayExpr, void> const&, tvm::runtime::ObjectRef const&)>::AssignTypedLambda<tvm::RelayExpr (*)(tvm::relay::Call const&, tvm::runtime::Array<tvm::RelayExpr, void> const&, tvm::runtime::ObjectRef const&)>(tvm::RelayExpr (*)(tvm::relay::Call const&, tvm::runtime::Array<tvm::RelayExpr, void> const&, tvm::runtime::ObjectRef const&))::{lambda(tvm::runtime::TVMArgs const&, tvm::runtime::TVMRetValue*)#1}::operator()(tvm::runtime::TVMArgs const&, tvm::runtime::TVMRetValue*) const+0x24b) [0x7fcd9d5bf9bb]
[bt] (1) /home/tjackson/.local/lib/python3.6/site-packages/tvm-0.8.dev0-py3.6-linux-x86_64.egg/tvm/libtvm.so(tvm::relay::quantize::Conv2dRealize(tvm::relay::Call const&, tvm::runtime::Array<tvm::RelayExpr, void> const&, tvm::runtime::ObjectRef const&)+0x829) [0x7fcd9d7163b9]
[bt] (0) /home/sachacon/.local/lib/python3.6/site-packages/tvm-0.8.dev0-py3.6-linux-x86_64.egg/tvm/libtvm.so(+0x1641a12) [0x7fcd9d713a12]
File "/home/tjackson/tvm/src/relay/quantize/realize.cc", line 215
TVMError: Check failed: rhs: