Hello,
I am working on quantization right now and I got some errors when running quantize() pass for ResNet50 and ResNet101 but succeeded for ResNet18 and ResNet34. Could anyone share how they quantize for ResNet50?
Best
My code is shown below:
mod, params= relay.frontend.from_mxnet(model_json,{‘data’:(1,3,data_size,data_size)}, arg_params=arg_params, aux_params=aux_params)
func = mod[“main”]
quan_graph = relay.quantize.quantize(func,params=params)
What’s the error?
I’m having some issues too, I get TVMError: Check failed: lhs->dtype == dtype (int8 vs. int32)
I posted the same question before.
It’s becuase the current relay quantization has limited operation supports. 
Thanks for reporting the issue, this is indeed a quantization issue that needs looking into. I can investigate, but first @ziheng @vinx13 might be familiar with the bug.
Thanks @thierry for the confirmation. We are working on benchmarking quantization pass to compile across a broad range of openly available networks. We think it might be useful to share the findings so that we can make quantization more robust. @ziheng @vinx13 Please have a look.
Currently, our goal can be to ensure that we dont crash. We can treat performance as a separate future (hopefully near-future 
) project.
| Model Name | Status | quantized top1/top5 | original top1/top5 | quantized performance | original performance |
|---|---|---|---|---|---|
| resnet18_v1 | Pass | 0.672/0.87928 | 0.70946/0.8992 | 8.195 | 3.435 |
| resnet34_v1 | Pass | 0.70228/0.89646 | 0.74396/0.91872 | 15.95 | 6.496 |
| resnet50_v1 | Error 1 | ||||
| resnet101_v1 | Error 1 | ||||
| resnet152_v1 | Error 1 | ||||
| densenet121 | Error 1 | ||||
| mobilenet1.0 | Pass | 0.54944/0.7849 | 0.70764/0.8997 | 9.352 | 9.323 |
| vgg11 | Pass | 0.62602/0.84828 | 0.6811/0.88248 | 33.362 | 13.016 |
| ssd_512_mobilenet1.0_voc | Error 2 |
Error 1 - resnet50_v1:
Traceback (most recent call last):
[bt] (8) /home/ubuntu/.local/lib/python3.6/site-packages/tvm-0.6.dev0-py3.6-linux-x86_64.egg/tvm/libtvm.so(tvm::relay::ForwardRewriter::VisitExpr_(tvm::relay::CallNode const*)+0x2c7) [0x7f4b9783bf37]
[bt] (7) /home/ubuntu/.local/lib/python3.6/site-packages/tvm-0.6.dev0-py3.6-linux-x86_64.egg/tvm/libtvm.so(tvm::relay::ForwardRewriter::GetTempExpr(tvm::relay::Expr const&)+0x15d) [0x7f4b9783aead]
[bt] (6) /home/ubuntu/.local/lib/python3.6/site-packages/tvm-0.6.dev0-py3.6-linux-x86_64.egg/tvm/libtvm.so(tvm::relay::ExprMutator::VisitExpr(tvm::relay::Expr const&)+0x9e) [0x7f4b9766c60e]
[bt] (5) /home/ubuntu/.local/lib/python3.6/site-packages/tvm-0.6.dev0-py3.6-linux-x86_64.egg/tvm/libtvm.so(tvm::relay::ExprFunctor<tvm::relay::Expr (tvm::relay::Expr const&)>::VisitExpr(tvm::relay::Expr const&)+0xd2) [0x7f4b97673152]
[bt] (4) /home/ubuntu/.local/lib/python3.6/site-packages/tvm-0.6.dev0-py3.6-linux-x86_64.egg/tvm/libtvm.so(std::_Function_handler<tvm::relay::Expr (tvm::NodeRef const&, tvm::relay::ExprFunctor<tvm::relay::Expr (tvm::relay::Expr const&)>), tvm::relay::ExprFunctor<tvm::relay::Expr (tvm::relay::Expr const&)>::InitVTable()::{lambda(tvm::NodeRef const&, tvm::relay::ExprFunctor<tvm::relay::Expr (tvm::relay::Expr const&)>)#6}>::_M_invoke(std::Any_data const&, tvm::NodeRef const&, tvm::relay::ExprFunctor<tvm::relay::Expr (tvm::relay::Expr const&)>*&&)+0x34) [0x7f4b9766e994]
[bt] (3) /home/ubuntu/.local/lib/python3.6/site-packages/tvm-0.6.dev0-py3.6-linux-x86_64.egg/tvm/libtvm.so(tvm::relay::ForwardRewriter::VisitExpr(tvm::relay::CallNode const*)+0x5f4) [0x7f4b9783c264]
[bt] (2) /home/ubuntu/.local/lib/python3.6/site-packages/tvm-0.6.dev0-py3.6-linux-x86_64.egg/tvm/libtvm.so(std::_Function_handler<void (tvm::runtime::TVMArgs, tvm::runtime::TVMRetValue*), tvm::runtime::TypedPackedFunc<tvm::relay::Expr (tvm::relay::Call const&, tvm::Array<tvm::relay::Expr, void> const&, tvm::NodeRef const&)>::AssignTypedLambda<tvm::relay::Expr ()(tvm::relay::Call const&, tvm::Array<tvm::relay::Expr, void> const&, tvm::NodeRef const&)>(tvm::relay::Expr ()(tvm::relay::Call const&, tvm::Array<tvm::relay::Expr, void> const&, tvm::NodeRef const&))::{lambda(tvm::runtime::TVMArgs const&, tvm::runtime::TVMRetValue*)#1}>::_M_invoke(std::_Any_data const&, tvm::runtime::TVMArgs&&, tvm::runtime::TVMRetValue*&&)+0xb0) [0x7f4b978017b0]
[bt] (1) /home/ubuntu/.local/lib/python3.6/site-packages/tvm-0.6.dev0-py3.6-linux-x86_64.egg/tvm/libtvm.so(tvm::relay::quantize::MulRealize(tvm::relay::Call const&, tvm::Array<tvm::relay::Expr, void> const&, tvm::NodeRef const&)+0x292) [0x7f4b97890ef2]
[bt] (0) /home/ubuntu/.local/lib/python3.6/site-packages/tvm-0.6.dev0-py3.6-linux-x86_64.egg/tvm/libtvm.so(dmlc::LogMessageFatal::~LogMessageFatal()+0x43) [0x7f4b9723db83]
File “/home/ubuntu/zxy/tvm/src/relay/pass/quantize.cc”, line 344
TVMError: Check failed: lhs->dtype == dtype (int8 vs. int32) :
Error 2 - ssd_512_mobilenet1.0_voc:
Segmentation fault: 11
Stack trace returned 10 entries:
[bt] (0) /home/ubuntu/.local/lib/python3.6/site-packages/mxnet/libmxnet.so(+0x21d8d4) [0x7f6295e4e8d4]
[bt] (1) /home/ubuntu/.local/lib/python3.6/site-packages/mxnet/libmxnet.so(+0x32856e6) [0x7f6298eb66e6]
[bt] (2) /lib/x86_64-linux-gnu/libc.so.6(+0x3ef20) [0x7f62c6cb2f20]
[bt] (3) /home/ubuntu/.local/lib/python3.6/site-packages/tvm-0.6.dev0-py3.6-linux-x86_64.egg/tvm/libtvm.so(tvm::relay::GetValidCountRel(tvm::Array<tvm::relay::Type, void> const&, int, tvm::Attrs const&, tvm::relay::TypeReporter const&)+0x17d) [0x7f624918c58d]
[bt] (4) /home/ubuntu/.local/lib/python3.6/site-packages/tvm-0.6.dev0-py3.6-linux-x86_64.egg/tvm/libtvm.so(std::_Function_handler<void (tvm::runtime::TVMArgs, tvm::runtime::TVMRetValue*), tvm::runtime::TypedPackedFunc<bool (tvm::Array<tvm::relay::Type, void> const&, int, tvm::Attrs const&, tvm::relay::TypeReporter const&)>::AssignTypedLambda<bool ()(tvm::Array<tvm::relay::Type, void> const&, int, tvm::Attrs const&, tvm::relay::TypeReporter const&)>(bool ()(tvm::Array<tvm::relay::Type, void> const&, int, tvm::Attrs const&, tvm::relay::TypeReporter const&))::{lambda(tvm::runtime::TVMArgs const&, tvm::runtime::TVMRetValue*)#1}>::_M_invoke(std::_Any_data const&, tvm::runtime::TVMArgs&&, tvm::runtime::TVMRetValue*&&)+0xd6) [0x7f624906f6d6]
[bt] (5) /home/ubuntu/.local/lib/python3.6/site-packages/tvm-0.6.dev0-py3.6-linux-x86_64.egg/tvm/libtvm.so(tvm::relay::TypeSolver::Solve()+0x107c) [0x7f624927590c]
[bt] (6) /home/ubuntu/.local/lib/python3.6/site-packages/tvm-0.6.dev0-py3.6-linux-x86_64.egg/tvm/libtvm.so(tvm::relay::TypeInferencer::Infer(tvm::relay::Expr)+0x55) [0x7f624925ebc5]
[bt] (7) /home/ubuntu/.local/lib/python3.6/site-packages/tvm-0.6.dev0-py3.6-linux-x86_64.egg/tvm/libtvm.so(tvm::relay::InferType(tvm::relay::Function const&, tvm::relay::Module const&, tvm::relay::GlobalVar const&)+0x39a) [0x7f624925fa2a]
[bt] (8) /home/ubuntu/.local/lib/python3.6/site-packages/tvm-0.6.dev0-py3.6-linux-x86_64.egg/tvm/libtvm.so(tvm::relay::ModuleNode::Add(tvm::relay::GlobalVar const&, tvm::relay::Function const&, bool)+0x826) [0x7f624902e636]
[bt] (9) /home/ubuntu/.local/lib/python3.6/site-packages/tvm-0.6.dev0-py3.6-linux-x86_64
I think it’s a bug in Realize pass. Use store_lowbit_output=False in QConfig can fix it. There is another PR https://github.com/dmlc/tvm/pull/3543 that will refactor this.
Great @xyzhou thanks you for doing the pass, I think I’ll have cycles this week to address this bug; it would be great to have #3543 merged beforehand.