# Segmentation Fault in relay.quantize.quantize

**URL:** <https://discuss.tvm.apache.org/t/segmentation-fault-in-relay-quantize-quantize/4328>\
**Category:** Troubleshooting\
**Created:** [October 14, 2019, 7:10pm UTC](https://discuss.tvm.apache.org/t/segmentation-fault-in-relay-quantize-quantize/4328 "2019-10-14T19:10:19Z")\
**Posts on this page:** 12\
**Page:** 1

<div class="post-metadata">

**Author:** ![alopez\_13](https://discuss.tvm.apache.org/user_avatar/discuss.tvm.apache.org/alopez_13/32/786_2.png) [@alopez\_13](https://discuss.tvm.apache.org/u/alopez_13)\
**Post date:** [October 14, 2019, 7:10pm UTC](https://discuss.tvm.apache.org/t/segmentation-fault-in-relay-quantize-quantize/4328/1 "2019-10-14T19:10:19Z")

</div>

Hello,

Started playing with quantization. Was able to compile and run a MobileNet model but when I try the ssd\_512\_mobilenet1.0\_voc from the zoo I get a segmentation fault:

Segmentation fault: 11

Stack trace returned 10 entries:

[bt] (0) 0 libmxnet.so 0x000000012cfcbc90 std::\_\_1::\_\_tree\<std::\_\_1::\_\_value\_type\<std::\_\_1::basic\_string\<char, std::\_\_1::char\_traits\<char\>, std::\_\_1::allocator\<char\> \>, mxnet::NDArrayFunctionReg\*\>, std::\_\_1::\_\_map\_value\_compare\<std::\_\_1::basic\_string\<char, std::\_\_1::char\_traits\<char\>, std::\_\_1::allocator\<char\> \>, std::\_\_1::\_\_value\_type\<std::\_\_1::basic\_string\<char, std::\_\_1::char\_traits\<char\>, std::\_\_1::allocator\<char\> \>, mxnet::NDArrayFunctionReg\*\>, std::\_\_1::less\<std::\_\_1::basic\_string\<char, std::\_\_1::char\_traits\<char\>, std::\_\_1::allocator\<char\> \> \>, true\>, std::\_\_1::allocator\<std::\_\_1::\_\_value\_type\<std::\_\_1::basic\_string\<char, std::\_\_1::char\_traits\<char\>, std::\_\_1::allocator\<char\> \>, mxnet::NDArrayFunctionReg\*\> \> \>::destroy(std::\_\_1::\_\_tree\_node\<std::\_\_1::\_\_value\_type\<std::\_\_1::basic\_string\<char, std::\_\_1::char\_traits\<char\>, std::\_\_1::allocator\<char\> \>, mxnet::NDArrayFunctionReg\*\>, void\*\>\*) + 2736

[bt] (1) 1 libmxnet.so 0x000000012ec611b6 mxnet::Storage::Get() + 4358

[bt] (2) 2 libsystem\_platform.dylib 0x00007fff7a53ab5d \_sigtramp + 29

[bt] (3) 3 ??? 0x0000000000000028 0x0 + 40

[bt] (4) 4 libtvm.dylib 0x00000001192c659f void tvm::runtime::detail::unpack\_call\_dispatcher\<bool, 0, 4, bool (_)(tvm::Array\<tvm::relay::Type, void\> const&, int, tvm::Attrs const&, tvm::relay::TypeReporter const&)\>::run\<tvm::runtime::TVMArgValue, tvm::runtime::TVMArgValue, tvm::runtime::TVMArgValue, tvm::runtime::TVMArgValue\>(bool (_ const&)(tvm::Array\<tvm::relay::Type, void\> const&, int, tvm::Attrs const&, tvm::relay::TypeReporter const&), tvm::runtime::TVMArgs const&, tvm::runtime::TVMRetValue\*, tvm::runtime::TVMArgValue&&, tvm::runtime::TVMArgValue&&, tvm::runtime::TVMArgValue&&, tvm::runtime::TVMArgValue&&) + 95

[bt] (5) 5 libtvm.dylib 0x00000001192c64f9 std::\_\_1::\_\_function::\_\_func\<void tvm::runtime::TypedPackedFunc\<bool (tvm::Array\<tvm::relay::Type, void\> const&, int, tvm::Attrs const&, tvm::relay::TypeReporter const&)\>::AssignTypedLambda\<bool (_)(tvm::Array\<tvm::relay::Type, void\> const&, int, tvm::Attrs const&, tvm::relay::TypeReporter const&)\>(bool (_)(tvm::Array\<tvm::relay::Type, void\> const&, int, tvm::Attrs const&, tvm::relay::TypeReporter const&))::‘lambda’(tvm::runtime::TVMArgs const&, tvm::runtime::TVMRetValue\*), std::\_\_1::allocator\<void tvm::runtime::TypedPackedFunc\<bool (tvm::Array\<tvm::relay::Type, void\> const&, int, tvm::Attrs const&, tvm::relay::TypeReporter const&)\>::AssignTypedLambda\<bool (_)(tvm::Array\<tvm::relay::Type, void\> const&, int, tvm::Attrs const&, tvm::relay::TypeReporter const&)\>(bool (_)(tvm::Array\<tvm::relay::Type, void\> const&, int, tvm::Attrs const&, tvm::relay::TypeReporter const&))::‘lambda’(tvm::runtime::TVMArgs const&, tvm::runtime::TVMRetValue\*)\>, void (tvm::runtime::TVMArgs, tvm::runtime::TVMRetValue\*)\>::operator()(tvm::runtime::TVMArgs&&, tvm::runtime::TVMRetValue\*&&) + 137

[bt] (6) 6 libtvm.dylib 0x0000000119602e78 tvm::TypedEnvFunc\<bool (tvm::Array\<tvm::relay::Type, void\> const&, int, tvm::Attrs const&, tvm::relay::TypeReporter const&)\>::operator()(tvm::Array\<tvm::relay::Type, void\> const&, int, tvm::Attrs const&, tvm::relay::TypeReporter const&) const + 328

[bt] (7) 7 libtvm.dylib 0x00000001196027f3 tvm::relay::TypeSolver::Solve() + 1155

[bt] (8) 8 libtvm.dylib 0x00000001195e2bab tvm::relay::TypeInferencer::Infer(tvm::relay::Expr) + 107

[bt] (9) 9 libtvm.dylib 0x00000001195e3a5a tvm::relay::InferType(tvm::relay::Function const&, tvm::relay::Module const&, tvm::relay::GlobalVar const&) + 570

Here is the relevant code:

import tvm  
from tvm import relay  
import tvm.relay.testing  
from gluoncv import model\_zoo

supported\_model = [  
“mobilenetv2\_1.0”,  
“ssd\_512\_resnet50\_v1\_voc”,  
“ssd\_512\_resnet50\_v1\_coco”,  
“ssd\_512\_resnet101\_v2\_voc”,  
“ssd\_512\_mobilenet1.0\_voc”,  
“ssd\_512\_mobilenet1.0\_coco”,  
“ssd\_300\_vgg16\_atrous\_voc”,  
“ssd\_512\_vgg16\_atrous\_coco”,  
“ssd\_512\_mobilenet1.0\_voc\_int8”  
]

model\_name = supported\_model[4]

# 

# Model parameters

# 

input\_name = “input”  
input\_dtype = “float32”  
out\_dtype = “int8”  
out\_numbits = 8  
dshape = (1, 3, 512, 512)

# 

# Declare the CPU type

# 

myTarget = “llvm -mcpu=haswell”  
ctx = tvm.cpu()

# 

# Read the model

# 

block = model\_zoo.get\_model(model\_name, pretrained=True)

# 

# Ingest model into compiler then build.

# 

def build(target):  
print("\nBuilding quantized code\n")  
mod, params = relay.frontend.from\_mxnet(block,  
{input\_name: dshape},  
dtype=input\_dtype)

```
with relay.quantize.qconfig(nbit_input=out_numbits,
                            nbit_weight=out_numbits,
                            nbit_activation=out_numbits,
                            dtype_input=out_dtype,
                            dtype_weight=out_dtype,
                            dtype_activation=out_dtype,
                            global_scale=8.0,
                            skip_conv_layers=[0],
                            round_for_shift=True,
                            store_lowbit_output=False,
                            debug_enabled_ops=None):
    mod["main"] = relay.quantize.quantize(mod["main"], params)

```

…

Any help is greatly appreciated.

Thanks!

---

<div class="post-metadata">

**Author:** ![alopez\_13](https://discuss.tvm.apache.org/user_avatar/discuss.tvm.apache.org/alopez_13/32/786_2.png) [@alopez\_13](https://discuss.tvm.apache.org/u/alopez_13)\
**Post date:** [October 15, 2019, 7:34pm UTC](https://discuss.tvm.apache.org/t/segmentation-fault-in-relay-quantize-quantize/4328/2 "2019-10-15T19:34:19Z")

</div>

Also, I have tried changing the type and number of bits of the activation layer to float32 and 32. Still get similar segmentation fault results.

---

<div class="post-metadata">

**Author:** ![gasgallo](https://discuss.tvm.apache.org/user_avatar/discuss.tvm.apache.org/gasgallo/32/821_2.png) [@gasgallo](https://discuss.tvm.apache.org/u/gasgallo)\
**Post date:** [November 20, 2019, 6:55am UTC](https://discuss.tvm.apache.org/t/segmentation-fault-in-relay-quantize-quantize/4328/3 "2019-11-20T06:55:36Z")

</div>

I’m facing a similar issue but in my case it’s model related, meaning that I’m able to quantize some models but I get `segmentation fault` with other (without any error trace).

Is it because some ops are still not supported for quantization?

---

<div class="post-metadata">

**Author:** ![alopez\_13](https://discuss.tvm.apache.org/user_avatar/discuss.tvm.apache.org/alopez_13/32/786_2.png) [@alopez\_13](https://discuss.tvm.apache.org/u/alopez_13)\
**Post date:** [November 21, 2019, 4:13pm UTC](https://discuss.tvm.apache.org/t/segmentation-fault-in-relay-quantize-quantize/4328/4 "2019-11-21T16:13:27Z")

</div>

It could be, the full explanation can be found here:

> **[Quantization Story](https://discuss.tvm.apache.org/t/quantization-story/3920)**
>
> I think it is worthwhile to have a high-level quantization post explaining the flow and mentioning developers who are involved in different steps. This should improve collaboration, while also putting a high-level story to anybody who wants to...

I haven’t tried going from TF-\>TVM recently as I am going through quantized model in TFLite-\>TVM route, it works but the performance on inference is worse than FP32 (original TF model) I think that the autotvm optimization is still being worked on, but once its ready I should expect better performance from INT-only models.

---

<div class="post-metadata">

**Author:** ![janimesh](https://discuss.tvm.apache.org/letter_avatar_proxy/v4/letter/j/b5e925/32.png) [@janimesh](https://discuss.tvm.apache.org/u/janimesh)\
**Post date:** [November 21, 2019, 6:00pm UTC](https://discuss.tvm.apache.org/t/segmentation-fault-in-relay-quantize-quantize/4328/5 "2019-11-21T18:00:12Z")

</div>

@alopez_13 Which TFLite model do you use and which Intel server machine? For Cascade lake, you should performance improvement. Let me know if you can share the details, I might be able to help.

---

<div class="post-metadata">

**Author:** ![alopez\_13](https://discuss.tvm.apache.org/user_avatar/discuss.tvm.apache.org/alopez_13/32/786_2.png) [@alopez\_13](https://discuss.tvm.apache.org/u/alopez_13)\
**Post date:** [November 22, 2019, 3:29pm UTC](https://discuss.tvm.apache.org/t/segmentation-fault-in-relay-quantize-quantize/4328/6 "2019-11-22T15:29:52Z")

</div>

I was working with a quantized MobileNet\_v2 from tensorflow zoo. The problem I had was that during graph optimization (auto tuning) I was getting an error about "_only support NCHW/HWCN currently"_ when calling:

```
tasks = autotvm.task.extract_from_program(sym["main"],
                                              target=target,
                                              params=params,
                                              ops=(relay.op.nn.conv2d,))

```

I thought it was because the convolution operator did not support quantization, but now that I think of it, I may need to call a specific relay operator? I’ll dig around in the forums…

---

<div class="post-metadata">

**Author:** ![alopez\_13](https://discuss.tvm.apache.org/user_avatar/discuss.tvm.apache.org/alopez_13/32/786_2.png) [@alopez\_13](https://discuss.tvm.apache.org/u/alopez_13)\
**Post date:** [November 22, 2019, 3:43pm UTC](https://discuss.tvm.apache.org/t/segmentation-fault-in-relay-quantize-quantize/4328/7 "2019-11-22T15:43:11Z")

</div>

By the way also looked at:

> **[Quantization int8 slower than int16 on skylake CPU](https://discuss.tvm.apache.org/t/quantization-int8-slower-than-int16-on-skylake-cpu/4390)**
>
> I am the first time user of TVM and I am doing some inference benchmarks of quantized models on CPU. Model is imported from mxnet, quantized and auto tuned. I am running the test on Google cloud, skyllake dual-core cpu. The int8 is always slower...

and, if I understood correctly, his proposed solution was to change _func\_create = “topi\_x86\_conv2d\_NCHWc”_ to _func\_create = “topi\_x86\_conv2d\_NCHWc\_int8”_ in the tuning function. Unfortunately, I still get the same error. I’ll keep digging, most likely I am not setting the tuning parameters right.

---

<div class="post-metadata">

**Author:** ![alopez\_13](https://discuss.tvm.apache.org/user_avatar/discuss.tvm.apache.org/alopez_13/32/786_2.png) [@alopez\_13](https://discuss.tvm.apache.org/u/alopez_13)\
**Post date:** [November 22, 2019, 4:55pm UTC](https://discuss.tvm.apache.org/t/segmentation-fault-in-relay-quantize-quantize/4328/8 "2019-11-22T16:55:36Z")

</div>

An update. Changing the tuner function will not help since the error is taking place on _autotvm.task.extract\_from\_program_ thus I think it has to do with the data layout that TFLite produces compared to what is expected in TVM. Now I remember reading about this in the forums.

---

<div class="post-metadata">

**Author:** ![janimesh](https://discuss.tvm.apache.org/letter_avatar_proxy/v4/letter/j/b5e925/32.png) [@janimesh](https://discuss.tvm.apache.org/u/janimesh)\
**Post date:** [November 22, 2019, 5:42pm UTC](https://discuss.tvm.apache.org/t/segmentation-fault-in-relay-quantize-quantize/4328/9 "2019-11-22T17:42:48Z")

</div>

I think you are right about the error. This might be also because of depthwise convolution in MobileNetV2. To further narrow down, there are 2 things that can be done

- Run w/o auto-tuning and see if the model passes.
- Try model w/o depthwise convolution.

---

<div class="post-metadata">

**Author:** ![alopez\_13](https://discuss.tvm.apache.org/user_avatar/discuss.tvm.apache.org/alopez_13/32/786_2.png) [@alopez\_13](https://discuss.tvm.apache.org/u/alopez_13)\
**Post date:** [November 22, 2019, 6:02pm UTC](https://discuss.tvm.apache.org/t/segmentation-fault-in-relay-quantize-quantize/4328/10 "2019-11-22T18:02:13Z")

</div>

Thanks. I know its auto tuning because I can do inference on that quantized model without it. I have not ran an exhaustive sweep to check for accuracy, but it does give me the same classes compared to the original TensorFlow and TFLite (without the compiler) models. Let me explore w/o depthwise convolution.

---

<div class="post-metadata">

**Author:** ![janimesh](https://discuss.tvm.apache.org/letter_avatar_proxy/v4/letter/j/b5e925/32.png) [@janimesh](https://discuss.tvm.apache.org/u/janimesh)\
**Post date:** [November 22, 2019, 6:02pm UTC](https://discuss.tvm.apache.org/t/segmentation-fault-in-relay-quantize-quantize/4328/11 "2019-11-22T18:02:49Z")

</div>

Which machine are you using?

---

<div class="post-metadata">

**Author:** ![alopez\_13](https://discuss.tvm.apache.org/user_avatar/discuss.tvm.apache.org/alopez_13/32/786_2.png) [@alopez\_13](https://discuss.tvm.apache.org/u/alopez_13)\
**Post date:** [November 22, 2019, 6:06pm UTC](https://discuss.tvm.apache.org/t/segmentation-fault-in-relay-quantize-quantize/4328/12 "2019-11-22T18:06:56Z")

</div>

I have ran in two x86 boxes, an old macOS (haswell) and a newer linux-based server (skylake). But I think the error is related to the data layout. TFLite does not use NCHW by default and that is why when you call _autotvm.task.extract\_from\_program_ it complains. The error happens before you actually do the tuning.