Trying compile moblienet model where the input image width and height are 1280x720 and we are seeing compilation error in the compilation stage. In this model we have upscale layer where the scale factor are different and from the error message we notice that looks like VAI_C compiler failed, please have a look at the below error message and help me to resolve this issue.
Error log:
INFO: Float Graph Check Done.
INFO: Calibrating for 1 iterations…
100% (1 of 1) |#############################################################################################################################################################| Elapsed Time: 0:06:40 Time: 0:06:40
INFO: Calibration Done.
2021-03-05 08:00:26.444441: W tensorflow/contrib/decent_q/utils/quantize_utils.cc:848] [DECENT_WARNING] Cannot find quantize info file: /tmp/tmp3xm1w16k//temp/Pad_23_aquant. Use default quantize info.
2021-03-05 08:00:26.444482: W tensorflow/contrib/decent_q/utils/quantize_utils.cc:848] [DECENT_WARNING] Cannot find quantize info file: /tmp/tmp3xm1w16k//temp/Pad_21_aquant. Use default quantize info.
2021-03-05 08:00:26.444652: W tensorflow/contrib/decent_q/utils/quantize_utils.cc:848] [DECENT_WARNING] Cannot find quantize info file: /tmp/tmp3xm1w16k//temp/Pad_18_aquant. Use default quantize info.
2021-03-05 08:00:26.444675: W tensorflow/contrib/decent_q/utils/quantize_utils.cc:848] [DECENT_WARNING] Cannot find quantize info file: /tmp/tmp3xm1w16k//temp/Pad_16_aquant. Use default quantize info.
INFO: Generating Deploy Model…
INFO: Deploy Model Generated.
********************* Quantization Summary *********************
INFO: Output:
quantize_eval_model: /tmp/tmp3xm1w16k/quantize_eval_model.pb
deploy_model: /tmp/tmp3xm1w16k/deploy_model.pb
[VAI_C][Fatal] Check failed for condition [(ceilf(scale_h) == scale_h) && (ceilf(scale_w) == scale_w)] in [/home/xbuild/conda-bld/dnnc_1592904456005/work/dnnc_impl/core/layers/deephiresize.cc:56] :Make sure new height and new width of current layer [nn_upsampling_94444937822288] is multiple of height and width of it’s input, input shape is [ 1 23 40 64 ]
***** Check failure stack trace: *****
Traceback (most recent call last):
File “Detector_tvm_vitis_cross.py”, line 46, in
module.run()
File “/workspace/tvm/python/tvm/contrib/graph_runtime.py”, line 206, in run
self._run()
File “tvm/_ffi/_cython/./packed_func.pxi”, line 322, in tvm._ffi._cy3.core.PackedFuncBase.call
File “tvm/_ffi/_cython/./packed_func.pxi”, line 257, in tvm._ffi._cy3.core.FuncCall
File “tvm/_ffi/_cython/./packed_func.pxi”, line 246, in tvm._ffi._cy3.core.FuncCall3
File “tvm/_ffi/_cython/./base.pxi”, line 160, in tvm._ffi._cy3.core.CALL
tvm._ffi.base.TVMError: AssertionError: Can’t retrieve right out tensor names from DNNC compiler output
At:
/workspace/.local/lib/python3.6/site-packages/pyxir-0.1.3-py3.6-linux-x86_64.egg/pyxir/contrib/target/components/DPUCZDX8G/vai_c.py(149): compile
/workspace/.local/lib/python3.6/site-packages/pyxir-0.1.3-py3.6-linux-x86_64.egg/pyxir/contrib/target/components/DPUCZDX8G/zcu104.py(67): xgraph_dpu_zcu104_compiler
/workspace/.local/lib/python3.6/site-packages/pyxir-0.1.3-py3.6-linux-x86_64.egg/pyxir/base.py(159): compile
/workspace/.local/lib/python3.6/site-packages/pyxir-0.1.3-py3.6-linux-x86_64.egg/pyxir/base.py(211): compile_opaque_func
/workspace/.local/lib/python3.6/site-packages/pyxir-0.1.3-py3.6-linux-x86_64.egg/pyxir/opaque_func.py(113): opaque_func_wrapper
/workspace/tvm/python/tvm/contrib/graph_runtime.py(206): run
Is this DPU limitation or again its tensorflow issue?
Thanks and Regards, Raju