Hi
I am trying to deploy a few onnx models using TVM. For that I need to import the onnx model and build it using relay and export the it as a library. I have successfully been able to do so for the models from the image classification model set from the Onnx Model Zoo.
I get errors to most of the onnx models except the image_classification models from ONNX Model Zoo.
Some of the error messages:
when I try building BiDaf
mod, params = relay.frontend.from_onnx(onnx_model, shape_dict)
File "/home/nitesh/nitesh_nfs_share/anaconda3/envs/tvm-build/lib/python3.8/site-packages/tvm/relay/frontend/onnx.py", line 4760, in from_onnx
mod, params = g.from_onnx(graph, opset)
File "/home/nitesh/nitesh_nfs_share/anaconda3/envs/tvm-build/lib/python3.8/site-packages/tvm/relay/frontend/onnx.py", line 4483, in from_onnx
self._nodes[i_name] = new_var(i_name, shape=i_shape, dtype=dtype)
File "/home/nitesh/nitesh_nfs_share/anaconda3/envs/tvm-build/lib/python3.8/site-packages/tvm/relay/frontend/common.py", line 610, in new_var
return _expr.var(name_hint, type_annotation, shape, dtype)
File "/home/nitesh/nitesh_nfs_share/anaconda3/envs/tvm-build/lib/python3.8/site-packages/tvm/relay/expr.py", line 476, in var
type_annotation = _ty.TensorType(shape, dtype)
File "/home/nitesh/nitesh_nfs_share/anaconda3/envs/tvm-build/lib/python3.8/site-packages/tvm/ir/tensor_type.py", line 41, in __init__
self.__init_handle_by_constructor__(_ffi_api.TensorType, shape, dtype)
File "tvm/_ffi/_cython/./object.pxi", line 126, in tvm._ffi._cy3.core.ObjectBase.__init_handle_by_constructor__
File "tvm/_ffi/_cython/./packed_func.pxi", line 279, in tvm._ffi._cy3.core.ConstructorCall
File "tvm/_ffi/_cython/./packed_func.pxi", line 257, in tvm._ffi._cy3.core.FuncCall
File "tvm/_ffi/_cython/./packed_func.pxi", line 246, in tvm._ffi._cy3.core.FuncCall3
File "tvm/_ffi/_cython/./base.pxi", line 163, in tvm._ffi._cy3.core.CALL
tvm._ffi.base.TVMError: Traceback (most recent call last):
1: TVMFuncCall
0: std::_Function_handler<void (tvm::runtime::TVMArgs, tvm::runtime::TVMRetValue*), tvm::runtime::TypedPackedFunc<tvm::TensorType (tvm::runtime::Array<tvm::PrimExpr, void>, tvm::runtime::DataType)>::AssignTypedLambda<tvm::{lambda(tvm::runtime::Array<tvm::PrimExpr, void>, tvm::runtime::DataType)#2}>(tvm::{lambda(tvm::runtime::Array<tvm::PrimExpr, void>, tvm::runtime::DataType)#2}, std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> >)::{lambda(tvm::runtime::TVMArgs const&, tvm::runtime::TVMRetValue*)#1}>::_M_invoke(std::_Any_data const&, tvm::runtime::TVMArgs&&, tvm::runtime::TVMRetValue*&&) [clone .cold]
3: TVMFuncCall
2: std::_Function_handler<void (tvm::runtime::TVMArgs, tvm::runtime::TVMRetValue*), tvm::runtime::TypedPackedFunc<tvm::TensorType (tvm::runtime::Array<tvm::PrimExpr, void>, tvm::runtime::DataType)>::AssignTypedLambda<tvm::{lambda(tvm::runtime::Array<tvm::PrimExpr, void>, tvm::runtime::DataType)#2}>(tvm::{lambda(tvm::runtime::Array<tvm::PrimExpr, void>, tvm::runtime::DataType)#2}, std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> >)::{lambda(tvm::runtime::TVMArgs const&, tvm::runtime::TVMRetValue*)#1}>::_M_invoke(std::_Any_data const&, tvm::runtime::TVMArgs&&, tvm::runtime::TVMRetValue*&&)
1: tvm::runtime::TVMArgValue::operator DLDataType() const
0: tvm::runtime::String2DLDataType(std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> >)
File "/home/nitesh/nitesh_nfs_share/anaconda3/conda-bld/tvm-package_1637002504757/work/include/tvm/runtime/packed_func.h", line 714
TVMError: In function ir.TensorType: error while converting argument 1: [15:15:47] /home/nitesh/nitesh_nfs_share/anaconda3/conda-bld/tvm-package_1637002504757/work/include/tvm/runtime/data_type.h:374: unknown type object
This is my shape dict and related variables for BiDaf
context_string = 'A quick brown fox jumps over the lazy dog.'
query_string = 'What color is the fox?'
shape_dict = {'context_word': [len(context_string), 1],
'context_char': [len(context_string), 1, 1, 16],
'query_word': [len(query_string),1],
'query_char': [len(query_string),1, 1, 16]}
Here is the output of graph.input:
graph input [name: "context_word"
type {
tensor_type {
elem_type: 8
shape {
dim {
dim_param: "c"
}
dim {
dim_value: 1
}
}
}
}
, name: "context_char"
type {
tensor_type {
elem_type: 8
shape {
dim {
dim_param: "c"
}
dim {
dim_value: 1
}
dim {
dim_value: 1
}
dim {
dim_value: 16
}
}
}
}
, name: "query_word"
type {
tensor_type {
elem_type: 8
shape {
dim {
dim_param: "q"
}
dim {
dim_value: 1
}
}
}
}
, name: "query_char"
type {
tensor_type {
elem_type: 8
shape {
dim {
dim_param: "q"
}
dim {
dim_value: 1
}
dim {
dim_value: 1
}
dim {
dim_value: 16
}
}
}
}
As you can see the element type is 8 that is enum as 8 in this dictionary that assigns the Dtype in onnx.py:
TENSOR_TYPE_TO_NP_TYPE = {
13 int(TensorProto.FLOAT): np.dtype('float32'),
14 int(TensorProto.UINT8): np.dtype('uint8'),
15 int(TensorProto.INT8): np.dtype('int8'),
16 int(TensorProto.UINT16): np.dtype('uint16'),
17 int(TensorProto.INT16): np.dtype('int16'),
18 int(TensorProto.INT32): np.dtype('int32'),
19 int(TensorProto.INT64): np.dtype('int64'),
20 int(TensorProto.BOOL): np.dtype('bool'),
21 int(TensorProto.FLOAT16): np.dtype('float16'),
22 int(TensorProto.DOUBLE): np.dtype('float64'),
23 int(TensorProto.COMPLEX64): np.dtype('complex64'),
24 int(TensorProto.COMPLEX128): np.dtype('complex128'),
25 int(TensorProto.UINT32): np.dtype('uint32'),
26 int(TensorProto.UINT64): np.dtype('uint64'),
27 int(TensorProto.STRING): np.dtype(np.object)
28 }
The dtype seems to string (8) (form here: Enum mappings) and will therefore have to of the type np.object. So there is no meaning for why I get the ’ unknown type object’ error here.
For MaskRCNN:
Incompatible broadcast type TensorType([1, 256, 31, 31], float32) and TensorType([1, 256, 32, 32], float32)
The type inference pass was unable to infer a type for this expression.
This usually occurs when an operator call is under constrained in some way, check other reported errors for hints of what may of happened.
Traceback (most recent call last):
File "prepare_test_libs.py", line 124, in <module>
prepare_model_lib(curr_path, sys.argv[1].upper()=='ARM',
File "prepare_test_libs.py", line 83, in prepare_model_lib
mod, params = relay.frontend.from_onnx(onnx_model, shape_dict)
File "/home/nitesh/nitesh_nfs_share/anaconda3/envs/tvm-build/lib/python3.8/site-packages/tvm/relay/frontend/onnx.py", line 4772, in from_onnx
mod, params = g.from_onnx(graph, opset)
File "/home/nitesh/nitesh_nfs_share/anaconda3/envs/tvm-build/lib/python3.8/site-packages/tvm/relay/frontend/onnx.py", line 4532, in from_onnx
op = self._convert_operator(op_name, inputs, attr, opset)
File "/home/nitesh/nitesh_nfs_share/anaconda3/envs/tvm-build/lib/python3.8/site-packages/tvm/relay/frontend/onnx.py", line 4661, in _convert_operator
sym = convert_map[op_name](inputs, attrs, self._params)
File "/home/nitesh/nitesh_nfs_share/anaconda3/envs/tvm-build/lib/python3.8/site-packages/tvm/relay/frontend/onnx.py", line 425, in _impl_v1
input_shape = infer_shape(data)
File "/home/nitesh/nitesh_nfs_share/anaconda3/envs/tvm-build/lib/python3.8/site-packages/tvm/relay/frontend/common.py", line 513, in infer_shape
out_type = infer_type(inputs, mod=mod)
File "/home/nitesh/nitesh_nfs_share/anaconda3/envs/tvm-build/lib/python3.8/site-packages/tvm/relay/frontend/common.py", line 488, in infer_type
new_mod = _transform.InferType()(new_mod)
File "/home/nitesh/nitesh_nfs_share/anaconda3/envs/tvm-build/lib/python3.8/site-packages/tvm/ir/transform.py", line 161, in __call__
return _ffi_transform_api.RunPass(self, mod)
File "tvm/_ffi/_cython/./packed_func.pxi", line 323, in tvm._ffi._cy3.core.PackedFuncBase.__call__
File "tvm/_ffi/_cython/./packed_func.pxi", line 257, in tvm._ffi._cy3.core.FuncCall
File "tvm/_ffi/_cython/./packed_func.pxi", line 246, in tvm._ffi._cy3.core.FuncCall3
File "tvm/_ffi/_cython/./base.pxi", line 163, in tvm._ffi._cy3.core.CALL
tvm.error.DiagnosticError: Traceback (most recent call last):
6: TVMFuncCall
5: std::_Function_handler<void (tvm::runtime::TVMArgs, tvm::runtime::TVMRetValue*), tvm::runtime::TypedPackedFunc<tvm::IRModule (tvm::transform::Pass, tvm::IRModule)>::AssignTypedLambda<tvm::transform::{lambda(tvm::transform::Pass, tvm::IRModule)#7}>(tvm::transform::{lambda(tvm::transform::Pass, tvm::IRModule)#7}, std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> >)::{lambda(tvm::runtime::TVMArgs const&, tvm::runtime::TVMRetValue*)#1}>::_M_invoke(std::_Any_data const&, tvm::runtime::TVMArgs&&, tvm::runtime::TVMRetValue*&&)
4: tvm::transform::Pass::operator()(tvm::IRModule) const
3: tvm::transform::Pass::operator()(tvm::IRModule, tvm::transform::PassContext const&) const
2: tvm::transform::ModulePassNode::operator()(tvm::IRModule, tvm::transform::PassContext const&) const
1: std::_Function_handler<void (tvm::runtime::TVMArgs, tvm::runtime::TVMRetValue*), tvm::runtime::TypedPackedFunc<tvm::IRModule (tvm::IRModule, tvm::transform::PassContext)>::AssignTypedLambda<tvm::relay::transform::InferType()::{lambda(tvm::IRModule, tvm::transform::PassContext const&)#1}>(tvm::relay::transform::InferType()::{lambda(tvm::IRModule, tvm::transform::PassContext const&)#1})::{lambda(tvm::runtime::TVMArgs const&, tvm::runtime::TVMRetValue*)#1}>::_M_invoke(std::_Any_data const&, tvm::runtime::TVMArgs&&, tvm::runtime::TVMRetValue*&&)
0: tvm::DiagnosticContext::Render()
File "/home/nitesh/nitesh_nfs_share/anaconda3/conda-bld/tvm-package_1637002504757/work/src/ir/diagnostic.cc", line 105
DiagnosticError: one or more error diagnostics were emitted, please check diagnostic render for output.
This is just for 3 of the error-prone onnx models from ONNX Model Zoo.
Any help will be appreciated!
Thanks