I trained my model with tensorflow and saved as pb file, when I use tensorflow frontend, I got errors. The main error info: Operator concatenate(axis=3, name=concatenate_1/concat) expects data1’s shape to be [1,4294967295,4294967295,0], but got [1,10,10,512]. data0 was from _resize_bilinear function, data1 was from some conv op, it seems _resize_bilinear infered shape was wrong, the infered shape shoud be [1,10,10,256].
Cross ref : https://github.com/dmlc/tvm/issues/2429
Can you share the arguments to nnvm.compiler.build( ?
Hope you called below API (or alternatively use add_shapes=True while exporting protobuf from tensorflow).
# Add shapes to the graph.
with tf.Session() as sess:
graph_def = nnvm.testing.tf.AddShapesToGraphDef(sess, 'softmax')
def _ProcessGraphDefParam(graph_def):
"""Type-checks and possibly canonicalizes `graph_def`."""
if not isinstance(graph_def, graph_pb2.GraphDef):
# `graph_def` could be a dynamically-created message, so try a duck-typed
# approach
try:
old_graph_def = graph_def
graph_def = graph_pb2.GraphDef()
graph_def.MergeFrom(old_graph_def)
except TypeError:
raise TypeError('graph_def must be a GraphDef proto.')
return graph_def
with tf.gfile.FastGFile(self.model_path, 'rb') as f:
graph_def = tf.GraphDef()
graph_def.ParseFromString(f.read())
graph = tf.import_graph_def(graph_def, name='')
# Call the utility to import the graph definition into default graph.
graph_def = _ProcessGraphDefParam(graph_def)
# Add shapes to the graph.
with tf.Session() as sess:
output_graph_def = tf.graph_util.convert_variables_to_constants(
sess,
sess.graph.as_graph_def(add_shapes=True),
self.output_node)
sym, params = nnvm.frontend.from_tensorflow(output_graph_def, layout=self.layout, outputs=self.output_node)
params = {k: tvm.nd.array(v.asnumpy().astype(self.dtype)) for k, v in params.items()}
shape_dict = {self.input_node: (1, *self.model_image_size, 3)}
dtype_dict = {self.input_node: self.dtype}
with nnvm.compiler.build_config(opt_level=3):
graph, lib, params = nnvm.compiler.build(sym, shape=shape_dict, target=self.target, target_host=self.target_host, dtype=dtype_dict, params=params)
Here is my code. I check the definition of nnvm.testing.tf.AddShapesToGraphDef and replace it with my own code because my model has multiple output nodes, here self.output_node is a list containing some output nodes name. Now I solved the infer shape problem by set attr[‘size’]'s value to (10,10) in _resize_bilinear , but when I change input size and rerun my code, I must modify attr[‘size’] again to match data1’s shape.
That’s a limitation with resize_bilinear now.
Current implementation takes resize shape from frozen graph information and set size attribute.
We may need to handle this situation specially.
Most of ops, like conv2d, can infer output shape correctly with given input size, so may I request for the implementation of shape inferring in resize_bilinear op? It may help a lot for us to deploy more tensorflow models in tvm.
@hxcai I took a second look on this issue and I see irrespective of input shape the resize_bilinear output size attr should be same.
Can you print attr, node.name, node.op in below location and check _output_shape attribute for nodes ?
If there is a situation for tensorflow add_shapes doesn’t give appropriate information we will try to work around this scenarios.
def _resize_bilinear():
def _impl(inputs, attr, params):
attr['size'] = attr['_output_shapes'][0][1:3]
inputs.pop(1)
# NHWC
attr['layout'] = 'NHWC'
return AttrCvt(op_name="resize",
ignores=['Tdim'],
extras={'method': "NEAREST_NEIGHBOR"})(inputs, attr)
return _impl
_convert_map = {
.........
'ResizeNearestNeighbor' : _resize_bilinear()
Above is what I have modified in source code to support tensorflow ResizeNearestNeighbor op, and I print related op info below.
node name: input_1 node op: Placeholder attr: {'shape': dim {
size: -1
}
dim {
size: -1
}
dim {
size: -1
}
dim {
size: 3
}
, 'dtype': tf.float32, '_output_shapes': [dim {
size: -1
}
dim {
size: -1
}
dim {
size: -1
}
dim {
size: 3
}
]}
node name: up_sampling2d_1/ResizeNearestNeighbor node op: ResizeNearestNeighbor attr: {'_output_shapes': [dim {
size: -1
}
dim {
size: -1
}
dim {
size: -1
}
dim {
size: 256
}
], 'T': tf.float32, 'align_corners': False}
node name: up_sampling2d_2/ResizeNearestNeighbor node op: ResizeNearestNeighbor attr: {'_output_shapes': [dim {
size: -1
}
dim {
size: -1
}
dim {
size: -1
}
dim {
size: 128
}
], 'T': tf.float32, 'align_corners': False
Here NHW = -1 because input size has not been set, when code goes to nnvm.compiler.build, I set the input shape and dtype.
By the way, I check my tensorflow model using tensorboard, and found that even if I set input placeholder shape and resize op get a determined input shape, it will give output size (?,?,?,C), so does this matter?
Please try out this patch where you can specify the out shapes explicitly as given below
diff --git a/nnvm/python/nnvm/frontend/tensorflow.py b/nnvm/python/nnvm/frontend/tensorflow.py
index a869abac..4891ffbf 100644
--- a/nnvm/python/nnvm/frontend/tensorflow.py
+++ b/nnvm/python/nnvm/frontend/tensorflow.py
@@ -1177,6 +1177,9 @@ class GraphProto(object):
self._output_shapes[node.name] = \
[tensor_util.TensorShapeProtoToList( \
tensor_value.tensor_shape)]
+ elif shape and node.name in shape:
+ # Give priority to user argument.
+ self._output_shapes[node.name] = [shape[node.name]]
elif '_output_shapes' in attr:
self._output_shapes[node.name] = \
[tensor_util.TensorShapeProtoToList(tshape) \
shape_dict = {'DecodeJpeg/contents': x.shape, 'ResizeBilinear': (1, 300, 300, 1)}
sym, params = nnvm.frontend.from_tensorflow(graph_def, layout=layout, shape=shape_dict)
Thanks a lot!!! It worked for me.
Great, I will PR it.