NNVM Error when image size is not 3x224x224

Questions
1

Hi, I am trying to run SSD detector on RK-3399 Firefly kit using NNVM with input image size being 300x300 instead of 224x224. It appears that this particular image size results in an intermediate feature map size which is not multiple of a certain factor thus resulting in an error during graph building for Mali GPU target. Any workaround for this will be helpful. Below is the error traceback. Thanks.

Traceback (most recent call last):
  File "nnvm_ssd_mobilent.py", line 175, in <module>
    approx_benchmark()
  File "nnvm_ssd_mobilent.py", line 125, in approx_benchmark
    dtype=dtype, target_host=target_host)
  File "/opt/nnvm/python/nnvm/compiler/build_module.py", line 290, in build
    graph = graph.apply("GraphFusePartition").apply("GraphFuseCompile")
  File "/opt/nnvm/python/nnvm/graph.py", line 234, in apply
    check_call(_LIB.NNGraphApplyPasses(self.handle, npass, cpass, ctypes.byref(ghandle)))
  File "/opt/nnvm/python/nnvm/_base.py", line 75, in check_call
    raise NNVMError(py_str(_LIB.NNGetLastError()))
nnvm._base.NNVMError: TVMCall CFunc Error:
Traceback (most recent call last):
  File "/opt/nnvm/tvm/python/tvm/_ffi/_ctypes/function.py", line 54, in cfun
    rv = local_pyfunc(*pyargs)
  File "/opt/nnvm/python/nnvm/top/nn.py", line 99, in compute_conv2d
    out = topi.nn.conv2d(inputs[0], kernel, strides, padding, layout)
  File "<decorator-gen-35>", line 2, in conv2d
  File "/opt/nnvm/tvm/python/tvm/target.py", line 345, in dispatch_func
    return dispatch_dict[k](*args, **kwargs)
  File "/opt/nnvm/tvm/topi/python/topi/mali/conv2d.py", line 128, in decl_conv2d
    return _decl_spatialpack(data, kernel, stride, padding, layout, out_dtype)
  File "/opt/nnvm/tvm/topi/python/topi/mali/conv2d.py", line 215, in _decl_spatialpack
    assert OW % VW == 0, "OW: %d  VW : %d" % (OW, VW)
AssertionError: OW: 150  VW : 4
2

Models are generally trained against a fixed size of input tensors for each layer.

We shouldn’t use different sizes like above.

You should resize the input and use, unless you redesign and train the entire model with different input size which generates new params (weights).

3

Even if it was already trained for 300x300, those assertions are about selecting an optimized scheduler. See this issue for possible mali schedule change!

4

@ehsanmok thanks. I will follow the issue you mentioned and see if I can finetune the schedule for this case.

I am trying ssd mobilenet detector which has input size of 300x300. No confusion here.