Autotvm - A fallback configuration is used, which may bring great performance regression

Rms45 · March 13, 2020, 10:06am

The online demo uses a pretrained torch model. I modified the target and ran the demo. The demo seems to download a torch model and convert to onnx and then to tvm module using the specified target. Hence i expect that model should already be tuned to be run on TX2. Or am i wrong somewhere?

Apart from this, for target = ‘cuda’, there seems to only one warning of this kind ,

WARNING:autotvm:Cannot find config for target=cuda, workload=(‘dense’, (1, 1280, ‘float32’), (27, 1280, ‘float32’), 0, ‘float32’). A fallback configuration is used, which may bring great performance regression.

This is the code which converts torch to tvm using relay:

github.com/mit-han-lab/temporal-shift-module

online_demo/main.py

832a758f0


      
          def torch2tvm_module(torch_module: torch.nn.Module, torch_inputs: Tuple[torch.Tensor, ...], target):
              torch_module.eval()
              input_names = []
              input_shapes = {}
              with torch.no_grad():
                  for index, torch_input in enumerate(torch_inputs):
                      name = "i" + str(index)
                      input_names.append(name)
                      input_shapes[name] = torch_input.shape
                  buffer = io.BytesIO()
                  torch.onnx.export(torch_module, torch_inputs, buffer, input_names=input_names, output_names=["o" + str(i) for i in range(len(torch_inputs))])
                  outs = torch_module(*torch_inputs)
                  buffer.seek(0, 0)
                  onnx_model = onnx.load_model(buffer)
                  relay_module, params = tvm.relay.frontend.from_onnx(onnx_model, shape=input_shapes)
              with tvm.relay.build_config(opt_level=3):
                  graph, tvm_module, params = tvm.relay.build(relay_module, target, params=params)
              return graph, tvm_module, params