[iOS- Auto Tvm] Auto Tuning was not happening for iOS metal target

Questions
4

hi @kazum, Sorry for the late response.

Can you run tests/ios_rpc_test.py successfully? --> Yes I’m able to run the ios_rpc_test.py and got the results for both iPhone CPU and Metal target.

Your tuning script can run against the llvm target? --> Yes, I’m able to tune the model on llvm target and I can see some change in the GFLOPS.

Tuning options for llvm target Code :

target = "llvm"

batch_size = 1
dtype = "float32"
model_name = "resnet-18"
log_file = "%s.log" % model_name
graph_opt_sch_file = "%s_graph_opt.log" % model_name

# Set the input name of the graph
# For ONNX models, it is typically "0".
input_name = "data"
num_threads = 1
os.environ["TVM_NUM_THREADS"] = str(num_threads)

tuning_option = {
    'log_filename': log_file,
    'tuner': 'random',
    'early_stopping': None,
    'n_trial': 100,
    "measure_option": autotvm.measure_option(
        builder=autotvm.LocalBuilder(),
        runner=autotvm.LocalRunner(
            number=1, repeat=10, min_repeat_ms=0, enable_cpu_cache_flush=True
        ),
    ),

Output :

[Task  1/12]  Current/Best:   10.13/  19.01 GFLOPS | Progress: (100/100) | 354.49 sCannot connect to tracker ('0.0.0.0', 9000), retry in 5 secs...
 Done.
[Task  2/12]  Current/Best:    4.21/  20.30 GFLOPS | Progress: (100/100) | 300.69 sCannot connect to tracker ('0.0.0.0', 9002), retry in 5 secs...
 Done.
[Task  3/12]  Current/Best:    9.94/  19.52 GFLOPS | Progress: (100/100) | 282.74 sCannot connect to tracker ('0.0.0.0', 9000), retry in 5 secs...
 Done.

Can you try the following patch and check if something changes or not? --> I’m working on this, I will update if any changes in the GFLOPS.

Please let me know if you have any suggestions.

Thanks,