[MetaSchedule] Database Loading Time

Troubleshooting
1

After tuning with many many trials with MetaSchedule, the size of the records file is beyond 1GB+. Then the database loading

 database = ms.tir_integration.tune_tir(
        mod=ir_module,
        target=target,
        work_dir=work_dir,
        max_trials_global=16,
        num_trials_per_iter=16,
        space=ms.space_generator.PostOrderApply(
                sch_rules="cuda-tensorcore", postprocs="cuda-tensorcore", mutator_probs="cuda-tensorcore")
    )

hangs there. Is there any method to improve the loading speed?

2

Hi @Maximilianxu ,

I also had a similar problem. So I’ve made a custom database backed by SQLite. The loading time will be significantly faster (70sec vs 0.5sec on my env) than the JSON database. I’ll post the source code after I got the permission to share it.

1 Like
3

Look forward to that! It does make sense that we can replace the JSON database with triditional databases.

4

@Maximilianxu

I’ve uploaded our SQLite-backed Database implementation. Hope you like it! Any comment is welcome!

1 Like