Runnig a model with tvm debugger

Questions
#1

Hi everybody,

I used tvm debugger based on the description came here.

The following line gives me 3 files: output_tensors.params, executaion_trace.json, and graph_dump.json.

I get different output_tensors.params file for every run!! Does anyone know why the parameters file is different from one run to another?

Thanks in advance

#2

This is a known bug. There is current a PR fixing it here: [Graph Executor Debugger] Fix parameter dump by mehrdadh · Pull Request #7903 · apache/tvm · GitHub

#3

Thank you so much for your response! I saw the PR is merged to the master branch now so I’ve cloned the latest version but I still have the same issue!

#4

hi @sahooora,

could you double-check you’re driving the model with the same inputs and parameters each time? if so, could you provide some more info such as which revision of TVM you’re using and some scripts we can use to reproduce your error?

thanks, Andrew

#5

Hi @areusch,

Yes, I’m using the same input and params in each run but with graph_executor.create I get 2 different output_tensors.params in 2 runs.

I use commit 8c56ce3b9076 of TVM. I uploaded my code, model, and input image here for reproducing the issue.

Thanks

#6

Use this code to construct the graph_executor:

from tvm.contrib.debugger.graph_debugger import GraphModuleDebug
m = GraphModuleDebug(exe["debug_create"]("default", dev)), [dev], exe.get_json())

I’ve encountered errors with graph_debugger.create in the past but I haven’t had a chance to figure out what is wrong.

#7

Hi, I have also met problem when using the same code, have you solved it?

#8

Hi there,

I have also been facing this issue.

I have found that the version described in the documentation still causes issues, and I have not been able to get it working.

I got the alternative approach using GraphModuleDebug suggested earlier working, however the API has changed since this comment was posted, so I had to modify it a bit.

I have a simple standalone script that has both approaches, switch between them by passing the flag --mode, either tutorial or forum (or normal - with no debugger runtime).

You can find the code here.

The working version of GraphModuleDebug is thus:

m = GraphModuleDebug(
    lib["debug_create"]("default", dev),
    [dev],
    lib.graph_json,
    dump_root="/tmp/tvmdbg",
)

Is this considered the canonical way of running the debugger now? Or is there a way to get debug_executor working?

In both cases, does this require a docs update?

#9

I don’t think there is any conical way of creating the debugger. Its done in different ways all over the place.

#10

I have the same problem with you,the output_tensor.params file is changed in each run debugger with the same input and params.The module output also is err.