How to use PAPI to measure FLOPs of a model compiled by TVM?

Questions
#1

I have read Getting Started With PAPI — tvm 0.8.dev0 documentation (apache.org) and followed codes in the doc.

exe = relay.vm.compile(mod, target, params=params)
vm = profiler_vm.VirtualMachineProfiler(exe, dev)

data = tvm.nd.array(np.random.rand(1, 3, 384, 384).astype("float32"), device=dev)
report = vm.profile(
    [data],
    func_name="main",
    collectors=[tvm.runtime.profiling.PAPIMetricCollector()],
)
print(report)

But when I ran to the code above, it reported error (See in the picture below)

image
image1808×772 69.1 KB

Are the “mod” and “params” wrong? In my code, the are the return value of the function “relay.frontend.from_pytorch”

mod, params = relay.frontend.from_pytorch(scripted_model, shape_list)

But in official doc, they are the return value of “mlp.get_workload(1)”

mod, params = mlp.get_workload(1)

How to use PAPI to measure FLOPs of a model compiled by TVM?

#2

Hi @iwangyuezhang, this error normally indicates that PAPI is not configured correctly. Could you provide the output of papi_avail and papi_native_avail?

#3

hi，@tkonolige，i meet a similar problem. When runing to the last line of code, error comes out:

image
image1905×388 81.1 KB
I have checked that perf tools don’t have (or support) stalled-cycles-fronted and stalled-cycles-backed under the command of perf stat ls:
image
image835×304 11.3 KB
These are the output of pap_avail:

image
image816×402 16.3 KB
image
image865×916 39 KB
image
image895×902 32.9 KB
image
image1242×347 16.6 KB

the output of papi_native-avail is too large to copy Do you have other suggestions, thanks a lot.

#4

@xueshe perf::STALLED-CYCLES-FRONTEND should be listed in the output of papi_native_avail. However, I don’t see stalled cycles in the output of perf stat ls, so my guess is that your platform does not support this hardware counter.

#5

Hello @tkonolige

Result of papi_native_avail is as below, even then i am facing this issue. Could you please tell what the issue might be?

$ papi_native_avail | grep -ie perf::STALLED
| perf::STALLED-CYCLES-FRONTEND                                              |
| perf::STALLED-CYCLES-BACKEND                                                 |

ERROR: 0: tvm::runtime::profiling::PAPIMetricCollectorNode::Init(tvm::runtime::Array<tvm::runtime::profiling::DeviceWrapper, void>) ** File “/home/tvm-code/vinay.n/TVM_OpenSource_OPENCL_profiling/tvm/src/runtime/contrib/papi/papi.cc”, line 200** PAPIError: -7 Event does not exist: perf::STALLED-CYCLES-FRONTEND.

PAPI Version: 
$ papi_version 
PAPI Version: 6.0.0.0

I have tried to run code given in this tutorial: Getting Started With PAPI — tvm 0.16.dev0 documentation