MergeComposite on QNN graph broken in the latest main?

Questions
1

@mbaret @lhutton1

I’m having an issue with the latest main branch, where I get an error from MergeComposite pass running on QNN graphs:

  File "/home/masa/projects/dev/tvm/src/relay/analysis/type_solver.cc", line 622
TVMError: 
---------------------------------------------------------------
An internal invariant was violated during the execution of TVM.
Please read TVM's error reporting guidelines.
More details can be found here: https://discuss.tvm.ai/t/error-reporting/7793.
---------------------------------------------------------------
  Check failed: false == false: [11:26:20] /home/masa/projects/dev/tvm/src/relay/qnn/op/../utils.h:175:

The error disappears if I revert this PR https://github.com/apache/tvm/pull/6704 that changed type inferencer to mixed mode, so I have a feeling that the PR broke merge composite on QNN graphs. Since we have no test in CI that exercises BYOC flow on QNN graphs, CI wouldn’t detect it.

Have you seen this issue using the latest main? I want to be sure before I dig deep.

2

We’re not seeing this sort of failure at the moment. The tests here: https://github.com/apache/tvm/blob/main/tests/python/contrib/test_ethosn/test_networks.py should at least partially cover QNN BYOC. One thing that may be worth trying (because we now do it for Ethos-N) is explicitly running InferType before running MergeComposite.

3

Thanks. I tried adding infer type before merge composite, but it didn’t help. Also tried running ethos-n tests up to merge composite, but since registering patterns requires support lib install, which I don’t have, merge composite returns no composite.

UPDATE: I fiddled with ethos patterns a bit to get merge composite working, and indeed, I can confirm that it is working.

Whether or not the error happens may depend on how frontend emits QNN graph (I use pytorch) or what composite is generated. I’ll try investigating what’s going on…

4

@mbaret The problem with the new type inferencer turned out to be real, it is solved in https://github.com/apache/tvm/pull/7074 by @mbrookhart

1 Like