BYOC pattern matching questions

Questions
#1

Hello,

I have a few questions about pattern matching in BYOC.

  1. Is it possible to keep a pattern such that conv & add are fused only if add is not followed by a max_pool?
  2. Is it possible to check if a particular op is going to multiple branches for example in the above example, I want to fuse conv & add only if add doesn’t have multiple branches from it, i.e, it is input on a single layer & not fed to multiple other layers as input.

Thanks in advance!

#2
  1. We might need a “not” operator in the pattern language. @mbrookhart what do you think?
  2. It’s impossible because we have no way to know who is going to use the output of a certain op.
#3

  1. A not pattern wouldn’t be that hard to implement, that could be an interesting addition, I could give that a shot next week.

  2. This is actually kind of complicated. I have this information deep in the guts of the pattern matcher, I need it for dominator analysis, but I’m not sure how to expose it. Maybe we could wrap the output in a NUsers kind of pattern that only matches if the contained node is used N times?

#4

I feel that NUsers is out of scope of pattern matching because it makes the pattern not self-contained and it would be hard for developers to reason why the pattern is matched or not matched, although it might be useful some times.

It seems to me that an ideal solution to this requirement is matching all conv+add patterns, and having another pass doing def-use analysis to cancel the partitions with more than one uses.

#5

@comaniac @mbrookhart , thank you for your replies.

I have raised question #2 because I haven’t faced any issue with single branch models but in multiple branch models, when conv + add (runs on byoc) is followed by max_pool (runs on byoc) in one of the branches and some of the other branches to which the output of add goes are run on tvm native code (not supported on byoc), I am getting error in well-formed checker saying “variable x is bound more than once, this is not valid IR”. Can you suggest some pointers to handle this?

Thanks!

#6

Hmm, that sounds like things are getting mis-partitioned. Any chance you can post a minimal reproducing example?

#7

Right, it sounds like a bug or mis-use to me. If the partition pattern is conv-add-max_pool, it should not be matched and partitioned if one of its intermediate results (in this case, the output of add) will be used by other ops. On the other hand, if it is partitioned, then it should expose the output of add to be one of the partitioned function outputs (i.e., (%3, %2), where %3 is the output of max_pool and %2 is the output of add). I don’t quite remember which one is the current implementation, but it should work anyways. Like @mbrookhart mentioned, it would be good to have a minimal reproducible example.