# \[RFC\] Type-Directed Relay Fuzzing Library

**URL:** <https://discuss.tvm.apache.org/t/rfc-type-directed-relay-fuzzing-library/12234>\
**Category:** RFC\
**Created:** [March 4, 2022, 4:22am UTC](https://discuss.tvm.apache.org/t/rfc-type-directed-relay-fuzzing-library/12234 "2022-03-04T04:22:26Z")\
**Posts on this page:** 1\
**Showing post:** 22

<div class="post-metadata">

**Author:** ![driazati](https://discuss.tvm.apache.org/user_avatar/discuss.tvm.apache.org/driazati/32/3303_2.png) [@driazati](https://discuss.tvm.apache.org/u/driazati)\
**Post date:** [October 26, 2022, 4:02pm UTC](https://discuss.tvm.apache.org/t/rfc-type-directed-relay-fuzzing-library/12234/22 "2022-10-26T16:02:11Z")

</div>

We discussed a few aspects of this at the community meeting this morning:

- When should we generate programs to test? When should the tests be run?
  - @slyubomirsky Other fuzzers have many different approaches, some generate and save programs and others generate them each time. Maybe we should do both in order to test the fuzzer
  - @driazati we should get the fuzzer running in CI at some scale now, we can grow or change it as we try it out

- As many (potentially thousands) of generated programs hit the same kinds of bugs, how do we group stack traces to simplify error reporting / bug hunting?
  - @slyubomirsky Fuzzing frameworks also differ here, some look at parts of the stack trace
  - @gromero Maybe it’s possible to look at program features to identify what triggers similar bugs

- How do we save generated programs without relying on the Relay text format?
  - @slyubomirsky `pickle`ing the IRModule works out of the box but is pretty opaque
  - @driazati `pickle` might be ok, we can probably disable the arbitrary code execution parts of it (another note: pickle also has a [text format](https://docs.python.org/3/library/pickletools.html#pickletools.dis) that might be easier to stomach, but there is no parser for it afaik)
  - @sebastianboblestetas We should do the JSON format if the serializer is simple enough

---

_[View the full topic](https://discuss.tvm.apache.org/t/rfc-type-directed-relay-fuzzing-library/12234)._
