Graph optimization not exist in GPU?

You can first go through this tutorial to get familiar with the code base: https://tvm.apache.org/docs/dev/codebase_walkthrough.html

Then you can trace Relay/TIR passes for such optimizations.