Example of a model using attention?

Application
1

My research collaborators are interested in porting some RNNs with attention into Relay. Are there existing public examples of Relay models that use attention that I could use as a guide to implement some of these RNNs? I would appreciate any examples or assistance

2

I think there is an example in PyTorch: NLP From Scratch: Translation with a Sequence to Sequence Network and Attention — PyTorch Tutorials 2.7.0+cu126 documentation

Hope this could help :slight_smile:

3

I think that will be very useful, thank you