How can we use this attention visualization to do attention weights visaulization on tranformer

[liperrino]

Hi. Thanks to this work. Please i would like to know how we can extend the attention visualization to the transformer (attention is all you need)

[sandrobarna]

@liperrino Hi, that's pretty straightforward, you just need to capture similarity scores between key and context vectors in attention layer during forward pass and then plot heatmap or something similar. Let me know if it makes sense.