Attention in transformers, visually explained | DL6

What are the neurons, why are there layers, and what is the math underlying it? Typo correction: At 14 minutes 45 seconds, the last index on the bias vector is n, when it’s supposed to in fact be a k. Thanks for the sharp eyes that caught that! There are two neat things about this…

This is the second of a series of 3 videos where we demystify Transformer models and explain them with visuals and friendly examples. 00:00 Introduction01:18 Recap: Embeddings and Context04:46 Similarity11:09 Attention20:46 The Keys and Queries Matrices25:02 The Values Matrix28:41 Self and Multi-head attention33:54: Conclusion

Learn prompt engineering techniques to get better results from ChatGPT and other LLMs.

https://www.youtube.com/watch?v=9-Jl0dxWQs8 AI Alignment forum post from the Deepmind researchers referenced at the video’s start:https://www.alignmentforum.org/posts/… Anthropic posts about superposition referenced near the end:https://transformer-circuits.pub/2022…https://transformer-circuits.pub/2023… Some added resources for those interested in learning more about mechanistic interpretability, offered by Neel Nanda Mechanistic interpretability paper reading listhttps://www.alignmentforum.org/posts/… Getting started in mechanistic interpretabilityhttps://www.neelnanda.io/mechanistic-… An interactive demo of sparse autoencoders (made…

Dec 3, 2024One-day workshop on topics in Generative AI IISc-IBM AI Day is being jointly organized by the Centre for Networked Intelligence (with support from Cisco CSR) and IBM-IISc Hybrid Cloud Lab, in collaboration with IBM India Research Lab. The goal of this workshop would be to apprise the audience of Generative AI, a set…