Visualizing transformers and attention | Talk for TNG Big Tech Day ’24

Timestamps:0:00 – Who this was made for0:41 – What are large language models?7:48 – Where to learn more

https://www.youtube.com/watch?v=9-Jl0dxWQs8 AI Alignment forum post from the Deepmind researchers referenced at the video’s start:https://www.alignmentforum.org/posts/… Anthropic posts about superposition referenced near the end:https://transformer-circuits.pub/2022…https://transformer-circuits.pub/2023… Some added resources for those interested in learning more about mechanistic interpretability, offered by Neel Nanda Mechanistic interpretability paper reading listhttps://www.alignmentforum.org/posts/… Getting started in mechanistic interpretabilityhttps://www.neelnanda.io/mechanistic-… An interactive demo of sparse autoencoders (made…

Dec 3, 2024One-day workshop on topics in Generative AI IISc-IBM AI Day is being jointly organized by the Centre for Networked Intelligence (with support from Cisco CSR) and IBM-IISc Hybrid Cloud Lab, in collaboration with IBM India Research Lab. The goal of this workshop would be to apprise the audience of Generative AI, a set…

If you’re interested in the herculean task of interpreting what these large networks might actually be doing, the Transformer Circuits posts by Anthropic are great. In particular, it was only after reading one of these that I started thinking of the combination of the value and output matrices as being a combined low-rank map from…

How does AI learn? Is AI conscious & sentient? Can AI break encryption? How does GPT & image generation work? What’s a neural network? #ai #agi #qstar #singularity #gpt #imagegeneration #stablediffusion #humanoid #neuralnetworks #deeplearning

This is CS50, Harvard University’s introduction to the intellectual enterprises of computer science and the art of programming. TABLE OF CONTENTS 00:00:00 – Welcome00:01:01 – Introduction00:03:13 – Image Generation00:08:23 – ChatGPT00:11:06 – Prompt Engineering00:12:40 – CS50.ai00:19:03 – Generative AI00:22:08 – Decision Trees00:26:33 – Minimax00:34:27 – Machine Learning00:42:56 – Deep Learning00:48:53 – Large Language Models00:53:36 –…