[1hr Talk] Intro to Large Language Models

"a large language model is just two files"

The most useful idea in the talk. Strip away the website, the company, and the chat interface, and what remains is a file of parameters plus a small program that runs them. His example is Meta's Llama 2 70B: 140 gigabytes of parameters and a runner of roughly 500 lines of C, self-contained on a laptop with no internet. Every later mystery is a mystery about how those numbers got into that file.

"you need about 6,000 gpus and you would run this for about 12 days"

The ingredients: ten terabytes of web text, thousands of specialized computers, twelve days, about two million dollars — producing 140 gigabytes of parameters. He calls the result "a zip file of the internet," then corrects himself: a lossy one. The original text is not in there; the gist is. His aside matters — these figures were already outdated in 2023, off "by factor of 10 or more" at the frontier. The numbers grew; the recipe did not change.

"this neural network basically is just trying to predict the next word in a sequence"

The single objective behind all of it: given some words, guess the next one. The Wikipedia page he pulls up makes it click — to predict the next word of a biography you must have absorbed who the person was. Run it forward and the model "dreams" documents, inventing product listings with fabricated ISBN numbers. That is the clearest account of hallucination you will find: not a malfunction, but the same machinery working normally where the gist lacks the fact.

"we don't know how these parameters collaborate to actually perform that"

The Transformer diagram appears — the only place a non-technical viewer should let the words wash past. The takeaway needs no diagram: we know exactly which mathematical operations run, and not what the parameters are doing. Pause for his example — a model names a celebrity's mother but fails the reverse question about her son. His phrase, "mostly inscrutable artifacts," is why these systems get judged by testing behavior rather than by reading their insides.

"we're now moving to the second stage of training which we call fine-tuning"

Everything so far is useless for conversation — feed it a question and it writes more questions. The fix: companies hire people, hand them a manual, and pay them to write ideal question-and-answer pairs. Around a hundred thousand, quality over quantity, a day of computing instead of months. Two details reward attention: the optional third stage at 21:11, where labelers merely rank candidate answers (the technique known as RLHF), and his account at 19:24 of how misbehavior gets fixed — a human writes the correct response into next week's training data.

"the first very important thing to understand about the large language model space are what we call scaling laws"

The most consequential section economically. The claim: prediction accuracy is a smooth, predictable function of two numbers — how big the model is and how much text it saw. Bigger and longer reliably means better, no cleverness required. He is explicit that this drives the industry's capital spending, since a larger cluster is the one path to a better model that requires no new idea. Whether the curve continues indefinitely, he does not settle.

"tool use is a major aspect in how these models are becoming a lot more capable"

The best few minutes to show someone who has never seen this work. He asks for a table of funding rounds and the model searches the web; asks it to fill gaps and it reaches for a calculator; asks for a chart and it writes and runs code; asks for a picture and it calls an image generator. Watch how ordinary the English stays. Everything now sold as an "AI agent" is this pattern, scaled up; the seeing-and-hearing preview at 33:10 is routine today.

"a lot more correct to think about it as the kernel process of an emerging operating system"

A forecast worth scoring two and a half years later. He wants models that trade time for accuracy — thinking for thirty minutes rather than answering instantly — and says flatly that none could in 2023. Reasoning models arrived about a year later. His speculation about self-improvement, drawn from AlphaGo, has partly arrived in narrow domains where answers can be checked automatically. The framing to carry forward is the "LLM OS": the model as a computer's coordinating process, its context window the working memory.

"we're going to have new security challenges that are specific to large language models"

Three attack families, each demonstrated concretely. Jailbreaks talk a model out of its refusals — the roleplay trick, or a request encoded in a format the safety training never covered. Prompt injection is the one to understand properly: text hidden in a web page or shared document, invisible to you, which the model reads as instructions. His search result smuggling in a phishing link is the whole problem in one screen. Data poisoning plants a trigger phrase during training. The demos have been patched; the categories have not.