The Bitter Lesson

Nine hundred words that explain more of the last decade of AI than most books. Sutton looks back over seventy years of AI research and finds one pattern repeating: researchers build human knowledge into their systems, it helps for a while, and then approaches that simply apply more computation — search and learning — sweep past them. Chess, Go, speech recognition, computer vision: each field resisted the pattern, and each eventually confirmed it.

Read it because it is the closest thing the current AI wave has to a founding document. When you hear that a lab "scaled up" a model, or that a clever hand-engineered system was abandoned for a bigger general one, this essay is the background assumption doing the work. It is also short, plainly written, and more careful than its reputation — Sutton is not saying that scale is all you need, but that methods which ride growing computation beat methods which encode how we think we think.