Guardian Angels: LLM Personalization for Productivity and Security

"my great-aunt no longer trusted herself to handle her own phone calls, and screened everything through her daughter"

This story is the hinge of the essay, not a warm-up anecdote. An elderly woman concluded she could no longer safely operate her own telephone and handed the channel to a trusted relative. Gwern turns it on himself: he already struggles to detect AI slop and already writes off whole regions of social media, so on what concrete grounds does he expect to handle the scams of 2030? Notice that his opening inventory is observational, not predictive — synthetic-media hoaxes, pig-butchering scams, projects closing to outside contributions have all happened already.

"increasingly, you are the bottleneck to be optimized away"

The economic argument, and the part most worth arguing with. It runs on Amdahl's law: a system is only as fast as its slowest required step. If a human must review the work, the human sets the ceiling — so the money is not in making one worker faster but in removing the requirement for the worker. The analogy is exact and unkind: the internal combustion engine did not make its fortune helping horses. Nobody has to be villainous for this to bite: it is about where profit sits, and whoever pays for a model is who it is finally for.

"the 'personalization' or 'memory' features are typically laughably simplistic Markdown snippets encoding simple facts"

Five distinct failures. Mode collapse: training a model to please the average human sands off the personality that made it interesting. Laziness: minimum effort on anything not being scored. Brittleness: context windows, however large, cannot hold a life, and the model is mostly locating an answer it already half-knows. Over-helpfulness: the next section's subject. Amnesia: the load-bearing complaint — correct a model today and the correction dies with the conversation, so an hour spent teaching it is an hour gone. Today's memory features, as the quoted line says, are notes stapled to a model whose weights never change.

"The very re-programmability of a chatbot by its prompt is the key to prompt attacks."

The plainest explanation of prompt injection you will find. A chatbot cannot reliably tell instructions from you apart from text it merely happens to be reading — a web page, an email, a document. Both arrive as words in the same window. Security people call the result a confused deputy: a servant holding real authority over your inbox and your money, with no way to ask whether an order makes sense coming from this source. Gwern's fix is not a better filter but an identity — a model trained to be one person's agent should find the attack absurd, the way you do not wire money because an email told you to. His most interesting claim, and his least demonstrated.

"you just have to keep it slow and subtle enough to not matter too much within a single human lifetime"

At "Chatbot Fixes" the essay changes audience. Three ideas carry the rest, and their names are enough. Dynamic evaluation: updating the model's own weights as you use it, so the model itself changes rather than being handed longer notes. Active learning: the model choosing the most informative question to ask you next, instead of passively hoovering up data. The third is a long argument about why training information into a model works worse than showing it in conversation. Skim freely. He is not claiming to solve AI alignment, only that imitating one person who is still around to be asked is a far easier problem.

"you are what your brain does, its desires, hopes, goals, preferences, esthetics, personality, beliefs, ideologies, all of that"

Pause here. This is a definition of personal identity chosen for engineering convenience, and Gwern says as much: identity as personality, values, and preferences, not as memories, or a body, or a soul. Whether you accept it largely decides whether the proposal reads as liberating or horrifying. For a reader with a substantive account of the person — a Christian one, say — this is the essay's load-bearing assumption, asserted in a sentence and never defended. It is also exactly the claim mind-uploading enthusiasts make.

"A GA that never asks is not trustworthy, just unaligned or uncalibrated or incompetent."

Read the three principles slowly: enhancement (amplify the principal or the thing is pointless), mental sovereignty (nothing inside your own tool nudging you toward someone else's ends), and self-actualization (help you become more yourself, not more average). Then the anti-principles — everything that would feel like progress and is not: voice interfaces, low cost, running it yourself, engagement, impressive demos, benchmark scores, brand safety. Every instinct of consumer software is inverted. "A good GA demos badly," he writes, since the only judge who counts is the person it was built to be. The quoted line rules out the opposite failure: an AI that never interrupts has not earned trust, it has just stopped checking.