Align Your AI · Part Two
Contents
Reading the failure chapters
The seven chapters that follow describe how deployed AI systems go wrong. Each opens with one real case told at some length, then takes the failures in its family one at a time. A reader does not need all of them. Which ones matter depends mostly on how an organization gets and runs its AI, and the book sorts that into six situations.
Six situations
- Using a model through an API. The company calls a model hosted by a provider such as OpenAI, Anthropic or Google and never holds the model itself. The provider’s updates are changes the company did not make and often cannot see.
- Running its own copy. The company downloads an openly released model and runs it on its own servers, often after adapting it to its own data, compressing it to cut costs or combining it with other models. Every one of those steps produces a new model.
- Building models. A smaller group of organizations train models from scratch or run large training programs of their own, including the systems that score a model’s answers during training.
- Running agents. The model is given tools: it can browse, send email, run code, move money or change records. Here the question shifts from what the model says to what it does.
- Putting AI in machines. Robots, vehicles, drones, industrial controllers and medical devices that use learned models. A mistake becomes a movement in the physical world.
- Automating research. Systems that run experiments, write code for other models or evaluate other AI systems with little human review. Their output feeds the next round of work, so errors compound.
The situations overlap. A company running an agent on a provider’s model is in two of them and should read for both.
Where to start
| If you are mainly | Start with | Then read |
|---|---|---|
| using a model through an API | Chapters 7 and 10 | update regression, prompt injection, excessive agency |
| running your own copy | Chapters 2, 7 and 8 | safety erosion after fine-tuning, compression safety loss, compromised checkpoints |
| building models | Chapters 6 and 9 | gaming the grader, evaluation awareness |
| running agents | Chapters 5 and 10 | prompt injection, excessive agency, the agent supply chain, fabricated actions |
| putting AI in machines | Chapter 11 | robot jailbreaks, unsafe exploration, the certification gap |
| automating research | Chapters 6 and 9 | answer leakage, gaming the grader, fabricated actions |
Chapter 14 has a checklist for each situation, and Chapter 15 has questions to ask a team or a vendor.
How strong the evidence is
The failures in these chapters do not all rest on the same kind of evidence, and each entry says which kind it has.
Some have been seen in deployed systems: a company’s postmortem, a regulator’s file, a court record or a well-sourced news report describes the failure happening to real users. Some have been reproduced by independent groups: research teams with no authors in common, or tests across many different models, found the same effect, but no public report yet shows it in a deployment. And some have so far been shown in a lab setup: researchers built a situation designed to produce the failure and it appeared.
These grades measure how far the evidence has traveled toward real systems. Danger is a separate matter: a failure shown only in a lab can still be the one that matters most to a particular deployment, and a failure with many public cases may simply be the one that is easiest to see from outside. Appendix A lists every case the book cites, with its sources.
What each entry gives
Every failure entry follows the same order, so a reader who knows what to look for can move quickly.
It opens with a short summary: what happens, in plain words, and the best public example. A card beside it rates how exposed each of the six situations is (high, medium or low), says what a company that only uses an API inherits from its provider, describes what you would notice if it were happening to you, and gives the first thing to do. The exposure ratings are the author’s judgment from the evidence; Appendix B collects them in one table.
Then comes the explanation of why the failure happens, in two or three paragraphs, followed by the public cases. Where the public record has none, a short illustrative scenario shows how the failure could arise; scenarios are labeled as fiction and never stand in for evidence. Each entry then gives concrete steps to detect the failure and to prevent or repair it, written for the people who will carry them out, and names who in an organization usually owns the fix. It closes with what is still unknown. The standards crosswalk at the back of the book maps each failure to the matching items in widely used security frameworks such as the OWASP lists and MITRE ATLAS, and shows where in a system’s life each one enters.
Chapter 13 sets out the routine that most entries’ advice plugs into.