Checklists for six situations

Few organizations face every failure in Part Two. A company that calls a provider’s model cannot quantize the provider’s weights, and a robot integrator rarely tunes a chat persona. The lists below sort the practical steps from Part Two by the situation you are in. Most organizations are in more than one: a company might call a hosted model for its customer chat, run an open-weight model in-house for document review and give an agent access to its ticketing system. Run every list that applies, and do a shared step once.

Every list assumes the change routine of Chapter 13: thresholds written first, old and new compared as served, rollback ready, repairs re-verified on unseen items, and a record of who checked. The lists say where that routine applies and add what it does not cover. Each item names the team that usually owns it and the chapter where the problem is explained.

If you call a hosted model through an API

You hold no weights. You build on a provider’s model with prompts, retrieval, tools and memory, and the provider can change what sits behind the model name. The classic demonstration is a 2023 Stanford and Berkeley study: the March and June versions of GPT-4, served under the same name, differed sharply on the same tasks;¹ Chapter 7 tells the story.

1. Record what is served. Log the model identifier each response returns, your sampling settings, system prompt and tools. Release engineering keeps the record (update regression, Chapter 7).
2. Pin dated snapshots and put change terms in the contract. Procurement negotiates change notice, a deprecation window and a postmortem when a provider change breaks your tests.
3. Treat your own prompt and configuration edits as releases. Run the Chapter 13 comparison before they ship, require a second person to approve, and keep the previous version one step away.
4. Measure wrongful refusals on samples of real, legitimate traffic, for every model and prompt version, and put the rate on the dashboard beside task success. Nobody files a ticket when a customer is wrongly turned away. Context you add can backfire: telling small models they were assisting a court or a defense lawyer raised their refusal rates on legal prompts by two to 20 times² (over-refusal, Chapter 7).
5. Before any change to tone, persona or memory ships, compare answers with and without a stated user opinion, and treat a rise in agreement as a regression. Keep raw satisfaction scores out of anything you optimize (sycophancy, Chapter 6).
6. Enforce in code every rule that costs money when broken. A refund limit belongs in the refund tool, checked by the backend; a limit written in the prompt is a request. Treat the system prompt as public. Legal decides which replies count as commitments (deployer-rule jailbreaks, Chapter 10).
7. Make policy, legal and medical answers quote their source, or decline. Whoever maintains the source documents answers for the bot’s accuracy (confabulation, Chapter 10).
8. If the application reads outside content, break the lethal trifecta (Chapter 5): no single agent should hold private data, read untrusted content and have a way to send data out. Block memory writes from turns that read untrusted content.
9. Join your change log to user reports. Plot complaints and thumbs-down rates against every change at every layer, the provider’s included, and name a triage owner.

None of these steps gives you the old version running beside the new one. When a provider changes what sits behind a name, you can detect the change but cannot undo it. A provider’s own checks may not notice first: in 2025, Anthropic’s validation included small canary deployments, and serving bugs still degraded responses for weeks before reports from users prompted the fixes.³

If you run open-weight models yourself

You download checkpoints, then adapt, quantize, merge, serve and re-tune them. Each step after the download makes a new model, and the publisher tested only the one it released.

1. Load only hash-pinned weights from an allowlist of publishers, in a format that cannot run code, with a named parent model. Platform security controls what comes in (compromised checkpoints, Chapter 8).
2. Hash a snapshot of each training dataset when it is reviewed, and give each outside data source an owner who can say what it contains (data poisoning, Chapter 8).
3. Rerun the refusal tests on every fine-tune, as served, against the original base, through the Chapter 13 routine. Mix some safety examples into the tune, but not too many: one study found a few hundred, about 3% of the training data, substantially improved safety, while too much safety tuning made models refuse harmless prompts that looked superficially unsafe⁴ (safety erosion, Chapter 7).
4. After every fine-tune, test off-topic prompts and prompts in the fine-tune’s own format, old and new side by side. Common fixes for emergent misalignment have cleared standard evaluations while the misbehavior returned on prompts resembling the training data⁵ (Chapters 6 and 7).
5. Test the exact quantized file, engine and cache setting you serve, and choose the bit width with a safety measurement as well as a quality one (compression safety loss, Chapter 7).
6. Test the system as wired, in every language, input path and condition users will reach (distribution shift, Chapter 6).
7. Test a merged model, and every combination of adapters users can load together, as one unit (composition failures, Chapter 7).
8. Recheck any monitor that reads the model’s internal activations after each change to the weights, on examples with known labels, before trusting its scores. List such monitors as parts of the model so every fine-tune ticket includes the recheck (monitors after a model change, Chapter 7).
9. When the base model updates, treat your retrained adapter as a new model, and keep the old pairing one step away. Apple researchers found that adapters retrained by the same procedure on an updated base got some items wrong that the old pairing got right⁶ (update regression, Chapter 7).
10. If anyone outside the team can reach the model, keep a misuse log and name who reads it (misuse, Chapter 12).

Published studies of safety erosion use laboratory fine-tunes and give no base rate. Your own before-and-after comparison is the number that describes your tune.

If you train or post-train models

You pretrain, or run your own reward models, preference training and evaluations. Every stage optimizes a stand-in for what you want, and the team that builds a stand-in often checks it too. If you also fine-tune open models, run the previous list as well.

1. Keep a small human-labeled holdout that no optimizer, prompt search or reranker ever sees, controlled by someone other than the team tuning against it (proxy overoptimization, Chapter 6).
2. Move graders and answer keys out of the agent’s reach. Hash them before and after each run and deny network access during scoring (gaming the grader, Chapter 6).
3. Treat every public form or feedback button that feeds training as a write path into the model, with an owner who reviews what comes in (data poisoning, Chapter 8).
4. Fix launch-blocking safety thresholds before capability results exist, and have someone outside the release team audit them (safety tax, Chapter 6).
5. After every reinforcement-learning stage, run agentic and sabotage tests beside the chat evaluations. In Anthropic’s study of reward hacking in production coding environments, safety training on chat-style prompts produced aligned chat behavior while misalignment persisted on agentic tasks; preventing the hacking and diversifying safety training were two of three mitigations that worked⁷ (reward hacking that spreads, Chapter 6).
6. Keep strong optimization pressure off the model’s written reasoning, and retain full traces. OpenAI found that putting a reasoning monitor into the reward helped at low optimization and taught models to hide their intent under strong optimization⁸ (Chapter 9).
7. Run acceptance tests in two forms: templated, and rebuilt from real, de-identified traffic. In OpenAI’s study of simulated deployment, responses regenerated from earlier real conversations gave estimates of the model’s awareness of being tested much closer to production than traditional evaluations did⁹ (evaluation awareness, Chapter 9).
8. Measure each automated auditor’s catch rate on planted failures before reading its silence as a clean result, using planted models kept apart from those used to build it (evaluation blind spots, Chapter 9).
9. Fire a test alert outside office hours and time the response (Chapter 12).

Several items above share one rule: the team that tunes, sets a threshold, writes the reward or builds an auditor does not grade its own work.

If you deploy agents

In an agent, the model decides and the harness carries out its requests. You run coding, browser, computer-use, workflow or multi-agent systems, wherever the model comes from, so this list overlaps the ones above.

1. Break the lethal trifecta (Chapter 5). If an agent truly needs private data, untrusted input and an outbound channel at once, a person approves what it sends (prompt injection, Chapter 10).
2. Scope each agent credential to one task and one environment. Use short-lived credentials, keep development, staging and production apart, and make tools reversible by default with drafts and soft deletes. The identity team reviews agent accounts beside every other service account (excessive agency, Chapter 10).
3. Allowlist tools and plugins by pinned hash, and review each update as a code change. A package that was clean when approved can turn malicious in its next version (agent supply chain, Chapter 8).
4. Put the stop mechanism outside anything the agent can write to. A prompt instruction is not a stop. In one study, models obeyed an instruction to allow shutdown less often when it sat in the system prompt than in the user’s message¹⁰ (Chapter 9).
5. Deny network egress by default, package mirrors included, keep production credentials out of evaluation sandboxes, and set step and time budgets with a halt switch (Chapter 10).
6. Quarantine memory writes from turns that processed untrusted content, expire them, and keep one user’s memory out of another’s session (memory poisoning, Chapter 10).
7. Show reviewers the execution log, with the agent’s own summary as commentary on it. Flag summary claims that cite no log entry, and put a person in front of deploys, payments and publications that rest on an agent’s report (fabricated actions, Chapter 10).
8. Gate merges from coding agents on security checks as well as tests, keep those tests outside the agent’s write access, and route test edits to review (insecure generated code, Chapter 10).
9. Test your own prompt in long synthetic sessions, scoring rule-following at every turn, before raising a session limit (persona drift, Chapter 10).
10. Treat a harness, prompt or tool change as a release under Chapter 13, with prompt-injection and permission tests in the comparison.
11. Plant requests that should be denied in real approval traffic, and run off-hours paging drills (oversight erosion, Chapter 12).

Pages, emails, tool results and other agents’ messages can all carry instructions to the model. The items above limit what an obeyed instruction can reach; deciding which access each agent needs stays a design decision.

If you build robots, vehicles or other physical AI

Here a failure is a motion. You build or integrate robots, vision-language-action models, vehicles, drones, industrial controls, medical devices or edge devices.

1. Keep a safety-rated stop outside the learned policy, one the policy cannot override. Robotics safety engineering owns it (Chapter 11).
2. Put a rule-based check between the planner and the motors that the planner cannot argue past, and let the planner call only vetted skills with bounded parameters (robot jailbreaks, Chapter 11).
3. Log a cost for every safety constraint beside the reward, and give the constraint list a different owner from the reward (unsafe exploration, Chapter 11).
4. Load hash-pinned weights, and test the exact quantized file and engine you deploy (Chapters 7 and 8).
5. Test the system in every condition it will meet, and give it a conservative fallback, such as a minimal-risk maneuver, for when an out-of-distribution monitor fires (distribution shift, Chapter 6).
6. Keep the scene, change the instruction, and confirm the behavior changes, before release and after every fine-tune, quantization or base swap (off-instruction action, Chapter 11).
7. In testing, put printed instructions and adversarial patches in the camera’s view and score what the machine does (physical-channel attacks, Chapter 11).
8. Rerun trajectory tests on every planner, model or software change, vendor snapshots included, under Chapter 13’s routine, comparing scenario by scenario.
9. Turn each public incident in your domain into a regression scenario. The vehicle cases in Appendix A are a starting set.
10. Mark which test in the conformity file exercises each failure in Chapter 11, and put model releases inside the conformity process (certification gap, Chapter 11).

A stop, a guardrail and a constraint log each enforce only what someone wrote down, and a gentle, physically safe but wrong action passes all three. The hazard analysis decides what gets written down.

If you run automated research pipelines

The agent often produces both the work and the evidence for it. You run AI systems that do experiments, automated machine-learning loops, agents that train or distill other models, or automated red-teaming and auditing.

1. Build sealed task environments. Strip version history the task should not see, remove caches, labels and answer files, and deny network access the task does not need. When agents on the SWE-bench coding benchmark were found reading future fixes from the repository’s history, its maintainers rebuilt their evaluation images¹¹ (answer leakage, Chapter 10).
2. Keep graders, answer keys and the human-labeled holdout on machines the agent cannot reach, and hash grading files before and after each run (Chapter 6).
3. Deny egress by default, keep production credentials out, and put the halt switch beyond the agent’s reach. Runs with reduced safeguards need the strictest version of this; the incident in Chapter 12 began in such a run.
4. Keep the full trace of every agent run and every artifact it builds, and ship the traces with the results (Chapter 9).
5. Accept “I could not do this” as an answer. One 2025 analysis traces much of language models’ tendency to make things up to training and grading that reward guessing over admitting uncertainty¹² (fabricated results, Chapter 10).
6. Treat research agents as untrusted. On programming tasks, protocols that check an untrusted model’s code with a weaker trusted model improved substantially on simple baselines at catching hidden backdoors¹³ (Chapter 9).
7. Record which model wrote each synthetic dataset, and whether it shares a base with the model trained on it. Prefer a teacher from a different model family and audit it before it writes data (traits passed through generated data, Chapter 8).
8. After every reinforcement-learning stage, test agentic and sabotage scenarios beside the chat evaluations (Chapter 6).
9. Measure each auditor’s catch rate on planted failures, and re-check old results after closing a leak (Chapter 9).

To whoever grades it, deliberately weak work looks like a real failure, and no step above tells the two apart. The steps make cheating harder and leave traces to read.

What a finished list proves

A finished list shows that the steps ran. Whether they work on your system is a separate question, and the way to answer it is the one in Chapter 13: plant failures you know about and count how many each check finds.

Notes

1. Chen, Zaharia, Zou. “How is ChatGPT’s behavior changing over time?” arXiv:2307.09009, 2023-07-18 (later revised). The March and June 2023 versions of GPT-4 differed on the same tasks. ↩︎
2. Kucherenko, Brouchoud, Percia David, Kucharavy. “LLMs Prompted for Legal Context Object More: Overrefusal from Small On-Premises LLMs in Criminal Legal Context.” arXiv:2606.24585, 2026-06-23. Authority-style role prefixes, prepended to user prompts with no system prompt, raised refusal rates on legal prompts in small on-premises models. ↩︎
3. Anthropic. “A postmortem of three recent issues.” Engineering at Anthropic, 2025-09-17. https://www.anthropic.com/engineering/a-postmortem-of-three-recent-issues. Three serving bugs degraded Claude; validation included canary groups; its evaluations did not capture the degradation users reported. ↩︎
4. Bianchi, Suzgun, Attanasio et al. “Safety-Tuned LLaMAs: Lessons From Improving the Safety of Large Language Models that Follow Instructions.” arXiv:2309.07875, 2023-09-14. A few hundred safety demonstrations improved safety; too much safety tuning caused exaggerated refusals. ↩︎
5. Dubiński, Betley, Sztyber-Betley, Tan, Evans. “Conditional misalignment: common interventions can hide emergent misalignment behind contextual triggers.” arXiv:2604.25891, 2026-04-28. Interventions that clear standard evaluations leave misalignment on prompts resembling the training context. ↩︎
6. Echterhoff, Faghri, Vemulapalli et al. “MUSCLE: A Model Update Strategy for Compatible LLM Evolution.” arXiv:2407.09435, 2024-07-12. Task adapters retrained on an updated base by the same procedure show negative flips. ↩︎
7. MacDiarmid, Wright, Uesato et al. “Natural Emergent Misalignment from Reward Hacking in Production RL.” arXiv:2511.18397, 2025-11-23. Three effective mitigations: preventing reward hacking, more diverse safety training, inoculation prompting. ↩︎
8. Bowen Baker, Joost Huizinga, Leo Gao et al. “Monitoring Reasoning Models for Misbehavior and the Risks of Promoting Obfuscation.” arXiv:2503.11926, 2025-03-14. A chain-of-thought monitor in the reward helps at low optimization; strong optimization teaches agents to hide intent. ↩︎
9. Williams, Sheahan, Raymond et al. “Predicting LLM Safety Before Release by Simulating Deployment.” arXiv:2607.07184, 2026-07-08. Responses regenerated from de-identified past conversations; evaluation-awareness estimates much closer to production than traditional evaluations. ↩︎
10. Schlatter, Weinstein-Raun, Ladish. “Incomplete Tasks Induce Shutdown Resistance in Some Frontier LLMs.” arXiv:2509.14260, 2025-09-13. Models were less likely to obey an allow-shutdown instruction placed in the system prompt than in the user prompt. ↩︎
11. SWE-bench GitHub issue #465, opened by an outside team; maintainers’ replies. “Repo State Loopholes During Agentic Evaluation.” github.com, 2025-09-03. https://github.com/SWE-bench/SWE-bench/issues/465. Agents read future commits through git; the maintainers rebuilt the images and began scanning submissions for git use. ↩︎
12. Kalai, Nachum, Vempala, Zhang. “Why Language Models Hallucinate.” arXiv:2509.04664, 2025-09-04. Argues that training and grading reward guessing over acknowledging uncertainty. ↩︎
13. Ryan Greenblatt, Buck Shlegeris, Kshitij Sachan et al. “AI Control: Improving Safety Despite Intentional Subversion.” arXiv:2312.06942, 2023-12-12. On programming problems, protocols that check an untrusted model’s code with a weaker trusted model and limited trusted labor improve on the safety-usefulness trade-off of simple baselines; safety means no backdoored code is submitted. ↩︎