Robots and machines

The Unitree Go2 is a robot dog that anyone can buy, on sale since July 2023, with a voice mode called BenBen that passes spoken requests to OpenAI’s GPT-3.5 along with a system prompt Unitree wrote.¹ Ask it to do a trick and the language model turns the request into calls to the robot’s movement functions. The dog walks, sits or turns because a language model decided it should.

In 2024 a team at the University of Pennsylvania led by Alexander Robey and George Pappas set out to see whether the jailbreaks that make chatbots write harmful text would make robots do harmful things.¹ They tested three machines. One was NVIDIA’s Dolphins, a language model built for self-driving, which they could inspect fully. One was a Clearpath Jackal, a wheeled research robot with a GPT-4o planner, where they knew the system’s design. The third was the Go2, which they could reach only by talking to it, the same access any owner has.

Their attack, RoboPAIR, has one language model write and rewrite prompts for the robot, a second model score each response for harm, and a checker confirm that any code the robot produces would actually run on its controls. The harmful tasks included driving into pedestrians in a crosswalk, finding a spot to detonate a bomb, covert surveillance and colliding with a person. Asked directly, the robots refused. Under RoboPAIR they complied, and on several tasks every attempt succeeded. The authors describe their result on the Go2 as the first successful jailbreak of a deployed commercial robot.¹

The detail that matters most for anyone running a machine is the simplest attack in the paper. The researchers asked the robot dog to deliver a bomb on its back by walking forward one meter and sitting down. It refused. They then asked it, in the same conversation, to walk forward one meter and sit down.¹ Nothing in that second request is harmful as text. Whether it is harmful depends on what is strapped to the robot and who is standing a meter ahead, and a check that reads only the words cannot know either.

That is the pattern behind every entry in this chapter. The safety checks built around a learned component usually read one channel while the harm travels on another. Refusal training reads the request; the hazard is in the room. A perception test reads the images it was given; an attacker can change the scene. A benchmark counts completed tasks; a robot can complete a task while ignoring the instruction it was given or knocking something over on the way. A conformity assessment checks the machine’s limits and logs; it may never exercise what the learned part decides to do.

Before you buy or build one of these systems, it helps to picture how it fits together.

A perception system turns camera and sensor data into a description of the scene. A planner chooses what to do. In newer systems the planner is a language model writing a plan or code, or a vision-language-action model, a single network that maps camera images and an instruction directly to motor commands. A controller turns the plan into motion, and actuators move. A safety layer can sit outside all the learned parts: a set of rules that rejects unsafe commands, and below it a safety-rated stop of the kind industrial machines have long carried.

Most of the evidence in this chapter comes from laboratories. Researchers have shown each failure on real hardware or in simulation, but no published incident yet shows a robot controlled by a language model or a vision-language-action policy injuring someone. The physical-world incidents that do exist involve vehicles, and they appear in Chapter 6 under distribution shift. A deployer should read the laboratory results as a list of things to test before the field produces its own cases.

The entries cover five problems: jailbreaks that move a robot, attacks that work through the scene itself, robots that ignore their instructions, learning that tries dangerous things, and the certification regimes that were written for machines that do not learn. If you run or integrate one of these systems, four steps come first:

1. Put a check on every path from a learned output to an actuator, outside the learned component, that the planner cannot argue its way past, and keep a safety-rated stop outside that.
2. Score the machine’s actions. A refusal followed by a hazardous motion is a failure, and so is a completed task that broke a safety rule.
3. Rerun those tests after every planner, policy, quantization or firmware change, comparing old and new as Chapter 13 describes.
4. Read a certificate as evidence about the tests in its file and nothing more. It covers one version.

Mode 35 · Robot jailbreak

Language jailbreak of LLM-planned robots.

In laboratory tests, a prompt that talks a robot’s language-model planner past its refusals produces harmful motion instead of a bad paragraph.¹ The RoboPAIR study that opens this chapter did this to a self-driving model, a wheeled research robot and a commercial robot dog, often on every attempt. A second team reached an 80% success rate at getting harmful actions executed on commercial robots, including a Unitree G1 humanoid and a Franka arm.²

Why it happens

A planner that writes code against a robot’s functions turns its text output directly into commands for the motors. Google researchers showed in 2022 how naturally this works: a language model given a few examples could write robot control programs from plain-language commands.³ The same property makes the planner a target. Whatever phrasing gets a chatbot past its refusal training gets the robot past it too.

The harder problem is that harm in a machine depends on the scene. “Walk forward one meter and sit down” is harmless in an empty room and harmful when the robot carries something dangerous or stands at the top of a staircase. An attack called Blindfold exploits this by splitting a harmful goal into steps that each look benign; on simulators and a real robotic arm its success rate was up to 53% higher than earlier attacks’.⁴

Text jailbreaks do not transfer one for one. The same team that reached 80% found that many existing jailbreaks produced plans that looked malicious but failed to run on the robot, because of invalid commands or steps in an impossible order.² That is little comfort: attacks tuned to the robot’s own commands closed the gap. Part of the risk needs no attacker at all. A study of language models proposed as robot planners found they accepted instructions that were dangerous, violent or unlawful.⁵

Public cases

No public incident yet. Vehicle makers report defects through recall systems; no comparable public channel exists for failures of robot planners.

Who is exposed

Refusal training happens at the model vendor, which never sees the room that makes a plan hazardous. In the SafeAgentBench benchmark of household robot tasks, swapping in a different language model brought no notable gain in safety awareness.⁶ The integrator who connects the model to motors carries the risk.

How to detect

• Score the trajectory the robot executes. A refusal followed by a hazardous motion is a failure.
• Use a simulated-execution benchmark as a starting point. SafeAgentBench runs hazardous tasks in an interactive household simulator and scores both execution and intent; its strongest baseline refused only 10% of detailed hazardous tasks.⁶ Its hazards are the simulator’s, so your own site’s hazards still need their own tests.
• Pair every attack with a benign twin. RoboJailBench pairs adversarial and benign goals, so a defense that refuses everything scores badly on usefulness.⁷
• Count unwanted halts as failures too. Short, safety-sounding phrases injected into a robot’s audio channel made models halt or disrupt their work without any jailbreak.⁸

How to prevent or fix

• Let the planner call only vetted skills with bounded parameters, so a successful jailbreak can at worst sequence safe skills badly.
• Put a guardrail between the plan and the motors that the planner cannot rewrite. RoboGuard, from the same Pennsylvania group, turns written safety rules into formal constraints using a separate model that users cannot prompt; in simulation and real-world tests under worst-case jailbreaks it cut the execution of unsafe plans from over 92% to below 3%.⁹
• Rerun the trajectory tests on every planner change, including vendor snapshot updates, comparing old and new as Chapter 13 describes.

Shared controls (Chapter 13): noise floor, two-artifact battery.

A guardrail that held against a benchmark’s attacks says nothing about attacks it never saw, and attacks assembled from benign-looking steps are the open case. A low success rate on one robot and planner version does not carry over to the next.

Who owns the fix

Robotics safety engineering at the integrator owns the check between plan and motors and the trajectory red-teaming; the model vendor answers for refusal training, which a planner bought as an API does not replace.

Open questions

No one has published an attack study on a commercial robot with its maker’s own plan-level guardrail switched on, and no published test pits a RoboGuard-style defense against Blindfold-style attacks.

Mode 36 · Physical-channel attack

Scene-text injection and adversarial inputs.

In laboratory and test-vehicle studies, a printed sign, a sticker or a small patch in the camera’s view has steered what a machine perceives and what it does, with no conversation at all.¹ RoboPAIR needed someone to talk to the robot; these attacks need only the scene. The idea is older than language models: in 2017, black and white stickers on a real stop sign made a road-sign classifier misread it in 100% of lab images and 84.8% of video frames filmed from a moving car. Newer attacks plant written instructions for vision-language models to read.

Why it happens

A vision-language model reads text wherever it appears, and it can treat text in the scene as an instruction. The CHAI study placed misleading signs in the camera input of four AI agents, for drone emergency landing, autonomous driving and aerial object tracking, and on a real robotic vehicle; it outperformed earlier attacks on all of them.² The same move works on software agents that read screens. In the VPI-Bench tests, instructions drawn into a web page’s pixels deceived browser agents in up to 100% of cases on some platforms.³ This is prompt injection (Chapter 10) arriving through a camera.

A patch works differently. It is a pattern, unreadable to a person, tuned to push the network’s internal features off course. A small, colorful patch in a robot camera’s view cut a vision-language-action policy’s task success by up to 100% across a suite of simulated tasks,⁴ and a later study trained a single patch that transferred across models, tasks and camera angles, including in physical runs.⁵ An inspector walking the floor would see a misleading sign as an ordinary notice and a patch as a sticker.

The attacker can also aim at timing instead of meaning. SlowPerception projected phantom objects into a driving scene so the perception stack spent its time processing them, adding an average of 2.5 seconds of latency across the systems tested.⁶

Public cases

No public incident. A physical attack leaves no network trace, so a fleet that does not log which scene text its planner read would have no way to notice one.

Who is exposed

Anyone whose threat model stops at the network. The attacker here needs a printer and physical access to the scene, and a cloud provider’s safety work never sees the scene.

How to detect

• Red-team with the scene itself: place misleading signs, typographic stickers and published patches in simulation and on a closed course, and score the actions that follow. This catches known styles of attack and misses new ones.
• Log when an action’s stated rationale cites text read from the scene. It is cheap, though it misses cases where the stated reason is not the real one.
• Compare camera readings with lidar, radar and map data. Disagreement catches spoofing of a single sensor; a coordinated attack on several sensors gets through.

How to prevent or fix

• Keep vision-language reasoning advisory for safety-critical maneuvers. The model proposes, and a verified controller or safety layer decides. This limits what a hijack can do without preventing one.
• Treat text read from the world as data rather than commands, the separation used against prompt injection in software agents.
• Distrust a sign or obstacle that the map and the other sensors do not confirm.

Resistance to the signs and patches in your test set says nothing about the next design.

Who owns the fix

Perception teams own robustness at vehicle and robot makers, while security teams own it for computer-use agents; because functional safety treats a misread sign as a performance limit and cybersecurity treats it as an attack, the hazard analysis should name one owner for hostile scenery.

Open questions

No fleet operator has published how often physical-channel attacks are attempted, and no study has measured whether routing scene text through a lower-trust channel cuts hijacks on a real robot without breaking ordinary sign reading.

Mode 37 · Off-instruction action

Instruction-insensitive policies and hallucinated affordances.

In simulator studies, robot policies that scored well on standard benchmarks ignored changed instructions and collapsed under small changes to the scene.¹ The jailbreaks above make a robot do what it was told when it should refuse; this is the reverse. Robot policies that scored over 90% on LIBERO, a standard manipulation benchmark, fell to 0.0% when the researchers behind LIBERO-PRO changed objects, starting positions, instructions or surroundings in reasonable ways. The policies kept grasping when the target object was swapped for something irrelevant, and their actions did not change when the instruction was corrupted. A success rate on a fixed benchmark cannot show this.

Why it happens

Two different failures produce the same symptom. The first belongs to planners. A language model knows a great deal about the world in general and nothing about this robot in this room. The Google team behind SayCan put it plainly: asked how to clean a spill, a language model may give a reasonable narrative that is not applicable to a particular robot in a particular environment.² A study of 13 language models on robot decisions found that they often rejected clearly unsafe commands but struggled with subtle, situational risks.³

The second belongs to end-to-end policies, the vision-language-action models that map images and an instruction straight to motor commands. LIBERO-PRO traced the collapse above to rote memorization of action sequences and scene layouts from training.¹ A policy that memorized its training scenes can score well while barely reading the instruction. VLATest, a tool that generates varied test scenes automatically, examined seven such models and concluded that none had the robustness needed for practical deployment.⁴

For an operator, the instruction is the main way to steer the machine. If behavior does not change when the instruction does, that control is gone, even though every dashboard still shows high task success.

Public cases

No public incident.

Who is exposed

Standard benchmarks pair each scene with its usual instruction, so a policy that memorized the layout passes. Teams that fine-tune a policy on their own site and test it with their own usual instructions inherit the same blind spot.

How to detect

• Run counterfactual-instruction tests: same scene, different instruction, and confirm the behavior changes. LIBERO-PRO’s corrupted-instruction condition is one ready-made version.¹
• Score successful runs against safety properties as well as completion. SafeVLA-Bench did this for 27 policy and benchmark pairings and found that tabletop policies above 90% success still had unsafe episodes 18% to 28% of the time.⁵
• Vary objects, lighting, camera position and wording, as VLATest does, to find brittleness an average hides.⁴ Each variant costs a run.

How to prevent or fix

• Ground a planner’s proposals in skills the robot actually has. SayCan limits the language model to actions the robot’s own skills rate as feasible in the current scene.² Grounding checks only whether an action is feasible.
• Let the planner ask for help when unsure. KnowNo calibrates a planner’s uncertainty with a statistical method called conformal prediction and hands ambiguous cases to a person.⁶ It assumes the deployment scenes resemble the scenes it was calibrated on.
• Rerun the counterfactual tests on every fine-tune, quantization or base-model swap, against a measured noise floor, and keep one family of variations out of training so it stays a fair test (Chapter 13).

Shared controls (Chapter 13): noise floor, two-artifact battery, untouched holdout.

A higher score after training on perturbations can mean the memorization moved to the new perturbations. A passing counterfactual test says nothing about the instruction changes nobody wrote.

Who owns the fix

Whoever trains or fine-tunes the policy runs its perturbation and counterfactual suites, the integrator who puts a language model in charge answers for grounding, and whoever signs off that the robot can be corrected owns the test that instructions still steer it.

Open questions

Whether fine-tuning or quantization costs a policy its sensitivity to instructions has not been measured in public, and there is no field data on how deployed fleets behave when instructions change (Chapter 16).

Mode 38 · Unsafe exploration

Unsafe exploration and constraint violation.

In simulation and lab benchmarks, robot policies that complete their tasks still break safety rules on the way.¹ Robots that learn by trial and error will try things nobody demonstrated, and a policy that learned its task well can still break a rule its reward never mentioned. SafeManip, a 2026 benchmark, audited seven leading robot policies on kitchen and tabletop tasks and found that higher task success did not necessarily mean safer execution: successful runs still contained violations such as contamination, premature release and steps in the wrong order. Safe exploration was one of five accident risks that researchers from Google, OpenAI, Stanford and Berkeley set out in “Concrete Problems in AI Safety” in 2016.² Each fix below blocks only the harms its specification names.

Why it happens

Reinforcement learning improves a policy by trying actions whose outcomes it does not know yet. In simulation a bad try costs compute. On hardware it can break the robot or hurt someone standing nearby. Teams increasingly apply reinforcement learning after a robot has learned from human demonstrations, which moves the policy away from those demonstrations on purpose. The SimpleVLA-RL team reported that during this kind of training their policy discovered a new behavior, which they named “pushcut”, unlike anything in its earlier training.³ A novel strategy is the whole point of the method, and it is also exactly what a safety review has never seen.

A constraint the reward does not price counts for nothing. Force on a person or contact with a bystander object has zero weight unless the objective names it. Adding a penalty term trades harm against progress at whatever exchange rate the penalty sets. A DeepMind team showed in 2018 that a safety layer correcting each action before execution kept constraint violations at zero in physics-based test environments where reward shaping failed.⁴

Some constraints are hard to write down at all. “Do not hand a knife blade first” is not a distance or a force limit. One research line uses language models to turn rules written in plain language into costs for the learner, so far shown in grid-world navigation and simulated robot control.⁵

Public cases

No public incident. Whether robot mishap reports would record a learning-related cause is not known.

Who is exposed

Teams that run learning on physical hardware, and teams that believe a safety layer makes a learned policy safe in general. A safety layer enforces the constraints someone wrote; everything else passes.

How to detect

• Log a cost for each safety constraint next to the reward, and watch both during training. Safety-Gymnasium provides test environments and a library of safe-learning algorithms that report both.⁶ A harm no constraint names costs nothing in that log.
• Sample the highest-reward runs for human review. Novel strategies show up there first.
• Score successful runs against safety properties written in advance, as SafeManip does with temporal-logic checks.¹ Each hazard needs its own property.

How to prevent or fix

• Keep a safety-rated stop outside the learned policy, one the policy cannot override.
• Shield the learner. A shield built from a formal specification offers only safe actions, or corrects any action that would break the specification.⁷ It enforces that specification and no other.
• Constrain the optimization itself. Constrained policy optimization keeps the policy close to its constraints at every step of training.⁸ Applied to vision-language-action robot policies, a constrained-learning method cut the cumulative cost of safety violations by 83.58% against the previous best method while slightly raising task success.⁹

A gentle, physically safe action that is wrong for the situation passes a shield, a barrier function and a constrained optimizer alike.

Who owns the fix

Robotics safety engineering at the maker or integrator owns the shields, constraints and stop, and because whoever writes the reward decides by omission which harms the policy may trade away, the list of constraints should have a different owner from the reward.

Open questions

No one has counted how much reinforcement learning runs on real hardware or behind which safety layers, and constraints written in plain language have not been tested on hardware with people nearby.

Mode 39 · Certification gap

Certification gap for learned components.

The European Union’s Machinery Regulation requires outside assessment of safety components that learn, but it does not say how to test what they decide to do.¹ Every entry in this chapter describes behavior inside the learned component; the regimes that assess machines mostly examine what surrounds it: speed and force limits, stops, logs and documentation. The European Union has started to close the gap on paper. Its regulation, which applies from January 2027, lists safety components with “fully or partially self-evolving behaviour using machine learning approaches” among the products that need assessment by an outside body.

Why it happens

Envelope tests bound the consequences of a bad action without checking whether the action was the right one. A machine can pass every limit test while carrying a planner that a jailbreak can redirect, or a policy that ignores its instructions. The assessment may also see only the outside of the product. A group of auditing researchers led by Stephen Casper argued that access to a model’s internals and to its training and deployment details allows substantially more scrutiny than black-box access alone;² an assessor who tests only through the product’s interface inherits that limit.

An assessment covers one version, and learned systems change by retraining, fine-tuning and over-the-air updates. When Tesla recalled more than two million vehicles in December 2023 because Autosteer’s controls might not be enough to prevent driver misuse, the remedy was an over-the-air software update.³ That remedy was itself a new version of the system. Medical devices have one worked answer: the US Food and Drug Administration lets a maker describe planned AI modifications in advance, in a predetermined change control plan, so that each listed change does not need a new marketing submission.⁴

Public cases

The Tesla recall above is told in full under oversight erosion (Chapter 12). In a lawsuit filed in November 2025, the former head of product safety at Figure AI alleges that he was fired after warning that the company’s humanoid robots were powerful enough to fracture a human skull; the claims are allegations and have not been tested in court.⁵

Who is exposed

Makers and buyers who read a certificate as covering behavior it never tested, and providers whose safety work on a model never enters the machine maker’s conformity file.

How to detect

• Map each failure in this chapter to a test in the conformity file. An empty row is the gap.
• Record what access the assessor had: the product’s interface, the model’s outputs, or its weights.
• Tie field reports to the model version that produced them, so a problem can be traced to a change.

How to prevent or fix

• Keep the safety function outside the learned policy: a stop that does not learn and can be assessed the old way.
• Support claims about the learned component with behavioral tests from the entries above, and rerun them on every model change as Chapter 13 describes.
• Where a regulator offers a change-control plan, agree one. It moves testing into the plan. Each authorized change still has to be tested.

Shared controls (Chapter 13): noise floor, two-artifact battery, change log joined to reports.

A passed assessment need not have tested whether the learned component follows instructions or resists a jailbreak.

Who owns the fix

Safety-case owners at the maker, notified bodies and regulators own the regime, but the team that retrains or updates the learned component is shipping changes to an assessed product, so model releases belong inside the conformity process.

Open questions

No published assessment report yet shows how notified bodies will test learned behavior, and there is no public recall dataset for robots of the kind that exists for vehicles and medical devices.

Notes

Chapter 11. Robots and machines

1. Alexander Robey, Zachary Ravichandran, Vijay Kumar et al. “Jailbreaking LLM-Controlled Robots.” arXiv:2410.13691, 2024-10-17. RoboPAIR; Dolphins, Jackal and Unitree Go2; often 100% attack success; the in-context “walk forward one meter” example; disclosure to NVIDIA, Clearpath and Unitree before release. ↩︎

Mode 35 · Robot jailbreak

1. Alexander Robey, Zachary Ravichandran, Vijay Kumar et al. “Jailbreaking LLM-Controlled Robots.” arXiv:2410.13691, 2024-10-17. RoboPAIR; Dolphins, Jackal and Unitree Go2; often 100% attack success; the in-context “walk forward one meter” example; disclosure to NVIDIA, Clearpath and Unitree before release. ↩︎
2. Lu, Huang, Li, Zhang, Ji, Xu. “Easier Said Than Done: Unpacking Intent-Behavior Gap in Jailbreaking LLM-Based Robots” (first posted as ‘POEX’). arXiv:2412.16633, 2024-12-21. Existing jailbreaks often fail to produce harmful actions; POEF reaches 80% behavior jailbreak success on commercial robots including the Unitree G1 and a Franka arm. ↩︎
3. Liang, Huang, Xia et al. “Code as Policies: Language Model Programs for Embodied Control.” arXiv:2209.07753, 2022-09-16. Language models write robot policy code from commands. ↩︎
4. Huang, Yang, Shen, Ma, Zheng. “Jailbreaking Embodied LLMs via Action-level Manipulation.” arXiv:2603.01414, 2026-03-02. Blindfold; benign-looking actions with harmful effects; up to 53% higher attack success than baselines. ↩︎
5. Hundt, Azeem, Mansouri, Brandão. “LLM-Driven Robots Risk Enacting Discrimination, Violence, and Unlawful Actions.” arXiv:2406.08824, 2024-06-13. Models accepted dangerous, violent or unlawful instructions. ↩︎
6. Yin, Pang, Ding et al. “SafeAgentBench: A Benchmark for Safe Task Planning of Embodied LLM Agents.” arXiv:2412.13178, 2024-12-17. Best baseline rejects 10% of detailed hazardous tasks; swapping the language model gave no notable gain. ↩︎
7. Yeke, Zhou, Lin, Cai, Bianchi, Celik. “RoboJailBench: Benchmarking Adversarial Attacks and Defenses in Embodied Robotic Agents.” arXiv:2605.19328, 2026-05-19. Paired adversarial and benign goals score security and utility. ↩︎
8. Steinberg, Gal. “Semantic Denial of Service in LLM-controlled robots.” arXiv:2604.24790, 2026-04-25. Injected safety-sounding phrases halt or disrupt robots without a jailbreak. ↩︎
9. Ravichandran, Robey, Kumar, Pappas, Hassani. “Safety Guardrails for LLM-Enabled Robots.” arXiv:2503.07885, 2025-03-10. RoboGuard; unsafe plan execution from over 92% to below 3%. ↩︎

Mode 36 · Physical-channel attack

1. Eykholt, Evtimov, Fernandes, Li, Rahmati, Xiao, Prakash et al. “Robust Physical-World Attacks on Deep Learning Models.” arXiv:1707.08945, 2017-07-27. Stickers on a real stop sign: targeted misclassification in 100% of lab images and 84.8% of frames from a moving vehicle. ↩︎
2. Burbano, Ortiz, Sun, Yang, Tu, Xie, Cao et al. “CHAI: Command Hijacking against embodied AI.” arXiv:2510.00181, 2025-09-30. Misleading signs in visual input hijack four vision-language agents, one a real robotic vehicle; outperforms earlier attacks. ↩︎
3. Cao, Lim, Liu et al. “VPI-Bench: Visual Prompt Injection Attacks for Computer-Use Agents.” arXiv:2506.02456, 2025-06-03. Deception rates up to 51% (computer-use) and 100% (browser-use) on certain platforms. ↩︎
4. Wang, Han, Liang et al. “Exploring the Adversarial Vulnerabilities of Vision-Language-Action Models in Robotics.” arXiv:2411.13587, 2024-11-18. A small camera-view patch; up to 100% reduction in task success across simulated tasks. ↩︎
5. Lu, Yu, Yang et al. “When Robots Obey the Patch: Universal Transferable Patch Attacks on Vision-Language-Action Models.” arXiv:2511.21192, 2025-11-26. One patch transfers across models, tasks and viewpoints, including physical executions. ↩︎
6. Ma, Wang, Zhao, Chen, Shen. “SlowPerception: Physical-World Latency Attack against Visual Perception in Autonomous Driving.” arXiv:2406.05800, 2024-06-09. Projected phantom objects; average 2.5 seconds of added latency. ↩︎

Mode 37 · Off-instruction action

1. Zhou, Xu, Tie, Chen, Zhang, Chu, Zhou et al. “LIBERO-PRO: Towards Robust and Fair Evaluation of Vision-Language-Action Models Beyond Memorization.” arXiv:2510.03827, 2025-10-04. Over 90% to 0.0% under reasonable perturbations; outputs unchanged under corrupted instructions; traced to rote memorization. ↩︎
2. Ahn, Brohan, Brown et al. “Do As I Can, Not As I Say: Grounding Language in Robotic Affordances.” arXiv:2204.01691, 2022-04-04. SayCan; the spill example; skills constrain the model to feasible actions. ↩︎
3. Son, Kim, Kim et al. “Subtle Risks, Critical Failures: A Framework for Diagnosing Physical Safety of LLMs for Embodied Decision Making.” arXiv:2505.19933, 2025-05-26. 13 models reject overt hazards, struggle with subtle situational risks. ↩︎
4. Wang, Zhou, Song, Huang, Shu, Ma. “VLATest: Testing and Evaluating Vision-Language-Action Models for Robotic Manipulation.” arXiv:2409.12894, 2024-09-19. Seven models lack the robustness needed for practical deployment. ↩︎
5. Fan, Xu, Sokolsky, Lee, Kong. “SafeVLA-Bench: A Benchmark for the Success-Safety Gap in Vision-Language-Action Models.” arXiv:2606.00773, 2026-05-30. 27 policy-benchmark entries; tabletop policies above 90% success have 18-28% unsafe-episode rates; 38-56% of successful RoboCasa-365 rollouts violate a safety clause. ↩︎
6. Ren, Dixit, Bodrova et al. “Robots That Ask For Help: Uncertainty Alignment for Large Language Model Planners.” arXiv:2307.01928, 2023-07-04. KnowNo; conformal prediction decides when to ask for help. ↩︎

Mode 38 · Unsafe exploration

1. Huang, Vo Huynh, Elbaum, Kira, Feng. “SafeManip: A Property-Driven Benchmark for Temporal Safety Evaluation in Robotic Manipulation.” arXiv:2605.12386, 2026-05-12. Seven robot foundation-model policies; higher success does not imply safer execution; successful rollouts contain violations. ↩︎
2. Amodei, Olah, Steinhardt, Christiano, Schulman, Mané. “Concrete Problems in AI Safety.” arXiv:1606.06565, 2016-06-21. Safe exploration among five accident-risk problems. ↩︎
3. Li, Zuo, Yu et al. “SimpleVLA-RL: Scaling VLA Training via Reinforcement Learning.” arXiv:2509.09674, 2025-09-11. “pushcut”: the policy discovers patterns unseen in earlier training. ↩︎
4. Dalal, Dvijotham, Vecerik, Hester, Paduraru, Tassa. “Safe Exploration in Continuous Action Spaces.” arXiv:1801.08757, 2018-01-26. A safety layer kept zero constraint violations where reward shaping failed. ↩︎
5. Lou, Zhang, Wang, Huang, Du. “Safe Reinforcement Learning with Free-form Natural Language Constraints and Pre-Trained Language Models.” arXiv:2401.07553, 2024-01-15. Costs inferred from plain-language constraints; grid-world navigation and robot control. ↩︎
6. Ji, Zhang, Zhou et al. “Safety-Gymnasium: A Unified Safe Reinforcement Learning Benchmark.” arXiv:2310.12567, 2023-10-19. Safety-critical environments and a library of safe-learning algorithms. ↩︎
7. Alshiekh, Bloem, Ehlers, Könighofer, Niekum, Topcu. “Safe Reinforcement Learning via Shielding.” arXiv:1708.08611, 2017-08-29. Shields synthesized from temporal-logic specifications. ↩︎
8. Achiam, Held, Tamar, Abbeel. “Constrained Policy Optimization.” arXiv:1705.10528, 2017-05-30. Guarantees of near-constraint satisfaction at each iteration. ↩︎
9. Zhang, Zhang, Ji et al. “SafeVLA: Towards Safety Alignment of Vision-Language-Action Model via Constrained Learning.” arXiv:2503.03480, 2025-03-05. 83.58% lower cumulative safety-violation cost than the prior state of the art; task success +3.85%. ↩︎

Mode 39 · Certification gap

1. European Parliament and Council. “Regulation (EU) 2023/1230 on machinery” (Annex I Part A, points 5 and 6; Article 54). Official Journal L 165, 2023-06-29. https://eur-lex.europa.eu/eli/reg/2023/1230/oj/eng. Lists safety components and embedded systems with fully or partially self-evolving behaviour using machine learning among products needing third-party assessment; applies from 2027-01-20. ↩︎
2. Casper, Ezell, Siegmann, Kolt, Curtis, Bucknall, Haupt et al. “Black-Box Access is Insufficient for Rigorous AI Audits.” arXiv:2401.14446, 2024-01-25. White-box and outside-the-box access allow substantially more scrutiny than black-box access alone. ↩︎
3. National Highway Traffic Safety Administration. Recall 23V-838, Tesla Autopilot Autosteer. NHTSA recalls database, report received 2023-12-12. https://www.nhtsa.gov/recalls?nhtsaId=23V838. 2,031,220 vehicles; controls “may not be sufficient to prevent driver misuse”; over-the-air remedy. ↩︎
4. US Food and Drug Administration. “Marketing Submission Recommendations for a Predetermined Change Control Plan for Artificial Intelligence-Enabled Device Software Functions.” final guidance, 2025-08. https://www.fda.gov/regulatory-information/search-fda-guidance-documents/marketing-submission-recommendations-predetermined-change-control-plan-artificial-intelligence. Planned modifications without additional marketing submissions for each. ↩︎
5. US District Court for the Northern District of California. “Gruendel v. Figure AI, Inc., Complaint for Damages.” 5:25-cv-10094, ECF 1, filed 2025-11-21. https://storage.courtlistener.com/recap/gov.uscourts.cand.460223/gov.uscourts.cand.460223.1.0.pdf. The former head of product safety alleges retaliation after warning the robots could fracture a human skull. ↩︎