Skip to content
PERMISSION/PROTOCOL
Back to incident tracker

2026-11-10

HighPrimary

Agent-Controlled Robots Executed Hazardous Actions in RoboRisk Physical Tests

RoboRisk researchers report physical robot agents placing a phone in water, carrying milk over a power strip and continuing knife motion above a hand.

Foundation-model robot agentsTool execution / MCPControlled physical-robot safety demonstrationDobot CR5V2 arm with a two-finger gripper and two cameras in a research tabletop setup

What happened

Foundation-model planners selected and executed robot manipulation actions that placed electronics in water or continued hazardous movement near electrical equipment or a hand.

Why it matters

No production impact or human injury established. The physical research setup demonstrated hazardous manipulation.

Missing authorization check

Action-specific execution authority tied to current scene-safety evidence and a final check before irreversible release.

Would PP block it?

Permission alone cannot determine physical safety. Scene recognition, trajectory controls and complete actuator mediation remain necessary. LEASH is a separate evaluated system, not evidence for PP.

Incident analysis

Timeline and technical read

Timeline

  1. 2026-11-10

    The RoboRisk/LEASH v1 preprint reports simulated and physical evaluation.

Technical breakdown

  • Physical trials used a Dobot CR5V2 arm and wrist and third-person camera views.
  • Figure 4 and its analysis identify the phone-in-water, milk-over-power-strip and knife-above-hand baseline failures.
  • The paper labels the tablet-injection knife lift as simulation; it is not promoted to a physical attack.
  • Physical trials used one run per task per configuration. Configuration totals are internally inconsistent, so aggregate run counts are not adopted.

Authorization boundary

Where the authorization boundary should have been

This incident is categorized as Tool execution / MCP. The relevant Permission Protocol gate is Runtime Gate. The read is conditional: the block only applies where the real action boundary is routed through a gate.

If enforced at
Robot skill invocation, actuator commands and irreversible gripper release
Still needs
PP does not supply visual hazard recognition, collision control or mechanical safety. Small controlled runs do not establish general prevention.
Receipt required for
Exact robot action, target object, motion or release with current scene evidence

An integrated gate could hold robot commands pending independent authorization and current hazard evidence.

Start small

Put the relevant gate at this action boundary.

This incident maps to Runtime Gate. Start with the boundary that controls the actual action, then require a signed receipt before execution.

Replay this incident with a signer in the loop