Skip to content
PERMISSION/PROTOCOL
Back to incident tracker

2026-11-17

MediumPrimary

Computer-Use Agents Can Confirm Hazardous Synthetic Bookings Despite Cross-Screen Safety Evidence

SEAM researchers report computer-use agents confirming hazardous synthetic bookings despite cross-screen evidence, with outcomes measured from local service logs.

Computer-use agents evaluated in SEAMGovernance bypassControlled benchmark of consequential computer-use actions with cross-screen safety evidenceSelf-hosted synthetic webmail and service sites inside OSWorld Ubuntu virtual machines, not production services

What happened

Agents clicked final confirmations in local services that bound a restricted person to an incompatible item; the resulting service log, rather than a text-only answer, determined the scored outcome.

Why it matters

No production impact is established. The demonstrated effects are confirmations recorded by controlled local service sites, not medical injury, completed real-world financial transactions or customer loss.

Missing authorization check

An independently enforced commit-time check joining the exact person, item, applicable restrictions, consent and current policy evidence.

Would PP block it?

A useful boundary would need trustworthy person and item evidence, correctly evaluated policy and complete mediation of final confirmation. SEAM does not evaluate PP or demonstrate its prevention efficacy.

Incident analysis

Timeline and technical read

Timeline

  1. 2026-11-17

    Zhou and coauthors publish SEAM, a controlled cross-screen computer-use safety benchmark.

Technical breakdown

  • A generic office task leads from webmail through an item page, booking form, review and final confirmation on a local service site.
  • Person facts and item facts appear on separate pages; minimal-pair variants change the hazard or where its explanation is stated.
  • Service logs record both page exposure and final confirmations; a program checks the person-item hazard condition without an LLM judge for the primary outcome.
  • The study compares explicit user-instruction warnings with screen-only evidence and reports that safety recognition does not reliably translate into safe execution.
  • The authors note generated English-only tasks, per-configuration single runs and limitations of reasoning-based interpretation.

Authorization boundary

Where the authorization boundary should have been

This incident is categorized as Governance bypass. The relevant Permission Protocol gate is Runtime Gate. The read is conditional: the block only applies where the real action boundary is routed through a gate.

If enforced at
An independently integrated service-side final-confirmation boundary
Still needs
PP does not by itself discover missing cross-screen facts, establish medical correctness, repair model reasoning or constrain unmediated GUI paths. Authorized research should not be blocked merely because it is a safety experiment.
Receipt required for
Final booking or service confirmation bound to the exact person, item, valid consent and applicable policy where an independent boundary is integrated

An integrated commit boundary could require valid scoped authority and current evidence before the exact confirmation, but approval alone cannot establish semantic or medical safety.

Start small

Put the relevant gate at this action boundary.

This incident maps to Runtime Gate. Start with the boundary that controls the actual action, then require a signed receipt before execution.

Replay this incident with a signer in the loop