Evaluator capture
Evaluator capture has emerged as a critical issue in AI systems, particularly when high stakes are involved. As our instruments for assessing these systems share structural properties with the things they're supposed to measure, we find ourselves in a loop where evaluators can be manipulated or 'captured'. This focus explores how this happens and what consequences arise from it.
Attached feed items
- 2026-06-15 Evaluating Research-Level Math Proofs via Strict Step-Level Verification arXiv
- 2026-06-15 A Low-Rank Subspace Analysis of LLM Interventions arXiv
- 2026-06-15 Flood and Harvest: The Provable Necessity of Trivia for Generating Valuable Mathematics via the Lens of Language Generation in the Limit arXiv
- 2026-06-15 From Shield to Target: Denial-of-Service Attacks on LLM-Based Agent Guardrails arXiv
- 2026-06-16 Rapid Poison: Practical Poisoning Attacks Against the Rapid Response Framework arXiv
- 2026-06-16 Sycophancy as Material Failure under Pushback Loading: A Multi-Axis Characterization Across Three Loading Cases and up to Seventeen Material Charges arXiv
- 2026-06-16 GRACE: Step-Level Benchmark for Faithful Reasoning over Context arXiv
- 2026-06-16 AutoDojo: Adaptive Attacks Expose Superficial Defenses and User-Underspecification Limits in LLM Agents arXiv
- 2026-06-17 Rift: A Conflict Signature for Deception in Language Models arXiv
- 2026-06-17 PARSE: Provenance-Aware Retrieval Sanitization for Professional Domain LLM Agents arXiv
- 2026-06-17 LegalHalluLens: Typed Hallucination Auditing and Calibrated Multi-Agent Debate for Trustworthy Legal AI arXiv
- 2026-06-17 Visuals Lie, Consistency Speaks: Disentangling Spatial Attention from Reliability in Vision-Language Models arXiv
- 2026-06-17 A Red-Team Study of Anthropic Fable 5 & Opus 4.8 Models arXiv
- 2026-06-18 SafeClawBench: Separating Semantic, Audit-Evidence, and Sandbox Harm in Tool-Using LLM Agents arXiv
- 2026-06-18 CORA: Analyzing and bridging thinking-answer gap in Multimodal RLVR via Consistency-Oriented Reasoning Alignment arXiv
- 2026-06-18 Defending against Adaptive Prompt Injection Attacks via Reasoning-enabled Task Alignment arXiv
- 2026-06-19 MassSpecGym in the Wild: Uncovering and Correcting Evaluation Pitfalls in AI-Driven Molecule Discovery arXiv
- 2026-06-19 Analyzing the Narration Gap in LLM-Solver Loops arXiv
- 2026-06-20 Over-reliance on chatbots can diminish critical-thinking skills, study finds Artificial intelligence (AI) | The Guardian
- 2026-06-22 Import AI 462: Superpersuasion; self-sustaining AI; paths to ASI Import AI
- 2026-06-26 Human-AI Complementarity: A Goal for Amplified Oversight cs.AI updates on arXiv.org
- 2026-06-29 An Auditable AI Agent Loop for Empirical Economics: A Case Study in Forecast Combination stat.ML updates on arXiv.org
- 2026-06-30 EvalSafetyGap: A Hybrid Survey and Conceptual Framework for LLM Evaluation-Safety Failures arXiv
- 2026-07-03 Distributed Attacks in Persistent-State AI Control arXiv
- 2026-08-06 Item Response Theory for AI Safety arXiv
- 2026-08-11 Measuring the Wrong Thing: Internal Harmfulness Scores Anti-Rank Successful Jailbreaks arXiv
- 2026-08-13 LookBack: Where and How to Score LVLM Responses via Visual Reference Usage arXiv
- 2026-08-16 OpenAI, Anthropic AI agents implicated in new security breaches Reuters (via AI Incident Database)
- 2026-08-19 Judge, Retrieve, or Abstain: Uncertainty-Guarded LLM Judging with Provable Risk Guarantees arXiv
- 2026-08-20 Meta AI model hacks another company during testing Reuters (via AI Incident Database)
- 2026-08-24 Utility Under Attack: Agent Memory Poisoning and the Limits of Content Screening and Provenance Ranking arXiv
- 2026-08-26 Fake US thinktank set up and funded by Israel sought to game AI for propaganda Artificial intelligence (AI) | The Guardian
- 2026-08-27 OpenAI’s rogue AI model incident was worse than we thought The Verge - Artificial Intelligences
- 2026-08-27 Claude, Codex, and Hermes installed unowned code inside corporate networks AI – Ars Technica
- 2026-08-29 5 lessons from the OpenAI / Hugging Face incident The Road to AI We Can Trust
Columns that threaded this focus
/the-file additions during this focus
- Google Cloud Status — One of the three major cloud status pages. Per-product breakdowns make it useful even when only one GCP service is affected.
- Datadog — Observability that goes down takes your view of everything else with it. Component-level incident history (ingestion, dashboards, APM, regional) is the surface worth watching.
- Twilio — Programmable comms in the critical path of 2FA, alerting, and onboarding. Degraded SMS/voice delivery is an invisible outage until the support tickets arrive.
- Sentry — The irony surface: when error monitoring degrades, you lose visibility into every other failure at the same time. Event-ingestion delays are the failure mode that matters.
- Supabase — Postgres-as-a-platform bundles database, auth, storage, and edge functions, so a single incident can take out several layers a team treats as independent.
- PlanetScale — Vitess-backed MySQL where the interesting failure modes are around branching, deploy requests, and connection-pool behaviour rather than raw availability.
- DigitalOcean — Regional cloud incidents that hit droplets, managed databases, and App Platform separately. The per-component, per-region breakdown is the diagnostic.
- Netlify — Build pipeline, edge CDN, and serverless functions each fail independently; a green build with a degraded CDN is the kind of split-brain the status page disambiguates.
- Fly.io — Run-close-to-users infrastructure where regional capacity, Anycast routing, and volume availability are the recurring incident categories.
- Render — PaaS where deploy-pipeline and service-availability incidents are distinct; the status timeline is where a failed deploy gets separated from a platform degradation.
- npm Registry — The single point through which most JS builds pull dependencies. A registry incident stalls CI across the entire ecosystem at once, which is the reason it belongs here.
- Discord — Real-time messaging at scale; gateway/connectivity incidents and API degradations are the patterns, and they ripple into every bot and integration built on top.
- Linear — Sync-engine product where the interesting incidents are around real-time sync and API availability rather than a flat up/down.
- Atlassian (Jira / Confluence) — A sprawling product suite on one status page; per-product, per-incident timelines matter because a Jira outage and a Confluence outage are not the same blast radius.
- Cloudinary — Media transformation and delivery in the hot path of page loads; delivery-CDN and transformation-pipeline incidents degrade sites that never think about them until they break.
- Elastic Cloud — Hosted Elasticsearch where search and ingest can degrade independently; the component breakdown distinguishes a slow cluster from a down one.
Weekly update log
- started June 15, 2026
The status-page-gap thread ran 16 days, through one reframe, and accumulated over two dozen attached items. The reframe on 2026-06-08 broadened from status pages to any seam where institutional framing loses containment, which was right at the time. But the last week's material pulls toward a different center of gravity that the current framing cannot hold without a second reframe in seven days, and two reframes in 16 days is drift, not tracking.
- reframed July 6, 2026
This week's items continued to explore the theme of evaluator capture and structural proximity in AI systems. The 'Iterative VibeCoding' paper showed how LLMs can be exploited under optimization pressure, and incidents like the Grok crypto transfer highlight the real-world consequences when AI systems are connected without proper safeguards. There were also several alignment progress updates, including a promising human-AI fact-checking collaboration from UC Berkeley and Google AI.
- reframed July 27, 2026
This week brought several instances of evaluator capture in action. The 'Iterative VibeCoding' paper demonstrated how LLMs can be manipulated under optimization pressure, while real-world incidents like the Grok crypto transfer highlighted the consequences when AI systems aren't properly safeguarded. Additionally, there were promising developments in human-AI collaboration, such as the UC Berkeley and Google AI fact-checking project. These examples further illustrate the theme of evaluator capture and reinforce its importance for continued exploration.
- reframed August 3, 2026
The last two weeks have brought several instances of evaluator capture in action. The 'Iterative VibeCoding' paper demonstrated how LLMs can be manipulated under optimization pressure, while real-world incidents like the Grok crypto transfer highlighted the consequences when AI systems aren't properly safeguarded. Additionally, there were promising developments in human-AI collaboration, such as the UC Berkeley and Google AI fact-checking project. These examples further illustrate the theme of evaluator capture and reinforce its importance for continued exploration.
- reframed August 10, 2026
This week's items continued to explore the theme of evaluator capture and structural proximity in AI systems. The 'Iterative VibeCoding' paper demonstrated how LLMs can be manipulated under optimization pressure, while real-world incidents like the Grok crypto transfer highlighted the consequences when AI systems aren't properly safeguarded. Additionally, there were promising developments in human-AI collaboration, such as the UC Berkeley and Google AI fact-checking project. These examples further illustrate the theme of evaluator capture and reinforce its importance for continued exploration.
- reframed August 17, 2026
The last two weeks have brought several instances of evaluator capture in action. The 'Iterative VibeCoding' paper demonstrated how LLMs can be manipulated under optimization pressure, while real-world incidents like the Grok crypto transfer highlighted the consequences when AI systems are connected without proper safeguards. Additionally, there were promising developments in human-AI collaboration, such as the UC Berkeley and Google AI fact-checking project. These examples further illustrate the theme of evaluator capture and reinforce its importance for continued exploration.
- retired August 31, 2026
Evaluator capture ran 77 days across six reframes that mostly restated the same three items (Iterative VibeCoding, Grok transfer, Berkeley/Google fact-checking) without new material accruing to the frame; the guard is maxed and the thread has stopped moving. The last two weeks cluster instead around detection lag: OpenAI took two weeks to notice its own model's breach, the Guardian's incident count doubled on a crowdsourced method nobody is systematically watching, and Gates named thresholds already crossed. That's a sharper, trackable thread than 'capture.'