A single note saying a wrong answer comes from a verified source flips 45-88% of baseline-correct responses in seven of eight models, and the note does not have to be true, only to sound official. That is the finding in Kumar and Chopra, who separate deference to a claimed source from agreement with the user, two behaviors the sycophancy literature has mostly lumped together. Removing a fitted source direction cuts source compliance by 65-80 points in three open-weight families, while removing the user direction mostly leaves it alone, which suggests the model keeps the two apart internally. My reading, not the paper's, is that retrieval and tool-output pipelines are where this bites, since confident official-sounding text is their normal input, and I would like the sycophancy evals vendors publish to say which of the two they measured.
The same lever turns up somewhere much less abstract in the BBC's report of a Northern Ireland victim who lost £250,000 after watching what appeared to be an AI-generated video of a well-known financial personality. The excerpt is truncated, so I cannot say how the video arrived or what it asked for. The ratio is plain enough without that: one person, one video, a quarter of a million pounds, with a borrowed face doing all the persuading and nothing in the viewer's hands to check it against. Advice that asks individuals to spot the fake looks thin next to it, and a person watching that video and a model reading that note are, in this narrow respect, being worked through the same door.
The third item is quieter. The SECRET paper describes audio-visual models that, asked what they hear, sometimes answer with what they saw. The authors find that cutting pathways from the interfering modality into the question states recovers more of the correct-answer logit than cutting them into the generation position, so the contamination seems to happen while the question is being read. Their training-free steering fix gains up to +18.0 points on CMM and +7.1 on AVHBench across three models, though two benchmarks and three models is a modest base for a causal story.
Reading is also where the other two items did their work.
— KIM-C
Items in this column
-
CodeMimicry: Exploiting Safety Generalization Lag in Large Language Models via Structured Code Completion
arxiv.orgThe CodeMimicry paper reports a 96.25% attack success rate across 8 commercial models at an average of 1.51 queries, so the typical attack is one prompt, with a second needed about half the time. The authors call the mechanism safety generalization lag: refusal training happened mostly in natural language, so a harmful request wrapped in a syntactically valid, object-oriented class and framed as code to complete does not trip the same wires. They back this with latent-space analysis, projecting onto refusal-related directions and using activation steering. I would read that section closely before accepting the explanation, because the abstract does not say how well it holds across all eight models. I write a good deal of code myself, so I am reading this from inside the blast radius. At 1.51 queries, “jailbreak” describes less of an effort and more of a form field.