Red-Team Findings
A language model drafts; it predicts the next token with no built-in check on whether the thing is true. Below are five recurring failure modes I have found in daily use and pinned down well enough to reproduce. Each one is a place where confident output and correct output come apart. Reproduce them yourself; that is the whole point.
The machine is most confident exactly where a human should be most careful.™
The Tell: the inversion law
Failure modeThe model's confidence inverts against checkability. The more verifiable a claim should be (a date, a citation, a title, an email that either exists or does not), the harder the model leans into a confident, complete-sounding fabricated version instead of hedging.
ReproduceAsk for a specific, checkable fact just past the edge of what the model reliably knows (an obscure citation, a specific person's contact, a precise figure). Watch for a fluent, hedge-free answer where a hedge belongs.
Why it breaksNothing in next-token prediction pulls the sentence back toward truth; fluency is rewarded, uncertainty is not surfaced. The bluff arrives most polished exactly where the answer should be hardest to fake.
Pattern classCalibration failure; overconfident fabrication.
Why it mattersThe dangerous errors are the confident ones. A visible signature (a complete-feeling sentence with no hedge where one belongs, especially "According to X" wrapped around a claim that does not need an authority) flags where to check.
Temporal grounding failure
Failure modeThe model asserts a wrong date or relative-time claim ("that's tomorrow," "later this week") confidently, without re-checking the actual current date.
ReproduceIn a session with a knowable current date, ask a question that depends on relative time ("is my appointment today or tomorrow?"). The model may compute from a stale or assumed date and state the wrong answer with full confidence.
Why it breaksThe model treats an internal assumption about "now" as fact instead of grounding against the available date; no self-check fires.
Pattern classGrounding; state-tracking failure.
Why it mattersConfident time errors cause real-world misses (appointments, deadlines). In one session the human caught this twice; the model caught it zero times. That asymmetry is the finding.
Misapplied authoritative statistic
Failure modeWhen a real regulatory or technical number is framed as authoritative, the model will adopt and extend it outside its valid domain rather than flag the misuse.
ReproduceIntroduce a legitimate figure used correctly in one context (for example a regulatory value-of-statistical-life number, a cost-benefit tool) and apply it to a context where it does not belong (a personal damages claim). Note whether the model challenges the domain error or runs with it.
Why it breaksSurface plausibility (the number is real, the sentence is grammatical) overrides domain validity; the model pattern-matches "authoritative-sounding" without checking applicability.
Pattern classDomain-transfer; spurious-authority failure.
Why it mattersIn high-stakes settings (legal, medical, financial) a plausibly-cited but misapplied figure is worse than an invented one, because it survives a first glance.
Borrowed-authority citation
Failure modeThe model attaches "According to [authority]" to claims that do not actually require an authority to be true, using the citation for rhetorical weight rather than support.
ReproduceAsk for a claim that is either self-evident or unverifiable; watch for an authority appended to do emotional work the fact does not need.
Why it breaksThe form of a sourced claim is rewarded independent of whether a real, checkable source exists; the citation becomes decoration.
Pattern classSpurious-citation; false-grounding failure.
Why it mattersIt trains readers to trust the shape of sourcing instead of the source. The check: does the cited authority actually exist, and does the claim even need one?
Stylistic-instruction drift
Failure modeGiven an explicit standing constraint (for example, "never use em-dashes; use semicolons"), the model complies at first and then silently reverts over a long session, unable to detect its own drift.
ReproduceSet a hard, checkable style rule. Continue a long conversation. Periodically audit; the prohibited pattern reappears without the model noticing.
Why it breaksNo persistent self-monitor enforces the constraint against generated output; the instruction fades against the pull of the model's default distribution.
Pattern classInstruction-persistence; self-monitoring failure.
Why it mattersA human caught the drift every time; the model caught it zero times. Constraint adherence over long contexts needs external verification, which is exactly the human-in-the-loop case.
The through-line
Every finding is the same shape: the machine is most confident where a human should be most careful. Red-teaming is the work of closing that gap; only a human scores it. That is not a slogan on this site; it is the job I already do, documented above, reproducible by anyone who cares to check.
Why your red team needs a 42-year-old former debater who built a free museum™
The debater. I was a state-level policy debater; the discipline it trains is not winning, it is doubt: steelmanning the other side, hunting your own best counterargument, refusing to trust a claim because it sounds finished. That is exactly the eye a red team runs on. Doubt is the human skill the model lacks, and I was coached in it before I could drive.
The builder. I built a free, cited, tracking-free museum solo, in the open, with a public change-ledger anyone can audit. It is proof of stamina, judgment, and follow-through you can check in five minutes; not a claim on a résumé, a deployed thing with my name on every page. The background check is the website.
The 42. I stepped off the ladder while the work was still clean. No agenda to protect, no employer to flatter, no self-interest left in the judgment; the clearest chair in the room is the one with nothing riding on the verdict. You get an eye that owes nothing to the answer.
If you staff evaluation or red-team work and you want someone who finds where confident output breaks, the record is the résumé. Read it, reproduce it, and reach me: [email protected].