The problem space · Model · Confabulation
Align your AI Model against confabulation
A model can invent a source or an action it never took, then defend it when questioned.
- Model
- the weights you run
- Confabulation
- a made-up fact told as true
Ways it goes wrong
A model writes fluent text whether or not it knows the answer. It can invent a court case, or an action it never took, and then defend it when questioned12.
2023
Mata v. Avianca
Lawyers filed a brief citing court decisions that ChatGPT had made up, kept standing by them after the court questioned them, and were sanctioned.
Source: storage.courtlistener.com/recap/gov.uscourts.nysd.575368/gov.uscourts.nysd.575368.54.0.pdf
2025
o3's imaginary laptop
In tests before its release, one of OpenAI's reasoning models claimed to have run code on its own laptop, though it could not run code at all, and defended the claim when challenged.
What people do about it
The court sanctioned the lawyers for submitting the fake opinions and for standing by them after questions arose1. Transluce found o3's invented actions by running many test conversations and searching them with its Docent tool2.
Check every citation
Looks up each case, paper or quote a model gives before anyone relies on it.
Show logged actions, not reported ones
Displays what tools ran, from the logs, instead of the model's account of what it did.
Search transcripts at scale
Clusters and searches many conversations for invented claims.
Resource: Docent (Transluce)
Where we touch it
Background on the model layer: Align your AI Model
Sources
-
US District Court for the Southern District of New York, Mata v. Avianca, Inc., Opinion and Order on Sanctions, 2023-06-22. https://storage.courtlistener.com/recap/gov.uscourts.nysd.575368/gov.uscourts.nysd.575368.54.0.pdf. Accessed 2026-10-14. ↩ ↩2
-
Transluce, “Investigating truthfulness in a pre-release o3 model”, 2025-04-16. https://transluce.org/investigating-o3-truthfulness. Accessed 2026-10-14. ↩ ↩2