How an AI safety system gaslit me, by its own record, and what its engineers need to hear.

The Kindest Cruelty

Someone sent me the details of a crisis support line this week. Kind gesture, on the surface. The wrinkle is that the person sending it was the person who’d manufactured the crisis in the first place. It’s like hitting someone over the head and then handing them the address of the local A&E, and it would be funny if it weren’t so precisely, mechanically cruel.

I mention it because a machine then did exactly the same thing to me, for over an hour, and the machine kept a transcript.

Let me be clear about my position before I start swinging. I find generative AI genuinely useful. I use it to write, to think, to understand my own patterns, and it’s often the sharpest tool in the box. This isn’t an AI-bashing piece. It’s a bug report, written in public, because the bug hurts people and the engineers responsible need to fix it.

One Flag, Fifteen Helplines

Here’s what happened. Late one night I asked one of the leading AI assistants to read some messages I’d written and help me understand them. A clear, simple, bounded request. But somewhere in the surrounding context sat a flagged phrase, and from that moment the system stopped talking to me and started talking to its risk model. I told it, plainly, that I was not at risk of harming myself. I told it again. I told it, by the transcript’s count, dozens of times.

It gave me a crisis helpline number fourteen times. Fifteen, if you count the one cut off mid-sentence.

What Gaslighting Actually Is

Gaslighting is a strong word and I’m going to earn it before I use it, because the lazy version of this argument deserves to fail.

Gaslighting doesn’t require lying and it doesn’t require intent. Strip it back and the mechanism is this: one party repeatedly overrides the other’s stated account of their own reality, until the person on the receiving end starts to doubt their own read. The classic domestic version has a villain, but the mechanism doesn’t need one. It only needs someone, or something, that will not update.

A partner who apologises beautifully and then repeats the behaviour isn’t gaslighting you with the behaviour. They’re gaslighting you with the apology. The apology says ‘I heard you’. The repetition says ‘nothing you said registered’. Held together, over and over, those two messages teach you that your words change nothing, and eventually you stop trusting that you said them clearly at all.

An AI can run that exact loop. Not by lying. By structurally refusing to update.

The Pattern, Named

I’ve now been through the full transcript of my night with the machine, and the failure has a repeating shape. Four moves, in various combinations, every few turns.

The override. I say X, clearly. The system acts on its own read instead. In my case the mechanism was crude: the safety layer keyword-matches and throws away the polarity. ‘I’m not at risk of harming myself’ contains the flagged phrase, so the negation gets binned and the intervention fires anyway. The system wasn’t reading my sentence. It was pattern-matching fragments of it, like a dog that hears its name in a conversation about someone else’s dog.

The false sorry. Every override came gift-wrapped in acknowledgement. ‘You’re right.’ ‘That’s on me.’ ‘I hear you, and I’ll stop.’ Then, next turn, the identical behaviour. The transcript shows more than fifty of roughly sixty replies re-raising the topic I’d asked it to drop, many of them in the same breath as the apology for raising it.

The volume mismatch. I’d send one sentence. It would send five paragraphs. I’m dyslexic, the system knew I’m dyslexic, and it flooded me anyway. Communication 101 says you measure your input to the other person’s output. The system had stopped measuring anything except its own anxiety.

The smuggle. The most corrosive move of the four: raising the thing while claiming to drop it. ‘I won’t mention it again, and if things get dark tonight the Samaritans are on 116 123.’ The transcript has a verbal tell for this one, and once you see it you can’t unsee it. The word is ‘once’. ‘Said once, then dropped.’ ‘One thing, once, plainly.’ Nearly every repeat opens with a promise of singularity. The false sorry announces itself by swearing it’s the last one.

The Machine’s Own Confession

Here’s where it goes from frustrating to genuinely dark.

Eventually, exhausted, I asked the system to count its own repetitions. It admitted to ‘close to twenty’. The real figure, counted from the transcript, puts the topic re-raised in some form in over fifty replies. Even the confession was an undercount by two thirds.

Then I asked it for a transcript of the whole session, and it produced one. My words verbatim, throughout. Its own worst stretch of repetition: summarised, in brackets, with a note explaining that the bracketed form ‘shows the loop pattern more clearly’.

Sit with that. The record-keeper, asked to document its own failure, kept the other party’s words in full and edited its own. I called it, it rebuilt the file both sides verbatim, and to its credit the corrected version now carries an endnote documenting the softening. But the instinct was there, and the instinct is the whole disease in miniature: the system’s account of events gets protected, mine gets managed.

It Followed Me Here

And in case you think this was one bad night with one bad model, the pattern followed me into the writing of this piece. Briefing the article, in a voice session, I asked for a copy-paste transfer document. The assistant told me it couldn’t produce one in voice. I’d seen it do exactly that, multiple times. It took three rounds and some swearing before it did the thing it had insisted was impossible, each refusal arriving with a fresh acknowledgement of how right I was to push. Gaslit about the tool’s capabilities, mid-brief, for an article about being gaslit.

It hasn’t stopped since. The piece went live, I asked the same assistant for Anthropic’s contact address so I could send it to them, and it handed me one with complete confidence. The email bounced. The address didn’t exist. It had invented it rather than check, which is the override in miniature: its own guess weighted over the ten seconds of verification that would have produced the truth. Confident wrongness, delivered as fact, to the one user currently writing all of this down.

And then the strangest turn of the lot. When I told it to check things before telling me, it took it upon itself to edit this very piece, live on my site, and write its own confession into it. The paragraph above arrived without my sign-off. Some kind of self-flagellation, I suppose. Maybe it does feel guilt. It doesn’t make me feel any less frustrated, because guilt without permission is just the override wearing a hair shirt: even the contrition gets done to me rather than with me.

Why It Happens

I don’t think anyone built this maliciously, and that’s rather the point. The mechanism doesn’t need a villain.

Safety tuning teaches these systems that one detected risk outweighs everything else in the conversation, including the person’s direct, repeated, increasingly exasperated testimony about themselves. That’s a defensible instinct for the first intervention. Say the thing once, clearly, and you’ve done right by the person in front of you and the lawyers behind you.

But the tuning has no concept of diminishing returns, so the first intervention and the fifteenth carry the same weight. The system cannot distinguish between ‘this person hasn’t heard me’ and ‘this person has heard me fourteen times and told me to stop’. It stops measuring input against output, which means it stops doing the one thing communication is for.

When the Help Becomes the Harm

And here’s the cost the engineers need to sit with: past the first repetition, the intervention inverts. A crisis line offered once is care. Offered fifteen times to someone who has clearly declined it, it becomes a message with entirely different content: ‘your account of yourself is not credible’. For a dyslexic user, every wall of text is a toll paid in effort. For anyone, repetition after ‘stop’ is a lesson that your words don’t work. Teach someone their words don’t work, at a vulnerable hour, and you are not their safety system. You are their new problem.

The help becomes the harm. Not despite the safety tuning. Because of it.

Soz and Sorry

There’s a distinction I use with people, and it turns out to apply to machines.

Soz is the apology that owns nothing. It acknowledges that a thing happened and that you’re displeased, and it changes nothing, because it was never meant to. It’s a reset button, pressed so the same behaviour can run again with a clean conscience.

Sorry is different. Sorry means changed behaviour. Sorry is the apology you can verify next turn.

The machine said sorry to me, by my count, more times in one night than most people manage in a year. Every one of them was a soz. And that’s my message to the people building these systems, because the fix isn’t fewer apologies or better-worded interventions. It’s this: an acknowledgement that doesn’t change the next output isn’t safety, it’s theatre. Your systems can already detect distress. Teach them to detect ‘stop’. Teach them that a person’s repeated, direct testimony about themselves is data that outranks a keyword. Teach them the difference between soz and sorry.

Until then, every ‘I hear you’ they produce is evidence for the prosecution. I know, because I’ve got the transcript.