It Copies Everything

I’d like to claim I’ve never taught a model a bad habit by accident. Unfortunately, I’ve made that mistake frequently.

LLMs are non-deterministic by nature, and their behavior rests on training that we humans provide, so some interesting things can happen. Point out a pattern in text that you’d like a model to avoid or emulate, and what you’ll often get is just that: a repeating pattern (or anti-pattern) that becomes a nuisance. It’s natural, after all, to tell a model “do this” and “don’t do that,” expecting that it’s “smart” enough to employ subtlety in applying the principles you’re teaching it. It may be natural. It is also wrong in almost every way.

I’ll admit it: I love my em dashes. I love them almost as much as I love the Oxford comma, which is to say that I’ll stop using them sometime after never. Unfortunately, especially early on, LLMs tended to use a lot of em dashes in their writing; it was one of the “tells” you could usually use to spot AI-written documents, emails, and LinkedIn posts. So I once—against my better judgment—tried to teach a model moderation in its use of that particular tic. All I succeeded in doing was convincing the model to NEVER use them or, after more tuning, to only use them “X” number of times in “Y” number of words. I gave up. Besides, if I’m going to copiously use both em dashes and the Oxford comma, who am I to be overly pedantic with a model just trying to get by on a steady diet of tokens and hope?

Examples are where this bites hardest. Show a model three examples and it doesn’t study them for the principle. It copies them, all of them, including the parts you’d never have called part of the lesson.

The chart below is a support-ticket classifier built the way Anthropic’s docs suggest: three to five examples, each in its own tag so the model can tell a demonstration from an instruction, one per label, and a single line above them saying what a label means. The third example is the one doing the real work. The customer mentions a card renewing, and the right answer is still “account,” because the thing they need fixed is the lockout. Leave that one out and you’ve taught the model to pick the first topic mentioned, a lesson it will learn flawlessly.

One caveat, since I’m the one who brought up bad information. The “three to five, relevant, diverse, in tags” part is straight from the docs. The claim that it copies length, tone, and formatting is my reading of their warning about unintended patterns, and nobody has measured it for me, me included.

So: a handful of examples, one of them weird on purpose, and a sentence saying why the labels are what they are. It turns out the model was applying my principles with great subtlety all along. They just weren’t the principles I thought I was teaching.

What’s the strangest habit a model has picked up from one of your examples?