Retrospring is shutting down on 1st March, 2025 Read more

Anon · 3mo

I am curious how do you notice if the work is ai generated? I have been reading a few works lately but I have not noticed anything, I am wondering if the authors didn't use it or if it just slipped past me and I was totally convinced it's the writer's work

i'm trying not to expose people using genAI to harm, and don't want to put a guide on how to avoid being detected out there, so i can only answer this vaguely ;-; also note that this is the current state of things, generative AI seems to still be steadily improving.

in general: everyone has stylistic tells that range from choice of words, sentence and work structure, grammar, themes and characterisation to punctuation and spelling.
generative ai, at least if you use one of the publicly available models not further trained on a specific data set, also has a "style". once you learned to recognise it, it feels like one single person was posting under dozens of different accounts, across all fandoms. and it's a "style" that's present outside of fic as well, a lot of magazines etc. use it too. once you're aware what it reads like, you'll see it everywhere; i can e.g. tell at a glance when people put their handwritten fic into an llm to generate a fic summary, sometimes a title alone is enough.

for everything listed below there will be cases where it's simply a part of someone's style, or they're genuine mistakes we make as writers. it's the accumulation of many or all of these aspects that point to a text being ai-generated. the more human input (editing, expansion) there is, the blurrier becomes the overall impression of a text—but it must be said that genAI leaves its traces (or shall i say: an indelible mark 🥴).

its most distinguishing feature is that it is bafflingly repetitive, to a degree no human writer is. you'll find words or phrases not specific to the text used up to a hundred times in a 100k text, as is not the case for a human writer. and this is across all ai-generated texts, to a point where seeing several of these word or phrase alone can be a sign. and it's not just words—within a text, you were until recently likely to find sentences or entire paragraphs that are a rephrased repeat of a previous one. writers do make these mistakes occasionally, but not as frequently and reliably as genAI, and most importantly: not at the level of writing this "style" would imply someone to have achieved (!), were they human.
they improved retainment of input (and possibly output too) recently, so the repetition of entire paragraphs and sentences was vastly reduced, but can still be present. the vocabulary doesn't seem to have changed, so word repetitions are still as abundant as ever.

a general marker of this "style" is that it depends on fancy words. it's a rather elegant and polished-seeming "style"—on the surface. when reading, it feels like biting into a cardboard replica of a slice of cake. big declarations with nothing behind them.

up until recently, this "style" aside, another big tell was inconsistency. if rooted in real life there'll be fake facts. then there's basic things like a character's age changing several times, time that passed not adding up, seasons not lining up, settings changing mid-scene. the rules of the world were laid out but not followed. the premise of the text got entirely lost to the point where an integral conflict stopped existing or warped into something else. things are said or happen but don't soak into the fabric of the text, characters seem suspended in their initial state.
still true currently: emotions make no sense. llms struggle greatly with portraying conflict and morally grey* areas, so emotional responses do not correspond to what happened in the text prior. usually, this manifests in an "offence" that is barely more than a flippant remark but is treated like the end of the world. conflicts are also often solved, then out of the blue reset to unresolved.
(*note that genAI can be used to generate text of any genre, including omegaverse and mafia aus, and also smut.)

with people who published work prior to the availability of llms, there's a vast difference in all elements of their style as laid out above that is so vast it cannot be attributed to finding a beta reader or an evolving style. most people's writing styles evolve very slowly, over the course of many years. oftentimes, this means the difference becoming visible not just in comparison to older work but within a text itself, when someone generated text and chose to edit or blend original writing into it.

although these texts tend to come with dialogue now (for a while they didn't, or it was clearly written by a person), the dialogue itself feels empty, often reads like therapist speak, and tends to state the obvious. it often doesn't contribute to the plot progressing. colloquialisms are often absent (unless edited in by hand).

another tell: ai-generated text comes without typos. (this stops being the case when the poster edited it.)

to be frank, personally i feel like characterisation is near absent. llms do the same as ai-generated images do: they have no knowledge of the subject so try to appeal to your feelings first and foremost, through heightened emotions, tragic situations, highlighted suffering, and prompt you to empathise so much you won't notice it's not real. if you look closely they're just cardboard cutouts of a character who could be anyone. which is why it's so tricky with fanfiction! we're used to projecting on fic. a fanfic writer can rely on the reader bringing their own ideas about the characters to the table and doing half of the work in this regard. ai-generated text passed off as fanfiction takes full advantage of this.
what is usually absent in addition to this are easter eggs, references to canon occurences like pairing moments, things they said or did. (don't use this particular observation about characterisation in isolation as a diagnostic tool though, many writers like to detach their characterisations from their however perceived personas.)

this comes almost last in this list although one would think it was obvious—but surprisingly enough, plentiful posting does not mean it's ai-generated text. one would think it would! and it can be, but in the majority of cases people post as irregularly or intensely as any fic writer, even leave chaptered texts unfinished. the exception would be those chaptered texts that rack up one million words over the course of three months.

in rare cases i found remnants of the llm responding to the prompt. there's no use looking out for this though, i've come across it maybe twice.

in general, one thing seems true for generative ai users: they generate to avoid the effort of writing and value ideas above their execution, and this mindset affects the quality of their ai-generated texts. although a lot of these texts could be easily edited and fixed—at least regarding most of the issues addressed above, not the general vibe of genAI—the majority don't fix them. make of that what you will.
we all make mistakes as writers, consciously or not, but the mistakes generative AI makes (at least regarding text) are predictable, repetitive, and lack the personal circumstance causing them to have indeed a very machine-y feeling to them.

tl;dr i find it has a noticeable "style" and also find it most bewildering to read, like setting foot into a palace with a perfect exterior that looks like a site of demolition on the inside. inescapably coming across it forced me to grow attuned to its telltale signs :( i'm not alone in this, many writers, publishers and teachers find it to be very distinct and obvious, partly due to lack of individual style, partly due to mentioned carelessness. i like to think it's a skill that can be learned through exposure (and also reading carefully and critically), but understand why people would choose not to spend leisure time reading ai-generated text, and also choose not to scrutinise each fic they come across. might as well be that you have nothing to worry about, or at the very least not yet. good luck out there ;-;

Retrospring uses Markdown for formatting

*italic text* for italic text

**bold text** for bold text

[link](https://example.com) for link