Speaking at least in regards to fiction writing - there was an article on Mark Lawrence's blog [1] a few years back where he posted 10 short fantasy stories with the theme “meeting a dragon.” with an even mixture of AI and human written ones, and the challenge was to see if you could tell which were which.
Rather than relying on simple regex matches for words like “delve” or checking for em dashes, the heuristic I used was this: if someone asked me to convince a friend to read a given story in a single sentence, could I do it?
I find that once you strip away much of the decorative finery from AI-generated copy, it just doesn’t have much to say. AI often tries to distract from its prosaicism with reams of descriptive prose.
If someone tries to hide that their writing contains AI, or otherwise gets their AI to write differently through prompting or a special model, I can’t tell. I can suspect through heuristics, like perfect grammar, rule of three, bad metaphors, vapidity (which is a particularly strong signal), and a hard-to-describe “feeling”. But some people write like that, especially if they've read lots of AI writing themselves; and the same phrase (especially if it's short) can have been generated by AI and written by a human.
However, I’ve noticed a lot of writing where it seems the person and model don’t even try to hide that it’s AI. Besides matching every heuristic above and an especially strong AI “feeling”, there are literal giveaways: “it’s not X, it’s Y”, “here’s the kicker”, “quiet part out loud”, “and the best part?” While technically a human can write like that (e.g. if they pretend to be AI), I feel it’s safe to assume such writing is at least AI-assisted.
Ultimately though, I always judge writing on the content itself and recommend you do too. Every piece of writing that I’ve known or suspected was mostly AI-generated has been long-winded and vapid, which AFAIK most people consider to be valid criticisms, so you're less likely to get pushback for claiming them. And at least for me, it’s easier to determine and prove that writing is long-winded and vapid than AI-generated. If a piece of AI writing is terse, informative, and high-quality in other ways, personally I would consider it high-quality overall (and I’ve already judged some writing as high-quality that I later learned was AI assisted, but it ended up being mostly human-written with the AI only rephrasing a few small parts).
AI written text is often characterized by a very peculiar choice of words — many people report "delve into" is a particularly telling sign of text written by AI.
If you want me to add more examples, feel free to ask me for more!
You've got to be below 85iq to fall for the rope a dope AI ads. The photorealism is wax, the scene setup isn't right and the language is odd from translation.
The default style of ChatGPT is very recognizable. Lots of weird metaphors, lots of sets of three bullet points, lots of "it's not x, it's y". But that's just the standard style, they can imitate any style if configured to do so.
You can't anymore. You can only verify something wasn't written by a AI, and that case is only true for text published before 2023 (true negative results).
Speaking at least in regards to fiction writing - there was an article on Mark Lawrence's blog [1] a few years back where he posted 10 short fantasy stories with the theme “meeting a dragon.” with an even mixture of AI and human written ones, and the challenge was to see if you could tell which were which.
Rather than relying on simple regex matches for words like “delve” or checking for em dashes, the heuristic I used was this: if someone asked me to convince a friend to read a given story in a single sentence, could I do it?
I find that once you strip away much of the decorative finery from AI-generated copy, it just doesn’t have much to say. AI often tries to distract from its prosaicism with reams of descriptive prose.
[1] - https://mark---lawrence.blogspot.com/2023/09/so-is-ai-writin...
There are 2 types of AI writing.
If someone tries to hide that their writing contains AI, or otherwise gets their AI to write differently through prompting or a special model, I can’t tell. I can suspect through heuristics, like perfect grammar, rule of three, bad metaphors, vapidity (which is a particularly strong signal), and a hard-to-describe “feeling”. But some people write like that, especially if they've read lots of AI writing themselves; and the same phrase (especially if it's short) can have been generated by AI and written by a human.
However, I’ve noticed a lot of writing where it seems the person and model don’t even try to hide that it’s AI. Besides matching every heuristic above and an especially strong AI “feeling”, there are literal giveaways: “it’s not X, it’s Y”, “here’s the kicker”, “quiet part out loud”, “and the best part?” While technically a human can write like that (e.g. if they pretend to be AI), I feel it’s safe to assume such writing is at least AI-assisted.
Ultimately though, I always judge writing on the content itself and recommend you do too. Every piece of writing that I’ve known or suspected was mostly AI-generated has been long-winded and vapid, which AFAIK most people consider to be valid criticisms, so you're less likely to get pushback for claiming them. And at least for me, it’s easier to determine and prove that writing is long-winded and vapid than AI-generated. If a piece of AI writing is terse, informative, and high-quality in other ways, personally I would consider it high-quality overall (and I’ve already judged some writing as high-quality that I later learned was AI assisted, but it ended up being mostly human-written with the AI only rephrasing a few small parts).
Well, that is a very good question!
AI written text is often characterized by a very peculiar choice of words — many people report "delve into" is a particularly telling sign of text written by AI.
If you want me to add more examples, feel free to ask me for more!
You've got to be below 85iq to fall for the rope a dope AI ads. The photorealism is wax, the scene setup isn't right and the language is odd from translation.
The default style of ChatGPT is very recognizable. Lots of weird metaphors, lots of sets of three bullet points, lots of "it's not x, it's y". But that's just the standard style, they can imitate any style if configured to do so.
It used to be em dashes. And those really obvious words like „delve into“ I remember there was even a story about those words.
EDIT: https://arxiv.org/abs/2406.07016
You can't anymore. You can only verify something wasn't written by a AI, and that case is only true for text published before 2023 (true negative results).