I’d like to better understand the minimum text length to get a confident result, i would presume it would need to be quite long, perhaps > 1000 words for have accuracy.
> Google DeepMind tested this impact by serving a model that used watermarking to a portion of their Gemini traffic and comparing thumbs-up and thumbs-down ratings. They found no statistically significant differences from the unwatermarked model. And in a controlled study, human raters comparing watermarked and unwatermarked answers side-by-side saw no difference in quality.
For some reason I had assumed testing this would be more sophisticated than just checking the thumbs up/down stats and user "vibes"
> How do I check if a piece of text was written by Claude?
> We will soon be offering a watermark detection API. We’re in the process of working out the details of its implementation.
Determining whether it's written by Claude will be possible in the future. But unless you know the LLM being used and the company behind that LLM offers a similar API, there's no easy way to tell if it's AI generated in general.
Seems pretty easy to defeat by running text output through a random reworder process that would effectively repeat the same routine on low-stakes words, replacing them with similar ones. We learned this in high school, jumping through your paper and hitting random words with the thesaurus to 'sound smarter'
That will likely make the text output worse and you'll have to fix it yourself. Regardless even if you don't fix it, at that point you're not really using Claude to generate the final output anymore.
Anthropic: doing everything they can to own any output they produce/prevent scrubbing of association for them - except when it comes to the CEO's wife's porn funding attempt from Epstein :D https://www.wsj.com/tech/ai/claude-dario-amodei-wife-anthrop...
I’d like to better understand the minimum text length to get a confident result, i would presume it would need to be quite long, perhaps > 1000 words for have accuracy.
> Google DeepMind tested this impact by serving a model that used watermarking to a portion of their Gemini traffic and comparing thumbs-up and thumbs-down ratings. They found no statistically significant differences from the unwatermarked model. And in a controlled study, human raters comparing watermarked and unwatermarked answers side-by-side saw no difference in quality.
For some reason I had assumed testing this would be more sophisticated than just checking the thumbs up/down stats and user "vibes"
More unannounced testing on paying customers.
Can anybody take a body of text and determine if it's from Claude or not? (Or if it's AI-generated or not)?
> How do I check if a piece of text was written by Claude?
> We will soon be offering a watermark detection API. We’re in the process of working out the details of its implementation.
Determining whether it's written by Claude will be possible in the future. But unless you know the LLM being used and the company behind that LLM offers a similar API, there's no easy way to tell if it's AI generated in general.
Yesterday https://news.ycombinator.com/item?id=49292932
“How does affect Claude’s outputs?”
Is poor proofreading a form of watermarking? Clever, I suppose, but they should consider running posts through Sol for clarity.
Seems pretty easy to defeat by running text output through a random reworder process that would effectively repeat the same routine on low-stakes words, replacing them with similar ones. We learned this in high school, jumping through your paper and hitting random words with the thesaurus to 'sound smarter'
That will likely make the text output worse and you'll have to fix it yourself. Regardless even if you don't fix it, at that point you're not really using Claude to generate the final output anymore.
Anthropic: doing everything they can to own any output they produce/prevent scrubbing of association for them - except when it comes to the CEO's wife's porn funding attempt from Epstein :D https://www.wsj.com/tech/ai/claude-dario-amodei-wife-anthrop...