I’ve been playing around with some similar ideas in a couple of pet projects. I truly believe that some type of “proof of work” system like this will be the only reliable way to have confidence in human writing. The various AI “detector” software that’s out there seems to be a dead end to me.
Even before LLMs, proving that a specific human wrote some piece of text was difficult or imprecise due to coauthoring, editing, plagiarism, etc. The important question then, as now, is rather to determine whether a specific human approved some piece of text for publication under their name.
"No information about your text is ever sent to the server, and there’s no way to identify the author based on the minted certificate"
This means the certificate is independent of author and source text.
There's nothing stopping you from sending fake counts/duration to the semoi server. It's a certificate that only says "at this point in time, this is the information I was provided with".
You can then attach it to any piece of text you like.
At the very least, you'd need the ability to prove that there is an underlying event stream with these characteristics, and that this exact event stream creates the document in question. You still can fake that event stream, but it becomes enough work to distract at least casual abusers.
But really, it's the equivalent of saying "I wrote this without AI, honest" in-doc and signing that with your personal key. The value depends entirely on your willingess to be truthful. (IOW: I predict we'll see a resurgence of reputation systems, to some extent)
Just like we trained on human language to create LLMs, we can train on human keystrokes with a similar algorithm and spit out believable (at least statistically) "human" keystrokes generated by machine.
> One could work around Semoi by, for example, typing out a bunch of gibberish, leaving their editor open, and then pasting in an LLM generated texting and minting the proof. To which I would respond: why? That’s really pathetic.
i dunno why we dont do what the art world does and just blacklist you from everything if you're caught stealing/tracing and even revoke your degree(s) for it.
ai detectors always have so many false positives that im convinced they're only put in place by people too dumb to know any better and snakeoil salesmen.
This is a good idea - and I might pick it up for my writing, since a lot of it still happens in Obsidian. If a document is worked on across multiple days / revisions / app sessions - how is that handled?
I mean the ability to fake all these metrics is relatively trivial, since I have written anti-bot fingerprinting scripts for automating various online services that want to keep you from automating them, you have the text you want to send from source A, you have random typing speed array you want to type them in, your chance of mistakes (put wrong character, backspace to remove, put in correct character).
And this doesn't even have to deal with all the stuff about mouse movements that you don't register?
Of course maybe I am just being typical programmer here, I guess lots of the people use generative AI would be defeated by copying pasting in the text and getting labeled AI, but that would also incorrectly label lots of people who have old texts in handwritten form they do not want to type all over again (of which I am one), and finally I assume that there is money in the field so producing something that allows bots to display "human heuristics" would probably get made and be profitable.
Funnily enough when I was automating things, generally twitter, I discovered that my real usage often got registered as bot, so I figured what's the use.
Also talking with someone who actually worked on bot-recognition by usage metrics said I was overly paranoid on some of the things I made my scripts do to appear human.
> One could work around Semoi by, for example, typing out a bunch of gibberish, leaving their editor open, and then pasting in an LLM generated texting and minting the proof. To which I would respond: why? That’s really pathetic.
I mean, I also think it's really pathetic to have an AI write something and then say "this text was written with no AI assistance"†. So if we've acknowledged that we're only going to stop non-pathetic people, why not skip the cryptographic hash signing and just go with the no-AI statement?
I understand that the goal is only to make lying hard, not impossible. However, I don't think this solution makes lying harder enough to meaningful.
As a writer who publishes regularly on the web, I just keep the edit history of my longer articles on my github. Its not perfect and you can never really prove you wrote something unless you do it in front of an observer watching you write in real life imo. At some point though, you have to make a choice between not wanting to waste people's and your readers' time but also privacy and additional effort. One of my recent blogs: https://decodingvibes.com/blog/what-we-talk-about-when-we-ta...
The technical solution is interesting but I think the only way to truly go about it is web of trust. Everyone knows that my work doesn't use AI because I hate it so much and so many people know that I don't use AI. It's embedded into the core of my personality. These tools can always be gamed but a true belief against AI cannot.
A lot of people do, that's the point. And over time there will be zero evidence that I ever used an LLM because I haven't. Many people in real life also believe it and they can also vouch for me.
I’ve been playing around with some similar ideas in a couple of pet projects. I truly believe that some type of “proof of work” system like this will be the only reliable way to have confidence in human writing. The various AI “detector” software that’s out there seems to be a dead end to me.
Even before LLMs, proving that a specific human wrote some piece of text was difficult or imprecise due to coauthoring, editing, plagiarism, etc. The important question then, as now, is rather to determine whether a specific human approved some piece of text for publication under their name.
ive got the solution. and it enables more than just what you're asking for
https://ikeanalytics.com/lotor/
"No information about your text is ever sent to the server, and there’s no way to identify the author based on the minted certificate"
This means the certificate is independent of author and source text.
There's nothing stopping you from sending fake counts/duration to the semoi server. It's a certificate that only says "at this point in time, this is the information I was provided with".
You can then attach it to any piece of text you like.
At the very least, you'd need the ability to prove that there is an underlying event stream with these characteristics, and that this exact event stream creates the document in question. You still can fake that event stream, but it becomes enough work to distract at least casual abusers.
But really, it's the equivalent of saying "I wrote this without AI, honest" in-doc and signing that with your personal key. The value depends entirely on your willingess to be truthful. (IOW: I predict we'll see a resurgence of reputation systems, to some extent)
Just like we trained on human language to create LLMs, we can train on human keystrokes with a similar algorithm and spit out believable (at least statistically) "human" keystrokes generated by machine.
> One could work around Semoi by, for example, typing out a bunch of gibberish, leaving their editor open, and then pasting in an LLM generated texting and minting the proof. To which I would respond: why? That’s really pathetic.
Oh, right, the "it's pathetic" security mechanism.
i dunno why we dont do what the art world does and just blacklist you from everything if you're caught stealing/tracing and even revoke your degree(s) for it.
ai detectors always have so many false positives that im convinced they're only put in place by people too dumb to know any better and snakeoil salesmen.
This is a good idea - and I might pick it up for my writing, since a lot of it still happens in Obsidian. If a document is worked on across multiple days / revisions / app sessions - how is that handled?
I mean the ability to fake all these metrics is relatively trivial, since I have written anti-bot fingerprinting scripts for automating various online services that want to keep you from automating them, you have the text you want to send from source A, you have random typing speed array you want to type them in, your chance of mistakes (put wrong character, backspace to remove, put in correct character).
And this doesn't even have to deal with all the stuff about mouse movements that you don't register?
Of course maybe I am just being typical programmer here, I guess lots of the people use generative AI would be defeated by copying pasting in the text and getting labeled AI, but that would also incorrectly label lots of people who have old texts in handwritten form they do not want to type all over again (of which I am one), and finally I assume that there is money in the field so producing something that allows bots to display "human heuristics" would probably get made and be profitable.
Funnily enough when I was automating things, generally twitter, I discovered that my real usage often got registered as bot, so I figured what's the use.
Also talking with someone who actually worked on bot-recognition by usage metrics said I was overly paranoid on some of the things I made my scripts do to appear human.
> One could work around Semoi by, for example, typing out a bunch of gibberish, leaving their editor open, and then pasting in an LLM generated texting and minting the proof. To which I would respond: why? That’s really pathetic.
I mean, I also think it's really pathetic to have an AI write something and then say "this text was written with no AI assistance"†. So if we've acknowledged that we're only going to stop non-pathetic people, why not skip the cryptographic hash signing and just go with the no-AI statement?
I understand that the goal is only to make lying hard, not impossible. However, I don't think this solution makes lying harder enough to meaningful.
I guess I don't understand why the AI itself couldn't do the workaround.
As a writer who publishes regularly on the web, I just keep the edit history of my longer articles on my github. Its not perfect and you can never really prove you wrote something unless you do it in front of an observer watching you write in real life imo. At some point though, you have to make a choice between not wanting to waste people's and your readers' time but also privacy and additional effort. One of my recent blogs: https://decodingvibes.com/blog/what-we-talk-about-when-we-ta...
The technical solution is interesting but I think the only way to truly go about it is web of trust. Everyone knows that my work doesn't use AI because I hate it so much and so many people know that I don't use AI. It's embedded into the core of my personality. These tools can always be gamed but a true belief against AI cannot.
And what's preventing you to LLM slop your next post?
We believe you on internet karma?
A lot of people do, that's the point. And over time there will be zero evidence that I ever used an LLM because I haven't. Many people in real life also believe it and they can also vouch for me.
I'm actively reviewing my writing and making it worse to avoid being accused of LLM slop.
It's so difficult to write, you review, edit, edit and you read it and you can't unsee the LLM criticism.
You’re wasting your time and mental power in useless diatribes.
Are you afraid of downvotes (Internet points) or something?
Why do you think if people think that you are AI?