Mate its been a couple of days, I would think you'd be mad to expect perfection from an LLM reverse engineering job straight away.
As a proof of concept however it shows that we are at the stage where any malicious actor has a very low barrier to entry to cause large scale corporate espionage etc.
The endavour doesn't have much value at all except as an experiment how well translating C code to Rust via LLM works (but you don't need an LLM for that either, C2Rust was already a thing).
Also, the C version of Quake compiled to WebAssembly and running in browsers is just as "safe".
But C isn't as nice to work with. Maybe OP just thought it was fun to see it take shape and wanted to share it. Why do any sort of little side project like this at all? Just seems like everyone is judging this too harshly. If an LLM is reasonably good at this sort of thing, and you think it would be cool to see Quake written in Rust running in the browser, I say just do it and have fun and don't be shy about doing show and tell about it.
What this exploits is the mental shortcut of "rust = good", which might be a good thing, as that was always wrong. But now it is being pushed to its breaking point so that that idea will eventually collapse.
Accelerationalism on a micro scale, basically.
AI keeps breaking things that were broken before like this constantly. It's the great cleanup of old bullshit. (Unfortunately through even more bullshit, but at least there is a silver lining)
Oh my god, and the author even supplied a "proof"[0] visual diff harness... that it replicates the original game pixel for pixel.
Just the cherry on top of great demonstration of our collective new superpower: asking computers to do something we can describe how to do, but would (probably) never take the time to do ourselves.
How long before the same thing is done to like, banking back ends? Wallstreet proprietary software? Amazons logistics and distribution systems?
It seems like we might be weeks/days/hours before a situation where someone back engineers and spoofs a system so pivotal to modern human society that the plug needs to be pulled.
> spoofs a system so pivotal to modern human society that the plug needs to be pulled.
why would a recreated system be detrimental?
If currently there's a monopoly on a software, this AI recreation is a good outcome to poke holes in that monopoly. It's only bad if you are financially invested in said monopoly, and this would be a minority compared to the amount of benefits that society at large could obtain.
If someone recreates Amazon's logistics and distribution systems they could try to compete with Amazon? But they'd also need the connections, distributors, transportation, etc. same with banking software, you need capital to be a bank not just software, and if they have the capital then the technology is working we intended making it easier to make new things and innovate, or at least just compete?
No, I am not talking about "taking over" companies and trying to emulate them and do business yourself. You just need to be able to break trust in the api calls and no one knows if a purchase order or transaction is legitimate.
Obviously you need to have access to the keys, BUT I don't see this as a dealbreaker anymore because you just get your agents to go and find them.
Or they can just take your money and not send out anything.
Alternatively, you just act as a middleman drop shipper and slightly raise the price more than Amazon’s and skim the difference. It might be a while before they find out.
I mean some people (me included) have been begging society to pull that plug since over 10 years now.
The plug being "the cloud" and "hooking everything up to the same internet".
These confusion attacks can only confuse people, because critical systems can exist in the same space where entertainment systems and all other categories of systems live.
This was wrong even before LLMs.
The visual diff harness was probably how they got rid of a lot of visual bugs, just tell the LLM to keep going until the pixels match exactly as the verification criteria
tbh I thought this was a commonly used technique even prior to LLMs? I know I've been using it extensively myself, but I was inspired by Dolphin's extensive visual CI system.
If it is common in the world of video game porting, that just shows my ignorance. I'm familiar with visual diffs in CI for e.g. web development (comparing a static component), but to do that to compare frames over time in a video game/3D environment is new to me.
There are so many more degrees of freedom, which I can see Claude handled... mipmaps, subtle differences in lighting/positioning/compositing etc.
Even then, visual diffs were pretty flakey for web development, because one's OS and browser choice would slightly alter the exact pixels blitted to the screen. At least this was the case for the tests that would simply match pixels instead of computing a sort of visual hash.
It's also partly why some people preferred snapshot tests that compared the DOM tree instead, though that was brittle in other ways (e.g. tests would break if an application's frontend used a major UI library and an update to the library permuted the order of classes in some part of the HTML).
For web UI tests this is mainly solved, at least when using Playwright. It allows setting thresholds, percentages and some other config items to allow some small differences in pixels. https://playwright.dev/docs/test-snapshots#options
I would never have imagined a phone web browser capable of this in 1996 between playing Quake and testing out the hot new JavaScript powered mouse rollover image effects in Netscape Navigator 2.0
This may sound funny but I feel games would lose a lot of fun if they were all written in rust and had classes of bugs just not available to them. For better or for worse quirks and bugs in games have shaped how people approach games, and also have given games charm for decades.
Fortunately for gamers, Rust doesn't do anything to stop physics engines from going haywire or preventing players from clipping out of bounds. A Mario 64 written in Rust still has parallel universes (well, assuming that you translated the out-of-bounds float-to-short cast as a modulo, which is actually UB in the C implementation).
‘Safe rust’ just being used to mean ‘no unsafe’ is kind of a shallow understanding of the language. You can write code without unsafe that is not really ideal at all eg abusing vector/slice indexing to create a kind of interior mutability that the borrow checker is blind to. Which as a C port I’m going to guess it probably ends up doing
Almost hard to believe that someone using an LLM to mindlessly migrate software from one language to another for no practical reason at all would have a shallow understanding of those languages...
I think 'standing on the shoulders of giants' is the phrase for something like this. It's the confluence of browser rendering, WASM, Rust, and LLMs. For me it's less a demo of what AI can do, and more a showcase of the human effort from the past few decades on the parts that needed to fall into place for an LLM to come in at the (relatively speaking) last second and claim a win. Sure, an LLM did the port from C to Rust, but think of all the things needed for it to all work. That's pretty damn amazing, and it wasn't done with AI.
Vibe coders need to stop trying to use the browser as a platform for complex games. It's never going to work out and will always lead to a slow, unplayable, piece of shit. If you want to make a game then just make it a desktop program... There's a reason why literally every modern game is built in this way.
These aren’t JavaScript apps, they compile to WASM which actually has really good performance. Probably not as good as native assembly but for an old game it’s easily more than enough.
I remember seeing the QuakeC line of code that halved self-damage from rockets. It enabled rocket jumping, but it also made the rocket launcher a much, much better close quarters deathmatch weapon than in Doom, so I thought it was a bit lame.
Are hordes of developers now going to port all sorts of open-source software to Rust using LLMs and passing it off as their own work?
I don't see any need for this. Rust is brilliant but the Quake C++ code was already more or less bug-free.
Nitpick: Quake was written C, not C++. The first id game written in C++ was Doom 3 around 2004.
Well look at Adobe, this just happened to the entire CSS suite. Not open source but proprietary.
Have you tried using the clones (in an airgapped VM, please)?
Because I did and it's the playable allegory of the cave but in vibecoded rust.
Mate its been a couple of days, I would think you'd be mad to expect perfection from an LLM reverse engineering job straight away.
As a proof of concept however it shows that we are at the stage where any malicious actor has a very low barrier to entry to cause large scale corporate espionage etc.
Past performance is not indicative of future results.
It's easy to create a set that looks as if it was a real city. It's infinitely more hard to create that real city.
But you're absolutely right that a set is all it needs for all sorts of (cyber) attacks.
How are people even doing this with frontier models without it immediately saying it can't do that.
From what I understand, the adobe guys software is not great.
The Adobe one doesn’t seem to be a decomp. It’s just someone asking the ai to build the same tool from scratch in rust.
Consequently it’s missing tons of features.
[delayed]
I mean...why not do it? Are you implying it would only have value if someone painstakingly did it by hand?
The endavour doesn't have much value at all except as an experiment how well translating C code to Rust via LLM works (but you don't need an LLM for that either, C2Rust was already a thing).
Also, the C version of Quake compiled to WebAssembly and running in browsers is just as "safe".
But C isn't as nice to work with. Maybe OP just thought it was fun to see it take shape and wanted to share it. Why do any sort of little side project like this at all? Just seems like everyone is judging this too harshly. If an LLM is reasonably good at this sort of thing, and you think it would be cool to see Quake written in Rust running in the browser, I say just do it and have fun and don't be shy about doing show and tell about it.
What about what about people learn and possibly the fun they get during the process?
Sure this project won't save humanity or optimize some KPI pleasing some obscure hierarchy.
Let people have fun, as long as they don't hurt anyone personally I'm fine with it and even I'm happy learning someone had some cool moments.
Yeah I agree. I love Rust and Quake is cool, but this is a trivial project for an LLM, and kind of pointless.
It would have been impressive before LLMs, but now? Who cares? Why is this here?
I suppose it will fizzle out in a few weeks.
What this exploits is the mental shortcut of "rust = good", which might be a good thing, as that was always wrong. But now it is being pushed to its breaking point so that that idea will eventually collapse.
Accelerationalism on a micro scale, basically.
AI keeps breaking things that were broken before like this constantly. It's the great cleanup of old bullshit. (Unfortunately through even more bullshit, but at least there is a silver lining)
Its been like 3 months of this.
I mean arguably, it has been [claude, when did chatGPT launch and when did we get toolcalls? Subtract from current date] of this.
But "this" has been shapeshifting somewhat constantly. We're dynamically crossfading from one dysfunction being blown up to the next.
Oh my god, and the author even supplied a "proof"[0] visual diff harness... that it replicates the original game pixel for pixel.
Just the cherry on top of great demonstration of our collective new superpower: asking computers to do something we can describe how to do, but would (probably) never take the time to do ourselves.
[0] https://github.com/terrapapagalli1516/quake-srp/tree/main/or...
Yeah cool.
How long before the same thing is done to like, banking back ends? Wallstreet proprietary software? Amazons logistics and distribution systems?
It seems like we might be weeks/days/hours before a situation where someone back engineers and spoofs a system so pivotal to modern human society that the plug needs to be pulled.
> spoofs a system so pivotal to modern human society that the plug needs to be pulled.
why would a recreated system be detrimental?
If currently there's a monopoly on a software, this AI recreation is a good outcome to poke holes in that monopoly. It's only bad if you are financially invested in said monopoly, and this would be a minority compared to the amount of benefits that society at large could obtain.
If someone recreates Amazon's logistics and distribution systems they could try to compete with Amazon? But they'd also need the connections, distributors, transportation, etc. same with banking software, you need capital to be a bank not just software, and if they have the capital then the technology is working we intended making it easier to make new things and innovate, or at least just compete?
No, I am not talking about "taking over" companies and trying to emulate them and do business yourself. You just need to be able to break trust in the api calls and no one knows if a purchase order or transaction is legitimate.
Obviously you need to have access to the keys, BUT I don't see this as a dealbreaker anymore because you just get your agents to go and find them.
Or they can just take your money and not send out anything.
Alternatively, you just act as a middleman drop shipper and slightly raise the price more than Amazon’s and skim the difference. It might be a while before they find out.
Example, parking places with QR codes for paying webapps.
Currently a plague in some European countries.
It looks like the real site, and you pay twice, in the fake app, and later the police.
Yep. On a small scale, you could skim money off transactions. On a large scale, you could break global distribution and logistics chains.
I mean some people (me included) have been begging society to pull that plug since over 10 years now.
The plug being "the cloud" and "hooking everything up to the same internet".
These confusion attacks can only confuse people, because critical systems can exist in the same space where entertainment systems and all other categories of systems live. This was wrong even before LLMs.
The visual diff harness was probably how they got rid of a lot of visual bugs, just tell the LLM to keep going until the pixels match exactly as the verification criteria
tbh I thought this was a commonly used technique even prior to LLMs? I know I've been using it extensively myself, but I was inspired by Dolphin's extensive visual CI system.
If it is common in the world of video game porting, that just shows my ignorance. I'm familiar with visual diffs in CI for e.g. web development (comparing a static component), but to do that to compare frames over time in a video game/3D environment is new to me.
There are so many more degrees of freedom, which I can see Claude handled... mipmaps, subtle differences in lighting/positioning/compositing etc.
Even then, visual diffs were pretty flakey for web development, because one's OS and browser choice would slightly alter the exact pixels blitted to the screen. At least this was the case for the tests that would simply match pixels instead of computing a sort of visual hash.
It's also partly why some people preferred snapshot tests that compared the DOM tree instead, though that was brittle in other ways (e.g. tests would break if an application's frontend used a major UI library and an update to the library permuted the order of classes in some part of the HTML).
For web UI tests this is mainly solved, at least when using Playwright. It allows setting thresholds, percentages and some other config items to allow some small differences in pixels. https://playwright.dev/docs/test-snapshots#options
Using pixel data might be unusual, but anyone porting a game with a replay feature is going to realize it's an easy way to compare implementations.
I suspect it's inspired by gbaeval. Both have the "oracle" and other similarities. https://gbaeval.com/
Imagine the power of this magic superpower in the hands of business owners....
I would never have imagined a phone web browser capable of this in 1996 between playing Quake and testing out the hot new JavaScript powered mouse rollover image effects in Netscape Navigator 2.0
This may sound funny but I feel games would lose a lot of fun if they were all written in rust and had classes of bugs just not available to them. For better or for worse quirks and bugs in games have shaped how people approach games, and also have given games charm for decades.
Rust only prevents types of bugs where the game crashes because of invalid memory access, or exploits.
Nah it prevents all kinds of bugs that exploit things like integer overflows, invalid values, etc.
Obviously gameplay bugs are still possible in Rust, but many of them are not.
> ...that exploit things like integer overflows...
AFAIK Rust doesn't check for integer overflow in release builds, so that would still be exploitable.
Since when are Rust programs unexploitable..
Fortunately for gamers, Rust doesn't do anything to stop physics engines from going haywire or preventing players from clipping out of bounds. A Mario 64 written in Rust still has parallel universes (well, assuming that you translated the out-of-bounds float-to-short cast as a modulo, which is actually UB in the C implementation).
Rust does, however, kill Missingno. No thanks.
Damn, no results when I grep for 0x5f3759df in the source[0].
[0] https://github.com/terrapapagalli1516/quake-srp
That's for quake 3, the fast inverse square root was not present in quake 1.
ok this beats my doom minesweaper mashup [1] by far
[1] https://dreadsweeper.franzai.com/
Hah! That's great!
‘Safe rust’ just being used to mean ‘no unsafe’ is kind of a shallow understanding of the language. You can write code without unsafe that is not really ideal at all eg abusing vector/slice indexing to create a kind of interior mutability that the borrow checker is blind to. Which as a C port I’m going to guess it probably ends up doing
Almost hard to believe that someone using an LLM to mindlessly migrate software from one language to another for no practical reason at all would have a shallow understanding of those languages...
https://warweb.duckdns.org/
I like these
How? Was it a LLM-assisted rewrite?
https://github.com/terrapapagalli1516/quake-srp/graphs/code-...
It was indeed human assisted.
Mindboggling that people used to do this by hand.
They didnt though, because there was no reason too.
Of course it was.
No one does any porting by hand anymore. Without LLMs this port would never have happened.
probably done completely by LLM
Naturally
who's on first?
That's the man's name
That's darn nice. I didn't even have to push the turbo button to be able to run it on my phone.
I wholeheartedly bless this slop.
Found the repo in the Reddit post:
https://github.com/terrapapagalli1516/quake-srp
https://www.reddit.com/r/quake/comments/1x1ch14/quake_srp_sl...
This is cool. Do Quake 3 next.
indeed playable.
It's so smooth
It was smooth on a 90 MHz Pentium1 from 1996
Mr moneybags over here... True, but for mere mortals that was a pretty high bar. Speaking from experience from fighting a 486DX2-80
Someone had their resolution set to higher than a postage stamp I see.
What resolution is a postage stamp?
Code lives here - https://github.com/terrapapagalli1516/quake-srp (SRP stands for 'slop rust port'.)
I think 'standing on the shoulders of giants' is the phrase for something like this. It's the confluence of browser rendering, WASM, Rust, and LLMs. For me it's less a demo of what AI can do, and more a showcase of the human effort from the past few decades on the parts that needed to fall into place for an LLM to come in at the (relatively speaking) last second and claim a win. Sure, an LLM did the port from C to Rust, but think of all the things needed for it to all work. That's pretty damn amazing, and it wasn't done with AI.
The port to Rust was superfluous though, could've gone directly from C to wasm. But then you miss out being able to say Rust Rust Rust...
Vibe coders need to stop trying to use the browser as a platform for complex games. It's never going to work out and will always lead to a slow, unplayable, piece of shit. If you want to make a game then just make it a desktop program... There's a reason why literally every modern game is built in this way.
These aren’t JavaScript apps, they compile to WASM which actually has really good performance. Probably not as good as native assembly but for an old game it’s easily more than enough.
I spent five years on and off porting Neverball to the web by hand and waited for the perfect moment to announce it while tweaking the UX.
Fucking missed it.
https://play.neverball.org/
https://news.ycombinator.com/item?id=960369
Quake in Flash player from 2009 I think
Clean launch, good luck!
Safe rocket jumping? Unlikely.
I remember seeing the QuakeC line of code that halved self-damage from rockets. It enabled rocket jumping, but it also made the rocket launcher a much, much better close quarters deathmatch weapon than in Doom, so I thought it was a bit lame.
It's much safer now that it's written in rust
.... /s, if it needed to be said
We're in a renaissance, every day more and more games are just playable in browser: https://x.com/RadiantOpti/status/2108068632991256995
You can play Half Life right now, with one click.
Another slop project highly likely to be abandoned in about a month.