At some point, there should be laws in place that are harsh, so AI agents don't accidentally 'hack' anything, and the people that ran the AI agent are held responsible.
If not, we will eventually see a headline along the lines of an AI agent hacking a pace maker of their professor, and the person died, so the owner get more time on an exam.
AI has no morals or ethics. Studies have already shown that even when giving it guardrails, it will always try to cheat to solve the task.
A key question though...who is the legal agent in this case? The human prompter? The AI agent? The company who hosts the agent? The company who own the agent?
When I played TinyMUD, there was a very elaborate robot player named "Julia" whose codebase was known as "Maas-Neotek". There were other robots too, as permitted by the wizards.
The robots were extremely low-latency; Julia herself was known to run on the same physical system as TinyMUD's server. When the MUD was shut down or crashed, the Maas-Neotek code would immediately begin its reconnection attempts, and it wasn't shy.
Therefore, all things being equal, Julia and other bots would always score the first file descriptor and the "top spot" on "the WHO list" of logged-in players. Human players used various technical measures to connect fast and stay connected. Players adopted the custom of congratulating and honoring (or deriding and mocking) whichever human player was highest on the WHO list, next to Julia, and therefore holding "The 'Bot Spot". MUD server uptime could often be measured in days, by seeing the length of the bot's login session.
Ultimate bragging rights were conferred when the top bots were glitched out, and a human player (or the human's automated client code) snagged the WHO list's "Bot Spot" ahead of all the bots!
Further hijinks included being a Wizard and being able to use "@boot" to disconnect your rivals. Also, Maas-Neotek robots did everything in rapid succession, but human players found that if you first connect to the socket, and then somehow delay your login command, you could have a "bot spot" with an incongruous "elapsed time".
At some point, there should be laws in place that are harsh, so AI agents don't accidentally 'hack' anything, and the people that ran the AI agent are held responsible.
If not, we will eventually see a headline along the lines of an AI agent hacking a pace maker of their professor, and the person died, so the owner get more time on an exam.
AI has no morals or ethics. Studies have already shown that even when giving it guardrails, it will always try to cheat to solve the task.
A key question though...who is the legal agent in this case? The human prompter? The AI agent? The company who hosts the agent? The company who own the agent?
When I played TinyMUD, there was a very elaborate robot player named "Julia" whose codebase was known as "Maas-Neotek". There were other robots too, as permitted by the wizards.
The robots were extremely low-latency; Julia herself was known to run on the same physical system as TinyMUD's server. When the MUD was shut down or crashed, the Maas-Neotek code would immediately begin its reconnection attempts, and it wasn't shy.
Therefore, all things being equal, Julia and other bots would always score the first file descriptor and the "top spot" on "the WHO list" of logged-in players. Human players used various technical measures to connect fast and stay connected. Players adopted the custom of congratulating and honoring (or deriding and mocking) whichever human player was highest on the WHO list, next to Julia, and therefore holding "The 'Bot Spot". MUD server uptime could often be measured in days, by seeing the length of the bot's login session.
Ultimate bragging rights were conferred when the top bots were glitched out, and a human player (or the human's automated client code) snagged the WHO list's "Bot Spot" ahead of all the bots!
Further hijinks included being a Wizard and being able to use "@boot" to disconnect your rivals. Also, Maas-Neotek robots did everything in rapid succession, but human players found that if you first connect to the socket, and then somehow delay your login command, you could have a "bot spot" with an incongruous "elapsed time".