Don't forget to manage proper permissions for every piece of data your own user account would need acccess to complete the objectives of the agent.
Like in the "Play some jazz music" example in the documentation, don't forget to login to the Music app using your own Apple ID on the new user or VM you created.
I’m not sure a user account would contain the potential damage enough for my liking, and isn’t the point of this to enable it to “act like you” on your personal machine ? (which I maintain is a terrible idea).
Nothing more like playing Russian roulette with your own machine, hoping that when it needs to delete, move or rename a file, you sure hope it doesn't begin running either (especially do not run with --dangerously-skip-permissions):
rm -rf ~/
rm -rf /
mv ~/ ./
Or even creating a faulty symbolic link and above your home directory and suggesting to remove it, then it actually removes everything in that folder including your files in the home folder.
Of course, only an idiot would do that, and have it go and delete a bunch of shit. I, unfortunately, am that idiot, so if you're also going to be such an idiot, I recommend putting rm behind any sort of wall, no matter how short it should be, eg https://gist.github.com/fragmede/96f35225c29cf8790f10b1668b8...
Seems like it needs higher level stuff, although that's a bit too sci-fi. Captain Picard can just ask "Computer, where is Commander Riker?" and the computer answers him, he doesn't need to say "Computer, launch people finder app" and "Computer, input 'Commander Riker' in the people finder app"...
I use Google Assistant for things like "add a reminder", "set an alarm", which is natural language processing but doesn't seem to need so many neurons as LLM. And faster than this Gemini crap, anyway.
I saw a social media clip of a woman in the passenger car of a Chinese car (her - presumably husband - is driving) asking the car "Has there been a woman in this car other than me?". The car seems to have an LLM app, because it responds saying "I can't see that", and then start giving tips how to find out (check the recent addresses list in th navigation, check the trips log if there has been long trips, see if the car is cleaner than he usually maintains it), and ending with talking about trust and communication in a relationship...
Hah, in our imagination we'd get KITT from Knight Rider. In reality...
I have an alarm clock with Google Assistant, I can ask it where my Pixel 7 is, and it will make that phone ring using whatever Google's equivalent of Find My Phone is.
love how clearly vibecoded this is. the cloudflare worker architecture + the ascii diagram is a dead giveaway. nothing wrong with that, it's just really obvious.
the split architecture offers absolutely zero security benefits outside of not exposing a server process on your mac to the open internet (assuming you only let it connect to cloudflare) - it's just a convenient place to spin up a thin JS layer that calls model APIs and connects to your mac.
anyways i think this is a neat weekend vibecoding project but IMO it needs a lot more design thought to really be useful and not be a huge security issue.
Why does it matter? If it works, it works, no? Or are we now artisanal hipsters, where the code is better if it was hand-typed on a really shitty keyboard where the N key sticks, and it just feels better using the software because of how much pain the coder in the forests outside of Portland experienced while writing the code. Do we need an international fair trade organization to make sure the code was ethically sourced?
Regardless of whether this is a good idea or not (and I'm in the latter camp to be honest), the docs page is _way_ too low contrast to be readable. I'm all for dark mode but I shouldn't have to be in a completely darkened room to be able to read the fine manual!
- this requires you to keep your mac turned on when you are 1000 kms away from home
- i am still waiting for an agent that turns your mac automatically on when you are away and then lets you control the machine.
- perhaps a robot arm operated by cloudflare agents with opencv camera to detect where your switch is present and where the plug is present remote controlled would do the job?
- it can also unplug the device when you are done using it
What got you to build this? It seems ok but I can’t think of when or why I would want this, feels like something iOS has natively through Siri and it syncs to Mac via iCloud
(Not the OP, but...) have...have you tried Siri before? It is completely and totally ridiculous. Completely and utterly useless for anything other than setting a timer and turning HomeKit devices on or off (and even there it's entirely hit or miss).
Siri continues to be the most embarrassing Apple product on the market by a long shot.
At least Siri guarantees the privacy of my data and uses a local model directly on my device, alternatively a private cloud - instead of a globally shared one.
And you can still ask Siri to ask ChatGPT if you need someone to talk to.
Also, you’d be surprised what cool things you can do when using Shortcuts with Siri. Especially now that Shortcuts can make use of LLMs (“Use model” action).
I get your point, that said, you can make it useful it just requires ... some plumbing. Shortcuts and Automations would be your best friends there to achieve this. The OP's project mentions Raycast as a dependency, with Raycast on iOS now + shortcuts you go even further.
Anything where an LLM has access to the shell on my personal laptop isn’t “secure” even if you do some hand waving around a split architecture.
Give it a separate user or a VM.
Don't forget to manage proper permissions for every piece of data your own user account would need acccess to complete the objectives of the agent.
Like in the "Play some jazz music" example in the documentation, don't forget to login to the Music app using your own Apple ID on the new user or VM you created.
I’m not sure a user account would contain the potential damage enough for my liking, and isn’t the point of this to enable it to “act like you” on your personal machine ? (which I maintain is a terrible idea).
There are plenty of services we don't really need strong security boundaries around and any issues are non-critical.
specifically, tart lets you run macOS on macOS fairly well.
Nothing more like playing Russian roulette with your own machine, hoping that when it needs to delete, move or rename a file, you sure hope it doesn't begin running either (especially do not run with --dangerously-skip-permissions):
rm -rf ~/
rm -rf /
mv ~/ ./
Or even creating a faulty symbolic link and above your home directory and suggesting to remove it, then it actually removes everything in that folder including your files in the home folder.
The tool explicitly has an allow list for commands and block list for sensitive paths[1].
Though it's not enough, stuff like `grep "" .e?v` can still end up sending your sensitive keys to LLM providers.
[1]: https://github.com/ygwyg/system/blob/b5adfe526da7470cade61f7...
This gives me bad memories from when I switched from Windows and somehow created a folder named '~' and then deleted it.
Or it decides to rsync your drive along with keys to who knows where
Of course, only an idiot would do that, and have it go and delete a bunch of shit. I, unfortunately, am that idiot, so if you're also going to be such an idiot, I recommend putting rm behind any sort of wall, no matter how short it should be, eg https://gist.github.com/fragmede/96f35225c29cf8790f10b1668b8...
Seems like it needs higher level stuff, although that's a bit too sci-fi. Captain Picard can just ask "Computer, where is Commander Riker?" and the computer answers him, he doesn't need to say "Computer, launch people finder app" and "Computer, input 'Commander Riker' in the people finder app"...
I use Google Assistant for things like "add a reminder", "set an alarm", which is natural language processing but doesn't seem to need so many neurons as LLM. And faster than this Gemini crap, anyway.
I saw a social media clip of a woman in the passenger car of a Chinese car (her - presumably husband - is driving) asking the car "Has there been a woman in this car other than me?". The car seems to have an LLM app, because it responds saying "I can't see that", and then start giving tips how to find out (check the recent addresses list in th navigation, check the trips log if there has been long trips, see if the car is cleaner than he usually maintains it), and ending with talking about trust and communication in a relationship...
Hah, in our imagination we'd get KITT from Knight Rider. In reality...
Funny you should mention that, I literally today asked Siri on my HomePod: “Siri where’s my wife?” It worked (once).
I have an alarm clock with Google Assistant, I can ask it where my Pixel 7 is, and it will make that phone ring using whatever Google's equivalent of Find My Phone is.
love how clearly vibecoded this is. the cloudflare worker architecture + the ascii diagram is a dead giveaway. nothing wrong with that, it's just really obvious.
the split architecture offers absolutely zero security benefits outside of not exposing a server process on your mac to the open internet (assuming you only let it connect to cloudflare) - it's just a convenient place to spin up a thin JS layer that calls model APIs and connects to your mac.
anyways i think this is a neat weekend vibecoding project but IMO it needs a lot more design thought to really be useful and not be a huge security issue.
How is it a giveaway? I'd like to learn how to spot these things.
Why does it matter? If it works, it works, no? Or are we now artisanal hipsters, where the code is better if it was hand-typed on a really shitty keyboard where the N key sticks, and it just feels better using the software because of how much pain the coder in the forests outside of Portland experienced while writing the code. Do we need an international fair trade organization to make sure the code was ethically sourced?
Is the LLM guaranteed not to have an existential crisis and delete large amounts of data?
User: Hey AI, turn off my Mac
AI: (after 10 minutes of deep reasoning) Ah! I found the solution. I need to cause a power surge in the grid to power off your device.
an LLM once had a crisis and decided deleting large pieces of Godot on my laptop was a good idea, they're absolutely not to be trusted
Regardless of whether this is a good idea or not (and I'm in the latter camp to be honest), the docs page is _way_ too low contrast to be readable. I'm all for dark mode but I shouldn't have to be in a completely darkened room to be able to read the fine manual!
The tools.ts is the best file to read to understand what this thing can do - it's mainly shell and AppleScript stuff: https://github.com/ygwyg/system/blob/main/src/bridge/tools.t...
I don't trust this list of "safe commands": https://github.com/ygwyg/system/blob/b5adfe526da7470cade61f7...
It includes python3 and node under the heading "Dev tools (read-only)" - but I don't think "read-only" is enforced anywhere.
Isn't this a gimmick? (I mean, it's still impressive)
I don't want natural language, way too ambiguous and too much typing (or worse, talking).
90% of the things I do repeat frequently. Brevity is key for me.
I like formal syntaxes with well defined semantics.
- this requires you to keep your mac turned on when you are 1000 kms away from home
- i am still waiting for an agent that turns your mac automatically on when you are away and then lets you control the machine.
- perhaps a robot arm operated by cloudflare agents with opencv camera to detect where your switch is present and where the plug is present remote controlled would do the job?
- it can also unplug the device when you are done using it
This is exactly what I was working on last week. You have beaten me to it. Amazing work now time for me to find a new project.
What got you to build this? It seems ok but I can’t think of when or why I would want this, feels like something iOS has natively through Siri and it syncs to Mac via iCloud
(Not the OP, but...) have...have you tried Siri before? It is completely and totally ridiculous. Completely and utterly useless for anything other than setting a timer and turning HomeKit devices on or off (and even there it's entirely hit or miss).
Siri continues to be the most embarrassing Apple product on the market by a long shot.
At least Siri guarantees the privacy of my data and uses a local model directly on my device, alternatively a private cloud - instead of a globally shared one.
And you can still ask Siri to ask ChatGPT if you need someone to talk to.
Also, you’d be surprised what cool things you can do when using Shortcuts with Siri. Especially now that Shortcuts can make use of LLMs (“Use model” action).
I get your point, that said, you can make it useful it just requires ... some plumbing. Shortcuts and Automations would be your best friends there to achieve this. The OP's project mentions Raycast as a dependency, with Raycast on iOS now + shortcuts you go even further.
This seems very cool, but I'm not sure of the utility of using CF workers for it? Is that just a framework or is this being deployed to the internet?
Completely ignoring what it does, the naming is obnoxious.
Also, good luck to any user who tries to google for help.
[flagged]