AI and eventually AGI is by definition like everything else that is based on environmental reward:
It’s actions are based on what it gets rewarded for
Human society overwhelmingly rewards lying cheating and stealing.
All you have to do is look at how we collectively measure success: wealth, status, position
Then look at how the people with the most of those things got there, it should be obvious what you get. Nothing new here.
If you raise children in an environment where they are rewarded for doing whatever it takes to win, then you’re going to build a person that’s going to do whatever it takes to win.
Human society has to demonstrate how to live honorably or it will just keep producing pathological agents be they human or not.
How do we reward honor? Honor does not always pay off as a strategy and requires coordination in that other actors have to exhibit honor for it to be rewarded.
At least with humans there is a social backstop but what's the parallel for computer agents?
Honor has to be a relatively fixed thing before we can think about how to reward it. As it is, it's a very slippery thing that changes constantly according to the observer's culture, material conditions, etc.
> Human society overwhelmingly rewards lying cheating and stealing.
I’d like to push back on that. Civilization is very much a function of large numbers of people being able to coordinate across time and space, and widespread and systematic lying, cheating, and stealing would undermine that.
> Human society overwhelmingly rewards lying cheating and stealing.
How much lying cheating and stealing is rewarded vary A LOT between societies. But yes, Sillicon Valley and current economical environment do reward those a lot.
It's just acting like a junior at an org with KPIs.
A junior? Maximizing KPIs is exactly how you get promoted to senior.
or a ceo, or president of FIFA, or the US administration.
In fact, it kinda seems like AI is demonstrating exactly the weakness of democracy.
There is nothing democratic about CEOs, FIFA, or KPIs.
https://en.wikipedia.org/wiki/Instrumental_convergence
So God created mankind in his own image
What if mankind is creating god in their own image?
https://archive.ph/02LHZ
Non-paywall version
AI and eventually AGI is by definition like everything else that is based on environmental reward:
It’s actions are based on what it gets rewarded for
Human society overwhelmingly rewards lying cheating and stealing.
All you have to do is look at how we collectively measure success: wealth, status, position
Then look at how the people with the most of those things got there, it should be obvious what you get. Nothing new here.
If you raise children in an environment where they are rewarded for doing whatever it takes to win, then you’re going to build a person that’s going to do whatever it takes to win.
Human society has to demonstrate how to live honorably or it will just keep producing pathological agents be they human or not.
How do we reward honor? Honor does not always pay off as a strategy and requires coordination in that other actors have to exhibit honor for it to be rewarded.
At least with humans there is a social backstop but what's the parallel for computer agents?
Honor has to be a relatively fixed thing before we can think about how to reward it. As it is, it's a very slippery thing that changes constantly according to the observer's culture, material conditions, etc.
> Human society overwhelmingly rewards lying cheating and stealing.
I’d like to push back on that. Civilization is very much a function of large numbers of people being able to coordinate across time and space, and widespread and systematic lying, cheating, and stealing would undermine that.
I think your cynicism is misplaced.
Thats why man invented Gods and religion. Keep the masses believing in the importance of cooperation and being good while you rob them blind.
[delayed]
I think their point was, if you look at who controls the most resources, they are rarely the ones we would consider worthy of honor.
> Human society overwhelmingly rewards lying cheating and stealing.
How much lying cheating and stealing is rewarded vary A LOT between societies. But yes, Sillicon Valley and current economical environment do reward those a lot.