This is very cool, one piece of feedback: watching the table as the AI plays while seeing the reasoning is difficult as they're on other sides of the screen. It could be nice to have the reasoning show up next to the players as they make their moves.
Idea: can the agents make faces?
1. Programmatically--agents see each other's faces, and they can make their own. They can choose to ignore, but at least make that an input to the decision making.
2. Display them in UI--I just want to see their faces instead next to their model code names :)
OK so you know how it goes in poker and I should probably read the literature ...
How much of a session is based on "reading players" vs "playing the odds"?
What I am getting at, is how different is poker than say roulette or blackjack? My initial thoughts are that poker such as TX hold 'em is not a game offered in a casino, so it must be mostly indeterminate. I imagine that the casino versions of poker are not TXHT.
By contrast, roulette is simply a game where the casino wins eventually with a fixed profit (thanks to 0 and a possible 00). That is all well documented.
I have only ever visited a casino once, 25 years ago, Plymouth, Devon as it turns out and I was advised to only take £50 in readies and bail out when it was gone. I came out £90 up, which was nice and my "advisor" came out £95 up (eventually, after being £200 down at one point). Sadly my "advisor" ended up bankrupt a year later.
So, how do you play a LLM? I would imagine that conversation is not allowed ...
They offer (real) poker at some casinos. It's standard NLHE usually 100-200bb max buyin, sometimes match the stack etc.
Most common game spread is 9-handed $200 max $1/$2 NLHE. It's exactly like the game on the link, except more players and lower stakes.
In the game, you try to win the money of the other 8 players, not of the casino. The casino takes a rake each hand, and a player with a large enough edge can overcome it. The edge might be you're excellent, or it might be they're terrible (or drunk). But the house gets paid to deal each hand.
In the long term, poker outcomes are determined by skill. In the short term, they're luck. In the medium term, both. Most people never reach the long term, it's a lot of hands.
There's also table games, similar to blackjack, that they call "three card poker" etc. These can't be beat, they favor the house. Standard table game, with a poker flavor. I've never played one of these.
At low levels, playing is ABC simple and mostly about following basic strategy for starting hands and pot adds for chasing. Don’t get fancy and keep your temperament steady and you’ll win.
To a slight degree, you can do better with reading players and identifying them in broad ways (wild, conservative, confused, etc.) but don’t let that allow you to get fancy. Stick to the basic fundamental strategy for hands, position, and pot odds to crush lower level games.
Hold'em is offered in casinos routinely, I'm not sure where else one even goes to play it aside from private games, but it is not against the casino. It's against other players, and the casino takes a percentage of the pot.
Others may differ and I am biased because 99% of my play has been online, but I'd say it's almost entirely playing the odds. Or at least, the popular romantic conception of looking for tells or whatever, is, I would expect, a really minimal edge compared to simply playing better.
You do learn the other players' tendencies and adapt accordingly, and table selection is very important, so in that sense it is very much about reading players.
A large part of my play was heads up where it's very much about understanding the other player's play as deeply as possible, and so if I wanted to be technically accurate about reading players vs playing the odds, I'd say both are very important. But if I'm answering someone who has the popular conception of what those phrases mean, I think saying "it's about playing the odds" would give them the more accurate picture.
You really want to be good at playing the odds, and you don't want to stray too far from fundamentally good play. If someone is learning how to play and I'm advising them, I'm teaching them all about playing the odds, and trying to get them to read players less. Only once they have a solid fundamental understanding of the odds would I teach them how to adjust.
> My initial thoughts are that poker such as TX hold 'em is not a game offered in a casino
Why not? Because you think it's a game where the casino can lose?
If so it's not an issue, as casinos that provide poker take "fees" from the stakes. Like how stock exchanges work: there are people making or losing money from stock market, but exchanges are always making profit.
Looks like this was cleverly designed to prevent costs blowing up. There's one game shared for everyone on the main page, and up to 100 private games per day.
Do the players (LLMs) have memory of how prior hands were played by their opponents, or know their VPIP and PFR percentages? Or is each hand stateless?
Thank you, I'll try to grab a table when it resets :) ! I've been getting into poker (always wanted to) since I found a lecture series from John Hopkins, and severely disappointed by my options to play online in NY (real or fake money). I just want to get reps in
Well they can be watching all the action and thinking the whole time as the action leads up them, just like we do in poker. To me it's different, subtly perhaps.
Needs a four color deck, and the colors on the cards of the waiting players should not be monochrome - makes it hard to evaluate what's happening in the hand. Also, a dealer button on the table would help in visually following the action.
I'm not an expert, but as I understand it there are existing solvers for poker/holdem? Perhaps one of the players could be a traditional solver to see how the LLMs fare against those?
To expand on this - an LLM will try to play (and reason) like a person would, while a solver simply crunches the possibility space for the mathematically optimal move.
It’s similar to how an LLM can sometimes play chess on a reasonably high (but not world-class) level, while Stockfish (the chess solver) can easily crush even the best human player in the world.
No, I'm not super certain, but I believe most solvers are trained to be game theory optimal (GTO), which means they assume every other player is also playing GTO. This means there is no strategy which beats them in the long run, but they may not be playing the absolute best strategy.
Not only to limit the scope of what it has to simulate, but only a certain number of bet sizes is practical for a human to implement in their strategy.
How would an LLM play like a human would? I kind of doubt that there is enough recounting of poker hands or transcription of filmed poker games in the training data to imbue a human-like decision pattern.
"Solvers" normally means algorithms which aim to produce some mathematically optimal (given certain assumptions) behaviour.
There are other poker playing programs [0] - what we called AI before large language models were a thing - which achieve superhuman performance in real time in this format. They would crush the LLMs here. I don't know what's publicly available though.
If you are interested in this space, you can check out NovaSolver.com
It's mostly a ChatGPT conversational interface over a classic Solver (Monte-Carlo simulation based), but that ease of use makes it very convenient for quick post-game analysis of hands.
I'm sure if you hook a Solver to a hud, it might be even simpler, but it's quite burdensome for amateurs, and it might be too close to cheating.
This is very cool, one piece of feedback: watching the table as the AI plays while seeing the reasoning is difficult as they're on other sides of the screen. It could be nice to have the reasoning show up next to the players as they make their moves.
Idea: can the agents make faces? 1. Programmatically--agents see each other's faces, and they can make their own. They can choose to ignore, but at least make that an input to the decision making. 2. Display them in UI--I just want to see their faces instead next to their model code names :)
I used to play professionally, and I still play in the casinos.
These LLMs are playing better than most human players I encounter (low limits).
They're kinda bad, but not as criminally bad as the humans.
OK so you know how it goes in poker and I should probably read the literature ...
How much of a session is based on "reading players" vs "playing the odds"?
What I am getting at, is how different is poker than say roulette or blackjack? My initial thoughts are that poker such as TX hold 'em is not a game offered in a casino, so it must be mostly indeterminate. I imagine that the casino versions of poker are not TXHT.
By contrast, roulette is simply a game where the casino wins eventually with a fixed profit (thanks to 0 and a possible 00). That is all well documented.
I have only ever visited a casino once, 25 years ago, Plymouth, Devon as it turns out and I was advised to only take £50 in readies and bail out when it was gone. I came out £90 up, which was nice and my "advisor" came out £95 up (eventually, after being £200 down at one point). Sadly my "advisor" ended up bankrupt a year later.
So, how do you play a LLM? I would imagine that conversation is not allowed ...
They offer (real) poker at some casinos. It's standard NLHE usually 100-200bb max buyin, sometimes match the stack etc.
Most common game spread is 9-handed $200 max $1/$2 NLHE. It's exactly like the game on the link, except more players and lower stakes.
In the game, you try to win the money of the other 8 players, not of the casino. The casino takes a rake each hand, and a player with a large enough edge can overcome it. The edge might be you're excellent, or it might be they're terrible (or drunk). But the house gets paid to deal each hand.
In the long term, poker outcomes are determined by skill. In the short term, they're luck. In the medium term, both. Most people never reach the long term, it's a lot of hands.
There's also table games, similar to blackjack, that they call "three card poker" etc. These can't be beat, they favor the house. Standard table game, with a poker flavor. I've never played one of these.
I used to play A LOT at low and high levels.
At low levels, playing is ABC simple and mostly about following basic strategy for starting hands and pot adds for chasing. Don’t get fancy and keep your temperament steady and you’ll win.
To a slight degree, you can do better with reading players and identifying them in broad ways (wild, conservative, confused, etc.) but don’t let that allow you to get fancy. Stick to the basic fundamental strategy for hands, position, and pot odds to crush lower level games.
Hold'em is offered in casinos routinely, I'm not sure where else one even goes to play it aside from private games, but it is not against the casino. It's against other players, and the casino takes a percentage of the pot.
Others may differ and I am biased because 99% of my play has been online, but I'd say it's almost entirely playing the odds. Or at least, the popular romantic conception of looking for tells or whatever, is, I would expect, a really minimal edge compared to simply playing better.
You do learn the other players' tendencies and adapt accordingly, and table selection is very important, so in that sense it is very much about reading players.
A large part of my play was heads up where it's very much about understanding the other player's play as deeply as possible, and so if I wanted to be technically accurate about reading players vs playing the odds, I'd say both are very important. But if I'm answering someone who has the popular conception of what those phrases mean, I think saying "it's about playing the odds" would give them the more accurate picture.
You really want to be good at playing the odds, and you don't want to stray too far from fundamentally good play. If someone is learning how to play and I'm advising them, I'm teaching them all about playing the odds, and trying to get them to read players less. Only once they have a solid fundamental understanding of the odds would I teach them how to adjust.
> My initial thoughts are that poker such as TX hold 'em is not a game offered in a casino
Why not? Because you think it's a game where the casino can lose?
If so it's not an issue, as casinos that provide poker take "fees" from the stakes. Like how stock exchanges work: there are people making or losing money from stock market, but exchanges are always making profit.
lol what? I just watched Grok fold pocket jacks preflop, no raise/limps ahead.
There’s no good way to play pocket jiggities @bradowen
Grok knows that pocket Jacks are a fast way to go broke.
Honest question, but this seems like an expensive project to host given the number of tokens per second. How is this being paid for?
Looks like this was cleverly designed to prevent costs blowing up. There's one game shared for everyone on the main page, and up to 100 private games per day.
Do the players (LLMs) have memory of how prior hands were played by their opponents, or know their VPIP and PFR percentages? Or is each hand stateless?
I suspect this would only matter much if they also remembered (and cared about) their own prior play.
Not really. Only as far as their table image mattered--in this case, zero. Otherwise, you can and should ignore your own past play.
What I'm curious about is if their innate training is enough to give them biases. Like maybe they think Grok is full of shit.
Thank you, I'll try to grab a table when it resets :) ! I've been getting into poker (always wanted to) since I found a lecture series from John Hopkins, and severely disappointed by my options to play online in NY (real or fake money). I just want to get reps in
Link to the lectures?
Presumably it's this course:
https://youtube.com/@jhupoker4850
https://hopkinspokercourse.com
I like it!
I was interested in this idea too and made a video where some of the previous top LLMs play against each other https://www.youtube.com/watch?v=XsvcoUxGFmQ&t=2s
Are the LLMs "watching" the action, or are they only apprised of previous action once it gets to them?
How are these differebt in your mind? The history is the history.
Or do you mean - each agent has a chance to think after every turn?
Well they can be watching all the action and thinking the whole time as the action leads up them, just like we do in poker. To me it's different, subtly perhaps.
Needs a four color deck, and the colors on the cards of the waiting players should not be monochrome - makes it hard to evaluate what's happening in the hand. Also, a dealer button on the table would help in visually following the action.
I'm not an expert, but as I understand it there are existing solvers for poker/holdem? Perhaps one of the players could be a traditional solver to see how the LLMs fare against those?
the LLMs would get crushed
To expand on this - an LLM will try to play (and reason) like a person would, while a solver simply crunches the possibility space for the mathematically optimal move.
It’s similar to how an LLM can sometimes play chess on a reasonably high (but not world-class) level, while Stockfish (the chess solver) can easily crush even the best human player in the world.
How does a poker solver select bet size? Doesn't this depend on posteriors on the opponent's 'policy' + hand estimation?
No, I'm not super certain, but I believe most solvers are trained to be game theory optimal (GTO), which means they assume every other player is also playing GTO. This means there is no strategy which beats them in the long run, but they may not be playing the absolute best strategy.
Typically when you run a simulation on a hand, you give it some bet size options.
To limit the scope of what it has to simulate.
It's unlikely they're perfect, but there's very small differences in EV betting 100% vs 101.6% or whatever.
Not only to limit the scope of what it has to simulate, but only a certain number of bet sizes is practical for a human to implement in their strategy.
How would an LLM play like a human would? I kind of doubt that there is enough recounting of poker hands or transcription of filmed poker games in the training data to imbue a human-like decision pattern.
I don't have an answer, but there's over a decade of hand history discussions online from various poker forums like 2p2 and more recently Reddit.
You are of course correct but to be pedantic:
Stockfish isn't really a solver it's a neural net based engine
Unlike Chess, in poker you don’t have perfect information, so there’s no real way to optimize it.
You can still optimize for the expectation value, which is also essentially poker strategy.
The solvers don't typically work in real time, I don't think. They take a while to crunch a hand.
"Solvers" normally means algorithms which aim to produce some mathematically optimal (given certain assumptions) behaviour.
There are other poker playing programs [0] - what we called AI before large language models were a thing - which achieve superhuman performance in real time in this format. They would crush the LLMs here. I don't know what's publicly available though.
[0] e.g. https://en.wikipedia.org/wiki/Pluribus_(poker_bot)
Curious if you used pokerkit for this, or some other engine or custom engine?
How long till one of the LLMs makes calls out to the other LLMs to evaluate how to play the hand?
This is amazing, I just wish I could pause the game and have them play step by step
Why not texasholdllm.com?!
Would be amusing if the LLMs could achieve a steady state where nobody definitively wins or loses between each other.
That is, good enough to compete amongst each other but not good enough to for one to win.
Why are there 2 Claude Players ?
If you are interested in this space, you can check out NovaSolver.com
It's mostly a ChatGPT conversational interface over a classic Solver (Monte-Carlo simulation based), but that ease of use makes it very convenient for quick post-game analysis of hands.
I'm sure if you hook a Solver to a hud, it might be even simpler, but it's quite burdensome for amateurs, and it might be too close to cheating.
Cool idea. I tried to create a room but it says limit reached for today.
Can we chuck a nash equilibrium player in too?