How I Built a Working Online Poker Bot, Part 8: Poker Hand Recognition, Comparison, Enumeration, and Evaluation

Thursday, August 21, 2008

Introduction

Thirty hands an hour.

When you're dealing a real-money game of No Limit Texas Hold'em, that's what you're shooting for. Thirty hands an hour, or roughly one hand every two minutes, like clockwork. Two minutes to wash, shuffle, cut the cards, deal them out in two concentric rings. Two minutes to gently coax a table of beer-drinking adults through four rounds of betting, keeping your sanity in the midst of two and three-way all-ins, split pots and side pots, pot bets and string bets, kills, half-kills, straddles, and other inventions of bored-out-of-their-mind poker players.

"Run it twice, dealer."

After a while, it becomes automatic. You get to where you can eyeball a stack of chips and tell it's one chip short of twenty. You get to where you can cut three, five, eight chips from a stack in one clean motion. Your dealing improves, as your hands learn the curt yet unhurried, almost lazy rhythm of high-throughput dealing. Wash, shuffle, shuffle, shuffle, cut, don't forget the deck guard, go. Two months before you were sending cards flying over the table rails in curlicues, now you're tossing them with pinpoint accuracy.

Like shuriken.

The rituals of the poker table are a kind of language, and as a fluent speaker of this language, you don't stop and think about what the individual words mean, or whether the grammar is correct. The words and the meaning they convey are one and the same. To hear the words is to understand them, but at the poker table, your "words" are bets, actions, the casual tossing of two useless cards into the muck, the spill of meaningful color we call the flop, the turn, the river, whatever. Symbols. Grammar. Inflection. Punctuation.

Language.

Think of a Texas Hold'em hand as a seven-letter word built from a 52-card alphabet. Consonants and vowels. In combination, they have meaning. One of your jobs as a poker dealer is to figure out when one person's "word" is better than another person's, and push the pot accordingly. As a poker player, you have to take things a step further. You have to understand the word. Not just its specific meaning, its denotation; but its connotations, associations, nuance, history. And you have to do all of this in the blink of an eye, automatically, without thinking.

It's almost robotic. 

Hand vs. hand range computation

Speaking of robots, this is the kind of stuff at which poker robots excel:

  • Hand Recognition
  • Hand Comparison
  • Hand Enumeration
  • Hand Evaluation

These terms are often used interchangeably, but I like to give them specific meanings.

  • Hand Recognition. Figuring out what a poker hand "is": what hand category it belongs to, the number of outs it has, it's equivalence class, and so forth.
  • Hand Comparison. Taking two or more poker hands and determining the winner.
  • Hand Enumeration. Enumerating all possibly combinations of cards, whether the cards in a player's hand, or the cards on the board, or both.
  • Hand Evaluation. Computing the win/loss percentage or pot equity for two or more hands or hand ranges.

And of course, it goes without saying that robust hand evaluation is the stepping-stone to a mature poker A.I. If your poker bot can't do this...

PokerStove

...it's fighting with one arm tied behind its back.

That, by the way, is a screenshot of Andrew Prock's PokerStove, a poker calculator capable of performing hand vs. hand-range analysis across multiple opponents, using exhaustive exploration or Monte Carlo simulation. There are many such tools out there, but PokerStove is the most popular, and probably the most functional, of the lot. PokerStove is free, so if you haven't already done so, I suggest downloading it and getting familiar with how it works.

Because we're going to teach our poker bots and related tools how to do what PokerStove does.

Subjective All-In Equity (SAIE)

Notice I said hand vs. hand-range analysis. The "range" part is crucial, because in poker, rarely are we able to put our opponents on specific hands. Instead, we assign them a range of hands, and narrow that range as the hand progresses. In order to compute our equity, we have to figure out how well our known hand fares not against another known hand, but against the range of hands we believe our opponent has. This is what's referred to as subjective all-in equity or SAIE.

I like Michael Maurer's definition:

SAIE is a player's pot equity given particular beliefs about the possible hands of the opponent(s) and assuming no further betting.

Provided we follow it with Andrew Prock's note in the PokerStove FAQ:

This is not the chance that a hand will win the pot. Rather it is the fraction of the pot that a hand will win on average over many repeated trials, including split pots. The equity for a hand is calculated by dividing the number of "pots" that the hand won by the number outcomes considered. Because two players can split a pot, a player can win fractional pots. Thus, it is possible for a hand to have non-zero equity despite the fact that it cannot win.

For example, in the above screen shot of PokerStove, we can see that the equity of pocket Jacks versus two opponents is around 36%, if we assign each of those opponents a range of [any pair; any AQ or better]. If you took this situation and played it out a million times, you'd find that on average, you'd win 36% of the pot (assuming no further betting, and assuming nobody folds).

This is a useful number to have.

It takes into account the fact that, if one of your opponent has Aces, and the other has Kings, you're equity is lower—only 15%. It takes into account the fact that, if one opponent has AK (any AK) and the other has AQ (any AQ), your equity is higher—about 44%. SAIE provides a sort of weighted average of how your hand fares against each of the hands in your opponents' range, taking into account the likelihood of each specific opponent holding. You see, it's quite a bit easier for an opponent to have AK (12 combinations) than it is for him to have AA (only 6 combinations). And the mathematics need to allow for that.

Once again: this is not the same thing as your chance of winning the pot! Another example should make this clear.

PokerStove

Here, Player 0's chance of winning the pot is only 11%, but his equity is around 57%. This is because most of the time—whenever a 4 or higher falls on the river—the hand is going to be tied. But when a 3 or 2 falls on the river, Player 0 wins with Four Aces and a Four kicker, which beats Player 1's Four Aces and a Three Kicker.

And although Player 1's pot equity is 44%, he has zero chance of winning the pot outright.

That's a contrived situation, but an illustrative one. By thinking in terms of pot equity, rather than chance of winning the pot, we neatly fold all those messy tie-game situations into our mathematics.

The Need For Speed

Believe it or not, calculating SAIE is easy. Calculating it efficiently may not be.

It all hinges on hand comparison: the ability to compare two or more poker hands and determine the winner. If you can do that, all you have to do is programmatically deal out large numbers of hands and start tallying wins, losses, and ties. When it comes to dealing those hands out programmatically, you have two options:

  • Exhaustive Exploration. Enumerate all possible outcomes: all the combinations of cards fitting your opponents' range times all the possibly combinations of board cards. Keep a tally of how many hands each player wins, loses, and draws. Afterwards, when you've enumerated every possibly combination, express these as a percentage.
  • Monte Carlo Simulation. Randomly generate hands fitting your opponents' ranges and board cards, keeping a tally of wins, losses, and draws for each player. Over many trials (hands or games), your results will start to converge.

Typically in the game of Hold'em, we use Monte Carlo simulation for preflop matchups, and exhaustive exploration for matchups on the flop or later. You'll notice PokerStove has a radio button allowing you to specify which technique to use although, as Andrew Prock notes:

Monte Carlo simulations work very well for most practical problems, and enumeration driven software is only going to be of interest to the true poker geeks who must have all of their i's dotted and t's crossed. Of course, I'm one of those geeks :)

Whether you're a geek or not, you're going to need throughput: the ability to evaluate large numbers of hands quickly. If you're going to be randomly generating, or consecutively enumerating, millions of hands, you can't afford to do four bubble sorts, nine redundant array copies, a Google search, and send off 3 emails every time you evaluate one of those millions of hands.

You've got to optimize.

(Remember, this is something poker bots and other tools need to do in real time, at multiple tables, on various and sundry hardware.)

So it's got to be fast.

I don't mean 2,000 hands per second fast. That's not fast, that's slow. I mean fast like a bat out of hell. Fast like the wind. Fast like the Flash. Speedy Gonzales. Run Forest Run! fast. Or even—dare I say it—as fast as that paragon of speed and grace, the Roadrunner.

The Roadrunner

MEEP-MEEP!

And the best part?

This is not a wheel you have to reinvent. Highly-optimized hand evaluation code is already written, and is publically available in multiple languages (C, C++, C#, Java, and Python, among others) with support for multiple poker variants (Hold'em, Omaha, Stud) provided you know where to look.

A Small Army of Poker Hand Evaluators

Some people collect butterflies, leaves, flowers; others collect coins, marbles and shot-glasses.

Me? I collect open-source poker code.

If it's code, and if it's about the game of poker, and if it's freely available, I want a copy. Even if the code is written in a language I'll never use. Even if it's slower, messier, or buggier than my existing code. A lot of clever people have devoted a lot of time and energy to building robust poker hand evaluators, you see, and I want to leverage their hard work.

So every time I stumble across a new specimen, I collect it. I haul it out to the garage, open my toolbox, and start hammering away. I clean up the code, throw it into a Visual Studio project or who knows, I might even crack open my Eclipse IDE and do some Java work. Whatever.

There is no "one size fits all" poker hand evaluator, so I want a hand evaluation library which incorporates multiple evaluators within it, all of them written by other people.

Here are just a few (all of which we'll be covering in detail):

  • The Pokersource Poker-Eval library
  • The Java enumeration library built on top of Pokersource
  • Keith Rule's C# port of the Poker-Eval library
  • Cactus Kev's 5-Card Evaluator
  • Paul Senzee's Improved 5-Card Evaluator
  • Paul Senzee's 7-Card Evaluator
  • Moritz Hammer's Directed Acyclic Graph-based 7-Card Evaluator
  • Steve Brecher's HandEval routines
  • The Two Plus Two 7-Card Evaluator

So instead of thinking of your poker hand evaluator as a single tool, think in terms of building a poker hand evaluation toolset; a collection of code which can be applied to the specific problems you're facing. Even within the relatively narrow context of an online poker bot, there's a lot of wiggle room. Are you building a multi-table online Texas Hold'em poker bot? If so, you need a fast 7-card evaluator. Are you building something which needs to support Hold'em as well as Omaha and 7-Card Stud. You need a good generic evaluator. And so forth.

If you think of poker hand evaluation as a smorgasbord, rather than a single component, you'll be doing yourself a favor.

Conclusion

Now, I know what you're thinking: where's the code? Where are the algorithms?

Well, it turns out that poker hand recognition, comparison, enumeration, and evaluation are such complex topics, it's impossible to fit them all into a single post, and at the same time, provide and explain the source code for a dozen different pre-existing evaluators. So I've broken these off into a set of background articles, to be published shortly:

  • Poker Hand Analysis, Part 1: Theory
  • Poker Hand Analysis, Part 2: 3rd-Party Evaluators and the XPokerEval library
  • Poker Hand Analysis, Part 3: Hand Ranges and Multiple Opponents

Today, I just wanted to introduce the subject, and tell you how you can make some money by writing hand evaluation code.

Coding the Wheel, in conjunction with a site to be named shortly, will be presenting an award of X dollars (we're not quite sure what X will be yet, but it will be hundreds of dollars, rather than thousands or tens) to the person or persons who can produce a substantial improvement to the hand evaluation code we'll be presenting.

In addition, that person's solution will be highlighted, with full credit, or anonymously if preferred, in a dedicated post on Coding the Wheel and possibly some other sites as well.

There are only a few requirements:

  • The solution must be unpublished
  • The solution must evaluate 7-card poker hands
  • The solution must be fast (noticeably faster than current evaluator implementations)

Full rules to follow. Stay tuned!

Tags: poker hand evaluation, poker bot, probability, online poker, poker

107 comment(s)

First!

Nice intro James, this article is worth it just for the links to 3rd-party hand evaluators. I'd seen a few of those but not all of them. Looking fwd to the contest...

In addition to exhaustive exploration and Monte Carlo-style simulation, you can use precomputed lookups for diff situations (as you probably know) but I think PokerStove uses preflop lookup tables, for certain matchups anyway.

Nice to see you back - you know, these postings will make a great book!

Also - the link http://www.codingthewheel.com/how-i-built-a-working-poker-bot doesn't work...

I'm definitely gonna have a go at the hand evaluator "competition", you gotta love algorithms. :D

I guess your latest few articles didn't get enough juice so you had to pump out another poker one. anyone that has taken your first ones to heart and have started making a bot will benefit zilch from this article.

it sounds like you have a half-working bot and you want to pay someone to come up with your algorithms for you. try rentacoder

Sorry Bismark, it ain't so. The reading public will have a very hard time suprassing the mentioned algorithms. Some of them are completely awesome.

Bismark, go troll elsewhere please. If you've read any of the articles you would see how much one could gain by implementing this into their homebrew poker bot.

Do you have an arbitrary environment to test it in?

James I agree with Jordan's question. How will you test the solutions? Will you run them on a dedicated machine? Enumerating all hands, generating random hands, both?

I suggest using the 2+2 method. Test the evaluator by enumerating all hands, then test it again by spawning 1 million random hands. Run each evaluator three times. Take the best result.

I AM ALL OVER THIS CONTEST. YOU GUYS MIGHT AS WELL JUST PAY ME THE MONEY NOW ;)

Given a test environment that I can build, I might be able to tweak my program a little bit more.

While AI/algorithms are really cool and I couldn't wait to dive into them, I can't help but feel we left the input/output section a bit too early. I still am unable to get the hole cards or all the table state from the current hand on pokerstars. My poor bot is still blind.

yeah, I haven't started on that (yet). I haven't finished reading Windows via C/C++ by Richter. So I'm reading the book, and programming the calculators. That way I don't get too bored on one task. There's top down, and bottom up... but for some reason I'm doing both. Just gotta make sure it meets in the middle.

I already win at poker, so I think I can make a decent AI later on. This project will definitely strengthen both poker and programming skills, and you can never get too much of either.

Does anyone know how they catch WoW bots these days? I was thinking that might be even more fun. I just wish there was a guide to the history of the cat-and-mouse regarding WoW.

This is a hand evaluator contest that we ran. All java sources are up for download and prepared with the same example so easy to get them running (and no half working stuff). I don't think anyone can beat RayW 7-card evaluator, in any language or by any approach, so the topic is pretty much closed. http://pokerai.org/pf3/viewtopic.php?f=3&t=16

hi james,

  • ¿Because create your own detoured.dll, because we do not use that comes with Detours ?
  • ¿Detourcreatewithdll have to be in a dll.?

"And how well is RayW optimized for multi-core?" -- RayW code is just a table lookup, you can't go any faster than that. It runs for 146,000,000 evaluatons per second. On a multi-core, for full evaluation purposes, you can lookup in several threads, and that's pretty much all you have to do. This isn't available as coding, but is a possible extension that one can easily do.

For a single hand evaluation there is no way to speed this up, as it's really mostly a table lookup, and this is pretty much a "single instruction operation".

P.S. By the way, there is another contest we run since some time with 1k$ reward, for preflop SAIE: http://www.pokerai.org/pf3/viewtopic.php?f=3&t=869

I'm guessing as usual (never have time to lookup/check my statements) But a shouldn't a algorithm running in L1 cache should be faster than a lookup table?!

The porting to C actually achieved 450m+ evaluatons per second. This means that it took less than half of a second to run ALL possible 7-card evaluations.

i've looked at all (okay, almost all) of these evaluators and one thing i never seen any benchmarks for is p. senzee's 7-card evaluator (see link) with the 52c7 table etc. anybody have stats on this?

and I agree with nyx. l1 cache should (might) be the fastest way to go..and other working set optimizations.

oh and also, cactus kev talks about being approached by one of the major poker software co's and asked for his algorithm, and whether it could be ported to 7-card. whatever became of that?

All seems very complex. Putting people on a range of hands is hard at the best of times let alone getting a robot that can always do the correct +EV move.

Hey James, can you tell how many tables u use to play when playing 30 Hands a hour? c u popo.

Angry, I don't think Senzee's 7-card evaluator is faster than the 2+2's lookup. I read that entire thread and lol. Can 1 programmer do any better? But I would like to see some hand vs. hand-range benchmarks which use that evaluator..

I understand you're just doing an intro here but I really hope the next two parts are hard-hitting like parts 4-7 were. I would say the best posts of the series are 1, 4, 5, 6, and 7. So far.

To win at poker, you don't have to make +EV decisions all the time. All you have to do is make less theoretical mistakes than your opponents make.

8 bits a card, 7 cards, and the size of the data pool isn't really an issue.. Can we do this thing with pointer arithmetic, theoretically? There would be 64 C 7 = 621,216,192 64-bit pointers. Find the base pointer, add it up. Using 64 cards instead of a standard deck of 52 side-steps complex combinatorial arithmetic. We're speaking the poker language, but also the computer language.

Step #1. Convert the seven cards into a 448 bit number (7 conjoined 64-bit twins) Step #2. Find the base pointer. Step #3. Add the pointers together.

This is after you have everything already stored. We waste raw memory for quick arithmetic. Am I wrong?

actually, a 64-bit number would hold an entire deck. Duh

Jordan, isn't that similar to what the 2+2 evaluator does ie, start with the 1st card, look it up in a 52-element array. From that array element, you have 51 vectors to the next array element/next card. And so forth, each card in the hand mapping to another 52-element subarray until you get to the last card, and there's your equiv. class. Is that similar to what you're suggesting

I'm not explaining very well, but I do have an idea that should save time compared to the

int p = HR[53 + *pCards++]; p = HR[p + *pCards++]; p = HR[p + *pCards++]; p = HR[p + *pCards++]; p = HR[p + *pCards++]; p = HR[p + *pCards++]; return HR[p + *pCards++];

but, I'm going to finish it before I say anything more in case there is some contest money. :-) I'll run some tests. Does anyone have a slick method of generating every single combo for test data? I mean, something cool! Something better than nested loops! I'm going to check out boost.

When does contest end?

Hey! What's wrong with nested loops?

James, I respect you and based on your articles I can tell you're just a smart guy who loves the game of poker and programming. Okay? Now let me tell you the problem I have with Coding the Wheel.

You had a huge (unheard of) success with the first post in this series, and you did what anybody would do: you turned those initial readers into a community. Fair enough. I don't blame you for this. If I though you were some sort of scam artist, I might, but I don't.

Now you're happily writing away about every aspect of botting from the mechanical and, with this post it looks like you'll be turning to the A.I. You've got anywhere between the #1 and #20 spot for "poker bot" on Google. Occasionally, you're ranking higher for that term than MSNBC articles, than WinHold'em, etc.

And your site is how many months old?

Now I'm sure, in your head, you don't think you're doing any harm. As you state before, you're not necessarily giving out any information that's not already out there (actually with some of the DLL injection stuff I think you have, but that's beside the point).

But I want you to consider what it means when every poker-bot-wannabe-script-kiddie stumbles across your site and sees, aha.. so THAT'S how they do it. This post is a prime example: take a script kiddie who probably never would've written anything more complex than a basic hand evaluator and tell him Hey! You can get the world's most powerful SAIE hand evaluation code for free! Just check out these links and if that doesn't help, wait a couple posts and I'll give you the source code!

It has the effect of turning incidental botting enthusiasts who never would've done much harm into guys that actually have the tools to get, okay maybe not a WINNING bot up and running, but a foldbot, a trashbot, whatever.

and all this at a time when we're trying to deal with UIGEA and regulation....

Furthermore, since you're not actually selling a botting product, there's no way for the sites to fight back. Your code and ideas will be taken and converted into a hundred different tools, each with a different name, a different set of goals. The sites can't spend time chasing down hobbyist bots, but it's the hobbyist bots that will ultimately ruin the game, not the big commercial bots (most of which are crap anyway).

So as a guy who enjoys your writing, I have to say: it's okay to give up the botting series and just write about normal stuff. Your community will stay with you. And don't you get tired of hashing this stuff out endlessly?

My favorite post of yours to date? The short, random ones about random computer/technology subjects. Not the botting posts, although those are interesting.

Give what I've said some thought and think about drawing the botting series to a close. You've done a good job so far, no need to knock it out of the ballpark.

From what I've seen the online game is already somewhat ruined, and good rittens (coming from a mid-stake live player)! It's frustrating since poker isn't what it was 5 years ago, understood. But, in my humble opinion, online poker deviates from what truly is poker. Besides, every wanna-be math kid has already given poker a try, and many of them made it (although many more of them did not).

Besides that, online poker sites don't do nearly enough to protect the players from bots. They should either make bots legal and regulated (Vegas style), or ban them altogether and use more sophisticated agreed-upon spyware to keep them out.

Furthermore, there is a lot of different directions one can go with this. I'm actually constructing a simulator, and don't take much use so far out of the articles themselves, but from books on the STL and boost, and Windows C++. Just because I learn all of these techniques taught on this site, it doesn't automatically mean I start rolling out bots.

And even still! It's going to be damn difficult to make one win. But, I think the grossest part of it all is that THE PUBLIC IS UNAWARE!!!

Now, you want to keep all of these secrets to yourself. I respect that too. But lets face it, a large chunk of the bots will get caught unless the builder has extensive knowledge on a large variety of topics. AI is an important topic that REALLY HAS NO VENUE so far, except for like airport security facial recognition, and biometrics. If you want truly intelligent machines in the future, I think that the best way is survival of the fittest.

I've played against bots 6 tabling $50-$100 sit 'n gos on Stars. He had an avatar of a man on an easy chair (looking for the conservative image) and played a very binary game. I always figured him as a bot. I never emailed Stars. You know what I did? I STOLE HIS BLINDS! And I'd also call him if he was committed to push on like the button or something. It was also fairly trivial. Of course he was a regular, but that didn't stop me from REGULARLY STEALING HIS BLINDS!

Of course, he might have just been the biggest most predictable nit in the world, but somehow I doubt it. And realistically, it doesn't make a difference, bot/nit mean just about the same thing to me.

Besides, this all could have reasonably been predicted as soon as the poker literature was kept out. The elite secrets of poker are few and far between. It is inevitable.

I agree that you should give the input and output stage methods and leave most of the rest of it alone. I actually want to figure this out on my own but my windows skills are weak (I'm getting the Windows via C/C++ book tomorrow). So it's great to have the very windows specific stuff from the input and output stages in there.

One thing I've realize is it's way better to get the input stage and logic going with a test app. Play some real hands, store all game text and other stuff to files. Then have a test app send that stuff to your bot like the poker client would. You can test pretty well doing that without having to debug your bot while the live PokerStars is running.

question about detection...

can't a poker client detect that the detoured.dll is in its working directory? that would be a telltale sign there's a bot. PokerTracker and the like don't do that.

I'm going to change the signature and name of that dll in my bot(if it works).

detours, or whichever support injection lib you use... isn't in the poker-site's working directory/folder. You compile an app that includes a dll that will be in your dir. When you run your app you'll inject your dll into the poker-sites address space. dll injections happen all the time on a PC. They could potentially determine that something is loaded into their address space... if they knew what to look for, and if they cared enough to look.

There are a few hundred poker-sites. Of all of then... there are just two that care enough to look. But they look for big name known botting apps, and some stat collection apps, and some sundry other poker support software that they deem to be in violation of their site guidelines.

A homegrown bot that may or may not work some day, that may or may not break even some day, that could have one name today and a different name tomorrow... just isn't a real threat... and just isn't worth their time. And we've all used different programming techniquesto get our code-projects working.

Now if you make a super great bot, and sell a few thousand copies of it, and it gets used in collusion rings and such... then, maybe, it will be worth their time. But worry about that then. In the meantime maybe your code-project in progress just needs to get semi-working first.

Hi, James.

There are some voices asking you to stop the bot series. Well, I ask you not to do it.

Since I discovered this site, a month ago, I have:

  • Learnt some "visual" C++, much different than the "standard" C++ I knew before.
  • Learnt some C#.
  • Learnt to make and use DLL.
  • Learnt to inject DLL and call its functions.
  • Learnt to use Detours (which I have just managed to get working).

And, in a near future, I'll learn to simulate human behaviour.

It's been all because of this series. Please, keep on it.

Yeah, I've learned a lot so far. I encourage you to keep going as well. I look forward to your writings every day.

To the anonymous guy who says...

[quote]... it's the hobbyist bots that will ultimately ruin the game, not the big commercial bots (most of which are crap anyway).[/quote]

I don't get this. I don't see the connection between bot proliferation and the ruination of the game. Crappy bots will add to the pot for players and add to the rake for poker rooms.

>[i]Give what I've said some thought and think about drawing the botting series to a close.[i]

Why? James' posts are fantastic to read and really draw you into the subject where most sites fail miserably. If the online poker industry fails because X number of people learn to make a homebrew bot from this site, they have more problems than just this. If the online poker industry is never challenged it will become stagnant, stale, and eventually fall apart. Challenging forces them to come up with new strategies, new ideas, and ultimately brand new inventions that might never have come about.

Nothing but good can come from this site!

Yeah, of course I agree w/ you Adam. I'd also add that since poker is not so strong right now, it's not such a bad idea to "retool". I wouldn't say retool or GTFO, but ya you know, retool. :-)

oh, and my idea for the contest wound up being really foolish (surprise surprise) maybe nested loops will do after all.

I made an algorithm, long ago, for my own use. It evaluates the hands just like a human could do: orders the 7 cards from bigger to lower (qsort), look if the first two cards are a pair, then look if the third card makes a third-of-a-kind or not; then look if the 4th card makes a four-of-a-kind or double pairs, and so forth.

It means that, once the cards are ordered, I only have to look each card once to know which hand it is and get its value. So, there are no nested loops and everything is done in a single step.

It's not slow: about 1.2M hands per second (in C), and was enough for my needs so far. Could be improved (making it return faster when having hands that are more probable, things like that), but it simply was good enough.

I'm not going to try to win the contest; I really don't think I can play against the above algorithms (which I hadn't heard about before). But I will stop using my own algorithm and use the winner instead. :-D

mine takes a 64-bit unsigned integer, and returns a 16-bit integer. There is one bit for each card. Barry Greenstein has one on his site, but it's not very efficient. It's TLC, lol.

I only have to cycle through the 64-bits one time, and then I have all the information on where the pairs, trips, quads, straights, flushes are. It wouldn't take too much modification to turn it into a low.

So, if you had 7 cards, you'd pass it like this Eval( 1 << card[0], 1 << card[1], 1 << card[2], 1 << card[3], 1 << card[4], 1 << card[5], 1 << card[6]);

No repeats! Huzzah!

I also have one of the traditional way, an array of cards. I don't know how memory works exactly, but my thinking is that you could have a 64-bit pointer to 16-bit unsigned integers and have it work insanely fast.

The catch is there isn't enough space in RAM to hold 2^52 64 bit pointers. But it might be useful to send messages this way from process to process. Didn't say it was a good idea, but it is an idea. :-P

jordan that's similar to what pokersource uses, a 52-bit bitfield one bit per card. then a bunch of masks and bitwise manipulations. it can be very fast but the 2+2 approach (cards are whole numbers 1-52) is i think even faster...

Do they do something like this?

&00000000000000001000110100000110000000000000001000000000010000000

where the memory location IS the bits that represent the cards? I'm no expert by any means, but if that could be done in RAM, that would boost things significantly.

what I find most promising are the suggestions in the 2+2 thread which were never implemented. Tweaking the size of the lookup table.. 130MBs seems puny. Surely we could get more speed by doubling the size of that table, or etc finding the right memory size vs. speed ratio for max performance.

Ed, I couldn't agree with you more. Memory is the closest thing you get to a free lunch. Obviously, you can't look-up table all of it, but I would think we could test a few different engines to see what works best.

There's going to be some choices on how you want to allocate the memory best. A chunk needs to be given to the look-up tables for fast look-up, the AI, and the mechanics.

So, what's up with the contest???

Thanks for the hand evaluator links. There were a few of those that I hadn't previously stumbled across :)

I look forward to seeing the contest results and the next article in the series.

[url=http://www.icmbot.com]poker bot[/url]

some of the above evaluators are hard to build, for me at least.. just spend 4 hours trying tobuild the 2+2 / ray w. evaluator. finally got it but... sheesh.any idea when's the next post coming?

Cactus Kevs 5 card eval and Senzee's fast eval modification (both in ANSI C) is relatively easy to build or port to another language .

PokerSource is quite difficult/complex to port. It has a C# interface, but I have no idea how to add some of the exports I need. It was quite difficult, but possible to build the dll with the 64 built in exports. Unfortunately it won't work with the C# api, as it uses some exports which weren't in the resulting dll. (And I'm not good at c++, so I were unable to add the missing exports)

I don't know when/if the next one is coming. But, I'm confident that reading all of this material that in a couple of years I could make one. Bots don't really matter though to the poker community.

1) If it walks and talks like a duck, it might as well be a duck. 2) 90% of bots are doomed to be failures. (guesstimate) 3) Isn't the attitude against bots eerily similar to the attitude against professionals? Professionals were unwelcome as they ciphened profit from the game. 4) Bots have their place in the poker culture, too. 5) Scientific advancement is often motivated by a buck. I strongly feel that these pet projects by intelligentsia to make cute bots that talk back is pointless and vain. We need bots that MAKE DECISIONS.

In this century, bots are going to become the new minority. Already we get a sneak peak at how they will be scapegoated.

Bots will always win in the long run, in any bot vs. human interaction. Their saving grace is that, unlike in the movies, bots and a.i. will create an unheard of quality of life for people in the new era. What's of value is intelligence. Whether it's packaged in the bot or in the brain is irrelevant to the point. But until that point is actually realized it will be go badly for some. No bots allowed, in this and other industries, until we get to the point to where denying people compute AI is like denying them air.

On September 10, 2008 I became aware of this series of articles. I made it up to part 4, got everything building successfully using boost in a Windows Vista 32 OS, but...after opening the Pokertime client and starting XPokerBot, nothing appeared in the XPokerBot GUI.

I saw someone else post about this problem and Mr. Devlin's reply was to wait for upcoming articles. I quickly scanned all the 8 articles and found no solution other than a reminder that nothing is tested to work in Vista. Maybe I missed something? Can someone point me to the solution for nothing appearing in the XPokerBot GUI under Vista?

Thanks.

I think it would defeat the purpose of these articles to eventually release a completed bot. It will be far more effective to explain components and different ways those components can interact, along with some strategies on how to fly under the radar.

James Devlin is in the process of seeding the poker AI gene pool. Our job is to go forth and multiply, mutate and evolve. The fittest will survive and prosper.

The model is Evolution, not Creationism.

@Damn Noobs

That's silly. The point of this is not to sell the bot, nor to provide it free for script kiddies to use without understanding how it works. Point of the blog is extending knowledge.

"Not everyone is as skilled as you obviously are in the coding arena and no matter how much effort is put in, nothing seems to stick."

I couldn't disagree with you more. Put enough effort in, and you'll sure reap the benefits. You just have to keep at it. You'll never learn anything with an attitude like that, in our outside of the coding arena.

In the monitor bot, in the XPokerbot.Hook.cpp, you left a comment saying you would get rid a global variables. How would you get rid of them?

James is right. I ran bots for a living for over four years and the poker-eval code is all you will ever need. I had to tweak a few things to give me certain specific information I wanted but anything else is overkill and a waste of cpu time. Strategy/AI is everything and this is where you will spend 95% of your coding time so save your brain cells for that and leave perfect hand evaluation to the Will Hunting wannabees. If you plan to be a bot runner you will have plenty of more important stuff to tackle.

Buy enough coffee to hold you over for at least a year while you work on strategy. I swear I kept the local Starbucks from filing bankruptcy while coding bot strategy.

It's all very well creating super fast enumeration code, but surely an accurate evaluator fed with a few million random (Monte Carlo) hands will yield a usable win % in a substantially faster time? As stated above, CPU time is better spent far more important decisions. In fact, i believe that a more efficient starting point for a bot is to actually IGNORE the cards and concentrate on player profiling and psychological and positional strategies...

I created a rather complicated player profiling system but found the bot didnt use it often enough to warrant all of the extra work and cpu usage. My bots played limit so its multiway most of the time and profiling is much less effective (at least in my opinion.) In fact it was very rare that the bot deviated from its basic strategy based on a player profile. In many cases you will not have enough data to make a correct decision anyway. You will need quite a bit of data in order to make a correct decision over 50% of the time. In NL I can see profiling to be much more important as you can isolate effectively and keep the game shorthanded or HU post flop against a weak opponet.

Many of the bot runners I knew spent way too much time on player profiling. The other mistake I saw was testing their bot against other bots. The only way to test a bot is to but it in the ring against real human opponents. You will lose money during this process and its unavoidable. Bot debugging can get frustrating and time consuming. My bot folded the nuts on the river many times before I got it all debugged. The only way to make money is to lose money for awhile. It took about 250k hands before I was happy with its play and performance and it began to make money. Have plenty of coffee on hand while your pouring over the thousands of hand histories each day.

from start to finish how long would it take to make your bot ,how many hours did it take to buils

Hi Joe. Mine took about two years and hundreds of hours before I was happy with it. You can get a simple bot playing the nut hands in just a week however. If you want to make a NL SNG bot I think it would not take that long. Many bot runners I know of are making bots for these games. Many SNG bot strategies are Jam or Fold from what I understand. Not nearly as much programming would be required for these as I believe they are mostly preflop strategies.

My experience was with limit bots and the reason it took me so long is that I had to get help from a pro to correct the mistakes. This of course took a huge amount of time.

I think a NL Cash bot would be the most challenging and probably the most difficult bot to attempt. This is where player profiling and all that would be really be needed. A system of putting people on a range of hands and comparing them to the board cards and their betting could get complicated but not difficult at all. All of this could be done with a rules based system which many people might not agree with. You could test a system like this in observe mode and fine tune it before putting the bot into live play and not risk a dime.

Great Post

http://www.poker-bots.com/

This has been a very eye-opening series of posts so far, but I would really love to get some more info on stealth and other things like that. For example, I don't think much has been done to address natural mouse movements, etc.

Hi james... and big thanks for your work !!!

in part 4, you promised updates for xpokerbot.zip, including compatilibility with other rooms like poker star... Is this project to publish release is still up to date ? i am very interested in this code because , i am not familiar with hooking... and c++ too

I wrote my own poker bot, using your code, and without you, i won't have the idea to begin this mad project... thanks a lot... it works pretty good, but i need to adjust its playing strategy... If you are interested, i have piece of code in C# to evaluate multiple odds opponants (monte carlo) using hash table in less than 2 seconds for a complete board and 9 opp...

this is an awesome series, are you still posting?

This ia an awesome series....Thanks a million James.

I have a quick question has anyone been able to get text out from the chat box in Poker Stars. If anyone has could you please tell me the process used...I have been trying to follow this whole series but not being a programmer myself don't know what tools to use. I know James has done a good job but it's too much info in a short span to grasp.

I loved the series though and am trying this out.

Anyone who has been sucessful could you please share the wisdon. PLEASE.

Nice job!

We did something similar too, you can download the free trial and check it out! http://www.pokerbot-smart.com/

Awesome

Very well written James, you should write a small book on this botting subject sometime :) Keep up the good work!

Great, But I have finished all you done. My English is poor, My work is in chinese. Here is screeshot: [url]http://www.eepoker.cn/eepoker/screenshot.jpg[/url] Have some more utility my website: [url=http://www.eepoker.cn]eepoker(????)![/url] ???????,??????????????????

I am awaiting the contestants' entries. This has been a very entertaining series of posts to follow. Keep up the good work!

Is this series over? I hope not!

Personally I'd like to see how to create dragable windows(controls?) into the client window like the ones you see that have the HUD displays. This is most likely a trivial thing to accomplish - unless you have no idea where to start...

Thank you for one of the most enjoyable and inspirational coding articles I have ever read. Your articles inspire me to challenge myself with my programming.

As noted in another one of your articles, Pokerstars no longer uses DrawText and ExtTextOut to populate the chat window:

quote: "I regret to be the bearer of bad tidings, but this was the purpose and intent of the change -- to deny malicious third party developers access to hooking 'DrawText' to extract data from the dealer chat window. We have completely avoided the use of 'DrawText' and 'ExtTextOut' in favor of a proprietary internal solution, and as such the contents of the chat cannot be extracted any longer."

With this change, it's no longer possible to program the "input" portion of the bot as you described in your tutorial articles. Will you have the time or the inclination to update us on a new method to "read" the chat window?

Thanks again!

This is intense man. You should write another blog about collusion and how it is done with all of the technical stuff included. I would really like to read that.

is anybody making money with this pokerbots. i would love to hear about more hear. this is an awesome blog.

With all that information you get you should be able to create a good winning poker bot at low stake tables or even higher.

Thanks for sharing

This is really interesting stuff, it seems like you are on track to building serious competitor. I really do wonder sometimes how many bots are operating out there in the poker rooms. Anyways good luck and keep up the good work.

John

Just a few words of comfort for those of you building bots. Do you think people would be selling them if they actually worked? No - unless you think there are charitable people who love helping cheats :O)

They would hide them like gold dust.

Personally I find I play better totally unaided.

Here is a thought, though you bot only needs to be 5% acurate cos you have to bet a 5% rake, I can get about 5% or very close unaided which means your bot can't win.

Try a simpler program in cash games which calculates how much each player wins, quite easy to write, you could do it in a week starting from scratch, code it up as you play.

You will find the only winner is a player with the same name on ever table,he is called................MR RAKE.

alternatively look at the poker sites profits and chuck in th ehuge running costs :O)

Since you built this a lot of sites have been cracking down on programs that run over their software. Has this affected you? Also, everytime sites update their software does this cause the bot software to stop working? The reason I ask is many poker sites are constantly updating.

How would you recommend going about extracting hole card data from a client like Absolute Poker, for example. The hole card info isn't available in the chat window, nor does it get written to any kind of log file in real time. I've used all kinds of monitoring software and I can't determine where in memory this info is being stored. any ideas?

Hi, This is a great set of articles! I'm very interested in making a roulette bot and I know the same concepts apply for reading the information from the roulette client. If anybody else is making a roulette bot please post and let know...

hi, just to mention i built my bot purely from vb6 and without hooks. That was about 2 years ago, it took me about 2 months to get a simple semi auto bot running, but i have been programming over 25 yrs so.. its a hobby, if anyone needs a programmer in this area, please email me. Danny7202@Boltblue.com

Man I hope the high you get from teaching newbies GDI hooking is worth it. I'm glad this stuff is all 4-5 years out of date, but still kinda sucks that you're giving all these idiots a leg up.

Hi everyone,

I would like to congratulate James Devlin because your web site is really amazing. From the technical point of view, it is 7 stars, not 5. It is outstanding.

Right now, I would like to find someone to cooperate with me on a poker software project whose goal is to improve SNG table selection (Pokerstars). As James Devlin states, table selection is very, very important and I need to improve that area. What I am looking for is someone that is able to develop a software that retrieves the content of any listbox on Pokerstars. I have already developed ALL the software needed to evaluate if we should or should not register on each specific tournament taking into account some statistic criteria ....

If you are interested, please post your email on a comment and I will contact you afterward.

Hey Poker of Aces, I work on another Poker project. I focus on cash games. But maybe we could exchange some knowledge and how to investigate new Poker rooms.

Feel free to write me an email: Klabautermann176@web.de

I see you have no update on this series for a while... do you plan to write more?

thanks.

Hello, I'm a new guy in this industry, this is my first reading about any poker related journal. so I found this very much helpful. Your explanations with charts & good pictures let me realize the whole thing very well.

Go ahead, get inspired and come up with more articles please.

loves that all fields are optional

SHIPP ITT!!!!

Use the form below to leave a comment.






Coding the Wheel has appeared on the New York Time's Freakonomics blog, Jeff Atwood's Coding Horror, and the front page of Reddit, Slashdot, Digg.

On Twitter

Thanks for reading!

If you enjoyed this post, consider subscribing to Coding the Wheel by RSS or email. You can also follow us on Twitter and Facebook. And even if you didn't enjoy this post, better subscribe anyway. Keep an eye on us.

Question? Ask us.

About

Poker

All in all you're just another spoke in the Wheel.


Hire

You've read our technical articles, you've tolerated our rants and raves. Now you can hire us anytime, day or night, for any project large or small.

Learn more

We Like

Speculation, by Edmund Jorgensen.