How I Built a Working Online Poker Bot, Part 3

Tuesday, June 10, 2008

[Update: two of the original authors of this post have moved on to greener pastures, and have asked us to mothball this content in preparation for an upcoming book. Thanks for all the feedback and stay tuned for more technical poker content in the future.]

Introduction

I'm a big fan of pet projects. You know the ones I mean: the projects we love to start and hate to finish. The two-week remodeling gig that takes two years. The '69 Mustang sitting on cinderblocks in the back yard while seasons rotate. The unfinished novel lurking on the nether regions of your hard drive. And for programmers and poker players around the world, a million unfinished tools and libraries ranging from the ingenious to the depressingly obscure.

Today, I'd like to talk to you about a pet project which is actually worth your time.

Tags: poker bot, A.I., online poker, poker

62 comment(s)

Love the "rambling" nature as you call it- not sure I would enjoy short posts as much- One thing I would really like to see is more detailed information on controlling the poker client. Excellent article though, interesting way of introducing the AI, and great diagram- is that the actual code base?

thanks was looking forward to it!

OKAY. YOU HAVE TO STOP TELLING PEOPLE HOW TO BUILD BOTS. WELL AND GOOD FOR PEOPLE TO SAY: I'D PLAY AGAINST A BOT ANY DAMN DAY. WELL AND GOOD FOR PEOPLE TO SAY BOTS ARE WEAK. NOT ME. I DON'T WANT TO PLAY AGAINST ONLINE POKER BOTS. THEY SCARE ME. PLEASE MAKE THEM STOP.

I don't think the AI will be the most difficult problem: it's staying undetected what looks to me the biggest problem. How are you ever going to evade the continuously increasing detection mechanisms of the poker sites?

Hey BOT THIS - don't you know that concealing information does not prevent the information from getting out? Information needs to be free....besides, just telling people how to do something does not in any way mean they are going to do it.

When you're developing a new poker strategy implementation do you set up simulated games and run fake hands to see how they compare to the existing implementations?

Your series on bots is very well done and as a professional coder, I've enjoyed reading them. I would be especially interested to hear about your debugging strategies and your testing strategies. Hopefully you can do a rambling post on those subjects soon.

I agree with BitBotGuy. In any case it's not like nobody ever thought of this before this article. The hardest part is being both a good programmer and a poker enthusiast, as this kind of project takes a lot of time to do. Not many of those people around I guess.

Dont get caught... Its now a class B felony

Just curious, but any plans to sell your "bot" down the road?

Hey. Good post. I dont really program, or play any poker, except the random texas holdem pokernights with friends. But I enjoyed reading the post. Especially funny to see that a project increases two skills simultaniously. And I guess you win more when you play after building the bot. Andif the bot is good, he prolly rakes in some cash to. GL+HF.

Good stuff, it’s an enjoyment to read, keep up the good work. I just happened to be a bot programmer myself, and just for the record, what you say about “big ball of mud” is true, I’ve been in the exact same situation ..

The biggest problem we have is staying undetected, any ideas beside hiding of process an rename the bot application to something like “paint.exe” ?

And for all of you “not programmers” but poker players reading this: surly if you are a weak player, a bot will beat you, but a human player will beat you as well, do not fear bots, if you play well you will win in the long run anyway.

[b]Fellow developers who are more into the subject than I, scoffed at me when I mentioned these articles, declaring them as blogspam for affiliate dollars. After reading this part and it failing to progress anywhere near towards turning theory into code, I accept this could turn out to be a waste of time for all involved except the author, who may end the series prematurely when he can find no more poker history to pad with.[/b]

@Thomas - i think you're overestimating what a poker site is able to "detect". If you're not manipulating the poker program directly (which he's not - he's actually causing mouse & keyboard events to fire), the information stream between the program and the server, or getting caught by their "collaboration heuristics" (which any human players could set off just as easily), there's nothing to detect. The poker site's program runs in windows, man. It doesn't know anything about mouse/keyboard drivers to be able to detect their use. So basically the bot interacts with a computer the same way as a human would. So the problem comes down to figuring out how to avoid being detected as a cheater/collaborator -- the answer to which is always going to be "DON'T CHEAT".

I've always been interested in digitally modeling player strategies (and in turn modeling how players model each others strategies), but you're 100% right about the amount of work required. In order for me to test models/agents against each other I need probably 90% of your framework coded (with none of the payoff of loyalty rewards!)... which has always seemed like way too much effort to even start on the part I'm interested in.

Are any of you aware of any open-source projects that handle the domain at this level of detail? Even if it's not in my preferred programming language it would help, I bet, to use as a reference... Damn. I think you just convinced me I need to build a poker framework as a pet project. ;)

Until online poker sites allow bots with some clear indication that they are bots (like ICC or FICS computer accounts), sharing information on this topic is a horrible idea.

Also, the bot itself is not a profitable side project (it may be profitable due to learning to code or learning poker), if you make a winning bot, you'll get caught, then your roll gets seized, the end.

Dude! You can write! Good job.

I insist that opponent modeling is not necessary if you plan on staying in the lower- to mid-stakes games. You can do very well with a static strategy, without introducing the complexity of modeling other players. Opponent modeling adds 2-4 more full dimensions to your input feature space, to a problem which may already be difficult enough (teaching the bot to play decent poker). On the other hand, so many players online don't even understand basic strategy, and will just hand money over to the players that do (you).

@anonymous--- blogspam? lol. single most entertaining& informative series on this topic i've ever read. maybe if the guy was trying to sell online poker signups or something but i've yet to see one online poker affiliate link. great thing about blogging nobody forces you to read.

Two facts which stand out for me -

  1. Using C++ for an online poker bot.

  2. The extreme verbiage and effort spent on this article - is it worth it?

Peace.

What a fascinating article! I've always wanted to code some accessory programs for online poker sites but I never really thought about it seriously until i read your series. Keep up the informative writing!

I'm curious about your statement that PStars uses a logfile that you can pull data from. I've examined this file, and it doesn't appear that it reports any cards on the screen besides your own hole cards. Can anyone clue me in on what I should be looking at?

One of my pet projects has been to code a simple program that keeps track of other people's folded cards in stud. Screen scraping is absolutely abysmal for this application unless you do nothing on your computer except play poker.

Awesome and superior. Please keep up the good work, and thanks!

@Anon who bolds his statements for some reason. Sorry the author doesn't just give out his code. He's giving you the tools to write the code. It does no one any good if he just gives out the source and EVERYONE uses it. The poker sites would catch on and block it and then you would be bitching about an updated bot that gets past the security. Do something for yourself for a change alright buddy?

BOT THIS Said: OKAY. YOU HAVE TO STOP TELLING PEOPLE HOW TO BUILD BOTS. WELL AND GOOD FOR PEOPLE TO SAY: I'D PLAY AGAINST A BOT ANY DAMN DAY. WELL AND GOOD FOR PEOPLE TO SAY BOTS ARE WEAK. NOT ME. I DON'T WANT TO PLAY AGAINST ONLINE POKER BOTS. THEY SCARE ME. PLEASE MAKE THEM STOP.

BOT THIS on 6/10/2008 12:22:10 PM (16 hours ago)

BOT THIS Said: OKAY. YOU HAVE TO STOP TELLING PEOPLE HOW TO BUILD BOTS. WELL AND GOOD FOR PEOPLE TO SAY: I'D PLAY AGAINST A BOT ANY DAMN DAY. WELL AND GOOD FOR PEOPLE TO SAY BOTS ARE WEAK. NOT ME. I DON'T WANT TO PLAY AGAINST ONLINE POKER BOTS. THEY SCARE ME. PLEASE MAKE THEM STOP.

Here's what I say: "STOP READING THIS IF YOU DON'T LIKE IT....LET THE REST OF US ENJOY THE GOOD READING THIS MAN HAS TO OFFER!"

Hey James, just want to say thanks very much for a very interesting series of articles! Looking forward to seeing more... keep it up!

great article, thanks so much for your insights

Major gambling sites attract tens of thousands of visitors in a day. There are even thousands of sites for those unwilling or unable to play for money: Many online gambling sites operate on a game credit basis, and you can play just for the fun of it without risking anything.

Extremely well written and organized articles, really enjoyed all parts.

Fact: Most experienced programmer are able to abstract the problems until this level.

Anyone start coding a poker bot based on this articles will spend a ton of money on programming books, a year of work on coding, then three tons until he will admit it is not an easy task to code better-than-abc bot.

Great 3rd installment to the series. You're absolutely right about the poker bots having an edge when it comes to playing short stack.

[b]The bot I made uses binary all-in or fold strategy.[b] [url]http://www.icmbot.com[/url]

While this type of bot isn't practical for cash games, it is quite lucrative at the lower stake turbo sit & go tournaments. It uses ICM (independant chip model) calculations and Nash Equilibrium to determine its push/folds.

I've run it at the turbo tournaments on Pokerstars and Full Tilt and its been surprising me with its success. Amazing how many players at the $6.5 tables will call all-in's with AKo or 44 at the 10/20 blind levels early on. My bot pushes QQ/KK/AA and sits tight during the early phases... it's lethal during 12bb> phases of the tournament, and uses within a 0.1% equity to push heads-up.

I'm looking forward to upcoming articles!

This is a very interesting article. I will post this and past it along to others that need to read this.

It's a shame about the dwindling poker scene in Dallas you started off talking about. In the UK, it is definately different and gambling laws seem to be loosening up rather than tightening!

As for the poker bot, you have some really interesting content, yet I couldn't fully grasp anything technical.

As usual, it's worth noting that the OpenHoldem project provides you the framework (and all of the source code) to do this yourself. It is architected in Input/Process/Output much like the author advocates, but the heavy liftinf for the Input & Output is already done. You need to create the Process part yourself of course.

Oh yeah, integration of PT stats is already there in the framework.

Come give it a try, and join a very vigorous community that discusses these topics every day, while you're there.

http://code.google.com/p/openholdembot/

James Devlin, please do not let sites like "openholdembot" and "icmbot.com" stop you nice project, it is an enjoyment to read you view on things – and very inspiring for my own bot work .. you are a good writer!!

And as for the icmbot, you can get that for less than 50$, this is just silly, a winning bot takes years to build and is worth a lot more than 50-310$, I wonder if icmbot can even can break-even! Buy a bot for 50$, you might as well go all-in with 50$ when you look down at 72o, it like throwing money out of the window ..

Just a suggestion: Images with text on therm (block diagrams for example) are better viewed (no blur) if saved as GIF or PNG but not JPEG format.

Another brilliantly presented and well informed article.

As a poker-botter myself, I find the subject totally facinating and look forward to each article.

Keep us the good work!

Net, net, I'm glad you're posting this. On the one hand, don't give away secrets that took me a while to discover. But on the other hand, I will definitely learn something. I've built a partial bot using some of the techniques you've mentioned but I didn't know about Detours and love it. Last summer, I felt my brain stagnating since I've retired so I decided to a). learn a new computer language and b). sharpen my poker skills. I built my little bot using VB.net and love it. I mainly use it to play my preflop strategy in tourneys while I read or watch tv. It solves my 'boredom' problem (playing hands I know I shouldn't but am bored) and I've done pretty well in some MTT tourneys with it. Of course, it's not a full-fledged bot. But your article is encouraging me to take the next step. I'm going to mix C++ and VB.net. You are an excellent writer and even if you keep it at this level (not much code), I'm getting a lot out of it. Also, I'm not afraid of a 'bot explosion' because it's still pretty difficult for the average joe to accomplish. Keep up the good work.

Did the men in white coats get him?

Great and excellent...

James I hope you will provide a full fledge sample-code-pack with detailed instructions. I m webdeveloper (many other languages) and interested in bot programming since several years (chats, http://www.alicebot.org/aiml.html etc.). My C++ knowledge is very limited, but I can compile and may make some modifications if I get a working example-pack.

If you provide it, it would be a great framework for people who are interested mainly in bot-logic and poker then c++ - professionals.

I hope it so mutch... and waiting on part 4

I opened an additional Thread for related discussions named Poker Hooks on: http://www.criticalsecurity.net/index.php?showtopic=30320

[...] Well, yes... but there are problems.

Firstly, checking for changes to the IAT relies on the IAT being changed AFTER the application starts. Start the application suspended, step it till the target DLL is loaded, change the IAT, inject the trampoline code as a locked page, unload the injector and then resume the application. The IAT never changes from the apps perspective... and the DLL's enumerate as normal.

Then there are transparent DLLs. You can create a passthrough DLL which gets loaded in place of the real DLL or even statically link a .lib version of the cryptoAPI into a replacement DLL with the wrapped functionality... Again, if the target DLL is third party then they cannot check for tampering as their system could break everytime microsoft did an update or they hit some alternative config - it would be a nightmare. again, they see no IAT patching and the DLL's enumerate normally.

Finally there are some nasty countermeasures if they are anal...

Lets say they periodically checksum all of their loaded modules and send them back for analysis... this rules out general build variations in third party tools and the system API because these they would see from many sources... they also audit memory management and any locked pages. Let's say short of rewriting the app theres no way we can inject anything into their space withoutthem knowing about it.

Okay, let's play with this... [...] http://www.criticalsecurity.net/index.php?showtopic=30320&pid=200845&st=0&#entry200845

ok... so what i described in the 2nd installment [the dsp co-processor] should be a drop-in intelligent agent, which shouldn't be insurmountable, since i'm only passing my hole cards and board cards each round to the dsp. This would allow for improvements in the dsp algorithms/hardware. And, I would, of course, want to keep estimated winning percentages[the output from the dsp to the bot process] as a generic/identical interface?

If any of you players or bot writers need hand histories for your stats (PT, HM etc) databases you should head to HandHQ.com.

Nice info but really is nothing new. A bot really just requires tons of time just like any programming project. Poker sites and bot makers are basically playing the game of virus writers and virus scanners play. It will go back and forth and there will never be a real winner.

Amazing thread, your online Poker Bot sorry is amazing.We would like to write a review on it when you are ready.

Thank you

Nice blog! More people should read it. If you want, you can register your blog Pokerweblogs.com. .It is free and and it automatically updates when you do an update, so visitors of our site can see when you updated your blog. The big advantage is that it will attract much more visitors to your blog.

Greets Peter

http://pokerbot.node64.com/

http://www.hotnastyworld.com

Nice job!

We did something similar here: http://www.pokerbot-smart.com/

Great work

Great 3rd installment to the series, I think thhttp://pokerfornia.comis a great story, and that it's great that you decided to share.

Very insightful thank you

I know you get hundreds of comments but would appreciate any input if you had some for my inquiry. Is it possible that poker sites predetermine who they want to win? I was recently playing on absolute poker in a heads up match Now my internet connection is good and all and through these last hands there was definitely no lag. The last hand the opponent had about 40 chips left, less than the blind. Before the flop even came out, the screen had already popped up telling me I won the tourney. Do you know of any reason for that? Obviously I could see the incentive for a poker site to do this, rapid construction and destruction of players BR's could create gambling addicts and a lot larger rake for the house? But it seems quite extreme that absolute would pre decide who is going to win? even though in the past the were involved in one major scandal. Feel free to give me an email if you have any ideas.

Regards, Brian

Very impressive article

It seems excellent idea to me is

A truly exciting read and I'm not even a coder. Just as a poker fan I find your approach to and analysis of the process of building an online poker bot fascinating!

Very nice article. Thank you very much

I have 36 computers avaiable from 10pm to 7am. If anyone wants to run their bot in my computers, we can share the profits. tobyniefers@hotmail.com

Hi everyone,

I would like to congratulate James Devlin because your web site is really amazing. From the technical point of view, it is 7 stars, not 5. It is outstanding.

Right now, I would like to find someone to cooperate with me on a poker software project whose goal is to improve SNG table selection (Pokerstars). As James Devlin states, table selection is very, very important and I need to improve that area. What I am looking for is someone that is able to develop a software that retrieves the content of any listbox on Pokerstars. I have already developed all the software needed to evaluate if we should or should not register on each specific tournament taking into account some statistic criteria ....

If you are interested, please post your email on a comment and I will contact you afterward.

Hey Poker of Aces, I work on another Poker project. I focus on cash games. But maybe we could exchange some knowledge and how to investigate new Poker rooms.

Feel free to write me an email: Klabautermann176@web.de

Comments for this post are closed.

Coding the Wheel has appeared on the New York Time's Freakonomics blog, Jeff Atwood's Coding Horror, and the front page of Reddit, Slashdot, Digg.

On Twitter

Thanks for reading!

If you enjoyed this post, consider subscribing to Coding the Wheel by RSS or email. You can also follow us on Twitter and Facebook. And even if you didn't enjoy this post, better subscribe anyway. Keep an eye on us.

Question? Ask us.

About

Poker

Coding the Wheel =
Code, poker, technology, games, design, geekery.


Hire

You've read our technical articles, you've tolerated our rants and raves. Now you can hire us anytime, day or night, for any project large or small.

Learn more

We Like

Speculation, by Edmund Jorgensen.