In Defense of Screen Scraping
Monday, February 18, 2008Hand-coded image recognition. Checking individual color values to try to figure out if the random smear of pixels forms a "L" or an "H". Dubious OCR schemes. Internet spambots. Click fraud. Hacking.
These are the things most people think of, when they hear the phrase screen scraping.
They're wrong, of course.
But you still won't find Screen Scraping 101 up at the local college. You won't find it in books, except here and there, occasionally, mentioned in passing, usually with a smirk. I've said before: screen scraping should always be a last resort. But you know what? Screen scraping has gotten a bad rap and it's not entirely deserved.
My experience?
Getting our applications to talk to one another has been something of a software Holy Grail. I give you DLL exports, COM, DCOM, Corba, OLE, Remoting, a dozen other technologies. It would be nice if we lived in a world in which each application (and each electronic gadget) exposed a clean, universal Automation interface allowing for external command and control. It should be a requirement for a well-behaved piece of hardware or software, that it allows itself to be controlled by other pieces of software.
if (DateTime.Now >= ac.WakeTime)
{
CoffeeMaker cm = House.Rooms["Kitchen"].Items["Coffee Maker"];
if (cm != null)
{
cm.Wake();
cm.Prepare(2); // 2 cups
cm.Brew(); // asynchronously brew the coffee
}
}
But we don't live in that world, at least not yet.
We live in a world which is more like the Tower of Babel.
Our world is a world of one hundred million competing pieces of software flogging it out across a dozen platforms, using different languages, different protocols, different storage mechanisms, different endian-ness, different everything. It's a huge mess and in part we're glad, because entire industries are born of these differences. Without them, many of us wouldn't have jobs..
But if I had to say in a word what the single most expensive characteristic of modern software is, complexity would not be my choice. I would choose words like interoperability, migration, conversion, transformation, adjustment—any word describing the enormous friction created by trying to fit billions of square pegs into billions of round holes, line by line of code, across the trillions of lines of code that comprise the world's code base.
In the midst of all this chaos, one thing we can usually count on, is the GUI.
No matter what happens in software, no matter how the underlying mechanisms change and improve, end-user applications will always display textual and graphical elements on a surface. That surface might be your computer screen, your iPod, or one day, your visual cortex. But there will always be a surface and there will always be elements painted on that surface, and the programmers responsible for coding those elements will try to make them pleasing to, and easily readable by, the human eye.
In order to do that they'll usually have to rely on publically available APIs or libraries because it's enormously difficult to paint good text, or good anything, from scratch. Font rendering is a subject for experts. Drawing anti-aliased lines quickly is a subject for experts. The mathematics of viewing frustums is a subject for experts. If you're looking for an exercise in futility: try to implement a good font-drawing routine from scratch. Get back to me in five years and let me know how it worked out.
No, by and large we're forced to use operating system APIs or third-party libraries because the complexities that they mask would be too costly to implement ourselves. And once we do that, we make our programs accessible not only to humans, but to software. To build an application based on common UI components is to implicitly state that yes, our application can be accessed and manipulated by other applications written by other people.
And this is why I say screen-scraping has gotten a bad rap. Screen scraping, true screen scraping, has very little to do with "pushing pixels" and everything to do with accessing display text and other UI elements in a robust way, regardless of whether such use was planned or intended by the creator(s) of the software. That's right: I'm saying that screen-scraping, implemented properly, is a clean, robust, generic, and a powerful addition to the programmer's bag of tricks.
It just suffers from a bad name, occasional misuse, and widespread misunderstanding.
Screen scraping is a technique in which a computer program extracts data from the display output of another program. The program doing the scraping is called a screen scraper. The key element that distinguishes screen scraping from regular parsing is that the output being scraped was intended for final display to a human user, rather than as input to another program, and is therefore usually neither documented nor structured for convenient parsing. Screen scraping often involves ignoring binary data (usually images or multimedia data) and formatting elements that obscure the essential, desired text data. Optical character recognition software is a kind of visual scraper.
I can't fault Wikipedia's definition of the phrase, but I think whoever wrote it thought that screen scraping is about how to extract meaning from pixels drawn to the screen. The definition is technically correct, but misleading. I'd like to suggest a different definition.
Screen scraping is a technique in which a computer program extracts data from the target program by examining the properties and behavior of the target program's GUI, and in particular, by examining and hooking the code structures that lie beneath the GUI.
By that definition, screen scraping has a lot more in common with system-level development techniques, or even disassembly, than it does with OCR or image recognition. And when we look at the code structures beneath the GUI, rather than trying to grok the meaning of pixellated randomness, we find that screen-scraping techniques are very robust indeed:
- DLL Injection
- API Detouring
- Window Subclassing
- Message Processing
In fact, we start to see that screen-scraping is an exercise in conventional development techniques. Windows. DLLs. Processes. Threads.
The stuff of which software is made.
So the next time you hear someone mention screen scraping, think twice before you chuckle. Screen scraping is about writing software with the sensory (visual) and motive (manipulaing the mouse, keyboard) capability of a human. It's not always (or even usually) appropriate, but like that most famous of martial arts techniques...
...when properly applied, it's extremely powerful.
Yes! Leave a Comment. All fields are optional.
- Sneak Peek: StopLight 1.0
- A Radioactive, Search-Hardened User Name for Online Poker
- Online Poker and the Multi-Tabling Effect
- A Primer Primer
- Moonlighting with OpenTK
- BROWSE ALL POSTS
Subscribe to Coding the Wheel over email or through any RSS reader. Coding the Wheel has been published since 2008.
- How I Built a Working Online Poker Bot, Part 1, 2, 3, 4, 5, 6, 7, 8
- Summoning the Harry Potter MMORPG
- Are Commercial Databases Worth It?
- 21 and the Monty Hall Paradox
- Visualizing the Internet (48)
SEO Packages wrote: Insightful, but the internet is a lot more difficult to portray imo. - 1,326 (41)
SEO Packages wrote: So disco isn't dead yet? - Dartboard-Driven Design (52)
SEO Packages wrote: Hilarious, very creative but I wouldn't guide myself after that. 90% of the surface of my target ... - How to Display Integrated Digg.com Comments on Your Website (27)
SEO Packages wrote: Very useful article, thanks for the info - Using Internet Conveyor Belts to Drive Traffic to Your Site (23)
SEO Packages wrote: Very useful info here, I am in the SEO business myself and I've learned some now stuff from your ... - Summoning the Harry Potter MMORPG (1048)
James Lloyd wrote: Is this Real LOOKS F*ING AMAZING :) can't wait till it is made :D - The Great Poker Hand Evaluator Roundup (119)
Cheap UGG Boots wrote: fddddffffff - The Great Poker Hand Evaluator Roundup (119)
mbt kisumu white wrote: dddddddddd - The Great Poker Hand Evaluator Roundup (119)
Discount MBT Shoes wrote: mbt kisumu black sandal and mbt kisumu white ar hot saling now in our website. This Summer,the mbt ... - How I Built a Working Online Poker Bot, Part 3 (133)
mr.cool wrote: Hello! I find this review very interesting ！Vibram FiveFingers - A Circular Dilemma Concluded (39)
online bollywood movies wrote: It seeks to show how rarely needed a high voltage working environment in which candidates are difficult ... - How I Built a Working Online Poker Bot, Part 3 (133)
wewr wrote: [Replica Watches][1] [1]: http://superior-replica.com/"Replica Watches" ... - How I Built a Working Online Poker Bot, Part 3 (133)
wewr wrote: Sorry i not mean spam,...just like your article,...cheers [Replica Watches][1] [1]: http://www.superior-replica.com/"Replica ... - How I Built a Working Online Poker Bot, Part 3 (133)
wewr wrote: Vibram FiveFingers - How I Built a Working Online Poker Bot, Part 3 (133)
Replica Watches wrote: Replica Watches - How I Built a Working Online Poker Bot, Part 3 (133)
Replica Watches wrote: Yeah, after a study it more deeply, I know now.Replica Watches - How I Built a Working Online Poker Bot, Part 3 (133)
Replica Watches wrote: I had not understood the purpose of this article, but after a study it more deeply, I know now. Replica ... - How I Built a Working Online Poker Bot, Part 3 (133)
Replica Watches wrote: posts that are very evocative readers like me to understand more about what is contained in this article ... - How I Built a Working Online Poker Bot, Part 7: Extracting Text from 3rd-Party Applications (318)
window cleaning wrote: Great series and I enjoyed each article very much. I hope now you've got a whole batch of articles ... - How I Built a Working Online Poker Bot, Part 3 (133)
property management wrote: fat burning furnace scam or real.
29 comment(s)
Is this post new?
I read all your posts a few weeks back and I don't remember this one.
This is the 2nd time you've used that Karate Kid pic. It is however somewhat epic in its geekery so I approve.
Anonymous on Sunday, July 20, 2008Is there any way that I could buy the poker bot you made?
Ewan on Sunday, August 17, 2008If one wants to start to build a screen scraper in C++, where to start? Anyone have links/sample code for me?
Anonymous on Wednesday, December 03, 2008Screen scraping work quite nicely when we take a screenshot and the then calculate a Sha256 sum to the needed areas and let a user spcify a meaning of that area. If I do handle 100 areas a time to make this 65 - 80 ms with C++ language. And this time is only for teaching not for playing.
hacker on Friday, May 29, 2009I don't use a GUI when scraping from, I use mainly the text output from a page/html/program. I thought screen scaping was mainly text/image resource/output gathering. The gui is somehow invisible. Does a html page have a gui? Or is that datamining?
Anonymous on Saturday, May 30, 2009Screen scraping is the technique of extracting information from an application by writing code to interrogate the application's user interface. This might mean doing pixel testing and OCR, it might mean hooking various operating system APIs, it might mean poking around in application memory, but it's always in reference to some visual element of the application.
In the browser/HTML world, on the other hand, "scraping" usually means extracting information from the raw HTML markup. Not quite the same thing.
James Devlin on Monday, June 01, 2009I recently coded a small app to rip data out of a online store and into a database for the purpose of building a similar site and I wanted an established price db for the same items (a collectible card game) and I used alot of RegEx's to extract the data from an HTML stream, what do you recommend down the other path as far as open sourced OCR packages?
Please keep the articles coming, I'm required to do a project for an operating systems class I'm taking and I'm thinking I can do something along the lines you have presented so far, in regards to the DLL injection and windows form/window management... plus I will learn a shitton on the way... any thoughts?
Mark on Thursday, January 14, 2010Introduced in 1990 as the Nike Air Max III, the Nike Air Max 90 turned more heads with an even larger visible Air unit and greater stability and cushioning. It has been re-released as a retro version with an immeasurable amount of colorways and themes. The Air Max 90 was referred to as the Air Max or the Air Max III until 2000, when it was reissued. It takes its current name from its launch year, 1990.The first colourway produced was the men¡¯s white / black / cool grey / red. It was striking to say the least: the radiant red around the visible Air window exaggerated the thickness of the midsole, and the matching thermoplastic straps and rubber heel patch added to the overall effect. The upper was made of Duromesh, synthetic suede and synthetic leather. The clean lines and dazzling colours made the Air Max a higly sought-after.
Nike Air Max 90 on Thursday, July 08, 2010nice article. keep post like this...
Fendi replica watches on Tuesday, July 20, 2010i believe you are a good writer, but have you erver thought to write some special artcals for peopel who likes shopping very much. for exaple, the artical about louis vuitton bag or vibram shoes,
uggs on Wednesday, July 21, 2010I loved this post so much. This was great. Keep it up!
Tudor replica watches on Saturday, July 24, 2010PerformingGraham for sale that will create a one-wayswiss watches backlink to your website whichMaurice Lacroix watches is a good point to have. It is Patek Philippe watchesa time-consuming Tag Heuer for saleprocess even though it produces good outcomes U-boat for saleand works. You can also swiss watchespost comments on blogs alongMaurice Lacroix watches together with your link, but make sureGraham watches these blogs are do-follow, which dolce & gabbana handbagsindicates the link that youreplica coach handbags simply post will probably be replica handbagscounted by the search engines jimmy choo replicaas a backlink.There are many thingsversace replica handbags that contribute to online businessjuicy couture replica failure, and the lack of good action thomas wyldeis surely one of the deadliest business-killersdolce gabbana handbags on the net. Take action like by no means chloe handbags replicaprior to, and don’t get struck by analysis paralysisburberry replica. Your achievement lies in your rapid actionchanel replica and how well you apply numerous strategies.
replica watches on Saturday, July 24, 2010nike air jordan 2010 nike air jordan 5 nike air jordan 3 nike air jordan 11 cheap nike air jordan 2 nike air jordan 1 nike air jordan shoes discount nike air jordan shoes nike air jordan 6 wholesale nike air jordan 1 discount nike air jordan 6 nike air jordan 7 nike air jordan 8 top quality nike air jordan shoes 23 nike air jordan 21 nike air jordan 22 nike air jordan 6 rings cheap nike air jordan 22 nike air lebron james shoes nike air lebron james shoes nike air jordan 2010 nike air joredan lebron james nike air jordan 2009 discount nike air jordan 2009 nike lebron james wholesale nike air jordan 2010 cheap nike air jordan 2009 discount nike air jordan 19 cheap nike air jordan 3 discount nike air jordan 18 brand nike air jordan 12 wholesale nike air jordan 2 nike air jordan 13 nike air jordan 14 nike air jordan 6 rings discount jordan shoes7 discount jordan shoes1 jordan shoes2010 jordan shoes 6 rings cheap jordan shoes2 discountjordan shoes3 discount jordan shoes4 jordan shoes uk23 jordan shoes classic bw22 cheap jordan shoes 21 ltd discount jordan shoes20 discountjordan shoes19 jordan shoes 18 uk nike air jordan shoes 17 cheap nike air jordan shoes 16 discount air jordan shoes 15 discount air jordan shoes 14 nike air jordan shoes 3 uk nike air jordan shoes 13 cheap nike air jordan shoes 11 discount air jordan shoes 6 discount jordan shoes 10 nike air jordan shoes 8 nike air jordan shoes 7 cheap louis vuitton bags cheap kobe bryant shoes cheap lebron james shoes .
cheapjordan on Saturday, July 24, 2010I will keep visiting this blog very often. Blu ray ripper /
ipad converter on Sunday, July 25, 2010air max shoes is an extremely comfortable shoe that uses a large air cushioning unit that can be seen from the side. This technology combined with the design of the shoe which is meant to compliment the spinal cord makes. Air Max shoes the ideal shoe for walking, running, playing basketball, or any other sport. nike air max ,nike air max 2009, nike air max 95, nike air max 90 and so on.Nike introduced its Air Cushioning Technology in 1979. By embedding Air Cushioning in the midsoles of its running shoes, it changed the way athletes approached and reacted to athletic performance forever.
nike air max on Monday, July 26, 2010ed hardy, cheap ed hardy, ed hardy clothing, ed hardy outlet, ed hardy jeans, ed hardy bags, ed hardy swimwear, ed hardy shirts, ed hardy tee, ed hardy caps, ed hardy purse, ed hardy board shorts, ed hardy shoes, ed hardy for men, ed hardy for women, ed hardy jeans for women.
cheap ed hardy on Tuesday, July 27, 2010ed hardy, cheap ed hardy, ed hardy clothing, ed hardy outlet, ed hardy jeans, ed hardy bags, ed hardy swimwear, ed hardy shirts, ed hardy tee, ed hardy caps, ed hardy purse, ed hardy board shorts, ed hardy shoes, ed hardy for men, ed hardy for women, ed hardy jeans for women.
cheap ed hardy on Tuesday, July 27, 2010It is a good timberland 6 inch boots thing that the God Who made us is not impatient with people like me with my timberland shoe company orchid. He lovingly thinks about us all the time timberland boots and he even temporarily gave up His heavenly home and his timberland traditional handsewn hold on diety to die for the sins of timberland mens custom world nearly 2,000 years ago. His death back then covered the sins of every cheap timberland boots person who will ever live. He then rose to life after three mens timberland chukka days and ever since has wanted us to accept His gracious act by loving Him and also those around us. When we do this, we will see people in a totally different timberland womens premium boots light — one that redefines what it means to be ugly or different or weird or strange. Instead, we see people with ugly timberland shoes store problems in whom God is working so they can bloom beautifully — just like I now see in my orchids whether they are “lumps” or blossoms! On a certain timberland work shoes day at a certain hour, we will pull into the station. Bands will be playing and flags waving. Once we get there, so many wonderful dreams will come true and the pieces of our timberland wheat shoes lives will fit together like a completed jigsaw puzzle. How restlessly we pace the aisles, *ing the minutes for timberland for you loitering --waiting, waiting, waiting for the station.
discount timberland boots on Wednesday, July 28, 2010Louis vuitton has many stylish purses.It is famous for its high quality, unique styles and fashionable designs.Louis vuitton online store leads the fashion trend in the ...
May 26, 2010 ... Which Louis vuitton look were you totally loving? Which ones weren't so much your favorites? Cast your vote for best Louis vuitton wallets now, ...
The best benefit ?s that you cаn burn calories when ?ou walk with th?se boots Louis vuitton, that got youг attention, did it not? Ye?! You can lose weight!
louisvuitton4 on Thursday, July 29, 2010Wholesale NFL Jerseys
Cheap NFL Jerseys
NFL Jerseys
NFL Football Jerseys
Women NFL Jerseys
Kid NFL Jerseys
Super Bowl Jerseys
Super Bowl NFL Jerseys
Throwback Jerseys
Throwback NFL Jerseys
Cheap Throwback Jerseys
Wholesale Throwback Jerseys
Cheap Super Bowl Jerseys
Wholesale Super Bowl Jerseys
youth nfl jerseys wholesale
youth nfl jerseys cheap
nfl youth jerseys
Baltimore Ravens Jerseys
Chicago Bears Jerseys
Cincinnati Bengals Jerseys
Denver Broncos Jerseys
Dallas Cowboys Jerseys
Green Bay Packers Jerseys
Indianapolis Colts Jerseys
Minnesota Vikings Jerseys
New England Patriots Jerseys
New Orleans Saints Jerseys
New York Jets Jerseys
Arizona Cardinals Jerseys
Buffalo Bills Jerseys
Atlanta Falcons Jerseys
Carolina Panthers Jerseys
Cleveland Browns Jerseys
Houston Texans Jerseys
Detroit Lions Jerseys
Jacksonville Jaguars Jerseys
Kansas City Chiefs Jerseys
Miami Dolphins Jerseys
New York Giants Jerseys
Philadelphia Eagles Jerseys
San Francisco 49ers Jerseys
Oakland Raiders Jerseys
Seattle Seahawks Jerseys
Pittsburgh Steelers Jerseys
St Louis Rams Jerseys
San Diego Chargers Jerseys
Tampa Bay Buccaneers Jerseys
Tennessee Titans Jerseys
Washington Redskins Jerseys
Ray Lewis Jerseys
Ed Reed Jerseys
Joe Flacco Jerseys
Terrell Suggs Jerseys
Dick Butkus Jerseys
Greg Olsen Jerseys
Matt Forte Jerseys
Devin Hester Jerseys
Walter Payton Jerseys
Brian Urlacher Jerseys
Lance Briggs Jerseys
Jay Cutler Jerseys
Carson Palmer Jerseys
Rey Maualuga Jerseys
Chad Johnson Jerseys
Demarcus Ware Jerseys
Jason Witten Jerseys
Terrell Owens Jerseys
Felix Jones Jerseys
Marion Barber Navy Jerseys
Marion Barber Jerseys
Emmitt Smith Jerseys
Miles Austin Jerseys
Roger Staubach Jerseys
Troy Aikman Jerseys
Tony Romo Jerseys
Eddie Royal Jerseys
Brandon Marshall Jerseys
Aaron Rodgers Jerseys
A.J. Hawk Jerseys
Donald Driver Jerseys
Greg Jennings Jerseys
Peyton Manning Jerseys
Dallas Clark Jerseys
Robert Mathis Jerseys
Marvin Harrison Jerseys
Drew Brees Jerseys
Marques Colston Jerseys
Pierre Thomas Jerseys
Reggie Bush Jerseys
Jeremy Shockey Jerseys
Tarvaris Jackson Jerseys
Adrian Peterson Jerseys
Jared Allen Jerseys
Mark Sanchez Jerseys
Thomas Jones Jerseys
Leon Washington Jerseys
Brett Favre Jerseys
buynflshop on Thursday, July 29, 2010Louis vuitton – probably is easy recognize, but sorry again, even a gorgeous Audry Hepburn does not look the best with her
outlet.html" tItlE="Louis vuitton outlet">Louis vuitton outlet Would be much better with
Buy at lower prices replica hangbags and shoes and save up to 80-90%. Buy Handbag Louis vuitton. We are a Louis vuitton
outlet store and offer cheap replica
Gillette holds strong as the “best for men” for personal care, owning 70 percent ... The lv brand is also gathering strong momentum in
China,
YouTube streaming video - Louis vuitton Journeys: the greatest game - For the world cup 2010 lv outlet launch a special ad
campaign with
louis vuitton on Thursday, July 29, 2010Pool Temperature Madera Jobs Madera Pest Control Madera Dentist Merced Dentist Visalia Dentist Modesto Dentist Fresno Limousine Fresno Granite briefcases leather leather briefcases prospect solution | prospect solutions | prospectsolution | prospectsolutions | prospectsolution.com prospect solution | prospect solutions | prospectsolution | prospectsolutions | prospectsolution.com prospect solution | prospect solutions | prospectsolution | prospectsolutions | prospectsolution.com
carmi on Friday, July 30, 2010essay essay writing essay writing service essay writing service uk
jane on Friday, July 30, 20102010 chanel handbags hot sale on buybury.com, you can discover all kinds of fashion chanel handbags in free shipping worldwide, and buybury.com provide chanel bags are all 70% discount, that attracted numerous chanel funs, and you can place an order for cheap chanel handbags will have the same discount. If you are looking for chanel handbags now, do not hesitate, chanel hobos on our site is really noble but cheaper. All chanel bags really are wonderful.
chanel handbags on Friday, July 30, 2010www.thesuitshoes.com can provide you the best MBT shoes to choice.
mbt shoes on Friday, July 30, 2010www.thesuitshoes.com can provide you the best MBT shoes to choice.
mbt shoes on Friday, July 30, 2010good news
hermes birkin on Friday, July 30, 2010thanks for your kindly sharing
abercrombie and fitch on Friday, July 30, 2010Want to get Ed hardy at home? No hesitation, get them at with big discount. We are promise you that all the ed hardy clothing are good in qualities. Meanwhile, buying ed hardy clothing online, that’s never a bad idea. By the way, whenever and wherever you need us, we will be there for you. Ed Hardy T-Shirts are waiting for you now, welcome you!
ed hardy on Friday, July 30, 2010