Introduction
So you've had some success using Digg.com as a venue to promote your content. You've managed to rack up a few posts with high Digg counts; maybe you've even had your day on the front page. You know that even moderately popular Digg postings get a lot of attention; so you start evaluating everything you write in terms of its "Diggability", for better or for worse.
Then you notice something: a lot of people are reading your content, and commenting on it; but most of them are doing so through your story widget on Digg.com, rather than domestically, on your website. Indeed, the more popular the article gets, the more it can feel like your site has gotten lost in the shuffle. It looks something like this:
- Story X, Digg.com - 156 comments
- Story X, AuthorsBlog.com - 2 comments
You're happy to get any comments, of course, on Digg.com or elsewhere. But it sure would be nice if you could magically "export" that juicy Digg comment list back to the page which originated the article, so that Digg comments and domestic comments are displayed seamlessly together. One big, happy family: isn't that how it should be?
So today, I'd like to talk about how to leverage the Digg API to accomplish just that. First, let me state the Golden Rule of Digg.com Customization:
You can access virtually any aspect of Digg content by making a plain-vanilla HTTP request, and massaging and displaying the response to that request using PHP, ASP.NET, Javascript, or what ever technology you want.
The smart folks behind Digg.com have put together an extremely flexible API which allows you to filter and extract just about anything - users, comments, digg counts - by querying a particular HTTP endpoint. If the term "HTTP endpoint" isn't particularly meaningful to you - allow me to rephrase. You use the Digg API by making requests for particular URLs, and interpreting the data those requests return. The format of the URL is determined (and documented) by Digg, as is the format of the data you get back.
The upshot is that there's no single way to implement Digg comments on your site. There are about a hundred ways, each slightly different than the next.
Digg's Request-Based API
When I say a "plain-vanilla" HTTP request, I mean something you could type into a browser window. Click the following link to see this in practice:
http://services.digg.com/stories/6200199/comments?appkey=http%3A%2F%2Fwww.codingthewheel.com
This is a real-world use of the Digg API. It says, "get me all the top-level Digg comments associated with Story #6200199, whatever that is."
(Every story on Digg is identified by a unique number, and 6200199 just happens to reference The Great Game, a post I published last week.)
Here's the data returned when the above URI is requested:
<events timestamp="1209451203" total="3" offset="0" count="3">
<comment date="1208754737" id="14636430" story="6200199" up="0" down="0" replies="0" replyto="" user="plsntvll" level="0" root="14636430">
<![CDATA[ Really interesting article, as always from him. Check out his site www.codingthewheel.com It's really fascinating what he comes up with.
]]>
</comment>
<comment date="1208738921" id="14631448" story="6200199" up="0" down="0" replies="1" replyto="" user="exby2004" level="0" root="14631448">
<![CDATA[ Loved the article... does anyone have a larger version of the image - <a class="user" href="http://www.codingthewheel.com/image.axd?picture=internet_blueprint2.jpg">http://www.codingthewheel.com/image.axd?picture=in ...</a> from this article?
]]>
</comment>
<comment date="1208716817" id="14623970" story="6200199" up="0" down="0" replies="0" replyto="" user="csiunatc" level="0" root="14623970">
<![CDATA[ Great Article, interesting info and good writing!
]]>
</comment>
</events>
Even if you're not an XML afficionado, you can see that there are three comments here. If you look closely, you'll see that each comment has both a "user" and a "date" attribute, along with some other data. Now, if only you could get your hands on that data, in the context of your website or blog... you'd have everything you need to display integrated Digg comments.
Digg Responses
You're not required to work with XML. You can instruct Digg.com to frame its responses in one of four formats:
- XML
- JSON
- Javascript
- PHP
You can easily tweak the response type by adding a "type" parameter to the request. Here's one that returns the exact data you saw above, but this time in a serialized PHP format:
http://services.digg.com/stories/6200199/comments?appkey=http%3A%2F%2Fwww.codingthewheel.com&type=php
Here's the response returned for the above request:
This is serialized PHP. While it may look like a lot of gibberish: it makes your life easier, and in practice you'll rarely, if ever, have to actually look at it. Instead, you'll deserialize it, and talk to a set of friendly objects.
What does the appkey query parameter do?
Every request to Digg.com should include an "app key" or application key, which identifies the particular website or application making the request.
http://services.digg.com/stories/6200199/comments?appkey=http%3A%2F%2Fwww.codingthewheel.com
The application key is just the absolute URL of your website, suitably encoded (URL-encoded, to be specific). According to the Digg.com API documentation:
The value of the appkey argument is up to you, within the constraints described below. Digg does not issue or authenticate appkeys at this time. It is used only for statistical purposes.
You may not be able to keep your selected appkey secret. We can't think of any reason someone would want to sniff your appkey and use it for their own, but silly mischief is always a possibility.
The value of the appkey argument must be a valid absolute URI (see IETF RFC 2396) that identifies the application making the request. The URI might point to:
- The application itself, if it's a web application.
- A web page describing the application.
- A web page offering the application for download.
- The author's web site.
Or you may use a URI that doesn't actually point to a document on the worldwide web, but uniquely identifies your application.
The value of the appkey argument must be URL encoded
So assuming your site is www.somesite.com, your application key would look like this:
appkey=http%3A%2F%2Fwww.somesite.com
Make sure to append that to the end of each and every request you make to the Digg API, and otherwise you can forget about it.
How to retrieve the Digg Story ID for your stories
One of the problems I had early on was figuring out how to retrieve the story ID for a given arbitrary story, given the original URL of the page on which the story was published.
In other words, given some hypothetical URL...
http://www.codingthewheel.com/archives/the-great-game
...how do you retrieve the Digg Story ID for this URL? (assuming it's been submitted to Digg in the first place)
Well, you can always do it manually, but it makes for a lot of busy-work:
- Each time you post a story to Digg.com, figure out it's Digg Story ID. Take that back to your original website where the content was posted, and store it in your database, or in a hidden field on the story's dedicated page.
- Use this "cached" Story ID whenever you need to make requests to the Digg.com API, for example, to display integrated Digg comments.
Not to mention: what happens when someone else submits one of your stories to Digg, without you knowing about it?
So what we want is a way to grab this information on the fly, for any arbitrary page on our site. As luck would have it, the Digg API allows you to do this through the link query parameter.
http://services.digg.com/stories?link=http%3A%2F%2Fwww.codingthewheel.com%2Farchives%2Fthe-great-game&appkey=http%3A%2F%2Fwww.codingthewheel.com
Go ahead and click the link if you want. You should see something like this:
<stories timestamp="1209508634" total="1" offset="0" count="1">
<story id="6200199" link="http://www.codingthewheel.com/archives/the-great-game" submit_date="1208662940" diggs="34" comments="4" status="upcoming" media="news" href="http://digg.com/programming/Life_Changing_Blogger_Success_The_Greatest_Game_Ever_Played">
<title>Life-Changing Blogger Success: The Greatest Game Ever Played</title>
<description>Required reading for web developers, bloggers, SEO consultants, online authors, web hobbyists, and so forth. Come one come all. Bring on the Game!</description>
<user name="nswrambo" icon="http://digg.com/img/udl.png" registered="1207465958" profileviews="31" fullname="John Smith" />
<topic name="Programming" short_name="programming" />
<container name="Technology" short_name="technology" />
<thumbnail originalwidth="700" originalheight="816" contentType="image/jpeg" src="http://digg.com/programming/Life_Changing_Blogger_Success_The_Greatest_Game_Ever_Played/t.jpg" width="80" height="80" />
</story>
</stories>
The XML response consists of a stories collection containing zero or more story elements. There can be multiple Digg story IDs associated with a particular URL, for example, in the case of duplicate submissions, or if a particular URL happens to contain two or more stories. But generally and on average, you'll find that there's a one-to-one correlation between the original URL of the article, and the Digg Story ID corresponding to that article.
I've bolded and underlined the relevant piece of data: the "id" attribute for the story. There you see it: 6200199. Once you have this piece of information, you can then make various other requests to Digg - such as retrieving a list of comments - which require that you have a Story ID.
How to retrieve the comments associated with a given story
So now that you know how to retrieve the numeric Story ID for a given page on your site, you can extract the list of Digg comments associated with the story. You've already seen (above) the format of the request you'll need to make:
http://services.digg.com/stories/6200199/comments?appkey=http%3A%2F%2Fwww.codingthewheel.com
Simple replace the story ID with the appropriate ID for your own Digg submissions.
Actually, the above request will only return the top-level comments for a given story. If you want to drill down into comments-on-comments, or comments-on-comments-on-comments, you'll need to make a separate request. This is beyond the scope of this article, but if you can get top-level comments working, you shouldn't have too much trouble drilling down and exposing sub-comments as well.
How to get the comment date into a readable format
Digg uses the Unix "epoch" format for date times. Under this format a date/time is expressed as:
the number of seconds elapsed since midnight Coordinated Universal Time (UTC) of January 1, 1970, not counting leap seconds
So decoding it is just a matter of calculation. Using C#, it would look something like this: (there are many, many code samples showing how to work with Unix datetime formats, in all languages - a good Google search will get you started)
{
double numberOfSeconds = double.Parse(unixTime);
DateTime myDate = new DateTime(1970, 1, 1, 0, 0, 0, DateTimeKind.Utc);
return myDate.ToLocalTime().AddSeconds(numberOfSeconds);
}
One potential implementation using ASP.NET
So you understand how to make a request to the Digg API - roughly. You understand that you can decode the data returned by that request - roughly. But what does it all look like when you put it together?
Here's a sample, extremely rough, non-refactored, naive, basic implementation using ASP.NET. It does the following:
- Assemble the appkey and link parameters.
- Create a valid HTTP request object, asking Digg.com for the Story ID corresponding to the original source URL of the Digg story (here on Coding the Wheel).
- Invoke that request.
- Parse the returned XML data to extract the Story ID.
You could place this code in a Page.Load event handler, for example. There are better approaches, such as placing your Digg comment extraction logic in an HTTP Handler, referenced by a piece of Javascript/AJAX on the client side, but that's beyond the scope of this article.
//string theAppKey = "http%3A%2F%2Fwww.codingthewheel.com";
string theAppKey = Server.UrlEncode("http://" + Request.Url.Host);
// Assumes Request.URL identifies the official URL for this post, on your website.
// However you get it, this URL needs to match the official URL for the story as submitted to Digg.com
string theLink = Server.UrlEncode(Request.Url);
// First, get the Digg story ID corresponding to the page's original URL
string diggEndpoint = String.Format("http://services.digg.com/stories?link={0}&appkey={1}", theLink, theAppKey);
HttpWebRequest req = (HttpWebRequest)HttpWebRequest.Create(diggEndpoint);
// Don't forget set the AppKey as the User Agent of the request
req.UserAgent = theAppKey;
// Zap - go out and get the response
HttpWebResponse resp = (HttpWebResponse)req.GetResponse();
// Cool, we got the response, and no exceptions thrown. Now extract the Story ID from the returned data.
// For simplicity, I'm looping across every node in the XML document. There are quicker/cleaner ways.
int storyID = 0;
using (XmlTextReader r = new XmlTextReader(resp.GetResponseStream()))
{
r.MoveToContent();
while (r.Read())
{
if (r.Name == "story" && r.NodeType == XmlNodeType.Element)
{
string id = r.GetAttribute("id");
storyID = Int32.Parse(id);
// woot! got it
break;
}
}
}
if (storyID == 0)
{
// blow up, throw an exception, return an error code, whatever..
return null;
}
// Now that we've got the story ID, we can get the story comments...
diggEndpoint = String.Format("http://services.digg.com/stories/{0}/comments?appkey={1}", storyID, theAppKey);
req = (HttpWebRequest)HttpWebRequest.Create(diggEndpoint);
req.UserAgent = theAppKey;
resp = (HttpWebResponse)req.GetResponse();
using (XmlTextReader r = new XmlTextReader(resp.GetResponseStream()))
{
r.MoveToContent();
while (r.Read())
{
if (r.Name == "comment" && r.NodeType == XmlNodeType.Element)
{
string user = r.GetAttribute("user");
string date = r.GetAttribute("date");
string text = r.ReadString();
// TODO: emit page content representing the comment for the given
// user, date, and comment text.
}
}
}
The code is fairly straightforward. We're making two separate HTTP requests to service.digg.com. The first request retrieves the story ID, the second request retrieves the list of comments associated with that story.
After that, it's up to you to figure out how to integrate this data with the rest of your comments, your particular CMS or blogging platform, etc. But you've seen the broad strokes.
Some other things to keep in mind
- I'm fairly new to Digg.com customization myself! So there will probably be better and/or easier ways to do everything I've described.
- Read the Digg API documentation, starting with the basic concepts.
- Review the specific Digg API page for whatever functionality you're trying to build
- If you get stuck, ask the Digg API community for help - that's where the real experts are.
- Don't use XML just because that's what I'm using, or that's what you see a lot of examples in. If you're running a PHP website: use PHP. If you're a Javascript guru: use Javascript.
Conclusion
This article barely scratches the surface of what's possible using the Digg API. It would be possible to write a book on this subject, but hopefully this article has given you enough to get started.
If you'd like to check out the (very simple, but demonstrative) way I'm using integrated Digg comments here on Coding the Wheel, check out the comments for my Great Game post. Or assuming this post itself gets Digged, you might be able to see some comments below.
Other than that, thanks for reading (as always), and good luck.
Posted by James Devlin 12 comment(s)
