How to Display Integrated Digg.com Comments on Your Website

Monday, April 28, 2008

Introduction

So you've had some success using Digg.com as a venue to promote your content. You've managed to rack up a few posts with high Digg counts; maybe you've even had your day on the front page. You know that even moderately popular Digg postings get a lot of attention; so you start evaluating everything you write in terms of its "Diggability", for better or for worse.

Then you notice something: a lot of people are reading your content, and commenting on it; but most of them are doing so through your story widget on Digg.com, rather than domestically, on your website. Indeed, the more popular the article gets, the more it can feel like your site has gotten lost in the shuffle. It looks something like this:

  • Story X, Digg.com - 156 comments
  • Story X, AuthorsBlog.com - 2 comments

You're happy to get any comments, of course, on Digg.com or elsewhere. But it sure would be nice if you could magically "export" that juicy Digg comment list back to the page which originated the article, so that Digg comments and domestic comments are displayed seamlessly together. One big, happy family: isn't that how it should be?

Integrated Digg comments

So today, I'd like to talk about how to leverage the Digg API to accomplish just that. First, let me state the Golden Rule of Digg.com Customization:

You can access virtually any aspect of Digg content by making a plain-vanilla HTTP request, and massaging and displaying the response to that request using PHP, ASP.NET, Javascript, or what ever technology you want. 

The smart folks behind Digg.com have put together an extremely flexible API which allows you to filter and extract just about anything - users, comments, digg counts - by querying a particular HTTP endpoint. If the term "HTTP endpoint" isn't particularly meaningful to you - allow me to rephrase. You use the Digg API by making requests for particular URLs, and interpreting the data those requests return. The format of the URL is determined (and documented) by Digg, as is the format of the data you get back.

The upshot is that there's no single way to implement Digg comments on your site. There are about a hundred ways, each slightly different than the next.

Digg's Request-Based API

When I say a "plain-vanilla" HTTP request, I mean something you could type into a browser window. Click the following link to see this in practice:

http://services.digg.com/stories/6200199/comments?appkey=http%3A%2F%2Fwww.codingthewheel.com

This is a real-world use of the Digg API. It says, "get me all the top-level Digg comments associated with Story #6200199, whatever that is."

(Every story on Digg is identified by a unique number, and 6200199 just happens to reference The Great Game, a post I published last week.)

Here's the data returned when the above URI is requested:

<?xml version="1.0" encoding="utf-8" ?>

<events timestamp="1209451203" total="3" offset="0" count="3">

   <comment date="1208754737" id="14636430" story="6200199" up="0" down="0" replies="0" replyto="" user="plsntvll" level="0" root="14636430">
   <![CDATA[ Really interesting article, as always from him.  Check out his site www.codingthewheel.com  It's really fascinating what he comes up with.
   ]]>
   </comment>

   <comment date="1208738921" id="14631448" story="6200199" up="0" down="0" replies="1" replyto="" user="exby2004" level="0" root="14631448">
   <![CDATA[ Loved the article... does anyone have a larger version of the image - <a class="user" href="http://www.codingthewheel.com/image.axd?picture=internet_blueprint2.jpg">http://www.codingthewheel.com/image.axd?picture=in ...</a> from this article?
   ]]>
   </comment>

   <comment date="1208716817" id="14623970" story="6200199" up="0" down="0" replies="0" replyto="" user="csiunatc" level="0" root="14623970">
   <![CDATA[ Great Article, interesting info and good writing!
   ]]>
   </comment>

</events>

Even if you're not an XML afficionado, you can see that there are three comments here. If you look closely, you'll see that each comment has both a "user" and a "date" attribute, along with some other data. Now, if only you could get your hands on that data, in the context of your website or blog... you'd have everything you need to display integrated Digg comments.

Digg Responses

You're not required to work with XML. You can instruct Digg.com to frame its responses in one of four formats:

  • XML
  • JSON
  • Javascript
  • PHP

You can easily tweak the response type by adding a "type" parameter to the request. Here's one that returns the exact data you saw above, but this time in a serialized PHP format:

http://services.digg.com/stories/6200199/comments?appkey=http%3A%2F%2Fwww.codingthewheel.com&type=php

Here's the response returned for the above request:

O:13:"DiggAPIEvents":5:{s:9:"timestamp";i:1209506457;s:5:"total";s:1:"3";s:6:"offset";i:0;s:8:"comments";a:3:{i:0;O:14:"DiggAPIComment":11:{s:4:"date";i:1208754737;s:2:"id";i:14636430;s:5:"story";i:6200199;s:2:"up";s:1:"0";s:4:"down";s:1:"0";s:7:"replies";s:1:"0";s:7:"replyto";N;s:4:"user";s:8:"plsntvll";s:7:"content";s:138:"Really interesting article, as always from him.  Check out his site www.codingthewheel.com  It's really fascinating what he comes up with.";s:5:"level";i:0;s:4:"root";i:14636430;}i:1;O:14:"DiggAPIComment":11:{s:4:"date";i:1208738921;s:2:"id";i:14631448;s:5:"story";i:6200199;s:2:"up";s:1:"0";s:4:"down";s:1:"0";s:7:"replies";s:1:"1";s:7:"replyto";N;s:4:"user";s:8:"exby2004";s:7:"content";s:242:"Loved the article... does anyone have a larger version of the image - <a class="user" href="http://www.codingthewheel.com/image.axd?picture=internet_blueprint2.jpg">http://www.codingthewheel.com/image.axd?picture=in ...</a> from this article?";s:5:"level";i:0;s:4:"root";i:14631448;}i:2;O:14:"DiggAPIComment":11:{s:4:"date";i:1208716817;s:2:"id";i:14623970;s:5:"story";i:6200199;s:2:"up";s:1:"0";s:4:"down";s:1:"0";s:7:"replies";s:1:"0";s:7:"replyto";N;s:4:"user";s:8:"csiunatc";s:7:"content";s:51:"Great Article, interesting info and good writing!";s:5:"level";i:0;s:4:"root";i:14623970;}}s:5:"count";i:3;}

This is serialized PHP. While it may look like a lot of gibberish: it makes your life easier, and in practice you'll rarely, if ever, have to actually look at it. Instead, you'll deserialize it, and talk to a set of friendly objects.

What does the appkey query parameter do?

Every request to Digg.com should include an "app key" or application key, which identifies the particular website or application making the request.

http://services.digg.com/stories/6200199/comments?appkey=http%3A%2F%2Fwww.codingthewheel.com

The application key is just the absolute URL of your website, suitably encoded (URL-encoded, to be specific). According to the Digg.com API documentation:

The value of the appkey argument is up to you, within the constraints described below. Digg does not issue or authenticate appkeys at this time. It is used only for statistical purposes.

You may not be able to keep your selected appkey secret. We can't think of any reason someone would want to sniff your appkey and use it for their own, but silly mischief is always a possibility.

The value of the appkey argument must be a valid absolute URI (see IETF RFC 2396) that identifies the application making the request. The URI might point to:

  • The application itself, if it's a web application.
  •  A web page describing the application.
  • A web page offering the application for download.
  • The author's web site.

 Or you may use a URI that doesn't actually point to a document on the worldwide web, but uniquely identifies your application.

The value of the appkey argument must be URL encoded

So assuming your site is www.somesite.com, your application key would look like this:

appkey=http%3A%2F%2Fwww.somesite.com

Make sure to append that to the end of each and every request you make to the Digg API, and otherwise you can forget about it.

How to retrieve the Digg Story ID for your stories

One of the problems I had early on was figuring out how to retrieve the story ID for a given arbitrary story, given the original URL of the page on which the story was published.

In other words, given some hypothetical URL...

http://www.codingthewheel.com/archives/the-great-game

...how do you retrieve the Digg Story ID for this URL? (assuming it's been submitted to Digg in the first place)

Well, you can always do it manually, but it makes for a lot of busy-work:

  1. Each time you post a story to Digg.com, figure out it's Digg Story ID. Take that back to your original website where the content was posted, and store it in your database, or in a hidden field on the story's dedicated page.
  2. Use this "cached" Story ID whenever you need to make requests to the Digg.com API, for example, to display integrated Digg comments.

Not to mention: what happens when someone else submits one of your stories to Digg, without you knowing about it?

So what we want is a way to grab this information on the fly, for any arbitrary page on our site. As luck would have it, the Digg API allows you to do this through the link query parameter.

http://services.digg.com/stories?link=http%3A%2F%2Fwww.codingthewheel.com%2Farchives%2Fthe-great-game&appkey=http%3A%2F%2Fwww.codingthewheel.com

Go ahead and click the link if you want. You should see something like this:

<?xml version="1.0" encoding="utf-8" ?>

<stories timestamp="1209508634" total="1" offset="0" count="1">

<story id="6200199" link="http://www.codingthewheel.com/archives/the-great-game" submit_date="1208662940" diggs="34" comments="4" status="upcoming" media="news" href="http://digg.com/programming/Life_Changing_Blogger_Success_The_Greatest_Game_Ever_Played">
<title>Life-Changing Blogger Success: The Greatest Game Ever Played</title>
<description>Required reading for web developers, bloggers, SEO consultants, online authors, web hobbyists, and so forth. Come one come all. Bring on the Game!</description>
<user name="nswrambo" icon="http://digg.com/img/udl.png" registered="1207465958" profileviews="31" fullname="John Smith" />
<topic name="Programming" short_name="programming" />
<container name="Technology" short_name="technology" />
<thumbnail originalwidth="700" originalheight="816" contentType="image/jpeg" src="http://digg.com/programming/Life_Changing_Blogger_Success_The_Greatest_Game_Ever_Played/t.jpg" width="80" height="80" />
</story>

</stories>

The XML response consists of a stories collection containing zero or more story elements. There can be multiple Digg story IDs associated with a particular URL, for example, in the case of duplicate submissions, or if a particular URL happens to contain two or more stories. But generally and on average, you'll find that there's a one-to-one correlation between the original URL of the article, and the Digg Story ID corresponding to that article.

I've bolded and underlined the relevant piece of data: the "id" attribute for the story. There you see it: 6200199. Once you have this piece of information, you can then make various other requests to Digg - such as retrieving a list of comments - which require that you have a Story ID.

How to retrieve the comments associated with a given story

So now that you know how to retrieve the numeric Story ID for a given page on your site, you can extract the list of Digg comments associated with the story. You've already seen (above) the format of the request you'll need to make:

http://services.digg.com/stories/6200199/comments?appkey=http%3A%2F%2Fwww.codingthewheel.com

Simple replace the story ID with the appropriate ID for your own Digg submissions.

Actually, the above request will only return the top-level comments for a given story. If you want to drill down into comments-on-comments, or comments-on-comments-on-comments, you'll need to make a separate request. This is beyond the scope of this article, but if you can get top-level comments working, you shouldn't have too much trouble drilling down and exposing sub-comments as well.

How to get the comment date into a readable format

Digg uses the Unix "epoch" format for date times. Under this format a date/time is expressed as:

the number of seconds elapsed since midnight Coordinated Universal Time (UTC) of January 1, 1970, not counting leap seconds

So decoding it is just a matter of calculation. Using C#, it would look something like this: (there are many, many code samples showing how to work with Unix datetime formats, in all languages - a good Google search will get you started)

public static DateTime ConvertUnixTimeToDateTime(string unixTime)
{
    double numberOfSeconds = double.Parse(unixTime);
    DateTime myDate = new DateTime(1970, 1, 1, 0, 0, 0, DateTimeKind.Utc);
    return myDate.ToLocalTime().AddSeconds(numberOfSeconds);
}

One potential implementation using ASP.NET

So you understand how to make a request to the Digg API - roughly. You understand that you can decode the data returned by that request - roughly. But what does it all look like when you put it together?

Here's a sample, extremely rough, non-refactored, naive, basic implementation using ASP.NET. It does the following:

  1. Assemble the appkey and link parameters.
  2. Create a valid HTTP request object, asking Digg.com for the Story ID corresponding to the original source URL of the Digg story (here on Coding the Wheel).
  3. Invoke that request.
  4. Parse the returned XML data to extract the Story ID.

You could place this code in a Page.Load event handler, for example. There are better approaches, such as placing your Digg comment extraction logic in an HTTP Handler, referenced by a piece of Javascript/AJAX on the client side, but that's beyond the scope of this article.

// Assemble the Digg.com "appkey" and "link" parameters...
//string theAppKey = "http%3A%2F%2Fwww.codingthewheel.com";
string theAppKey = Server.UrlEncode("http://" + Request.Url.Host);

// Assumes Request.URL identifies the official URL for this post, on your website.
// However you get it, this URL needs to match the official URL for the story as submitted to Digg.com
string theLink = Server.UrlEncode(Request.Url);

// First, get the Digg story ID corresponding to the page's original URL
string diggEndpoint = String.Format("http://services.digg.com/stories?link={0}&appkey={1}", theLink, theAppKey);
HttpWebRequest req = (HttpWebRequest)HttpWebRequest.Create(diggEndpoint);

// Don't forget set the AppKey as the User Agent of the request
req.UserAgent = theAppKey;

// Zap - go out and get the response
HttpWebResponse resp = (HttpWebResponse)req.GetResponse();

// Cool, we got the response, and no exceptions thrown. Now extract the Story ID from the returned data.
// For simplicity, I'm looping across every node in the XML document. There are quicker/cleaner ways.

int storyID = 0;
using (XmlTextReader r = new XmlTextReader(resp.GetResponseStream()))
{
    r.MoveToContent();
    while (r.Read())
    {
        if (r.Name == "story" && r.NodeType == XmlNodeType.Element)
        {
            string id = r.GetAttribute("id");
            storyID = Int32.Parse(id);
            // woot! got it
            break;
        }
    }
}

if (storyID == 0)
{
    // blow up, throw an exception, return an error code, whatever..
    return null;
}

// Now that we've got the story ID, we can get the story comments...
diggEndpoint = String.Format("http://services.digg.com/stories/{0}/comments?appkey={1}", storyID, theAppKey);
req = (HttpWebRequest)HttpWebRequest.Create(diggEndpoint);
req.UserAgent = theAppKey;
resp = (HttpWebResponse)req.GetResponse();

using (XmlTextReader r = new XmlTextReader(resp.GetResponseStream()))
{
    r.MoveToContent();
    while (r.Read())
    {
        if (r.Name == "comment" && r.NodeType == XmlNodeType.Element)
        {
            string user = r.GetAttribute("user");
            string date = r.GetAttribute("date");
            string text = r.ReadString();

            // TODO: emit page content representing the comment for the given
            // user, date, and comment text.
        }
    }
}

The code is fairly straightforward. We're making two separate HTTP requests to service.digg.com. The first request retrieves the story ID, the second request retrieves the list of comments associated with that story.

After that, it's up to you to figure out how to integrate this data with the rest of your comments, your particular CMS or blogging platform, etc. But you've seen the broad strokes.

Some other things to keep in mind

  1. I'm fairly new to Digg.com customization myself! So there will probably be better and/or easier ways to do everything I've described.
  2. Read the Digg API documentation, starting with the basic concepts. 
  3. Review the specific Digg API page for whatever functionality you're trying to build
  4. If you get stuck, ask the Digg API community for help - that's where the real experts are.
  5. Don't use XML just because that's what I'm using, or that's what you see a lot of examples in. If you're running a PHP website: use PHP. If you're a Javascript guru: use Javascript.

Conclusion

This article barely scratches the surface of what's possible using the Digg API. It would be possible to write a book on this subject, but hopefully this article has given you enough to get started.

If you'd like to check out the (very simple, but demonstrative) way I'm using integrated Digg comments here on Coding the Wheel, check out the comments for my Great Game post. Or assuming this post itself gets Digged, you might be able to see some comments below.

Other than that, thanks for reading (as always), and good luck.

Tags: Digg.com, XML, ASP.NET, social networking

21 comment(s)

Cool! I'll be trying this out shortly. Thanks for the information. -John

it sure would be nice if you could magically "export" that juicy Digg comment list back to the page which originated the article, so that Digg comments and domestic www.stapleygaragedoor.com comments are displayed seamlessly together. One big, happy family: isn't that how it should be?

digg sucks i hate digg.com with a passion all you looooosers need to learn some truth and come on over to reddit.com where the big boys play...

There are better approaches, such as placing your Digg comment extraction logic in an HTTP Handler, referenced by a piece of Javascript/AJAX on the client side, but that's beyond the scope of this article

Great article, James! Another way to get information on a story is by using the "clean title", from the URL on Digg:

http://digg.com/programming/Life_Changing_Blogger_Success_The_Greatest_Game_Ever_Played

...becomes:

http://services.digg.com/story/Life_Changing_Blogger_Success_The_Greatest_Game_Ever_Played?appkey=http%3A%2F%2Fwww.codingthewheel.com

You can also get all stories from your domain like this: http://services.digg.com/stories?domain=codingthewheel.com&appkey=http%3A%2F%2Fwww.codingthewheel.com

Some borked HTML crept into my comment up there, all I pasted was some raw URLs.

Interesting post- new twist- I see you are still working out the 'kinks', but looking pretty good. what i want now to know is: any way for to comment replies made on CTW back onto Digg? 2-way?

One thing I am not sure about is how to inject the DIGG comments into my existing comments list for that post. I could implement an extension that retrieves during the page load but wouldn't that happen everytime someone loaded the blog entry? Wouldn't I want to somehow evaluate what comments have already been scraped off the DIGG site and then append any new comments to my list of comments?

Suggestions?

I love my ghd outlet Glattetang! It heats up very quickly and works well. My glamour hair is annoyingly thick and poofy, but ghdstraighteneroutletaustralia works wonders! I have never been so fully satisfied with just ghd straightener outlet supplier! What a pleasure shopping at this ghd outlet australia! Thank you very much for this wonderful shopping experience. I will be shopping ghd outlet very very often.

ghd rettetang has lasted for a long time, with no proplems. Few hairs are snagged and my hair is left shiny and nice. ghd rettetang til salgsis my PREFERRED brand, and thanks for the best price.  I have heard a number of people saying how good ghd glattetang are so when I got the chance to use them I found they were not a patch on my one, so don't be fooled by the name!! I would be recommending the ghd rettetang i norge to everyone and would tell people to try both brands and put them to the test and billig ghd will certainly come out on top. kjøp ghd rettetang is the most amazing ghd there possibly could be.My first choice every time for Totally excellent ghd rettetang !

Bike

[b][url=http://www.comelygirls.com/hair straightener][b] The Solia 1?" flat iron features the Dynamic Alignment System to ensure perfect contact between your hair and the plates. It supplies even heat distribution for true silky straight hair. Its ceramic/tourmaline ion technology will generate gentle far-infrared heat and extra negative ions to create the perfectly frizz-free hair without heat damage. Kiss bad hair days goodbye!

クリントン米国務長官は4日、ミャンマーの国会補選実施などの民主化努力を評価し、同国に対する金融サービス投資禁止や政府高官の渡米禁止など一部制裁の緩和措置を取る用意があると発表した。また、駐ミャンマー米大使を近く指名すると明らかにした。  民主化運動指導者アウンサンスーチー氏と同氏率いる国民民主連盟(NLD)が補選で圧勝したことを受け、制裁緩和に着手する姿勢を示すことで、一層の民主化を促すのが狙い。ただ、緩和の対象を限定し、全政治犯釈放や北朝鮮との軍事協力停止を含めた改革を推進するよう圧力を維持する方針も示した。

鈴木克昌幹事長代理、樋高剛総括副幹事長、古賀敬章副幹事長、村上史好、萩原仁、畑浩治、京野公子、大西孝典=以上、幹事長補佐、松崎哲久、階猛、横山北斗、中村哲治=以上、政策調査会副会長、大谷啓、菅川洋、木内孝胤、笠原多見子、石山敬貴、友近聡朗=以上、政策調査会長補佐、玉城デニー、岡本英子、大山昌宏、三宅雪子=以上、広報副委員長、水野智彦、木村剛司、金子健一=以上、企業団体対策副委員長

The industry leaders in monitoring systems and voice communications. Our systems work on new, old, any type and brand of elevator equipment. Integrated Display Computers serve as the foundation for visualization control of processes and machinery. Thanks for sharing the informative post. Regards, Team Building Activities

Nice post.Thank you for taking the time to publish this information very useful! I've been looking for books of this nature for a way too long. I'm just glad that I found yours. Looking forward for your next post. Thanks :) Regards, Sam - Botox Injections Schaumburg

beats by dre of Northern Europe and Denmark, monster by dre belly muscles, which agency that about-face the approach handset, beats by dre headphones the two styles in atramentous

Do you mind if I quote a couple of your articles as long as I provide credit and sources back to your weblog? My blog site is in the exact same niche as yours and my users would certainly benefit from some of the information you provide here. Please let me know if this ok with you. Appreciate it!

Use the form below to leave a comment.






Coding the Wheel has appeared on the New York Time's Freakonomics blog, Jeff Atwood's Coding Horror, and the front page of Reddit, Slashdot, Digg.

On Twitter

Thanks for reading!

If you enjoyed this post, consider subscribing to Coding the Wheel by RSS or email. You can also follow us on Twitter and Facebook. And even if you didn't enjoy this post, better subscribe anyway. Keep an eye on us.

Question? Ask us.

About

Poker

Coding the Wheel =
Code, poker, technology, games, design, geekery.


Hire

You've read our technical articles, you've tolerated our rants and raves. Now you can hire us anytime, day or night, for any project large or small.

Learn more

We Like

Speculation, by Edmund Jorgensen.