Minty Django Feeds

As I've mentioned before, I recently moved my blog over to Django. For the most part the transition has gone smoothly. But one area that has bugged me for a while is that BirdFeeder Pepper, my RSS feed analytics provided by the wonderful analytics package Mint, hasn't been working, so I can't see how many people are reading my site through RSS.

Normally, an RSS feed written in PHP would just require:

define('BIRDFEED', 'Hicks-Wright.net');
include($_SERVER['DOCUMENT_ROOT'].'/feeder/index.php');

at the top of the page to track hits, but since everything is written in Python now, there's no easy way to do that.

So I Googled around a bit to see if anyone else had run into this problem. One person realized they could use Django's database introspection command to get a model for the Mint database. This sounded like a pretty good idea; I would just have to reverse engineer the PHP code for the insertion part of BirdFeeder into Python, and everything else would just work.

Unfortunately, this was much harder than I expected. Shaun Inman, creator of Mint and generally nice guy, has built a pretty sophisticated infrastructure in Mint that he uses extensively. It obviously has made his life easier, but it was not going to do the same for mine.

So, I thought of a new way: fake it. BirdFeeder works by being included by PHP in the feed code, and from there it grabs the relevant stats from the headers and makes the database insertion. But it has some protections on it to keep it from being accessed directly, so the first thing is to move

define('BIRDFEED', 'Hicks-Wright.net');

into /feeder/index.php. Next, I add another condition to /feeder/index.php by changing:

if (!defined('BIRDFEED'))

into

if(!defined(BIRDFEED') || $_GET['pw'] != 'password')

This is a simple hack, but it should be enough to keep people from messing with my stats.

Next, I create a new view in Django called feeds

def feeds(request, url, feed_dict):
    headers = {'HTTP_HOST':'Host',
               'HTTP_USER_AGENT':'User-Agent',
               'HTTP_ACCEPT_ENCODING':'Accept-Encoding',
               'HTTP_ACCEPT_LANGUAGE':'Accept-Language',
               'HTTP_REFERER':'Referer',
               'HTTP_COOKIE':'Cookies'}

    # Call into BirdFeeder with the request headers.                            
    o = urllib2.build_opener()
    o.addheaders = [(headers[k], request.META.get(k, '')) \
            for k in request.META.keys() if k in headers.keys()]
    f = o.open("http://hicks-wright.net/feeder/index.php?pw=password")
    f.read()
    f.close()

    return django.contrib.syndication.views.feed(request, url=url, feed_dict=feed_dict)

This opens an HTTP connection to BirdFeeder with my password attached, then forwards the rest of the request on to the built-in Django feed view.

Finally, I wanted to also track Seeds, the number of incoming links I got from my feed to my site. In the old PHP feed, it was done like this:

print "<link>".$BirdFeeder->seed($title, $url)."</link>\n";

But, being in Python, I couldn't do that. So instead, I just replaced the link in the Feed object by defining the method item_link():

class BlogFeed(Feed):
    title = "Hicks-Wright.net"
    link = "http://hicks-wright.net"
    ...
    def item_link(self, item):
        return '%s/feeder/?FeederAction=clicked&feed=%s&seed=%s&seed_title=%s' % \
               (self.link,
                urlencode(self.title), 
                urlencode(self.link + item.get_absolute_url()), 
                item.title)

This passes the site URL, feed title, item URL, and item title to BirdFeeder, which counts the Seed and then forwards the person on to the article.

Posted September 14, 2007