HOWTO: Adding an RSS feed to a Subversion Server

I have now gone on stage twice and talked about how useful an RSS feed is for a subversion server. About time I actually document how to set up such a thing.

This tutorial will start by creating a local subversion server. The rest of the directions are universally applicable with the caveat that you need to make sure that the user running the RSS generation actually has write access to wherever the RSS feed(s) are to be written.

Of course, this all assumes Mac OS X. It would only require a bit of modification to make this work on any random Unix derivative and a slight bit more for Windows.

You’ll need to have a Subversion package installed that includes the server bits. I typically grab the bits from darwinports or http://fink.sourceforge.net/. Wilfredo Sanchez’s iDisk has prebuilt bits, too.

Create a Local Subversion Repository

The following command line will create a subversion repository at the given path.

svnadmin create --fs-type fsfs /Volumes/Data/svn

Grab the Necessary RSS Generation Scripts

From Dalke Scientific’s Python page, you’ll need to grab PyRSS2Gen (tarball).

Copy this to the hooks directory of the repository. In my case:

cp PyRSS2Gen.py /Volumes/Data/svn/hooks/

Grab svn2feed.py from the Subversion subversion repository’s contributed hooks directory.

cd /Volumes/Data/svn/hooks/
curl -O http://svn.collab.net/repos/svn/trunk/contrib/hook-scripts/svn2feed.py

Setup the Commit Hook to Generate RSS on Commit

cd /Volumes/Data/svn/hooks/
# copy the code below
pbpaste > post-commit
chmod a+x post-commit

This is the contents of the RSS feed only post-commit.

#!/bin/sh

REPOS="$1"
REV="$2"

/usr/bin/python /Volumes/Data/svn/hooks/svn2feed.py \
    --svn-path /usr/bin/ \
    --max-items=100 \
    --format=atom \
    --revision "$REV" --item-url "http://localhost/svn/" \
    --feed-url="http://localhost/rss/svn.rss" \
    --feed-file "/Library/WebServer/Documents/rss/svn.rss" \
    "$REPOS" &

See the svn2feed.py usage and documentation for more information on what all the parameters actually mean.

Update: Fitz made fun of me for not including the & at the end. The post-commit hook doesn’t care about the exit codes of sub-commands and, therefore, there is no reason to make the user wait for the execution of the various commands.

python svn2feed.py --help

Setup the RSS Directory on the WebServer

This is where the permissions can be tricky. In the following, it is configured to allow read/write by anyone. Solves the permissions problem while completely sacrificing security in that directory. A better solution would be to make the directory owned by and writable by whatever user executes the svn2feed.py script; typically, the web server’s user when using svn-via-http. For non-HTTP svn servers, the directory will likely have to be group writable with the setGID bit set as the “server” side of svn may be executed as different users.

sudo mkdir /Library/WebServer/Documents/rss
sudo chown nobody /Library/WebServer/Documents/rss
sudo chmod a+rwx /Library/WebServer/Documents/rss

Notes

That should be it. It should “just work”.

But, of course, it probably won’t. To troubleshoot, go to the hooks/ directory in a terminal and copy/paste the svn2feed.py command line above. Fill the $REV and $REPOS variables as appropriate and execute the command.

Keep futzing with that until it generates something useful.

Then plug the result back into post-commit and try executing a transaction against the subversion repository. If the feed doesn’t update, it is most likely because there is something different about the execution environment within subversion vs. the user shell.

Since there isn’t a log file generated by the execution of the post-commit script, add “ &> /tmp/munch.log” after the "$REPOS" (but before the & to (hopefully) log some useful diagnostics into /tmp/munch.log.

The formatting of the resulting feed is really really ugly in both NetNewsWire and Safari.

Ugly enough that I couldn’t stand it. Grab svn2feed.py from my subversion repository and see if it works better for you.

It does for me in NetNewsWire, but I’m finding that Safari’s feed caching makes testing difficult. As well, the HTML entities seem to leak into Safari’s output whereas NNW renders them “correctly” (where “correctly” is, in the classic ambiguous HTML sense of the words, defined as “looks good enough for me, HTML definition be damned”).

Forgot to mention: Patches are assuredly welcome!



20 Responses to “HOWTO: Adding an RSS feed to a Subversion Server”

  1. Dominik Wagner says:

    Of course you could just get a hosted account at versionshelf which will start business next week.

    Or also, which is quite simple, grab svnxslt by martin from map’s portfolio which uses xsltproc instead of python. Much more appropriate if you ask me, since svn already emits xml.

  2. bbum says:

    Except that I don’t want stuff on my local machine hosted anywhere but my local machine, nor do many companies want their code hosted at an external site, quality of service guarantee be damned.   Versionshelf looks quite cool, though, if a project falls into the set of projects that can be hosted at such a service.

    Seems like a bit of overkill to fire up an XSLT transformation engine against svn generated XML to spew the feed in comparison to just dumping the RSS/Atom DOM directly based on svn generated plain text.   Neat solution and all, but I fail to see any significant advantage over the somewhat mediocre solution I outlined.

  3. Dominik Wagner says:

    I understand your concerns. But I know I would have been more than happy if such a service would have been around just for my freelance work. Setting up your own secure server isn’t a piece of cake. You’ll have to manage a dedicated server to keep everyone else out, if you do that already, versionshelf won’t be of much interest, of course. Regarding having code only on my local machine: I on the other hand take comfort in the fact that if my local machine breaks and my backups haven’t been the best (maybe that will change with Time Machine) my code still is intact. But I’m glad that you don’t deleted my shameless plug right away.

    As rss generation goes: If xsltproc is available (which it is on mac os x) I just think of xslt as a much cleaner solution. But that’s just me, and maybe I’m to anal about this ;). But running a xslt transformation engine isn’t so much more overkill than running a python script, IMHO.

  4. inkBlots » RSS for Subversion and Dreamweaver says:

    [...] Here’s a nifty article on how to set up an RSS feed for your subversion server. One caveat, though: “Of course, this all assumes Mac OS X. It would only require a bit of modification to make this work on any random Unix derivative and a slight bit more for Windows.” [...]

  5. Daniel J. Luke says:

    trac (http://trac.edgewall.org/) can also be used to generate rss feeds of your svn repository.

    It’s meant to be more than just a feed generator, though, so it’s much more heavyweight.

    (but it’s also available from darwinports).

  6. Michael Heilemann says:

    Or you can have it send a mail (which is pretty common for a repository to do) to a Google Group (which is free, fast and easy to set up), which in turn has its own feeds.

  7. Chris Mear says:

    I have WebSVN installed — it’s a PHP app from the same house as Subversion itself. Along with letting you browse your repository and do diffs in your web browser, it has built-in RSS feeds for any directory. And it’s dead easy to set up (one line of config on most Unix boxes to tell it where your repository lives).

  8. Aristotle Pagaltzis says:

    There’s also subatom, written in Perl. The setup effort for it is roughly identical with what you show here.

  9. Oriol News Portal says:

    Generant RSS de les revisions de Subversion

    Quan treballes en un projecte on hi ha ms d’una persona treballant contra un repositori de codi s molt bona idea poder-se sindicar via RSS a les noves revisions que es van publicant del codi i als…

  10. Jehiah says:

    Insurrection (the worlds best web based svn viewer) also provides rss feeds. In many ways it’s similar to trac; but without the bug tracking, and with more advanced code browsing features.

  11. Arek says:

    And for those who just want to publish ChangeLogs on their webpage there is svn2log script.

  12. links for 2006-09-13 at onwalking.org says:

    [...] bbum’s weblog-o-mat » Blog Archive » HOWTO: Adding an RSS feed to a Subversion Server This tutorial will start by creating a local subversion server. The rest of the directions are universally applicable with the caveat that you need to make sure that the user running the RSS generation actually has write access to wherever the RSS feed(s) (tags: subversion svn rss) [...]

  13. The Random Musings » Blog Archive » links for 2006-10-21 says:

    [...] bbum’s weblog-o-mat » Blog Archive » HOWTO: Adding an RSS feed to a Subversion Server A howto for setting up an RSS feed on a Subversion server. (tags: subversion rssfeed) [...]

  14. Web 2.0 Announcer says:

    bbum?s weblog-o-mat Blog Archive HOWTO: Adding an RSS feed to a Subversion Server

    [...][...]

  15. Jack says:

    For the other 99% of normal world that uses WINDOWS, is there a simple solution to set this up? Does anyone make or know of a solution that can just be installed on a windows box that runs apache and SVN? God-Forbid we make life easy instead of pure frustration to get this to work.

  16. bbum says:

    Hah — your percentages amuse me. In the subversion using world, the % of developers running a Unix variant is much greater. Most Windows developers I know use some god awful windows specific SCM system (their words, not mine) or something terribly expensive (which is what I mostly ran into when I did financial district consulting in NYC and Chicago).

    In any case, there isn’t anything about this particular solution that couldn’t be done on Windows. You just need to install Python and set the paths up correctly.

  17. Sam says:

    I am running a service that does this right now. Just go to: http://svnfeed.com. Here is the blog entry that I wrote about it: http://www.javarants.com/2007/09/22/track-subversion-svn-changes-with-an-rss-feed/

  18. Jack says:

    Sam, you svnfeed site has a mis-spelling. publically should be publicly.

  19. John says:

    Dude I run SVN on windows and use viewvc.cgi , setting up a RSS feed on a windows box isn’t that hard.

    As bbum said, most Windows developers don’t use SVN (which is foolish IMHO) as it’s dirt easy to use and cheap!

    At first the learning curve for the older windows devs was a little nasty but once they ‘got it’ they LOVE SVN…. plus it pushes all of the backup requirements to me as the sysadmin :)

    Cheers, get post btw thanks!

  20. Peter says:

    So – this solution works great with our UberSVN installation on CentOS. Thanks for your post.

    The only thing I came across was that if the message log contained multiple lines then this was being truncated so only the first line would be displayed. With some hackery (and I do apologise) I have came up with a solution to add multiple lines of the log message.

    Using the svn2feed.py linked here in: def _get_item_dict(self, revision): I added the following:


    ....
    info_lines = child_out.readlines()
    child_out.close()
    child_in.close()
    child_err.close()

    #loop info_lines for additional log message lines
    info_line_log = ''
    for i, v in enumerate(info_lines):
    if i >= 3:
    info_line_log += v + ''

    Then adjusted the line that reads:


    % (info_lines[0], info_lines[1], revision, info_lines[3], formatted_changed_data))

    to


    % (info_lines[0], info_lines[1], revision, info_line_log, formatted_changed_data))

    Voilla – multiple lines in the log message.

    Thanks!

Leave a Reply

Line and paragraph breaks automatic.
XHTML allowed: <a href="" title=""> <abbr title=""> <acronym title=""> <b> <blockquote cite=""> <cite> <code> <del datetime=""> <em> <i> <q cite=""> <strike> <strong>