Clickstream

RSS Sucks For Mashups, But It's Ubiquitous

Another thing we learned while banging away at web sites to get data was that RSS stinks for slinging web data, yet the tools available don't deal well with Atom feeds. Atom is great for this purpose, but hard to locate tools for (other than code libraries). Being both ignorant and lazy, I'd prefer to integrate tools rather than write 3x that much code in PHP and Python.

Our first attempt to grab data from Salesforce.com generated an Atom feed. We did that because we wanted to get a bunch of data and feed different elements to other extractors to collect data. Sadly, none of the tools we were using could consume Atom. We had a choice of programming a custom widget for parsing Atom to get the data we wanted or switching to RSS. We did the latter because it was expedient given the time constraints.

The problem with using RSS for data is that it only has a few fields to store data in. We ended up using the title, description, link, and even the date fields. This meant having to take 8 fields of extracted data and publish two separate RSS feeds, or stuffing multiple values into a single field like description. Luckily, we didn't need some fields broken out so stuffed things into fewer fields. Still, I had something like nine RSS feeds going at one point.

The other side was getting feeds. Lots of sites generate RSS feeds, but no Atom. So we're stuck with crappy feeds or we do it ourselves, but then have to DIY the consuming side as well. Let's hope Atom usage picks up in the future.

Labels: Atom, integration, mashup, mashup camp, mashupcamp4, RSS, web2.0

Posted by Mark Wednesday, July 25, 2007 12:09:00 AM | permalink |