Providing JSON Feeds in Addition to RSS ⁂ starbreaker.org

RSS was revolutionary twenty years ago. We finally had an alterative to the sort of push technology Microsoft was trying to sell us with active channels. Instead of having stuff pushed to you, you could subscribe to a feed using any client application that implemented the standard and set up your client to request updates at your convenience — whether you pulled the feed hourly, daily, weekly, or even monthly was up to you.

However, RSS is still a limited standard by default, though being XML-based it could be extended with namespaced elements. For example, it has a pubDate element for when an item had been created, but did not provide a corresponding date for when that item had been updated.

It gets worse, though. RSS was plainly designed to provide summaries with links to the original website.

full-text RSS feed sizes

However, if you were determined to provide a full-text feed before RSS 2.0, you had to stick your HTML inside a CDATA section and then stick that inside your description element. RSS 2.0 gave us a content:encoded element for article text, but you still needed your CDATA block.

It is, quite frankly, a pain in the ass. I know this from experience. So, I had been thinking about providing JSON feeds for a more modern approach to syndication.

why not Atom feeds?

I did consider providing Atom as well as RSS. Atom provides a content element specifically for full-text feeds, as well as an updated date. However, Atom dates to 2005 and has not been updated. While you can put HTML inside Atom’s content element with type="html", it must be escaped. If you use text="xhtml" instead, your content must validate as XHTML and be wrapped inside a div element with a xmlns="http://www.w3.org/1999/xhtml" attribute.

I’ve tried generating full-text Atom feeds with a shell script before. Getting the result to validate was not fun. Atom doesn’t seem to account for HTML5, which I’m using on this website. I might try Atom again as a replacement for RSS, and it isn’t worth my while to provide two different XML feed formats.

JSON feeds are a fairly mature format, though to my knowledge the spec hasn’t been adopted by IETF or the W3C. Unfortunately, support for JSON feeds in GNU/Linux feed reader apps like Liferea, newsboat, and Akregator isn’t there yet. However, macOS and iOS has NetNewsWire, which does support JSON feeds.

The spec for JSON feeds was easy enough to follow, and not hard to implement using variations on the shell scripts I had written to generate my RSS feeds. However, escaping the HTML so I could include it in my feed’s content_html fields was fun. I ended up running incoming HTML through jq -sR . to escape quotes, newlines, and anything else that might prevent my JSON feed from validating.

Right now my JSON feeds are mainly for use with EchoFeed. It would be nice if social platforms would just grab my feeds or let me specify a feed source for my account, but I’ve been griping about that since Atom was the new hotness. It would also be nice if plugins like Awesome RSS for Firefox showed JSON feeds as well as RSS and Atom feeds since I made a point of exposing them in each page’s head. But Rome wasn’t built in a day. I did my part; the rest is up to others.

I would like to eventually provide only one format, but that’s a concern best left to my future self. Never mind that I could have stuck to only providing RSS, but unfortunately web development has generally moved away from XML in favor of JSON because JavaScript developers find dealing with the older, more verbose format painful. For now I am content to take a cue from Bryce Wray, who explains his approach in Why have both RSS and JSON feeds? and Not so fast, there, fella.