Introduction

Imagine you have an application that stores all your personal content, such as photos, music, videos etc. You share all these with your friends and family regularly, add more photos or videos to already existing shares, add comments to whatever they have shared with you and so on. Since your friend cannot be expected to remember and visit your application to see if there is anything new, your application must notify them each time there is an activity which takes place. There are several methods of notifying users today – Email and Web Syndication being the two most popular ones. Though there are several reasons why Email is still effective; primary being that it is the most universally accepted method, and a good one - to - one communication medium, there are various disadvantages too:

• As volumes increase, users are overwhelmed and lose interest in the content.
• Email messages can also be blocked by filters, and may get marked as spam.
• The notification emails contain useful information like title, description, images, links etc. However they are designed to be human readable. Since the format of these mails is not standardized, there is no reliable method in which these applications can parse the emails and extract the required information.

Syndication

Syndication is a notification mechanism that lets applications inform others of new and updated content. The main benefits of creating syndications include:

1. Syndication is an opt-in process for the user, meaning only if the user is interested in the content does he receive it. This allows users to effectively combat junk or spam mails.
2. It is also the easiest way to keep track of a large number of your favourite websites or blogs, without having to remember to check each site manually or clutter your email Inbox. This saves you from the effort of visiting each of your friend’s Dekoh network site to check the new or modified content shared for you.
3. Syndication feeds are not blocked and you can be sure that the subscribers are notified.
4. Syndication feeds contains event information in a standardized format, making it possible for users to build tools around them. Such tools can aggregate information from different sources, mine this information and offer value added features based on this information. Applications can transform this information to other formats enabling interaction between different applications.

What Dekoh Provides

Dekoh allows its users to keep up-to-date with both forms - timely email notifications and syndication feeds — when, where and how they want it.

How RSS is used in Dekoh

The figure above shows how Dekoh aggregates recent activity information from each of your contacts. Dekoh discovers syndication feeds from all your contacts and automatically subscribes you to these available feeds. Whenever your contacts share/comment a Book, Photo or Music, the corresponding application updates the feed. The new events in the updated feed are displayed on your Dekoh Desktop.

Understanding Web Syndication

Web Syndication is a family of Web feed formats used to publish frequently updated content such as blog entries, news headlines or podcasts. Web syndication, also called a "feed" or "web feed" is basically the backbone of web syndication. It captures event information in a stadard format, which can be either a summary of content from an associated web site or the full text. Web syndication makes it possible for people to keep up with their favorite sites in an automated manner that's easier than checking them manually.

Syndication content is read using software called a "feed reader" or an "aggregator." The user subscribes to a feed by entering the feed's link into the reader or by clicking syndication icon in a browser that initiates the subscription process. The reader checks the user's subscribed feeds regularly for new content, downloading any updates that it finds. We need to understand the constituents of a feed better before leveraging it.

Constituents of a feed

• Event summary: Event summaries are fundamental building blocks of a syndication feed, enacapsulating information about an event. The event summary comprises of identifiers and metadata associated with the event, such as title, description, link, image, author and so on. These are mostly rendered as latest headlines.
• Feed: A syndication feed or feed is a standard dataformat for grouping event summaries together. Typically, feeds are published at an advertised URLs on the web. The users subscribe to the feeds using a feed reader. The feed reader, checks regularly for an updated version of all the subscribed feeds and notifies the subscribers of the new event summaries as they are published.
• Channel: When there are several events being published, it is advisible to groups related events together. This lets subscribers to quickly get to the events they are most interested in. This grouping of events is called a Channel. Not all feed formats support publishing of multiple channels per feed. In the feed formats that support channels, one feed contains one or more channels and each channel contains the event summaries (as opposed to a feed directly containing all the even summaries).

Popular Feed Formats

The two popular formats for publishing syndication feeds are RSS and Atom. Both the formats are based on XML, a widely used and well-understood standard for communicating information between applications on the Internet. Adoption for both the feeds is already high. Most web logs emit notifications of a new posting in the form of a feed. Various big names like Firefox, Google and Bloglines offer software to read feeds once the user has configured the source URLs.

Atom File Format

An Atom Feed Document is a representation of an Atom feed, including metadata about the feed, and some or all of the entries associated with it. Its root is the atom:feed element.

Feeds are composed of a number of items, known as "entries", each with an extensible set of attached metadata. For example, each entry has a title. A feed maps to a Feed tag, event summary maps to an “entry” tag and atom does not have any channels.

RSS 2.0 File Format

An RSS document is a <rss> element, with a mandatory attribute called version, that specifies the version of RSS that the document conforms to. A feed maps to an “rss” tag, next level of the <rss> element is a single <channel> element (maps to the Channel), which contains information about the channel (metadata) and its contents. A channel may contain any number of <item>s.

An item may represent a "story" -- much like a story in a newspaper or magazine, basically the description is a synopsis of the story, and the link pointing to the full story. The “title” tag is the name of the channel. It's how people refer to your service. If you have an HTML website that contains the same information as your RSS file, the title of your channel should be the same as the title of your website. Each event summary maps to the “item” tag in an RSS document.

Discovering Feeds

As websites publish multiple feeds, or the same feed in various formats; it becomes increasingly difficult for programs/readers to discover all the feeds or the site summary. OPML or (Outline Processor Markup Language) is the solution. OPML is an XML format, where you advertise all the URLs of your sites’ feeds. OPML is used for many types of outline and list. As OPML is an open format, multiple vendors, service providers and websites use OPML to list RSS feeds. Most RSS readers/aggregators, such as IE7, Firefox and Google Reader support importing an OPML file. Following is a sample OPML that advertises two RSS feeds emitted by a fictional website called mashyourwebanddesk.com.

<?xml version="1.0" encoding="UTF-8"?>
<opml version="1.0">
   <head>
      <title></title>
      <ownerName></ownerName>
      <ownerEmail></ownerEmail>
   </head>
   <body>
	<outline description="Description of Feed1" htmlUrl="http:// mashyourwebanddesk.com/feeds/rss1" language="en" title="Title" type="rss" version="RSS1" xmlUrl=""/>
      <outline description="Description of Feed1" htmlUrl="http:// mashyourwebanddesk.com/feeds/rss1" language="en" title="Title" type="rss" version="RSS1" xmlUrl=""/>
</body>
</opml>

Dekoh Syndication API

Dekoh syndication API allows applications to leverage web syndication as a notification mechanism. Dekoh applications can use this API to publish, discover, subscribe and view syndication feeds.

Benefits of using Dekoh Syndication API

The main benefits of Dekoh Syndication API include:

No need to handle multiple feed formats

There are multiple feed formats (RSS 0.9, RSS 0.91, RSS 1.0, RSS 2.0, Atom 0.2, Atom 0.3) making it very cumbersome for application developers to support all the formats. Dekoh Syndication API abstracts a feed using a simple object model. Developers need to work with only this object model, letting the API takes care of publishing the object model to various feed formats and translating various feed formats to the object model.

Discovering all feeds at one go

All the applications deployed on Dekoh would like to publish their own feeds. The list of feeds published using the Syndication API is available in the OPML file format at the standard URL: http://<dekohUserID>.dekoh.net/dekohportal/feeds/opml (where <dekohUserID> is your Dekoh user ID). The Dekoh Portal application automatically subscribes users to their contacts’ feeds advertised in these OPML files. Hence, feeds published using the Dekoh Syndication API, show up in the contacts’ Dekoh Desktop homepage automatically.

Saving bandwidth through compression

Dekoh Syndication API supports gzip compression for feeds without requiring any additional code/configuration from the developer. Whenever Dekoh gets requests from RSS readers that support reading compressed content, it sends out the response in this compressed format. Similarly, whenever the syndication API is used to subscribe to feeds, it will attempt to download the feeds in compressed format. This saves bandwidth and improves response times.

Caching

Feed Readers poll frequently to check for new content. To do this, they can download the feed frequently and check if it has changed since the last time it was downloaded. This increases the bandwidth requirement heavily. Most feed readers, now support the standard HTTP “If-modified-since” header. When these readers download the feed for the first time; they cache the response and remember the “last-modified” time from the response header value. The next time this Feed Reader requests for the feed, it includes a request header “If-Modified-Since” with the “last-modified” time of the previous response.

If the feed file has changed since the “If-Modified-Since” time in this request header, the syndication API responds with the new Feed file, else it responds saying not modified and does not send a Feed file. For feeds subscribed using the syndication API, Dekoh aggregator using the same “If-modified-since” and “last-modified” mechanism to download feeds only when they have been modified, thus reducing bandwidth consumption.

Publishing your applications’ RSS with Dekoh Syndication API

EventSummaries can be grouped into channels and then into feeds or directly to create the feeds. Since the syndication API supports both the formats, we have chosen Channel as the default mechanism to group the summaries. When publishing the summaries to formats that do not support channels, the event summaries from all the channels get merged into a single feed.

Dekoh’s SyndicationService publishes the OPML advertising all the Channels registered with it. Each application might want to publish one or more channels. All these channels have to be registered with the SyndicationService for them to be listed/discovered. Application developers should register a ChannelProvider for each of the Channels they want to publish with the SyndicationService. ChannelProvider is the creator of Channels. A Channel simply translates the events from application domain to SyndEntries which encapsulate a event summary.

If you want to offer a feed of your content or application using Dekoh Syndication API, simply create applications that publish their activity summaries. Dekoh exposes its API to make it easy for applications to publish their activity summaries, and not worry about publishing in various formats such as RSS 1.0, RSS 2.0 Your application must implement a “Channel” for events that need to be published. The information enclosed between the <channel> tags is used to describe the feed itself. The code snippet below illustrates a typical Dekoh Channel description

public class CommentsChannel extends DekohChannel {
	List<SyndEntry> getSyndEntries() { //Makes use of rome API
		List<Comment> comments = getRecentComments();
		return createSyndEntry(comments);
	}
}

For each Channel that you have created, you must create a ChannelProvider. ChannelProviders are used to control how data is read from a given Channel.

public class CommentsChannelProvider extends DekohChannelProvider {
Channel getChannel(String publisher, String visitor, HttpServletRequest request) throws FeedException, FileNotFoundException, CouldnotGenerateFeedException {
	return commentsChannel;
}

long getLastModified(HttpServletRequest request) {
	return getMostRecentComment().getPostedAt().getTime();
}
}

You must register the ChannelProvider with Dekoh's SyndicationService Once the channels have been registered with the service; these channels can be discovered by OPML.

public class MyApplicationBootstraper implements ServletContextListener {	private final SyndicationService syndicationService = SyndicationService.getInstance();
public String getProviderName() {
	return “myApplicationChannel”;
}
public void contextInitialized(ServletContextEvent sce)
{
	CommentsChannelProvider provider = new  CommentsChannelProvider();
	syndicationService.registerChannelProvider(provider);
}
}

The OPML of all the feeds emitted by Dekoh for the publisher is available at http://localhost/dekohportal/feeds/opml

The feed for the above registered ChannelProvider in RSS 2.0 format will be published at http://localhost:1025/dekohportal/feeds/rss2?n=myApplicationChannel

Dekoh RSS Widget and Aggregation

Many popular feed readers (including Firefox's Live Bookmarks, & Google Reader) do not support reading authenticated feeds. All feeds published with Dekoh are served only after authentication. Using Dekoh platform API, Dekoh applications can choose to publish the creation of Share as an RSS feed and your contacts can subscribe to them. If your contacts have Dekoh installed on their machines, this RSS feed is automatically aggregated. Similarly, Dekoh applications can publish RSS feeds about contacts visiting the share or posting comments. These feeds are shown on the local user's Dekoh Desktop.

RSS for new Share created

Extend ShareChannel to provide implementations of that will pull from the application event stream a list of shares created recently for a Contact. An implementation of ShareChannelProvider should be registered with SyndicationService to publish this feed in various feed formats (RSS 1, RSS2).

RSS feed of Comments

Extend CommentChannel to provide implementations of that will pull most recent comments on shares created by the given local user. An implementation of CommentChannelProvider should be registered with SyndicationService to publish this feed in various feed formats (RSS 1, RSS2).

RSS feed of Contact visit

Not all your contacts that visit the shared content will post a comment. So knowing that someone has visited will be indicative of the interest the shared content drawing in a user's contact network. Dekoh applications can record the contact's visit to view the share, by calling dekoh.portal.Stats.recordBuddyVisit(HttpServletRequest) in the relavant JSP.

Application should extend BuddyVisitChannel to provide implementations of that will supply access link and preview image for the visited share to be displayed in the RSS.

An implementation of BuddyChannelProvider should be registered with SyndicationService to publish this feed in various feed formats (RSS 1, RSS2).

Subscribing to Feeds

Feeds emitted by Dekoh need an accessor to login via the basic authentication method. Many of the existing feed readers like FireFox's Live Bookmarks and Google Reader do not support reading feeds that require a basic authentication.

Dekoh Feed Aggregation Service

Dekoh Portal includes an RSS Feed Aggregator that you can configure to periodically poll and aggregate feeds from various sources which require authentication (Basic authentication only). Dekoh's RSS feed aggregator builds on the open source software "rome-fetcher".

To create an instace of Dekoh's RSS Aggregator:

• Create a SubscriptionList, choose the polling frequency and instanciate RSSFeedAggregator.

SubscriptionList subscriptionList = new InMemorySubscriptionList();
subscriptionList.add(new URI(“http://newsrss.bbc.co.uk/rss/newsonline_world_edition/front_page/rss.xml”));
subscriptionList.add(new URI(“http://feeds.feedburner.com/TechCrunch”)
RSSFeedAggregator feedAggregator = RSSFeedAggregatorFactory.getInstance().createDekohAggregator(subscriptionList);
feedAggregator.scheduleAggregator(5*60*1000);

• This will create an aggregator that will poll the list of URLs every five minutes. The aggregated information can be accessed with the URL:

String feedUrl = feedAggregator.getAggregatorFeedURL();

• Each Dekoh Desktop installation automatically creates an instance of this aggregator and polls for news from registered feeds on all your contacts' Dekoh installations and feeds sources that are emitted locally.

Viewing Feeds in Dekoh

The Dekoh Desktop main page includes a widget for viewing RSS feeds. This widget is configured to read RSS from the above mentioned urls. Dekoh also includes a Try icon that reads the same urls to display summaries of all new events as and when they occur.

Attachments