Backbone Live Collections

A Backbone collection can be filled in several ways:

1. By individually fetching or creating models and then adding them to the collection. In this situation, the collection is not responsible for any interaction with a server.
2. By fetching a group of models using a single request made by the collection object itself. In this situation we define a url property on the collection and fetch it when we're ready for the data.

When your collection is actually responsible for communicating with an API endpoint to fetch its models, there are several types of data that you might be interested in fetching. For example:

1. You might have a finite set of data that you need to fetch once for your application.
2. You might have a feed of data that is constantly updating and thus your collection periodically check for updates.

Dealing with the first scenario is quite simple. It would require defining a Collection constructor that has a url property. Calling fetch on an instance of that collection would then fetch the data and that would be sufficient.

Dealing with the second scenario is somewhat more complex for several reasons:

1. The end point needs to be queried periodically rather than just once.
2. The data coming back may already be in your collection (because not enough new data was generated on the server and thus the subset returned is mostly identical.)
3. The existing collection may need to be ammended or added to.
4. Some UI component may want to update based on new data being retrieved, but only if there is new data.

How would one go about creating this system?

Let's create a system that watches a specific Twitter query and updates a list of tweets that match it.

You can search Twitter for anything using the following end point:

http://search.twitter.com/search.json?q=cats

The results of your query will look as follows:

First, let's write a Backbone Model & View that will represent a single tweet:

Now, let's create our actual collection and a quick view to render it

We can now instantiate a Tweets collection for a particular query like so:

You can see it in this fiddle: http://jsfiddle.net/iros/Pg2aU/18/

Making it live

While this works, note that if we want to treat this end point as a live feed, we need to call fetch repeatedly. Instead of the above method, we can chose to do this instead:

Note that we also removed the success callback. How would we actually paint new tweets then? Well, conveniently every time a model is added to a collection an “add” event is fired. This allows us to subscribe to that add event instead of rendering an entire collection of models. Let's rewrite our tweets collection view accordingly:

You can see it in this fiddle http://jsfiddle.net/iros/Pg2aU/19/

Don't forget that by adding new models to your collection, it will continue to grow in size. You may want to remove items as new ones are added, but that's outside the scope of this article.

Reusable StreamCollection

One quick way to use this "streaming" pattern is to extend the default Backbone.Collection to allow for streaming.

This allows us to start the stream by simply calling:

catTweets.stream({
  interval: 2000,
  add: true
});

You can see it in this fiddle http://jsfiddle.net/iros/VuJVa/9/

The Dreaded Duplicates

Now if you let the above fiddle run for a few cycles, you might have noticed there's quite a bit of tweet duplication. This is an unfortunate side-effect of the way adding models is implemented in backbone. You would think given the nature of relational databases that a “duplicate model” would be identified by using an id attribute (since most of the time, there's a uniqueness constraint on your data.) However, the way Backbone checks whether a model already exists in a collection is by using its cid as follows:

Given that your models come in from the server, there is no reason why they would have a duplicate cid (seeing as on creation of a new model a unique cid is assigned to it and it's never persisted unless you intentionally save it.)

There are two ways to work around this problem:

1. Overwrite the model's cid to match its id. This will ensure duplicates are detected. In our example this would look as follows: While this solution works, it will still result in an error being thrown every time a duplicate model is encountered. You can always surrounded in a try/catch block where appropriate in your application. You can see that in the following fiddle if you open your console. You can see it in this fiddle: http://jsfiddle.net/iros/4pLUR/6/

2. Alternatively, you could overwrite your collection's add method to check for a duplicate before actually adding the model in question and only adding it if it doesn't already exist. In our example that would look like so: Again, not the most elegant solution but at least we got rid of the thrown errors. You can see it in this fiddle: http://jsfiddle.net/iros/GVaWe/6/

Ideally, this wouldn't be an issue and duplicates would be detected by id as well as the cid. This was in fact once fixed in the Backbone source but then subsequently reverted in this commit.

How do we solve this then??

What we're proposing is adding a new options property to the Collection.prototype.add method:

{ unique : true }

This will guarantee that only unique models (by cid and id as a fallback) will be added to the collection if you so desire. There's a pull request awaiting feedback and your love, if you care to dispense it: https://github.com/documentcloud/backbone/pull/808

What do you think?

Much thanks to Tim Branyen and Adam Sontag for their feedback.

This entry was posted by Irene Ros (@ireneros) on December 22, 2011 in Backbone, JavaScript and Open Web Apps.

Tweet