For the past year I’ve been using the Wordnik API in my projects to generate random words. I’ve made extensive use of their
randomWords feature in projects like All the Things, Metaphor-a-Minute!, and even Amazon Random Shopper but for every project I’d copy/paste code for interfacing with Wordnik. I’d been meaning to write my own interface to the API, but I’d been putting it off, because, well, I’m lazy. But a couple of weeks ago I took our Building Web Applications with Backbone class, and I was finally inspired to build a promises-based interface to Wordnik using Backbone, which I called wordnik-bb.
It was while building my library that I noticed that Wordnik had recently added rhyming to their API! That is, you can send an English word to their relatedWords API call and it’ll send you back an array of words that rhyme with it. I had never really played with the power of their powerful semantic “word graph” before and now I finally had an excuse to do so.
Whenever I start a creative coding exercise, I do some simple prototyping to see if the core idea I have posesses any potential to be interesting. Initially I just brute forced a few API calls using the excellent interactive tools over at the Wordnik API documentation. I wrote the first thing that came to mind:
I was pretty happy with the output of this. For example:
I once knew someone who was Iraqi Their manners were very wacky
My rule of thumb for creative coding is: if it makes me laugh, it’s probably worth pursuing.
At any rate, I showed this to my friend Cameron Kunzelman and he immediately responded, “So basically you’re telling me this rhyming machine is going to be able to make late 1980s rap pretty consistently.” This reminded me that I’d wanted to make a “bad freestyle rapper” bot for ages, and now I had the means to do so.
One equation for making a cool textual generator is: randomness + formulaic writing = hilarity. So I pretty quickly realized that having my program generate a battle rap was the way to go. A good resource for this is Adam Bradley’s Book of Rhymes: The Poetics of Hip Hop, which lays out some of the formula behind battle rhymes.
The basic algorithm for RapBot has remained the same from the start:
This algorithm would have been a lot harder to write if I weren’t using a promises implementation. If you’re not familiar with promises, here’s an introduction to jQuery’s implementation of promises. The short version is: if you’re making asynchronous calls to a service (like I’m doing with Wordnik), you can’t be guaranteed when you’ll get the data back. It could be in 200ms, or 2 seconds, or 20 seconds. So instead of writing a function that returns a value like the part of speech for a word, you write a function that returns a promise for the part of speech of a word. Then you can define callbacks on the promise telling it what to do if it’s resolved as a success, rejected as a failure, etc.
So now that you’re an expert on promises (cough), you can now see that an algorithm where it says “make this async call, then when that’s done make that async call” and chains it a bunch of times is a great place for us to use promises.
Here’s an example of promises in RapBot, when we ask the Wordnik API for a part of speech of a word:
Compare this to the structure of the
getLine, which is a good ol’ synchronous function:
I did a “soft launch” of RapBot on a Saturday night to see how it would perform. The answer: not terribly well. Wordnik’s API caps the number of calls per hour at 5000 calls. That might seem like a lot, but at the time of my soft launch, every page load made somewhere close to 50 API calls! That’s because each couplet would:
So each couplet is 4 API calls, and for 12 couplets that could be as many as 48 API calls.
Fortunately, I was able to get the number of API calls down to about 29 from 48. In addition to returning individual random words, Wordnik can also return an array of up to 1000 random words in a single API call. You can also ask for an array of random words of a particular part of speech. Looking at the first two steps of creating a couplet, you can see that I’m getting a word and getting its part of speech. The better approach would be: on page load, grab an array of random words for each part of speech that I need. Since I support 5 parts of speech, that’s 5 API calls. Then in each couplet I say: pick one of our parts of speech we support, then access the array of words for it. This means that the couplet algorithm now looks like this:
So now each couplet is only 2 API calls, which means that the 12 couplets make 24 API calls total. Add in the 5 API calls on page load and we’re now down to 29 calls from 48. (As often happens with refactoring for performance, my code became more modular and suddenly I was able to tweak the frequency of appearance of each part of speech. So now I can say: give me a noun 40% of the time, an adverb 10% of the time, etc.)
There’s further work that could be done. Instead of making the 5 API calls on page load, the application could make the 5 API calls on startup and store it globally on the server. But that would reduce the variety of results on each page load. I could increase the randomness by updating those random word arrays silently in the background, making the API calls once every half hour, and appending the results to the existing arrays.
Beyond that I could cache a whole bunch of couplets and just refer to the cached results on every page load, but again that would further reduce the randomness of the results and make the whole exercise less fun.
I do have to give a shout-out to the team at Wordnik for increasing my API call limit, too!
With any creative coding project there will always be a long list of ways to improve. Potential improvements include:
All of these things will result in more API calls, so before I do anything else, the next step would involve caching results to reduce API calls by an order of magnitude to something like 2 or 3 per page.
In closing, I leave you with some cryptic words of wisdom about the Open Source ecosystem, courtesy of RapBot:
If you're gonna battle me, then you gotta be noncommercial When I rock a mic you know I rock it real uncontroversial