Making a RapBot with JavaScript

This post talks about the development of RapBot, my freestyle 80s battle rap generator. You might want to see it in action before reading on, and you can check out the source code here.

For the past year I’ve been using the Wordnik API in my projects to generate random words. I’ve made extensive use of their randomWords feature in projects like All the Things, Metaphor-a-Minute!, and even Amazon Random Shopper but for every project I’d copy/paste code for interfacing with Wordnik. I’d been meaning to write my own interface to the API, but I’d been putting it off, because, well, I’m lazy. But a couple of weeks ago I took our Building Web Applications with Backbone class, and I was finally inspired to build a promises-based interface to Wordnik using Backbone, which I called wordnik-bb.

It was while building my library that I noticed that Wordnik had recently added rhyming to their API! That is, you can send an English word to their relatedWords API call and it’ll send you back an array of words that rhyme with it. I had never really played with the power of their powerful semantic “word graph” before and now I finally had an excuse to do so.

Experimenting with creative code

Whenever I start a creative coding exercise, I do some simple prototyping to see if the core idea I have posesses any potential to be interesting. Initially I just brute forced a few API calls using the excellent interactive tools over at the Wordnik API documentation. I wrote the first thing that came to mind:

I was pretty happy with the output of this. For example:

I once knew someone who was Iraqi
Their manners were very wacky

My rule of thumb for creative coding is: if it makes me laugh, it’s probably worth pursuing.

At any rate, I showed this to my friend Cameron Kunzelman and he immediately responded, “So basically you’re telling me this rhyming machine is going to be able to make late 1980s rap pretty consistently.” This reminded me that I’d wanted to make a “bad freestyle rapper” bot for ages, and now I had the means to do so.

One equation for making a cool textual generator is: randomness + formulaic writing = hilarity. So I pretty quickly realized that having my program generate a battle rap was the way to go. A good resource for this is Adam Bradley’s Book of Rhymes: The Poetics of Hip Hop, which lays out some of the formula behind battle rhymes.

Battles are better with promises

The basic algorithm for RapBot has remained the same from the start:

• Get a random word and its part of speech. Get a ‘line’ for that word based on the part of speech. For example if it’s a noun, it’ll pick from a bunch of noun lines, like “I’m the illest MC to ever rock the [noun]”
• Once I have that first word, get a list of words that rhyme with it. (Wordnik is sort of inconsistent. Some words just lack any rhyme data.)
• If I get back any rhyming words, then I pick one at random and also get its part of speech.
• Much like I did with the first word, I get a line to match its part of speech.
• Now I have a couplet! I repeat this 12 times, which will get us anywhere from 0 (if it couldn’t find rhymes for any words at all, a rare case) to 12 couplets.

This algorithm would have been a lot harder to write if I weren’t using a promises implementation. If you’re not familiar with promises, here’s an introduction to jQuery’s implementation of promises. The short version is: if you’re making asynchronous calls to a service (like I’m doing with Wordnik), you can’t be guaranteed when you’ll get the data back. It could be in 200ms, or 2 seconds, or 20 seconds. So instead of writing a function that returns a value like the part of speech for a word, you write a function that returns a promise for the part of speech of a word. Then you can define callbacks on the promise telling it what to do if it’s resolved as a success, rejected as a failure, etc.

So now that you’re an expert on promises (cough), you can now see that an algorithm where it says “make this async call, then when that’s done make that async call” and chains it a bunch of times is a great place for us to use promises.

Here’s an example of promises in RapBot, when we ask the Wordnik API for a part of speech of a word:

Compare this to the structure of the getLine, which is a good ol’ synchronous function:

Reducing API calls

I did a “soft launch” of RapBot on a Saturday night to see how it would perform. The answer: not terribly well. Wordnik’s API caps the number of calls per hour at 5000 calls. That might seem like a lot, but at the time of my soft launch, every page load made somewhere close to 50 API calls! That’s because each couplet would:

• Get a random word (1 API call)
• Get the part of speech for it (1 API call)
• Get a list of words that rhyme with it (1 API call)
• Pick a word from that list (not an API call)
• Get the part of speech for it (1 API call)

So each couplet is 4 API calls, and for 12 couplets that could be as many as 48 API calls.

Fortunately, I was able to get the number of API calls down to about 29 from 48. In addition to returning individual random words, Wordnik can also return an array of up to 1000 random words in a single API call. You can also ask for an array of random words of a particular part of speech. Looking at the first two steps of creating a couplet, you can see that I’m getting a word and getting its part of speech. The better approach would be: on page load, grab an array of random words for each part of speech that I need. Since I support 5 parts of speech, that’s 5 API calls. Then in each couplet I say: pick one of our parts of speech we support, then access the array of words for it. This means that the couplet algorithm now looks like this:

• Pick one of our five parts of seech (not an API call)
• Get a word from the array of random words from that part of speech (not an API call)
• Get a list of words that rhyme with it (1 API call)
• Pick a word from that list (not an API call)
• Get the part of speech for it (1 API call)

So now each couplet is only 2 API calls, which means that the 12 couplets make 24 API calls total. Add in the 5 API calls on page load and we’re now down to 29 calls from 48. (As often happens with refactoring for performance, my code became more modular and suddenly I was able to tweak the frequency of appearance of each part of speech. So now I can say: give me a noun 40% of the time, an adverb 10% of the time, etc.)

There’s further work that could be done. Instead of making the 5 API calls on page load, the application could make the 5 API calls on startup and store it globally on the server. But that would reduce the variety of results on each page load. I could increase the randomness by updating those random word arrays silently in the background, making the API calls once every half hour, and appending the results to the existing arrays.

Beyond that I could cache a whole bunch of couplets and just refer to the cached results on every page load, but again that would further reduce the randomness of the results and make the whole exercise less fun.

I do have to give a shout-out to the team at Wordnik for increasing my API call limit, too!

Looking ahead

With any creative coding project there will always be a long list of ways to improve. Potential improvements include:

• More complicated rhyme schemes
• Bots that respond to each other (1: “You’re nothing but a [noun1]!” 2: “You call me a [noun1]? You’re just a [noun2]!”)
• Take meter into account, so the rhymes scan right

All of these things will result in more API calls, so before I do anything else, the next step would involve caching results to reduce API calls by an order of magnitude to something like 2 or 3 per page.

In closing, I leave you with some cryptic words of wisdom about the Open Source ecosystem, courtesy of RapBot:

If you're gonna battle me, then you gotta be noncommercial 
When I rock a mic you know I rock it real uncontroversial

This entry was posted by Darius Kazemi (@tinysubversions) on February 20, 2013 in JavaScript and Node.js.

Tweet