Rake Part 3: Rules

If you're a Ruby programmer, you've almost certainly used Rake, the build utility created by the late Jim Weirich. But you might not realize just how powerful and flexible a tool it can be. I certainly didn't, until I decided to use it as the basis for Quarto, my e-book production toolchain.

This post is part of a series on Rake, starting with the basics and then moving on to advanced usage. It originated as a series of RubyTapas videos published to subscribers in August-September 2013. Each post begins with a video, followed by the script for those who prefer reading to viewing.

My hope in publishing these episodes for free is that more people will come to know and love the full power of this ubiquitous but under-appreciated tool. If you are grateful for Rake, please consider donating to the Weirich Fund in Jim's memory.

In the last two episodes we constructed a Rakefile for building Markdown files into HTML. The file is working as it stands now, but it contains some duplication. There are two nearly identical rules for building HTML files, one that looks for source files with a .md extension, and one that looks for source files with a .markdown extension. It would be nice if we could combing them into a single, more generic rule.

We’ll start by removing the second rule. If we run rake now, it fails:

Before we go any further, let’s talk about this error message a little bit. It says: “Don’t know how to build task ‘ch4.html'”. This doesn’t tell us a lot. It’s also a little bit confusing, because it’s talking about a task called ch4.html. But ch4.html is a file we want to build, not a task, right?

As it turns out Rake thinks about all of the things it is asked to build as tasks. The only difference between plain tasks and file tasks is that with a file task, Rake knows that if there is a file matching the name of the task, and that file is newer than any of its prerequisites, it needn’t bother running the task at all.

In this case we know why Rake can’t build this file, because we just removed the rule telling it how. But what if we didn’t know? This message doesn’t give us a whole lot to work with.

To get a better understanding of what Rake is up to, we can pass the trace flag to it. This time, Rake leaves a breadcrumb trail to follow, telling us what it tried to do. First, it invoked the default task. We’ve made the default task dependent on the “html” task, so it invoked that one next.

The next step after that is an abrupt notification that rake has aborted because it didn’t know how to build ch4.html, followed by a Ruby stack trace.

Let’s ask Rake why it was trying to build =ch4.html= in the first
place. We can do this by running Rake with the =-P= flag, which tells
it to dump a list of prerequisites.

This output makes it clear that the html task depends on a list of files, including ch4.html.

Remember, we’re still pretending we don’t know what the problem is with this Rakefile. We’ve gathered a lot of insight into Rake’s thinking, but so far we’re none the wiser about the connection between HTML files and Markdown files, something we would need to understand in order to fix this problem.

In order to peer even deeper into Rake’s thought process, we next set an option called Rake.application.options.trace_rules to true in our Rakefile. Enabling this option does exactly what its name suggests: it tells Rake to give us tracing information about rules defined in the Rakefile.

NOTE: After I made this video, Jim Weirich pointed out that this option is also available as the command-line option –rules.

We run rake -trace again, and this time in addition to the breadcrumb trail of task invocation, we see some new information. For each file build, Rake tells us that it is attempting to use a rule in which a .html depends on a corresponding .md file. When it gets to ch4.html, it fails. It doesn’t explicitly say that the prerequisite file ch4.md couldn’t be found. But with the information in front of us now, we can reasonably deduce what the problem is.

Now it’s time to make the rule work again. We start by defining a method, source_for_html. It will take the name of an HTML file, and return the name of the corresponding Markdown file. In order to do so, it needs access to the source files list. Right now the list is a local variable, which won’t be accessible inside this method. We change it to a constant.

We then search the source files list for the first source file whose base name matches the base name of the given HTML file name. In order to compare just the base names, we use the #ext method again. You might remember that we used this method on the source file list in order to derive a list of HTML output file names. This time we pass an empty string to #ext in order to remove the file extension entirely.

“Wait just a darn minute!” I can hear you saying. “We sent the #ext message to a FileList before. But here we’re sending it to individual file name strings! How does that work?”

As it turns out, Rake modifies Ruby’s String class to support some of the same methods that FileList supports, so that we can do the same operations on FileLists and individual file names interchangeably.

Now we have a method which, given an HTML file name, can search back to find the source Markdown file needed to generate it. Now we need to make the .html rule use this method.

We do this by replacing the .md dependency in the rule with a lambda. Inside the lambda, we take the sole argument and pass it into our #source_for_html method. When Rake tries to build a .html file, it will now pass the name of the target file to the lambda we provided as a prerequisite. It will take the return value of this lambda and see if it matches an existing file. If so, it considers the rule a match and proceeds to execute the associated code.

We still have rule tracing enabled, so we get a window into how Rake reasons using our updated rule. When it comes to the ch4.html target, it correctly determines that the prerequisite is ch4.markdown, not ch4.md . Finding that file, it goes ahead and builds the ch4.html file.

We now have a generic rule for building HTML files from Markdown files with either long or short extensions. But more importantly, we have a lot more insight into how Rake works with rules to determine what to build, and how. Happy hacking!

BTW, here’s the final Rakefile:

(This post is also available in Japanese at Nilquebe Blog.)

I hope you've enjoyed this episode/article on Rake. If you've learned something today, please consider "paying it forward" by donating to the Weirich Fund, to help carry on Jim's legacy of educating programmers. If you want to see more videos like this one, check out RubyTapas. If you want to learn more about Rake, check out my book-in-progress The Rake Field Manual.

P.S. Have you used Rake in a particularly interesting way? I want to hear from you.