The Internet no longer a wild west

From a Wired article:

"There was a brief moment in the tech market from 1995 to now where anyone could simply attach a server to the internet and be in business," Wilson writes in response to a commenter. "That moment is coming to an end."

I'm glad I grew up in a time when the Internet was kind of a "wild west"; it didn't feel ruled by corporations or governments. Sad to see it losing some of that.

A deep dive into Express's static middleware

This guide assumes you know how to use Express and have used its static middleware. No need to have done anything complex with it, though! If you need help, you can check out my intro to Express or my book on the topic. This guide was last updated for Express 4.6.1.

This guide is a little bit outdated; check out the new docs.

If you're like me, you like Express. And if you're like me, you've used its static middleware, express.static. And if you're like me, you thought it was that simple. But this feature is only mentioned offhandedly in the documentation and it has a ton of not-well-documented features.

Get ready, maggots. We're going to go nerd spelunking.

Explaining the stack

If you go looking for the word "static" in the Express source, you basically find it in one place: aliasing express.static to a module called serve-static. If you go spelunking in serve-static, you'll find that it depends on another module, called send.

At a high level, here's how the three modules are put together:

  1. send is at the lowest level. It's basically a function that sends a file over HTTP. To get a bit more technical, send takes an HTTP request and a path, and it returns a stream that you pipe to your HTTP response. For a sense of scope, it's about 600 lines of code.
  2. serve-static wraps send up into generic middleware and adds a couple of options. It's what you think of as express.static. It's smaller, at about 150 lines of code, but it still does a fair bit.
  3. express.static is just an alias for serve-static; there's just one line of code here.

Worth noting that Express's res.sendFile also uses send (and never touches serve-static, because that's middleware!).

With these three parts, you can customize the hell out of your static middleware. Some of the options are dealt with in serve-static while others get passed down into send. In any case, there are way more options than I expected.

Caching options

The static middleware does no server-side caching (I thought that it did!), but it does let you do two methods of client-side caching: ETag and Max-Age. If you don't know what those are, get ready to learn.

ETags

ETag is a horrible name and is short for "entity tag", a name that is even worse. It's one way to do caching, and here's how it works:

Let's say I'm a web browser and I'm loading jokes.edu/offensive.html for the first time. When I get the response back, I display it to the user. In addition to the content of the page, the server might also send an HTTP header that looks like this:

ETag: 1234567

If the browser sees the ETag, it will cache offensive.html and say that its corresponding ETag is "1234567".

The next time the browser loads jokes.edu/offensive.html, the browser asks, "is the ETag still '1234567'?" It does this by setting the following header in the request:

If-None-Match: 1234567

If nobody's edited offensive.html, then the file is exactly the same, and so is its ETag. Instead of sending all the bytes again, the server responds with an HTTP 304 status code (which means "not modified") and saves a bunch of bandwidth.

But if someone has offensive.html, then the file will have a different ETag, and so everything will be sent over the wire.

Servers can recalculate ETags however they please, often by using a checksum or hash function or whatever you want to call it. For your reference, Express (send, really) uses the MD5 hash function, because those rarely have collisions and are fast to calculate.

By default, the static middleware has ETags enabled. It'll set the ETag header (unless you set them sometime beforehand, which I wouldn't recommend). To disable it, you can do something like this:

app.use(express.static(myStaticPath, {
  etag: false
}))

You might want to disable ETags for a few reasons:

  1. You don't want any kind of caching, even of static files. This caching is pretty reliable, though, so that shouldn't really be a concern.
  2. You want strong ETags; send only supports weak ones. If you don't know what these are, you probably don't need strong ETags.
  3. You don't trust Express's implementation for some weird reason. Maybe you're worried about rare MD5 hash collisions?

I'd recommend that you leave this alone, because the above reasons aren't compelling (at least to me), but it's your call.

Max-Age

Max-Age is another fun caching mechanism that Express supports, and it's a little different from ETags.

With ETags, we can reduce the amount of bytes sent over the wire, but clients still have to make an HTTP request every time, just to make sure their cache is still valid. With Max-Age, the server basically says, "Here's a resource, which you can cache for a week" (or however long you'd like). On one hand, it saves bandwidth, but on the other hand, the people making the server had better be pretty sure that the content will be good for a certain amount of time!

Unlike ETags, Max-Age isn't itself an HTTP header. It tags along with a header called Cache-Control. If a server wanted to tell the client to cache something for one day (86,400 seconds), it'd send a header like this:

Cache-Control: public, max-age=86400

Cache-Control turns out to be a pretty complicated HTTP header; it's got a long spec. But Express's static middleware only deals with a small subset of it, like so:

var oneDay = 86400000; // in milliseconds
app.use(express.static(myStaticPath, {
  maxage: oneDay
}))

This code example will set the max-age to just one day, just like the header above. Now, a browser will only request that resource after one day has passed.

Notice that the time is in specified milliseconds, not seconds like the header above. This is because almost everything in JavaScript is millisecond-based, not second-based; send will do the conversion for you.

You can also pass times as strings (which internally uses the ms module):

app.use(express.static(myStaticPath, {
  maxage: '2h'
}))

This sets the max-age to two hours, as you might expect.

A couple of notes about this option:

  • The maximum max-age that Express allows is 365 days.
  • Passing a negative number will just set the max-age to 0.
  • As you can see, you're passing a property called maxage. The static middleware also supports maxAge (note the capitalization difference). send, however, doesn't support capital-A maxAge. To be safe, I'd stick with the all-lowercase maxage for consistency.

If you're pretty sure resources won't be updated for an amount of time, I'd recommend adding a max-age to your files. There are asset helpers that modify the filenames so that browsers don't cache old assets; I won't cover those here, but things like connect-assets can help with this.

Personally, I usually omit max-age. Leaving it out is slightly less performant but much less developer headache. Once again: your call!

The index

Ugh, caching is hard. Let's do something easy: serving the index.

You've undoubtedly encountered the wonderful world of index.html sometime in your life; when you visit a directory, it's often the case that index.html is served to you. But did you know that the static middleware can change all that?

By default, the static middleware (via send) serves up a file called index.html when you visit the folder's root. It's as if you did this:

app.use(express.static(myStaticPath, {
  index: 'index.html'
}))

As you might imagine, you can change it. Let's say you want the filename to be different:

app.use(express.static(myStaticPath, {
  index: 'jokes.txt'
}))

Now we'll load jokes.txt when we visit the root, instead of index.html.

You can also pass an array. If it finds the first file, it'll send that as the index. If not, it'll send the second file, and the third, and so on. If it never finds it, it'll continue to the next middleware.

app.use(express.static(myStaticPath, {
  index: ['jokes.txt', 'index.html']
})

You can also ignore the index completely. In that case, the only way to see a file called index.html is to visit index.html directly.

app.use(express.static(myStaticPath, {
  index: false
}))

In this case, trying to visit the root will give a 404 error.

Not too crazy, and pretty useful!

Setting custom headers

The static middleware also supports a setHeaders property, which is a function that's called right before HTTP response headers are set. Let's quickly look at a couple of examples of its usage to see how it's used and why we'd want to use it.

If your browser sees Content-Disposition: attachment in the HTTP response headers, it'll open a download dialog rather than trying to display the response in the browser. If you've ever clicked "download this file", seen a download dialog, and wondered why your browser doesn't just try to render the file, it's because of Content-Disposition.

If you want to serve all static files as attachments, you can combine that with Express's res.attachment, like so:

app.use(express.static(myAttachmentsPath, {
  setHeaders: function(res, path) {
    res.attachment(path)
  }
}))

This basically sets Content-Disposition: attachment for your files. This is perhaps the most common case.

You might also want to do this selectively. Let's say we want to send the file as an attachment if the word "download" is in the path. Here's how we might do that:

app.use(express.static(myAttachmentsPath, {
  setHeaders: function(res, path) {
    if (path.indexOf("download") !== -1) {
      res.attachment(path)
    }
  }
}))

You might also use this method to log things or set special debug headers, though I think the attachment recipe is the most common.

Trailing slashes on directories

Imagine a world where you have your static files in a directory called static, and within that directory is another folder called comedy_pix. If I visited /comedy_pix/, I'm obviously visiting the directory. But what if I'm visiting /comedy_pix without the trailing slash?

By default, the static middleware (not send any more) will redirect you with a nice 303 "see other" HTTP request. If you want to disable that behavior, you can!

app.use(express.static(myStaticPath, {
  redirect: false
}))

Now, if you visit /comedy_pix without the trailing slash, the middleware will never happen.

I think this is a pretty obscure feature. You might want to do this if you have a folder called comedy_pix and a separate route that maps to comedy_pix, for example. It's there in case you need it, but you likely don't.

It's probably not important, but this only works if you pass in false as false; you can't pass 0 or null or new Boolean(false) or other falsy values. There's no good reason that you should be doing this anyway!

Exposing hidden dot files

You probably know about hidden files: files considered "hidden" don't show up in most listings by default. On OS X and Linux, a file starting with a period is considered hidden, and is sometimes called a "dotfile" because it starts with a dot. On Windows, it's a little different, but the middleware doesn't support this.

The middleware supports sending these dotfiles. By default, though, they're ignored. It's as if you specified this option:

app.use(express.static(myStaticPath, {
  dotfiles: 'ignore'
}))

If you want to serve hidden files for some reason (which can put you in the Danger Zone, because these files are usually hidden for a reason):

app.use(express.static(myStaticPath, {
  dotfiles: 'allow'
}))

You can also choose to send a 403 Forbidden error when trying to access a dotfile. Clients will know that there's a dotfile there, but they won't be able to get inside:

app.use(express.static(myStaticPath, {
  dotfiles: 'deny'
}))

This is sensibly disabled by default, and I can't think of a great reason to change it.

Some other Fun Facts™

  • send (and therefore everything else) will set a bunch of headers if they're not already specified: Accept-Ranges, Date, Cache-Control (for max-age stuff), Last-Modified, and ETag (unless you disable it). If you want to remove those headers, check out this simple example which uses the on-headers module.

  • send supports a root property, but I wouldn't set it unless you're using send directly. A root property in the options will override the previously-specified root. You can do horrible things this way:

    // don't do this!!
    app.use('/path-foo', express.static(myStaticPath, {
      root: '/path-bar'
    }))
    
  • All three parts of this stack expose the mime module as an alias, so express.static.mime will work.

All done!

I don't know about you, but I didn't expect Express's static middleware to be so complicated! Luckily, I think they've done a good job choosing sensible defaults so that you don't have to worry about this stuff 99% of the time.

Hopefully you've enjoyed this little dive into the wonders of serving static files with Express!

Mostly on screens, at scales unimaginable

From "The Secret of Minecraft":

We're in a new century now, and its hallmark is humans doing things together, mostly on screens, at scales unimaginable in earlier times.

A quick romp through default values in CoffeeScript

This was last updated for CoffeeScript 1.7.1.

Imagine that this sentence is a beautifully-crafted, flowing intro paragraph. "Wow, what a great introduction to this CoffeeScript guide," you whisper. Now, let's quickly explore all of CoffeeScript's ways to set default values.

Before you start, make sure you know about undefined, null, and falsy values in JavaScript. Check this out if you need help.

As a function parameter

To quote the CoffeeScript documentation, functions can have "default values for arguments, which will be used if the incoming argument is missing (null or undefined)." Here's how you might use it:

shootLaserBeam = (color = "red") ->
  # ...

Now, if the first parameter is undefined or null, it'll be defined as the string "red". If it's anything else (even falsy values like 0 and the empty string), it won't be set to the default value.

Using the existential operator

CoffeeScript's existential operator can be used to set default values.

sendThreateningMessageFromUFO = (options) ->
  options.message ?= "I will hold your pathetic planet hostage"
  # ...

You can also set another variable using a sexier version of a ternary operator.

# this...
message = evilMessage ? "Exterminate!"

# ...is equivalent to this:
if evilMessage?
   message = evilMessage
else
   message = "Exterminate!"

# ...or this:
message = if evilMessage? then evilMessage else "Exterminate!"

The ternary operator makes sure that the variable isn't undefined and isn't null.

You'll get a compile error for if you try to define not-yet-defined variables. For example, this gives a compiler error:

someUndefinedVariable ?= "I will destroy your worlds"

This makes sense, right? Why set a default value for something that's totally undefined? If you, for some reason, need to do this, you can use the existential operator on the variable:

someUndefinedVariable = "Flee, puny humans" unless someUndefinedVariable?

Discarding falsy values, too

If you want to set defaults for any falsy value (not just undefined and null), you can use or=.

angryAliens = false
angryAliens or= true
# angryAliens is now true

# ||= is totally equivalent to or=
ufoCount = 0
ufoCount ||= 50
# ufoCount is now 50

# compare it to the existential operator:
numberOfAliens = 0
numberOfAliens ?= 1000
# numberOfAliens is still zero

The important takeaway here: ?= only discards null and undefined, where or= filters any falsy value.

For undefined only

Most of CoffeeScript either tests against falsy values or "is it null or undefined". You might want to set things only if it's undefined and nothing else, not even null. Here's how you might do that:

cropCircleCount = 69 if cropCircleCount is undefined

Now, if cropCircleCount is null or anything falsy other than undefined, it'll stay that way.

While this isn't as built-into-the-language as some of the features above, the fact that it's on one line is a little nicer than regular JavaScript. undefined is an undefined keyword in ECMAScript 3 and below. This means that this JavaScript fails on older browsers:

var isUndefined = window.myFunVariable === undefined;
// This fails on older browsers because "undefined" is, well, undefined.

CoffeeScript makes sure that this works in older browsers (by compiling undefined to void 0 to reliably produce an undefined value).

A summary

And now, a summary.

  • Use default parameters if it's a function parameter. Only sets to default if the value is null or undefined.
  • Use ?= to set a value to a default if the value is null or undefined.
  • Use or= (or ||=) to reject other falsy values like 0 and the empty string.

Go forth and set your CoffeeScript defaults, friend.

Wearable Tech or Technical Wear?

From "Wearable Tech or Technical Wear":

I'm excited for the Glass-Luxottica joint task force. It's a step away from futuristic looking wearable tech and a step towards technological wearbles for which I would dish out that dolla. If wearables are designed to be seamlessly and beautifully integrated into everyday life with all of the consumer base in mind—yes, even the omniscient white male—then I believe Wired's prediction that wearable tech being as big as the smartphone could come true.

I agree; wearables are currently being made for nerds, not for fashionable people.