Mitigating Denial of Service Attacks with Web Resources

Another interesting security discussion from our on-going security push and the saga around finding a solution that's worth sharing (and hopefully reading)... this time it's about Web Resources which I had blogged about here.

As background for this, recall that the URL for a Web resource takes the form: WebResource.axd?a=MyControls&r=Bold.gif&t=632059604175183419

The interesting parameter is "t". This parameter is the timestamp of the assembly containing the resource. We add this parameter to the URL, so that rebuilding the assembly generates a new URL, causing the browser to ignore what it has cached, and fetch the new data. We aggressively cache for best performance by telling the browser to cache the data and never check for an updated version. Then we vary the output cache by the "t" parameter on the server.

Unfortunately however, this means that some malicious person out there could be simulating requests with random "t" values, and we'd just fill up the cache. Therein lay the denial of service (DOS) attack. Though these are somewhat hard to deal with, we didn't want to contribute to the problem.

So the first thing that we could do was validate that the incoming "t" value matched the timestamp of the assembly or returning an error status code otherwise. This takes care of the DOS attack, but introduces a new problem. The timestamp of the assembly on each machine on a Web farm isn't the same. So this simply doesn't work.

The next solution was to drop the "t" and put in assembly version. This actually got checked in... but it actually breaks the development scenario (oops!), where the assembly version isn't updated every build (nor should it be). In fact, assembly versions typically don't change even when a patch is put out. For example, the assembly version of System.Web doesn't change between QFEs. So this change obviously had to be backed out, but before making anymore changes, we had to ensure we had a solution that worked.

To lay down the requirements: We needed a "t"-like parameter so the browser's cache model would work. This also enabled the control development scenario really well, wherein changes were automatically picked up when the resource was modified. Pretty compelling and not something to give up on just yet. But we also didn't want to vary the cache per "t", since that led to the security hole.

A little discussion actually led us to realize a solution that was actually pretty easy, and was probably staring at us for the longest time. Just needed a few folks from the team to gather around and have a focused discussion on the solution. By default we were storing the cached data in disk-output cache, whose lifetime went beyond the lifetime of an app-domain. The "t" was allowing us to work against the disk-based output cache. What we simply needed to do was turn off disk-based output cache, and rely on in-memory cache alone (we still get the full benefits of the cache, including kernel-mode caching, so we aren't losing much). Now, whenever the bin directory was updated causing a restart or the application was explicitly restarted to pick up GAC changes, the in-memory cache would be cleaned up, and new content would be read from the assembly resource on-demand. To finish off the fix, we just needed to drop the version parameter, and bring back "t" with its prior semantics sans the vary-by behavior.

That takes care of denial of service attacks and retains performance. Just what we were looking for!

As a side note, watch out what you're caching in your own application as well, and make sure you're not susceptible to attacks that make you flood the cache with spurious entries.