Making Fast and Lightweight Webpages (When Every Byte Counts)

Note: this was first published over on Intranel's blog

As computers and internet connections have improved for the top tier of users, it has become common to see a webpage download several megabytes of HTML, CSS, JavaScript, images, and other assets. This can quickly turn into a poor and expensive user experience, yet it’s all but free for the developer. Storage and server bandwidth are incredibly cheap, only becoming expensive when dealing with gigabytes of data delivered at vast scale. But what happens when you’re suddenly limited by your storage? And not to hundreds of megabytes, but only a few. One of my recent projects was to develop a fast wifi set-up page for a device built with an embedded system housing a total of 4MB of storage space.

Fun fact: a Google search for “large webpages” loads 1.34mb of data uncompressed (the Google results page itself). We’ll come back to that number momentarily.

The boot loader and system libraries are given 1MB of space, leaving just 3MB. However, that memory is then divided into thirds:

  1. One for the currently running firmware
  2. One for the incoming firmware update, and
  3. One for the factory default firmware.

That’s just 1MB for the entire system, including the webpage. That page of search results from Google? It wouldn’t even fit by itself.

The purpose of the webpage is to assist the owner in finding nearby wifi connections and connect the device to them. Though the device is internet-connected, it is by no means a “high-tech” type of device: the consumers are not necessarily going to be first-adopter technophiles. The webpage needs to work on as many browsers as possible and stay on-brand to the product to assure people they are in the right place.

Let me walk you through some of the methods I used to make this page as lightweight as possible with good user experience, while still easy to work on as a developer.

JavaScript

The first easy win was to not use any 3rd-party libraries. Libraries and frameworks are useful, but always contain code that isn’t used for a particular project. With our small amount of storage space, this project couldn’t afford any unused bytes.

Loading libraries from content delivery networks (CDNs) wasn’t possible since the customer is connected locally to the on-board WiFi at this point (with the goal of connecting it to the wider internet). Any library would need to live on the system. With libraries not being a viable option, everything was custom written in regular, vanilla Javascript.

To make my life as a developer easier (depending on your point of view), I could have used Typescript and set a build target for high compatibility. However, there wasn’t enough scripting to get a major benefit from that. Also, at the time of the work, my knowledge of Typescript wasn’t very good and I preferred the direct control over the Javascript. I’d reconsider this approach if this project came up today now that I have Typescript experience.

CSS

To help with CSS browser compatibility, I used SCSS and Autoprefixer (while I didn’t have a good knowledge at the time of Typescript, I did of SCSS and was confident I wouldn’t be adding unnecessary bloat). Where it didn’t matter, I lived with design inconsistencies.

As wiser people before me have said, a webpage does not need to look the same in every browser. This meant not bringing in Normalize.css or a similar library to help smooth the differences between browsers.

As an example of minor inconsistencies, I used CSS Grids to layout one of the components. The modern syntax isn’t supported in Internet Explorer but could have been made to work with Autoprefixer. Doing so turned out to generate so much extra code that I decided a slight difference in layout was acceptable and opted to use floats instead.

To keep both markup and CSS sleek, the majority of the selectors were written using the semantic markup of the HTML rather than sprinkling extra classes around.

Images

The only image on the site is the logo. To begin, I worked with the designer to convert it to SVG format. SVGs have a lot of optimisation opportunities. I ran it through SVGOMG, selecting the lowest precision while still maintaining the shapes. From there, the file was further optimised by moving the styles in-line rather than using classes. This may seem counter-intuitive as it creates more code, but it allows for improved compression, which is covered later.

Build Process

I used the task runner Grunt to compile and minify my sources and then put all the pieces together to build the webpage. Here are the steps for that:

SCSS

First, the SCSS gets compiled with a compressed style and no sourcemap. This removes comments and whitespace. From there, cross-browser styles are applied with Autoprefixer.

JavaScript

The scripts are processed using Uglify which does a lot to minify them. Mangling (to shorten variable and function names to a single character) was used with the toplevel option set to true as there was no namespacing to worry about.

HTML

The last couple of steps were to insert the compiled CSS and minified JS directly into the HTML file. Because the embedded system isn’t running a true web server, it was much simpler to create a single GET endpoint for the index.html file, than work to allow access to each individual file.

With those files injected, the entire HTML file was minified: comments removed, whitespace collapsed safely, plus attributes and class names sorted consistently. This last piece, the sorting, was to help with compression, the final task.

GZip

By far the biggest savings came from GZipping the file. Since the embedded system isn't going to compress the file on-the-fly, GZipping was done during the build process.

Simply put, GZipping works by replacing repeated characters with references. This is why putting HTML attributes into the same order is important as it creates larger and more repeatable patterns, resulting in higher compression ratios.

It’s interesting to note that this means the DRY principle may not be the best method of writing code—from the perspective of characters typed, not the overarching philosophy. Prior to introducing GZip to the process, I was doing everything I could to help the JavaScript minifier.

I took a dozen calls similar to var $form = document.querySelector("form"); and converted them to this:

var q = document.querySelector.bind(document);
var $form = q("form");
...

While this meant the minifier could get rid of a lot of extra bytes, it wasn’t as efficient for GZip. Discovering these non-intuitive optimisation methods was a rewarding part of the project.

File Savings

  • The raw source files are around 18kB*
  • Minified code from the build process is around 65.7% smaller
  • The GZip compressed file is roughly 27.5% smaller than the size of the source files (about 41.8% smaller than the minified code)

* These are updated numbers from when I wrote the article. They are slightly higher as features have been added, but still reflect the build process savings.

Conclusion

Recall that the “partition” for the firmware is only 1MB. The firmware size is a bit more than 900kB all told, most of that necessary libraries, leaving around 80kB of breathing room.

The webpage is 4.5kB compressed. While I may not have needed to worry about every little byte, adding in a library or being careless with an image could easily have put it above that 80kB threshold. And with over-the-air updates possible, those 80kB may be needed for future improvements!

It was a good little project and fun to see what could be done with such a small amount of available space. Interactive webpages do not need all the frameworks or all the code to be successful.