Remember when static websites were the only things that were served on the internet? I don’t know about you but I definitely remember the pain of uploading all my sources files via FTP to a web server only to realize there was a mistake in one of the HTML files. Shnit!
Nowadays, while web has become significantly more complex, there are companies, mom and pop shops and individuals looking to make it simpler. Most websites are only as good as the software engineers maintaining it but what if you could reduce the risk, increase the speed at which your site is served and save money in the process? By using a static website generator framework. These frameworks come in all shapes and sizes and work with Ruby, Go, Python, and even Javascript.
By the end of this article, you will know all the reasons why I have run my websites statically for the past 4 years and will continue into the future. You will undoubtably be inspired to maybe even configure your own. So, without further ado, let’s figure out how these things work!
How Does It Work
A static website generator has a few nuances and typical configurations that you should understand. From what I can tell across different frame works they are similar if not the same. Let’s, for instance, talk about the file structure.
File Structure
➡️ layout folder - this folder contains all the pertinent views required to generate your website. Every ‘view’ is essentially a piece or small portion of HTML code that is taken and then stitched into the bigger picture. For instance, I may have a Mailchimp signup box on my website but i’ve decided to keep it in it’s own sub-layout file. That way I can import that file in multiple locations on my website without having to copy, paste, and edit redundant code!
➡️ static files - these files are your CSS, Javascript, etc. They can be preprocessed by your website generator but generally they can be left alone
➡️ images - this is where your images live (obviously). One of the cool things about images on some platforms is the fact that you can reference a larger photo and the generator will automatically resize and crop to the necessary photo sizes. This optimizes your website and ensures that you’re not loading a 6 megabyte photo in a 100x100px space.
➡️ content - this is where your blog content or pages live. They are typically written in some type of Markdown format. This enables you to commit your complete website to a source control program like Git and then track and push changes with ease. If you’re a frequent user of Git, using this type of framework will come to you easily. (Extra bonus if you already know how to write documentation in Markdown, if not you can check out the guide here: www.markdown.com)
Compiling
The use of a static website generator is very similar to the process of compiling a program. A website generator will traverse of every piece of source code (typically HTML, Markdown or even data files like TOML,YAML and JSON) that is used for a your website. It then takes that source and combines layout files to create dynamic pages as if it were served to you via a framework like Wordpress, or Ruby on Rails. So, in other words, the website generator will programmatically create links to all your content in all the right places. For instance, if you are using a blog, your main blog page probably has 25+ links to other pages on your site. Every single one of these are generated dynamically and then placed into that particular page’s index.html. Everything is set in stone and there is no more CPU time required to generate this page other than to serve it over HTTPS.
Usually, when a website is generated, it is placed into a _site folder or public folder. These folders can then be synced to a web server via rsync or if you’re using a turnkey service like Netlify, then Netlify will be waiting for changes to your git repository and rebuild your site right then and there.
An example rsync command may look something like this. I highly recommend using rsync because not only is it thorough but it’s also secure (over SSH!)
rsync -rtzh --delete \_site/ <user>@<your-site>>.com:/home/deploy/<your-servers-public-folder> --exclude-from ./exclude.txt
If you’re building on your production machine you can use the following:
rsync -rtzh --delete \_site/ /home/deploy/<site folder name> --exclude-from ./excludes.txt
Syncing to something like Amazon S3 is also straight forward. You can use a commands like the following which will recursively add files as needed:
# Upload to S3!
echo "\n\n-> Uploading to S3"
# Sync media files first (Cache: expire in 10weeks)
echo "\n--> Syncing media files..."
s3cmd sync --acl-public --progress --exclude '\*.\*' --include '\*.jpg' --include '\*.jpg' --include '\*.ico' --add-header="Expires: Sat, 20 Nov 2020 18:46:39 GMT" --add-header="Cache-Control: max-age=6048000" dist/ s3://<site-name>
# Sync Javascript and CSS assets next (Cache: expire in 1 week)
echo "\n--> Syncing .js and .css files..."
s3cmd sync --acl-public --progress --exclude '\*.\*' --include '\*.css' --include '\*.js' --add-header="Cache-Control: max-age=604800" dist/ s3://<site-name>
# Sync html files (Cache: 2 hours)
echo "\n--> Syncing .html"
s3cmd sync --acl-public --progress --exclude '\*.\*' --include '\*.html' --include '\*.xml' --add-header="Cache-Control: max-age=7200, must-revalidate" dist/ s3://<site-name>
# Sync everything else, but ignore the assets!
echo "\n--> Syncing everything else"
s3cmd sync --acl-public --progress --exclude '.DS_Store' dist/ s3://<site-name>
# Sync: remaining files & delete removed
s3cmd sync --acl-public --progress --delete-removed dist/ s3://<site-name>
All website generators vary, for instance Hugo has a themes folder which, in my opinion, complicates things. By far the most stable and feature rich out of all of them is Jekyll. But there are some trade offs, of which I’ll speak of later in this post.
Local editing
One of the additionally nice things is that these generators all have a way to watch for file changes and then update a local copy while editing. This is useful when you’re polishing a blog article for publication or if you’re tweaking visual elements in HTML or CSS.
These commands spin up a local web server and serve your files directly. For instance to load up the preview on hugo execute the hugo server command. For jekyll execute the jekyll preview command.
Additionally, there are services out there that act as CMS for static website generators. There is a huge list at Headless CMS the same guys that run Netlify and StaticGen. That way, you as a web developer can handle the way things look while someone less technical can deal with filling the site up with the necessary content.
Saving Time and Money ($$$)
Say you’re running a traditional web server. A typical web transaction may look something like this:
➡️ Send GET request for the url to server ➡️ Server pieces together the requested page by fetching a template ➡️ It then ensures that all sub-templates are also populated ➡️ It fills in all the pertinent variable data for the page. This may require a database lookup (which adds time!) ➡️ The server packages that html file up and sends it back.
Not only does this transaction cost more time to process but it also costs CPU and Disk time. Considering Amazon Web Services charges based on the use of bandwidth and CPU time, we can reduce costs significantly by serving a completely pre-generated static website. Here’s what a typical GET response may look like.
➡️ Sends GET request for the url to the server ➡️ Server fetches static file from disk ➡️ Server packages that file and sends it back
As you can see, this is much simpler and doesn’t require much work on your server side. The reduced requirement on CPU time essentially makes it much easier to accept a larger amount of connections. So, if you’re serving up a popular blog or product website, a static website will always out-perform a dynamically generated one.
Bye Bye Wordpress!
One of the other main advantages of this system is that there is no pre-processing before the content is sent to you. If you have ever owned or used a Wordpress site, you may have seen this error message at one point in time:
With statically generated sites, unless your web server is turned off or disconnected from the internet, it will always serve the content that you provided for it. There is nothing dynamically created on the server side thus keeping your server overhead to a minimum.
This also protects you from bugs and security leaks that may be in Apache, Wordpress or in the PHP renderer itself. One of the scariest things about Wordpress is that anyone could gain access to your admin console which could lead to some disastrous results. With a website generator framework, your work is backed up in Git and can be re-deployed to AWS S3, Netlify, or your own server with one or two commands (or a few clicks).
Which Static Site Generator?
I have had the chance to experiment with a few static site generators over the past few years. Jekyll was first on the scene during the heyday of Ruby-on-Rails. Thus, you could probably guess Jekyll is completely Ruby based. Since then there have been a few more very popular static site generators come on the scene.
Hugo is a static site generator based on Google’s Go programing language. I have been using it for this site for a few months now and i’ve been impressed with how much work they’ve accomplished in a short amount of time. The only thing it’s really lacking is support for dynamic image resizing and asset optimization. I have fixed that though with a custom Gulpfile which does post-processing after Hugo has generated the site.
There are tons of other site generators out there but those happen to be the most used. If you are a person who likes minimal frameworks I highly suggest you check out Middleman. Its a bit more complicated to set up, especially for a blog, but it’s worth the time investment. It’s also much more finely polished compared to Jekyll or Hugo.
For more ideas, if you’re particular about which language you use, you can check out StaticGen where they sort the most popular generators and indicate the framework, and templating language used.
Note: All my statically generated sites use Hugo. After experiementing with it over the past few years, I’m pretty pleased with it and it just keeps getting better!
Deploying wherever
The nice thing about a static server is you can deploy it anywhere on the web. From a personal server running on Digital Ocean, to Amazon to Netlify there is an option for everyone. I have personally done all three and have found that a full turnkey solution far outweighs the benefits of using any other service. Let me outline some pluses and minuses of each
Personal VPS (Digital Ocean, Linode, etc)
I ran a Ubuntu server from about 2012 using Linode and Digital Ocean up until just recently. Over the past 4 years, I have found it increasingly more frustrating to manage, keep updated and keep it from crashing. Depending on how complex your stack is, it’s a full time job to keep a personal server up and running without breaking. As I had mentioned, running a static site is much simpler on a standard VPS but, in most cases, you need to roll your own method for generating the site and then syncing it to the public web directory. This, in an of itself, can be error prone especially if you add or remove custom features or Ruby Gems!
Pluses:
- You have the ultimate control over this server and your source files
- Cost is relatively low
- Can be configured which ever way you’d like
Minuses:
- You are ultimately responsible whether your server is working properly or not
- You are solely responsible for getting HTTPS certificates generated and installed correctly
- When you are adding sites, you are responsible for configuring your web server (like Nginx) to route web requests correctly
- You are responsible to ensure that your site stays up and running when large of amounts of traffic hits your latests and greatest blog post
- You are responsible for optimizing your web server for web page speed and reliability (compression, cache, asset optimization, etc)
Amazon S3 + Cloudfront
Amazon is the de-facto standard for hosting anything web wise. Having a robust content distribution network an awesome track record (for the most part) for staying available it makes sense why it’s the go to for so many folks. They recently came out with a feature called “Website Hosting Quickstart” which automatically sets up their service for a statically hosted site. More information for that service can be found here.
When implementing everything I had to figure out a few things though:
- How to programatically upload my files and update only the files necessary
- How to optimize my files for the fastest delivery to the end user
As you can tell, if you spend too much time on optimization you’ll never have any time to write more content. Isn’t writing good content the most important thing in the first place?
Pluses:
- Reliability and speed of a robust CDN
- De-facto standard for anything web in 2017
- Costs are minimal
- One click HTTPS configuration
Minuses:
- Less leeway and freedom. Still requires you to upload content from a machine. i.e. Automated posts still require a “compilation” machine which will compile your site and update S3 accordingly.
- Frustrating to optimize for speed. This is based on my use of Google’s Page Speed Insight tool.
- Requires use of outside tools like
s3cmd - Requires use of Amazon’s proprietary DNS
- DNS can be tricky to configure
Netlify
I eventually discovered Netlify on Hacker News (good one guys) and ignored it as I went about my business running my life into the ground as I spent countless hours dealing with server and optimization crap that I should never have to do. After the search for something Hugo related, I noticed again Netlify was sitting right before me. So, after a minute of thought I said “something that will save my time and help me focus on content without costing an arm and a leg? Hell yes!”
Within a few hours or so after converting all my submodules to subtrees in all my repositories, (For Hugo it was my theme and for my Jekyll site it was all my plugins) I was able to launch all my statically generated sites with a simple Git push. What a sigh of relief. Things just worked.
Additionally, after running my site through Google’s Page Speed Insight tool or Pingdom’s speed tool, I noticed that Netlify had optimized their service to be as fast as possible. I was amazed that all my problems were solved with a simple git push.
Pluses:
- Optimized web experience without a single configuration file or http header to futz with
- Deployed with a simple
git push - One click HTTPS configuration
- Easy domain name configuration
- Can trigger a compilation by a webhook ping. (I’m using Zapier to ping Netlify)
- Also has the ability to send a webhook to Slack, etc so I know when things are working and when they aren’t
Minuses:
- If something is broken, you are at the whim of Netlify (or on your debug skills depending on how brave you are)
- If you’re using some esoteric utilities to generate your site, they may not have it and thus your site may not compile.
- A bit more costly than the above alternatives
As you can see, there are some consistent pluses and minuses to each system. The biggest tradeoffs that I could see were the difference between having complete control versus operating in a ‘server-less’ environment where you, as the website engineer/designer, don’t have to worry about such things. With a server-less configuration, all the hard work is done for you.
Some folks like control, and I was one of those people for a long time, but that also potentially means your setup will be fraught with server issues, updates and security holes on a constant basis. Whereas the other solutions are a bit more bullet proof because security comes down to how secure the static web server is and how strong your SSL cert is versus the password strength of your Wordpress site. (Talk about things that keep me up at night. 😨)
Securing Everything Down
There’s been a lot of hubub on the web since the House of Representatives repealed an action that kept normal citizens safe from the prying eyes of big internet companies like AT&T, etc.
What does this all mean though? Now, there is no recourse for companies (that you pay for service) to intercept your unencrypted website traffic to see what you’re doing on certain websites. Is the equivalent of tapping a phone line and listening to the conversation.
So how can you protect your site and also the folks who visit your site? Enable HTTPS This will enable your web server to securely serve your website with a less of a chance for internet service providers to inject code into your documents to track what you or your viewers may be doing at any one time.
Lots of services now have complementary certificate that you can generate to make your site HTTPS compliant. For instance, AWS gives you the option to to generate your HTTPS certificate on Cloudfront here:
Their system makes it extremely easy to get up and running. It’s a simple click of the mouse.
Also, as I mentioned before, I’m a very happy customer of Netlify. They also give you the ability to create a one click security certificate which can be used to encrypt the transmission of your website. Again a simple click of a button and you’re ready to go!
HTTPS is additionally even more important when you’re transmitting and receiving sensitive customer information across the web. So, no matter what, you should always serve it with HTTPS/SSL.
Conclusion
Programming on the web is always about picking the right tool. For most websites concentrating on making it easier to get your websites to your customers will help you in the long run by staying focused on what matters. Why spend hours configuring NGINX when it’s much more important to create new products or write new blog posts. Seriously, all that nitty gritty stuff can be left to the engineers at Amazon or Netlify. Remember, that extra few dollars you pay per month may go far compared to the alternatives of spending your own valuable time doing the same things.
What are your biggest pains in running a site? Send me an email comments@jaredwolff.com or write a comment below.
Last Modified: 2020.3.7
