Menu Search
Jump to the content X X
Smashing Conf Barcelona 2016

We use ad-blockers as well, you know. We gotta keep those servers running though. Did you know that we publish useful books and run friendly conferences — crafted for pros like yourself? E.g. upcoming SmashingConf Barcelona, dedicated to smart front-end techniques and design patterns.

How To Automate Optimization and Deployment Of Static Content

A lot of traffic between users and your site comes from the static content you’re using to set up the user interface, namely layout graphics, Stylesheets and Javascript files.

This article shows a method to improve the providing of static content for a web platform. Further, it will show you a way to automate the deployment of these files, so you can deliver them with least effort but with maximum performance.

This tutorial will take some time to set it up, but it’s going to save you hours of work in the future and will improve your page speed significantly.

1. Why do I need this? Link

There are several approaches to optimize the delivery of contents.
Some use on-the-fly compression via the server itself or a scripting language, which costs performance and does not optimize the structure and content of the files.

The method shown here prepares the files once and also merges and optimizes the code of CSS and Javascript files before the files are compressed, which makes the delivery of them even faster.

  • Most browsers download only two files from a source at once1. If a page needs to load more files from one domain, they get queued.
  • More files to transfer mean more requests to the server, more traffic and more usage of the server’s performance. For the users of the platform, this means longer loading times.
  • The more steps you need to deploy these files, the more space for mistakes is given.
  • Deployment is boring. It’s far more exciting to invest some time once, to set up a reusable automation, than wasting time doing the same copy/paste/upload actions over and over again.

Compare this two screenshots that show the same content before and after the optimization.
The apricot colored parts of the bars stand for the status “in queue” while loading the page.

Before optimization
The “in queue” status means nothing less than: Wasted time of the user.

In this example, the loading time was reduced by 33%, the transferred data size was reduced by 65% and the number of requests to the server even by 80%.

After optimization
By using CSS sprites and merged CSS and Javascript files, there is no queue for the loading of the basic static content.

2. How to improve your static content Link

Besides caching, there are some principles to make the whole setup of static content more efficient right from the start of development.

  • Use CSS sprites2 for your layout graphics. It not only saves you a lot of traffic and loading time. If you got used to maintain your graphics like this, you’ll notice that it can be much more comfortable to have your layout elements e.g. in a single Photoshop file.
  • Don’t blow up the loading time with your CSS and Javascript files. Combine the files of each kind into one single file, minimize them (e.g. by removing line breaks and other unnecessary characters) and compress them using GZip to get the load even smaller.
    This is covered in this tutorial.

3. Automating the deployment using Ruby Link

This automation is written in the Ruby language3. So you need to have Ruby installed on your development computer. The installation is pretty simple on Windows and Mac OS X even ships with Ruby.

Please keep in mind: The scripts are completely independent from the language you’re using to run your site! You can also use them to deploy content e.g. for a PHP project.

If you’re not used to work on the command line, the next steps might look cryptic to you. Don’t hesitate, clench your teeth and take care that you always pass the correct path of the files to deal with. You only have to go through this once. When you’re finished, all you need to do is to type one single command to start the whole process.

To make it easier for you, this is the folder structure used for this tutorial:

+ css/
- fonts.css
- grids.css
- layout.css
- reset.css
- static.min.css (generated)
- static.min.css.gz (generated)

+ js/
- framework.js
- gallery.js
- plugin1.js
- plugin2.js
- start.js
- static.min.js (generated)
- static.min.js.gz (generated)
 
+ deploy/
- batch
- ftp.rb
- gzip.rb

index.php (or something similar)

Requirements for this tutorial Link

Don’t worry, besides Ruby you won’t need anything, that you don’t probably already have:

  • A simple text editor to save the source code.
  • A command line tool, like Windows cmd.exe or Mac OS X Terminal to call the scripts.
  • A FTP account to try out the upload script.
  • A project to enhance.
  • Optional: The Firefox add-on Firebug4 (or something similar for other browsers), to see the enhancements afterwards.

Juicer Link

Christian Johansen created a tool called Juicer which enables you to merge and minimize CSS and Javascript files. To install it on Windows, simply type

gem install juicer

in the command line. When you’re using Mac OS X use

sudo gem install juicer

After the successful install, Juicer will ask you to extend it with YUI Compressor5 and JSLint6 for Javascript compression and verification. You do this by typing

juicer install yui_compressor

and after that

juicer install jslint

in the command line.

Preparing your files Link

The good thing is, you don’t have to change anything in your present files. But to make sure, the files are merged in the order you want to, you need to set up two additional files.

Let’s assume you have four CSS files and five Javascript files:

<link rel="stylesheet" href="./css/reset.css" type="text/css" media="screen" />
<link rel="stylesheet" href="./css/fonts.css" type="text/css" media="screen" />
<link rel="stylesheet" href="./css/grids.css" type="text/css" media="screen" />
<link rel="stylesheet" href="./css/layout.css" type="text/css" media="screen" />

<script type="application/javascript" src="js/framework.js"></script>
<script type="application/javascript" src="js/plugin1.js"></script>
<script type="application/javascript" src="js/plugin2.js"></script>
<script type="application/javascript" src="js/gallery.js"></script>        
<script type="application/javascript" src="js/start.js"></script>

Create a new Javascript file, called static.js with the following content:

/**
 * @depends framework.js
 * @depends plugin1.js
 * @depends plugin2.js
 * @depends gallery.js
 * @depends start.js
 */

After that, create a CSS file static.css with this content:

@import url("reset.css");
@import url("fonts.css");
@import url("grids.css");
@import url("layout.css");

Now you’re ready to run Juicer in your command line.

For Javascript:

juicer merge -i --force ./js/static.js

The parameter -i means that the merging process won’t be cancelled, if JSLint thinks you have errors in your Javascript code. The parameter --force means older versions of the minified file will be overwritten.

For CSS:

juicer merge --force ./css/static.css

As a result you will see two new generated files named static.min.js and static.min.css. You probably want to know if they still work for your site, so go ahead and test it, by replacing the old bunch of link and script tags in your html header with the two new ones.

GZip Compression Link

When your minified files work fine, you can go on to compression. If you get Javascript errors or your CSS layout looks weird, you should re-check the order of the files to merge.

Below you see a small Ruby script that saves a gzipped copy of a file.

require 'zlib'

file_to_zip = ARGV[0]

puts "Gzipping #{file_to_zip}..."

base_name = File.basename(file_to_zip)

file_name_zip = "#{file_to_zip}.gz"

base_name_zip = "zip_#{base_name}"

File.open(file_name_zip, 'w') do |f|

  gz = Zlib::GzipWriter.new(f)
    
  IO.foreach(file_to_zip) {|x| 
    gz.write x
  }   
    
  gz.close
  
end

puts "Gzipped version saved as #{file_name_zip}"

Save it as gzip.rb and call it like this:

ruby gzip.rb ../js/static.min.js

When you look at your folder, you’ll notice a new file called static.min.js.gz. Now do the same for the CSS file:

ruby gzip.rb ../css/static.min.css

Important: Make sure that your server provides the right content encoding and content type information to the clients, so they understand that this files are gzipped content.
You can do this in many ways. For Apache, an example for the .htaccess file would be:

<FilesMatch ".(js.gz)$">
AddType text/javascript .gz
AddEncoding x-gzip .gz
</FilesMatch>

<FilesMatch ".(css.gz)$">
AddType text/css .gz
AddEncoding x-gzip .gz
</FilesMatch>

FTP-Upload Link

You’re on the home stretch. Now you have all improved files together and need to upload them to your host. Again, here’s a little Ruby script that does exactly that for you.

require 'net/ftp'

ftp_host = ARGV[0]
ftp_user = ARGV[1]
ftp_password = ARGV[2]


localfile = ARGV[3] #e.g. "../js/static.min.js.gz"
remote_dir = ARGV[4] #e.g. "www/js"

ftp = Net::FTP.open(ftp_host, ftp_user, ftp_password) 

puts "FTP - Status: #{ftp.status}."

puts "FTP - Go to directory: #{remote_dir}."
ftp.chdir(remote_dir)
puts "FTP - Uploading file: #{localfile}..."
ftp.putbinaryfile(localfile)

files = ftp.list puts "FTP - Your file was uploaded here:" puts files

ftp.close

Save it as upload.rb, put in the credentials of your host, eventually change the destination directory and then call it like that:

ruby ftp.rb example.org admin mysecret ../js/static.min.js www/js

Which stands for:
ruby ftp.rb ftp-host ftp-user ftp-password file-to-upload ftp-destination-folder

The minified Javascript file should be uploaded now. Repeat this for the CSS file and the gzipped versions of both.

Please note that the folders on the ftp server, where the files are copied to, must already exist.

Putting it all together Link

You got all the pieces of this puzzle and now you can finally put them together, by grouping the commands in a batch file. On Windows it would look like this:

@echo off

echo -------- MERGING JS FILES --------

call juicer merge -i --force ../js/static.js 

echo -------- FINISHED MERGING JS FILES --------

echo -------- MERGING CSS FILES --------

call juicer merge --force ../css/static.css

echo -------- FINISHED MERGING CSS FILES --------

echo -------- COMPRESSING FILES --------

ruby gzip.rb ../js/static.min.js

ruby gzip.rb ../css/static.min.css

echo -------- FINSISHED COMPRESSING FILES --------

echo -------- UPLOADING FILES --------

ruby ftp.rb example.org admin mysecret ../js/static.min.js www/js

ruby ftp.rb example.org admin mysecret ../js/static.min.js.gz www/js

ruby ftp.rb example.org admin mysecret ../css/static.min.css www/css

ruby ftp.rb example.org admin mysecret ../css/static.min.css.gz www/css

echo -------- FINISHED UPLOADING FILES --------

Save it in a file called e.g. batch.bat, insert your FTP credentials and call it like this:

cd deploy //switch to subdirectory
batch.bat

On OS X you can do it like that:

#!/bin/bash

echo -------- MERGING JS FILES --------

juicer merge -i --force ../js/static.js 

echo -------- FINISHED MERGING JS FILES --------

echo -------- MERGING CSS FILES --------

juicer merge --force ../css/static.css

echo -------- FINISHED MERGING CSS FILES --------

echo -------- COMPRESSING FILES --------

ruby gzip.rb ../js/static.min.js

ruby gzip.rb ../css/static.min.css

echo -------- FINSISHED COMPRESSING FILES --------

echo -------- UPLOADING FILES --------

ruby ftp.rb example.org admin mysecret ../js/static.min.js www/js

ruby ftp.rb example.org admin mysecret ../js/static.min.js.gz www/js

ruby ftp.rb example.org admin mysecret ../css/static.min.css www/css

ruby ftp.rb example.org admin mysecret ../css/static.min.css.gz www/css

echo -------- FINISHED UPLOADING FILES --------

Save it in a file called e.g. batch, insert your FTP credentials and call it like this:

cd deploy //switch to subdirectory
sh batch

And the content gets merged, minified, compressed and uploaded in one rush.
Additionally to the CSS and Javascript files, it would be a good idea, if the graphics file for the CSS sprites would be uploaded too? Just add an additional call of the upload script in the batch file. It can be used with any filetype.

4. In your template Link

Further development and bug fixing is hard to do in a minified file. But you can add an if-clause in your templates to vary the referencing of your files, depending on your environment.

Take the original files for your local development and the minified for your online version. Additionally you should check if the users browser supports gzipped content (most modern browsers do).

An example for PHP:

if($_SERVER["SERVER_NAME"] == "localhost") {
  //local development server - load all files
} else {
  if (substr_count($_SERVER['HTTP_ACCEPT_ENCODING'], 'gzip')){
    //production system - client accepts gzip     
  } else {
    //production system - client does not accept gzip
  } 
}

And an example for Ruby on Rails:

<% if RAILS_ENV == 'development' %>
  <%# local development server - load all files %>
<% else %>
  <% if self.request.env['HTTP_ACCEPT_ENCODING'].match(/gzip/) %>
    <%# production system - client accepts gzip %>
  <% else %>
    <%# production system - client does not accept gzip %>
  <% end %>
<% end %>

5. Get even faster with subdomains Link

Like mentioned before, most browsers only download two files per host simultaneously. You can bypass that rule by creating some subdomains for your static content and referencing the files through them.

For example, create a subdomain on your web hosting account, that’s called img.example.org, which points to example.org/img.

Do this for all of your folders, containing static content and use the subdomains to reference your resources:

img.example.org/sprites.jpg
css.example.org/static.min.css
js.example.org/static.min.js

This way, the rule won’t be applied and all files can be downloaded at the same time.

6. Conclusion Link

If you got all of this working for one of your projects, you surely recognized the advantages for you as a developer and for the users as the consumers of your content.

These examples don’t claim to be perfect. But if you didn’t approach to automate recurring tasks in your daily workflow as a developer, you hopefully have an idea now, how it could look like.

Always keep in mind, that computers were invented to spare you the boredom of repeating, simple tasks. So save your time for more important things.

Of course, this doesn’t have to be the end of the line. If you’re interested in the automatic deployment of whole applications, you should have a look at Capistrano. Again, it’s written in Ruby, but you can also use it for PHP platforms.

Further Resources Link

Footnotes Link

  1. 1 http://kb.mozillazine.org/Network.http.max-persistent-connections-per-server
  2. 2 https://www.smashingmagazine.com/2009/04/27/the-mystery-of-css-sprites-techniques-tools-and-tutorials/
  3. 3 http://www.ruby-lang.org
  4. 4 http://getfirebug.com/
  5. 5 http://developer.yahoo.com/yui/compressor/
  6. 6 http://www.jslint.com/
  7. 7 http://developer.yahoo.com/performance/rules.html
  8. 8 http://developer.yahoo.com/yslow/
  9. 9 http://betterexplained.com/articles/how-to-optimize-your-site-with-gzip-compression
  10. 10 http://www.uwserver.de/?p=16
SmashingConf Barcelona 2016

Hold on, Tiger! Thank you for reading the article. Did you know that we also publish printed books and run friendly conferences – crafted for pros like you? Like SmashingConf Barcelona, on October 25–26, with smart design patterns and front-end techniques.

↑ Back to top Tweet itShare on Facebook

Advertisement

Christian Bäuerlein recently obtained his degree in Media System Design. He is passionate about Web development and creating all kinds of projects with JavaScript, PHP, Ruby and Java. You can follow him on this journey through his Twitter profile.

  1. 1

    Heh. Writing an article on optimization and failing to notice the javascript bottleneck. Good job. You should move scripts to the bottom of the page to avoid scripts hindering the download of other files.
    Furthermore, if you’re going to advocate a technique, why not actually reference it by name? Using subdomains for content are called Content Delivery Networks (CDN).

    -1
  2. 2

    This looks interesting, let me give it a try.

    Thanks :)

    0
  3. 3

    In my view it’s very important to optimize the delivery of contents. Also i like the Atomization technique. Nice tutorial Christian, Thanks for sharing.

    DKumar M.
    @instantshift

    0
  4. 4

    Armig Esfahani

    July 19, 2009 1:11 am

    pretty interesting… thank you for the post..

    0
  5. 5

    I agree with most, except the sub domains.

    Creating sub domains will increase the number of parallel downloads, but reduce the speed of DNS lookups. The more domains the more lookups.

    Most modern browsers do 6 to 8 parallel downloads. IE7 does 4.

    Therefore, depending on your visitors, you might be better off using one domain for every 6 http requests. You should not count the html file, since it will need to be downloaded by itself first, and then tell the browser what else is needed.

    Also your css images will not start downloading until the CSS file has been downloaded by the browser. Most large websites (yahoo, google) use inline CSS, because it reduces http requests and speeds up the image downloads.

    1
  6. 6

    Emil Bonsaksen

    July 19, 2009 1:20 am

    Great article and great tips on optimizing!

    0
  7. 7

    Very interesting stuff. I’m going to give this a shot first thing Monday. Thanks!

    0
  8. 8

    There is a project called Web optimizator (Web optimizator) that aim to automate all clientside improvements for websites. The main features are: very simple installation on server, website’s resource compression, support for some popular CMS/CMF systems (e.g. Joomla, Drupal, WordPress, Symfony, …) and much-much more.

    0
  9. 9

    Well, i always use gzip and Inline css and JavaScript. I just file_get_contents(…); them with PHP. And I use sprites ofcourse…

    0
  10. 10

    Nice Tutorial. Greets from MSD =)

    0
  11. 11

    great article, this is what exactly i’m going to do next week :)

    0
  12. 12

    Great and useful tips… thanks

    0
  13. 13

    sudo gem install juicer
    Building native extensions. This could take a while…
    ERROR: Error installing juicer:
    ERROR: Failed to build gem native extension.

    /System/Library/Frameworks/Ruby.framework/Versions/1.8/usr/bin/ruby extconf.rb install juicer
    can’t find header files for ruby.

    Gem files will remain installed in /Library/Ruby/Gems/1.8/gems/hpricot-0.8.1 for inspection.
    Results logged to /Library/Ruby/Gems/1.8/gems/hpricot-0.8.1/ext/hpricot_scan/gem_make.out

    Installing Juicer didn’t work on Mac OS X 10.5.7 – any ideas? I’m totally new to Ruby.

    0
  14. 14

    @Frederic: Two possible solutions:
    – Install XCode http://developer.apple.com/tools/xcode/index.html like mentioned here http://www.macruby.org/trac/wiki/Troubleshooting#cantfindheaderfilesforruby

    – Try to update Ruby Gems with this command: gem update –system – see http://rubygems.org/read/chapter/3#page14

    0
  15. 15

    Sunny Singh

    July 19, 2009 3:16 am

    GZIP and sprites are definitely the easiest to do, for me at least.

    Not sure about inline CSS, since it can’t get cached but in most cases (page specific css) can go between style tags. Also I have one main css file “uecore.css” so everything is referenced from one file such as resets and template styles.

    This form should really have a website input by the way, most blogs do it and it’s useful, why not smashing magazine?

    0
  16. 16

    Oh god what is the world coming to? RoR in Smashing Magazine Examples without PHP examples. *sigh*

    1
  17. 17

    Jasmin Halkić

    July 19, 2009 3:42 am

    Nice tips, thanks!

    0
  18. 18

    very helpful! Thanks!

    0
  19. 19

    Oliver Schrenk

    July 19, 2009 5:41 am

    Nice article, but you should rely on rsync instead of f t p. It offers synchronizing over a secure connection and best of all only syncs what is necessary, which is helpful if you have big projects.

    0
  20. 20

    Hi Andy,
    your remark about the JS bottleneck is absolutely correct. But please mind, that this article is not a complete round up of all optimization techniques, it just shows a optimization method for the file contents itself. Further possibilites – like the placement of javascript, expiring headers and etags – can be found via the section “Further Resources”, e.g. the Yahoo Best Practices.

    A CDN usually depends on a distributed system, not only on a subdomain.

    0

↑ Back to top