Addressing The Responsive Images Performance Problem: A Case Study

Advertisement

Five-inch mobile devices are on the market that have the same screen resolution as 50-inch TVs. We have users with unlimited high-speed broadband as well as users who pay money for each megabyte transferred. Responsive design for images is about optimizing the process of serving images to users. In this article, we will share our responsive image technique, the “padding-bottom” technique, which we researched and implemented on the mobile version of the Swedish news website Aftonbladet.

The techniques presented here are the result of a few weeks of research that we did in October 2012. We were fortunate enough to be a part of the team that built a new responsive mobile website for Aftonbladet. Aftonbladet is Sweden’s largest website, and its mobile version gets about 3 million unique users and up to 100 million page views per week.

With that amount of users, we felt it was our responsibility to make a fast and well-optimized website. Saving just 100 KB of image data per page view would translate into a lot of terabytes of data traffic saved in Sweden per year. We started out by researching other responsive image techniques, but because none of them was a perfect match, we ended up combining some of the best hacks into our own solution. Please note that this project covered only a responsive mobile website; but do not worry — the technique presented here applies to all types of responsive websites.

The Specification

We started out by creating a simple specification in order to select a suitable responsive image solution. The solution had to:

  • be easy to cache,
  • multiserve images.

Let’s go through these requirements and see what they mean.

Easy to Cache

With a website that gets traffic peaks of over 10,000 requests per second, we wanted to keep the server logic as simple as possible. This means that we didn’t want to use server-side device detection or a cookie-based solution that serves multiple versions of the HTML. We needed a single HTML file to be served to all users, although manipulating the HTML with JavaScript after it has loaded is acceptable. The same rules apply to images; we needed to be able to put the images on a content delivery network (CDN), and we did not want any dynamics in the image-serving logic.

Multiserving Images

We wanted to serve different image versions to different devices. One big complaint about our previous mobile website was that high-DPI iPhones and Android devices did not get the high-resolution images they deserved. So, we wanted to improve image quality, but only for the devices that were capable of displaying it.

Loading Images With JavaScript

JavaScript, if placed in the footer where it should be, will load after the HTML and CSS has been parsed. This means that, if JavaScript is responsible for loading images, we can’t take advantage of the browser’s preloader, and so an image will start downloading a fair bit later than normal. This is not good, of course, and it reveals another problem: The page might reflow every time the JavaScript inserts an image into the DOM.

Reflowing happens when the browser recalculates the dimensions of the elements on the page and redraws them. We have set up a demo page and a video that demonstrate this effect. Note that the demo page has an inserted delay of 500 milliseconds between each image in order to simulate a slow connection speed.

As you can see from the video, another very annoying feature is that the user will likely get lost in the reflowing when returning to a page with the “Back” button. This is actually a serious problem for websites such as Aftonbladet. Having a functional “Back” button will keep users longer on the website.

The reflowing problem would not really be present on a website that is not responsive because we would be able to set a width and height in pixels on the image tag:

<img src="img.jpg" width="60" height="60"/>

One important aspect of responsive Web design is to remove those hardcoded attributes and to make images fluid, with CSS:

img {
	max-width: 100%;
}

No More max-width: 100%

We needed to find a solution whereby we could reserve space for an image with only HTML and CSS and, thus, avoid reflowing. That is, when the JavaScript inserts an image into the page, it would just be inserted in the reserved area and we would avoid reflowing. So, we threw out one of the cornerstones of responsive Web design, img { max-width: 100% }, and searched for another solution that could reserve space for a responsive image. What we needed was something that specifies just the aspect ratio of the image and lets the height shrink with the width. And we found a solution.

The Padding-Bottom Hack

This technique is based on something called intrinsic ratios, but because none of our team’s members could remember, understand or pronounce the term “intrinsic” we just called it the “padding-bottom hack.”

Many people learned about this feature back in 2009 in A List Apart’s article “Creating Intrinsic Ratios for Video,” by Thierry Koblentz, and the technique is often used to embed third-party content and media on responsive websites. With the technique, we define the height as a measure relative to the width. Padding and margin have such intrinsic properties, and we can use them to create aspect ratios for elements that do not have any content in them.

Because padding has this capability, we can set padding-bottom to be relative to the width of an element. If we also set height to be 0, we’ll get what we want.

.img-container {
	padding-bottom: 56.25%; /* 16:9 ratio */
	height: 0;
	background-color: black;
}

The next step is to place an image inside the container and make sure it fills up the container. To do this, we need to position the image absolutely inside the container, like so:

.img-container {
	position: relative;
	padding-bottom: 56.25%; /* 16:9 ratio */
	height: 0;
	overflow: hidden;
}

.img-container img {
	position: absolute;
	top: 0;
	left: 0;
	width: 100%;
	height: 100%;
}


The container reserves the space needed for the image.

Now we can tweak our demo, applying the padding-bottom hack, and the user will no longer get lost in the reflowing that we saw earlier. Also, the “Back” button functions as expected. See the new video and demo.

This technique improves the user experience of the website quite a bit over the traditional max-width approach, but the experienced reader will by now have noticed two things:

  1. We need to know the aspect ratio of the image before we load an image.
  2. Images could be scaled up larger than their original size.

To handle the aspect ratios, you need either to have a content management system with which you can control the templates or to have a limited, fixed set of aspect ratios for images. If you have something in between, whereby you cannot affect how image tags are rendered, then this method will probably be hard to use.

At Aftonbladet, we decided to calculate the padding-bottom percentage on the server and print it out as an inline style in the HTML, as you will see in the following code snippets. For the second problem, we found that, for our use case, letting the image scale up if needed (and losing some quality) was actually better than setting a fixed maximum width for the image.

Choosing An Image-Loading Technique

Now that we’ve allowed ourselves to load images with JavaScript, because we’ve minimized the reflowing, we can set up the requirements for this:

  • The resulting HTML should be a single img tag.
  • The DOM elements should be minimal.
  • It should execute as quickly as possible.
  • It should not break when JavaScript is disabled.

Based on this simple specification, we created an inline vanilla JavaScript, based on the “noscript” technique. The idea is to add the information about different image sizes to the HTML as data attributes in a noscript tag. The content of the noscript tag would be an img tag and would be shown to browsers that have JavaScript turned off. Let’s look at the markup:

<noscript data-src-small="img-small.jpg" 
    data-src-medium="img-medium.jpg" 
    data-src-high="img-high" 
    data-src-x-high="img-x-high.jpg">
        <img src="img-small.jpg">
</noscript>

The job of the JavaScript, then, is to parse the content of the page, identify images that should be lazy-loaded, check the size of the device’s screen and pick the correct image. The following code would look for images to load and insert them into the DOM. It is important that the JavaScript be inline and load as soon as possible after the HTML. The script would also retrieve the alt tag from the noscript tag and insert it into the newly created img tag.

	var lazyloadImage = function (imageContainer) {

		var imageVersion = getImageVersion();

		if (!imageContainer || !imageContainer.children) {
			return;
		}
		var img = imageContainer.children[0];

		if (img) {
			var imgSRC = img.getAttribute("data-src-" + imageVersion);
			var altTxt = img.getAttribute("data-alt");
			if (imgSRC) {
				var imageElement = new Image();
				imageElement.src = imgSRC;
				imageElement.setAttribute("alt", altTxt ? altTxt : "");
				imageContainer.appendChild(imageElement);
				imageContainer.removeChild(imageContainer.children[0]);
			}
		}
	},
	lazyLoadedImages = document.getElementsByClassName("lazy-load");

	for (var i = 0; i < lazyLoadedImages.length; i++) {
		lazyloadImage(lazyLoadedImages[i]);
	}

Picking The Perfect Image

So far, the techniques described here generally apply to any website that implements responsive images. The last step, selecting the image to send to the browser, is different in the way that it has to be adapted to the needs of the website.

Many factors need to to be considered when choosing the optimal image to send to a particular device, such as screen size, network speed, cacheability, overall page weight and the user’s preference. The website we built for Aftonbladet mainly targets mobile browsers, and we were lucky enough to have a lot of statistics on the average user’s behavior. By analyzing the numbers, we could identify some trends.

First, the vast majority hold their device in portrait mode. For reading and browsing articles, portrait mode is the natural choice. And while screen size varies a lot, over 99% of the traffic we analyzed represent devices with a viewport width of either 320 or 360 pixels.

Secondly, most of the visiting devices have high-density screens, with a native width of 480, 640, 720 or 1080 pixels. The highest resolutions come from newer phones, such as the Galaxy S4 and Xperia Z; while a 1080 pixel-wide image looks great on those phones, tests showed that a 720 pixel-wide image would look good enough, with less of a bandwidth cost.

After analyzing the data, we settled on three versions for each image:

  • small (optimized for a 320 pixel-wide screen),
  • medium (optimized for a 640 pixel-wide screen),
  • large (optimized for a 720 pixel-wide screen).

(Devices without JavaScript would get the small image.) We believe these settings are reasonable for a mobile website, but a fully responsive website that targets all kinds of devices would probably benefit from different settings.

We give the versions logical names, instead of specifying media queries in the markup. We choose to do it this way for flexibility. This makes it easier to evolve the JavaScript in the future to, for example, adapt to network speed or enable a user setting that overrides which image to use. In its simplest form, the engine for selecting image versions could be implemented as in the following example (although, to support Internet Explorer, we’d need another function as a workaround for the absence of window.devicePixelRatio).

var getImageVersion = function() {
        	var devicePixelRatio = getDevicePixelRatio(); /* Function defined elsewhere.*/
        	var width = window.innerWidth * devicePixelRatio;
        	if (width > 640) {
                    	return "high";
        	} else if (width > 320) {
                    	return "medium";
        	} else {
                    	return "small"; // default version
        	}
};

We also tried to take the screen’s height into account when selecting the right image. In theory, this would have been a nice complement to make sure that the image is well suited to the device. But when we tested the theory in the real world, we soon found too many edge cases that made it impossible to detect the height in a way that was good enough.

Apps that embed Web views inside scroll views reported a seemingly random height between 0 and 3000 pixels, and features such as Samsung’s TouchWiz system, which has a split-screen mode, made us abandon the screen’s height as a reliable value for choosing image sizes.

We created a simple demo page that has all of the JavaScript and CSS needed. But keep in mind that our code is targeted at mobile devices, so it doesn’t work out of the box in, say, old Internet Explorer browsers.

Making Smaller Images

Large beautiful images use up a lot of bandwidth. Downloading a lot of 720 pixel-wide images on a cellular network can be both slow and costly for the user. While a new image format such as WebP would help some users, it is not supported by enough browsers to be viable as the only solution. Fortunately, thanks to research by Daan Jobsis, we can take advantage of the fact that a high compression rate doesn’t affect the perceived quality of an image very much if the image’s dimensions are larger than displayed or if the image is displayed at its native size on a high-density screen.

With aggressive JPEG compression, it is, therefore, possible to maintain a reasonable download size while still having images look beautiful on high-density displays. We already had an image server that could generate scaled, cropped and compressed images on the fly, so it was just a matter of choosing the right settings.

This is also one reason why we didn’t include an image version for 480-pixel screens. Scaling down a 640 pixel-wide image with a high compression level made for a better-looking image at a smaller size than we could achieve with an image that had the native resolution of the 480-pixel screen. In this case, we decided that making the device scale the image to fit was worth it.

Red Areas Don’t Compress Well

A high compression rate is no silver bullet, though. Some images look terrible when compressed, especially ones with prominent bright-red areas, in which JPEG compression artifacts would be clearly visible and could spoil the overall impression of the image. Unfortunately, the editors at Aftonbladet have a fondness for images with prominent bright-red areas, which made our task just a little more challenging.

artifacts
These two images are saved with a 30% quality setting. While the image on the left might be passable even on a normal screen, the red circle in the right image looks bad even on a high-density screen.

Finding a Compromise

We could solve this problem in a few ways. We could increase the dimensions and compression of the images even more, which would make the artifacts less visible. This would mean that the browser has to scale images while rendering the page, which could have a performance impact. It would also require larger source images, which in practice are not always available. Another solution would be to let the editors choose how compression should be handled for each image, but this would add another step to the editors’ workflow, and we would need to educate them on the intricacies of how image compression, size, performance and mobile devices work together.

In the end, we settled on a compromise and avoided using high compression rates for important images and image galleries, instances where the image is the center of attention. In these cases, we also make sure to load only the visible images, to not waste the user’s bandwidth.

quality
The teaser images on Aftonbladet’s section pages (left) work really well with high compression levels, while a full-screen image gallery (right) benefits from higher-quality images.

Generating the different images can be problematic. We have the luxury of an existing back end that can scale, compress and crop images on demand. Rolling your own might entail a lot of work, but implementing it as, for example, a WordPress plugin (using the WP_Image_Editor class) would actually be pretty straightforward.

The Bottom Line

Three years after Ethan Marcotte introduced responsive Web design, we’re still struggling to find a good solution to the problem of how to handle images. All responsive image solutions are more or less a hack, and so is this one. But we have managed to escape the excessive reflowing problem, we have not introduced a lot of unneeded DOM elements, we have a cacheable website, and we prevent images that aren’t used from being downloaded.

Aftonbladet’s home page on a mobile device has around 40 images and, with this technique, ends up being around 650 KB on a “large screen,” 570 KB on a medium screen and 450 KB on a small screen (although the size varies according to the content). Without the high compression rate, the large and medium versions would be over a megabyte in size. And compared to the old website, we’ve managed to move from blurry low-resolution images to high-quality images tailored to each device, with just a 25% increase in download size.

So, we are still waiting for the perfect solution to responsive images. In the meantime, what we have outlined above has been a success for Aftonbladet’s new responsive website and hybrid apps. The new website (whose perceived loading time is twice as fast as that of the old one) has led to a huge boost in traffic and user interaction; and all through the summer, traffic to Aftonbladet’s mobile version has been higher than traffic to the desktop and tablet versions.

(al) (ea)

↑ Back to top

Anders loves to build stuff on the web. He specialises in building responsive and mobile web solutions and works as a consultant for Valtech in Stockholm.

Tobias is a front-end and back-end web developer with a passion for web performance, scalability and security. He is the lead developer at Aftonbladet, where he's trying to reshape the news industry from within.

  1. 1

    Nice approach and a good explanation. I had the same idea and made a library for it with pure frontend javascript (no backend): http://joeyvandijk.github.io/rimg/

    It will listen to DOM events, browser resizes, the DOM load event or you can manually initiate the change.

    Any feedback/PR is welcome.

    ;)

    1
  2. 3

    Cool Idea! Thanks

    0
    • 4

      But it doesn’t cause any SEO problem?

      0
      • 5

        Nope, images inside noscripts would, together with its alt attributes, be parsed by search engines.

        0
        • 6

          Can someone corroborate Joan’s comment? I know that the SEO concern of wrapping images in a noscript tag was a major concern for a lot of developers. However, it is the approach used for no-js image fallbacks in picturefill.

          0
          • 7

            Most search engines, and most certainly Google can parse JavaScript. So even if your images are shown with only JS chances are Google will see them just fine. Especially on a site that is trafficked as often as Aftonbladet.

            0
          • 8

            There are also search engines that don’t render javascript and those search engines would indeed pick-up the images defined in the noscript-tags. A lot of black hat SEO engineers have abused the noscript-tag therefore to spam a lot of keywords into a page to achieve a higher ranking. So search engines have a filter regarding the use of noscript-tags. However, if you can say with certainty that your code in the noscript-tag is in no way an attempt into tricking the search engines, which this reponsive image technique definitely isn’t, then you’ll be fine.

            0
          • 9

            No SEO issue with this clever techinique. I’ve tried out on a website I built some time ago for a guy ( manueledepretto.com ). If you type his name on google you’ll find out all the images coded in the website.

            By the way, I used picturefill.

            0
        • 10

          I’m not sure if it is a recommend practice and would not rely on the point that spiders does index links within the noscript tag in the same way they would index normal image sources. According to John Mueller:

          “One of the problems with noscript is – as others have mentioned – that it’s been abused quite a bit by spammers, so search engines might treat it with some suspicion. ”

          http://goo.gl/ciNWCx

          … but I would be thankful for any examples.

          0
      • 11

        This is my number one concern in regards to responsive image techniques that involve client side render. I’m surprised it’s always left unsaid in articles, and even in popular libs like picturefill. Is there any real world proof or test case that googles indexes such images fine ?

        0
        • 12

          Yri:

          It is hard to find concrete answers to this topic, but it is old and well-discussed one. Below is a solid proof that it does work: a search result that shows images from my website’s Pinterest board. Pinterest’s content is populated by JS.

          http://bit.ly/1drkXtj

          0
          • 13

            I don’t think Pinterest images are populated with JS, since they work with JS disabled and have the same src there. Plus it has a permanent url for each pin, where it most certainly renders the regular server-side images. Pinterest layout is certainly rendered with JS, but it doesn’t look like there’re client-side image selection solutions present.

            Anyway, I’d like to hear the authors’ take on this, since SEO is of critical importance for resources like Aftonbladet.

            0
  3. 14

    Great technique! But if I understand well, the padding (padding-bottom: 56.25%; /* 16:9 ratio */) always has to be related to the images, right?

    0
    • 15

      Yes, definitely. This value is dependent on the aspect ratio of the image. Since our images vary widely in size, we actually set it as an inline style attribute on the image container element.

      0
  4. 16

    We’re developing a site using very much the similar design patterns right now (padding-top in percentages, calculating the best image resolution on a per image basis using available pixels, etc) and like it very much. Glad we’re on the same page here! :)

    /Anton

    0
  5. 17

    Thank you for sharing your approach. :-)

    0
  6. 18

    I am interested in the standard moving forward…

    0
  7. 19

    COME ON!! Every time I implement something “new” I end up reading about a better way of doing it with better performance. For once could I read this FIRST then work on my project?!

    Now I have to go find a project to implement this in. You realize you are making have to work more ;)

    0
  8. 20

    I’d love to implement this, but I’m afraid I don’t have as clear a picture of my users’ behavior. How did you get your great statistics?

    0
  9. 21

    Thanks for sharing this technique. I’m running into a little bit of an obstacle. I want to left float some lazyloaded thumbnails, and have text wrap around it, but when I apply float left to the .img-container, the image inside disappears.

    The only way I can get it to work is to add another container div to wrap around the already existing .img-container div. Is there a more elegant way to accomplish this without the extra markup?

    0
    • 22

      The image is absolute positioned so it is easier to run into trouble and get into conflicts with other CSS rules, but most things are solvable. It will be easier for someone to help you out if you add a code example on codepen.io or similar.

      0
  10. 23

    Hey! that’s Great!!! This is the one I was searching for my project myoffstreet.com I am in that team and we are now making it responsive. We are using lazy load. This article is really helpful for us. Thanks Smashing :)

    0
  11. 24

    There’s a part of me that just wishes the web were a more mature medium than it currently is; so that we didn’t need to hack so heavily in order to achieve functionality that should be the status quo, but I think I’m living in a Fool’s Paradise to keep that mindset. (Could you blame me? The rent was great!)

    That being said, for the small amount of additional markup that this hack requires, and the fact that I had not yet even heard of reflow (thankfully your video better demonstrated the issue so I could see that yes, I noted the issue at hand as well in the browser demo) but I have to say, the fact that you guys took on issues as seemingly minuscule as this is very commendable; Thanks for sharing it.

    0
    • 25

      Geert van der Heide

      September 18, 2013 1:12 am

      I agree Aaron, more mature tech to handle things like this should already exist by now. But web technology has always been a step behind the latest use cases, and that’s probably not going to change.

      Ideally, the browser would automatically communicate screen size, screen density and most of all: bandwidth, to the server, and the server would then automatically send the most appropriate image back to the browser. This process should be cached so that images are resampled only once for each size and quality. Something like this would require no markup, configuration, preparing of images or javascript. But a solution like this would require refactoring of browsers, server tech and maybe the protocols as well. So I’ll keep dreaming for now, and wait for someone way smarter than me to someday create a system like this.

      0
  12. 26

    Thanks for the solution. I have been searching for good solution for the same and this sounds great. Does this solution work better when the page has more than 100 large images (around 1980*1020)? I have little concern regarding this.

    0
  13. 27

    Only 25%..?

    I’d think that 25% is quite a big amount. 25% pay rise, anyone?

    0
  14. 28

    Geert van der Heide

    September 18, 2013 3:24 am

    Nice article, but it focusses way too much on image size / quality, and too little on bandwidth and performance. For Western countries with fast internet, that might make some sense. But even then, bandwidth and performance should never be lower on the list than image quality.

    Specifically, I think mobile users with High-DPI screens would very often prefer non-retina images if it means their loading speed doubles. User bandwidth is, unfortunately, not as easy to grab as screen size, but in my opinion it should be the leading metric in deciding which image to show to which user.

    0
  15. 29

    One problem with this technique is that it assumes one width/pageview. Snap view (Win 8) changes or just regular browser resizes either result in extremely pixelated images (when not monitored) or excessive loading (load new images) and increased javascript monitoring (bad for performance).

    It’s a dangerous assumption :)

    0
  16. 30

    Great post, I came up with a similar solution here. http://caracaldigital.com/retina-handling-code/ . The no script with data attributes is a great idea, I Instead went for just a js object detailing the image cuts and I used a .js css rule prefix to hide images until my script kicked in. I think the no padding hack is great too, I’ll definitely incorporate these tricks into my future sites.

    One thing extra mine has is it re-evaluates things on resize, only really useful on desktops if you hit a screen in a small window then increase your window size. Hmmm as I’m typing this I’m thinking it would be good to think about browser zoom level too, apparently that’s possible to detect.

    0
  17. 31

    This method has a certain disadvantage. We look at an example:

    data-src-small = “img/320.jpg”
    data-src-medium = “img/640.jpg”
    data-src-high = “img/720.jpg”
    data-src-x-high = “img/720.jpg”>
    src = “img/320.jpg”

    The example must be created in advance 5 images. Some known frequency screens 240-320 is for 40% more screen sizes in the range of 1-5%. Consequently, in a month, perhaps, the majority of images are no longer needed. Unwanted images should be deleted. A more reasonable approach would be the automatic generation of the desired screen size and their subsequent removal after some time, as unnecessary.

    0
    • 32

      Yes, on Aftonbladet we had the setup that you describe: an image server that could scale images automatically and routines for deleting old unused images.

      0
  18. 33

    Instead of throwing away img { max-width: 100% } you could have used transparent SVG placeholder images…

    0
  19. 36

    I like this solution. However, users on high resolution screens are usually using mobile devices with 3G, 4G etc. There is a chance that their traffic is limited. Would be great to hear an idea how to handle this.

    0
  20. 37

    Are we not getting lost in resolving the reflow problem and not addressing the actual problem, which is serving the correct image quickly. The img tag always has the small.jpg associated with it. So this is being prefetched by the browser, so technically you are making two requests. An initial request for the small image then another one after the browser dimension is calculated.

    0

Leave a Comment

Yay! You've decided to leave a comment. That's fantastic! Please keep in mind that comments are moderated and rel="nofollow" is in use. So, please do not use a spammy keyword or a domain as your name, or else it will be deleted. Let's have a personal and meaningful conversation instead. Thanks for dropping by!

↑ Back to top