Responsive images have been around long enough for most of us to have taken them for a spin, or at least to have learned from the experiences of those who have. Beyond doubt, the responsive images1 specification is a great win for the web. However, quite a few reports from the front lines suggest that responsive images can become pretty ugly.
Further Reading on SmashingMag:
- Preload: What Is It Good For?2
- Responsive Images Done Right: A Guide To <picture> And srcset3
- Introducing The Responsive Image Breakpoints Generator4
- Why Perceived Performance Matters5
This article will not demonstrate the challenges of responsive images. Several other sources have already6 done7 that8. Instead, we’ll focus on how to address the issues, with a little help from the web server and a new way for the browser, or client, to request images with specific properties. Even if this is called “Client Hints”, it is pretty specific. Let’s dive in!
What Are Client Hints? Link
Client Hints is a new feature already available with Chrome 4610 and Opera 33. More browser vendors are following. It is an initiative by Google11 (spearheaded by Ilya Grigorik), and, as its progress indicates, it is likely to be “a thing”. The initiative was also recently adopted by the HTTP Working Group12.
You can think of Client Hints as the missing link between the browser and the server when it comes to layout information. Instead of specifying every possible image breakpoint, pixel density and format in your responsive images markup, Client Hints append the current setting to the HTTP request, allowing the web server to pick the perfect fit — also known as content negotiation13.
The current way of dealing with responsive images is typically to specify different image sources based on the receiving device’s pixel density, preferred image format and viewport size. If you go through the process of picking breakpoints and formats, your markup might end up something like this:
I think many of us agree with Brad on this.
To be fair, the example above include different crops of the image too. Which is something Client Hints alone can’t help you with. Still, crafting markup like this is not something a developer or designer should be spending their time on17. We need an automated process. Even if the server can generate dynamic markup automatically, client hints decouples the markup from the image which makes it easier to perform operations on the image without having to worry too much about the markup.
Let The Web Server Know Link
Imagine if the web server knew the pixel density, the viewport size and also the actual size and format of an image. This is what Client Hints does! Browsers that support Client Hints add a few HTTP headers to requests. The latest draft18 mentions these hints:
This stands for “device pixel ratio,” the ratio of physical pixels on the screen to CSS pixels.
This is the width of the viewport in CSS pixels. (CSS pixels means the units used in CSS to describe a layout. A CSS width of 100 pixels would be 200 device pixels (DP) if the device pixel ratio (DPR) was 2.)
This is the actual width of an image in real physical pixels (similar to the
wdescriptor made famous by responsive images).
This is the client’s maximum download speed.
This boolean indicates whether extra measures should be taken to reduce the payload.
Save-Data are not available in Chrome just yet, but you can imagine their purpose. Let’s focus on the currently available hints. First, we have to tell the browser to send the hints.
Enabling Client Hints In Your HTML Link
You’ll have to opt in to enable Client Hints. The reason for this is that additional data shouldn’t be added to a request unless it is used for something. If it’s not used, it would work against the very purpose by adding data to the payload. Web servers can also advertise their support by adding an HTTP header to the HTML response which lists the hints accepted by the server:
Accept-CH: DPR, Width, Viewport-Width
If adding an HTTP header is not an option, you can also add this meta tag inside the
<head> element in your markup:
<meta http-equiv="Accept-CH" content="DPR,Width,Viewport-Width">
That’s all we need. The browser will now append the
Width, which, in practice, applies only to images).
Aside from images, Client Hints might be useful if the CSS file contains breakpoints based on the viewport size or device pixel ratio. Knowing the viewport size before the CSS is returned to the browser, the server can strip unqualified blocks from the CSS file before sending the response. That is another story. For now, let’s look at an example with images.
Our Old Friend,
Imagine we have a page with the image tag below. Let’s display the
flower.jpg image at a width of 200 CSS pixels.
<img src="flower.jpg" width="200">
With Client Hints enabled, the browser will send the following request to the server:
The browser “hints” to the server that the requesting device has a pixel ratio of 2. As a bonus, we get the size of the viewport, too, since this is known by the browser at the time when the request is being made. Because the DPR is 2 and our page’s design need this image to be 200 pixels wide, we would like to serve an image that is 400 actual pixels wide (200 pixels × 2).
But we’re missing information about the intended display size, even if the
img tag says
width="200". The specification explains23 that the
sizes attribute has to be set in order for the
Width header to be sent. In the near future, the
width attribute on the image element is likely to be included in the algorithm too, but for now, we’ll have to stick with
sizes attribute describes the layout and display size of the image. In the beginning, it might be less intimidating to think of it like the good old
width attribute or like a CSS property:
<img src="flower.jpg" sizes="200px">
<img src="flower.jpg" sizes="25vw">
Now, the request for
flowers.jpg will look like this:
The browser calculates the intended display size of the image based on the current size of the viewport and the device pixel ratio of the device in time for the preloader to fetch it. In the example above, the viewport (
Viewport-Width) is 774 pixels wide . The
<img> tag specifies that the image should cover 25% of the viewport size: 193.5 CSS pixels.
Because it’s a high density display, with a pixel ratio (
DPR) of two, we multiply the CSS pixels by the pixel ratio and arrive at 387 actual pixels (
Width). You might recognize this from how the selection process work with “regular” markup for responsive images with multiple image sources listed. The difference here is that the information is appended to the HTTP request, rather than an image source being picked from the
Mind. Blown. Link
What just happened? Basically, we boiled our verbose responsive images tag down to something that looks very familiar, but with the same responsive functionality. With the pixel ratio and width information, the server can now pick, or generate, the appropriate size of the requested image.
DPRheader takes care of resolution switching. We don’t need
xdescriptors in our markup.
Widthheader tells the server the exact width of the image (relative to the viewport) that the browser need to fit in the layout.
Actually, we don’t need
srcset at all. The
sizes attribute is our new hero! Based on the relative value of
sizes, the browser translates this into the actual size in physical pixels. Also, remember that media conditions can be applied if you want the image to have different sizes as the width of the viewport changes:
<img src="flower.jpg" sizes="(min-width: 30em) 100vw, 50vw">
But what about
type? Responsive images allows you to define different formats, or mime types, of an image by using the
Client Hints does not cover this, but its older brother, the
Accept header, may contain valuable information.
This example is from Chrome, which is nice enough to tell us that WebP28 is preferred. Other browsers might just say
*/* which basically means, “Send me anything.” In those cases, you could apply your own rules, or, even better, the server could implement more advanced device intelligence solutions to decide the best image format to return to the client.
The Server Side of Client Hints Link
We could say that, with Client Hints, we’re moving the responsibility of selecting an image source from the browser to the server. This means, of course, that we need to implement some logic to act on the Client Hints server-side.
The bonus of involving the server is that, rather than just selecting the best match from a list of pre-generated image files like the browser does, the server can generate the perfect fit on the fly! On a small scale, this is quite achievable because we have all the information we need in the HTTP header.
However, if this task is a bit daunting and if performance is a priority, a few web services — image proxies, if you will — already support Client Hints. One of them is the free ImageEngine29. Using ImageEngine, for example, we first have to “prefix” our images with the URL of the service.
If your image
src’s URL is
http://example.com/image.jpg, then we’d have to change the
[key] is the personal token you get when you sign up. As long as the meta tag is present and that
sizes attribute is present in the image tags, we’re good to go. Looking at the response with cURL, we can see how the server responds:
$ curl -I http://try.imgeng.in/http://web.wurfl.io/assets/sunsetbeach.jpg -H "DPR: 2" -H "Width: 150" -H "Viewport-Width: 800" HTTP/1.1 200 OK Content-Type: image/jpeg Vary: Width Content-DPR: 2 …
The request has a
DPR of 2, a
Width of 150 pixels and a
Viewport-Width of 800. The server then responds with the
Content-DPR header, whose purpose is to confirm to the browser30 what the pixel ratio is of the returned image, so that the browser can fit it on the page correctly.
In the example above, the
Content-DPR will always be the same as the
DPR request header, because ImageEngine is scaling the inputted image to the exact value of
Width. As a bonus, even if
Width is not set, ImageEngine will fall back to
Viewport-Width and then to screen size data from WURFL, a database of devices.
If you implement the server yourself, and you chose to mimic browser behavior by selecting the closest match from a set of pre-generated image sources, then the
Content-DPR header might be a different number than the
DPR hint from the client. The browser will make use of
Content-DPR to scale the image to its display dimensions.
Also worth noting is the
Vary header. The purpose of this header is to tell the client (the browser or proxy) that the response from this URI will vary according to the value of the
Width header. This enables web proxies and content delivery networks to cache the image better — at least compared to caching based on the
Non-Supporting Browsers Link
Before you run off and start implementing support for Client Hints, you should know that not all browsers support them. The last
<img> tag above will likely mess up a page viewed in a browser that does not support Client Hints. So, what are our options?
Until support for Client Hints has reached critical mass, the safest approach is to combine hints with explicit – but relative – widths and heights to make sure the layout doesn’t break. If the browser doesn’t send Client Hints, but the image server expects it, you need the layout to handle an oversized image if that is the default behavior of the image server. Moreover, to reduce the risk of serving too big an image to a non-supporting browser, an image optimization service is recommended, as described above. ImageEngine is useful because it handles mobile devices fairly well (although I’m biased here!). If a mobile device does not support Client Hints, then ImageEngine will never serve an image wider than that particular device’s screen.
Aside from automation, the motivation for implementing Client Hints is, of course, image performance. It’s difficult to compose a fair test case, but to give an idea of how a Client Hints-based image request performs differently than a “regular” responsive images request, I’ve put together a little demo33 with the two scenarios. Below is a table showing the bytes transferred and the actual size of the image put on the wire at different viewports. The selected image breakpoints and viewport sizes in the demo are arbitrary.
Responsive images w/srcsetClient Hints with server scaling
|Viewport width||KBytes (w/srcset)||Actual width (pixels, w/srcset)||KBytes (w/client hints)||Actual width (pixels, w/client hints)|
While the preselected image breakpoints span portions of the viewport’s size, Client Hints, along with an image server capable of resizing images, addresses the continuum of viewport sizes. With Client Hints, we have surgical precision. On average, the Client Hints approach serves 19% less data. If we exclude the 1400-pixel viewport, to which the responsive images approach serves too small an image (1024 pixels), then the data served is 32% less with Client Hints, which is substantial.
The data sample in the chart above is too small to draw any conclusions, but it illustrates the purpose of Client Hints well. No surprises there. Worth noting, though, is the use of DNS prefetching37 for the external host name
try.imgeng.in. Referring to the table above, the time it takes to download the bytes is as expected. Client Hints works as advertised.
(Almost) Ready For Prime Time Link
While responsive images (
<img>) element with
sizes) implemented in markup only allows the browser to pick the closest match from a list of image resources, Client Hints enable the web server to serve an image tailored to the browser’s needs, which in most cases mean less data and better image quality. It does require some programming if you’re implementing server-side support by yourself; luckily, some content delivery networks and web services already support it.
The future of Client Hints looks promising. Adding hints about the user’s connection and instructions to reduce data traffic are on the way38. With this information, the server can increase image compression, avoid serving high-density images or reduce the byte size of images in other ways.
As developers, we have no reason not to start exploring Client Hints right now. The key benefits are less verbose and more maintainable responsive image tags, fewer image bytes transferred and, ultimately, happier end users. Only a few steps are needed from you:
- add the meta tag to the
- put the
sizesattribute in your image tags.
Then make, or pick, your image server. I’ve mentioned the free ImageEngine39 optimization service above, which supports Client Hints. The best way to find other services that support Client Hints is to stay tuned and search on Google40 (duh!), because this is a new thing and more providers are announcing support as we speak.
If you want to implement support for Client Hints yourself, the article “Efficient Image Resizing With ImageMagick41” is a great start. Also, the open-source imaging service Thumbor is also considering Client Hints42.
- 1 https://www.smashingmagazine.com/2014/05/responsive-images-done-right-guide-picture-srcset/
- 2 https://www.smashingmagazine.com/2016/02/preload-what-is-it-good-for/
- 3 https://www.smashingmagazine.com/2014/05/responsive-images-done-right-guide-picture-srcset/
- 4 https://www.smashingmagazine.com/2016/01/responsive-image-breakpoints-generation/
- 5 https://www.smashingmagazine.com/2015/09/why-performance-matters-the-perception-of-time/
- 6 http://www.scientiamobile.com/page/responsive-images-specification-real-world-scenarios
- 7 http://codepen.io/Tigt/post/when-responsive-images-get-ugly
- 8 https://dev.opera.com/articles/responsive-images/
- 9 https://www.smashingmagazine.com/2014/05/responsive-images-done-right-guide-picture-srcset/
- 10 https://youtu.be/X1F8GEiZf9o?t=122
- 11 https://github.com/igrigorik/http-client-hints
- 12 https://tools.ietf.org/html/draft-ietf-httpbis-client-hints-00
- 13 https://en.wikipedia.org/wiki/Content_negotiation
- 14 https://twitter.com/brad_frost/status/599676745176997889
- 15 http://alistapart.com/article/responsive-images-in-practice
- 16 https://twitter.com/brad_frost/status/599676745176997889
- 17 http://blog.cloudfour.com/responsive-images-101-part-9-image-breakpoints/
- 18 http://igrigorik.github.io/http-client-hints/
- 19 https://www.smashingmagazine.com/wp-content/uploads/2015/12/02-clienthints101-opt.png
- 20 https://www.smashingmagazine.com/wp-content/uploads/2015/12/02-clienthints101-opt.png
- 21 https://www.smashingmagazine.com/wp-content/uploads/2015/12/03-Headers1-opt.png
- 22 https://www.smashingmagazine.com/wp-content/uploads/2015/12/03-Headers1-opt.png
- 23 https://github.com/igrigorik/http-client-hints/blob/master/browser_implementation_considerations.md#width
- 24 http://www.w3.org/TR/css3-values/#relative-lengths
- 25 https://www.smashingmagazine.com/wp-content/uploads/2016/01/04-Headers2-opt.png
- 26 https://www.smashingmagazine.com/wp-content/uploads/2016/01/04-Headers2-opt.png
- 27 http://blog.cloudfour.com/responsive-images-101-part-5-sizes/
- 28 https://www.smashingmagazine.com/2015/10/webp-images-and-performance/
- 29 http://web.wurfl.io/#image-engine
- 30 http://igrigorik.github.io/http-client-hints/#confirming-selected-dpr
- 31 https://remysharp.com/2010/10/08/what-is-a-polyfill
- 32 http://www.stevesouders.com/blog/2013/04/26/i/
- 33 http://wurfl.github.io/smdemo.html
- 34 https://www.smashingmagazine.com/wp-content/uploads/2015/12/05-wptsmdemox-opt.png
- 35 http://www.webpagetest.org/result/151117_DY_HEF/
- 36 https://www.smashingmagazine.com/wp-content/uploads/2015/12/05-wptsmdemox-opt.png
- 37 https://developer.mozilla.org/en-US/docs/Web/HTTP/Controlling_DNS_prefetching
- 38 http://igrigorik.github.io/http-client-hints/#the-downlink-client-hint
- 39 https://web.wurfl.io/#image-engine
- 40 https://www.google.com/webhp?q=image+optimizing+client+hints&gws_rd=cr&ei=hWtUVr3FGIWlsgHzuo_wAg#
- 41 https://www.smashingmagazine.com/2015/06/efficient-image-resizing-with-imagemagick/
- 42 https://github.com/thumbor/thumbor/issues/549