Menu Search
Jump to the content X X
Smashing Conf San Francisco

We use ad-blockers as well, you know. We gotta keep those servers running though. Did you know that we publish useful books and run friendly conferences — crafted for pros like yourself? E.g. upcoming SmashingConf San Francisco, dedicated to smart front-end techniques and design patterns.

Accessibility APIs: A Key To Web Accessibility

Web accessibility is about people. Successful web accessibility is about anticipating the different needs of all sorts of people, understanding your fellow web users and the different ways they consume information, empathizing with them and their sense of what is convenient and what frustratingly unnecessary barriers you could help them to avoid.

Armed with this understanding, accessibility becomes a cold, hard technical challenge. A firm grasp of the technology is paramount to making informed decisions about accessible design.

How do assistive technologies present a web application to make it accessible for their users? Where do they get the information they need? One of the keys is a technology known as the accessibility API (or accessibility application programming interface, to use its full formal title).

Reading The Screen Link

To understand the role of an accessibility API in making Web applications accessible, it helps to know a bit about how assistive technologies provide access to applications and how that has evolved over time.

A World of Text Link

With the text-based DOS operating system, the characters on the screen and the cursor position were held in a screen buffer in the computer’s memory. Assistive technologies could obtain this information by reading directly from the screen buffer or by intercepting signals being sent to a monitor. The information could then be manipulated — for example, magnified or converted into an alternative format such as synthetic speech.

Getting Graphic Link

The arrival of graphical interfaces such as OS/2, Mac OS and Windows meant that key information about what was on the screen could no longer be simply read from a buffer. Everything was now drawn on screen as a picture, including pictures of text. So, assistive technologies on those platforms had to find a new way to obtain information from the interface.

They dealt with this by intercepting the drawing calls sent to the graphics engine and using that information to create an alternate off-screen version of the interface. As applications made drawing calls through the graphics engine to draw text, carets, text highlights, drop-down windows and so on, information about the appearance of objects on the screen could be captured and stored in a database called an off-screen model. That model could be read by screen readers or used by screen magnifiers to zoom in on the user’s current point of focus within the interface. Rich Schwerdtfeger’s seminal 1991 article in Byte, “Making the GUI Talk1,” describes the then-emerging paradigm in detail.

Off-Screen Models Link

Recognizing the objects in this off-screen model was done through heuristic analysis. For example, the operating system might issue instructions to draw a rectangle on screen, with a border and some shapes inside it that represent text. A human might look at that object (in the context of other information on screen) and correctly deduce it is a button. The heuristics required for an assistive technology to make the same deduction are actually very complex, which causes some problems.

To inform a user about an object, an assistive technology would try to determine what the object is by looking for identifying information. For example, in a Windows application, the screen reader might present the Window Class name of an object. The assistive technology would also try to obtain information about the state of an object by the way it is drawn — for example, tracking highlighting might help deduce when an object has been selected. This works when an object’s role or state can easily be determined, but in many cases the relevant information is unclear, ambiguous or not available programmatically.

This reverse engineering of information is both fallible and restrictive. An assistive technology could implement support for a new feature only once it had been introduced into the operating system or application. An object might not convey useful information, and in any case it took some time to identify it, develop the heuristics needed to support it and then ship a new version of the screen reader. This created a delay between the introduction of new features and assistive technology’s ability to support it.

The off-screen model needs to shadow the graphics engine, but the engines don’t make this easy. The off-screen model has to independently calculate things like white-space management and alignment coordination, and errors would almost inevitably mount up. These errors could result in anomalies in the information conveyed to assistive technology users or in garbage buildup and memory leaks that lead to crashes.

Accessibility APIs Link

From the late 1990s, operating system accessibility APIs were introduced as a more reliable way to pass information to assistive technologies. Instead of applying complex heuristics to determine what an on-screen object might be, assistive technologies could query the accessibility API for specific information about each object. Authors could now provide the necessary information about an application in a form that they knew assistive technology would understand.

An accessibility API represents objects in a user interface, exposing information about each object within the application. Typically, there are several pieces of information for an object, including:

  • its role (for example, it might be a button, an application window or an image);
  • a name that identifies it within the interface (if there is a visible label like text on a button, this will typically be its name, but it could be encoded directly in the object);
  • its state or current condition (for example, a checkbox might currently be selected, partially selected or not selected).

The first platform accessibility API, Microsoft Active Accessibility (MSAA), was made available in a 1997 update to Windows 95. MSAA provided information about the role and state of objects and some of their properties. But it gave no access to things like text formatting, and the relationships between objects in the interface were difficult or impossible to determine.

In 1998, IBM and Sun Microsystems built a cross-platform accessibility API for Java. Java Swing 1.0 gave access to rich text information, relationships, tables, hyperlinks and more. The Java Jive screen reader, built on this platform, was the first time a screen reader’s information about the components of a user interface included role, state and associated properties, as well as rich text formatting details.

Notably, Java Jive was written by three developers in roughly five months; developing a screen reader through an off-screen model typically took several years.

Accessibility APIs Go Mainstream Link

In 2001 the Assistive Technology Service Provider Interface (AT-SPI) for Linux was released, based on the work done on Java, and in 2002 Apple included the NSAccessibility protocol with Mac OS X (10.2 Jaguar).

Meanwhile on Windows, the situation was getting complicated. Microsoft shipped the User Interface Automation (UIA) API as part of Windows 7, while IBM released IAccessible2 as an open standard for Windows and Linux, again evolved from the work done on Java.

Accessibility APIs existed for mobile platforms before touchscreen smartphones became dominant, but in 2009 Apple added the UI Accessibility API to iOS 3, and Android 1.6 (Donut) shipped with the Accessibility Framework.

By the beginning of 2015, Chrome OS stood out as the most mainstream platform lacking a standard accessibility API. But Google was beta testing its Automation API, intended to fill that gap in the platform.

Modern Accessibility APIs Link

In modern accessibility APIs, user interfaces are represented as a hierarchical tree. For example, an application window would contain several objects, the first of which might be a menu bar. The menu bar would contain a number of menus, each of which contains a number of menu items, and so on. The accessibility API describes an object’s relationship to other objects to provide context. For example, a radio button would probably be one “sibling” within a group.

Other features such as information about text formatting, applicable headers for content sections or table cells and things such as event notifications have all become commonplace in modern accessibility APIs.

Assistive technologies now make standard method calls to the operating system to get information about the objects on the screen. This is far more reliable, and far more efficient, than intercepting low-level operating system messages and trying to deconstruct them into something meaningful.

From The Web To The Accessibility API Link

In browsers, the platform accessibility API is used both to make information about the browser itself available to assistive technologies and to expose information about the currently rendered content.

Browsers typically support one or more of the available accessibility APIs for the platform they’re running on. For example, on Windows, Firefox, Chrome, Opera and Yandex support MSAA/IAccessible and IAccessible2, while Internet Explorer supports MSAA/IAccessible and UIAExpress. Safari and Chrome support NSAccessibility on OS X and UIAccessibility on iOS.

The browser uses the HTML DOM, along with further information derived from CSS, to generate an accessibility tree hierarchy of the content it is displaying, and it passes that information to the platform accessibility API. Information such as the role, name and state of each object in the content, as well as how it relates to other objects in the content, can then be queried by assistive technologies.

Let’s see how this works with some HTML:

<p><img src="mc.png" alt="My cat" longdesc="meeow.html">Rocks!</p>

We have an image, rendered as part of a paragraph. A browser exposes several pieces of information about the image to the accessibility API:

  1. It has a role of “image” (or “graphic” — details vary between platforms). This is implicitly determined from the fact that it is an HTML img element.
  2. Its name is “My cat”. For images, the name is typically derived from the alt attribute.
  3. A description is available on request, at the URL meeow.html (at the same “base” as the image).
  4. The parent is a paragraph element, with a role of “text.”
  5. The image has a “sibling” in the same container, the text node “Rocks!”

An assistive technology would query the accessibility API for this information, which it would present so the user can interact with it. For example, a screen reader might announce, “Graphic: My cat. Description available.”

(Does a cat picture need a full description? Perhaps not, but try explaining that to people who really want to tell you just how amazing and talented their feline friends actually are — or those of their readers who want to know all about what this cat looks like! Meanwhile, the philistines among us can ignore the extra information.)

Roles Link

Most HTML elements have what are called “roles,” which are a way of describing elements. If you are familiar with WAI-ARIA, you will be aware of the role attribute, which sets a role explicitly. Most elements already have implicit roles, however, which go along with the element type. For example:

  • <ul> and <ol> have “list” as implicit role,
  • <a> has “link” or “hyperlink” as implicit role,
  • <body> has “document” as implicit role.

These role mappings are being standardized and documented in the W3C’s “HTML Accessibility API Mappings2” specification.

Names Link

While roles are typically derived from the type of HTML element, the name (sometimes referred to as the “accessible name”) of an object often comes from one of several different sources. In the case of a form field, the name is usually taken from the label associated with the field:

<input type="radio" id="tequila" name="drinks" checked>
<label for="tequila">Reposado</label>

In this example, a button has the “radio button” role. Its accessible name will be “Reposado,” the text content of the label element. So, when a speech-recognition tool is instructed to “Click Radio button Reposado,” it can target the correct object within the interface.

The checked attribute indicates the state of the button, so that a screen reader can announce “Radio button Reposado Checked” or allow a user to navigate directly between the checked options in order to rapidly review a form that contains multiple sets of radio buttons.

Authors have an important role to play, providing the key information that assistive technologies need. If authors don’t do the “right thing,” assistive technologies must look in other places to try to get an accessible name — if there is no label, then a title or some text content might be near the radio button, or its relationship to other elements might help the user through context.

It is important to note that authors should not rely on an assistive technology’s ability to do this, because it is generally unreliable. It is a “repair” strategy that gives assistive technology users some chance of using a poorly authored page or website, such as the following:

<p>How good is reposado?<br>
<!--BAD CODE EXAMPLE: DON'T DO THIS-->
<input type="radio" id="fantastic" name="reposado" checked >
<label for="reposado">Fantastic</label><br>
<input type="radio" id="notBad" name="tequila"><br>
<input type="radio" id="meh" name="tequila" title="meh"> Meh

Faced with this case, a screen reader might provide information such as “second of three options,” based on information that the browser provides to the accessibility API about the form. Little else can be determined reliably from the code, though.

Nothing in the code associates the question with the set of radio buttons, and nothing informs the browser of what the accessible name for the first two buttons should be. The for and id attributes of the <label> and <input> for the first button do not share a common value, and nothing associates the nearby text content with the second button. The browser could use the title of the third button as an accessible name, but it duplicates the nearby text and unnecessarily bloats the code.

A well-authored version of this would use the fieldset element to group the radio buttons and use a legend element to associate the question with the group. Each of the buttons would also have a properly associated label.

<fieldset><legend>How good is reposado?</legend>
<!-- THIS IS A BETTER WAY TO CODE THE EXAMPLE -->
<input type="radio" id="fantastic" name="reposado" checked>
<label for="fantastic">Fantastic</label><br>
<input type="radio" id="notBad" name="reposado">
<label for="notBad">Not bad</label><br>
<input type="radio" id="meh" name="reposado">
<label for="meh">Meh</label><br>
</fieldset>

Making this information available through the accessibility API is more efficient and less prone to error than relying on assistive technologies to create an off-screen model or guess at the information they need.

Conclusion Link

Today’s technologies — operating systems, browsers and assistive technologies — work together to extract accessibility information from a web interface and appropriately present it to the user. If appropriate content semantics are not available, then assistive technologies will use old and unreliable techniques to make the interface usable.

The value of accessibility APIs is in allowing the operating system, browser and assistive technology to efficiently and reliably give users the information they need. It is now easy to make an interface developed with well-written HTML, CSS and JavaScript very accessible and usable for assistive technology users. A big part of accessibility is, therefore, an easily met responsibility of web developers: Know your job, use your tools well, and many pieces will fall into place as if by magic.

With thanks to Rich Schwerdtfeger, Steve Faulkner and Dominic Mazzoni.

(hp, al, ml)

Footnotes Link

  1. 1 http://www.paciellogroup.com/blog/2015/01/making-the-gui-talk-1991-by-rich-schwerdtfeger/
  2. 2 http://rawgit.com/w3c/aria/master/html-aam/html-aam.html

↑ Back to top Tweet itShare on Facebook

Advertisement

Léonie Watson is a Senior Accessibility Engineer with The Paciello Group (TPG) and owner of LJ Watson Consulting. Amongst other things she is a director of the British Computer Association of the Blind, a member of the W3C HTML and SVG working groups, and HTML Accessibility Task Force. In her spare time Léonie blogs on tink.uk, talks on the AccessTalk podcast, and loves cooking, dancing and drinking tequila (although not necessarily in that order).

Chaals McCathie Nevile is part of the CTO group at Russian internet giant Yandex and a member of various W3C groups including co-cordinator of the HTML accessibility Task Force. His first paid accessibility job would now be illegal use of child labour, and he has also worked as a barman, cleaner, and indulging the fantasies of people who wanted to learn how to program their video recorder. He immensely enjoys his job working out how to improve the way the Web helps people. He also enjoys cooking, travel, and reading books.

  1. 1

    Thanks for this article!

    In conclusion: use correct, standards compliant markup? :) Would that sum it up?

    2
    • 2

      Using correct, standards complaint markup is a good start…

      In order to get there, you have to understand the assitive, browser, OS technology well and how they interact. Most importantly it’s about having a fundamental understanding of user needs. Once you have all that, the web and it’s markup make sense.

      Fantastic article!

      5
    • 3

      Léonie Watson

      March 17, 2015 4:05 pm

      “In conclusion: use correct, standards compliant markup? :) Would that sum it up?”

      For accessibility at the code-level, semantic markup is definitely a good thing. It’s also a good thing for developers too, since it means the browser is doing some of the accessibility work (so you don’t have to do so much).

      3
      • 4

        Taliesin L. Smith

        March 18, 2015 4:50 am

        And “standards compliant markup” or “code-level, semantic markup” is good for designers as well as it provides clear hooks for the CSS.

        0
  2. 5

    Thanks for the article.

    According to me, use correct, standards compliant markup is the basics for any web developper.
    It is a very good start.
    But to go further, take a look a the WCAG (http://www.w3.org/WAI/intro/wcag).

    0
  3. 6

    Hi Léonie & Chaals,

    Thanks for the detailed background. It really helps highlight the need for the API approach. It’s already challenging enough for web developers to come to grips with how they should be making their sites accessible, since most of us aren’t users of screen readers, and having multiple competing screen reader models is a strong disincentive to do good testing.

    Thanks,
    John

    1
  4. 7

    Yahoo Forgot Password Help

    March 17, 2015 1:33 pm

    Nice blog and thanks for share your a good piece of content with us.

    -2
  5. 8

    Great article.

    I’m always banging on at work about web accessibility – start with clean semantic markup (which will benefit SEO too!), then progressively enhance with ARIA, etc, where needed.

    Don’t just use “display:none” / “display:block” to hide/show content without thinking how an AT would “see” that content.

    And don’t forget users who can’t use a mouse (so they’re restricted to keyboard or some other pointing device), or who may not be able to differentiate certain colours, or who may have cognitive issues.

    Remember… 1 day we’ll get old, and we may develop accessibility issues ourselves. So code for the “future you”.

    :)

    6
  6. 9

    @Koj, @John, @Basher,

    thanks for the encouragment, and for the sound comments. As we began, there is a lot of technical challenge here, so the more we can make the important knowledge part of people’s “reflexes”, the more they can focus on the getting the harder stuff right too…

    0
  7. 10

    I dont think accessibility is something EVERY website should care about, its a very specific demographic that very few websites should take into account.

    -65
    • 11

      Patrick H. Lauke

      March 17, 2015 11:22 pm

      “I dont think accessibility is something EVERY website should care about, its a very specific demographic that very few websites should take into account.”

      I don’t think EVERY shop should serve people of colour…or women…or the irish…

      26
    • 12

      Taliesin L. Smith

      March 18, 2015 4:40 am

      Hood, accessibility is not only the right thing to do, morally. It is good for business. People with physical disabilities are the largest minority group in the U.S. And that’s just one category. We are not getting any younger and as we age we, well, develop challenges. Please see more stats here: http://www.disabled-world.com/disability/statistics/

      2
    • 13

      @HOOD
      Accessibility is not “for a specific demographic”; there are people with disabilities in EVERY demographic.

      2
    • 14

      Lynn Holdsworth

      March 20, 2015 9:48 am

      For me this translates to “so you’re blind. So clear off and visit some site created for special people”. I fall into that “specific demographic” you speak of. I’m also a wife, mother, business owner, tax-payer, shopper, book reader, transport user and web developer, and my right to use any website I want or need to is enshrined in the laws of most civilised countries. Accessible websites change people’s lives immeasurably for the better. Not having to ask someone to take me shopping or read me the newspaper is incredibly empowering. Are you unaware, lacy or just cold?

      14
    • 15

      Hi @HOOD,

      Actually, it isn’t a very specific demographic, since it cuts across more or less *all* populations. From driver training to sky-dive bookings, from military procedures to small shops, and by way of some things you may not expect like “adult” services and video systems, there are many reasons (in the form of individuals who want to buy products or need to use services) why accessibility is basically part of the requirements in *any* market.

      There are a couple of secondary reasons, too.

      One, often noted, is that search engines are critical to today’s web, and they take accessibility into account in various ways (our detailed algorithms are secret, but we’ll tell you for free that there are a number of accessibility requirements that you also need to meet for a search engine).

      Another is that given life’s unpredictability, even a small intimate network of people recording things for themselves won’t be making a mistake by future-proofing against the possibility of accidents, aging, and other common causes of disability. Imagine storing important data such as your passwords, only to discover that when you need access to the bank you can only do it by asking someone else. It seems more obvious that services which manage passwords for a mass market need to handle accessibility.

      There is a more interesting discussion about how far “mainstream” products should provide specific solutions, and how much should be done by specialised tools.

      Indeed, the accessibility APIs that are the core subject of this article are developed and shipped inside mainstream products (your OS) in order to allow specialised tools (screenreaders, magnifiers, voice control systems, etc) to be built on top of them more efficiently.

      But the trend, as with products in general, is for things that begin as specialist niche products to become cheaper, more efficient, and actually more useful as they are integrated across more mainstream systems. 20 years ago the speech synthesis that screen readers used was provided by an expensive special piece of hardware, but 15 years ago that had changed, bringing the cost down astronomically. Bad news for people whose business didn’t adapt, but good news for millions of actual users – including those whose need for speech output couldn’t justify the extra hardware but who benefit significantly from the fact that some level of speech output is now basically free on any system – and the developers who have used that to enhance products for many different markets from kid’s toys to military systems.

      (I’ll stop before I write a whole new article into the comments…)

      3
  8. 16

    Michael Spellacy

    March 18, 2015 2:22 pm

    Really, Hood? Really?

    0
  9. 17

    gopinath srinivasan

    March 19, 2015 1:41 pm

    Nice article for this Web API! it’s interested and easy way to learn!!!!

    0
  10. 18

    Excellent post! Thanks

    For completeness you could mention that the APIs also allow AT to perform actions on the objects and also receive events about status changes. Indeed this makes them ideal for test automation, a direction Microsoft seemed keen on based on the name the chose for their 2nd attempt and how they promoted it.

    A point of information – IAccessible2 was not really available for Linux (though the Linux foundation manage the spec). Rather it attempted to harmonize Linux (GNOME) AT-SPI and windows, largely as MSAA was so feature poor.

    0
    • 19

      Léonie Watson

      March 20, 2015 7:45 pm

      @gopinath @Steve

      Thanks both, and in particular to Steve for the additional useful information. Good to know.

      0
  11. 20

    @HOOD – hmm, at some point you’ll develop age related disabilities or have a mishap (I hope not). I suspect you’ll appreciate it then. :)

    0
  12. 21

    MobilePundits

    March 23, 2015 7:41 am

    This is what we can say motivation for the people who have eyes or who are blind. Whatever strategy is chosen for making the testing easier, team has to be aware that altering APIs and providing details about the processing may also be a security breach. Therefore, these hooks and additions need to be either hidden and protected or simply stripped from the builds that are promoted to production environments.

    0
    • 22

      Lynn Holdsworth

      March 23, 2015 1:04 pm

      Rather than making changes to API’s, the idea is to make full use of the functionality already offered in the accessibility API’s of most browsers. This should have no effect whatsoever on security.

      0
    • 23

      Léonie Watson

      March 23, 2015 2:04 pm

      @MobilePundits
      What makes you think the platform accessibility APIs might be a security breach? Can you share some examples or more information?

      4
    • 24

      Hi @MobilePundits,

      to be honest, I am not sure we understood your point, and it would be helpful if you have time to help us make sure we do.

      What I think you are pointing out is that knowing a lot about what a user does through the accessibility API, or giving web apps the ability to manipulate the content of the accessibility API, can introduce risks to the user.

      Assuming I got that right: In the first case there is a less direct risk because it only actually enables tracking of the user. However this in turn does indeed pose problems, especially because people with disabilities are sometimes more vulnerable to “evil” uses of tracking.

      In the second case, of manipulating what goes into the API, the risk is clearer because it enables things like more sophisticated phishing attacks. Currently the API is not available to web apps in any real way, although they can influence what is presented through the API by using ARIA (I’d like to write about this at length soon). There are moves to provide more direct access, for the good things it enables, but these concerns will have to be balanced. In particular, there is an ongoing discussion among the relevant technical communities about the question of trust and security in the context of providing information about users.

      If that doesn’t answer the point, please help us to understand it better

      0
  13. 25

    Raghavendra Satish Peri

    May 12, 2015 11:57 am

    A detailed & wonderful article..I never thought so much goes into accessibility.

    0

↑ Back to top