Creating a Universal Usability Agenda

By Whitney Quesenbery

Published: November 6, 2006

“HAVA called for improved standards for voting systems and required that they allow individuals with disabilities to vote “in a manner that provides the same opportunity for access and participation (including privacy and independence) as for other voters.”

Disclaimer: The views in this article are those of the author and do not necessarily represent the views of the Elections Assistance Commission or the National Institute of Standards and Technology.

How do you keep usability, accessibility, and user experience requirements on track while developing standards? It is part of the very nature of standards to focus on details—and in the process, to sometimes lose sight of the real goals. This is especially true when a standards-making process goes on for a long time, a situation is highly political, or most people are focused on technology issues. For over two years, I’ve worked in just such a situation as part of the Technical Guidelines Development Committee (TGDC) creating federal standards for voting systems in the United States.

The Help America Vote Act of 2002 (HAVA) mandated the TGDC and its members—who include election officials, members of the US Access Board, and other experts, working with scientists from the National Institute of Standards and Technology (NIST). The TGDC is an advisory committee, so we don’t actually create regulations or standards, but offer our advice to the Elections Assistance Commission (EAC).

For most people, the mention of voting systems conjures up one of two thoughts: either the hanging chads of a terrible usability disaster during the 2000 presidential election or the need for paper audit trails and the importance of security in voting systems. With everything that people have written and said about security, transparency, trust, and the necessity of our being able to accurately recount ballots, it’s easy for something like usability to seem like a trivial “nice to have.”

But it does matter, and HAVA called for improved standards for voting systems and required that they allow individuals with disabilities to vote “in a manner that provides the same opportunity for access and participation (including privacy and independence) as for other voters.” (HAVA 301(a)(3)). The subcommittee on Human Factors and Privacy is responsible for drafting guidelines for usability and accessibility, according to the provisions of this law.

“Standards are all about details, and it’s only too easy to lose sight of the big picture while writing and debating specific requirements.”

This article looks at how we created an agenda to guide our work, including decisions about how to create the new requirements. Our approach is also applicable in other situations, such as creating usability guidelines for a product or focusing corporate attention on user experience.

Standards are all about details, and it’s only too easy to lose sight of the big picture while writing and debating specific requirements. A committee of appointed members not only brings together different perspectives, but means there will likely be process, political, and administrative overhead. Sound familiar?

To combat this entropy, we created a set of guiding principles and voted to adopt them as official resolutions. Viewed as a whole, the following resolutions created an agenda for a universal approach to the usability and accessibility of voting systems:

  • By agreeing on basic principles in advance, the committee could focus on the details of a complex standard with a shared understanding of its goals.
  • By defining relationships and important dependencies at the beginning of the project, longer-term work could start immediately, so it would be ready when needed.
  • The resolutions could span the development of several versions of the standard.

These resolutions provide both a high-level view of our usability and accessibility goals, as well as specific directives for how to organize the work. This was important for a committee that included many stakeholders, not just user experience and human factors experts. Most importantly, the resolutions define the full scope of the project, and we can use them as a measure of success for the completed standard.

The Human Factors and Privacy Resolutions

Accessible Voting Systems—The first resolution acknowledged the HAVA requirement for accessible voting systems and called for the guidelines to both incorporate the latest available accessible technology and draw on existing regulations for accessibility. This allowed us to build on the work of the Access Board, W3C, and other international standards bodies.

Human Factors and Privacy Requirements for Capturing Indication of a Voter’s Choice—This companion to the first resolution focused on how more usable systems could support voters’ ability to be accurate and efficient in casting their ballots.

Human Factors and Privacy of Voting Systems at the Polling Place—US elections use a secret ballot, and this resolution made the point that usability, accessibility, and privacy are functions of both the system we use to vote and the environment of the polling place. Although our guidelines cover only the system itself, we believed there was an opportunity to provide guidance on good election procedures to support the equipment requirements.

Human Performance-Based Standards and Usability Testing—This was one of the most forward-looking resolutions. It called for standards for performance that are measurable with usability testing. Performance standards allow innovative solutions to the challenges of creating a voting system and avoid narrow design requirements, while still requiring systems to be usable and accessible.

Accommodating a Wide Range of Human Abilities—The voting population includes not only people with specifically identified disabilities, but also an aging population, language minorities, and people with other special needs. This resolution set out the principle that all voting systems—not just so-called accessible systems—should accommodate a wide range of abilities. This universal usability approach to the guidelines will make voting systems more usable for all.

Usability Guidance for Instructions, Ballot Design, and Error Messages—Instructions for voters and poll workers, ballot design, error messages, and Help are important components of any system, so this resolution called for improving the usability of instructions, error messages, and Help through plain language and good ballot design, in all necessary formats.

General Voting System Human Factors and Privacy Considerations—This resolution ensured that the committee would consider the usability and accessibility of all aspects of the system, including security features and other functional requirements, and review all guidelines for their implications for human factors.

Usability of the Standards—Anyone who has read a standard knows that they are often written in arcane language. We wanted to express the requirements in clear, plain language that is understandable to experts in usability testing, voting officials—who may not be experts in human factors—and advocates who are interested in voting systems.

Availability of Voting Machines for Validating Benchmarks and Conformance Test Protocols—The last resolution covered important housekeeping. Creating a standard usability test meant that the staff needed access to a variety of real voting systems. This resolution enabled NIST to set up a program under which vendors could make their systems available to the team developing and testing these guidelines.

How Well Did It Work?

No standard is perfect—especially one that is created in the political hot-house environment of special interests in elections and voting systems. The standard, called the Voluntary Voting Systems Guidelines (VVSG2005) includes the first usability requirements for voting systems and enhanced accessibility requirements for voters with disabilities. There are specific requirements covering partial vision, blindness, dexterity, mobility, hearing, and cognition. See the sidebar for a timeline for the creation of the VVSG2005.

During the process of drafting the VVSG2005 and an update that is currently in progress, the resolutions have allowed us to keep the usability and accessibility work on track:

  • We’ve used the resolution on universal usability to support our approach of creating general usability guidelines that make all systems as accessible as possible.
  • The resolution on performance guidelines provides the basis for creating ground-breaking benchmarks and has enabled us to start the research and pilot testing necessary to write performance-based requirements and create a repeatable test for conformance.
  • Our insistence that the standards themselves be usable has resulted in a prolonged tug-of-war as we have tried to both write the individual requirements clearly and structure the whole document in a way that reduces its complexity and the number of nested levels.

One of the most interesting things that happened occurred during the comment and revision process: Some of our guidelines were upgraded. For example, the draft version only recommended usability testing as part of the certification testing. However, public comments—such as one from Senator Christopher Dodd, of Connecticut, who called for “requiring … not merely encouraging them, to conduct summative usability tests”—supported the idea strongly enough that the final version of the standards requires usability testing—both with the general public and people with disabilities. It may be the only federal regulation to do so.

In the end, the principles embodied in the resolutions helped keep the committee focused on the goals for the voting system standards, but also an approach to meeting those goals.


Human Factors and Privacy subcommittee members—Whitney Quesenbery (Chair), Philip Pearce (US Access Board), John Gale (Secretary of State, Nebraska), Alice Miller (Director of Elections-District of Columbia), and Sharon Turner Buie (Director of Elections-Kansas City, Missouri)

NIST staff for Human Factors and Privacy—Sharon Laskowski and John Cugini

Past members include James Elekes (US Access Board) and Donetta Davidson (Secretary of State, Colorado), now on the Elections Assistance Commission.


If you would like to learn more about the effort to create federal standards for usable, accessible voting systems in the United States, here are some online resources:


The fact that you got a senator to utter—and presumably understand—the phrase “summative usability testing” is a major accomplishment.

But seriously, good work and thanks for sharing this information.—Paul Sherman

This sort of consensus building approach—“creating…guiding principles”—applies to more than just voting. In fact, I’ve shared this among my colleagues as an example of a great strategy for many user experience groups.

Paul—I think that work and discussions around voting systems have done a lot to help people understand the importance of the user experience. If all the pieces don’t fit together, elections don’t work. And they have to.

Thanks for the support. Writing standards is a slow process, so we savor any successes.—Whitney

J—Thanks!! I also thought that this strategy might work well in other situations, especially in large organizations for corporate standards—for UX, branding, usability, accessibility…. It’s so easy to get pulled off into the details, so having the resolutions to keep us on track was invaluable.—Whitney

Bravo, Whitney! Your comprehensive approach to universal usability and application to voting is an inspiration. It’s great to read about such successes.

Join the Discussion

Asterisks (*) indicate required information.