Today’s Rubicon Labs report from the trenches deals with the importance of doing proper usability testing and taking ownership of your digital projects.
One of our corporate clients has recently asked us to do an AODA compliance assessment on an AngularJS/Ruby web application they were about to release to their customers. We were pretty shocked to find out that the near-final product had a number of serious flaws, including
- non-semantic HTML used throughout, practically every single element was a generic <div>, including headings, links and form elements
- color contrast ratio not meeting the WCAG 2.0 standards in several areas
- general lack of logical structure/flow to the application interfaces and interactions
The latter caught our special attention since the project manager insisted they performed extensive usability testing before making decisions on the interface and interaction design. We were given the videos of the user testing sessions, only to discover that the methodology they applied was completely wrongheaded.
In essence, they let their users pick and choose designs instead of observing their interactions with the interfaces. Their idea of “A/B testing” was to show a user 2 or 3 interface options, ask them which one they liked the most, and then make the design decisions based on user feedback. The development was outsourced to an external vendor with very little communication taking place between the initial requirements briefings and the final product delivery.
It was not the most pleasant of conversations, but we had to communicate to the executives about the lack of UX design expertise on their team as well as inability of the application to meet even the most basic accessibility requirements. With hundreds of thousands of dollars already invested and with vendor demanding extra fees to fix accessibility issues (despite AODA compliance present in the submitted requirements doc), our client is seriously considering taking a legal action against the vendor.
There is a couple of things that can be learned from this story…
First, it is the obvious lack of expertise on the client’s team and their wrong approach to usability testing. The first rule of usability is… Don’t listen to your users! More often than not, users do not know what they want. Instead, observe their interactions with the interface and measure the efficiency with which the tasks are performed.
You don’t need to code the interface either before running tests on it. Paper designs with one team member performing a role of a human computer work great! Here’s one example of such tests in action:
Second, it is extremely important to stay in charge of your projects. If the work is being outsourced to an external vendor, a digital lead should be assigned to monitor progress and maintain constant communication with the vendor, ensuring the project is progressing as planned and all the requirements are being met.
While it was a very expensive lesson for our client, it was encouraging to hear them acknowledge the issues we raised and to commit to applying our recommendations to avoid similar situations in the future.