Usability testing

(Previously posted and amended from LJMU Digital Services Blog)

Over the years I and teams I work with have carried out quite a few usability tests. Whilst those close to user research understand the benefits of usability testing, to a lot of people they are something new, something that on paper doesn’t seem to be of value. So I’d thought I would write a post to explain them, the reasons we do them and a whole batch of common questions we get. Expect this post to grow over time!

So what is a usability test?

From government digital services:

User testing is a ‘qualitative’ research method, used to gauge how easy and intuitive a (product, service, website) is to use and whether it supports the needs of its intended audience. Combined with more ‘qualitative’ techniques it helps us understand the users’ motivations and attitudes as well.

Why do we test?

Simply put, we test to understand, learn about and improve upon our digital services. As we are driven by user needs, we need to test constantly to validate against these needs and to uncover new needs.

What do we test?

We test our digital services regardless of its status in the design cycle. So this can be as varied as wireframes and sketches, to prototypes or live services.

When do we test?

All the time. We try for one test a week; but often it’s once a fortnight which fits in with our 4 week sprints, so testing a development twice before it goes live.

Where do we test?

Often we test in a lab based environment (our own DIY lab) using a test room and an observation room. Sometimes we test users in their own environment remotely using video conferencing technology or we go outside and test in pop-up environments (cafe’s, student spaces, libraries…). Recruitment of your target audience isn’t an easy process so we need to be flexible in location.

Why not focus groups?

Focus groups…you get the answers you want to hear, not the natural, raw and emotive feedback we need to hear

Whilst we do utilise workshops, we don’t run focus groups. Focus groups simply don’t uncover user insights. Because they are led activities, you get the answers you want to hear, not the natural, raw and emotive feedback we need to hear. Equally groups tend to have leaders and once one person shouts their opinion; others follow or hide, leaving us with little variation in feedback. With a one-on-one usability test we ask our target audience to complete realistic tasks. We observe and see if they complete tasks, noting down pain points or problems. We get them to talk aloud so we can understand the thought process too.

Why so few people to interview?

Typically we ask small samples of our target audience to take tests. Normally 3-5 people per test. But why so few? Don’t you need lots of people to have some significance? Well if the test was purely on how many can complete a task, then yes we would need hundreds of people to validate a test. But we are not running a test for that purpose. We are running it to see if the digital service meets the users needs and if it can be used. It takes very few to uncover if a site cant be used hence 3-5 people. But don’t take my word for it. GDS, Steve Krug and Jakob Nielsen all concur.

What about stakeholders, they know what we need to do anyway?

Stakeholders are important to design thinking. Like members of the Digital Services team  we are both experts, but we are not users. What we assume is just that, an assumption and our usability tests often validate those assumptions. Making a development based of an assumption (normally contained in heavy specification documentation) is very dangerous. Time, effort and resource have been spent on something the user defines as a failure. We’re all guilty of this and according to Forbes or Genpact, digital failures happen for 90% or 66% of developments respectively. Ouch!

To combat assumption, we get stakeholders and members of the Digital Service team to attend usability tests. They are an amazing experience, at times a humble experience but always a learning experience.

“You are not your user and cannot think like a user unless you are meeting them regularly”
– Leisa Reichelt

In summary

Those who value users, know the importance of usability testing. Government Digital Services are pioneers in this area and have certainly had an impact on us especially in defining our Blueprint.

If you’re reading this and want to know more, look no further than Steve Krug – Don’t make me think. It’s the bible!