Usability testing during COVID-19 – how we conducted remote representative usability testing for adult social services
20 January 2021
Following the findings of our discovery phase we’ve now moved into the alpha phase of our Adult Social Care Digital Transformation Project and created a prototype status update text and website. These are designed to help keep people informed with updates about their request to Adult Social Care, meaning that there is less frustration and confusion about what is happening. Throughout this phase, we have been testing our ideas with real users to evaluate whether they meet user needs.
What is usability testing?
Usability testing is a key user research method to evaluate prototypes (design ideas that we’ve turned into something someone can interact with). In a usability test, people who would normally use the service or product being designed are asked to carry out typical tasks to let us know how well these prototypes work.
Usability testing is often referred to as the ‘gold standard’ of UX evaluation methods, and with good reason. It allows us to collect observed data, as well as reported data. In other words, it lets us see how our ideas work and where they can be improved, rather than relying only on what we hear about them.
Staying safe – remote usability testing
Usability testing is easiest done in person, where the researcher can observe more: what is the participant focusing on? What does their body language tell us?
However, in order to keep our participants and ourselves safe during COVID, we’ve opted to run all of our research remotely. Remote usability testing is often conducted using video conferencing tools like Zoom or Microsoft Teams, where we can screenshare to show our designs, and even give control of the mouse to the participant so that they can roam around the screen as if it was their own. It also allows us to see people’s reactions taking place.
While this approach can work well for people who are confident using technology, for those less familiar, it can be a barrier to their inclusion in the research. With our focus on Adult Social Care, many of our users are less familiar with technology, and it was crucial that we involve them in our research to properly evaluate our designs and ensure their valuable input.
How do we usability test remotely when we can’t use video conferencing software?
To tackle this problem, we decided to split our approach. For those comfortable using video conferencing software, we conducted our sessions that way. For those less familiar, we conducted sessions over the phone. “But how can a usability test happen over the phone?” I hear you ask!
Here are the steps we took to ensure we could surface valid findings this way:
- take the time to build rapport in advance – building rapport at the start of research sessions is key to helping people feel at ease in what can sometimes be quite an unfamiliar situation. It helps ensure people feel comfortable to tell you the truth about your prototype, and helps you as a researcher to understand how they like to communicate. Running sessions over the phone provides less opportunity for rapport building due to the fact that you can’t see each other. You are essentially just a strange voice on the end of a phone. So how do you make this feel less strange for people?
What we lack in visual cues, we can make up in time spent. To do this, we called every participant in advance to arrange their session, and to spend a little bit of time having a chat. This meant that when we came to pick up our phones for the session, we knew each other and it felt more familiar for the participants. Which leads us on to point 2…
- brief the participants in advance about what to expect from the session – It was important to ensure that people were comfortable both with us as researchers, and also with the session itself. We took the opportunity in that initial phone call to explain a bit about the structure of the session and what to expect (without giving too much away in advance)! This meant that when people came to take part, they were clearer about the plan and what their role was, helping the whole experience feel calmer, and helping us to focus on the research at hand
- focus on what you can observe – without visual information such as facial expressions and body language, there’s a lot less to be observed. However, there are still things that we can pay attention to: the tone of someone’s voice, the words they use, any pauses in their speech. When we can’t see what someone is doing on a screen, it’s important that we pay attention to these things, as they can tell us a lot
- encourage people to think out loud – in addition to observing, we can also ask people to relay to us what they are doing, and what they are thinking as they are doing it. This is a technique called a ‘concurrent think-aloud’ and helps us create a clearer picture of what is going on, especially when we can’t see what is happening. It can feel quite unnatural to do this, so where needed you can also boost this technique by asking elicitation questions to encourage people to speak up
- follow up afterwards – as with the other points on this list, this is general good practice, but something that’s even more important when you’ve not had the chance to meet your participants. We found that alongside posting out vouchers, it was helpful to send out a thank you letter to each participant, as well as an information sheet and contact details, so that they could always follow up with any questions or requests
As we continue into the next phase of this project (beta), we’ll continue to conduct user research to ensure the texts and website are valuable for the people using them.
