22 December 2021
Everyone understands the role of a researcher on a project, or at least they think they do. But there is more to carrying out interviews and telling others what they’ve learned. And it takes a whole research team to make sure it’s robust and valuable.
Kat Anderson, user researcher for our accessing adult social care private beta explains:
Project background
I recently joined the team who embarked on a journey to solve a problem that people are faced with when asking for help from adult social services - long waiting times and lack of information on the progress as their case if referred to the relevant department.
‘Track my request’ is a text notification service that is now in private beta (this means it is being tested with a small live audience), currently to the less vulnerable people who need help from adult social services. The aim of the service is to keep people updated on progress through a series of 4 text messages sent out at different times in the referral journey. The goal for the local authority is to reduce the number of calls that are made by people seeking reassurance that they are “still in the system”.
The trial has been operating in Neath Port Talbot Council since September 2021.
The planning phase
Research begins with a planning phase. We decided on the research questions and the logistics on how we were going to run the research.
Mark Sharwood, public services development manager at Torfaen County Borough Council was an observer at the research sessions:
“I've been part of the research gang, acting as an observer. We've been taken through the whole process, and it takes a lot of time. I thought I was used to doing a certain amount of user research, but after this experience, I will have to think again. It's not just about setting up a timetable of interviews and thinking about the research questions, it’s also about the recruitment of potentially vulnerable people.
How are you going to collect the data? Who’s going to write the notes? Where are the notes going to be held and what happens to them when they've been made? Are all the observers on the same wavelength when it comes to taking notes? So, a certain amount of training goes on just to make sure everybody is on the same page”.
Rhian Johns, development officer at Neath Port Talbot Council adds: “We underestimated the amount of planning that is required to undertake user research and booking out time in advance to attend all relevant sessions.
You can imagine our calendars; days full of meetings, why is the researcher inviting me to observe the sessions? Can’t they just tell me what happened?”
During research sessions
The purpose of research is building empathy with the people who are or who will be using the service, to really understand what they are experiencing.
Nita Sparkes, principal organisational development consultant, Neath Port Talbot Council:
“It's good to hear first-hand what the service user is saying. Having empathy means that we can design services that work well for people, services that don't need instruction manuals!”
So how do we do it? We do it by active participation. Any observer coming to a research session has a responsibility to capture what people have said. It’s not easy it’s a skill that has to be developed.
There is time for interpretation later, and we do it together, as a team.
Nita continues “Taking notes almost verbatim helps capture useful quotes that can be used as part of the research findings. It makes you think about what you have recorded in your notes and the validity.”
Mark adds “An observer needs to listen not only to the actual words that are being said, so you can quote them and not misinterpret them, but also understand the context and to a large extent their body language (if we were lucky enough to have them on video). Keeping up with that in an interview is incredibly hard. I'm thankful that there were two observers on every interview, because I knew there were sentences and bits that I missed in the hope that the other observer caught it as well. The expression that research is a team sport is absolutely true, I was really relying on my teammates to make sure that they had caught the bits that I had missed.”
After research sessions
Interpreting the findings starts with a discussion about what we learnt immediately after the session. “It's a killer to do it straight away but important as it’s fresh in your memory - if it’s left to another time you may end up with a blurred recollection” says Nita.
Research debriefs are important both for the researcher and the observers, particularly when talking to vulnerable people. The circumstances that we hear are not always easy to digest. We look after our psychological safety by taking time to discuss what we’re heard, and we let it go. We look out for each other.
Jon Lewis, senior systems support officer at Neath Port Talbot Council, was drafted in as subject matter expert to help interpret the findings: “It wasn’t something I had done before, so it was a bit overwhelming. You need to be in the right headspace to take in all of the information, make sense of it and give feedback.”
All the research data was put on our ‘research board’. Nita adds “The board had all of the findings from all interviews so at first it was a lot to take in”
Mark adds “You're looking at sorting raw data into insights. One of the things that I learned is what an insight really is. It is not a collection of data or a collection of observations. You must understand the background of the situation and of the person and the problem, or barriers that they are having to achieve an outcome.
Ultimately, an insight is all of that. That insight gives us something actionable to help change the way in which we currently do or design things. That to me was an important part of the synthesizing process.”
Don’t forget the last part of research, presenting the findings. I worked with a team of 10 observers who actively participated by taking notes and engaging in collaborative sense making. That is 10 people who can confidently help share the findings with other stakeholders. The more senior the observers, the more likely research is going to reach the right people.
User research is a team sport
Does this mean the observers can now do research themselves? Not quite. Whist working collaboratively has huge advantages for everyone involved, the researcher still plays the lead role in orchestrating how the research is structured and carried out. We help each other look out for bias, but it takes years of practice to develop the knowledge needed to interpret the findings in a meaningful way that is going to be useful to future decision making. Bad research is worse than no research at all, as it can build up false impressions and lead you in the wrong direction.
Mark adds “Understanding what the considerations are for us as local authorities was very powerful for me. We were lucky to have experts helping us through this, and next time we might not have that. If we were planning to do our own user research, what sort of things would we have to think about? Do we have people with user research skills in our organisation? And if we do, do they have the capacity? And can we act on our lessons learnt? Projects within organisations do not always do lessons learned logs, so sometimes those lessons are lost. Sometimes those lessons are only shared within a certain part of the organisation and don’t benefit other parts.”
“It’s important to discuss lessons learnt from the research process, so that the next time we do research, we can do it better.”
This blogpost was written by Kat Anderson, supported by Nita Sparkes, Jon Lewis and Rhian Johns, Neath Port Talbot County Borough Council and Mark Sharwood, Torfaen County Borough Council.