In previous blog posts, we’ve shared our work exploring how service assessments should work in Wales. At the heart of this alpha is an experimental approach that relies on user research to help us make evidence-based decisions about how to move forward. 

What we’ve been working on 

This month, we have been working on experiment 2 basing our work on this hypothesis: 

We believe that providing assessments tailored to delivery phases with clear ‘met/not met’ outcomes will drive changes aligned with the Digital Service Standard for Wales. 

What we designed 

To test our hypothesis, we designed a user journey that introduced the concept of a two-stage service assessment and integrated some of the insights from our previous round of research. We then designed three Figma prototypes that brought elements of the journey to life: a checklist, an outcome report and a web page explaining the two-stage service assessment. The prototypes were designed to quite high fidelity so our research participants could get a better understanding of the concept. 

What we learned 

After our first few research interviews we started to believe that our hypothesis was not quite right. Participants were using the prototypes as expected but talking about using them in user journeys that did not match the one we had created. 

By the time we had completed all nine interviews it was clear our hypothesis was totally wrong. 

Our hypothesis hadn’t considered: 

  • how differently teams are working across Wales 
  • the variation in how organisations view the importance the service standards 
  • the desire from some individuals/teams to do good work 

What happens next 

Our hypothesis-based approach helped us to test-run a skeleton service while it was still a concept. Although it wasn’t the right direction, it did help us to learn. We were able to: 

  • get more clarity on what users needed from the process (by telling us what it shouldn’t be) 
  • understand the complexity that could impact the success of service assessments 
  • get more confidence on how to move forward 

We’d recommend taking an experiment/hypothesis based approach when thinking about developing a service. Working in the open (a key principle within the Digital Service Standard) allowed us to openly share that what we initially thought wasn’t quite right. In our next blog post, we’ll cover our final recommendations on what a future service assessments service could look like.