We’ve recently supported Welsh Government to explore digital products to help make the neurodivergence referral and assessment more effective and efficient.

We developed a set of criteria to evaluate these digital products and the Digital Service Standard for Wales played an essential role in our process. 

Background

In early 2024, CDPS, in partnership with the Welsh Government's neurodivergence and learning disabilities team, undertook research to explore ways to provide faster and easier access to resources and support for those awaiting neurodivergence referrals or assessments.

The research – involving both professionals and service users – also aimed to improve the effectiveness of the processes involved in delivering and gathering information to support these services.

One of the recommendations was to design and test a digital information gathering product: a centralised digital tool designed to streamline information gathering, addressing current inefficiencies and time-consuming processes. This product would hopefully benefit users and professionals by enabling more efficient collection and dissemination of information. 

Overview 

Following the discovery, CDPS have just finished a rapid phase of work exploring and assessing existing digital information gathering products and evaluate their suitability for use in the neurodivergence referral process.

Key to this piece of work was developing a methodical – yet flexible - process we could follow. The approach we took was to establish a set of evaluation criteria, drawn from key categories, which adhere to the Digital Service Standard for Wales:

  • User needs: How well the tools met the specific needs of users, based on the insights captured in the discovery report.  
  • Organisational needs: How well the tools met the requirements outlined by the organisation commissioning, coordinating and/or delivering the service. In this piece of work, we focused on the Welsh Government's organisational needs. Depending on the project and its phase, this could mean understanding the needs of multiple organisations. We worked with the Welsh Government team to break down these requirements into ‘essential’ and ‘desirable’ criteria. This was helpful to make more meaningful comparison, and we anticipate that agreeing ‘essential’ and ‘desirable’ criteria is essential when multiple organisations are involved.  
  • User experience (UX) best practices: How well the tools adhered to widely accepted principles of accessibility, usability, and intuitive design. 
  • Neurodiverse design considerations: How well the tools were designed with the needs and preferences of neurodiverse audiences, ensuring inclusivity and equitable access. While this criteria might seem specific to the context of this project, ‘make sure everyone can use the service’ is a key principle in the Digital Service Standard for Wales. For example, when designing for neurodivergent people, it’s important to consider diverse cognitive and sensory needs. When reviewing products, we were looking for things such as simple navigation, avoiding cluttered layouts and the use of plain language ensuring those products are as easy to use as possible 
  • Service target outcomes: Ensuring the products have the potential to support the desired outcomes for people in Wales, including efficiency, reliability, and ease of use. Setting target outcomes for the service is important to do even at the early stages of exploration as they can serve as clear guiding goals to keep the team and project aligned. 

Once we finalised the categories, we developed an evaluation matrix to structure our analysis, breaking down each category into specific, measurable criteria. Each existing product was assessed against these criteria, which meant we could compare products and identify the most suitable solutions for the neurodivergence referral process.

What we’ve learned from the process

  • There were two of us working together on the evaluation, often reviewing the same product from different and overlapping angles. This meant we covered as much ground as possible, whilst also filling gaps the other may have missed. 
  • Working as a pair also meant we not only learned from each other’s approaches, we also challenged each other along the way! 
  • It was tricky to assess some things when we didn't have access to the back end of the product platforms. However, this isn't meant to be “perfect”, and the report is a tool to aid discussion and help the team in Welsh Government  make informed decisions. 
  • We found it was really useful to map out the user journey through the referral process. This made sure there were criteria relating to each stage and each channel. 
  • The process and tools we developed during the project were valuable in making sure our evaluations were fair, thorough and aligned to the Digital Service Standard for Wales. 
  • By designing the evaluation against a set of categories, dividing these into individual, measurable criteria and plotting into an evaluation matrix, we set ourselves up for a methodical and comprehensive approach to the evaluation. 

We’re excited about the potential for this framework and believe these categories, with the Digital Service Standard as their backbone, can be a useful tool to help with evaluating other digital products and tools.  

If you would like a copy of our evaluation matrix, or want to find out more about the Digital Service Standard for Wales, please contact our Head of Standards, Jemima Monteith-Thomas by emailing jemima.monteith-thomas@digitalpublicservices.gov.wales