Digital Landscape Review – alpha report

Contents

1. Executive summary
2. Landscape Review: the evidence in detail 
3. 16 opportunities to improve services 
4. Next steps 

1. Executive summary: mapping Welsh digital services

The aim of CDPS’s Digital Landscape Review (DLR) is to develop a better understanding of existing digital public services in Wales to:

• identify where CDPS can help to link up teams and services
• prioritise areas for development and investment

The DLR completed its alpha (exploratory) phase at the end of 2021. As part of alpha, the team gathered four distinct sets of evidence:

• 120 responses to 2 surveys of Welsh services, covering service standards and technology
• interviews with public service leaders from 30 organisations and findings from focus groups with more than 100 service owners
• findings from four workshops exploring CDPS’s potential future offering
• a partial list of Welsh public services (with an analysis of their shared characteristics and user needs)

1.1 Identified: opportunities for support

From these four sets of evidence, CDPS identified 16 opportunities for CDPS to support the development of public services in Wales.

In beta (the second, experimental phase of Agile service development), the DLR team will be working with CDPS senior leadership to shape the organisation’s work plan in light of alpha findings.

The rest of this report details the evidence the DLR gathered in alpha and the 16 support opportunities it derived from that evidence.

2. Landscape Review: the evidence in detail

The 4 sets of evidence the DLR gathered in alpha (survey responses, interviews, workshop findings and a service list) covered:

• 280+ services across Welsh Government, local government and sponsored bodies
• 200+ general services that we understand most Welsh local authorities provide

We believe we covered all services for the organisations we have spoken to but we did not cover all Welsh public sector organisations.

2.1 Survey evidence: service characteristics

The DLR team did 2 surveys in alpha, covering service standards and technology. There were responses from 120 services, either in focus groups or online.

Respondents typically did not answer all questions. Some services struggled to share data numbers or service costs (approximately one-third provided this data).

Survey highlights

• 30 organisations involved in the alpha phase
• 71% of services reported the end-to-end user journey could be completed digitally
• 5,000 median number of annual user journeys for services that provided user data
• £250k – £300k median annual running cost for services that provided cost data

Table: Overview of service survey responses by type of organisation

Table: Main users of each service in the Digital Landscape Review

2.2 Survey evidence: service standards

Survey highlights

• 47% of service owners either hadn’t heard of the digital service standards, or have not applied them to their service
• 25% of services are not fully available in the Welsh language
• 52% of services reported using metrics to evaluate the effectiveness of their survey
• 40% of services indicated they undertook user research*
• 49% of service owners did not know whether their service met the international accessibility standard
• 47% of services were not developed using an iterative approach

*_{This excludes responses indicating user research involved complaints, or a feedback form only. This figure may include some services that interpreted user research as complaints or a feedback form}

2.3 Survey evidence: service characteristics

Results are based on 26 survey responses, including 5 from the discovery phase (where questions varied from the alpha survey).

Survey highlights

• 47% of responding organisations had over 4,000 employees compared with 27% with under 500
• 20 organisations involved in the alpha phase
• 46% of respondents could estimate the annual cost of their HR and finance systems
• 11 respondents answered on behalf of their service technology vs 15 for their organisation

Table: Organisations that responded to the DLR surveys

Table: Proportion of survey respondents by position

2.4 Technology survey: hosting, products and team positioning

Table: Which DLR services are hosted where

Table: Relationship between IT and Digital teams in DLR services

Wordcloud: responses to ‘Commercial off-the-shelf and SaaS products used?’

2.5 Services: components needing development

Table: percentage of services that would benefit from specific components

Findings based upon analysis of the DLR’s list of:

• 280-plus services from Welsh Government, sponsored bodies and health and care organisations
• 200-plus services from local authorities.

3. In detail: 16 opportunities to improve services

Based on the evidence summarised above, the DLR team developed a list of 16 opportunities to improve Welsh public services. Note that these are areas for CDPS involvement and do not refer to specific services.

The opportunities include a mix of:

• broad opportunities to improve how public services are designed and delivered
• specific opportunities to develop shared components
• opportunities to work with organisations on focused support for service delivery

Opportunity 1: Ensure products and services meet user needs through direct user research

Issue (mentioned by 68% of focus group organisations)

There is little evidence of services speaking directly to people about their experience as service users, with most research involving surveys. This lack of direct user research worsens user experience.

Opportunity benefits

More direct user research will:

• increase effectiveness and take-up of services, as they will be designed with a better understanding of users’ needs
• reduce re-work​ as user needs are understood up front
• increase digital inclusion (the range of people, including those with accessibility barriers such as disability, who can use a service)
  

Table: responses to ‘Do you do user research to understand how effective your service is for users?’

Opportunity 2: Make products and services inclusive and accessible to all

Issue (mentioned by 76% of focus group organisations)

Services are concerned about digital inclusion and, to a lesser extent, accessibility but don’t always know how to resolve these issues

With the pandemic and the rapid move to online service delivery, not all services were designed with digital inclusion and accessibility in mind.

Services are not always usable by users who lack digital literacy, have hearing or visual impairments or have learning disabilities.

Opportunity benefits

More inclusive and accessible services will:

• reduce work across multiple channels as more users use the principal route
• lead to positive experiences and improved outcomes for digitally excluded people, who are often the most in need of public services

Table: Proportion of kinds of feedback that DLR services have received

Opportunity 3: Develop consistent, high-quality services through meeting the Digital Service Standards

Issue (mentioned by 24% of focus group organisations)

The Digital Service Standards for Wales are not widely known and followed. Only around half of services are using the service standards as a guide to designing and delivering their service. Other evidence suggests many services are not meeting all of the standards. We want services to understand the standards and how to change their service to meet them.

Opportunity benefits

Greater awareness and adoption of the Digital Service Standards will lead to:

• better awareness of good service design practice and where services might be lacking, improving in turn the effectiveness and uptake of digital public services
• more digital inclusion, better cybersecurity and more data driven decision making
  

Table: responses to ‘Have you heard of the Welsh Service Standards?’

Opportunity 4: Improve experience of Welsh speakers with bilingual service design

Issue (mentioned by 32% of focus group organisations)

Not all services have a Welsh-language version. Those that do are mostly translated rather than designed with Welsh speaking users in mind. People who want to interact in Welsh usually have a worse user experience than users of English.

Opportunity benefits

More services designed for Welsh speakers will lead to:

• the same level of positive experiences for Welsh speakers as for English speakers
• increased usability, leading to greater use and less duplication of work in other channels
• more services following government guidance on promoting the Welsh language
  

Table: Welsh language offering among DLR services

Opportunity 5: Iterate during service design and delivery

Issue (mentioned by 20% of focus group organisations)

Services are built, then left, not iterated (improved in small stages). This ‘waterfall’ approach can worsen service quality for users and reduce how long services are in use.

Opportunity benefits

Greater service iteration will lead to:

• testing of a wide variety of ideas, letting developers ‘fail fast’
• cost-effective integration of user feedback
• users adopting services more readily, as problems are smoothed out during development
  

Table: responses to ‘Is the service designed and/or delivered using an iterative approach?’

Opportunity 6: Empower service owners

Issue (mentioned by 12% of focus group organisations)

Services are not led by empowered service owners. Almost 90% of services were able to name a service owner, but this role was almost always a department or function-level role, rather than a service level one.

Services would benefit from having an empowered service owner at operational, service level to push improvements.

Opportunity benefits

Empowerment of service owners will lead to:

• better services, as one individual has responsibility over the end-to-end process and the ability to unite policy and strategy, software development teams and user feedback ​
• service owners tracking performance metrics and budget for their service, ensuring that both are met

Chart: owner roles among DLR services

Opportunity 7: Use service performance metrics systematically

Issue (mentioned by 28% of focus group organisations)

Service teams don’t use metrics on service performance (particularly online user behaviour) in a systematic way. When they do use metrics, they tend not to capture them straightforwardly.

Only around 50% of services the DLR surveyed said that they tracked metrics, and fewer than one-third tracked anything more than user satisfaction and usage statistics.

Opportunity benefits

By tracking metrics more effectively, services could better understand the needs of users and continuously improve services to meet those needs. Specifically, better metrics would lead to easier:

• identification of services that are struggling
• assessment of how each service iteration is performing, helping to shape a picture of good practice
• identification of service components that are ineffective and need user testing and redesign
  

Table: responses to ‘Do you use any metrics to evaluate the effectiveness of your service?’

Opportunity 8: Protect sensitive information and keep data secure

Issue (mentioned by 28% of focus group organisations)

Good cybersecurity and information security, and related expertise, are lacking among Welsh services. Poorly defined security practices can result in breach or loss of sensitive information.

Such shortfalls open organisations up to ethical breaches through misuse of information, as well as legal, financial and reputational challenges over data protection and storage of personally identifiable information.

Opportunity benefits

Better cybersecurity and information security will reduce:

• the risk of ethical and financial breaches
• dependence on the private sector for cyber and information security services, reducing spending
  

Opportunity 9: Design and deliver services in an Agile way

Issue (mentioned by 68% of focus group organisations)

There is a lack of skills in Agile service development and programme management within the Welsh public sector. There is also a lack of support for these practices from leadership.

Only a small number of organisations mentioned Agile skills development as part of their digital transformation.

Opportunity benefits

CDPS wants to embed Agile skills at all levels within Welsh public sector organisations:

• allowing teams to work in an iterative and user-centred way
• promoting a culture of collaboration and working in the open, where team members can test and fail fast to reach the best possible solution
  

Opportunity 10: Design and deliver services with DDaT roles, even in smaller organisations and teams

Issue (mentioned by 12% of focus group organisations)

Welsh services rarely use the central government digital, data and technology (DDaT) role names explicitly. Organisations also have difficulty understanding how DDaT could be implemented in smaller organisations and teams.

Many organisations had smaller team structures than the DDaT framework suggests. Some organisations described themselves as “traditional”.

Opportunity benefits

Services and organisations would benefit from embedding DDaT roles and team structures to deliver digital services. That would:

• create a foundation for effective and well-rounded teams that can design and deliver better, user-centred services
• help recruitment and retention and create a pathway for continuing professional development

Table: number of DDaT roles in DLR service teams during design and delivery phases

Opportunity 11: Procuring effective digital services

Issue (mentioned by 56% of focus group organisations)

Organisations find procurement of technical services to be difficult and slow, and it is often blocked by organisational bureaucracy. The result is services that often fail to meet expectations.

Services would benefit from guidance on how to procure technical services more efficiently.

Opportunity benefits

Better technical procurement would:

• avoid scenarios where products fail to integrate or to meet user needs
• lead to greater confidence among teams and better value for money
• lead to rapid service improvement, bringing the best of private sector development to users
  

Opportunity 12: Recruit and retain staff needed to deliver good digital services

Issue (mentioned by 44% of focus group organisations)

Organisations struggle to recruit and retain staff to deliver their services, particularly citing difficulty in matching private sector pay.

Many services mentioned lack of people as a challenge – teams were stretched thin, with no capacity to think about continuous improvement. In some teams, essential skills were concentrated within a few individuals who moved on, leaving a significant gap.

Opportunity benefits

Organisations would benefit from thinking about how they recruit for digital roles so:

• they meet current shortages, taking service teams from “keeping the lights on” to being able to improve their services in line with user needs
• organisations have a growing base of expertise, reducing time spent training new staff members and leading to long-term, consistent oversight of service transformation
  

Opportunity 13: Link up services within user accounts

Issue (mentioned by 28% of focus group organisations)

Organisations, and in particular local councils, are very interested in linking up services within single user accounts. Health and care organisations also showed strong interest.

Opportunity benefits

Linking services within user accounts leads to:

• improved experience for users – fewer logins, entering information less often, ability to check their own data
• better reflection of user needs in user journeys – for example, linking job centre and unemployment benefit services
• streamlined service development through development of linked services at the same time
  

Table: Percentage of DLR services that would benefit from authenticating a user

Opportunity 14: Build forms rather than PDFs

Issue (mentioned by 36% of focus group organisations)

Form-building services are widespread in some areas but non-existent in others. In some organisations, users can interact with most services through a form and have an end-to-end digital user journey.

However, about one-quarter of services do not support a fully digital user journey, many requiring users to download and email PDF forms.

Opportunity benefits

Implementing form building across services would improve user and staff experience through reducing:

• manual work for teams and the risk of data loss through paper storage
• duplicate spending on private providers, as forms are essential components of many services
  

Table: Percentage of DLR services that require inputting and storing data from users

Opportunity 15: Track transactions through case management system

Issue (mentioned by 48% of focus group organisations)

Many services are still manually tracking transactions through email chains, even when the user interacted with the service digitally at the front end. Other services said they needed better case management systems.

Opportunity benefits

Better case management would lead to:

• better experiences for users as the ‘thread’ of transactions is not dropped ​
• reduced manual workload for service teams, particularly where non-digital channels are involved in following up on cases
  

Table: Percentage of DLR services that need to track transactions

Opportunity 16: Use centrally developed services

Issue (mentioned by 24% of focus group organisations)

Services in Wales do not always adopt centrally developed services. Many organisations did not know about or did not use GOV.UK services like Notify or Pay or Welsh services like Hwb, for example.

Opportunity benefits

Adopting central solutions or using a central design system would bring consistency and quality to services while:

• ​reducing time and money spent on procuring alternative private suppliers
• streamline use of resource as central services typically have a core supporting team
• creating a uniform experience for users, regardless of location
  

4. Next steps for the Digital Landscape Review

Referring to the opportunities identified in the alpha phase, during beta the DLR team will:

• work with CDPS leadership to refine CDPS’s portfolio
• do more research to underpin our findings
• carry out a specific study on digital inclusion, with the aim of producing a comprehensive directory of inclusion activity in Wales

Digital Landscape Review – alpha report

CDPS’s Digital Landscape Review aims to understand Welsh digital public services in depth to prioritise development and investment.