Contents
1. Executive summary: mapping Welsh digital services
The aim of CDPS's Digital Landscape Review (DLR) is to develop a better understanding of existing digital public services in Wales to:
- identify where CDPS can help to link up teams and services
- prioritise areas for development and investment
The DLR completed its alpha (exploratory) phase at the end of 2021. As part of alpha, the team gathered four distinct sets of evidence:
- 120 responses to 2 surveys of Welsh services, covering service standards and technology
- interviews with public service leaders from 30 organisations and findings from focus groups with more than 100 service owners
- findings from four workshops exploring CDPS’s potential future offering
- a partial list of Welsh public services (with an analysis of their shared characteristics and user needs)
1.1 Identified: opportunities for support
From these four sets of evidence, CDPS identified 16 opportunities for CDPS to support the development of public services in Wales.
In beta (the second, experimental phase of Agile service development), the DLR team will be working with CDPS senior leadership to shape the organisation's work plan in light of alpha findings.
The rest of this report details the evidence the DLR gathered in alpha and the 16 support opportunities it derived from that evidence.
2. Landscape Review: the evidence in detail
The 4 sets of evidence the DLR gathered in alpha (survey responses, interviews, workshop findings and a service list) covered:
- 280+ services across Welsh Government, local government and sponsored bodies
- 200+ general services that we understand most Welsh local authorities provide
We believe we covered all services for the organisations we have spoken to but we did not cover all Welsh public sector organisations.
2.1 Survey evidence: service characteristics
The DLR team did 2 surveys in alpha, covering service standards and technology. There were responses from 120 services, either in focus groups or online.
Respondents typically did not answer all questions. Some services struggled to share data numbers or service costs (approximately one-third provided this data).
Survey highlights
- 30 organisations involved in the alpha phase
- 71% of services reported the end-to-end user journey could be completed digitally
- 5,000 median number of annual user journeys for services that provided user data
- £250k - £300k median annual running cost for services that provided cost data
Table: Overview of service survey responses by type of organisation
| Local authorities | 42 |
|---|---|
| Sponsored bodies | 29 |
| Welsh Government | 27 |
| Health and care | 11 |
| Voluntary sector | 3 |
| UK government | 2 |
Table: Main users of each service in the Digital Landscape Review
| Members of the public | 84 |
|---|---|
| Frontline public servants (such as doctors or teachers) | 14 |
| Internal civil servants or local government officers | 12 |
2.2 Survey evidence: service standards
Survey highlights
- 47% of service owners either hadn’t heard of the digital service standards, or have not applied them to their service
- 25% of services are not fully available in the Welsh language
- 52% of services reported using metrics to evaluate the effectiveness of their survey
- 40% of services indicated they undertook user research*
- 49% of service owners did not know whether their service met the international accessibility standard
- 47% of services were not developed using an iterative approach
*This excludes responses indicating user research involved complaints, or a feedback form only. This figure may include some services that interpreted user research as complaints or a feedback form.
2.3 Survey evidence: service characteristics
Results are based on 26 survey responses, including 5 from the discovery phase (where questions varied from the alpha survey).
Survey highlights
- 47% of responding organisations had over 4,000 employees compared with 27% with under 500
- 20 organisations involved in the alpha phase
- 46% of respondents could estimate the annual cost of their HR and finance systems
- 11 respondents answered on behalf of their service technology vs 15 for their organisation
Table: Organisations that responded to the DLR surveys
| Local authorities | 10 |
|---|---|
| Sponsored bodies | 7 |
| Health and care | 6 |
| Welsh Government | 3 |
Table: Proportion of survey respondents by position
| Senior management | 16 |
|---|---|
| Management | 6 |
| Team leader or supervisor | 2 |
| C-suite executive or director | 1 |
2.4 Technology survey: hosting, products and team positioning
Table: Which DLR services are hosted where
| Cloud | 5 |
|---|---|
| On Prem only | 7 |
| Hybrid | 13 |
Table: Relationship between IT and Digital teams in DLR services
| Digital and IT team together | 3 |
|---|---|
| Digital team | 7 |
| IT team | 11 |
| Other | 3 |
Wordcloud: responses to ‘Commercial off-the-shelf and SaaS products used?’
2.5 Services: components needing development
Findings based upon analysis of the DLR's list of:
- 280-plus services from Welsh Government, sponsored bodies and health and care organisations
- 200-plus services from local authorities.
3. In detail: 16 opportunities to improve services
Based on the evidence summarised above, the DLR team developed a list of 16 opportunities to improve Welsh public services. Note that these are areas for CDPS involvement and do not refer to specific services.
The opportunities include a mix of:
- broad opportunities to improve how public services are designed and delivered
- specific opportunities to develop shared components
- opportunities to work with organisations on focused support for service delivery
Opportunity 1: Ensure products and services meet user needs through direct user research
Issue (mentioned by 68% of focus group organisations)
There is little evidence of services speaking directly to people about their experience as service users, with most research involving surveys. This lack of direct user research worsens user experience.
"User research has been done (…) by way of a post review questionnaire"
Quote from a service team member
Opportunity benefits
More direct user research will:
- increase effectiveness and take-up of services, as they will be designed with a better understanding of users' needs
- reduce re-work as user needs are understood up front
- increase digital inclusion (the range of people, including those with accessibility barriers such as disability, who can use a service)
Table: responses to 'Do you do user research to understand how effective your service is for users?'
| No | 13 |
|---|---|
| Through feedback forms and/or complaints | 9 |
| Yes - but not for the users of this specific service | 3 |
| Yes - during development | 15 |
| Yes - during development and an ongoing basis to iterate the service | 33 |
| I don't know | 4 |
Opportunity 2: Make products and services inclusive and accessible to all
Issue (mentioned by 76% of focus group organisations)
Services are concerned about digital inclusion and, to a lesser extent, accessibility but don't always know how to resolve these issues.
With the pandemic and the rapid move to online service delivery, not all services were designed with digital inclusion and accessibility in mind.
Services are not always usable by users who lack digital literacy, have hearing or visual impairments or have learning disabilities.
"One of our ambitions is looking at how we can improve [the] journey [of people with disabilities]. If you could help in that arena!"
Quote from a service team member
Opportunity benefits
More inclusive and accessible services will:
- reduce work across multiple channels as more users use the principal route
- lead to positive experiences and improved outcomes for digitally excluded people, who are often the most in need of public services
"The push to digital platforms is great for those who can but lots get left behind."
Quote from a service team member
Table: Proportion of kinds of feedback that DLR services have received
| Feedback in service-specific elements | 10 |
|---|---|
| Access and usability positives | 2 |
| Access and usability problems | 9 |
| Poor timelines of processing updates | 4 |
Opportunity 3: Develop consistent, high-quality services through meeting the Digital Service Standards
Issue (mentioned by 24% of focus group organisations)
The Digital Service Standards for Wales are not widely known and followed. Only around half of services are using the service standards as a guide to designing and delivering their service. Other evidence suggests many services are not meeting all of the standards. We want services to understand the standards and how to change their service to meet them.
"Some sort of standard, even voluntary, would help us to tell more difficult stakeholders that they had to engage more, for example with user research"
Quote from a service team member
Opportunity benefits
Greater awareness and adoption of the Digital Service Standards will lead to:
- better awareness of good service design practice and where services might be lacking, improving in turn the effectiveness and uptake of digital public services
- more digital inclusion, better cybersecurity and more data driven decision making
Table: responses to 'Have you heard of the Welsh Service Standards?'
| Yes - we believe our service meets all of the service standards | 31% |
|---|---|
| Yes - we are working on this service meeting them | 22% |
| Yes - but we have not considered whether this service meets them | 11% |
| No | 36% |
Opportunity 4: Improve experience of Welsh speakers with bilingual service design
Issue (mentioned by 32% of focus group organisations)
Not all services have a Welsh-language version. Those that do are mostly translated rather than designed with Welsh speaking users in mind. People who want to interact in Welsh usually have a worse user experience than users of English.
"Making [the service] of equal quality in English and Welsh was a challenge"
Quote from a service team member
Opportunity benefits
More services designed for Welsh speakers will lead to:
- the same level of positive experiences for Welsh speakers as for English speakers
- increased usability, leading to greater use and less duplication of work in other channels
- more services following government guidance on promoting the Welsh language
“Our software is largely English-led, it’s not set up to admit [language] legislation for Wales. We are not able to support these applicants effectively”
Quote from a service team member
Table: Welsh language offering among DLR services
| Fully available in Welsh | 75% |
|---|---|
| Partially available in Welsh | 21% |
| Not available in Welsh | 5% |
Opportunity 5: Iterate during service design and delivery
Issue (mentioned by 20% of focus group organisations)
Services are built, then left, not iterated (improved in small stages). This 'waterfall' approach can worsen service quality for users and reduce how long services are in use.
“Typically, you find that the initiation of a product is sufficiently funded, but the ongoing support and development of a product tend to be starved… as a consequence, you get a product that ages very quickly”
Quote from a service team member
Opportunity benefits
Greater service iteration will lead to:
- testing of a wide variety of ideas, letting developers 'fail fast'
- cost-effective integration of user feedback
- users adopting services more readily, as problems are smoothed out during development
Table: responses to 'Is the service designed using an iterative approach?'
| Yes | 50 |
|---|---|
| No | 23 |
| Don't know | 21 |
Table: responses to 'Is the service delivered using an iterative approach?'
| Yes | 54 |
|---|---|
| No | 23 |
| Don't know | 18 |
Opportunity 6: Empower service owners
Issue (mentioned by 12% of focus group organisations)
Services are not led by empowered service owners. Almost 90% of services were able to name a service owner, but this role was almost always a department or function-level role, rather than a service level one.
Services would benefit from having an empowered service owner at operational, service level to push improvements.
Opportunity benefits
Empowerment of service owners will lead to:
- better services, as one individual has responsibility over the end-to-end process and the ability to unite policy and strategy, software development teams and user feedback
- service owners tracking performance metrics and budget for their service, ensuring that both are met
Owner roles among DLR services:
| Department/function, such as Head of Highways, Transport and Recycling | 72% |
|---|---|
| Service, such as project manager | 16% |
| Organisation, such as Chief Executive | 12% |
Opportunity 7: Use service performance metrics systematically
Issue (mentioned by 28% of focus group organisations)
Service teams don't use metrics on service performance (particularly online user behaviour) in a systematic way. When they do use metrics, they tend not to capture them straightforwardly.
Only around 50% of services the DLR surveyed said that they tracked metrics, and fewer than one-third tracked anything more than user satisfaction and usage statistics.
“We don’t track at which point [users] drop off – it doesn’t tell us much”
Quote from a service team member
Opportunity benefits
By tracking metrics more effectively, services could better understand the needs of users and continuously improve services to meet those needs. Specifically, better metrics would lead to easier:
- identification of services that are struggling
- assessment of how each service iteration is performing, helping to shape a picture of good practice
- identification of service components that are ineffective and need user testing and redesign
“The number of hits on the website – do [they] mean anything?”
Quote from a service team member
Table: responses to 'Do you use any metrics to evaluate the effectiveness of your service?'
| No answer | 41 |
|---|---|
| No | 14 |
| Yes - user satisfaction | 4 |
| Yes - usage | 15 |
| Yes - additional KPIs | 35 |
Opportunity 8: Protect sensitive information and keep data secure
Issue (mentioned by 28% of focus group organisations)
Good cybersecurity and information security, and related expertise, are lacking among Welsh services. Poorly defined security practices can result in breach or loss of sensitive information.
Such shortfalls open organisations up to ethical breaches through misuse of information, as well as legal, financial and reputational challenges over data protection and storage of personally identifiable information.
“Accessibility and performance of the application are restricted by current IT security safeguards”
Quote from a service team member
Opportunity benefits
Better cybersecurity and information security will reduce:
- the risk of ethical and financial breaches
- dependence on the private sector for cyber and information security services, reducing spending
“How easy is it for some kid these days, sat at home, to cut through [our] security?”
Quote from a service team member
Opportunity 9: Design and deliver services in an Agile way
Issue (mentioned by 68% of focus group organisations)
There is a lack of skills in Agile service development and programme management within the Welsh public sector. There is also a lack of support for these practices from leadership.
Only a small number of organisations mentioned Agile skills development as part of their digital transformation.
"We need an institution wide Agile plan. [Leadership] don’t realise they can be Agile throughout the whole workplan, it’s not just an IT thing"
Quote from a service team member
Opportunity benefits
CDPS wants to embed Agile skills at all levels within Welsh public sector organisations:
- allowing teams to work in an iterative and user-centred way
- promoting a culture of collaboration and working in the open, where team members can test and fail fast to reach the best possible solution
"We lack Agile skills, and full-time programme managers – we’ve got no specialist skills or dedicated time"
Quote from a service team member
Opportunity 10: Design and deliver services with DDaT roles, even in smaller organisations and teams
Issue (mentioned by 12% of focus group organisations)
Welsh services rarely use the central government digital, data and technology (DDaT) role names explicitly. Organisations also have difficulty understanding how DDaT could be implemented in smaller organisations and teams.
Many organisations had smaller team structures than the DDaT framework suggests. Some organisations described themselves as “traditional”.
"We're undergoing a cultural shift towards working in multidisciplinary teams and doing UX"
Quote from a service team member
Opportunity benefits
Services and organisations would benefit from embedding DDaT roles and team structures to deliver digital services. That would:
- create a foundation for effective and well-rounded teams that can design and deliver better, user-centred services
- help recruitment and retention and create a pathway for continuing professional development
"We're starting to recruit people into DDaT roles, but at the moment don't have the right broader frameworks or project structures to support them"
Quote from a service team member
Table: number of DDaT roles in DLR service teams during design phases
| Service owner | 46% |
|---|---|
| Delivery manager | 38% |
| Developer | 38% |
| Designer | 37% |
| Content designer | 30% |
| Product manager | 26% |
| Business analyst | 23% |
| User research | 21% |
Table: number of DDaT roles in DLR service teams during delivery phases
| Service owner | 44% |
|---|---|
| Delivery manager | 35% |
| Developer | 27% |
| Designer | 23% |
| Content designer | 19% |
| Product manager | 25% |
| Business analyst | 20% |
| User research | 13% |
Opportunity 11: Procuring effective digital services
Issue (mentioned by 56% of focus group organisations)
Organisations find procurement of technical services to be difficult and slow, and it is often blocked by organisational bureaucracy. The result is services that often fail to meet expectations.
Services would benefit from guidance on how to procure technical services more efficiently.
"We struggled with procurement and have been left with a system that potentially doesn't integrate"
Quote from a service team member
Opportunity benefits
Better technical procurement would:
- avoid scenarios where products fail to integrate or to meet user needs
- lead to greater confidence among teams and better value for money
- lead to rapid service improvement, bringing the best of private sector development to users
"We have a 96% satisfaction rate on the services we have transformed end-to-end using new digital systems"
Quote from a service team member
Opportunity 12: Recruit and retain staff needed to deliver good digital services
Issue (mentioned by 44% of focus group organisations)
Organisations struggle to recruit and retain staff to deliver their services, particularly citing difficulty in matching private sector pay.
Many services mentioned lack of people as a challenge – teams were stretched thin, with no capacity to think about continuous improvement. In some teams, essential skills were concentrated within a few individuals who moved on, leaving a significant gap.
"We struggle to find sufficient numbers, so we tend to promote people internally, but that just creates gaps throughout the service"
Quote from a service team member
Opportunity benefits
Organisations would benefit from thinking about how they recruit for digital roles so:
- they meet current shortages, taking service teams from "keeping the lights on" to being able to improve their services in line with user needs
- organisations have a growing base of expertise, reducing time spent training new staff members and leading to long-term, consistent oversight of service transformation
"We’ve got a new digital apprentice"
Quote from a service team member
Opportunity 13: Link up services within user accounts
Issue (mentioned by 28% of focus group organisations)
Organisations, and in particular local councils, are very interested in linking up services within single user accounts. Health and care organisations also showed strong interest.
"[The council] doesn’t have a notion of an account (…) Joining up that information is a challenge"
Quote from a service team member
Opportunity benefits
Linking services within user accounts leads to:
- improved experience for users – fewer logins, entering information less often, ability to check their own data
- better reflection of user needs in user journeys – for example, linking job centre and unemployment benefit services
- streamlined service development through development of linked services at the same time
"If someone applies to free school meals, why doesn’t that talk to council tax reductions? People shouldn’t be made to fill in endless forms"
Quote from a service team member
Table: Percentage of DLR services that would benefit from authenticating a user
| Health and care | 64% |
|---|---|
| Sponsored bodies | 34% |
| Welsh Government | 41% |
| Local government | 79% |
Opportunity 14: Build forms rather than PDFs
Issue (mentioned by 36% of focus group organisations)
Form-building services are widespread in some areas but non-existent in others. In some organisations, users can interact with most services through a form and have an end-to-end digital user journey.
However, about one-quarter of services do not support a fully digital user journey, many requiring users to download and email PDF forms.
"We’re on a journey from paper/PDF forms to online forms. We’ve got 70 live, so we’re about halfway"
Quote from a service team member
Opportunity benefits
Implementing form building across services would improve user and staff experience through reducing:
- manual work for teams and the risk of data loss through paper storage
- duplicate spending on private providers, as forms are essential components of many services
"It was all essentially non-digital (…) Now some [user forms] can be created digitally and never see paper, others still are paper but scanned"
Quote from a service team member
Table: Percentage of DLR services that require inputting and storing data from users
| Health and care | 80% |
|---|---|
| Sponsored bodies | 81% |
| Welsh Government | 83% |
| Local government | 85% |
Opportunity 15: Track transactions through case management system
Issue (mentioned by 48% of focus group organisations)
Many services are still manually tracking transactions through email chains, even when the user interacted with the service digitally at the front end. Other services said they needed better case management systems.
"We have a veneer of digitisation – we take an input and churn out an email"
Quote from a service team member
Opportunity benefits
Better case management would lead to:
- better experiences for users as the 'thread' of transactions is not dropped
- reduced manual workload for service teams, particularly where non-digital channels are involved in following up on cases
"We are in the process of building a new case management system that will replace our current systems"
Quote from a service team member
Table: Percentage of DLR services that need to track transactions
| Health and care | 65% |
|---|---|
| Sponsored bodies | 68% |
| Welsh Government | 83% |
| Local government | 88% |
Opportunity 16: Use centrally developed services
Issue (mentioned by 24% of focus group organisations)
Services in Wales do not always adopt centrally developed services. Many organisations did not know about or did not use GOV.UK services like Notify or Pay or Welsh services like Hwb, for example.
"I didn't know GOV.UK services even existed"
Quote from a service team member
Opportunity benefits
Adopting central solutions or using a central design system would bring consistency and quality to services while:
- reducing time and money spent on procuring alternative private suppliers
- streamline use of resource as central services typically have a core supporting team
- creating a uniform experience for users, regardless of location
"Because it's not mandated that organisations will use these solutions, some have stayed with incumbent providers, which prevents us realising the full cash benefit"
Quote from a service team member
4. Next steps for the Digital Landscape Review
Referring to the opportunities identified in the alpha phase, during beta the DLR team will:
- work with CDPS leadership to refine CDPS's portfolio
- do more research to underpin our findings
- carry out a specific study on digital inclusion, with the aim of producing a comprehensive directory of inclusion activity in Wales