Assessing if artificial intelligence is the right solution

AI is just another tool to help deliver services. 

Designing any service starts with identifying user needs. If you think AI can help you meet user needs, consider your data and the specific technology you want to use. 

When assessing if AI can help you meet users’ needs, consider if:

• there’s data containing the information you need, even if disguised or buried
• it’s ethical and safe to use the data - see the UK Government’s Data Ethics Framework
• you have a large quantity of data for the model to learn from
• the task is large scale and repetitive enough that a human would struggle to carry it out
• it would provide information a team could use to achieve outcomes in the real world

AI is not an all-purpose solution. Unlike a human, AI cannot infer, and can only produce an output based on the data a team inputs to the model.

As with all technology projects, make sure you can change your mind at a later stage and you can adapt the technology as your understanding of user needs changes.

When identifying whether AI is the right solution, work with:

• specialists who have a good knowledge of your data and the problem you’re trying to solve, such as data scientists
• at least one domain knowledge expert who knows the environment where you will be deploying the AI model results

For your AI model to work, it needs access to a large quantity of data. 

Work with specialists who know your data to assess your data state, such as data scientists. 

You can assess whether your data is high enough quality for AI using a combination of:

• accuracy
• completeness
• uniqueness
• timeliness
• validity
• sufficiency
• relevancy
• representativeness
• consistency

If your problem involves supporting an ongoing business decision process, plan to establish ongoing, up-to-date access to data while following data protection laws.

There is no one ‘AI technology’. Currently, widely-available AI technologies are mostly either supervised, unsupervised, or reinforcement machine learning. 

The machine learning techniques that can provide you with the best insight depend on the problem you’re trying to solve.

You can buy or adapt commercially available products for some types of problems for which machine learning is commonly used.

It can be difficult to be specific about the benefits from an AI project because of its experimental and iterative nature. 

To explore this uncertainty and provide the right level of information around the potential benefits, you can:

• carry out some initial analysis on your data to help you understand how hard the problem is and how likely the project’s success would be
• build your business case around a small-scale proof of concept (PoC) and use its results to prove your hypothesis

Once you have secured budget, allow enough time and resources to conduct a discovery to show feasibility. 

Discovery for projects using AI can often takes longer for similar projects that do not use AI.

When assessing if AI could help you meet user needs, consider how you will procure the technology. 

Define your purchasing strategy in the same way as you would for any other technology. 

Whether you build, buy or reuse (or combine these approaches) will depend on different considerations, including:

• if the needs you’re trying to meet are unique to your organisation or you could fulfil users’ needs with generic components
• the maturity of commercially available products that meet those needs
• how your product needs to integrate with your existing infrastructure

Address ethical concerns about the use of AI from the start of the procurement process.

Read the guidelines for AI procurement from GOV.UK.

Your team can build or adapt off-the-shelf AI models or open-source algorithms in-house.

When making this decision, you should work with data scientists to consider whether:

• your team has the skills to build an AI project in-house
• your operations team can run and maintain an in-house AI solution

You may be able to buy your AI technology as an off-the-shelf product, especially if it’s a common application of AI like optical character recognition. 

Buying your AI technology is not always suitable as the specifics of your data and needs can mean the supplier needs to significantly customise an existing model or build it from scratch.

Your AI solution needs to be integrated into an end-to-end service for your users, even if you can buy significant components off the shelf.

Read the guidelines for AI procurement from GOV.UK.

You must understand who is responsible if the system fails when using AI. The problem may lie in a number of areas, including failures with the: 

• data chosen to train the model
• design of the model
• coding of the software, or its deployment

Establish a responsibility record which defines who is responsible for different areas of the AI. Consider if:

• the models are achieving their purpose and business objectives
• there is a clear accountability framework for models in production
• there is a clear testing and monitoring framework in place
• your team has reviewed and validated the code
• the algorithms are robust, unbiased, fair and explainable
• the project fits with how citizens and users expect their data to be used

Depending on your organisation’s maturity, you could set up a dedicated board, committee or forum to handle AI data and model governance.

You can keep a central record of all AI technologies you use, including:

• where AI is in use
• what the AI is used for
• who’s involved
• how it’s assessed or checked
• what other teams rely on the technology

Read the Algorithmic Transparency Recording Standard on GOV.UK.