So which AI use case should we actually start with?
That was the question Mr Saeki, who leads DX at manufacturer Company A from the corporate planning department, let slip in a meeting that had brought together the heads of sales, IT and customer support. Six months earlier, the company had more than twenty candidate AI use cases lined up: drafting meeting minutes, handling enquiries, drafting sales proposals, searching internal knowledge, and so on. But each department insisted its own project should come first, and the AI investment decision was put off, meeting after meeting.
Today the company scores each candidate out of five on impact, feasibility, risk, cost and reusability, and verifies only the top three each quarter. Over the three months from April to June 2026, comparing eighteen candidates against the same evaluation matrix cut each meeting from ninety minutes to forty-five, and shifted the discussion from whose department gets its way to which theme is easiest to test for AI ROI.
This article is for companies with rather too many AI candidates and no clear way to prioritise them . It sets out how to prioritise AI use cases with an evaluation matrix. The aim is a state in which limited budget and headcount can be directed towards themes that promise real impact and can realistically be tested. That said, an evaluation matrix is no panacea. It leads to reproducible AI implementation only when paired with buy-in on the ground, proper data, operating rules and regular review.
Why AI use cases are so hard to put in order
The further a company gets with exploring AI, the sooner it tends to run into one particular problem: too many candidates.
Generative AI is useful across a remarkably broad sweep of work, from drafting and summarising to search, classification, enquiry handling, document preparation and analytical support. Gather ideas from each department and, in short order, you have a great many use cases on the table.
Implementing all of them at once is, of course, not realistic. Budget, staff, data preparation, system integration, security checks and internal training are all finite. The more candidates pile up, the harder it becomes to see where to begin.
Each department defines impact differently
For the sales team, impact might mean shorter proposal-writing time or better-prepared meetings. For customer support, what matters is more consistent enquiry responses and a quicker first reply. The HR team may prize efficient training-content creation or faster handling of internal queries.
In other words, the same word, impact, points to different metrics depending on who is using it.
Try to set priorities in this state and the discussion tends to turn subjective.
This is the task the front line struggles with most.
The bigger business impact lies over here.
That use case is too early on security grounds.
A row of opinions like these does nothing to move the AI investment decision forward. What is needed is not to overrule any one department, but to create a state in which they can all be compared on the same yardstick.
Decide by who shouts loudest, and the investment wobbles
When AI themes are settled in meetings, the wishes of the more vocal departments, or those closest to senior management, tend to win out.
Prioritising genuinely urgent issues is, naturally, not unreasonable in itself. But decide without a clear yardstick and you will struggle, later, to explain why this theme went first.
An investment decision you cannot explain is one that struggles to win cooperation on the ground.
AI implementation requires drawing out, from the people who own the target task, the workflow, the data used, the exceptions and the judgement criteria applied in practice. If the front line does not understand why their task was chosen, the verification becomes a box-ticking exercise.
A glut of PoCs leaves little learning behind
Press ahead with AI without setting priorities, and small PoCs start springing up all at once.
A PoC for generating meeting minutes, one for an enquiry bot, one for sales-document drafting, one for internal FAQ search. The mood is positive at first, but with attention and resources spread thin, few of them are seen through, and little usable learning is left behind. Worse, the sense that AI did not really deliver can take root, making the next attempt harder.
What an evaluation matrix is
An evaluation matrix is a table for comparing several AI use cases against common criteria and settling their order of priority.
Rather than simply drawing up a wish list, you score each candidate against the same set of axes. For example:
- Impact
- Feasibility
- Risk
- Cost
- Reusability
Rate each out of five and, where it helps, apply weightings. Looking at the totals and the overall balance makes it easier to judge which use case to test first.
That said, an evaluation matrix is not a table that spits out the right answer for you.
The matrix’s role is not to replace decision-making with something mechanical, but to move the discussion away from personal preference and departmental interest towards a shared basis for judgement. In short, it is a common language for reaching agreement.
When adopting and using AI, transparency, accountability, education and literacy, and appropriate disclosure are all important points. The OECD AI Principles set out, among other things, that those deploying AI should provide stakeholders with meaningful information about a system’s capabilities and limitations and about appropriate and inappropriate uses, within reason.
The five axes for evaluating AI use cases
Impact
Impact is the size of the benefit a given use of AI would deliver.
When you look at impact, hold these questions in mind: how many people and departments it touches, whether it shortens the work or improves quality, and whether it has any bearing on revenue or the customer experience. The aim is to gauge, in business terms, how much a use case would actually move the needle.
Feasibility
Feasibility is whether the use case can realistically be implemented.
However large the impact, if the necessary data is not in order, if system integration is complicated, or if the on-the-ground workflow is not standardised, you cannot get going straight away.
When you look at feasibility, check the following.
- Whether the input data exists
- Whether the data is in a consistent format
- Whether the work procedure is reasonably standardised
- Whether there are too many exceptions to handle
- Whether you can secure the cooperation of the using department
- Whether integration with existing systems is required
Internal FAQ search, for instance, may promise real impact, yet if the FAQs and rules are out of date the AI may end up citing the wrong information. In that case the knowledge needs sorting out before the AI is configured.
Where you use a tool that handles AI chat, AI summarisation and training-data management within the same project, it becomes easier to move from sorting candidates through to verification. Kanata, for example, offers business-support features such as AI chat, AI summarisation and e-learning, with a project library built around the idea of organising AI settings, prompts and training data in one place. Whichever tool you choose, though, the freshness of the source material and the rules for managing it still need checking separately.
Risk
Risk is the size of the problems a given use of AI could cause.
Generative AI is convenient, but there are things to watch: wrong answers, information leakage, copyright, personal data, security and accountability. Particular care is needed where customer information, contract information, HR information or non-public information is involved.
When you look at risk, apply these lenses.
- Whether the impact of a wrong answer would be large
- Whether it handles personal or confidential information
- Whether legal, audit or compliance checks are required
- Whether accountability to customers comes into play
- Whether it is work in which a human can review the output
- Whether AI output might be sent straight to outside parties
High risk does not necessarily mean a use case should be pushed to the back. But high-risk cases call for a narrowed scope of testing, an added review step and proper masking rules, rather than aiming for full production use from the outset.
In managing generative-AI risk, privacy, security, intellectual property and misinformation are among the principal concerns. The NIST AI Risk Management Framework, including its generative-AI profile, is a useful reference for managing the risks specific to generative AI.
Cost
Cost is the burden required to test, implement and run a use case.
In AI work, treating the tool subscription as the only cost leads you astray. In practice, costs such as these arise.
- Effort for initial design
- Effort for data preparation
- Effort for prompt design
- Effort for interviewing the front line
- Effort for security checks
- Effort for internal training
- Effort for review and improvement once live
An enquiry-handling bot, for instance, may carry large impact, but tends to be a high-burden theme operationally, demanding FAQ preparation, answer-rule design, escalation conditions and a defined process for handling wrong answers.
By contrast, work that assumes a final human check, such as minute summarisation or email drafting, can often be started at relatively low cost.
Reusability
Reusability is whether something that works in one department can be rolled out to others, or to other tasks.
Rather than ending in a one-off efficiency gain, AI tends to leave the organisation with more usable learning when you start from themes that can be spread sideways.
Once a minute-summarisation template is in place, for instance, it can be applied to sales meetings, development meetings, board meetings, HR interviews and more. Sorting out the knowledge base for enquiry handling could likewise be extended to HR, general affairs, IT and customer support.
When you assess reusability, look at the following.
- Whether other departments have the same task
- Whether prompts and templates can be reused
- Whether the training data can be repurposed
- Whether the operating rules can be spread sideways
- Whether the results are easy to explain
Highly reusable use cases make it easier to turn the lessons of the first round of testing into an organisation-wide foundation for AI.
How to build an evaluation matrix
List candidate use cases by business process
First, set out what you want AI to do.
Here, rather than working in units as sweeping as “make sales more efficient with AI”, break things down by business process.
In the sales domain, you might split it like this.
- Pre-meeting company research
- Designing questions for first meetings
- Summarising meeting notes
- Drafting the structure of a proposal
- Drafting email replies
- Classifying reasons for lost deals
For customer support, you might divide it as follows.
- Classifying the content of enquiries
- Drafting a first-response suggestion
- Extracting FAQ candidates
- Summarising the content of complaints
- Analysing the response history
Too fine a unit becomes hard to manage; too broad a one cannot be evaluated. The important thing is a granularity at which you can say who, in which task, inputs what and gets what kind of output.
Define each axis on a five-point scale
Next, decide the scoring standard for each axis.
For impact, for instance, you might define it as follows.
| Score | Rough guide to impact |
|---|---|
| 5 | Promises large reductions in effort, quality gains, or effects on revenue and customer experience across multiple departments |
| 4 | Promises a clear reduction in effort or a quality gain in a specific department |
| 3 | Some improvement is likely, but the reach is limited |
| 2 | There is an effect, but it is hard to quantify |
| 1 | The effect is small or unclear |
For feasibility, think along these lines.
| Score | Rough guide to feasibility |
|---|---|
| 5 | The data exists, the workflow is sorted, and testing can begin at once |
| 4 | Testing is possible with a little preparation |
| 3 | Some data preparation or process tidying is needed |
| 2 | System integration or major process change is required |
| 1 | The conditions for implementation are not in place at present |
For risk, defining a higher score as lower risk makes it easier to produce a total.
| Score | Rough guide to how low the risk is |
|---|---|
| 5 | Handles no confidential information, and human review is straightforward |
| 4 | Handles internal information but the scope of impact is limited |
| 3 | Touches confidential or personal information; rules need defining |
| 2 | The impact of wrong answers or information-management failures would be significant |
| 1 | Carries significant legal or security risk and demands careful judgement |
Decide the weightings
You can give every axis the same weight, but depending on the company’s stage, weighting can make the judgement clearer.
For a company just starting with AI, it may be better to lean on feasibility and risk. Take on a high-difficulty theme from the outset and the burden on the front line is heavy, with little to show for it but the memory of failure.
For a company that already has several AI successes behind it, on the other hand, the stage may be set to weight impact and reusability and move on to larger investment decisions.
In an early-adoption phase, weightings of this sort might apply.
| Axis | Weight |
|---|---|
| Impact | 30% |
| Feasibility | 25% |
| Low risk | 20% |
| Low cost | 15% |
| Reusability | 10% |
There is no single right answer for weightings. What matters is that senior management, DX, IT and the front-line owners agree in advance on what they are prioritising this time round.
Score the candidate use cases
Once the axes are set, score each candidate.
By way of example, let us compare these three.
| Use case | Impact | Feasibility | Low risk | Low cost | Reusability | Total |
|---|---|---|---|---|---|---|
| Automatic summarising for minutes | 4 | 5 | 4 | 5 | 5 | 23 |
| Internal enquiry bot | 5 | 3 | 3 | 3 | 4 | 18 |
| Drafting sales proposals | 4 | 4 | 4 | 4 | 4 | 20 |
In this example, automatic summarising for minutes comes out highest, because the data is easy to assemble, risk is easy to keep down given the human-check assumption, and it spreads readily to other departments.
The internal enquiry bot, by contrast, carries large impact but scores rather lower on feasibility and cost, since it needs FAQ preparation, answer rules and a design for handling wrong answers.
So, rather than looking only at the total, see which items are low; that tells you what to prepare next.
Choose the top themes as test candidates
Themes that come out near the top of the matrix should be chosen as test candidates first, not rolled straight into production.
The key point here is that high priority and ready for immediate production are not the same thing.
The matrix is for deciding which tests to spend limited resources on. For the top themes, draw up a test plan along these lines.
- Target task
- Target department
- Test period
- Number of users
- Success metrics
- Risk mitigations
- Human review step
- Conditions for the go-to-production decision
For minute summarisation, for instance, a test might look like this.
Over one month in July 2026, trial AI-assisted minute-writing across ten of the sales team’s regular meetings. Compare it with the usual drafting time, number of revisions and participant satisfaction, and judge whether it is fit for production use.
Making the period, target and metrics this explicit means the test results feed readily into the next investment decision.
Putting the matrix to work in AI investment decisions
An evaluation matrix is not something you build and then set aside. To put it to work in investment decisions, how you read the ranking matters.
Start with “high impact, high feasibility”
The easiest place to begin is with use cases that are both high impact and high feasibility.
This is the territory where results are easy to confirm in short order and a sense of success is easy to build internally. Minute summarisation, email drafting, summarising internal documents and tidying meeting notes can fall into it.
In the early phase, starting from themes like these helps lower the psychological barrier the front line feels towards AI.
Manage “high impact, low feasibility” as a medium-to-long-term theme
Some use cases carry large impact yet score low on feasibility.
Company-wide knowledge search, automated customer handling, contract review and integration across multiple systems often come into this bracket. These themes could become the main targets for AI adoption. However, preparations such as data organisation, access management, workflow design and security checks are needed. Rather than rejecting them outright because they cannot be implemented immediately, manage them as medium-to-long-term themes. On the matrix itself, set up separate preparation tasks aimed at improving their feasibility scores.
Such themes are best parked as medium-to-long-term candidates, with the conditions, namely the data, the rules and the security checks, put in place before you take them on in earnest.
“Low risk, low cost” is the doorway to adoption on the ground
Even where impact is only moderate, a use case that is low in risk and low in cost makes an effective doorway into AI.
Rewriting text, drawing up meeting agendas, tidying weekly reports, brainstorming and checking comprehension in internal training are examples.
Judged purely on business impact, these may look like low priorities. But they matter when it comes to building the habit, on the ground, of using AI day to day.
AI implementation does not advance on tool adoption alone. As the front line accumulates the experience of feeling “this is a task I can hand to AI”, it leads on to larger transformation of the work itself.
Turn the reasons for a low score into the next improvement theme
Use cases that scored low on the matrix need not be discarded.
If the enquiry bot scored low on feasibility because the FAQs are out of date, then the next quarter should go on sorting out the FAQs. If contract review scored high on risk because the rules of use are not yet in place, then you need to define the review scope with the legal team.
The matrix is not merely a table for deciding what is in and what is out. It is also a tool for finding which preconditions you should put in place in order to move AI forward.
Practical points for running the evaluation and testing
Prioritising AI use cases throws up a good deal of information to organise: sorting candidates, summarising meeting notes, drafting the evaluation axes, sharing test results, and more.
Here, without tying ourselves to any one tool, let us set out an approach that works in practice.
Sort the candidates with an AI chat
First, feed the AI ideas gathered from each department into an AI chat and classify them by business area.
You might ask it, for instance, as follows.
Below are AI ideas gathered from each department.
Please organise them by business area, intended users, input data, expected impact and risk.
Merge any overlapping ideas, and bring them to a granularity that sits well on an evaluation matrix.
This sorts out the variations in wording and the duplicates. That said, the classification the AI produces is a draft. In the end, the department heads and the people who do the work need to check it.
Structure meetings and interviews with AI summarisation
What you gather in interviews with each department becomes easier to evaluate once organised through AI summarisation.
In particular, the remarks of people on the ground carry important information that is hard to quantify.
This work spikes suddenly at month-end.
If the reviewer is away, everything grinds to a halt.
There is a manual, but in practice we just ask the old hands.
Information of this kind bears on the feasibility and risk assessments. Summarise the meeting records and notes and organise them by axis, and the basis for each score becomes easier to explain.
Manage the evaluation materials in a reusable form
The matrix, the scoring standards, the test plans, the prompts and the approach to organising training data need not be built from scratch each time.
It helps to keep things like the following in an internal knowledge-management tool, document-management system or AI platform.
- Prompts for evaluating AI use cases
- An evaluation-matrix template
- Interview questions
- A PoC-plan template
- A minute-summarisation template
- A risk-check checklist
Where you want to manage several AI features within the same business project, choosing a tool that handles prompts and training data together makes the running of it easier. A service such as Kanata, which lets you handle AI chat, AI summarisation and a project library in the same environment, suits this kind of reuse-oriented design. Whether it fits your existing internal systems and security requirements, though, is something each company needs to check.
Feed the test results back into the matrix
When a round of AI testing is done, feed the results back into the matrix.
If a minute-summarisation use case you had rated “impact 4, feasibility 5” in fact took a lot of correcting and went down poorly on the ground, the rating needs revisiting.
Conversely, if email drafting, which you had thought of as moderate impact, ends up used across the whole sales team and saving thirty minutes a week per person, there is room to raise its reusability and impact scores.
The matrix is not a fixed table but a working record, updated as the test results come in.
How to avoid coming unstuck when running the matrix
The matrix is handy, but use it the wrong way and it can, if anything, make decision-making rigid. Here are the points to watch as you run it.
Do not decide on the score alone
A high total does not necessarily mean that theme should go first.
If the total is high but the front-line owner is in no position to help just now, the test will not progress. Conversely, even a slightly lower total is worth trying first if the front line is ready to cooperate and you can learn quickly.
The matrix is decision material, not the decision itself.
Do not ignore effects that are hard to quantify
Not every benefit of AI can be reduced to hours saved.
Effects like the following, for instance, are hard to quantify yet matter to the organisation.
- New joiners pick up the work faster
- Fewer questions pile up on the old hands
- The points at issue in meetings get sorted out
- Internal documents become more consistent in quality
- Employees grow accustomed to using AI
When thinking about AI ROI, it is important to record quantitative and qualitative effects separately.
Do not underestimate the risk
Leaning too heavily on “a human checks it, so we are fine” is also dangerous.
Even with a human review step, you cannot keep risk down if the reviewer is rushed off their feet, there are no judgement criteria, or the basis for the output is unclear.
For use cases touching customer handling, contracts, HR, finance or security in particular, you need to be clear on how far AI output may be used.
The OECD frames the task as managing the risks while drawing on the benefits of generative AI, and treats privacy risk and the efforts of individual countries and regions as live concerns. The same themes of continuous improvement in AI governance and risk management are widely held to matter.
Review the axes regularly
In the early days, leaning on feasibility and low risk is natural. But once experience has built up internally and the operating rules and data are in better order, you may reasonably give more weight to impact and reusability.
Setting aside a session once a quarter to revisit the axes and weightings makes it easier to keep your judgement in step with where things actually stand.
Decide the order, not the rejection
Using the matrix throws up use cases that sit lower down.
Treat these as “this idea has no value” and you will tend to lose the goodwill of the front line.
What the matrix decides is not whether something has value, but the order in which to take it on.
- Feasibility is low for now
- The risk mitigations are not yet enough
- The data needs sorting out first
Put it this way and you can keep front-line ideas alive as future candidates rather than dismissing them.
In summary
A growing number of AI use cases is not a bad thing in itself. If anything, it is a sign that the front line has begun to think of AI’s possibilities as their own.
But the more candidates there are, the more you need a mechanism for setting priorities.
- Choose on impact alone and you may pour time into a theme you cannot implement.
- Choose on feasibility alone and you may shut yourself into small improvements.
- Choose on risk alone and you may be left with no theme worth attempting.
That is precisely why it matters to lay out impact, feasibility, risk, cost and reusability side by side and compare them on the same matrix.
The matrix is not there to make the AI investment decision mechanically. It is a tool for senior management, DX, IT and the department heads to argue it out while looking at the same map.
You do not need a perfect matrix from the start. The realistic approach is to narrow the field to around ten candidates, score them on the five axes, and test the top two or three on a small scale.
Then, on the strength of those results, you update the axes and the scores. Repeated, that cycle becomes the foundation for moving beyond one-off PoCs to genuine business-process redesign and AI implementation.
Q&A
Who should decide the priority of AI use cases?The realistic answer is that senior management, the DX team, IT and the front-line owners decide it together. Leave it to senior management alone and it tends to drift away from how things really work; leave it to the front line alone and the investment judgement and risk management tend to be weaker.
Do you really need five axes on the matrix?Not necessarily five. But we would recommend including, at a minimum, impact, feasibility and risk. Where you are also weighing investment decisions and sideways rollout, adding cost and reusability makes comparison easier.
Should you always start with the highest-scoring use case?Not always. The score is one piece of decision material. You also need to weigh the front line’s readiness to cooperate, the timing for testing, the state of the data and the company’s top-level priorities.
How should you think about AI ROI?Consider time saved, quality gains, fewer reworks, faster customer responses and employee satisfaction separately. Fixating on a precise monetary conversion from the outset makes progress hard, so in the early stages it is best to start with easy-to-measure metrics such as time saved, number of revisions and user satisfaction.
How often should you review the matrix?Reviewing it about once a quarter makes it easier to run. AI tools, internal data, the front line’s familiarity and your risk appetite all shift, so rather than fixing the axes once and leaving them, it is important to update them as the test results come in.