How to Measure AI Value: Lessons from Ireland’s AI Projects and Business Leaders

When a leadership team asks me how to measure AI value, I bring three perspectives to the discussion.

Through the AI Awards, I see examples of organisations applying AI to specific problems. Through the AI Ireland Podcast, I speak with people building and implementing these systems. Through our leadership research, I examine the challenges organisations report as they move towards adoption.

Those perspectives help me ask better questions.

What changed in the work? Who benefited? What evidence supports the result? And what would another organisation need before attempting something similar?

The AI Awards 2026 finalists offer useful examples of why those questions matter. They also show why a single measure, such as “hours saved”, cannot describe every AI project.

Start with the outcome the work is meant to improve

Consider three examples from the published finalist summaries.

Headon Boxing Academy’s Pulse identifies patterns in member behaviour that may indicate cancellation risk, helping staff prioritise outreach.

SPRYT’s Asa supports appointment booking, reminders and patient queries.

The Mental Health Commission uses human-supervised generative AI to help turn inspection findings into draft regulatory reports. AI Awards 2026 finalist summaries

These are different jobs. They need different measures.

For a retention project, I would ask whether outreach reaches the right people and whether it improves retention compared with a suitable baseline.

For appointment administration, I would examine staff workload, booking completion, attendance and the experience of people using the service.

For report drafting, I would examine total preparation and review time, factual accuracy, omissions and the corrections required before approval.

These are the measures I would investigate—not claims about results those finalists have achieved. The published summaries describe the projects; they do not independently verify every outcome.

The leadership lesson is straightforward: define value around the task before choosing the metric.

Measure the checking, not just the creation

Producing a draft quickly is useful only if the full process improves.

Imagine a proposal that previously required 60 minutes to prepare and draft, followed by 20 minutes of review.

With AI assistance, drafting falls to 25 minutes, but review increases to 35 minutes.

The total falls from 80 to 60 minutes. The net saving is 20 minutes.

These figures are illustrative, but the distinction matters. Measuring only the drafting stage would exaggerate the improvement.

The Mental Health Commission finalist example makes this a useful leadership question: when AI supports drafting, how do we account for expert validation?

In your own pilot, record preparation, checking, correction and approval. Keep the quality standard consistent.

Treat assurance as part of performance

In my AI Ireland Podcast conversation with Cyril Treacy, Co-Founder and Chief Operating Officer of Disseqt AI, we explored governance, assurance and the importance of testing.

The episode distinguishes the policies that guide AI use from the evidence needed to assess how a system behaves. Listen to the conversation with Cyril Treacy

My practical takeaway for leaders is this: include reliability in the value calculation.

A system that produces more output but creates additional checking, errors or customer complaints may be moving work elsewhere.

Alongside time and cost, examine:

• Whether outputs meet the agreed standard.
• How often people need to intervene.
• What happens when information is missing.
• Whether errors are identified before they affect customers.
• Whether the system stays within its approved permissions.

My AI agent permissions checklist explores those boundaries in more detail.

Use leadership research to understand the conditions for success

AI Ireland’s earlier published 2026 research draws on responses from 130 AI leaders. Its published overview identifies integration, skills and governance among the challenges organisations face. Read the published research

That matters when designing a pilot.

If employees cannot access reliable information, a better prompt will not resolve the underlying problem. If nobody owns the process, a successful demonstration may never become a dependable way of working.

Survey findings help frame the questions. Evidence from your own process should determine the next decision.

Use the 5Ps to prepare the test

I use the 5Ps to connect an AI opportunity with the organisation’s readiness.

People: Who will use the system, review its work and own the outcome?

Process: Which task is changing? What happens when the normal sequence breaks down?

Platforms: Can the approved tools support the workflow and enforce the required permissions?

Proprietary Data: Is the information accurate, current and appropriate for this use?

Products & Services: What should improve for the customer or the business?

For example, an appointment assistant could be technically capable but still struggle with incomplete scheduling information or an unclear route to a staff member.

The useful question is whether the whole service works.

Turn the evidence into a 30-day pilot

For a frequent, manageable task, a month can provide an initial review point.

Days 1–5: Establish the baseline.
Record current effort, quality and completion time across several comparable tasks. Agree what an acceptable result looks like.

Days 6–10: Define the change.
Choose the step AI will support. Name the owner, approve the information and tools, and set the review boundaries.

Days 11–20: Test and record.
Track preparation, checking, corrections and exceptions. Compare similar tasks where possible.

Days 21–25: Investigate the differences.
Find out where the approach helps and where it struggles. Separate routine tasks from unusual cases.

Days 26–30: Decide.
Continue, adjust, stop or gather more evidence.

Thirty days will not suit every project. An infrequent process or an outcome such as customer retention may require a longer observation period. The review date should prompt an honest decision, not a premature success claim.

Distinguish released capacity from cash savings

If a pilot releases ten hours of staff time, ask what happens to those hours.

Does the team clear a backlog? Respond sooner? Improve the quality of its work? Handle more demand? Reduce paid overtime?

Each could have value, but they are different outcomes.

Include setup, training, subscriptions, review and maintenance costs. Avoid turning every minute saved into a financial return unless there is a defensible basis for doing so.

A useful leadership scorecard should show the benefit, the cost and the uncertainty.

What this perspective brings to a leadership workshop

The value of bringing together finalist projects, practitioner interviews and leadership research is the ability to examine an opportunity from several angles.

The project provides a concrete example. The conversation explores implementation and judgement. The research adds context about the challenges leaders report.

My role is to connect those perspectives to your organisation and help you decide what to do next.

Preparation can include a leadership readiness survey to establish your team’s experience and priorities. We can then use relevant examples to define a manageable test through the 5Ps.

The aim is to leave with a clear problem, a responsible owner and evidence you can use to make the next decision.

Discuss an AI leadership workshop for your team, or read what a workshop should deliver.