Automation KPIs: Measuring What Actually Matters

Updated May 2026. Rewritten with the actual automation KPIs we use with clients, including the ones we have dropped and the ones we now defend in board meetings.

Automation KPIs sit in a tricky spot. They have to convince a CFO that a programme is paying back, prove to an ops team that the work was worth doing, and not corrupt the behaviour of the people building the automations along the way. Most automation programmes start with the wrong set of metrics and quietly switch to telling stories instead. The right set of automation KPIs makes the storytelling unnecessary.

We run a small AI and automation consultancy in Brisbane and have shipped enough automations across healthcare, recruitment, and finance to have a strong opinion on what to measure and what to ignore. This piece is the working set of automation KPIs we use, the categories they fall into, and the gotchas we have learned the hard way.

For the upstream context on picking which processes to automate at all, our note on business process mapping techniques is the right starting point. This piece picks up after the automation is live.

Why Most Automation KPIs Quietly Fail

The default automation KPIs that show up in every consultancy deck look reasonable on paper. Hours saved. Tickets automated. Processes digitised. The problem is that none of these survive contact with a sceptical finance team.

“Hours saved” sounds great until the CFO asks where those hours went. If the answer is “they got reassigned to other work”, that is a productivity gain, but it is not money the business has back. If the answer is “we did not backfill a role when someone left”, that is real money. Both deserve to be counted, but they need to be counted differently.

“Tickets automated” is even worse. It rewards volume over impact. A team can automate 800 trivial requests and miss the one process that was eating two FTE worth of time. The KPI looks good. The business outcome is the same.

The set we now recommend covers four things: business value, automation reliability, build velocity, and risk. Each of those needs its own KPI. None of them can be replaced by a count of “things done”.

The Four Categories of Automation KPIs

Every working automation programme tracks at least one KPI in each of these four categories. Pick fewer than this and you will get a blind spot. Pick more and people stop reading the dashboard.

Business Value Automation KPIs

The numbers that prove the programme is paying for itself. Cost saved, revenue protected, throughput increased, SLAs met. These are the ones the CFO cares about. They are also the hardest to attribute cleanly because the business changes around the automation.

Reliability Automation KPIs

The numbers that prove the automation is doing the work it claims to be doing. Success rate, error rate, mean time to recovery, percentage of runs that needed human escalation. Most teams skip these and discover six months later that 40 percent of “automated” transactions are actually being patched up by someone in ops.

Build Velocity Automation KPIs

How fast the team is shipping new automations and how many they retire. New process automations live, average time from idea to production, technical debt cleanups. These tell you whether the automation team is healthy or whether they are slowly suffocating under their own backlog.

Risk and Compliance Automation KPIs

The numbers that prove your automations are not going to make tomorrow’s incident review interesting. Number of automations with documented owners, percentage with run logs retained for the compliance period, percentage that have been reviewed in the last 12 months. These are boring and unfashionable and they will save your programme when something breaks.

Automation KPIs for Business Value

The single most important automation KPI in this category is the one that can be defended in a finance meeting without arm-waving. Three that work in practice:

Cost Avoided (Not Cost Saved)

“Saved” implies money came back. “Avoided” is honest: we did not have to hire the extra two FTE we would have needed at this volume. Cost avoided is calculated by taking the headcount the team would have needed to handle current volume without the automation, multiplying by fully loaded cost, and subtracting the automation’s running cost. A typical mid-sized invoice automation we have shipped avoids $80,000 to $150,000 AUD per year in fully loaded ops cost.

Throughput Per FTE

How many of the unit of work (invoices, claims, applications, leads) the team handles per full-time-equivalent. This metric survives reorgs, headcount changes, and seasonality. A 2x improvement in throughput per FTE over 12 months is a serious automation win and survives any conversation with finance.

Cycle Time Reduction

How long it takes one unit of work to go from triggered to done, end to end. The killer metric for customer-facing automations. We have shipped onboarding automations that took application-to-active from 9 days to under 6 hours. That kind of cycle time win usually unlocks revenue you could not previously bill.

Automation KPIs for Reliability

The fastest way to lose credibility for an automation programme is to claim 100 percent success when the team is quietly patching up edge cases. Reliability automation KPIs are how you stay honest.

• Straight-through processing rate. The percentage of runs that completed without any human touch. A healthy mature automation sits between 85 and 97 percent. Below 80 percent, you are running a workflow tool with humans plugging the holes, and you should be honest about that.
• Error rate. The percentage of runs that failed outright. Should be under 2 percent for a stable automation. Spikes above that mean something upstream changed.
• Mean time to recovery. When an automation breaks, how long until it is back. For non-critical workflows, under 24 hours. For revenue-touching ones, under 2 hours. Track this and you will quickly see which automations need more attention than they are getting.
• Exception backlog age. How long the oldest unresolved exception has been waiting. A growing exception backlog is the canary in the coal mine for an automation programme. We have seen it predict major outages weeks in advance.

If you are tracking only one reliability number, make it straight-through processing rate. It is the one that catches the silent erosion of automation value.

Automation KPIs for Build Velocity

Automation programmes die from two directions. Too slow, and the backlog of waiting processes turns into a credibility problem. Too fast, and the team ships fragile automations that erode the previous category. The build velocity automation KPIs we use:

• Average time from request to production. Idea to live. For straightforward workflows we aim for 2 to 4 weeks. For complex AI-touching builds, 6 to 10 weeks. If this number creeps past 12 weeks for most builds, something in the pipeline is broken.
• Net new automations per quarter. A simple count. Most teams report 4 to 8 per quarter at steady state. A team with fewer than 2 is either understaffed or stuck in approval limbo.
• Automations retired per quarter. The one nobody tracks. Automations have a shelf life. The process changes, the source system changes, the business logic changes. A healthy programme retires 1 to 2 automations per quarter. A programme that retires zero is accumulating debt.

The single biggest debugging session we have lost time on in a client engagement was an automation that had been live for two years past the point where the underlying process changed. Nobody had retired it. The data was wrong for 18 months before anyone noticed.

Automation KPIs for Risk and Compliance

These automation KPIs are the ones nobody wants to look at until the day they urgently do. We recommend three:

• Percentage of automations with a named owner. Should be 100 percent. Anything below that is an orphan automation, which is the most dangerous kind. When it breaks, nobody knows.
• Percentage of automations with run logs retained for the compliance period. 7 years for financial data in Australia under APRA, 10 years for some health records, varies elsewhere. Should be 100 percent for anything that touches regulated data.
• Percentage reviewed in the last 12 months. An annual review forces the owner to check that the automation still does what it is supposed to do. We do not let this fall below 80 percent without raising a flag.

If you are running automations that touch APP-regulated data, APRA-regulated data, or anything covered by My Health Records, these three KPIs are not optional. We have walked into post-incident reviews where the absence of these numbers was the biggest single finding.

How to Roll Up Automation KPIs to ROI

Eventually, someone in finance will ask for a single ROI number. Resist the temptation to make one up. The most defensible rollup we use:

1. Total cost avoided per year (from business value KPIs)
2. Less the running cost of the automation stack (licences, infrastructure, ops time)
3. Less the team’s fully loaded cost
4. Divided by the total programme investment to date

For a mid-sized programme running for two years, we typically see ROI in the 250 to 600 percent range when calculated this way. Anything claiming over 1,000 percent should be treated with suspicion. Our companion piece on calculating ROI of business process automation walks through this in more detail.

Automation KPIs Vary by Programme Maturity

The set of automation KPIs that matter changes as the programme matures. Tracking the wrong category at the wrong time is the most common mistake we see.

Pilot Phase (Zero to Six Months)

Track cycle time reduction and straight-through processing rate. That is it. You are proving the technology works on a specific problem. Trying to track cost avoided in month two is dishonest because you do not have enough data. Throughput per FTE will move randomly because you are still tuning.

Scale-Up Phase (Six to Eighteen Months)

Add cost avoided, throughput per FTE, and build velocity. You now have enough data to start comparing baselines. This is where the programme either earns its budget or quietly gets defunded.

Mature Phase (Eighteen Months Onwards)

Add the risk and compliance category. Now you have enough automations live that “all of them have owners” stops being trivially true. This is also when the maintenance cost of old automations starts to become a real number, so the “automations retired per quarter” KPI starts mattering.

When Automation KPIs Are the Wrong Thing to Measure

Automation KPIs are good for steady-state programmes shipping incremental wins. They are bad for one-off transformational projects, brand-new AI agent builds where the right behaviour is still being discovered, or initiatives where the goal is to change a customer experience rather than make a back-office cheaper.

For those, use qualitative measures: post-launch customer feedback, support ticket sentiment, NPS, employee feedback on whether the work has actually got better. These are softer and harder to defend in a board pack, but they are what the actual outcome looks like.

If you are exploring whether to build internal automations at all, our work on AI agent development covers the cases where agentic automations replace whole categories of the kind of work the KPIs above are designed to count. Different problem, different metrics. Worth a chat if you are not sure which side of the line you are on.

Frequently Asked Questions About Automation KPIs

What are the most important automation KPIs to track?

The four we recommend across every programme: cost avoided (business value), straight-through processing rate (reliability), average time from request to production (build velocity), and percentage of automations with named owners (risk). If you can only track three, drop the build velocity one.

How do you measure ROI on automation?

Take cost avoided per year, subtract running cost of the automation and the team’s loaded cost, and divide by the programme investment to date. For a mid-sized programme running 18 to 24 months, this typically lands in the 250 to 600 percent range. Anything above 1,000 percent should be checked carefully.

How much does it cost to track automation KPIs properly?

For a small programme, $0 to $200 AUD per month using existing tools (workflow logs, a Google Sheet, a simple dashboard). For a mid-sized programme, $500 to $2,000 AUD per month for BI tooling plus integration work. The cost is mostly the time of whoever maintains the dashboard, not the tooling.

What is a good straight-through processing rate?

For a mature automation, 85 to 97 percent. Below 80 percent means humans are doing a lot of patching up in the background. Above 97 percent often means you are not catching edge cases that should be flagged for human review, which is a different risk.

Should automation KPIs be tied to team bonuses?

Almost never. Tying automation KPIs to compensation creates incentives to ship fragile automations that look great in month one and rot quietly. If you reward automation teams, do it on qualitative output and customer satisfaction, not on count of automations shipped.

How often should automation KPIs be reviewed?

Weekly at the team level for reliability KPIs (so you catch breakage). Monthly at the leadership level for business value and build velocity. Quarterly for the full programme review including risk and compliance. Annual is too slow to fix anything.

How do you measure success of AI automation versus traditional automation?

The reliability KPIs are the same. The risk KPIs gain an extra one: hallucination rate, which is the percentage of outputs that need to be corrected because the model fabricated something. For human-in-the-loop AI automations, also track the override rate, which tells you how often the human disagreed with the model’s suggestion.

What automation KPIs do CFOs actually care about?

Two. Cost avoided per year (with the workings shown) and throughput per FTE. Everything else is operational detail they trust the team to manage. If you can defend those two numbers in a 30-minute meeting with three slides, you have a programme that will keep its budget.

If you want to design a working set of automation KPIs for your programme, or you suspect your current set is measuring activity rather than value, get in touch. We have built enough of these scorecards to know which ones hold up under finance scrutiny.