Outcomes Over Output: Measuring Whether a Transformation Is Working

Ask how an agile transformation is going and you will often hear about activity: how many teams were trained, how many coaches are in place, how many ceremonies are running, how velocity has trended. Those numbers are easy to collect. They rarely answer the question a board, a cabinet secretary or a chief executive actually cares about, which is whether customers, users and the organization are better off.

This guide shows how to build a measurement approach around outcomes, drawing on Scrum.org's Evidence-Based Management Guide, the measurement sections of the major scaling frameworks, and the metrics chapter of the U.S. Government Accountability Office's Agile Assessment Guide (GAO-24-105506).

Five kinds of measures

The Evidence-Based Management (EBM) Guide, updated in 2024, sorts measures into five categories:

• Inputs: things the organization spends money on.
• Activities: things people do, such as meetings, writing code or creating reports.
• Outputs: things the organization produces, such as releases, features and reports.
• Outcomes: desirable things a customer or user experiences, representing a capability they did not have before.
• Impacts: results the organization or its other stakeholders achieve when customers reach their desired outcomes, such as increased revenue or market share.

The guide's diagnosis is direct: "measuring activities and outputs is easy, while measuring outcomes is difficult." It adds that working more hours and delivering more features "does not necessarily lead to improved customer experiences." It also cautions that impacts can be improved without improving customer outcomes, but that doing so usually harms the organization over time.

Most transformation dashboards are built almost entirely from inputs, activities and outputs. That is the gap to close.

Start from goals, then form hypotheses

EBM describes three levels of goals. A Strategic Goal is a large, uncertain aspiration. Intermediate Goals indicate the organization is on the path toward it. Immediate Tactical Goals are the current focus of improvement work. Organizations then form hypotheses about what will move them toward their tactical goals, run experiments and measure the results.

The guide's public health example makes the distinction clear. If the strategic goal is to eradicate the effects of a disease, the goal is measured by how many people fall ill, not by vaccinating a certain percentage of the population. Vaccination may be a necessary activity, but it is not the goal.

GAO makes the same point in program terms. It says connections between strategic goals and objectives should be traceable to agile artifacts such as the road map and backlog, and that without that traceability the organization may not have the right information to make decisions about prioritization and re-planning.

Balanced lenses on value

The frameworks offer different but compatible lenses for balancing measures:

• EBM uses four Key Value Areas: Current Value, Unrealized Value, Time-to-Market and Ability to Innovate. It says the first two reflect customer outcomes, while the second two reflect the organization's capability to deliver them.
• The Scrum@Scale Guide does not require specific metrics but suggests that, at a minimum, organizations measure productivity, value delivery, quality and sustainability, giving team happiness as an example of sustainability.
• SAFe's Measure and Grow approach uses three measurement domains: outcomes, flow and competency.
• DORA's guide for business leaders says executives should measure software against its purpose. If it was built to cut response times, measure response times.

Whichever lens you choose, the principle is balance: something about value to customers, something about speed and flow, something about quality, and something about the health of the people doing the work.

What GAO expects from agile metrics

GAO's metrics chapter is useful even outside government because it describes what auditors look for. It lists six best practices: identify key metrics based on the program's agile framework; ensure metrics align with and prioritize organization-wide goals; establish and validate metrics early and align them with incentives; establish management commitment; commit to data-driven decision making; and communicate performance information frequently and efficiently.

Several details matter for leaders:

• Metrics should be quantifiable, meaningful, repeatable and consistent, and actionable.
• Organizations should distinguish product performance, such as code quality or user satisfaction, from development team performance.
• The value of a metric should exceed the effort needed to collect it.
• Management should set target values for critical metrics, such as how long it should take from program launch to deploying a minimum viable product.
• Forcing agile programs to report metrics designed for waterfall development, GAO warns, will not only impede agile adoption but also fail to provide accurate insight.

GAO's case studies illustrate the cost of gaps. In one, a program did not measure internal user satisfaction and, as GAO reported, limited its understanding of the value delivered with each release.

Balance metrics so they cannot be gamed

Any single number becomes a target. GAO advises against optimizing one metric, such as velocity, at the expense of others, such as quality. It notes that velocity is specific to a team, cannot be used to compare teams, and can be raised simply by inflating story point estimates without adding value.

GAO suggests counterbalancing combinations instead. For example, pair velocity with the number of stories that are rejected, defective or unfinished, so a team cannot look faster by cherry-picking easy work. DORA similarly recommends using several metrics, including some with healthy tension between them.

A one-page measurement plan

For example, imagine a hypothetical permitting office modernizing its online application process. A one-page plan might include:

• Strategic goal: residents get permits faster with fewer errors.
• Outcome measures: time from application to decision, and the share of applications returned for corrections.
• Delivery measures: lead time for changes and how often working software reaches users.
• Quality and health measures: defects found after release, and a short periodic team survey.
• Current experiment: a redesigned upload step, with the hypothesis that correction requests will fall.
• Review cadence: monthly, matching the release rhythm, with the authority to change course.

Notice what is missing: counts of people trained or ceremonies held. Those may be worth tracking for the transformation team, but they are not evidence that the transformation is working.

Key takeaways

• Separate inputs, activities and outputs from outcomes and impacts, and build your dashboard around the last two.
• Start with goals and hypotheses, and trace them to the road map and backlog as GAO recommends.
• Balance value, flow, quality and people measures rather than relying on any one number.
• Never compare teams by velocity; GAO says it is team-specific and easy to inflate.
• Review metrics on the same cadence as delivery, and be ready to act on what they show.

Sources

• Scrum.org, Evidence-Based Management Guide (2024)
• U.S. Government Accountability Office, Agile Assessment Guide: Best Practices for Adoption and Implementation, GAO-24-105506 (reissued with revisions December 15, 2023)
• Jeff Sutherland and Scrum Inc., The Scrum@Scale Guide, Version 2.1 (February 2022)
• Scaled Agile, Inc., Measure and Grow (updated September 8, 2025)
• Steve Fenton, Mark Kropf and Denali Lumma for DORA, How to empower software delivery teams as a business leader (updated April 19, 2024)
• Nathen Harvey for DORA, DORA's software delivery performance metrics (updated January 5, 2026)