Metrics & Reporting

Automation ROI measures the return on investment from test automation — comparing the upfront and ongoing cost of building and maintaining automated tests against the time saved versus manual execution over the automation's lifetime. It answers whether automating a given area was actually worth it.

Code coverage measures the percentage of source code — lines, branches, or functions — that gets executed when a test suite runs, giving a quantitative signal for how much of the codebase tests actually touch. It's a coverage-of-execution metric, not a measure of whether that execution actually verified anything.

Cyclomatic complexity is a metric that counts the number of independent linear paths through a piece of code, based on its conditional branches — a higher number means more possible execution paths, and generally more test cases needed to achieve meaningful branch coverage of that code.

Defect density measures the number of confirmed defects relative to the size of the software — typically defects per thousand lines of code, or per feature/module — allowing comparison of quality across different components, releases, or teams on a normalized basis rather than raw defect counts alone.

Defect leakage measures the percentage of defects that escape one testing phase and are only caught in a later phase — found in production instead of QA, or found in UAT instead of system testing — indicating where the testing process itself has gaps worth investigating.

Defect Removal Efficiency (DRE) measures what percentage of total defects — those found during testing plus those found later in production — were actually caught before release. A DRE of 90% means 90% of all eventual defects were caught pre-release, and 10% escaped to production.

Escaped defects are bugs discovered in production, after release, that testing failed to catch beforehand. Tracking this count and its trend over time is one of the clearest, most outcome-focused signals of whether a testing process is actually working, since it measures real-world impact, not just testing activity.

Mean Time to Detect (MTTD) measures the average time between a defect or issue being introduced and it actually being discovered — whether by testing, monitoring, or a user report. A shorter MTTD generally means less time for a problem to cause real-world impact before anyone notices.

Mean Time to Repair (MTTR) measures the average time between a defect being detected and it being fully resolved — a key indicator of how efficiently a team responds to and fixes known issues, especially critical production incidents.

Pass/fail rate is the percentage of executed test cases that passed versus failed in a given test run or cycle — one of the simplest, most immediately understandable testing metrics, commonly used as a quick health signal for a build or release.

Requirements coverage measures what percentage of documented requirements have at least one associated test case verifying them, exposing gaps where a requirement exists but nothing actually checks whether it's been correctly implemented.

Test case effectiveness measures how well a test suite actually catches real defects — commonly evaluated through mutation testing (do the tests catch deliberately introduced bugs) or by tracking how many escaped defects existing tests failed to catch despite covering the affected area.

Test execution rate measures how many test cases have actually been run out of those planned for a given cycle, expressed as a percentage or a count — a basic progress indicator during an active testing cycle, distinct from pass/fail rate, which measures the outcome of those executions.