Announcement · Testing

Patch, the patchcov hermit crab, mending a hole in its quilted shell

Announcing patchcov

patchcov answers one question precisely: which lines did this change add that no test ran? It is a command-line tool that reads the coverage report you already produce, and a GitHub Action that turns its answer into a pull-request comment and a gate. Neither needs an account, a token or a server.

  • patchcov scores the lines a change added, lists the untested ones as file:line, and exits non-zero when a gate fails. patchcov-action runs it on every pull request, with a baseline from the merge base.
  • That output is meant for automation as much as for people. An AI agent can loop on it, "add tests until the gate passes", without reading a dashboard.
  • The project-wide percentage is the wrong signal for a pull request. A change can leave a third of its new code untested and barely move the total.

Why another coverage tool

More and more code is written by agents in a loop: write, run the checks, read the result, fix, repeat. That loop is only as good as the signals it can read. Coverage tooling was built to give people a picture, and an agent needs an instruction.

Three things kept getting in the way:

patchcov is the smallest tool that fixes all three. It reads report files from disk, joins them to git diff, and prints a result that needs no human to interpret it.The mission, from the README: help projects that use agentic workflows keep their new code properly tested, with minimal friction.

Predict: a project has 4,000 executable lines at 92.00%. A pull request adds 6 executable lines and tests 4 of them. How far does the project total move?

From 92.00% to 91.96%, a move of −0.04 percentage points. (3,680 + 4) ÷ (4,000 + 6) = 91.96%. That is under the 0.05 pp tolerance patchcov uses for its headline delta, so the total renders as neutral ⚪. The patch coverage for the same change is 66.67%, with two lines named. Try it below.

Patch coverage, hands on

Patch coverage is the share of the added executable lines that some test ran. Click a line's mark to decide whether a test covers it, and watch both numbers.

  1. 40 pub fn parse_header(line: &str) -> Result<Header, Error> {
  2. 41 let line = line.trim();
  3. 42+ // A shebang is not a header.
  4. 43+ if line.starts_with("#!") {
  5. 44+ return Ok(Header::Shebang);
  6. 45+ }
  7. 46+ let Some((key, value)) = line.split_once(':') else {
  8. 47+ return Err(Error::MissingColon);
  9. 48+ };
  10. 49+ if key.is_empty() {
  11. 50+ return Err(Error::EmptyKey);
  12. 51+ }
  13. 52 Ok(Header::Field(key.into(), value.trim().into()))
  14. 53 }

Lines 42, 45, 48 and 51 are added but not executable: a comment and closing braces never appear in a coverage report, so they are not counted.

66.67%Patch coverage (4/6 new lines)
91.96%Project total ⚪ −0.04 pp, neutral
Patch: 66.67% (4/6 new lines covered)
Uncovered new lines: src/parser.rs:47, src/parser.rs:50
Error: patch coverage 66.67% is below the --fail-under-patch threshold of 80.00%
exit 1
The total barely moves however many lines you leave untested; patch coverage swings by 16.67 points per line and names each gap. “Loop” plays the agent’s part: it writes a test for the first line patchcov lists, runs the gate again, and stops when it exits 0.

How patchcov works

patchcov is a join between two things you already have: a per-line coverage report, and the lines git says the change added.

Head reportlcov · Cobertura · … Baseline reportoptional git diffmerge-base..HEAD Parsemap paths to git Analyzeline by line Reportmd · JSON · YAML Exit code0 pass · 1 gate
Notice that nothing leaves the machine: every input is a file or a git object, and every output is stdout or an exit code.

Read the report you already have. lcov, llvm-cov JSON, Cobertura, JaCoCo XML and Go coverprofiles are detected from the content, so one command serves Rust, Go, Java, Kotlin, TypeScript, Python, C++ and more. Paths are made repo-relative by a fixed, configurable pipeline, with no guessing: a wrong guess would attribute coverage to the wrong file without a sign.

Ask git what the change added. By default that is everything from the merge base with the default branch to HEAD, so lines that reached main while your branch was open are not counted against you.

Score each added line. It is covered, uncovered, or not executable. Lines the report doesn't mention, such as comments and braces, are left out of the denominator.

Optionally, compare with a baseline. A report from the merge base adds per-file deltas and indirect changes: lines whose coverage flipped without their text changing. These are scoped to the files the diff touched, because two separate test runs always differ a little elsewhere.

Print, then exit. The markdown is a ready-made pull-request comment; JSON and YAML follow a documented schema. The exit code is 0, or 1 when --fail-under-patch or --fail-under-lines fails, and 2–8 for a usage, report, marker, config, git or path problem. So a script can tell a failed gate from a broken run.

Built for the agent loop

Because the answer is a list of file:line and an exit code, an agent can act on it without a person in between.

Agent Test run patchcov tests + coverage head.lcov diff --fail-under-patch 80 exit 1 · parser.rs:47, :50 write 2 tests tests + coverage diff --fail-under-patch 80 exit 0
The agent never reads a chart. Each round trip is a command, a list of lines, and a yes or no.

The agent runs the test suite under any coverage tool that writes a supported report, such as cargo llvm-cov, go test -coverprofile, coverage.py or JaCoCo.

patchcov exits 1 and names the gaps. With -o json, they arrive as uncovered_new_lines, and --error-format json puts the cause on stderr as an object too.

The agent writes tests for exactly those lines and runs both steps again. The same command works on a laptop, in a sandbox and in CI, so the agent can pass the gate before it pushes.

A signal you can trust

A gate that cries wolf gets ignored, by people and agents alike. patchcov is quiet where coverage is noisy and loud where it is broken.

// patchcov: coverage tolerate reason="AVX2 path; runners differ in CPU"
fn popcount_avx2(words: &[u64]) -> u64 { /* … */ }
// patchcov: coverage end
Go deeper: why tolerated lines still count in the patch

A tolerate region masks deltas: each tolerated line is scored with its status in the baseline, so a flap reads as no change. The displayed percentages stay the real, measured values. An added line has no baseline counterpart, so it keeps its real status and stays in the patch-coverage denominator: new code should still be tested, even if it flaps later. To remove lines from every number, use ignore instead.

patchcov-action: the same answer on every pull request

The command answers the question; the action wires it into GitHub. It installs a cached patchcov binary, finds the right baseline, posts one sticky comment, and applies the gates last.

main CI runon push to main Baseline artifactone per commit PR CI runon pull_request patchcov diffhead vs baseline Sticky commentposted first Gatesapplied last publish merge base’s
Notice that the baseline is pinned to the pull request’s fork point, not to the tip of main, so the deltas belong to this pull request alone.

Every push to main publishes a baseline: that commit's per-line report, as an artifact named coverage-baseline.

On a pull request, produce the head report. In fat mode (the default) the action runs cargo-llvm-cov for a Rust workspace. In thin mode you bring a report from any language, and the action only diffs, comments and gates.

Fetch the baseline for the merge base. If that commit has none, it tries up to 10 first-parent ancestors, nearest first, and says so in the comment. As a last resort in fat mode, it rebuilds coverage at the merge base in a git worktree. A miss is a warning: patch coverage never needs a baseline.

Post one comment and keep it current. It has the patch coverage, the per-file deltas and the uncovered file:line list, and is updated in place on every push.

Gate last. fail-under-patch and fail-under-lines run after the comment posts, so a failing pull request still gets its explanation.

The action also handles the topologies real projects grow into: sharded test runs merged into one result, merge queues, and files a CI runner can't execute.

jobs:
  coverage:
    runs-on: ubuntu-latest
    permissions:
      contents: read
      actions: read            # download the merge base's baseline
      pull-requests: write     # post the coverage comment
    steps:
      - uses: actions/checkout@v7
        with:
          fetch-depth: 0       # full history, so the merge base resolves
      - uses: action-works/patchcov-action@v2
        with:
          fail-under-patch: 80
      - run: pytest --cov --cov-report=lcov:coverage.lcov
      - uses: action-works/patchcov-action@v2
        with:
          run-coverage: false
          report: coverage.lcov
          fail-under-patch: 80
  coverage:
    needs: shard               # a failed shard fails this job
    steps:
      - uses: actions/checkout@v7
        with:
          fetch-depth: 0
      - uses: actions/download-artifact@v8
        with:
          pattern: coverage-shard-*
          merge-multiple: true
          path: shards
      - uses: action-works/patchcov-action@v2
        with:
          run-coverage: false
          shard-reports: shards/shard-*.lcov
          fail-under-patch: 80
cargo install patchcov --locked     # or: cargo binstall patchcov
cargo llvm-cov --lcov --output-path head.lcov
patchcov diff --report head.lcov --fail-under-patch 80

When to choose patchcov

patchcov is a signal, not a dashboard. Pick it for the question it answers, and keep the other tools for the questions they answer.

patchcov

A command and an action that score the change, in your own CI.

  • Answers "what did this change leave untested, and where?"
  • Machine-readable output and distinct exit codes, ready for agents
  • No service, account or upload token: the gate works whenever your CI does
  • Runs the same locally, in a sandbox and in CI
  • Built for noisy coverage and sharded runs
  • No history, trend graphs or cross-repository views
  • Line coverage; branch scoring is opt-in and limited to lcov and Cobertura

A hosted coverage service

You upload reports; the service keeps them and comments on pull requests.

  • Coverage history and trends over months
  • Dashboards and views across many repositories
  • A third party between your tests and your merge button
  • Reports leave your CI; tokens to manage
  • The result is learned after pushing, not before

A code-quality platform

Coverage as one input to a wider quality gate.

  • Static analysis, security scanning and coverage in one place
  • Another server to run or to depend on
  • Much more to set up than a coverage gate needs
Choose by the question you need answered.
If you need…Choose
To know whether a pull request added untested code, and wherepatchcov
A gate an agent or script can act on, in a looppatchcov
A coverage gate that can't be blocked by someone else's outagepatchcov
One combined result from a sharded test runpatchcov
Coverage history, trend graphs, or views across repositoriesA hosted service, alongside patchcov
Per-branch percentages, or JaCoCo and llvm-cov branch dataYour coverage tool's own report
Static analysis and security scanning as wellA code-quality platform

What it doesn't tell you

Get started

  1. Install. cargo install patchcov --locked, cargo binstall patchcov, or a prebuilt binary for Linux, macOS or Windows from the GitHub releases.
  2. Run it on your branch. Produce a per-line report from your test run, then patchcov diff --report head.lcov.
  3. Add the gate. Add action-works/patchcov-action@v2 to your pull-request workflow with fail-under-patch, and let the comment and the exit code do the rest.

Further reading