Gating Reconciliation Deploys with GitHub Actions

A change to matching logic is a change to a financial control, and it should not reach production the way a UI tweak does. This page builds a GitHub Actions pipeline that treats a reconciliation deploy as a controlled release: it runs the control test suite, replays the candidate matcher against a frozen shadow dataset, compares the resulting match rate against the last known-good baseline with Decimal arithmetic inside a defined regression bound, and only then allows promotion through a protected environment that requires named reviewers. Every decision the gate makes — pass, fail, or bypass attempt — is written to a structured audit log carrying a trace_id, source_hash, and match_decision, so a SOX auditor can reconstruct exactly why a given deploy was allowed through months later. It sits alongside CI/CD Gating for Financial Controls inside Compliance, Audit Trails & Financial Controls as the concrete implementation of that control point.

Prerequisites

Step 1 — Define the workflow graph and required jobs

The workflow separates three concerns into distinct jobs: control-tests runs the deterministic unit and control-assertion suite, shadow-replay runs the candidate matcher against the frozen dataset and produces a match summary artifact, and gate consumes both and decides whether deploy is allowed to run. deploy targets a GitHub Environment with required reviewers and a protected branch rule, so the human approval step is enforced by GitHub itself, not by application code that a misconfigured script could skip.

yaml
name: reconciliation-deploy-gate

on:
  pull_request:
    branches: [main]
  push:
    branches: [main]

jobs:
  control-tests:
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v4
      - uses: actions/setup-python@v5
        with:
          python-version: "3.12"
      - run: pip install -r requirements.txt
      - run: pytest tests/control_assertions -q --junitxml=control-results.xml
      - uses: actions/upload-artifact@v4
        with:
          name: control-results
          path: control-results.xml

  shadow-replay:
    needs: control-tests
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v4
      - uses: actions/setup-python@v5
        with:
          python-version: "3.12"
      - run: pip install -r requirements.txt
      - run: python -m recon.shadow_replay --sample-size "${SHADOW_SAMPLE_SIZE:-5000}" \
              --dataset fixtures/shadow_dataset.parquet \
              --out candidate_summary.json
        env:
          SHADOW_SAMPLE_SIZE: 5000
      - uses: actions/upload-artifact@v4
        with:
          name: candidate-summary
          path: candidate_summary.json

  gate:
    needs: [control-tests, shadow-replay]
    runs-on: ubuntu-latest
    outputs:
      decision: ${{ steps.evaluate.outputs.decision }}
    steps:
      - uses: actions/checkout@v4
      - uses: actions/download-artifact@v4
        with:
          name: candidate-summary
      - uses: actions/setup-python@v5
        with:
          python-version: "3.12"
      - run: pip install -r requirements.txt
      - id: evaluate
        run: python -m recon.deploy_gate \
              --candidate candidate_summary.json \
              --baseline fixtures/baseline_summary.json \
              --source-hash "${{ github.sha }}" \
              --trace-id "${{ github.run_id }}-${{ github.run_attempt }}"

  deploy:
    needs: gate
    if: needs.gate.outputs.decision == 'ALLOW'
    runs-on: ubuntu-latest
    environment: production-reconciliation   # required reviewers configured in repo settings
    steps:
      - uses: actions/checkout@v4
      - run: ./scripts/deploy_reconciliation_service.sh

The deploy job’s environment: production-reconciliation line is what gives the required-reviewer rule teeth: GitHub blocks the job from starting until the configured approvers sign off, and that approval is itself logged in the repository’s deployment history, independent of anything the gate job wrote.

Step 2 — Load candidate and baseline match-rate summaries

The shadow-replay job produces a summary that the gate script parses into a strictly typed model. Match rates are stored as Decimal strings in the JSON artifact and parsed as Decimal immediately — never through float() — so the regression comparison in the next step never accumulates binary floating-point error.

python
from decimal import Decimal, InvalidOperation
from pydantic import BaseModel, field_validator
import json

class MatchSummary(BaseModel):
    build_ref: str
    sample_size: int
    matched_count: int
    match_rate: Decimal
    tolerance_config_hash: str

    @field_validator("match_rate", mode="before")
    @classmethod
    def parse_decimal_rate(cls, v: object) -> Decimal:
        try:
            rate = Decimal(str(v))
        except InvalidOperation as exc:
            raise ValueError(f"match_rate is not a valid Decimal: {v!r}") from exc
        if not (Decimal("0") <= rate <= Decimal("1")):
            raise ValueError(f"match_rate out of bounds: {rate}")
        return rate

def load_summary(path: str) -> MatchSummary:
    with open(path, encoding="utf-8") as fh:
        return MatchSummary.model_validate(json.load(fh))

tolerance_config_hash is a hash of the matching-tolerance configuration — the same date-window and amount-tolerance rules the matcher used to produce the summary — so the gate can detect if the candidate build silently changed its tolerance bounds without a corresponding threshold-review sign-off.

Step 3 — Compare match rate within bound and detect tolerance drift

The core comparison is two checks against Decimal bounds: the candidate’s match rate must not fall below the baseline by more than max_matchrate_regression, and the tolerance configuration hash must match the baseline’s unless an explicit override is present. Both checks run before any deploy decision is made.

python
from dataclasses import dataclass

@dataclass(frozen=True)
class GateResult:
    decision: str          # ALLOW | BLOCK
    reason_code: str | None
    regression: Decimal

def evaluate_gate(
    candidate: MatchSummary,
    baseline: MatchSummary,
    max_matchrate_regression: Decimal = Decimal("0.0050"),
    tolerance_override_hash: str | None = None,
) -> GateResult:
    regression = baseline.match_rate - candidate.match_rate

    if regression > max_matchrate_regression:
        return GateResult("BLOCK", "MATCHRATE_REGRESSION", regression)

    drift = candidate.tolerance_config_hash != baseline.tolerance_config_hash
    override_matches = tolerance_override_hash == candidate.tolerance_config_hash
    if drift and not override_matches:
        return GateResult("BLOCK", "TOLERANCE_DRIFT", regression)

    return GateResult("ALLOW", None, regression)

regression is a signed Decimal — a negative value means the candidate’s match rate actually improved on the baseline, which is always allowed regardless of max_matchrate_regression. The check is intentionally one-sided: gates that block improvements as readily as regressions train engineers to route around them.

Step 4 — Emit the structured audit log and fail the job deterministically

The gate’s decision is meaningless to an auditor without the context that produced it. The final step logs the full evaluation — including the trace_id passed in from the workflow’s github.run_id, the source_hash from github.sha, and the match_decision — then sets a non-zero exit code and a GitHub Actions step output so the deploy job’s if: condition can react to it.

python
import logging
import os
import sys
from datetime import datetime, timezone

log = logging.getLogger("recon.deploy_gate")

def run_gate(candidate_path: str, baseline_path: str, source_hash: str, trace_id: str) -> int:
    candidate = load_summary(candidate_path)
    baseline = load_summary(baseline_path)
    result = evaluate_gate(candidate, baseline)

    log.info(
        "deploy_gate.evaluated",
        extra={
            "trace_id": trace_id,
            "source_hash": source_hash,
            "match_decision": result.decision,
            "reason_code": result.reason_code,
            "regression": str(result.regression),
            "candidate_match_rate": str(candidate.match_rate),
            "baseline_match_rate": str(baseline.match_rate),
            "evaluated_at": datetime.now(timezone.utc).isoformat(),
        },
    )

    output_path = os.environ.get("GITHUB_OUTPUT")
    if output_path:
        with open(output_path, "a", encoding="utf-8") as fh:
            fh.write(f"decision={result.decision}\n")

    if result.decision != "ALLOW":
        print(f"::error::deploy gate blocked — {result.reason_code} (regression={result.regression})")
        return 1
    return 0

if __name__ == "__main__":
    import argparse
    parser = argparse.ArgumentParser()
    parser.add_argument("--candidate", required=True)
    parser.add_argument("--baseline", required=True)
    parser.add_argument("--source-hash", required=True)
    parser.add_argument("--trace-id", required=True)
    args = parser.parse_args()
    sys.exit(run_gate(args.candidate, args.baseline, args.source_hash, args.trace_id))

Because run_gate returns a real process exit code, the gate job in the workflow fails outright on a BLOCK decision — there is no code path where a blocked gate can still report success, which closes the door on a GATE_BYPASS scenario where a step silently swallows a non-zero return.

Configuration boundary table

Parameter Default Valid range Notes
max_matchrate_regression 0.0050 0.0001–0.0200 Decimal fraction; candidate rate below baseline minus this triggers MATCHRATE_REGRESSION
tolerance_drift_bound 0 0–1 Hash-equality check by default; 1 allows a signed override hash to pass
shadow_sample_size 5000 500–50000 Transactions replayed per gate run; larger samples reduce variance in the rate estimate
required_reviewers 2 1–6 Enforced by the GitHub Environment protection rule, not the gate script
protected_env production-reconciliation environment name Must match an Environment configured with reviewer and branch restrictions

Verification and testing

A fixture pair — one baseline summary and one deliberately regressed candidate — lets the gate’s boundary behavior be asserted without touching GitHub Actions at all.

python
def test_gate_blocks_regression_beyond_bound():
    baseline = MatchSummary(
        build_ref="main@abc123",
        sample_size=5000,
        matched_count=4900,
        match_rate=Decimal("0.9800"),
        tolerance_config_hash="cfg-9f2e",
    )
    candidate = MatchSummary(
        build_ref="pr-482@def456",
        sample_size=5000,
        matched_count=4830,
        match_rate=Decimal("0.9660"),
        tolerance_config_hash="cfg-9f2e",
    )
    result = evaluate_gate(candidate, baseline, max_matchrate_regression=Decimal("0.0050"))
    assert result.decision == "BLOCK"
    assert result.reason_code == "MATCHRATE_REGRESSION"
    assert result.regression == Decimal("0.0140")

def test_gate_allows_within_bound():
    baseline = MatchSummary(
        build_ref="main@abc123", sample_size=5000, matched_count=4900,
        match_rate=Decimal("0.9800"), tolerance_config_hash="cfg-9f2e",
    )
    candidate = MatchSummary(
        build_ref="pr-483@aa11bb", sample_size=5000, matched_count=4885,
        match_rate=Decimal("0.9770"), tolerance_config_hash="cfg-9f2e",
    )
    result = evaluate_gate(candidate, baseline, max_matchrate_regression=Decimal("0.0050"))
    assert result.decision == "ALLOW"

Run these as part of control-tests, not gate, so a broken gate implementation itself fails the pipeline before it ever gets a chance to evaluate a real candidate. In the live workflow, keep fixtures/baseline_summary.json updated by promoting the most recent ALLOW decision’s candidate summary to baseline as a separate, reviewed commit — never by letting the gate job overwrite it automatically.

Troubleshooting

  • GATE_BYPASS — a deploy reached production without a passing gate job run. Root cause: a branch protection rule that lists deploy as a required check but not gate, so a direct push or an admin merge can skip evaluation entirely. Fix: mark gate itself as a required status check on the protected branch, and require deploy to declare needs: gate as shown in Step 1.
  • MATCHRATE_REGRESSION — the gate blocks a deploy that “looks fine” in manual testing. Root cause: the shadow dataset sample is small enough that normal variance crosses max_matchrate_regression. Fix: increase shadow_sample_size or widen the bound slightly, but only through a reviewed config change — never by rerunning the job until it passes.
  • TOLERANCE_DRIFT — the gate blocks a build that only touched unrelated code. Root cause: a dependency bump or a formatting change altered how the tolerance config is serialized, changing tolerance_config_hash without changing actual matching behavior. Fix: hash the parsed tolerance values, not the raw config file bytes, so cosmetic changes don’t register as drift.
  • MISSING_APPROVAL — deploy stays queued indefinitely after gate reports ALLOW. Root cause: the production-reconciliation environment’s required-reviewer list references a team or user with no active members. Fix: audit the environment’s protection rules in repository settings and keep required_reviewers staffed with at least two people who are not the change’s author.
  • GATE_BYPASS (variant) — the gate step reports success despite a Python exception. Root cause: a try/except around run_gate that logs the error but doesn’t propagate a non-zero exit. Fix: let exceptions inside the gate script terminate the process; only evaluate_gate’s explicit BLOCK/ALLOW result should determine the exit code, as in Step 4.

Part of CI/CD Gating for Financial Controls within Compliance, Audit Trails & Financial Controls.