Building Deterministic Composite Keys for Transaction Hashing in Python

The same wire transfer usually shows up twice — once from the bank’s settlement API with a nanosecond-precision timestamp and an upper-case reference, once from the ERP’s general-ledger export with a date-only value date and a lower-case, trailing-whitespace reference. Both rows describe the identical economic event, but only if the composite key built from their fields reduces to the same bytes will Exact Match & Hash Comparison treat them as one. Get the key construction wrong and the gate at the front of Transaction Matching Algorithms & Logic fails open silently: nothing crashes, the digest index simply never records a hit, and a transaction that should have matched with mathematical certainty is pushed into the slower tolerance and fuzzy stages instead. This page is the narrow how-to for that one piece — the exact sequence of canonicalisation, ordering, and serialisation that turns two differently-formatted records of the same event into one deterministic source_hash.

Prerequisites

Step 1 — Define and freeze the canonical field contract

The fields that compose the key, and their order, are a versioned contract, not an implementation detail. Pick the smallest set of fields that is economically sufficient to identify the event — typically currency, amount, value date, and a reference key — and store both the tuple and a version tag in configuration so a later change is a deliberate, logged migration rather than a silent re-keying of every digest in the index.

python
from __future__ import annotations

from typing import Final

# Fixed order. Reordering this tuple changes every downstream digest —
# treat any change as a versioned migration, never a hotfix.
COMPOSITE_KEY_FIELDS: Final[tuple[str, ...]] = (
    "currency", "amount", "value_date", "reference",
)
COMPOSITE_KEY_VERSION: Final[str] = "v2"
FIELD_SEPARATOR: Final[str] = "|"

Step 2 — Canonicalise each field independently

Each field gets its own pure, side-effect-free normaliser. The amount is quantised as a Decimal at a fixed precision so 19.9 and 19.90 never diverge; the currency is upper-cased against ISO 4217; the timestamp is floored to a UTC date so a 23:59:59 posting on one side and a 00:00:01 posting on the other, separated only by settlement-clock skew, still collapse to the same day; the reference is trimmed, casefolded, and NFC-normalised so "INV-2201 " and "inv-2201" are indistinguishable.

python
import unicodedata
from datetime import datetime, timezone
from decimal import Decimal, ROUND_HALF_UP, InvalidOperation

ISO_4217_ALIASES = {"US$": "USD", "€": "EUR", "£": "GBP"}

def canonical_amount(raw: str | Decimal, precision: int = 2) -> Decimal:
    """Quantise to ledger-native precision. Never accepts float input."""
    if isinstance(raw, float):
        raise TypeError("FLOAT_IN_KEY: amount must be Decimal or str, not float")
    try:
        return Decimal(str(raw)).quantize(
            Decimal(f"1e-{precision}"), rounding=ROUND_HALF_UP
        )
    except InvalidOperation as exc:
        raise ValueError(f"unparseable amount: {raw!r}") from exc

def canonical_currency(raw: str) -> str:
    code = raw.strip().upper()
    return ISO_4217_ALIASES.get(code, code)

def canonical_date(ts: datetime, granularity: str = "day") -> str:
    """Floor a UTC timestamp to a fixed granularity, returned as ISO-8601."""
    if ts.tzinfo is None:
        raise ValueError("TZ_NOT_NORMALISED: timestamp must be timezone-aware")
    ts_utc = ts.astimezone(timezone.utc)
    if granularity == "day":
        return ts_utc.date().isoformat()
    if granularity == "hour":
        return ts_utc.replace(minute=0, second=0, microsecond=0).isoformat()
    raise ValueError(f"unsupported date_granularity: {granularity}")

def canonical_reference(raw: str) -> str:
    stripped = unicodedata.normalize("NFC", raw.strip())
    return stripped.casefold()

Step 3 — Serialise the canonical string deterministically

Serialisation is where most drift bugs enter: locale-aware str() formatting, dict iteration order in versions of Python that don’t guarantee it, or an accidental float slipping into an f-string. Build the string explicitly from the frozen COMPOSITE_KEY_FIELDS tuple so the order can never depend on insertion order or platform locale.

python
from typing import Mapping

def build_composite_key(
    canonical_fields: Mapping[str, str | Decimal],
    field_order: tuple[str, ...] = COMPOSITE_KEY_FIELDS,
) -> str:
    """Concatenate canonicalised fields in a fixed, explicit order."""
    missing = [f for f in field_order if f not in canonical_fields]
    if missing:
        raise KeyError(f"FIELD_ORDER_DRIFT: missing fields {missing}")

    parts: list[str] = []
    for field in field_order:
        value = canonical_fields[field]
        if isinstance(value, float):
            raise TypeError(f"FLOAT_IN_KEY: field {field!r} is a float")
        parts.append(f"{field}={value}")
    return FIELD_SEPARATOR.join(parts)

Locale is deliberately excluded from the equation: Decimal.__str__ and date.isoformat() are both locale-independent, which is exactly why floats — whose repr can vary with platform and formatting context — are rejected outright rather than coerced.

Step 4 — Hash the canonical string and emit the audited decision

The canonical string is UTF-8 encoded and passed through a pinned hash algorithm. Every computation — success or rejection — emits a structured audit line carrying trace_id, source_hash, and match_decision, so a later examiner can reconstruct exactly which composite key produced which digest without re-deriving it from raw source data.

python
import hashlib
import logging
import uuid

logger = logging.getLogger("reconciliation.composite_key")

def source_hash(
    tx: Mapping[str, str],
    *,
    algorithm: str = "sha256",
    precision: int = 2,
    date_granularity: str = "day",
    trace_id: str | None = None,
) -> str | None:
    trace_id = trace_id or str(uuid.uuid4())
    try:
        canonical_fields = {
            "currency": canonical_currency(tx["currency"]),
            "amount": canonical_amount(tx["amount"], precision=precision),
            "value_date": canonical_date(tx["value_date"], date_granularity),
            "reference": canonical_reference(tx["reference"]),
        }
        key = build_composite_key(canonical_fields)
    except (TypeError, ValueError, KeyError) as exc:
        logger.warning(
            "composite_key.reject",
            extra={"trace_id": trace_id, "source_hash": None,
                   "match_decision": "REJECTED_MALFORMED", "reason": str(exc)},
        )
        return None

    digest = hashlib.new(algorithm, key.encode("utf-8")).hexdigest()
    logger.info(
        "composite_key.hashed",
        extra={"trace_id": trace_id, "source_hash": digest,
               "match_decision": "FINGERPRINTED", "algorithm": algorithm,
               "key_version": COMPOSITE_KEY_VERSION},
    )
    return digest

Step 5 — Guard against collisions and field-order drift

Two failure classes threaten a composite key long after it ships: a genuine hash collision (astronomically unlikely with SHA-256, but still worth defending against with a contract check) and a field-order drift — a deploy that silently reorders or adds a field without bumping COMPOSITE_KEY_VERSION. Hash the field-order tuple itself into a contract_hash and compare it against the value baked into the running index; a mismatch means the index and the hashing code disagree about what a key even is, and probing it would produce meaningless results.

python
def contract_hash(field_order: tuple[str, ...] = COMPOSITE_KEY_FIELDS) -> str:
    blob = FIELD_SEPARATOR.join(field_order).encode("utf-8")
    return hashlib.sha256(blob).hexdigest()[:12]

def assert_contract(index_contract_hash: str) -> None:
    current = contract_hash()
    if current != index_contract_hash:
        raise RuntimeError(
            f"FIELD_ORDER_DRIFT: index built under {index_contract_hash}, "
            f"code expects {current}"
        )

def check_for_collision(digest: str, index: Mapping[str, list[str]],
                         canonical_key: str, stored_keys: Mapping[str, str]) -> bool:
    """Return True only if two DIFFERENT canonical keys share one digest."""
    for existing_id in index.get(digest, []):
        if stored_keys.get(existing_id) != canonical_key:
            return True
    return False

assert_contract should run once at process startup and again before every batch reconciliation run, so a deploy that changes field order fails fast in CI rather than quietly corrupting a production index. check_for_collision is the last line of defence: it only fires when two records hash identically and their stored canonical strings differ, which distinguishes a true HASH_COLLISION from the far more common case of two legitimately identical records (duplicates or same-day repeat charges).

Configuration boundary table

Parameter Default Valid range Notes
amount_precision 2 0–8 Must equal the ledger’s native decimal scale; a mismatch makes equal amounts hash apart.
date_granularity day day, hour Coarser granularity absorbs settlement-clock skew but raises collision risk on high-frequency feeds.
ref_normalisation nfc+casefold+trim fixed set Do not add locale-specific case folding; str.casefold() is Unicode-correct without a locale dependency.
field_order (COMPOSITE_KEY_FIELDS) currency, amount, value_date, reference project-defined Any reorder is a breaking change; bump COMPOSITE_KEY_VERSION and reindex.
hash_algo sha256 sha256, blake2b Pin in config; changing it invalidates every existing digest in the index.

Verification and testing

The property under test is simple to state and easy to get wrong in practice: two records from different sources, describing the same event, must canonicalise to the same source_hash, even when their raw formatting diverges completely.

python
def test_two_sources_same_event_same_hash():
    bank_feed_record = {
        "currency": "usd",
        "amount": "1499.005",
        "value_date": datetime(2026, 7, 14, 23, 59, 3, tzinfo=timezone.utc),
        "reference": "  INV-2201 ",
    }
    erp_export_record = {
        "currency": "USD",
        "amount": Decimal("1499.01"),
        "value_date": datetime(2026, 7, 15, 0, 0, 41, tzinfo=timezone.utc),
        "reference": "inv-2201",
    }

    hash_a = source_hash(bank_feed_record, trace_id="test-a")
    hash_b = source_hash(erp_export_record, trace_id="test-b")

    # Different clock offsets, casing, and whitespace — same canonical event.
    assert hash_a is not None and hash_a == hash_b

Note the deliberately adversarial fixture: the two source timestamps straddle a UTC midnight boundary, so date_granularity="day" is what makes them converge — a coarser or finer setting than the ledger’s actual settlement window would either merge unrelated events or split this one apart. Run this fixture, plus a negative case where the reference genuinely differs, in CI on every deploy so a change to canonical_amount, canonical_reference, or COMPOSITE_KEY_FIELDS is caught before it reaches production data.

Troubleshooting

  • FLOAT_IN_KEY — digests vary between runs on the same data. Root cause: an amount or other field was passed as a Python float before reaching canonical_amount, and float repr is not guaranteed stable across platforms or CPython versions. Fix: enforce Decimal or str at the ingestion boundary; canonical_amount and build_composite_key both raise TypeError rather than silently coercing.
  • FIELD_ORDER_DRIFT — a previously matching pair stops matching after a deploy. Root cause: COMPOSITE_KEY_FIELDS was reordered, extended, or a field renamed without bumping COMPOSITE_KEY_VERSION and reindexing. Fix: call assert_contract() at startup against the index’s stored contract_hash, and treat any mismatch as a hard failure, not a warning.
  • TZ_NOT_NORMALISED — ValueError raised from canonical_date, or two genuinely same-day events land on different dates. Root cause: a naive datetime (no tzinfo) reached the key builder, or a timestamp was floored in local time instead of UTC. Fix: enforce timezone-aware UTC datetimes at the schema boundary described in Exact Match & Hash Comparison, before any canonicalisation runs.
  • HASH_COLLISION — two records with different canonical strings share one digest. Root cause: either a genuine (vanishingly rare) SHA-256 collision, or — far more likely — the stored canonical string wasn’t compared, so a coincidental multi-hit was misreported as a collision. Fix: always run check_for_collision, which compares the stored canonical key, not just the digest, before flagging a true collision; route confirmed collisions to a tie-breaker rather than an alert that pages someone at 3 a.m.

Part of Exact Match & Hash Comparison within Transaction Matching Algorithms & Logic.