Case 04 · configuration postmortem

The config changed on Saturday. Which data did it change?

A service account reduces batch size and Monday throughput collapses. The audit question is not only who changed the file. It is which exact outputs were produced under that configuration, and whether the record of what ran is still the record that was signed.

The code this page is about. Repeated from The output that catches it below.

$ makoto verify bundle handoffs/edited-signed-metadata/bundle \
  --policy receiver/policy.json \
  --schema-catalog receiver/catalog.json \
  --expected-manifest sha256:2b051dc3aeea5530c72cf2a92da9d57d1e929ff4e77041aa4e0751625a2606dd \
  --expected-head sha256:c4af1b433b03c69ae3d4ed795d4f39e189118d1cc3c4dc52397f911f13d6dc3d \
  --expected-artifact receiver/expected/edited-signed-metadata.artifact.json \
  --evaluation-time 2026-09-16T16:00:00Z
PASS        load-safely
PASS        parse-strictly
PASS        index-payloads
PASS        core-schemas
FAIL        signatures
FAIL        authorization-thresholds
PASS        metadata-profiles
FAIL        authorization
PASS        graph-dependency-artifacts
SKIPPED     graph
SKIPPED     roots-and-heads
FAIL        completeness-anchor
PASS        freshness-anchors
SKIPPED     artifact-bytes
SKIPPED     artifact-profiles
PASS        decision
DENY        decision
manifest sha256:2b051dc3aeea5530c72cf2a92da9d57d1e929ff4e77041aa4e0751625a2606dd
E_SIGNATURE_INVALID: signature verification failed
E_SIGNER_UNAUTHORIZED: no matching policy rule met its signature threshold
E_MANIFEST_SET: final artifact is not an exact terminal subject of its declared head
E_MANIFEST_SET: signed manifest sets do not equal the verified graph and bundle index
$ echo $?
1

Evidence boundary. Every terminal block on this page was captured by running the command shown against the checked-in reference fixtures with the Makoto core verifier at the pinned commit 276008e, which the gate below runs. Verification-summary tables are computed from the published reports with its property rules. The verification reports are byte-for-byte the files published under the end-to-end proof. Nothing below was typed by hand.

The problem

Monday morning an alert fires: pipeline throughput dropped sharply over the weekend. The configuration shows a batch size nobody remembers setting. Version control says a service account made the change. Chat search finds nothing. Two hours of cross-referencing later, someone finds that an engineer lowered it to debug a memory error and never reverted it.

A postmortem is only as good as the record it reads. If the record of what an operation did can be edited after the fact, whether to tidy it up or to cover a mistake, the investigation reconstructs a history that never happened.

The runnable fixture tests exactly that. After the reference handoff is signed, the recorded operation of the final step is rewritten. One variant ships the edited statement with its original signature. The other edits the payload in place, leaving the bundle index still claiming the original statement digest.

The raw example

This is the signed record of the final step: what operation ran, when, and exactly which input bytes and predecessor statement it consumed.

$ statement() { b=handoffs/$1/bundle; d=$(jq -r '.artifacts[0].statementDigest.sha256' $b/bundle.json); jq -r .payload $b/attestations/$d.dsse.json | base64 -d; }
$ statement positive | jq '.predicate | {operation, event, inputs}'
{
  "operation": {
    "name": "Remove direct identifiers and bucket ages",
    "type": "urn:makoto:demo:operation:public-safe"
  },
  "event": {
    "id": "urn:uuid:33333333-3333-4333-8333-333333333333",
    "occurredAt": "2026-09-16T16:00:00Z"
  },
  "inputs": [
    {
      "digest": {
        "sha256": "421e19cf002aa25cf6f2f9ed788e04c9badfd9cbc6b00030b0b089810b2ae4f8"
      },
      "name": "normalized",
      "provenance": {
        "statementDigest": {
          "sha256": "1f28b72bcd4c1e9b7df71403ac6bb1670c2f2b09628ca6d76a2fa384db9a0848"
        },
        "subjectName": "customers.normalized.json"
      }
    }
  ]
}

How it failed

Variant one, an edited record. The operation name is rewritten. The signature is copied unchanged, so the envelope still looks signed.

Variant two, an edited payload under an old index entry. The same edit is written into the envelope file in place. The bundle index still names the file by the original statement digest, 99aee76d…, but the bytes inside now hash to c4af1b43….

$ statement() { b=handoffs/$1/bundle; d=$(jq -r '.artifacts[0].statementDigest.sha256' $b/bundle.json); jq -r .payload $b/attestations/$d.dsse.json | base64 -d; }
$ envelope() { b=handoffs/$1/bundle; d=$(jq -r '.artifacts[0].statementDigest.sha256' $b/bundle.json); cat $b/attestations/$d.dsse.json; }
$ diff <(statement positive | jq .predicate.operation) <(statement edited-signed-metadata | jq .predicate.operation)
2c2
<   "name": "Remove direct identifiers and bucket ages",
---
>   "name": "Attacker-edited metadata",
$ diff <(envelope positive | jq .signatures) <(envelope edited-signed-metadata | jq .signatures) && echo 'signature bytes: unchanged'
signature bytes: unchanged
$ b=handoffs/statement-digest-mismatch/bundle
$ jq -r '.attestations[2].statementDigest.sha256' $b/bundle.json
99aee76db46a720405da6d0015d427c791b0d4e02367b9e2bc36ce82ae812bce
$ jq -r .payload $b/$(jq -r '.attestations[2].path' $b/bundle.json) | base64 -d | shasum -a 256
c4af1b433b03c69ae3d4ed795d4f39e189118d1cc3c4dc52397f911f13d6dc3d  -

Put Makoto in the path

History is re-verified on a schedule, not only when it is first received. A nightly job re-runs the gate over every archived handoff, so an after-the-fact edit is found the next morning instead of in the middle of the next incident.

The workflow

# Nightly re-verification of archived handoffs. An edit to the
# recorded history after the fact fails here instead of in the
# middle of the next postmortem.
#
# Copy to .github/workflows/makoto-gate.yml in the repository that
# receives handoffs. Layout it expects:
#   handoffs/<name>/bundle/                  what the sender transferred
#   receiver/policy.json                     receiver-owned trust policy
#   receiver/catalog.json, receiver/resources/  pinned private schemas
#   receiver/expected/<name>.artifact.json   expected final artifact
name: Makoto nightly archive re-verification

on:
  workflow_dispatch:
  schedule:
    - cron: "17 3 * * *"

permissions:
  contents: read

env:
  # Pinned so these fixtures replay identically. Delete this line to
  # evaluate against the runner clock.
  EVALUATION_TIME: "2026-09-16T16:00:00Z"

jobs:
  gate:
    runs-on: ubuntu-latest
    strategy:
      fail-fast: false
      matrix:
        include:
          # The sender publishes manifest and head digests out of band.
          # The receiver pins them here, never inside the bundle.
          - handoff: edited-signed-metadata
            manifest: sha256:2b051dc3aeea5530c72cf2a92da9d57d1e929ff4e77041aa4e0751625a2606dd
            head: sha256:c4af1b433b03c69ae3d4ed795d4f39e189118d1cc3c4dc52397f911f13d6dc3d
          - handoff: statement-digest-mismatch
            manifest: sha256:e24aae77ba1374162edff49aa0ab1d8dd7e38c333e134a2128ad1598fec99fb7
            head: sha256:99aee76db46a720405da6d0015d427c791b0d4e02367b9e2bc36ce82ae812bce
    steps:
      - uses: actions/checkout@v4
      # Pin the verifier to an exact core commit, like any dependency.
      # Run it from a checkout: the CLI resolves its core schemas from
      # the repository tree. This is the core commit the hosted
      # schemas are pinned to.
      - uses: actions/checkout@v4
        with:
          repository: makoto-project/makoto
          ref: 276008eeec0f10d5322c4c627b8c103222fee841
          path: .makoto-core
          persist-credentials: false
      - uses: astral-sh/setup-uv@v6
      - name: Verify the received bundle
        id: verify
        shell: bash
        env:
          HANDOFF: ${{ matrix.handoff }}
          EXPECTED_MANIFEST: ${{ matrix.manifest }}
          EXPECTED_HEAD: ${{ matrix.head }}
        run: |
          verify() {
            uv run --project .makoto-core makoto verify bundle "handoffs/$HANDOFF/bundle" \
              --policy receiver/policy.json \
              --schema-catalog receiver/catalog.json \
              --expected-manifest "$EXPECTED_MANIFEST" \
              --expected-head "$EXPECTED_HEAD" \
              --expected-artifact "receiver/expected/$HANDOFF.artifact.json" \
              --evaluation-time "$EVALUATION_TIME" "$@"
          }
          status=0
          verify --json > makoto-report.json || status=$?
          verify || true
          exit "$status"
      - name: Keep the verification report
        if: always()
        uses: actions/upload-artifact@v4
        with:
          name: makoto-report-${{ matrix.handoff }}
          path: makoto-report.json

  postmortem-index:
    # Runs only when every handoff in the matrix was allowed.
    needs: gate
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v4
      - name: Refresh the postmortem evidence index
        run: echo "replace with your existing step"

Download makoto-gate.yml.

The gate keeps the JSON report as a build artifact, so a later job can summarize it into a signed verification summary without running verification again.

What the receiver requires of every handoff

The receiver trusts one handoff key and requires the manifest, head, and final artifact to match its own pinned expectations.

$ jq '{handoff: .handoff | {authorizedKeyIds, requireExpectedManifest, requireExpectedHead, requireExpectedArtifacts}}' receiver/policy.json
{
  "handoff": {
    "authorizedKeyIds": [
      "sha256:d016df3d83373617c06b5e1d6359caa06eeba8b3af2c7077234f109f2194516c"
    ],
    "requireExpectedManifest": true,
    "requireExpectedHead": true,
    "requireExpectedArtifacts": true
  }
}

The output that catches it

Edited record

The copied signature does not verify over the edited payload, so signatures fails and no rule can authorize the statement.

$ makoto verify bundle handoffs/edited-signed-metadata/bundle \
  --policy receiver/policy.json \
  --schema-catalog receiver/catalog.json \
  --expected-manifest sha256:2b051dc3aeea5530c72cf2a92da9d57d1e929ff4e77041aa4e0751625a2606dd \
  --expected-head sha256:c4af1b433b03c69ae3d4ed795d4f39e189118d1cc3c4dc52397f911f13d6dc3d \
  --expected-artifact receiver/expected/edited-signed-metadata.artifact.json \
  --evaluation-time 2026-09-16T16:00:00Z
PASS        load-safely
PASS        parse-strictly
PASS        index-payloads
PASS        core-schemas
FAIL        signatures
FAIL        authorization-thresholds
PASS        metadata-profiles
FAIL        authorization
PASS        graph-dependency-artifacts
SKIPPED     graph
SKIPPED     roots-and-heads
FAIL        completeness-anchor
PASS        freshness-anchors
SKIPPED     artifact-bytes
SKIPPED     artifact-profiles
PASS        decision
DENY        decision
manifest sha256:2b051dc3aeea5530c72cf2a92da9d57d1e929ff4e77041aa4e0751625a2606dd
E_SIGNATURE_INVALID: signature verification failed
E_SIGNER_UNAUTHORIZED: no matching policy rule met its signature threshold
E_MANIFEST_SET: final artifact is not an exact terminal subject of its declared head
E_MANIFEST_SET: signed manifest sets do not equal the verified graph and bundle index
$ echo $?
1

Output of jq '{decision, primaryError, errors, notPassed: [.checks[] | select(.status != "pass") | {id, status}]}' makoto-report.edited-signed-metadata.json against the report the workflow step wrote.

{
  "decision": "deny",
  "primaryError": "E_SIGNATURE_INVALID",
  "errors": [
    {
      "causedByCheck": "signatures",
      "code": "E_SIGNATURE_INVALID",
      "context": {
        "keyid": "sha256:8cef065b7af83669150b7d32704d3d3e75c3e9aee0d4f6740544c68e81b377aa",
        "statementDigest": {
          "sha256": "c4af1b433b03c69ae3d4ed795d4f39e189118d1cc3c4dc52397f911f13d6dc3d"
        }
      },
      "message": "signature verification failed",
      "step": 5
    },
    {
      "causedByCheck": "authorization-thresholds",
      "code": "E_SIGNER_UNAUTHORIZED",
      "context": {
        "statementDigest": {
          "sha256": "c4af1b433b03c69ae3d4ed795d4f39e189118d1cc3c4dc52397f911f13d6dc3d"
        }
      },
      "message": "no matching policy rule met its signature threshold",
      "step": 6
    },
    {
      "causedByCheck": "completeness-anchor",
      "code": "E_MANIFEST_SET",
      "context": {
        "artifactDigest": {
          "sha256": "ecd821dbe6f4204c4aad64878643d7954cb8c94abbf8d0b28f9c2bfc272f781e"
        },
        "bundleId": "urn:makoto:demo:negative:edited-signed-metadata",
        "head": {
          "sha256": "c4af1b433b03c69ae3d4ed795d4f39e189118d1cc3c4dc52397f911f13d6dc3d"
        },
        "subjectName": "customers.public.json"
      },
      "message": "final artifact is not an exact terminal subject of its declared head",
      "step": 11
    },
    {
      "causedByCheck": "completeness-anchor",
      "code": "E_MANIFEST_SET",
      "context": {
        "bundleId": "urn:makoto:demo:negative:edited-signed-metadata"
      },
      "message": "signed manifest sets do not equal the verified graph and bundle index",
      "step": 11
    }
  ],
  "notPassed": [
    {
      "id": "signatures",
      "status": "fail"
    },
    {
      "id": "authorization-thresholds",
      "status": "fail"
    },
    {
      "id": "authorization",
      "status": "fail"
    },
    {
      "id": "graph",
      "status": "skipped"
    },
    {
      "id": "roots-and-heads",
      "status": "skipped"
    },
    {
      "id": "completeness-anchor",
      "status": "fail"
    },
    {
      "id": "artifact-bytes",
      "status": "skipped"
    },
    {
      "id": "artifact-profiles",
      "status": "skipped"
    }
  ]
}

What a verification summary records

A verification summary over this report is FAILED with E_REPORT_DENIED, because the decision is deny. Its evaluation sidecar still records each verified property's exact rule against the report's checks:

Property	What its rule reads in this report	Result
`MAKOTO_AUTHORIZED`	`signatures` fail, `authorization-thresholds` fail, `authorization` fail; handoff authorization pass	Not established: `E_PROPERTY_AUTHORIZED`
`MAKOTO_GRAPH_COMPLETE`	`index-payloads` pass, `graph-dependency-artifacts` pass, `graph` skipped, `roots-and-heads` skipped, `completeness-anchor` fail	Not established: `E_PROPERTY_GRAPH_COMPLETE`
`MAKOTO_FRESHNESS_ANCHORED`	`freshness-anchors` pass; methods: `expected-artifacts` pass, `expected-heads` pass, `expected-manifest` pass, `max-age` not_checked, `nonce` not_checked	Established
`MAKOTO_SCHEMA_CONFORMANT`	`core-schemas` pass, `metadata-profiles` pass, `artifact-profiles` skipped; the policy requires 2 digest-pinned profiles	Not established: `E_PROPERTY_SCHEMA_CONFORMANT`
`MAKOTO_REPRODUCED`	Needs a second `allow` report from an independent Transform L2-or-higher platform; none is supplied	Not established: `E_PROPERTY_REPRODUCED`

Computed from the published report and the policy whose digest it records, with the property rules in core at the pinned commit. Track levels are not shown: they also need the statements themselves and any platform assessments.

Open the complete edited-signed-metadata report.

Edited payload under an old index entry

The verifier recomputes every statement digest from the decoded payload before trusting the index. index-payloads fails with E_STATEMENT_DIGEST as the primary error. Because the edited statement is no longer the head the manifest signed, the verifier also reports the required final-artifact profile as absent from that head.

$ makoto verify bundle handoffs/statement-digest-mismatch/bundle \
  --policy receiver/policy.json \
  --schema-catalog receiver/catalog.json \
  --expected-manifest sha256:e24aae77ba1374162edff49aa0ab1d8dd7e38c333e134a2128ad1598fec99fb7 \
  --expected-head sha256:99aee76db46a720405da6d0015d427c791b0d4e02367b9e2bc36ce82ae812bce \
  --expected-artifact receiver/expected/statement-digest-mismatch.artifact.json \
  --evaluation-time 2026-09-16T16:00:00Z
PASS        load-safely
PASS        parse-strictly
FAIL        index-payloads
PASS        core-schemas
PASS        signatures
PASS        authorization-thresholds
PASS        metadata-profiles
PASS        authorization
PASS        graph-dependency-artifacts
SKIPPED     graph
SKIPPED     roots-and-heads
FAIL        completeness-anchor
PASS        freshness-anchors
SKIPPED     artifact-bytes
FAIL        artifact-profiles
PASS        decision
DENY        decision
manifest sha256:e24aae77ba1374162edff49aa0ab1d8dd7e38c333e134a2128ad1598fec99fb7
E_STATEMENT_DIGEST: indexed statement digest differs from decoded payload bytes
E_MANIFEST_SET: final artifact is not an exact terminal subject of its declared head
E_MANIFEST_SET: signed manifest sets do not equal the verified graph and bundle index
E_REQUIRED_PROFILE_MISSING: required final-artifact profile is absent from its signed head
E_REQUIRED_PROFILE_MISSING: required final-artifact profile is absent from its signed head
$ echo $?
1

Output of jq '{decision, primaryError, errors, notPassed: [.checks[] | select(.status != "pass") | {id, status}]}' makoto-report.statement-digest-mismatch.json against the report the workflow step wrote.

{
  "decision": "deny",
  "primaryError": "E_STATEMENT_DIGEST",
  "errors": [
    {
      "causedByCheck": "index-payloads",
      "code": "E_STATEMENT_DIGEST",
      "context": {
        "declared": {
          "sha256": "99aee76db46a720405da6d0015d427c791b0d4e02367b9e2bc36ce82ae812bce"
        },
        "statementDigest": {
          "sha256": "c4af1b433b03c69ae3d4ed795d4f39e189118d1cc3c4dc52397f911f13d6dc3d"
        }
      },
      "message": "indexed statement digest differs from decoded payload bytes",
      "step": 3
    },
    {
      "causedByCheck": "completeness-anchor",
      "code": "E_MANIFEST_SET",
      "context": {
        "artifactDigest": {
          "sha256": "ecd821dbe6f4204c4aad64878643d7954cb8c94abbf8d0b28f9c2bfc272f781e"
        },
        "bundleId": "urn:uuid:55555555-5555-4555-8555-555555555555",
        "head": {
          "sha256": "99aee76db46a720405da6d0015d427c791b0d4e02367b9e2bc36ce82ae812bce"
        },
        "subjectName": "customers.public.json"
      },
      "message": "final artifact is not an exact terminal subject of its declared head",
      "step": 11
    },
    {
      "causedByCheck": "completeness-anchor",
      "code": "E_MANIFEST_SET",
      "context": {
        "bundleId": "urn:uuid:55555555-5555-4555-8555-555555555555"
      },
      "message": "signed manifest sets do not equal the verified graph and bundle index",
      "step": 11
    },
    {
      "causedByCheck": "completeness-anchor",
      "code": "E_REQUIRED_PROFILE_MISSING",
      "context": {
        "profileDigest": {
          "sha256": "68169d043c628fda5435cbd7845b02ea3e0d850b7a509c0b35fd304463fffacb"
        },
        "profileId": "https://schemas.example.internal/makoto/customer-public-v1.schema.json",
        "subjectName": "customers.public.json"
      },
      "message": "required final-artifact profile is absent from its signed head",
      "step": 11
    },
    {
      "causedByCheck": "artifact-profiles",
      "code": "E_REQUIRED_PROFILE_MISSING",
      "context": {
        "profileDigest": {
          "sha256": "68169d043c628fda5435cbd7845b02ea3e0d850b7a509c0b35fd304463fffacb"
        },
        "profileId": "https://schemas.example.internal/makoto/customer-public-v1.schema.json",
        "subjectName": "customers.public.json"
      },
      "message": "required final-artifact profile is absent from its signed head",
      "step": 13
    }
  ],
  "notPassed": [
    {
      "id": "index-payloads",
      "status": "fail"
    },
    {
      "id": "graph",
      "status": "skipped"
    },
    {
      "id": "roots-and-heads",
      "status": "skipped"
    },
    {
      "id": "completeness-anchor",
      "status": "fail"
    },
    {
      "id": "artifact-bytes",
      "status": "skipped"
    },
    {
      "id": "artifact-profiles",
      "status": "fail"
    }
  ]
}

What a verification summary records

Property	What its rule reads in this report	Result
`MAKOTO_AUTHORIZED`	`signatures` pass, `authorization-thresholds` pass, `authorization` pass; handoff authorization pass	Established
`MAKOTO_GRAPH_COMPLETE`	`index-payloads` fail, `graph-dependency-artifacts` pass, `graph` skipped, `roots-and-heads` skipped, `completeness-anchor` fail	Not established: `E_PROPERTY_GRAPH_COMPLETE`
`MAKOTO_FRESHNESS_ANCHORED`	`freshness-anchors` pass; methods: `expected-artifacts` pass, `expected-heads` pass, `expected-manifest` pass, `max-age` not_checked, `nonce` not_checked	Established
`MAKOTO_SCHEMA_CONFORMANT`	`core-schemas` pass, `metadata-profiles` pass, `artifact-profiles` fail; the policy requires 2 digest-pinned profiles	Not established: `E_PROPERTY_SCHEMA_CONFORMANT`
`MAKOTO_REPRODUCED`	Needs a second `allow` report from an independent Transform L2-or-higher platform; none is supplied	Not established: `E_PROPERTY_REPRODUCED`

Open the complete statement-digest-mismatch report.

Model configuration as transformation evidence

Bind the config bytes Record the exact configuration artifact or digest used by the operation.
Name the executor Sign the transformation with the workload identity and let receiver policy decide whether that identity was authorized.
Append affected outputs Each output statement references the input data, predecessor statement, operation, and configuration evidence.
Traverse the interval Postmortem analysis can identify outputs produced after the changed config and before the corrected one.

What remains outside Makoto

Change approval, human intent, alerting, and root-cause judgment remain separate systems. Makoto supplies attributable, tamper-evident claims about artifacts; it does not decide whether the configuration was wise.

Concrete postmortem record

Before	Configuration artifact A is bound to outputs produced under the earlier settings.
Change	Configuration artifact B is committed; the next transformation statement names its digest and authorized workload signer.
Blast radius	Traverse descendants produced after B and before corrective artifact C.
Finding	The report identifies attributable claims and affected bytes without pretending to know whether the human change was justified.

Configuration belongs in the evidence as an exact input artifact, not merely a free-text note or ticket URL. The reference fixture does not yet bind a configuration file as an input; the tamper checks shown above apply unchanged when it does.

What else this handles

Feature-flag changes that cause regressions in production.
Connection-pool or retry settings modified during on-call firefighting.
Infrastructure-as-code drift, where deployed state diverges from the recorded plan.
Audits that require a complete, unedited history of configuration changes.

Key insight. Configuration is data too. A signed record makes the postmortem fast, and re-verifying that record makes it honest: an edit to history fails verification instead of becoming the history.

Make the postmortem reproducible

Attach the exact configuration and output bytes to the transformation claim.

Run the append-only pattern Read the casebook entry