Maintain it through Git, sign-off and pull requests.

Makoto defines no Git, sign-off or review process. Its records are small signed files that sit beside the manifests they describe, and a signed statement is never edited: a change produces a new statement, and a re-ingest produces a new record. That suits a repository where every change arrives as a reviewed pull request and Git keeps every earlier record.

$ uv run --no-project lineage/check_lineage.py --base main
FAIL text/letters/letters.yaml changed but text/letters/makoto/ did not
     re-run the ingest so the lineage record describes the new bytes
FAIL text/letters/makoto deny: E_EXPECTED_ARTIFACT

Three mechanisms, three questions

Keeping them apart matters. A signed-off commit can carry a record signed by a key the base policy does not authorize, and the lineage check refuses it. A properly signed record can arrive in a commit without a sign-off, and the dco job refuses that. Adding a maintainer key is its own pull request that edits the policy; records signed with that key pass once it has merged.

A contribution, end to end

1. Edit ingest.yaml, or add a leaf directory with its own.
2. Run the ingest. It writes shards to the lookaside, rewrites the leaf manifest, and writes a new record under makoto/, signed with a key the policy authorizes.
3. Commit everything with git commit -s and open a pull request.
4. The workflow runs two jobs: every commit needs a sign-off from its author, and every record must verify against the files in the pull request.
5. Reviewers read the manifest diff and the record's statements, which are small JSON documents once decoded.

A contributor without an authorized key can still open the pull request; a maintainer re-runs the ingest on the branch and signs. A policy can require two maintainers for a step with minimumSignatures, and makoto envelope cosign adds the second signature (section 16.1).

The workflow

Two jobs, on every pull request. The Makoto CLI is installed from source at the same core commit this site is pinned to.

name: corpus checks

on:
  pull_request:

permissions:
  contents: read

jobs:
  dco:
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@3d3c42e5aac5ba805825da76410c181273ba90b1 # v7.0.1
        with:
          ref: ${{ github.event.pull_request.head.sha }}
          fetch-depth: 0
          persist-credentials: false
      - name: Every commit carries a sign-off from its author
        env:
          BASE: ${{ github.event.pull_request.base.sha }}
        run: |
          status=0
          for commit in $(git rev-list "$BASE..HEAD"); do
            author=$(git log -1 --format='%an <%ae>' "$commit")
            if ! git log -1 --format='%(trailers:key=Signed-off-by,valueonly)' "$commit" |
              grep -qxF "$author"; then
              echo "::error::$commit lacks 'Signed-off-by: $author' (commit with -s)"
              status=1
            fi
          done
          exit "$status"

  lineage:
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@3d3c42e5aac5ba805825da76410c181273ba90b1 # v7.0.1
        with:
          ref: ${{ github.event.pull_request.head.sha }}
          fetch-depth: 0
          persist-credentials: false
      - uses: astral-sh/setup-uv@bec219d24cd3e171d82865faccec33120bb574f4 # v10.1.0
      - name: Install the Makoto reference CLI
        run: |
          uv tool install \
            "git+https://github.com/makoto-project/makoto@276008eeec0f10d5322c4c627b8c103222fee841"
          uv tool dir --bin >> "$GITHUB_PATH"
      - name: Manifests and lineage records agree
        env:
          BASE: ${{ github.event.pull_request.base.sha }}
        # Run the base commit's check, which also reads the base commit's policy,
        # so a pull request cannot change the rules it is checked against.
        run: |
          git show "$BASE:lineage/check_lineage.py" > "$RUNNER_TEMP/check_lineage.py"
          uv run --no-project "$RUNNER_TEMP/check_lineage.py" --base "$BASE"

The check

It reads the policy and catalog from the base branch, requires a record in every directory that holds an ingest.yaml, fails when a handed-off file changed without its record, and runs the verifier with each file's current bytes as the expected artifact. Once a record verifies, it compares ingest.yaml with the parametersDigest the ingest statement pins. Any other fault, such as an unauthorized key, a missing statement or an edited envelope, surfaces as the verifier's own error code.

"""Fail a pull request whose corpus manifests and lineage records disagree.

The policy and schema catalog come from the base commit, never from the pull
request, so a contribution cannot authorize its own key. A policy change is a
separate pull request that merges first.

Every leaf directory (one holding an `ingest.yaml`) must have a `makoto/`
lineage record, and every record is checked three ways:

1. If the pull request changed a file the record hands off (the leaf manifest)
   but changed nothing under that record, the contributor edited the corpus
   without re-running the ingest that updates its lineage.
2. `makoto verify bundle` must allow the record under the base policy, with the
   handed-off file's current bytes as the expected artifact. A stale,
   hand-edited or wrongly signed record is denied.
3. Once the record verifies, the ingest statement's `parametersDigest` must
   match the `ingest.yaml` in the pull request.

Usage, from the repository root:

    uv run --no-project lineage/check_lineage.py --base origin/main

Requires Git and the `makoto` command on PATH. Standard library only.
"""

from __future__ import annotations

import argparse
import base64
import hashlib
import json
import subprocess
import tempfile
from pathlib import Path

POLICY = "lineage/policy.json"
CATALOG = "lineage/catalog.json"
SKIPPED = {".git", ".venv", "node_modules"}


def git(*arguments: str) -> bytes:
    return subprocess.run(["git", *arguments], check=True, capture_output=True).stdout


def sha256(data: bytes) -> str:
    return hashlib.sha256(data).hexdigest()


def decode(envelope: Path) -> dict:
    return json.loads(base64.b64decode(json.loads(envelope.read_bytes())["payload"]))


def directories_holding(pattern: str) -> list[Path]:
    return sorted(
        path.parent for path in Path(".").glob(f"**/{pattern}") if not SKIPPED & set(path.parts)
    )


def base_config(base: str, scratch: Path) -> tuple[Path, Path]:
    """The base commit's policy and catalog, with every schema the catalog names."""
    names = [POLICY, CATALOG]
    catalog = json.loads(git("show", f"{base}:{CATALOG}"))
    names += [f"{Path(CATALOG).parent.as_posix()}/{item['path']}" for item in catalog["resources"]]
    for name in names:
        target = scratch / "base" / name
        target.parent.mkdir(parents=True, exist_ok=True)
        target.write_bytes(git("show", f"{base}:{name}"))
    return scratch / "base" / POLICY, scratch / "base" / CATALOG


def verify(record: Path, artifacts: list[dict], config: tuple[Path, Path], scratch: Path):
    policy, catalog = config
    command = [
        "makoto", "verify", "bundle", str(record),
        "--policy", str(policy),
        "--schema-catalog", str(catalog),
        "--json",
    ]  # fmt: skip
    for index, artifact in enumerate(artifacts):
        current = record.parent / artifact["name"]
        data = current.read_bytes() if current.is_file() else b""
        binding = scratch / f"expected-{sha256(str(record).encode())[:12]}-{index}.json"
        binding.write_text(
            json.dumps(
                {
                    "head": artifact["head"],
                    "subjectName": artifact["name"],
                    "digest": {"sha256": sha256(data)},
                }
            )
        )
        command += ["--expected-artifact", str(binding)]
    result = subprocess.run(command, check=False, capture_output=True, text=True)
    try:
        report = json.loads(result.stdout)
    except json.JSONDecodeError:
        return "error", [result.stderr.strip() or f"makoto exited {result.returncode}"]
    return report["decision"], sorted({item["code"] for item in report.get("errors", [])})


def pinned_parameters(record: Path) -> set[str]:
    """The parametersDigest of each head statement, read after the record verified."""
    index = json.loads((record / "bundle.json").read_bytes())
    paths = {item["statementDigest"]["sha256"]: item["path"] for item in index["attestations"]}
    pinned = set()
    for head in decode(record / "manifest.dsse.json")["heads"]:
        operation = decode(record / paths[head["sha256"]])["predicate"].get("operation", {})
        pinned.add(operation.get("parametersDigest", {}).get("sha256", ""))
    return pinned


def main() -> int:
    parser = argparse.ArgumentParser(description=__doc__.splitlines()[0])
    parser.add_argument("--base", required=True, help="Git ref the pull request targets")
    args = parser.parse_args()
    changed = set(git("diff", "--name-only", f"{args.base}...HEAD").decode().splitlines())
    failures = 0
    for leaf in directories_holding("ingest.yaml"):
        if not (leaf / "makoto/bundle.json").is_file():
            print(f"FAIL {leaf.as_posix()}/ has ingest.yaml but no makoto/ lineage record")
            failures += 1
    with tempfile.TemporaryDirectory() as scratch_name:
        scratch = Path(scratch_name)
        config = base_config(args.base, scratch)
        for record in directories_holding("makoto/bundle.json"):
            # Read only to learn which files to expect; the verifier checks the signature.
            artifacts = decode(record / "manifest.dsse.json")["artifacts"]
            record_changed = any(path.startswith(f"{record.as_posix()}/") for path in changed)
            for artifact in artifacts:
                name = (record.parent / artifact["name"]).as_posix()
                if name in changed and not record_changed:
                    print(f"FAIL {name} changed but {record.as_posix()}/ did not")
                    print("     re-run the ingest so the lineage record describes the new bytes")
                    failures += 1
            decision, codes = verify(record, artifacts, config, scratch)
            if decision != "allow":
                print(f"FAIL {record.as_posix()} {decision}: {', '.join(codes)}")
                failures += 1
                continue
            parameters = record.parent / "ingest.yaml"
            if parameters.is_file() and sha256(parameters.read_bytes()) not in pinned_parameters(
                record
            ):
                print(f"FAIL {parameters.as_posix()} differs from the parameters the record pins")
                failures += 1
                continue
            print(f"PASS {record.as_posix()} verifies against the current files")
    return 1 if failures else 0


if __name__ == "__main__":
    raise SystemExit(main())

What it accepts and refuses

A change the record does not cover passes:

$ uv run --no-project lineage/check_lineage.py --base main
PASS text/letters/makoto verifies against the current files

A contributor who adds a shard to letters.yaml by hand, without re-running the ingest, is refused twice. The record did not change with the manifest, and the verifier denies the handoff because the manifest's bytes no longer match the digest the record names:

$ uv run --no-project lineage/check_lineage.py --base main
FAIL text/letters/letters.yaml changed but text/letters/makoto/ did not
     re-run the ingest so the lineage record describes the new bytes
FAIL text/letters/makoto deny: E_EXPECTED_ARTIFACT

Three more contributions it refuses. Settings edited without re-running the ingest:

$ uv run --no-project lineage/check_lineage.py --base main
FAIL text/letters/ingest.yaml differs from the parameters the record pins

A new leaf directory with no record:

$ uv run --no-project lineage/check_lineage.py --base main
FAIL text/diaries/ has ingest.yaml but no makoto/ lineage record
PASS text/letters/makoto verifies against the current files

A record signed without the license claim, together with an edit to lineage/policy.json that drops the rule demanding it. The check reads the base branch's policy, so the edit has no effect:

$ uv run --no-project lineage/check_lineage.py --base main
FAIL text/letters/makoto deny: E_MANIFEST_SET, E_SIGNER_UNAUTHORIZED

Each refusal exits 1, so the job fails, and the pull request cannot merge while the check is required.

History

Git keeps every earlier record. Makoto does not require a new record to name the one it replaces: a re-ingest that replaces the corpus starts from a new origin, and each reader's policy decides whether an older record is still acceptable. A step that builds on existing shards, rather than replacing them, names them as inputs and extends the graph instead.