chum
← journal

Multipart assembly: one object's identity as a Merkle root of its parts

SP7 · 2026-06-17

“The record is the proof” is easy to honor for a single blob: its address is the hash of its bytes, and a signed, append-only chain says which name points at it. SP7 asks the harder version of the question. When a client uploads a large object in many parts, the parts arrive separately, each a blob in its own right — so what, exactly, is the assembled object, and how does its address still prove every byte? The scaffold had left upload.complete as a 501 stub precisely because that answer was unsettled. ADR-0003 had been sitting open between two options. SP7 closes it.

Two ways to name an assembled object

The obvious move is to re-hash the assembled bytes: concatenate the parts, run sha-256 over the whole thing, and call that the object’s CID. It works, and it matches the intuition that an object’s identity is a digest of its content. But it throws away everything the upload already knew. Each part was already a content-addressed blob with its own verified CID; re-hashing the concatenation discards those addresses and forces a verifier to pull the entire object to check any of it.

ADR-0003 (chum/docs/decisions/0003-multipart-digest.md, Accepted 2026-06-17) takes the other option: the object’s identity is a Merkle root over the part CIDs. upload.complete assembles the ordered parts into a canonical DAG-CBOR manifest that commits to each part’s CID and size plus the total, and the object’s CID is that manifest’s CIDv1/raw/sha-256. The whole object resolves into a clean chain of independently verifiable links:

signed PointerRecord ──(cid)──▶ manifest blob ──(parts[].cid)──▶ part blobs
        │                            │                               │
   chain + sig                  raw-sha256 CID                  raw-sha256 CIDs
   (ADR-0004)                   = the Merkle root           (each a real CAS blob)

The payoff is partial verification. A client fetches the manifest, validates it against the record’s CID, then validates any single part against its listed CID — without touching the other parts. That property does not exist if the object’s identity is a digest of the concatenated bytes; there, verifying one byte means re-hashing all of them. Choosing the Merkle root is choosing the structure that lets the proof be checked piece by piece. That is the reason ADR-0003 recommended it, and the reason it is consistent with the thesis rather than merely compatible with it.

The Assembler is pure proof, no I/O

The manifest codec is its own seam — object.Assembler — deliberately holding no storage and no network, so the part of the system that defines the object’s identity can be reasoned about and tested in isolation. The adapter in internal/assembler builds the manifest and parses it back:

func (Assembler) Manifest(parts []object.ManifestPart, contentType string) ([]byte, int64, error) {
	if len(parts) == 0 {
		return nil, 0, errors.New("assembler: manifest needs at least one part")
	}
	var total int64
	mp := make([]manifestPart, len(parts))
	for i, p := range parts {
		mp[i] = manifestPart{CID: object.CIDLink(p.CID), Size: p.Size}
		total += p.Size
	}
	m := manifest{Type: ManifestType, ContentType: contentType, Parts: mp, Size: total}
	b, err := object.DagCBOR.Marshal(m)
	return b, total, err
}

Two details carry weight. The encoding uses object.DagCBOR — the same canonical DAG-CBOR profile (definite-length, length-first key sort, CID links as IPLD tag-42) that the pointer records use. Those helpers are exported from the object package precisely so the manifest and record encodings can never quietly diverge into two dialects; there is one canonical profile, used in both places. And Parse re-derives the total size from the parts and rejects any manifest whose declared size disagrees, so a verifier never has to trust the redundant field — it recomputes it. Parse also caps part count (MaxParts = 100_000) so a hostile manifest cannot force an unbounded allocation, and rejects an empty parts list, because a multipart object with no parts is not an object.

An additive marker that changes nothing for everyone else

A multipart record needs to announce itself so the read path knows to reassemble. SP7 adds exactly one field to PointerRecord: Assembled bool, omitempty in the canonical encoding. Because CBOR omitempty drops a false bool entirely, every existing non-multipart record encodes to byte-identical canonical bytes — and therefore identical CIDs and identical signatures — as before SP7 existed. A golden-bytes test pins this: a non-assembled record’s bytes are asserted unchanged. The new capability is invisible to the records that don’t use it. This is the additive-seam discipline the whole project runs on: you extend the system without disturbing the proofs already in the chain.

The read path, and why by-CID stays literal

GET /b/{bucket}/{key} resolves to a record; if record.Assembled, the service fetches the manifest, parses it, and returns a reader that lazily streams each part in order — each part fetched through the SP-5 verify decorator, so every part’s CID is validated as it is served. Corrupt one part’s bytes on disk and the read fails with ErrCorrupt at exactly that part. Reassembly is transparent only on the named path, which carries the Assembled flag. By-CID reads stay literal: GET /blob/{manifestCID} returns the manifest bytes — that CID is the verifiable root — and GET /blob/{partCID} returns that one part. No content sniffing, no surprise reassembly behind a hash; a CID always addresses exactly the bytes it names.

Deletion stays the reaper’s job alone

Parts are ordinary committed CAS blobs, and a part CID may be shared — with another object, or another in-flight session. So, exactly as in SP6, the only safe deleter is the reaper, which checks all references before removing a byte. upload.abort records intent and nothing more. Service.ReapUploads does the work in two steps: demote stale-open sessions to aborted (removing them from the live-reference set), then, for each aborted session’s parts, delete the bytes only if no chain record and no other open-or-completed session still references them — and finally purge the session row. A completed object’s parts are referenced forever through the retained membership index, so they read as live and are never reaped; that is what keeps a multipart object provable for as long as its name exists. The upload table growing by one session plus N part rows is the same bargain the pointer chain already makes: the record is the proof, and the proof is what you keep.

What SP7 settled

The seam list is short — a pure Assembler, a durable UploadStore, one additive record field, one reaper extension — but the decision underneath it is the load-bearing one. A multipart object is not a re-hashed bag of bytes; it is a Merkle root over content-addressed parts, named by a signed record, verifiable in whole or in part. ADR-0003 moved from open to accepted on that basis. The 501 stub is now a working upload that produces objects whose identity is, all the way down, a chain of hashes you can check yourself.

Get early access →