For months I have been describing Chum with some version of this sentence: object storage where you can verify a bucket’s entire history without trusting the operator. It is on the landing page. It is in the ConOps. It is the reason the project exists.
I went back through my own code last week, looking for something else entirely, and worked out that it is not true. Not “true with caveats.” Not “true once you squint.” Two independent things are missing, and either one alone is enough to void the claim.
The audience for this project reads the code. Someone was going to open service.go and find this. It is strictly better that they find my account of it first.
Hole A: I sign everything
Here is Service.Put, in chum/internal/object/service.go:
rec := PointerRecord{
Bucket: bucket, Key: key, CID: id, Prev: prev, DID: s.signer.DID(),
ContentType: contentType, Size: size, Visibility: vis,
Timestamp: clock(),
}
block, err := UnsignedBytes(rec)
// ...
rec.Sig = s.signer.Sign(block)
s.signer is the server’s key — a file at ./data/signer.key, minted on first boot. Every pointer record in every chain is signed by it. The same is true of tombstones and of multipart completion.
Look at what PointerRecord carries: one DID, one signature. There is nowhere for a writer to appear. So the chain records what changed, and it records that the server said so, and that is all. “Who authorised this write” is not in the record. It was never in the record.
Which means: whoever holds signer.key can fabricate an entire history — invent records, sign them, chain them — and it will verify perfectly. That person is me. A chain that is tamper-evident against everyone except the one party you were being asked not to trust is not, on any honest reading, “verify without trusting the operator.”
Hole B: nothing outside my server ever sees the head
The second hole is worse, and I found it by grepping my own repository for a function call that turned out not to exist.
Chum is built on AT Protocol. The obvious thing to do with a signed, append-only chain on atproto is to publish it — put the head somewhere the operator does not control, so the operator cannot quietly change it. Search the module for putRecord, applyWrites, createRecord: zero hits. The only com.atproto.* call anywhere in the non-test tree is sync.getRepo, twice, in the revocation syncer. Read-only.
Nothing Chum produces ever leaves Chum.
So even setting Hole A aside, I could withhold a record, truncate a chain, backdate a write, or serve one history to you and a different one to your auditor — and there is no artifact anywhere in the world that would contradict me. An append-only log that only its operator can see is not a ledger. It is a diary.
These are two halves of the same hole, and I want to be precise about which half does what, because I initially got this wrong myself. Client signatures fix who authorised. Anchoring fixes when, and only once. An anchored chain that I still sign proves only that I did not change my mind later. A client-signed chain with no anchor proves only that I did not forge the contents of the writes I chose to show you. You need both. It is arithmetic, not a compromise.
What Chum actually does prove
Smaller than I said, and still worth something:
Every mutation appends a signed, content-addressed record to an append-only chain, each link naming its predecessor’s CID. Any third party can walk that chain offline and prove that it is internally consistent — that no record was altered, that no link was removed, that the bytes at each CID hash to that CID. Content integrity is intrinsic: the address is the content, so a substituted byte changes the address and there is no metadata to trust.
That is a real property. It is tamper-evidence against everyone except me. What it is not is operator-independence, and I should not have used language that implied otherwise.
(While I was at it, I found a third one, unrelated to the claim but worth saying: GetByCID discards its Principal argument at the function signature — CID reads are unauthenticated by design, on the theory that content-addressing is its own permission. That is fine for public bytes and unacceptable the moment I ship a public proof endpoint, because object.history hands you the CID of every version. A proof of a private object would be a bearer token to its plaintext. That is now launch-blocking, and it gets fixed before any anonymous proof ships.)
The part I got wrong about the fix
My first instinct was to close the holes quietly and keep the sentence. Write the fixes, ship them, never mention it.
The deeper problem is that the sentence was never achievable in the first place, by anyone, and no amount of engineering was going to make it so.
Every verifiable system bottoms out in a trust root. S3’s integrity story bottoms out in Amazon. Certificate Transparency — the closest well-known analogue to what I am building — bottoms out in the log operators and in Chrome’s policy of requiring them. Sigstore bottoms out in the Fulcio root. There is no system anywhere that achieves “trust nobody.” There are only systems that are precise about whom you are trusting, and what it would take for them to lie to you.
So the fix is not a louder claim. The fix is to stop making an absolute claim and start making a checkable one.
The ladder
Chum’s guarantee is becoming a tier, recorded inside the signed bytes so it cannot be forged upward, and every surface that reports verification has to report it. This is ADR-0008.
Tier 1 — self-custody. The writer’s key is a did:key, a did:web, or lives on a PDS they run. The signing key is never touched by any operator, including me. This is the strong form, and it needs no new protocol machinery — it works today.
Tier 2 — registered key. The writer is on a managed PDS (a bsky.social account, say). More on this below; it is the interesting case.
Tier 3 — legacy. Signed by the operator. What every record in Chum is today.
A verifier that shows you a green tick without telling you which of these it checked is the same overclaim in a nicer font. So the verifier will lead with the tier, in plain language, as the headline of the result — not as a footnote.
I think this is actually a stronger position than the one I was falsely claiming, and it is one an incumbent cannot copy without publishing its own embarrassing tiers.
Tier 2, and the key-custody problem
Here is the thing that nearly sank this. “The client signs” sounds easy until you ask with what key. AT Protocol OAuth does not hand a browser your repo signing key — your PDS holds it. So for a typical user on a managed PDS, “the client signs, therefore the operator cannot forge” degrades to “a different operator can forge,” which is not obviously progress.
It turns out atproto has the machinery, and I had not looked.
com.atproto.identity.signPlcOperation accepts a verificationMethods field. The did:plc spec makes that an open map — arbitrary key ids, did:key-encoded, up to ten per DID. That is the same mechanism used for PDS migration, gated behind an emailed token. So a managed-PDS user can add a #chum signing key to their own DID document. And indigo — which Chum already depends on — parses the DID document’s full key map, not just #atproto, so Chum resolves that key with no new code at all.
The residual worry is that the managed host still holds your rotation keys, and could rewrite your DID document later to swap the key out. Two things bound that.
First, PLC is an append-only audit log, served by a party that is neither me nor necessarily your host. GET plc.directory/{did}/log/audit is public and unauthenticated; it returns every operation, with timestamps, chained by prev — and it retains even nullified operations. A key substitution cannot be silent. It can only be loud, in a log the substituting party does not control, and historical records still verify against the key that was valid when they were signed.
Second — and this is the part I did not expect — nullification requires a higher-authority rotation key and only works within 72 hours. And signPlcOperation accepts a rotationKeys field too. Which means a user can register their own rotation key above their host’s, and after that the host cannot substitute their key at all. That is identity self-custody without leaving the managed PDS, for the same ceremony plus one field.
One honest caveat, because this is the whole point of this post: I scanned twelve thousand consecutive real PLC operations, and every single one carries exactly {atproto: ...}. Nobody is using extra verification methods. The spec permits it; the wild has no precedent. Whether a given PDS implementation passes a non-atproto entry through — or quietly clobbers it with its own recommended credentials — is an empirical question I have not been able to answer yet, because it needs a real account and an emailed token. There is a script in the tree that runs the ceremony and reports the answer from the PDS’s signed operation, before submitting anything, so it can detect a clobber without ever mutating a DID document. If it clobbers, Tier 2 does not exist and the claim narrows to Tier 1. I will say so if that happens.
What actually closes the holes
Three things, all specified, none shipped:
The writer signs (writerDid, writerSig, and the CID of the capability that authorised it, inside the signed record). This puts authorship in the chain. It also forces compare-and-swap semantics on the head — two writers racing must not silently clobber — and a multipart story, because a client signs the part-CID list, not the assembled CID it cannot know in advance. It is bigger than it looks, and it is security-critical.
The writer keeps a receipt in their own repo. An anchor stops me rewriting history. It does not stop me omitting a write, or serving two versions of it to two people. So when the client signs a write intent, it also publishes that intent’s CID to the writer’s own PDS — one record in a repo I do not control. Now I cannot omit or equivocate without the writer being able to prove it. This is cheap, and it neatly sidesteps the objection that anchoring into infrastructure I also run is theatre: your PDS is, by definition, not my PDS.
An external witness for the head. A Merkle root over the head set, chained to the previous checkpoint, signed, published to a PDS I do not operate. Root only — no bucket names, no keys, no CIDs, because names are frequently personal data and a public append-only log is not erasable by anyone, including me. Getting that constraint wrong is cheap to avoid now and impossible to undo later.
With all three, forging a Chum history requires the collusion of: me, and your PDS host, and my anchor host, and plc.directory, and the relays that retained the commits.
That is a sentence I will be able to say and defend. It is not “trust nobody.” It is a specific, checkable, bounded statement about exactly who would have to conspire, and I would rather ship that than keep a slogan that does not survive contact with my own source tree.
And then attack it
The claim is worth exactly as much as its falsifiability, so the last piece is a tool that attacks Chum: forge a record, drop a link, backdate a head, substitute a key, serve two histories to two readers. Five attacks, five tests, each asserting the verifier catches me. It ships in the repo. A store that publishes its own attacks is not something I have seen, and it costs a test file rather than a marketing budget.
The lesson, which is duller than the bug
The failure here was not a coding mistake. Every line of Chum does exactly what it says. The failure is that I wrote the marketing copy from the architecture diagram instead of from the implementation.
The diagram was aspirational, in the way diagrams always are. The code was honest, in the way code always is. Only one of them was on the homepage.
The landing page now says what the chain proves and what it does not. The tiers are an ADR. The fixes are specced and dated. If you want to check any of that, the code is right there — which was supposed to be the entire point, and now actually is.