coop/docs/concord-community-discovery-plan.md

Concord discovery: why no community ever reaches subscribe

Audit + fix plan. Read alongside docs/concord-usage.md and docs/concord-simplification-plan.md.

Symptom

crates/community/src/lib.rs::subscribe is never called, so no wrap is ever subscribed to and the sidebar stays empty. community load: 0 state document(s) found is the only clue.

Root cause

CommunityRegistry::load only ever reads the local database. Nothing in the discovery path touches a relay.

community::init
  └─ SignerChanged → load
       └─ sync::load
            ├─ store::load_states(client)   → client.database().query(..)   // local only
            └─ load_list(client, ..)        → client.database().query(..)   // local only, .limit(1)
       → track([])
       → sync_subscriptions: `for community in self.communities` runs zero times
       → subscribe never called
       → no relay is ever queried
       → the database never fills
       → load stays empty forever

The loop is self-reinforcing: the local database is populated by the subscriptions that the empty load prevents. That is why an account which belongs to several communities in another client still shows nothing — a fresh install has no concord/* state document, and coop has no way to ask for one.

Confirmed by inspection:

Location	What it does
crates/community/src/lib.rs:167-191	load → sync::load, then track(states)
crates/community/src/sync.rs:125-137	load = store::load_states + load_list
crates/concord/src/store.rs:314-341	load_states queries client.database() only
crates/community/src/sync.rs:139-153	load_list queries client.database() only, .limit(1)
crates/community/src/lib.rs:233-276	sync_subscriptions skips everything when communities is empty

subscribe itself is correct. Do not debug it.

What the protocol actually says

Read from the spec (concord-protocol/concord, the submodule referenced by accordion.chat): 02.md §8 and examples.md §6.2.

A member's memberships live in the Community List, on relays:

• Kind 33302, addressable, NIP-44-encrypted to self, signed by the member's real key, one event per fragment with d = the fragment index in decimal ("0", "1", …). 13302 is explicitly retired ("the single-event Community List, superseded by 33302 once it outgrew one event — a replaceable kind cannot fragment", 02.md:314).
• Every 32-byte value at any depth is unpadded base64url, not hex. This is section-scoped: CORD-05 invite fields stay hex (examples.md §6.3).
• Join material is the membership subset — owner, owner_salt, community_root, root_epoch, control_pk, channels, relays, name, plus control_root when held. It is the only durable home of a member's keys.
• The two snapshots solve opposite problems: seed is the earliest epoch held (backfill anchor), current the latest ("so a fresh device reconstructs the Community instantly with no epoch-by-epoch walk"). seed is omitted when equal to current; embedded snapshots omit community_id (inherited).
• A client holds the complete List when it holds a fragment at every index below frags; it unions fragments and merges, so a partial read is safe.

Two consequences for coop:

1. The state document is a coop invention. store::{save_state, load_state, load_states} write kind 30078 with d = concord/<id>, signed by a per-process LOCAL_KEYS, and never leave the machine. No equivalent exists anywhere in the spec. It is a local cache and must never be treated as the discovery source.
2. Discovery is: subscribe to my 33302 → materialize a community from current join material → subscribe to its planes → fold. The fold produces the authoritative state; the List only supplies the keys to start.

Divergences (coop vs spec)

#	Spec	coop today
1	kind 33302, addressable	was cord02::list::KIND_COMMUNITY_LIST = 13302 (retired) — fixed in Phase A
2	one event per fragment, d = index, frags declared	was no frags, single event, d unused, load_list .limit(1) — fixed in Phase A
3	32-byte values unpadded base64url at any depth	was hex for JoinMaterial.owner/control_root, CommunityId serde, ChannelGrant.key — fixed in Phase A
4	seed omitted when equal to current; embedded snapshot omits community_id; seed's cosmetic fields rewritten from current	was both snapshots emitted verbatim, community_id always present — fixed in Phase A
5	fetch from relays	local database only — fixed in Phase C
6	materialize CommunityState from join material	was no such path; only CommunityState::from_genesis — fixed in Phase B
7	publish the List on create/join (read-modify-write)	build_list_event is referenced only by tests and docs — fixed in Phase D
8	private channel keys ride in join material	ChannelKeyRef has a key field, but private planes are still not subscribed

Divergences 1–7 are resolved. 8 remains, in the narrow sense that planes() still skips private channels rather than deriving their addresses from the granted key.

Plan

Ordered so each phase is independently reviewable and testable. Nothing here touches the frozen HKDF derivations or cord01 envelope semantics.

Phase A — make the List interoperable (pure, no I/O) — DONE

crates/concord/src/cords/cord02/list.rs

1. KIND_COMMUNITY_LIST → 33302; add frags: u64 to CommunityList and is_complete(&self, frags) -> bool.
2. Add a base64url codec for the §8 value set and apply it to every 32-byte field at every depth. Because JoinMaterial currently types owner and control_pk as PublicKey (nostr's hex serde), this needs either wire newtypes or serialize_with/deserialize_with helpers. Keep it local to the List: cord05 stays hex.
3. Implement the two §8 MUSTs: omit community_id on an embedded snapshot, omit seed when it byte-equals current, and rewrite seed's cosmetic fields (name, relays, each channel's name) from current on every serialization.
4. build_list_event/parse_list_event take the fragment index and emit/read the d tag.

Tests: round-trip the examples.md §6.2 payload verbatim; merge convergence for two devices and mixed-age fragments; frags disagreement resolves to the larger value; a repack does not shed unknown fields.

As built. The §8 rules live behind private wire structs (WireList, WireEntry, WireSnapshot, WireChannel), so a writer re-encodes on every serialization while the public types keep their internal hex/PublicKey spellings and cord05 stays hex. Three deviations from the sketch above:

• is_complete takes the set of fragment indices a client holds, not a count: a count is wrong when the indices are sparse.
• The reader tolerates non-zero base64url trailing bits. The spec's own §6.2 example has five such values, so a strict decoder rejects the worked example; the writer still emits the canonical spelling.
• The third omission MUST was implemented too: an entry whose added_at does not outrun its tombstone is not written. It is a serialization rule exactly like the other two, so it belongs here rather than in Phase D.

parse_list_event validates the d tag but returns just the CommunityList; fragment_index(event) reads the index, which kept sync.rs untouched until Phase C. MAX_MEMBERSHIPS = 50 is kept as a stopgap (see risks): §8 has no membership limit, and removing the cap needs write-time fragmentation.

Phase B — materialize a community from join material (pure) — DONE

crates/concord/src/store.rs, crates/concord/src/cords/cord02/list.rs

1. CommunityState::from_join_material(material: &JoinMaterial, added_at_ms: u64) -> Result<Self>: identity/owner/salt/root/root_epoch from the material; control_pks = { root_epoch → control_pk }; relays parsed; channels from the grants; control_root when present; heads empty (the first control fold fills them); banned empty; dissolved false.
2. Carry the private channel key: add key: Option<[u8; 32]> to ChannelKeyRef (or a parallel map) so a grant's key has a home. Without this, a private channel is silently read-only-until-rekey.

Tests: a material with and without control_root; a private grant's key survives; from_join_material then planes() yields the control control_pk plus the guestbook and public channels, i.e. a subscription filter that addresses real planes.

As built. from_join_material does not verify community_id against owner/owner_salt: the List is signed by the member's own key and encrypted to self, and the invite path already validates that binding in CommunityInvite::validate. private on a materialized channel is simply key.is_some() — the spec's channels carry only the Private Channel keys a member was granted, so a grant with no key is a public channel. Nothing else changed: from_genesis and apply_fold construct every channel with key: None, and planes() still skips private channels, whose address derives from the granted key rather than the community_root. Carrying the key is what makes subscribing to them possible later; it is not needed to fix discovery.

Two tests. In concord, from_join_material (with and without control_root, a granted key surviving, a public grant staying keyless). In community, planes() plus subscription_filter over a state built field-by-field (control

• guestbook + public channel addressed, private skipped) — JoinMaterial and ChannelGrant cannot be constructed from community because their extra field's type is crate-private, so the materialization and the plane derivation are each proved where they live.

Phase C — the List drives load — DONE

crates/community/src/sync.rs, crates/community/src/lib.rs

1. subscribe_list(client, self_pk) subscribes to Kind::Custom(33302) author(self_pk) under a dedicated concord/list subscription id, using ReqTarget::auto. With gossip enabled, auto breaks the filter down by author, so it queries the account's NIP-65 write relays and adds/connects them itself — bootstrap relays alone would miss a List published elsewhere.
2. CommunityRegistry calls subscribe_list once per signer (signer change and the initial defer). It is deliberately not called from load: re-subscribing on every List event would re-deliver the List and loop. reset does not unsubscribe it either — subscribe_list replaces the subscription itself, and a reset-issued unsubscribe could race the replacement and cancel discovery.
3. The notification listener routes a concord/list event to a new Signal::List, whose consumer re-runs load. Community planes keep using Signal::Event(id).
4. load_list reads every 33302 event by self_pk from the database, keeps the newest event per fragment index, decrypts and merges them. .limit(1) is gone. An incomplete List is read normally — a missing fragment is news not yet heard.
5. load unions two sources: every live List entry (materialized with from_join_material, or refreshed if a state document already exists) and every held local state the List does not mention. A held membership is dropped only when a tombstone outranks its added_at_ms; absence from the List is never a fact. Each list-derived state is save_stated, so the next load is warm.
6. refresh(held, fresh) keeps the fold's authority (heads, banned, dissolved) and the control planes it learned, and takes the List's identity, relays, and channel keys. Channels are merged by id rather than replaced, so a public channel the fold discovered is not shed by a List snapshot that predates it.

As built, deviating from the sketch above. The plan called for client.fetch_events(..); the SDK's own recommendation is to keep the request path on a subscription and read the database. This is safer than it sounds: a relay's event is persisted at nostr-sdk/src/relay/inner.rs:1291 before the notification is emitted, so a subscription plus a database read loses nothing and needs no explicit save. The subscription is set up with ReqTarget::auto rather than a hand-built NIP-65 relay map, because gossip already resolves the author's write relays and connects them on demand.

Tests (no network, in crates/community/src/sync.rs): a membership the List carries materializes a community even though no state document was ever written for it, and discovery writes the document so the next load is warm; a held membership the List never mentions is kept alongside the one it does; a tombstone outranks a held membership and drops it; and the concord/list id is not read as a community subscription. Fragment events are built with store::list_entry + CommunityList::joined + build_list_event and saved straight into a memory database, so the tests exercise the real seal/parse/merge path without a relay.

Phase D — publish — DONE

crates/community/src/sync.rs, crates/concord/src/store.rs, crates/concord/src/cords/cord02/list.rs

1. create mints the genesis, folds it into a state, and saves that state locally as before, then announces the community: the genesis wraps to its relay set, and the membership to the account's own List. Both publishes are best-effort — a relay that is down is a warning, not a failed create.
2. The List write is a read-modify-write over the copy already held (§8). create reads the newest held fragment, unions its own entry in with CommunityList::joined, builds fragment 0, and publishes it. Publishing saves it locally as a side effect of send_event, before any relay is resolved, so the fragment survives a relay that is down and no explicit database write is needed.
3. The fragment's created_at is max(now, previous + 1), so an addressable relay can never quietly keep the copy the write meant to replace.

As built, deviating from the sketch above. Three decisions the sketch did not cover:

• The List goes to the account's NIP-65 write relays (.to_nip65()), not the community's metadata relays. The List is the member's own document, and it is the same relay set subscribe_list resolves for its author filter — the two halves must agree or a write can land where nothing reads. The genesis wraps, which belong to the community and not the member, do go to the metadata relays.
• The entry is built by a new store::list_entry(state, name). JoinMaterial' extra field is crate-private, so the community crate cannot build one; name is passed in because the state does not carry it — the name lives in the Control fold, and a created community has it in the metadata.
• A List that already spans more than one fragment is left alone: placing a new membership needs a repack (which fragment does it belong in?), and §8 allows a repack only against the complete List. load keeps a membership the List never mentions, so the community is still tracked locally; the remote write is deferred with a warning rather than performed wrongly.

Tests: create records a membership the List round-trips, and a second create unions into the same document instead of replacing it.

Phase E — verify live

RUST_LOG=info cargo run -p coop, sign in with the accordion account that already belongs to communities. Expect community {id}: subscribing to .. and rows in the sidebar. This is the first time the path can be exercised at all.

Validation per phase

• cargo test -p concord (A, B), cargo test -p community (B, C, D).
• cargo clippy --workspace --all-targets, cargo fmt --all -- --check.
• A is provable against the spec's worked example, so it needs no relay.
• C is provable with nostr-memory: fragments are built with build_list_event and saved as the subscription would have, then load reads them. No relay, no LocalRelay.
• E is the only step that needs real relays.

Risks and open decisions

• Base64url is case-significant and coop's ids are hex everywhere else. Confine the codec to cord02::list; any normalisation that case-folds will silently corrupt §8 values. Resolved in Phase A: the codec is private to list.rs and never case-folds.
• MAX_MEMBERSHIPS = 50 is not in the spec. §8 has no membership limit; its only bound is the 65,536-byte encoded event. fits() still measures the NIP-44 plaintext, which understates that by roughly a third. Phase D kept the count cap and added a guard: a List that already spans more than one fragment is not appended to, because placing a new membership needs a repack. So a member with more than one fragment gets no remote write until fragmentation lands; the community stays local and visible.
• Relay selection is the difference between finding the account's List and not. Resolved in Phase C by ReqTarget::auto, whose gossip path resolves the filter's author to their NIP-65 write relays and connects them. A List published only to relays with no NIP-65 entry is still unreachable; that is a user-visible relay setting if it ever bites.
• Private channels stay unsubscribed until planes() derives their address from the granted key (Phase B gave ChannelKeyRef a home for it, but the discovery fix does not need it). Public discovery works regardless.
• Two writers, one key. Once coop publishes 33302, an account used from both accordion and coop has both clients writing the List. §8's read-modify-write is what keeps that from losing memberships — it is not optional.
• A create racing the first list sync can publish over an unseen List. record_membership unions into what the local database holds, and on a fresh sign-in that is empty until the concord/list subscription has delivered. A create in that window writes a one-entry fragment 0, and an addressable relay then replaces the account's fuller List with it. The window is the ordinary sign-in-to-create interval, so it is small but not zero. The honest fix is to treat the List write as part of the sync loop — republish list ∪ local memberships whenever the subscription settles — rather than doing it inside create; an EOSE flag is not enough on its own, because an account with no NIP-65 relays never reaches EOSE and would then never write at all.
• store::save_state signs with a per-process random key. Harmless while it stays local, but it means the state document can never be published or compared; if a future phase wants it on the wire, it needs the account signer.
• The deployed reference client still writes the retired kind 13302. The spec this plan implements (concord-protocol/concord main) moved the List to 33302 in PR #18, merged 2026-08-15. The applesauce concord branch that accordion.chat builds against still declares 13302, single-event, capped at 50 memberships, at its head of 2026-08-05; accordion's pin predates even that (0.0.0-concord-20260804145327). So an account whose memberships were written by that build stores them under a kind coop deliberately does not read, and will show an empty sidebar until the client is updated to the fragmented kind. This is not a bug in the discovery path — Phases C and D are correct against the current spec — but it is the first thing to check if a live sign-in still shows nothing. Supporting 13302 alongside 33302 is a deliberate non-goal until the reference client moves.