Concord is an encrypted community/channel protocol over Nostr: shared-key "Private Streams" (CORD-01), communities with a self-certifying owner id and three planes (CORD-02), public/private channels (CORD-03), an owner-rooted signed roster (CORD-04), invites (CORD-05), rekeys/refoundings (CORD-06) and disappearing messages (CORD-08).
Scope of this plan: **backend + public Rust API**. No views, no widgets, no copy.
Appendix A (derivations) and Appendix B (kinds) of CORD-02 are **frozen**: every labeled byte and every kind number is part of the wire format. Treat both as constants with golden-vector tests.
Reference implementations for cross-checking behaviour (not for copying code): Vector (`crates/vector-core/src/community/v2/*`), Armada, Grimoire.
## 3. Reuse map — nostr-sdk APIs we build on
Verified against the pinned revision (`nostr` 0.45.4 / `nostr-sdk` 0.45.2, git `b230cec`).
| Concord need | Existing API |
| --- | --- |
| NIP-44 under a raw conversation key | `nostr::nips::nip44::v2::{ConversationKey, encrypt_to_bytes_with_nonce, decrypt_to_bytes}` |
| NIP-44 conversation key for a keypair | `ConversationKey::derive(&SecretKey, &PublicKey)` (self-ECDH for streams) |
| NIP-44 under a signer | `nip44::{encrypt, decrypt}` (already wrapped by `state::UniversalSigner`) |
**Not needed.**`secp256k1` (use `nostr::SecretKey::from_slice` + `Keys::new`), `base64` (use `data_encoding::BASE64`, already a workspace dep), `bech32` (NIP-19 is in the SDK), any new storage engine (the client's LMDB database is enough), any new HTTP client.
**Add one dependency:**`hkdf = "0.12"` (already in `Cargo.lock` transitively). Add it to `[workspace.dependencies]` and to the new crate. `sha2` is already a workspace dep.
## 4. Crate layout
New crate `crates/concord`, picked up automatically by the `crates/*` workspace member glob.
```
crates/concord/
Cargo.toml
src/lib.rs init, ConcordRegistry, ConcordEvent, signal bus, subscriptions, ingest pipeline
src/store.rs local persistence + opened-rumor cache + history queries
```
`Community` and `Channel` GPUI entities live in `src/lib.rs` next to the registry — they are the public surface, not a separate concern. Ten modules, each with real content; no single-fn files.
Dependencies: `common`, `state`, `person`, `device`, `settings`, `gpui`, `nostr` (for `nip44` features), `nostr-sdk`, `hkdf`, `sha2`, `serde`, `serde_json`, `anyhow`, `flume`, `smallvec`, `itertools`, `futures`, `log`, `instant`. Everything under `cfg(not(target_arch = "wasm32"))` follows the `chat`/`state` split so the crate still builds for web.
## 5. Core types
```rust
pubstructCommunityId([u8;32]);// sha256 commitment, never on the wire
pubstructChannelId([u8;32]);
pubstructEpoch(pubu64);
/// A derived stream: signing keypair + the self-ECDH conversation key that
/// encrypts the wraps. Memoised in a bounded process-wide cache.
pubfninvite_bundle_key(token: &[u8;16])-> [u8;32];// raw hkdf32 output; used as a NIP-44 conversation key
```
Rules that must be enforced by construction, not by convention:
- Hex is lowercase everywhere; pubkeys are x-only hex, never bech32; tag numbers are decimal with no leading zeros (`"4"`, never `04`/`+4`).
- The epoch field is *omitted*, not zeroed, for labels with no epoch (`concord/dissolved`, locators, `concord/community`).
-`scalar_normalize` retries by appending a counter byte to the same `info`, starting at `0`.
- Labels and commitments are append-only. A test asserts every label is unique and that the label table matches Appendix A.6 exactly.
**Golden vectors.**`derive.rs` carries a `#[cfg(test)]` block pinning every derivation output, seeded from the independent Python vectors published by the Vector implementation (channel/control/control-signer/guestbook at epoch 0 and at `0x0102030405060708`, dissolved, all four locators, invite key, community id, epoch commitment). One vector is missing upstream — `pins_locator` — so we mint it from our own implementation and pin it, flagged in the test as self-referential. Changing any pinned value means the wire format changed.
- The wrap is signed by the **stream key** with a random ephemeral `p` tag — NIP-59 reversed. `extra` is how the caller mirrors a NIP-40 expiration onto the wrap.
- The seal is signed by the **real author** and carries `created_at` equal to the rumor's. It is never published bare.
- Control plane **must** use the plaintext seal; chat, guestbook and rekey planes **must** use the encrypted one. Each plane asserts its own form at both ends.
- The control plane is a write-restricted stream: the wrap key derives from `control_root` while the content is encrypted under the `community_root`-derived conversation key. `open_wrap_at` takes the two halves separately for this reason.
- Open order: kind → address match → wrap signature (only when `verify_wrap_sig`) → NIP-44 open → seal kind → seal signature → rumor parse → `rumor.pubkey == seal.pubkey` → recompute the rumor id and reject a mismatch → strict `ms` resolve.
- Enforce the 65 535-byte NIP-44 cap at every nesting layer before publishing.
- Do not verify the wrap signature for ordinary planes: every reader holds the group key, so the signature proves nothing. It matters only for the restricted control plane and for rekeys.
- The ephemeral wrap keypair is returned to the caller so a client may NIP-09-scrub its own wrap later.
## 8. Planes, state and folds
### 8.1 Editions and authority (`edition.rs`, `control.rs`)
- Tag grammar: `["vsk", sub]`, `["eid", hex32]`, `["ev", decimal]`, `["ep", hex32]`, `["vac", eid, version, hash]`. Duplicates of any of the five reject the edition; `ev` must pass a decimal check before parsing.
- Tie-break at equal version is the lower **inner rumor id**, never `created_at`.
-`gap` is a safety signal: a tracking client (already holds the floor) fails closed for that entity and refetches; a bootstrapping client (floor 0) may accept the highest authority-verified head, which is what makes compaction re-wrapping work.
- Entity coordinates are `vsk 0` → `community_id`, `1` → `role_id`, `2` → `channel_id`, `3` → `grant_locator`, `4` → `banlist_locator`, `8` → `invite_links_locator`, `11` → `pins_locator`. All derive from `community_id` only, so a refounding re-wraps heads verbatim.
```rust
pubconstP_MANAGE_ROLES: u64=1<<0;// …bit table from CORD-04 §3, frozen; retired bits are burned
pubfnis_staff(&self,member: &PublicKey,owner: &PublicKey)-> bool;// the six control bits, CORD-04 §3
}
```
Authority rules to encode once and test hard:
- The owner is position 0, derived from `community_id`, and is never removable.
- No edition may claim a `position` at or above its own signer's, including the owner: no Role may claim 0.
- The actor must hold the required bit **and strictly outrank** the target. Equal cannot act on equal.
- A `vac` citation is a sync floor, not a verdict: block until the cited Grant version is folded, verify its hash, then judge against the *current* roster.
- A staff-making Grant carries `control_wrap`, a NIP-44 pairwise ciphertext of `epoch_be[8] ‖ control_root[32]`, and is adopted **only if it derives to the `control_pk` the member already holds** for the named epoch.
- Banlist is one replaced entity; mutations carry a re-heal step (re-fold after publish, re-apply if the addition lost the tiebreak).
### 8.2 Communities, channels, metadata
`CommunityMetadata` carries `name` (≤ 64 bytes), `description` (≤ 10 000 bytes), `relays` (truncated on read and write to 5), `icon` and `banner` as encrypted-blob pointers (`{url, key, nonce, hash}`), `message_expiration`, and the optional `custom` object. `ChannelMetadata` carries `name`, `private`, optional `voice`, `deleted`, optional `custom`.
Every content struct uses `#[serde(flatten)] extra: serde_json::Map<String, Value>` and round-trips unknown fields. A name edit by an older client must not wipe another client's `custom` keys. Round-trip discipline gets its own test.
Channel keying follows CORD-03 §1: a public channel derives from `community_root` at the base epoch, a private one from its own random key at its own epoch. Public→private is a rekey at the next channel epoch (monotonic, never reset); private→public derives from the base again and the prior private history stays unreadable to later joiners.
### 8.3 Guestbook and member list (`guestbook.rs`)
- Entries dated more than an hour ahead of local time are dropped. An `ms` outside `0..999` drops the entry rather than being interpreted.
- Coalesce is per npub, one final state each, by millisecond time, ties on the lower inner rumor id.
- A Kick counts only when its signer holds `KICK` and outranks the target.
- A Snapshot counts only from the npub whose Refounding minted the epoch. There is deliberately no owner fallback.
- The member list is `coalesced Joined ∪ observed authors − banlist`, and observation counts **forward only** (an author re-enters on activity newer than their latest Leave/Kick/Ban). A Grant holder with `ms == 0` is present by construction.
### 8.4 Chat plane (`chat.rs`)
```rust
pubstructChatMessage{
pubid: EventId,// recomputed rumor id
pubauthor: PublicKey,
pubchannel: ChannelId,
pubepoch: Epoch,
pubkind: Kind,// 9 | 1111 | 3302 | 1740 | 15
pubcontent: String,
pubmedia: Vec<SharedUri>,
pubmentions: Vec<Mention>,
pubreply_to: Option<EventId>,// lowercase `e`/`q`
pubthread_root: Option<EventId>,// uppercase `E` for 1111
pubat_ms: u64,
pubexpiration: Option<Timestamp>,
pubedited_at: Option<u64>,// folded from 3302
pubdeleted: bool,// folded from 5
pubreactions: BTreeMap<PublicKey,String>,
}
```
Sends funnel through one function so the rules cannot drift:
It builds the seal + wrap, mirrors the NIP-40 tag onto the wrap for durable kinds, publishes via `send_event(..).to(relays)`, retains the ephemeral wrap key for later NIP-09 scrubbing, and locally echoes its own wrap through the same ingest path so send-then-read works without waiting on a relay round-trip.
Disappearing messages (CORD-08) live here: `message_expiration` is read from the folded metadata, `["expiration", created_at + t]` is attached to every durable Chat rumor and to the wrap, kinds 5 and 1740 are exempt, ingest refuses an already-expired rumor, a periodic sweep purges stored ones, and the kind 1740 timer notice renders only when its author holds `MANAGE_METADATA`.
The link rides `naddr` (`Nip19Coordinate` for kind 33301, link signer, empty `d`) in the path and the token + bootstrap relays in the fragment. A fragment is never sent to a server. The bundle is decrypted with `invite_bundle_key(token)`, and the joiner must recompute `community_id` from `owner` + `owner_salt`.
Bounds before allocation: reject a bundle with more than 256 channels, truncate the relay list to 5, refuse an expired one.
- The subscription for rekeys is precomputed from the *next* epoch's address, per private channel and once for the base.
- A receiver accepts a key only after: locating its blob, decrypting with the rotator↔recipient conversation key, checking the bound `scope` and `epoch` inside the plaintext, and matching `prevcommit` against the key it currently holds.
- Only after holding **all**`n` chunks of one `(rotator, newepoch, prevcommit)` set, with none containing its locator, may a client conclude it was removed.
- Send cap 80 blobs per event, accept cap 120 (Vector's documented erratum: the CORD-01 double envelope pushes 120 blobs past a 64 KB relay limit). Record the reason in a comment so nobody "fixes" it back.
- Compacted control heads are re-wrapped with their original signature intact, which is exactly why the control plane uses the plaintext seal.
- Two concurrent refoundings converge on the lexicographically lowest new base key; the heal is down-only.
- Authority: a channel rekey needs `MANAGE_CHANNELS`, a refounding needs `BAN`, and in both the rotator must strictly outrank every removed target. Holding a key is never authority.
Dissolution (CORD-02 §9) also lives here: a chainless, owner-signed `vsk 10` tombstone at `dissolved_group_key(id)`, plaintext-sealed, and a verifier **must** refuse any tombstone whose `eid` is not the community's own id (including the all-zero placeholder — accepting it lets an owner's genuine tombstone for one community be re-wrapped at another of theirs and kill it permanently). On sight the community is sealed read-only: subscriptions halt, nothing new is honored, existing history stays readable, and a member's delete of their own message is still honored.
## 9. Storage (`store.rs`)
Three layers, no new storage engine:
1.**Raw wraps** (kind 1059) are persisted automatically by the SDK's relay pool when a subscription or fetch matches a filter. Nothing to write.
2.**Opened rumors** are cached locally as NIP-78 `Kind::ApplicationSpecificData` events signed by a session-local keypair, exactly like `chat::set_rumor`. Tags: `["d", rumor_id]` (replace key), `["c", channel_hex]`, `["p", author]`, `["k", kind]`, `["e", wrap_id]`, `["t", "concord"]`. Contents are the rumor JSON.
- The `c`/`t` keys deliberately differ from chat's `r` key so the two message namespaces can never collide in one database.
- The read path dedupes by rumor id and keeps the newest `created_at`, because the local signing key changes per session.
3.**Community state** — one local document per community, `Kind::ApplicationSpecificData` with `["d", "concord/<community_id>"]`:
```rust
pubstructCommunityState{
pubid: CommunityId,
pubowner: PublicKey,
pubowner_salt: [u8;32],
pubcommunity_root: [u8;32],
pubroot_epoch: Epoch,
pubcontrol_root: Option<[u8;32]>,// present iff the holder is staff
Writes are debounced (a fold head changes on every edition); reads load once at init.
**Decision, stated for the record:** this document stores raw community keys unencrypted in a plaintext local database. That matches the existing posture — `chat` already caches decrypted message rumors in the same LMDB. If that posture ever changes, the state document is the one to wrap with NIP-44-to-self, since it is the only local artifact holding keys.
`query_messages` reads the local cache (`Filter::new().kind(ApplicationSpecificData).custom_tag(LOWERCASE_C, channel_hex)`); `backfill` pages relays newest-first with `until`, deduplicating by wrap id and stepping past same-second walls.
## 10. Sync engine and GPUI conventions
`ConcordRegistry` mirrors `ChatRegistry`'s shape exactly: a foreground GPUI entity holding `Entity<Community>` handles, a `flume` signal bus, one background notification listener, one foreground consumer, and task slots that are cleared when the signer changes.
**Subscription.** Community relays come from the folded metadata. `init`/`join` add them to the client (`client.add_relay(url).and_connect()`), then:
Targeted subscribe against the community relays, with a pool-wide subscribe as the fallback path. Rebuild idempotently whenever a plane's address changes (join, channel added, rekey folded).
**Routing.**`dispatch` matches on the `subscription_id` carried in `RelayMessage::Event`, dedupes by wrap id (both subscriptions and several relays deliver the same wrap), then recognises the plane by **wrap author** against the derived addresses it holds — never by trial decryption. Recognition order: held channel planes → guestbook → control signer → rekey addresses → dissolved.
**Ingest pipeline.** Unwrap, verify and fold all happen inside `cx.background_spawn`, never on the foreground thread: secp256k1 verification per edition and per seal is far too expensive for the UI thread.
The consumer is `cx.spawn(async move |this, cx| { while let Ok(signal) = rx.recv_async().await { this.update(cx, |this, cx| this.apply(signal, cx))?; } })`, which updates entities and calls `cx.notify()`.
Rules taken from the project guidelines:
- Crypto, folding, database queries and network I/O only in `cx.background_spawn`.
- Foreground tasks are `cx.spawn` with `this.update(cx, ..)`; any entity update happens there, and the inner `cx` is always used.
- Tasks are stored in fields (`tasks`, `listener`, `consumer`) so they are cancelled on signer change and dropped with the registry. `detach()` only for genuinely fire-and-forget work such as the local state save.
- Long-running paging is bounded by explicit page and step caps, not by unbounded loops.
- Every fallible path returns `Result` and surfaces through `ConcordEvent::Error`; nothing is silently swallowed.
`CommunityEvent` and `ChannelEvent` mirror `ChatEvent`: one variant per thing the UI has to react to (`Updated`, `Members`, `Added`, `Removed`, `Dissolved`, `Error`, plus channel-level `Incoming`, `Reload`).
## 11. Integration with existing crates
1.**`crates/chat/src/lib.rs` — required fix.** `handle_notifications` currently treats *every* kind 1059 event as a NIP-17 wrap for the current user and pushes an unwrap failure into the trash. Concord wraps are kind 1059 with an ephemeral `p` tag, so they would flood the trash and leak error toasts. Route by `subscription_id` from `RelayMessage::Event` against `sub_id1`/`sub_id2`, and drop the `if rumor.tags.is_empty()` heuristic once the real recipient check is in place.
2.**`desktop/src/main.rs` and `web/src/lib.rs`** — add `concord::init(window, cx)` after `chat::init(window, cx)`.
3.**`Cargo.toml`** — add `hkdf = "0.12"` to `[workspace.dependencies]`; add the crate to `desktop` and `web` dependencies. No other workspace changes.
4.**No changes** to `state`, `person`, `device`, `settings`, `common`, or `ui`.
## 12. Security invariants to test, not to assume
Each of these has burned a real implementation, or is a documented cross-client trap:
- Recompute every rumor id and reject a claimed mismatch; never trust an embedded `id`.
- Require `rumor.pubkey == seal.pubkey`.
- Require the plaintext seal form on Control and the encrypted form on Chat/Guestbook/Rekey — a strict reader must drop a mis-sealed edition rather than fold a chain a later compaction would fork.
- Check `channel`**and**`epoch` against the plane whose key opened the wrap; reject duplicates of either tag.
- Reject duplicate `vsk`/`eid`/`ev`/`ep`/`vac` tags; require decimal-with-no-leading-zeros on every numeric tag.
- Refuse a tombstone whose `eid` is not this community's id.
- Adopt a `control_root` from a Grant only if it derives to the `control_pk` held for that epoch; adopt a rekey blob only if its bound plaintext matches the scope and epoch and its `prevcommit` matches the key currently held.
- Never conclude removal from a partial rekey chunk set.
- Drop guestbook entries more than an hour in the future; treat an out-of-range `ms` as malformed, not as an interpretation opportunity.
- Never honour a Snapshot from anyone but the refounder of that epoch.
- Refuse to write a Pin List from a list the writer could not read.
- Enforce the NIP-44 65 535-byte cap at every layer before publishing, and the 5-relay / 256-channel / 50-membership / 100-roles / 64-roles-per-member / 500-banlist / 25-pin caps at their ingest and write points.
- Lowercase hex only; x-only pubkeys only; no version tag anywhere.
## 13. Milestones
| # | Deliverable | Done when |
| --- | --- | --- |
| M0 | Crate skeleton, `derive.rs`, golden vectors, workspace wiring | `cargo test -p concord` pins every derivation from an independent vector set; all labels match Appendix A.6 |
| M1 | `stream.rs` + `store.rs` | seal/wrap/open round-trips for both seal forms; malformed inputs rejected in the documented order; local cache reads back after a restart |
| M2 | `edition.rs` + `control.rs` genesis | a community is created and published; its two genesis wraps open at a second client sharing the keys; edition hash matches the cross-client vector |
| M3 | Control fold + roster + metadata/channels | fold tests for chains, gaps, downgrade refusal, fork tiebreak, compaction dangle; metadata and channel edits visible to a second client |
| M4 | Chat plane | send/receive/edit/delete/reaction across two identities; history pages backwards across an epoch boundary; binding checks reject a foreign channel/epoch |
| M5 | Guestbook + member list + moderation | join/leave/kick/ban converge to the same memberlist on both clients; every authority rule from §8.1 has a negative test |
| M6 | Invites + Community List | link mint → fetch → join round-trips; revoked link refuses; direct invite lands in the recipient's giftwrap inbox via the `k` tag; a second device reconstructs membership from 13302 |
| M7 | Rekeys + refounding + dissolution | a removed member stops reading after a rekey; continuity and race rules tested; a tombstone seals the community and a foreign-id tombstone is refused |
| M8 | Pins + disappearing messages + hardening | pins verify from a keyless reader's view; expiry is refused at ingest and purged by the sweep; the audit of §12 is complete with a test per bullet |
Ordering is deliberately dependency-first: each milestone is usable on its own, and nothing in M2+ depends on a later milestone.
## 14. Open questions and risks
1.**Community List kind.** CORD-02 §8 specifies `13302`, replaceable. Vector has retired it in favour of fragmented `33302`, because a replaceable kind holds one event per pubkey and so cannot shard past the NIP-44 size cap. We implement `13302` per spec, enforce the 50-membership cap and pre-publish size check, and treat `33302` as an interop follow-up. Confirm with Armada before writing the multi-device code.
2.**NIP-42 for stream-authored REQs.** Relays that gate kind 1059 by author (for example `ditto-relay`'s `AUTH_KINDS`) need an AUTH event signed by that plane's derived key. `nostr-sdk`'s `Authenticator` is per-client and returns one identity, while a Concord client holds many plane keys — so this cannot be solved by swapping the authenticator. Options: contribute a per-REQ auth hook upstream, or accept that such relays are unsupported and prefer relays without the gate. Decide before M7; the default is "documented limitation" plus a relay-capability check.
3.**`invite_bundle_key`.** Appendix A.6 says the label "yields the public-invite decrypt key" without stating whether that is the raw HKDF output used as a NIP-44 conversation key or the `conv_key` of a normalized keypair. The reference implementation uses the raw output. Pin a vector and verify against Armada early — this one decides whether links open at all.
4.**Missing golden vector for `pins_locator`.** Upstream publishes none. Ours will be self-referential; flag it in the test.
5.**Relay set.** Up to 5 recommended, and both reads and writes fan out across them. Coop's client is a gossip client with `no_background_refresh`, so community relays must be added explicitly and re-added on metadata change.
6.**Local plaintext state.** §9 records the decision. Revisit only if the local database stops being treated as trusted.
7.**Was a `community_id` ever hashed into a tag?** No — it must never appear on the wire. A lint-style test should assert it appears in no published event other than inside an invite bundle and a direct invite.
## 15. Test strategy
- **Unit, pure:** derivations against golden vectors, edition hash, fold, coalesce, memberlist, blob codec, caps and rejection paths. These need no GPUI context and should be exhaustive — they are where cross-client divergence is caught.
- **Integration, GPUI:** `TestAppContext` with two registries sharing an in-memory database, driving wraps through the ingest path; timeouts and delays use `cx.background_executor().timer(..)` per the project guidelines, never `smol::Timer`.
- **Round-trip:** every builder paired with its parser, asserting the parse produces the identical structure, including unknown-field round-tripping on all content types.
- **Negative:** every bullet in §12 gets a test that constructs the hostile input and asserts the drop.
Blocking a user prevents them from interacting with repositories, such as opening or commenting on pull requests or issues. Learn more about blocking a user.
reya