From 319b1038d2dbd476082a64acbc078b8a3589e99e Mon Sep 17 00:00:00 2001 From: Ren Amamiya Date: Wed, 16 Sep 2026 11:03:33 +0700 Subject: [PATCH] add plan --- PLAN.md | 560 ++++++++++++++++++++++++++++++++++++++++++++++++++++++++ 1 file changed, 560 insertions(+) create mode 100644 PLAN.md diff --git a/PLAN.md b/PLAN.md new file mode 100644 index 00000000..048a6a01 --- /dev/null +++ b/PLAN.md @@ -0,0 +1,560 @@ +# Concord support — backend & API plan + +Concord is an encrypted community/channel protocol over Nostr: shared-key "Private Streams" (CORD-01), communities with a self-certifying owner id and three planes (CORD-02), public/private channels (CORD-03), an owner-rooted signed roster (CORD-04), invites (CORD-05), rekeys/refoundings (CORD-06) and disappearing messages (CORD-08). + +Scope of this plan: **backend + public Rust API**. No views, no widgets, no copy. + +## 1. Scope + +**In scope** + +- New `crates/concord` crate: derivations, stream codec, control/chat/guestbook planes, authority fold, ephemeral state, invites, rekeys, dissolution, storage, sync engine. +- Public API for a future UI: `ConcordRegistry` + `Entity` / `Entity` + events, mirroring the shape of `crates/chat`. +- The minimum surgical edits to existing crates required for coexistence (see §11). + +**Out of scope** + +- Any UI work. +- CORD-07 audio/video. Reserve `23313`, the `concord/voice-*` labels and the `voice` metadata flag so nothing else claims them, and implement nothing. +- Pins (CORD-04 §7) ship in the last milestone; the design accounts for `vsk 11` early so the fold is not retrofitted. +- Cross-client interop testing (Vector/Armada/Grimoire). Tracked as follow-up work, not blocking. + +## 2. Sources of truth + +| Doc | What we take from it | +| --- | --- | +| CORD-01 | Stream event shape, seal forms 20013/20014, encoding rules, binding, deletions | +| CORD-02 | `community_id`, `community_root`, `control_root`, epochs, 3 planes, metadata, invites, Community List, dissolution | +| CORD-03 | Channel keying, metadata, message kinds, `channel`/`epoch` binding, threads vs quotes | +| CORD-04 | Editions, `vac`, the roster, permission bits, banlist, the three removals, pins | +| CORD-05 | Bundle, link (naddr + fragment), relay dictionary, Invite List, Registry, Direct Invite | +| CORD-06 | Rekey blobs, chunking, `prevcommit` continuity, Refounding, compaction, races | +| CORD-08 | `message_expiration`, NIP-40 tagging, ingest/purge enforcement, timer notice 1740 | + +Appendix A (derivations) and Appendix B (kinds) of CORD-02 are **frozen**: every labeled byte and every kind number is part of the wire format. Treat both as constants with golden-vector tests. + +Reference implementations for cross-checking behaviour (not for copying code): Vector (`crates/vector-core/src/community/v2/*`), Armada, Grimoire. + +## 3. Reuse map — nostr-sdk APIs we build on + +Verified against the pinned revision (`nostr` 0.45.4 / `nostr-sdk` 0.45.2, git `b230cec`). + +| Concord need | Existing API | +| --- | --- | +| NIP-44 under a raw conversation key | `nostr::nips::nip44::v2::{ConversationKey, encrypt_to_bytes_with_nonce, decrypt_to_bytes}` | +| NIP-44 conversation key for a keypair | `ConversationKey::derive(&SecretKey, &PublicKey)` (self-ECDH for streams) | +| NIP-44 under a signer | `nip44::{encrypt, decrypt}` (already wrapped by `state::UniversalSigner`) | +| Event id recomputation | `EventId::compute(pubkey, created_at, kind, tags, content)`, `UnsignedEvent::compute_id` | +| Event (de)serialization | `Event::{from_json, as_json, verify}`, `UnsignedEvent::from_json` | +| Building events | `EventBuilder::new(..).tags(..).custom_created_at(..)`, `FinalizeEvent::finalize(&Keys)` | +| Tags | `Tag::{custom, identifier, public_key, expiration}`, `Tags`, `SingleLetterTag` | +| Kinds | `Kind::GiftWrap` (1059), `Kind::Custom(21059|20013|20014|3308|…)`, `Kind::is_ephemeral` | +| Publish | `Client::send_event(&event).to(relays).ack_policy(AckPolicy::none())` | +| Subscribe | `Client::subscribe(target).with_id(..).close_on(..)`, `SubscribeAutoCloseOptions`, `ReqExitPolicy` | +| Backfill | `Client::fetch_events(target)`, `Client::stream_events(target)` | +| Local persistence | `Client::database()` → `query(Filter)`, `save_event(&Event)` | +| Invite link parsing | `Nip19::from_bech32` → `Nip19::Coordinate(Nip19Coordinate)` | +| Signer abstraction | `state::UniversalSigner` (`AsyncSignEvent` + `AsyncNip44`) | +| Relay auth | `nostr_sdk::{Authenticator, SignerAuthenticator}` | + +**Not needed.** `secp256k1` (use `nostr::SecretKey::from_slice` + `Keys::new`), `base64` (use `data_encoding::BASE64`, already a workspace dep), `bech32` (NIP-19 is in the SDK), any new storage engine (the client's LMDB database is enough), any new HTTP client. + +**Add one dependency:** `hkdf = "0.12"` (already in `Cargo.lock` transitively). Add it to `[workspace.dependencies]` and to the new crate. `sha2` is already a workspace dep. + +## 4. Crate layout + +New crate `crates/concord`, picked up automatically by the `crates/*` workspace member glob. + +``` +crates/concord/ + Cargo.toml + src/lib.rs init, ConcordRegistry, ConcordEvent, signal bus, subscriptions, ingest pipeline + src/derive.rs frozen HKDF / group_key / locators / commitments + golden vectors + src/stream.rs CORD-01: seal + wrap + open (SealForm, OpenedStream), channel/epoch binding + src/edition.rs CORD-04 §1: canonical signing bytes, edition hash, parse, fold + src/control.rs control plane view, genesis, content types, roster fold, authority checks + src/guestbook.rs CORD-02 §5: join/leave/kick/snapshot, coalesce, complete memberlist + src/chat.rs CORD-03: channel plane — message/edit/delete/reaction builders + message view + src/invite.rs CORD-05: bundle, link, registry, Invite List, Direct Invite + src/rekey.rs CORD-06: blob codec, continuity, refounding, compaction, dissolution + src/store.rs local persistence + opened-rumor cache + history queries +``` + +`Community` and `Channel` GPUI entities live in `src/lib.rs` next to the registry — they are the public surface, not a separate concern. Ten modules, each with real content; no single-fn files. + +Dependencies: `common`, `state`, `person`, `device`, `settings`, `gpui`, `nostr` (for `nip44` features), `nostr-sdk`, `hkdf`, `sha2`, `serde`, `serde_json`, `anyhow`, `flume`, `smallvec`, `itertools`, `futures`, `log`, `instant`. Everything under `cfg(not(target_arch = "wasm32"))` follows the `chat`/`state` split so the crate still builds for web. + +## 5. Core types + +```rust +pub struct CommunityId([u8; 32]); // sha256 commitment, never on the wire +pub struct ChannelId([u8; 32]); +pub struct Epoch(pub u64); + +/// A derived stream: signing keypair + the self-ECDH conversation key that +/// encrypts the wraps. Memoised in a bounded process-wide cache. +pub struct GroupKey { keys: Keys, conversation: ConversationKey } +impl GroupKey { + pub fn pk(&self) -> PublicKey; + pub fn keys(&self) -> &Keys; + pub fn conversation(&self) -> &ConversationKey; +} + +pub enum SealForm { Encrypted, Plaintext } + +pub struct OpenedStream { + pub rumor_id: EventId, + pub author: PublicKey, // the seal's verified pubkey + pub seal_form: SealForm, + pub seal: Event, // retained: compaction re-wraps plaintext seals verbatim + pub wrapper_id: EventId, + pub at_ms: u64, // created_at * 1000 + ms tag + pub rumor: UnsignedEvent, +} +``` + +Ordering everywhere uses `at_ms`, never `created_at`, and ties break on the lower inner rumor id. + +## 6. Frozen derivations (`derive.rs`) + +```rust +fn build_info(label: &str, id: &[u8; 32], epoch: Option) -> Vec; // label ‖ 0x00 ‖ id[32] ‖ epoch_be[8]? +fn hkdf32(ikm: &[u8], info: &[u8]) -> [u8; 32]; // HKDF-SHA256, zero-length salt, L = 32 +fn hkdf_to_secret_key(ikm: &[u8], info: &[u8]) -> SecretKey; // A.3 scalar_normalize, counter from 0 + +fn group_key(label: &str, secret: &[u8], id: &[u8; 32], epoch: Option) -> GroupKey; + +pub fn channel_group_key(secret: &[u8; 32], channel: &ChannelId, epoch: Epoch) -> GroupKey; +pub fn control_group_key(root: &[u8; 32], id: &CommunityId, epoch: Epoch) -> GroupKey; // read key +pub fn control_signer_group_key(root: &[u8; 32], id: &CommunityId, epoch: Epoch) -> GroupKey; // write key +pub fn guestbook_group_key(root: &[u8; 32], id: &CommunityId, epoch: Epoch) -> GroupKey; +pub fn channel_rekey_group_key(root: &[u8; 32], channel: &ChannelId, epoch: Epoch) -> GroupKey; +pub fn base_rekey_group_key(root: &[u8; 32], id: &CommunityId, epoch: Epoch) -> GroupKey; +pub fn dissolved_group_key(id: &CommunityId) -> GroupKey; // no epoch field + +pub fn community_id_of(owner_xonly: &[u8; 32], owner_salt: &[u8; 32]) -> CommunityId; // plain SHA-256 +pub fn epoch_key_commitment(epoch: Epoch, key: &[u8; 32]) -> [u8; 32]; // plain SHA-256 +pub fn grant_locator(id: &CommunityId, member: &[u8; 32]) -> [u8; 32]; +pub fn banlist_locator(id: &CommunityId) -> [u8; 32]; +pub fn pins_locator(id: &CommunityId, channel: &ChannelId) -> [u8; 32]; +pub fn invite_links_locator(id: &CommunityId, creator: &[u8; 32]) -> [u8; 32]; +pub fn recipient_locator(rotator: &[u8; 32], recipient: &[u8; 32], scope: &[u8; 32], epoch: Epoch) -> [u8; 32]; +pub fn invite_bundle_key(token: &[u8; 16]) -> [u8; 32]; // raw hkdf32 output; used as a NIP-44 conversation key +``` + +Rules that must be enforced by construction, not by convention: + +- Hex is lowercase everywhere; pubkeys are x-only hex, never bech32; tag numbers are decimal with no leading zeros (`"4"`, never `04`/`+4`). +- The epoch field is *omitted*, not zeroed, for labels with no epoch (`concord/dissolved`, locators, `concord/community`). +- `scalar_normalize` retries by appending a counter byte to the same `info`, starting at `0`. +- Labels and commitments are append-only. A test asserts every label is unique and that the label table matches Appendix A.6 exactly. + +**Golden vectors.** `derive.rs` carries a `#[cfg(test)]` block pinning every derivation output, seeded from the independent Python vectors published by the Vector implementation (channel/control/control-signer/guestbook at epoch 0 and at `0x0102030405060708`, dissolved, all four locators, invite key, community id, epoch commitment). One vector is missing upstream — `pins_locator` — so we mint it from our own implementation and pin it, flagged in the test as self-referential. Changing any pinned value means the wire format changed. + +## 7. Stream codec (`stream.rs`) + +```rust +pub const KIND_WRAP: u16 = 1059; +pub const KIND_WRAP_EPHEMERAL: u16 = 21059; +pub const KIND_SEAL_ENCRYPTED: u16 = 20013; +pub const KIND_SEAL_PLAINTEXT: u16 = 20014; +pub const NIP44_MAX_PLAINTEXT: usize = 65_535; + +pub fn seal_content(rumor: &UnsignedEvent, form: SealForm, group: &GroupKey) -> Result; +pub fn build_seal(rumor: &UnsignedEvent, form: SealForm, group: &GroupKey, author: &Keys) -> Result; +pub fn wrap_seal(seal: &Event, group: &GroupKey, wrap_kind: u16, at: Timestamp, extra: &[Tag]) -> Result<(Event, Keys), StreamError>; + +pub fn open_wrap(wrap: &Event, group: &GroupKey) -> Result; +pub fn open_wrap_at(wrap: &Event, address: &PublicKey, conversation: &ConversationKey, verify_wrap_sig: bool) -> Result; + +pub fn channel_binding_tags(channel: &ChannelId, epoch: Epoch) -> Vec; +pub fn check_channel_binding(rumor: &UnsignedEvent, channel: &ChannelId, epoch: Epoch) -> Result<(), StreamError>; +pub fn build_rumor(kind: u16, author: PublicKey, content: &str, tags: Vec, at_ms: u64) -> UnsignedEvent; // appends ["ms", n] +pub fn resolve_ms_strict(rumor: &UnsignedEvent) -> Result; +``` + +Design points that are easy to get wrong: + +- The wrap is signed by the **stream key** with a random ephemeral `p` tag — NIP-59 reversed. `extra` is how the caller mirrors a NIP-40 expiration onto the wrap. +- The seal is signed by the **real author** and carries `created_at` equal to the rumor's. It is never published bare. +- Control plane **must** use the plaintext seal; chat, guestbook and rekey planes **must** use the encrypted one. Each plane asserts its own form at both ends. +- The control plane is a write-restricted stream: the wrap key derives from `control_root` while the content is encrypted under the `community_root`-derived conversation key. `open_wrap_at` takes the two halves separately for this reason. +- Open order: kind → address match → wrap signature (only when `verify_wrap_sig`) → NIP-44 open → seal kind → seal signature → rumor parse → `rumor.pubkey == seal.pubkey` → recompute the rumor id and reject a mismatch → strict `ms` resolve. +- Enforce the 65 535-byte NIP-44 cap at every nesting layer before publishing. +- Do not verify the wrap signature for ordinary planes: every reader holds the group key, so the signature proves nothing. It matters only for the restricted control plane and for rekeys. +- The ephemeral wrap keypair is returned to the caller so a client may NIP-09-scrub its own wrap later. + +## 8. Planes, state and folds + +### 8.1 Editions and authority (`edition.rs`, `control.rs`) + +```rust +pub const EDITION_LABEL: &[u8] = b"vector-community/v1/edition"; // frozen, cross-client + +pub fn edition_hash(entity: &[u8; 32], version: u64, prev: Option<&[u8; 32]>, content: &[u8]) -> [u8; 32]; +pub struct ParsedEdition { author: PublicKey, vsk: String, entity: [u8; 32], version: u64, + prev: Option<[u8; 32]>, content: String, self_hash: [u8; 32] }; +pub fn parse_edition(rumor: &UnsignedEvent) -> Result; + +pub struct FoldResult { pub head: Option, pub gap: bool, pub anchored: bool } +pub fn fold(editions: &[EditionMeta], floor: u64, floor_hash: Option<&[u8; 32]>) -> FoldResult; +pub fn bootstrap_head(editions: &[EditionMeta], floor: u64) -> Option; +``` + +- Tag grammar: `["vsk", sub]`, `["eid", hex32]`, `["ev", decimal]`, `["ep", hex32]`, `["vac", eid, version, hash]`. Duplicates of any of the five reject the edition; `ev` must pass a decimal check before parsing. +- Tie-break at equal version is the lower **inner rumor id**, never `created_at`. +- `gap` is a safety signal: a tracking client (already holds the floor) fails closed for that entity and refetches; a bootstrapping client (floor 0) may accept the highest authority-verified head, which is what makes compaction re-wrapping work. +- Entity coordinates are `vsk 0` → `community_id`, `1` → `role_id`, `2` → `channel_id`, `3` → `grant_locator`, `4` → `banlist_locator`, `8` → `invite_links_locator`, `11` → `pins_locator`. All derive from `community_id` only, so a refounding re-wraps heads verbatim. + +```rust +pub const P_MANAGE_ROLES: u64 = 1 << 0; // …bit table from CORD-04 §3, frozen; retired bits are burned +pub struct CommunityRoles { roles: BTreeMap<[u8; 32], Role>, grants: BTreeMap } +impl CommunityRoles { + pub fn permissions_of(&self, member: &PublicKey) -> u64; // union of role bits + pub fn position_of(&self, member: &PublicKey, owner: &PublicKey) -> u32; + pub fn is_authorized(&self, actor: &PublicKey, owner: &PublicKey, bit: u64) -> bool; + pub fn is_authorized_in(&self, actor: &PublicKey, owner: &PublicKey, channel: &ChannelId, bit: u64) -> bool; + pub fn outranks(&self, actor: &PublicKey, owner: &PublicKey, target_position: u32) -> bool; + pub fn can_act_on(&self, actor: &PublicKey, owner: &PublicKey, target: &PublicKey, bit: u64) -> bool; + pub fn is_staff(&self, member: &PublicKey, owner: &PublicKey) -> bool; // the six control bits, CORD-04 §3 +} +``` + +Authority rules to encode once and test hard: + +- The owner is position 0, derived from `community_id`, and is never removable. +- No edition may claim a `position` at or above its own signer's, including the owner: no Role may claim 0. +- The actor must hold the required bit **and strictly outrank** the target. Equal cannot act on equal. +- A `vac` citation is a sync floor, not a verdict: block until the cited Grant version is folded, verify its hash, then judge against the *current* roster. +- A staff-making Grant carries `control_wrap`, a NIP-44 pairwise ciphertext of `epoch_be[8] ‖ control_root[32]`, and is adopted **only if it derives to the `control_pk` the member already holds** for the named epoch. +- Banlist is one replaced entity; mutations carry a re-heal step (re-fold after publish, re-apply if the addition lost the tiebreak). + +### 8.2 Communities, channels, metadata + +`CommunityMetadata` carries `name` (≤ 64 bytes), `description` (≤ 10 000 bytes), `relays` (truncated on read and write to 5), `icon` and `banner` as encrypted-blob pointers (`{url, key, nonce, hash}`), `message_expiration`, and the optional `custom` object. `ChannelMetadata` carries `name`, `private`, optional `voice`, `deleted`, optional `custom`. + +Every content struct uses `#[serde(flatten)] extra: serde_json::Map` and round-trips unknown fields. A name edit by an older client must not wipe another client's `custom` keys. Round-trip discipline gets its own test. + +Channel keying follows CORD-03 §1: a public channel derives from `community_root` at the base epoch, a private one from its own random key at its own epoch. Public→private is a rekey at the next channel epoch (monotonic, never reset); private→public derives from the base again and the prior private history stays unreadable to later joiners. + +### 8.3 Guestbook and member list (`guestbook.rs`) + +```rust +pub enum GuestbookEntry { + Join { member: PublicKey, at_ms: u64, invited_by: Option<(String, String)> }, + Leave { member: PublicKey, at_ms: u64 }, + Kick { actor: PublicKey, target: PublicKey, at_ms: u64, citation: Option }, + Snapshot { refounder: PublicKey, members: Vec, snapshot_id: [u8; 32], chunk: (u32, u32), at_ms: u64 }, +} + +pub fn coalesce(events: &[GuestbookEvent], now_ms: u64, snapshot_authority: Option<&PublicKey>, + can_kick: impl Fn(&PublicKey, &PublicKey, Option<&AuthorityCitation>) -> bool) + -> BTreeMap; + +pub fn complete_memberlist(coalesced: &BTreeMap, + observed: &BTreeMap, // author → newest ms published + banned: &BTreeSet, banned_at: &BTreeMap, + refound: Option<&Refound>) -> BTreeSet; +``` + +- Entries dated more than an hour ahead of local time are dropped. An `ms` outside `0..999` drops the entry rather than being interpreted. +- Coalesce is per npub, one final state each, by millisecond time, ties on the lower inner rumor id. +- A Kick counts only when its signer holds `KICK` and outranks the target. +- A Snapshot counts only from the npub whose Refounding minted the epoch. There is deliberately no owner fallback. +- The member list is `coalesced Joined ∪ observed authors − banlist`, and observation counts **forward only** (an author re-enters on activity newer than their latest Leave/Kick/Ban). A Grant holder with `ms == 0` is present by construction. + +### 8.4 Chat plane (`chat.rs`) + +```rust +pub struct ChatMessage { + pub id: EventId, // recomputed rumor id + pub author: PublicKey, + pub channel: ChannelId, + pub epoch: Epoch, + pub kind: Kind, // 9 | 1111 | 3302 | 1740 | 15 + pub content: String, + pub media: Vec, + pub mentions: Vec, + pub reply_to: Option, // lowercase `e`/`q` + pub thread_root: Option, // uppercase `E` for 1111 + pub at_ms: u64, + pub expiration: Option, + pub edited_at: Option, // folded from 3302 + pub deleted: bool, // folded from 5 + pub reactions: BTreeMap, +} +``` + +Sends funnel through one function so the rules cannot drift: + +```rust +fn publish_chat(store, client, community, channel, epoch, group, rumor, at_ms, ephemeral) -> Task, Error>>; +``` + +It builds the seal + wrap, mirrors the NIP-40 tag onto the wrap for durable kinds, publishes via `send_event(..).to(relays)`, retains the ephemeral wrap key for later NIP-09 scrubbing, and locally echoes its own wrap through the same ingest path so send-then-read works without waiting on a relay round-trip. + +Disappearing messages (CORD-08) live here: `message_expiration` is read from the folded metadata, `["expiration", created_at + t]` is attached to every durable Chat rumor and to the wrap, kinds 5 and 1740 are exempt, ingest refuses an already-expired rumor, a periodic sweep purges stored ones, and the kind 1740 timer notice renders only when its author holds `MANAGE_METADATA`. + +### 8.5 Invites (`invite.rs`) + +```rust +pub struct CommunityInvite { community_id, owner, owner_salt, community_root, root_epoch, + control_pk: Option, channels: Vec, + relays: Vec, name: String, icon: Option, + expires_at: Option, creator_npub: Option, label: Option, + extra: Map } +impl CommunityInvite { pub fn validate(&self) -> Result<(), InviteError>; // recompute community_id + pub fn expired(&self, now_ms: u64) -> bool; } + +pub fn build_bundle(token: &[u8; 16], link_signer: &Keys, invite: &CommunityInvite) -> Result; // 33301, d = "" +pub fn build_revocation(link_signer: &Keys) -> Result; // vsk 9 +pub fn parse_link(input: &str) -> Result; +pub fn encode_fragment(relays: &[RelayUrl], token: &[u8; 16]) -> String; // version byte 4, flags, ≤ 3 relays, base64url +pub fn decode_fragment(fragment: &str) -> Result<(Vec, [u8; 16]), InviteError>; +pub fn build_direct_invite(receiver: &PublicKey, invite: &CommunityInvite, signer: &UniversalSigner) -> Task>; // 3313 rumor → 13 seal → k-tagged 1059 +``` + +The link rides `naddr` (`Nip19Coordinate` for kind 33301, link signer, empty `d`) in the path and the token + bootstrap relays in the fragment. A fragment is never sent to a server. The bundle is decrypted with `invite_bundle_key(token)`, and the joiner must recompute `community_id` from `owner` + `owner_salt`. + +Bounds before allocation: reject a bundle with more than 256 channels, truncate the relay list to 5, refuse an expired one. + +### 8.6 Rekeys and refoundings (`rekey.rs`) + +```rust +pub enum RekeyScope { Channel(ChannelId), Base } +pub fn encode_blob_plaintext(scope, epoch, new_root, control_pk, control_root) -> Vec; // 72 | 104 | 136 bytes +pub fn parse_blob_plaintext(bytes: &[u8], scope, epoch) -> Result; +pub fn build_rekey_rumor(rotator, scope, new_epoch, prev_epoch, prev_commit, blobs, chunk) -> UnsignedEvent; +pub fn plan_refounding(fold, removed: &[PublicKey]) -> Result; +pub fn compact(fold, epoch, new_control_root, ...) -> Vec; // re-wrap heads verbatim, plaintext seals preserved +``` + +- The subscription for rekeys is precomputed from the *next* epoch's address, per private channel and once for the base. +- A receiver accepts a key only after: locating its blob, decrypting with the rotator↔recipient conversation key, checking the bound `scope` and `epoch` inside the plaintext, and matching `prevcommit` against the key it currently holds. +- Only after holding **all** `n` chunks of one `(rotator, newepoch, prevcommit)` set, with none containing its locator, may a client conclude it was removed. +- Send cap 80 blobs per event, accept cap 120 (Vector's documented erratum: the CORD-01 double envelope pushes 120 blobs past a 64 KB relay limit). Record the reason in a comment so nobody "fixes" it back. +- Compacted control heads are re-wrapped with their original signature intact, which is exactly why the control plane uses the plaintext seal. +- Two concurrent refoundings converge on the lexicographically lowest new base key; the heal is down-only. +- Authority: a channel rekey needs `MANAGE_CHANNELS`, a refounding needs `BAN`, and in both the rotator must strictly outrank every removed target. Holding a key is never authority. + +Dissolution (CORD-02 §9) also lives here: a chainless, owner-signed `vsk 10` tombstone at `dissolved_group_key(id)`, plaintext-sealed, and a verifier **must** refuse any tombstone whose `eid` is not the community's own id (including the all-zero placeholder — accepting it lets an owner's genuine tombstone for one community be re-wrapped at another of theirs and kill it permanently). On sight the community is sealed read-only: subscriptions halt, nothing new is honored, existing history stays readable, and a member's delete of their own message is still honored. + +## 9. Storage (`store.rs`) + +Three layers, no new storage engine: + +1. **Raw wraps** (kind 1059) are persisted automatically by the SDK's relay pool when a subscription or fetch matches a filter. Nothing to write. +2. **Opened rumors** are cached locally as NIP-78 `Kind::ApplicationSpecificData` events signed by a session-local keypair, exactly like `chat::set_rumor`. Tags: `["d", rumor_id]` (replace key), `["c", channel_hex]`, `["p", author]`, `["k", kind]`, `["e", wrap_id]`, `["t", "concord"]`. Contents are the rumor JSON. + - The `c`/`t` keys deliberately differ from chat's `r` key so the two message namespaces can never collide in one database. + - The read path dedupes by rumor id and keeps the newest `created_at`, because the local signing key changes per session. +3. **Community state** — one local document per community, `Kind::ApplicationSpecificData` with `["d", "concord/"]`: + +```rust +pub struct CommunityState { + pub id: CommunityId, + pub owner: PublicKey, + pub owner_salt: [u8; 32], + pub community_root: [u8; 32], + pub root_epoch: Epoch, + pub control_root: Option<[u8; 32]>, // present iff the holder is staff + pub control_pks: BTreeMap, + pub channels: Vec, // id, key, epoch, name, private + pub epoch_keys: Vec<([u8; 32], Epoch, [u8; 32])>, // (scope, epoch, key) — the history backfill index + pub relays: Vec, + pub heads: BTreeMap<[u8; 32], (u64, [u8; 32], EventId)>, // entity → (version, self_hash, inner id) + pub guestbook: Vec, + pub observed: BTreeMap, + pub banned: BTreeSet, + pub dissolved: bool, + pub added_at_ms: u64, +} +``` + +Writes are debounced (a fold head changes on every edition); reads load once at init. + +**Decision, stated for the record:** this document stores raw community keys unencrypted in a plaintext local database. That matches the existing posture — `chat` already caches decrypted message rumors in the same LMDB. If that posture ever changes, the state document is the one to wrap with NIP-44-to-self, since it is the only local artifact holding keys. + +History queries: + +```rust +pub async fn query_messages(&self, channel: &ChannelId, until: Option, limit: usize) -> Result, Error>; +pub async fn backfill(&self, plane_authors: &[PublicKey], relays: &[RelayUrl], until: Option, limit: usize) -> Result, Error>; +``` + +`query_messages` reads the local cache (`Filter::new().kind(ApplicationSpecificData).custom_tag(LOWERCASE_C, channel_hex)`); `backfill` pages relays newest-first with `until`, deduplicating by wrap id and stepping past same-second walls. + +## 10. Sync engine and GPUI conventions + +`ConcordRegistry` mirrors `ChatRegistry`'s shape exactly: a foreground GPUI entity holding `Entity` handles, a `flume` signal bus, one background notification listener, one foreground consumer, and task slots that are cleared when the signer changes. + +**Subscription.** Community relays come from the folded metadata. `init`/`join` add them to the client (`client.add_relay(url).and_connect()`), then: + +```rust +let filter = Filter::new() + .kinds([Kind::GiftWrap, Kind::Custom(KIND_WRAP_EPHEMERAL)]) + .authors(plane_authors) // guestbook, control signer, all held channel planes, + // rekey addresses for epoch + 1, dissolved address + .since(Timestamp::from_secs(now - FRESH_WINDOW)) + .limit(0); // live tail only; history comes from backfill +client.subscribe(target).with_id(SubscriptionId::new(CONCORD_SUB)).await?; +``` + +Targeted subscribe against the community relays, with a pool-wide subscribe as the fallback path. Rebuild idempotently whenever a plane's address changes (join, channel added, rekey folded). + +**Routing.** `dispatch` matches on the `subscription_id` carried in `RelayMessage::Event`, dedupes by wrap id (both subscriptions and several relays deliver the same wrap), then recognises the plane by **wrap author** against the derived addresses it holds — never by trial decryption. Recognition order: held channel planes → guestbook → control signer → rekey addresses → dissolved. + +**Ingest pipeline.** Unwrap, verify and fold all happen inside `cx.background_spawn`, never on the foreground thread: secp256k1 verification per edition and per seal is far too expensive for the UI thread. + +```rust +enum Signal { + Chat { community: CommunityId, channel: ChannelId, message: Box }, + Control { community: CommunityId, heads: Vec, roster: Box }, + Guestbook { community: CommunityId, members: BTreeSet }, + Rekey { community: CommunityId, scope: RekeyScope, epoch: Epoch }, + Dissolved(CommunityId), + Eose(SubscriptionId), + Error(ConcordError), +} +``` + +The consumer is `cx.spawn(async move |this, cx| { while let Ok(signal) = rx.recv_async().await { this.update(cx, |this, cx| this.apply(signal, cx))?; } })`, which updates entities and calls `cx.notify()`. + +Rules taken from the project guidelines: + +- Crypto, folding, database queries and network I/O only in `cx.background_spawn`. +- Foreground tasks are `cx.spawn` with `this.update(cx, ..)`; any entity update happens there, and the inner `cx` is always used. +- Tasks are stored in fields (`tasks`, `listener`, `consumer`) so they are cancelled on signer change and dropped with the registry. `detach()` only for genuinely fire-and-forget work such as the local state save. +- Long-running paging is bounded by explicit page and step caps, not by unbounded loops. +- Every fallible path returns `Result` and surfaces through `ConcordEvent::Error`; nothing is silently swallowed. + +**Registry API.** + +```rust +pub fn init(window: &mut Window, cx: &mut App); +pub struct ConcordRegistry { /* … */ } +impl ConcordRegistry { + pub fn global(cx: &App) -> Entity; + pub fn loading(&self) -> bool; + pub fn communities(&self) -> Vec>; + pub fn community(&self, id: &CommunityId, cx: &App) -> Option>; + pub fn find(&self, query: &str, cx: &App) -> Vec>; + + pub fn create(&mut self, params: CommunityParams, cx: &mut Context) -> Task>; + pub fn join(&mut self, link: &str, cx: &mut Context) -> Task>; + pub fn accept_direct_invite(&mut self, rumor: &UnsignedEvent, cx: &mut Context) -> Task>; + pub fn leave(&mut self, id: &CommunityId, cx: &mut Context); + pub fn discard_invite(&mut self, id: &CommunityId, cx: &mut Context); + pub fn refresh(&mut self, id: &CommunityId, cx: &mut Context); + pub fn shutdown(&mut self, cx: &mut Context); // halt subscriptions, keep our own state +} +``` + +**Community API** (`Entity`, `EventEmitter`): + +```rust +pub fn id(&self) -> CommunityId; +pub fn owner(&self) -> PublicKey; +pub fn name(&self) -> SharedString; pub fn description(&self) -> Option; +pub fn icon(&self) -> Option; +pub fn relays(&self) -> Vec; +pub fn epoch(&self) -> Epoch; +pub fn dissolved(&self) -> bool; +pub fn channels(&self) -> Vec>; +pub fn channel(&self, id: &ChannelId, cx: &App) -> Option>; +pub fn members(&self) -> BTreeSet; +pub fn banned(&self) -> BTreeSet; +pub fn roles(&self) -> &CommunityRoles; +pub fn permissions(&self, member: &PublicKey) -> u64; +pub fn is_staff(&self, member: &PublicKey) -> bool; +pub fn message_expiration(&self) -> Option; + +// authority actions — each returns a publish task and nothing optimistic +pub fn set_metadata(&mut self, meta: CommunityMetadata, cx: &mut Context) -> Task>; +pub fn create_channel(&mut self, name: &str, private: bool, cx: &mut Context) -> Task>; +pub fn edit_channel(&mut self, id: &ChannelId, meta: ChannelMetadata, cx: &mut Context) -> Task>; +pub fn create_role(&mut self, role: Role, cx: &mut Context) -> Task>; +pub fn assign_roles(&mut self, member: &PublicKey, roles: &[[u8; 32]], cx: &mut Context) -> Task>; +pub fn ban(&mut self, members: &[PublicKey], cx: &mut Context) -> Task>; +pub fn unban(&mut self, members: &[PublicKey], cx: &mut Context) -> Task>; +pub fn kick(&mut self, member: &PublicKey, cx: &mut Context) -> Task>; +pub fn rekey_channel(&mut self, id: &ChannelId, removed: &[PublicKey], cx: &mut Context) -> Task>; +pub fn refound(&mut self, removed: &[PublicKey], cx: &mut Context) -> Task>; +pub fn dissolve(&mut self, cx: &mut Context) -> Task>; +pub fn create_invite(&mut self, params: InviteParams, cx: &mut Context) -> Task>; +pub fn revoke_invite(&mut self, token: &[u8; 16], cx: &mut Context) -> Task>; +pub fn direct_invite(&mut self, receiver: &PublicKey, cx: &mut Context) -> Task>; +pub fn save_community_list(&mut self, cx: &mut Context) -> Task>; // kind 13302, multi-device sync +``` + +**Channel API** (`Entity`): `id`, `name`, `private`, `epoch`, `deleted`, plus + +```rust +pub fn messages(&self, until: Option, limit: usize, cx: &App) -> Task, Error>>; +pub fn send(&self, content: &str, reply_to: Option, cx: &App) -> Task, Error>>; +pub fn send_file(&self, file: FileAttachment, reply_to: Option, cx: &App) -> Task, Error>>; +pub fn edit(&self, id: EventId, content: &str, cx: &App) -> Task, Error>>; +pub fn delete(&self, id: EventId, cx: &App) -> Task, Error>>; +pub fn react(&self, id: EventId, emoji: &str, cx: &App) -> Task, Error>>; +pub fn typing(&self, cx: &App) -> Task>; // kind 23311, ephemeral +pub fn pin(&self, id: EventId, cx: &App) -> Task>; // vsk 11, PIN_MESSAGES +``` + +`CommunityEvent` and `ChannelEvent` mirror `ChatEvent`: one variant per thing the UI has to react to (`Updated`, `Members`, `Added`, `Removed`, `Dissolved`, `Error`, plus channel-level `Incoming`, `Reload`). + +## 11. Integration with existing crates + +1. **`crates/chat/src/lib.rs` — required fix.** `handle_notifications` currently treats *every* kind 1059 event as a NIP-17 wrap for the current user and pushes an unwrap failure into the trash. Concord wraps are kind 1059 with an ephemeral `p` tag, so they would flood the trash and leak error toasts. Route by `subscription_id` from `RelayMessage::Event` against `sub_id1`/`sub_id2`, and drop the `if rumor.tags.is_empty()` heuristic once the real recipient check is in place. +2. **`desktop/src/main.rs` and `web/src/lib.rs`** — add `concord::init(window, cx)` after `chat::init(window, cx)`. +3. **`Cargo.toml`** — add `hkdf = "0.12"` to `[workspace.dependencies]`; add the crate to `desktop` and `web` dependencies. No other workspace changes. +4. **No changes** to `state`, `person`, `device`, `settings`, `common`, or `ui`. + +## 12. Security invariants to test, not to assume + +Each of these has burned a real implementation, or is a documented cross-client trap: + +- Recompute every rumor id and reject a claimed mismatch; never trust an embedded `id`. +- Require `rumor.pubkey == seal.pubkey`. +- Require the plaintext seal form on Control and the encrypted form on Chat/Guestbook/Rekey — a strict reader must drop a mis-sealed edition rather than fold a chain a later compaction would fork. +- Check `channel` **and** `epoch` against the plane whose key opened the wrap; reject duplicates of either tag. +- Reject duplicate `vsk`/`eid`/`ev`/`ep`/`vac` tags; require decimal-with-no-leading-zeros on every numeric tag. +- Refuse a tombstone whose `eid` is not this community's id. +- Adopt a `control_root` from a Grant only if it derives to the `control_pk` held for that epoch; adopt a rekey blob only if its bound plaintext matches the scope and epoch and its `prevcommit` matches the key currently held. +- Never conclude removal from a partial rekey chunk set. +- Drop guestbook entries more than an hour in the future; treat an out-of-range `ms` as malformed, not as an interpretation opportunity. +- Never honour a Snapshot from anyone but the refounder of that epoch. +- Refuse to write a Pin List from a list the writer could not read. +- Enforce the NIP-44 65 535-byte cap at every layer before publishing, and the 5-relay / 256-channel / 50-membership / 100-roles / 64-roles-per-member / 500-banlist / 25-pin caps at their ingest and write points. +- Lowercase hex only; x-only pubkeys only; no version tag anywhere. + +## 13. Milestones + +| # | Deliverable | Done when | +| --- | --- | --- | +| M0 | Crate skeleton, `derive.rs`, golden vectors, workspace wiring | `cargo test -p concord` pins every derivation from an independent vector set; all labels match Appendix A.6 | +| M1 | `stream.rs` + `store.rs` | seal/wrap/open round-trips for both seal forms; malformed inputs rejected in the documented order; local cache reads back after a restart | +| M2 | `edition.rs` + `control.rs` genesis | a community is created and published; its two genesis wraps open at a second client sharing the keys; edition hash matches the cross-client vector | +| M3 | Control fold + roster + metadata/channels | fold tests for chains, gaps, downgrade refusal, fork tiebreak, compaction dangle; metadata and channel edits visible to a second client | +| M4 | Chat plane | send/receive/edit/delete/reaction across two identities; history pages backwards across an epoch boundary; binding checks reject a foreign channel/epoch | +| M5 | Guestbook + member list + moderation | join/leave/kick/ban converge to the same memberlist on both clients; every authority rule from §8.1 has a negative test | +| M6 | Invites + Community List | link mint → fetch → join round-trips; revoked link refuses; direct invite lands in the recipient's giftwrap inbox via the `k` tag; a second device reconstructs membership from 13302 | +| M7 | Rekeys + refounding + dissolution | a removed member stops reading after a rekey; continuity and race rules tested; a tombstone seals the community and a foreign-id tombstone is refused | +| M8 | Pins + disappearing messages + hardening | pins verify from a keyless reader's view; expiry is refused at ingest and purged by the sweep; the audit of §12 is complete with a test per bullet | + +Ordering is deliberately dependency-first: each milestone is usable on its own, and nothing in M2+ depends on a later milestone. + +## 14. Open questions and risks + +1. **Community List kind.** CORD-02 §8 specifies `13302`, replaceable. Vector has retired it in favour of fragmented `33302`, because a replaceable kind holds one event per pubkey and so cannot shard past the NIP-44 size cap. We implement `13302` per spec, enforce the 50-membership cap and pre-publish size check, and treat `33302` as an interop follow-up. Confirm with Armada before writing the multi-device code. +2. **NIP-42 for stream-authored REQs.** Relays that gate kind 1059 by author (for example `ditto-relay`'s `AUTH_KINDS`) need an AUTH event signed by that plane's derived key. `nostr-sdk`'s `Authenticator` is per-client and returns one identity, while a Concord client holds many plane keys — so this cannot be solved by swapping the authenticator. Options: contribute a per-REQ auth hook upstream, or accept that such relays are unsupported and prefer relays without the gate. Decide before M7; the default is "documented limitation" plus a relay-capability check. +3. **`invite_bundle_key`.** Appendix A.6 says the label "yields the public-invite decrypt key" without stating whether that is the raw HKDF output used as a NIP-44 conversation key or the `conv_key` of a normalized keypair. The reference implementation uses the raw output. Pin a vector and verify against Armada early — this one decides whether links open at all. +4. **Missing golden vector for `pins_locator`.** Upstream publishes none. Ours will be self-referential; flag it in the test. +5. **Relay set.** Up to 5 recommended, and both reads and writes fan out across them. Coop's client is a gossip client with `no_background_refresh`, so community relays must be added explicitly and re-added on metadata change. +6. **Local plaintext state.** §9 records the decision. Revisit only if the local database stops being treated as trusted. +7. **Was a `community_id` ever hashed into a tag?** No — it must never appear on the wire. A lint-style test should assert it appears in no published event other than inside an invite bundle and a direct invite. + +## 15. Test strategy + +- **Unit, pure:** derivations against golden vectors, edition hash, fold, coalesce, memberlist, blob codec, caps and rejection paths. These need no GPUI context and should be exhaustive — they are where cross-client divergence is caught. +- **Integration, GPUI:** `TestAppContext` with two registries sharing an in-memory database, driving wraps through the ingest path; timeouts and delays use `cx.background_executor().timer(..)` per the project guidelines, never `smol::Timer`. +- **Round-trip:** every builder paired with its parser, asserting the parse produces the identical structure, including unknown-field round-tripping on all content types. +- **Negative:** every bullet in §12 gets a test that constructs the hostile input and asserts the drop.