From 319b1038d2dbd476082a64acbc078b8a3589e99e Mon Sep 17 00:00:00 2001 From: Ren Amamiya Date: Wed, 16 Sep 2026 11:03:33 +0700 Subject: [PATCH 01/12] add plan --- PLAN.md | 560 ++++++++++++++++++++++++++++++++++++++++++++++++++++++++ 1 file changed, 560 insertions(+) create mode 100644 PLAN.md diff --git a/PLAN.md b/PLAN.md new file mode 100644 index 00000000..048a6a01 --- /dev/null +++ b/PLAN.md @@ -0,0 +1,560 @@ +# Concord support — backend & API plan + +Concord is an encrypted community/channel protocol over Nostr: shared-key "Private Streams" (CORD-01), communities with a self-certifying owner id and three planes (CORD-02), public/private channels (CORD-03), an owner-rooted signed roster (CORD-04), invites (CORD-05), rekeys/refoundings (CORD-06) and disappearing messages (CORD-08). + +Scope of this plan: **backend + public Rust API**. No views, no widgets, no copy. + +## 1. Scope + +**In scope** + +- New `crates/concord` crate: derivations, stream codec, control/chat/guestbook planes, authority fold, ephemeral state, invites, rekeys, dissolution, storage, sync engine. +- Public API for a future UI: `ConcordRegistry` + `Entity` / `Entity` + events, mirroring the shape of `crates/chat`. +- The minimum surgical edits to existing crates required for coexistence (see §11). + +**Out of scope** + +- Any UI work. +- CORD-07 audio/video. Reserve `23313`, the `concord/voice-*` labels and the `voice` metadata flag so nothing else claims them, and implement nothing. +- Pins (CORD-04 §7) ship in the last milestone; the design accounts for `vsk 11` early so the fold is not retrofitted. +- Cross-client interop testing (Vector/Armada/Grimoire). Tracked as follow-up work, not blocking. + +## 2. Sources of truth + +| Doc | What we take from it | +| --- | --- | +| CORD-01 | Stream event shape, seal forms 20013/20014, encoding rules, binding, deletions | +| CORD-02 | `community_id`, `community_root`, `control_root`, epochs, 3 planes, metadata, invites, Community List, dissolution | +| CORD-03 | Channel keying, metadata, message kinds, `channel`/`epoch` binding, threads vs quotes | +| CORD-04 | Editions, `vac`, the roster, permission bits, banlist, the three removals, pins | +| CORD-05 | Bundle, link (naddr + fragment), relay dictionary, Invite List, Registry, Direct Invite | +| CORD-06 | Rekey blobs, chunking, `prevcommit` continuity, Refounding, compaction, races | +| CORD-08 | `message_expiration`, NIP-40 tagging, ingest/purge enforcement, timer notice 1740 | + +Appendix A (derivations) and Appendix B (kinds) of CORD-02 are **frozen**: every labeled byte and every kind number is part of the wire format. Treat both as constants with golden-vector tests. + +Reference implementations for cross-checking behaviour (not for copying code): Vector (`crates/vector-core/src/community/v2/*`), Armada, Grimoire. + +## 3. Reuse map — nostr-sdk APIs we build on + +Verified against the pinned revision (`nostr` 0.45.4 / `nostr-sdk` 0.45.2, git `b230cec`). + +| Concord need | Existing API | +| --- | --- | +| NIP-44 under a raw conversation key | `nostr::nips::nip44::v2::{ConversationKey, encrypt_to_bytes_with_nonce, decrypt_to_bytes}` | +| NIP-44 conversation key for a keypair | `ConversationKey::derive(&SecretKey, &PublicKey)` (self-ECDH for streams) | +| NIP-44 under a signer | `nip44::{encrypt, decrypt}` (already wrapped by `state::UniversalSigner`) | +| Event id recomputation | `EventId::compute(pubkey, created_at, kind, tags, content)`, `UnsignedEvent::compute_id` | +| Event (de)serialization | `Event::{from_json, as_json, verify}`, `UnsignedEvent::from_json` | +| Building events | `EventBuilder::new(..).tags(..).custom_created_at(..)`, `FinalizeEvent::finalize(&Keys)` | +| Tags | `Tag::{custom, identifier, public_key, expiration}`, `Tags`, `SingleLetterTag` | +| Kinds | `Kind::GiftWrap` (1059), `Kind::Custom(21059|20013|20014|3308|…)`, `Kind::is_ephemeral` | +| Publish | `Client::send_event(&event).to(relays).ack_policy(AckPolicy::none())` | +| Subscribe | `Client::subscribe(target).with_id(..).close_on(..)`, `SubscribeAutoCloseOptions`, `ReqExitPolicy` | +| Backfill | `Client::fetch_events(target)`, `Client::stream_events(target)` | +| Local persistence | `Client::database()` → `query(Filter)`, `save_event(&Event)` | +| Invite link parsing | `Nip19::from_bech32` → `Nip19::Coordinate(Nip19Coordinate)` | +| Signer abstraction | `state::UniversalSigner` (`AsyncSignEvent` + `AsyncNip44`) | +| Relay auth | `nostr_sdk::{Authenticator, SignerAuthenticator}` | + +**Not needed.** `secp256k1` (use `nostr::SecretKey::from_slice` + `Keys::new`), `base64` (use `data_encoding::BASE64`, already a workspace dep), `bech32` (NIP-19 is in the SDK), any new storage engine (the client's LMDB database is enough), any new HTTP client. + +**Add one dependency:** `hkdf = "0.12"` (already in `Cargo.lock` transitively). Add it to `[workspace.dependencies]` and to the new crate. `sha2` is already a workspace dep. + +## 4. Crate layout + +New crate `crates/concord`, picked up automatically by the `crates/*` workspace member glob. + +``` +crates/concord/ + Cargo.toml + src/lib.rs init, ConcordRegistry, ConcordEvent, signal bus, subscriptions, ingest pipeline + src/derive.rs frozen HKDF / group_key / locators / commitments + golden vectors + src/stream.rs CORD-01: seal + wrap + open (SealForm, OpenedStream), channel/epoch binding + src/edition.rs CORD-04 §1: canonical signing bytes, edition hash, parse, fold + src/control.rs control plane view, genesis, content types, roster fold, authority checks + src/guestbook.rs CORD-02 §5: join/leave/kick/snapshot, coalesce, complete memberlist + src/chat.rs CORD-03: channel plane — message/edit/delete/reaction builders + message view + src/invite.rs CORD-05: bundle, link, registry, Invite List, Direct Invite + src/rekey.rs CORD-06: blob codec, continuity, refounding, compaction, dissolution + src/store.rs local persistence + opened-rumor cache + history queries +``` + +`Community` and `Channel` GPUI entities live in `src/lib.rs` next to the registry — they are the public surface, not a separate concern. Ten modules, each with real content; no single-fn files. + +Dependencies: `common`, `state`, `person`, `device`, `settings`, `gpui`, `nostr` (for `nip44` features), `nostr-sdk`, `hkdf`, `sha2`, `serde`, `serde_json`, `anyhow`, `flume`, `smallvec`, `itertools`, `futures`, `log`, `instant`. Everything under `cfg(not(target_arch = "wasm32"))` follows the `chat`/`state` split so the crate still builds for web. + +## 5. Core types + +```rust +pub struct CommunityId([u8; 32]); // sha256 commitment, never on the wire +pub struct ChannelId([u8; 32]); +pub struct Epoch(pub u64); + +/// A derived stream: signing keypair + the self-ECDH conversation key that +/// encrypts the wraps. Memoised in a bounded process-wide cache. +pub struct GroupKey { keys: Keys, conversation: ConversationKey } +impl GroupKey { + pub fn pk(&self) -> PublicKey; + pub fn keys(&self) -> &Keys; + pub fn conversation(&self) -> &ConversationKey; +} + +pub enum SealForm { Encrypted, Plaintext } + +pub struct OpenedStream { + pub rumor_id: EventId, + pub author: PublicKey, // the seal's verified pubkey + pub seal_form: SealForm, + pub seal: Event, // retained: compaction re-wraps plaintext seals verbatim + pub wrapper_id: EventId, + pub at_ms: u64, // created_at * 1000 + ms tag + pub rumor: UnsignedEvent, +} +``` + +Ordering everywhere uses `at_ms`, never `created_at`, and ties break on the lower inner rumor id. + +## 6. Frozen derivations (`derive.rs`) + +```rust +fn build_info(label: &str, id: &[u8; 32], epoch: Option) -> Vec; // label ‖ 0x00 ‖ id[32] ‖ epoch_be[8]? +fn hkdf32(ikm: &[u8], info: &[u8]) -> [u8; 32]; // HKDF-SHA256, zero-length salt, L = 32 +fn hkdf_to_secret_key(ikm: &[u8], info: &[u8]) -> SecretKey; // A.3 scalar_normalize, counter from 0 + +fn group_key(label: &str, secret: &[u8], id: &[u8; 32], epoch: Option) -> GroupKey; + +pub fn channel_group_key(secret: &[u8; 32], channel: &ChannelId, epoch: Epoch) -> GroupKey; +pub fn control_group_key(root: &[u8; 32], id: &CommunityId, epoch: Epoch) -> GroupKey; // read key +pub fn control_signer_group_key(root: &[u8; 32], id: &CommunityId, epoch: Epoch) -> GroupKey; // write key +pub fn guestbook_group_key(root: &[u8; 32], id: &CommunityId, epoch: Epoch) -> GroupKey; +pub fn channel_rekey_group_key(root: &[u8; 32], channel: &ChannelId, epoch: Epoch) -> GroupKey; +pub fn base_rekey_group_key(root: &[u8; 32], id: &CommunityId, epoch: Epoch) -> GroupKey; +pub fn dissolved_group_key(id: &CommunityId) -> GroupKey; // no epoch field + +pub fn community_id_of(owner_xonly: &[u8; 32], owner_salt: &[u8; 32]) -> CommunityId; // plain SHA-256 +pub fn epoch_key_commitment(epoch: Epoch, key: &[u8; 32]) -> [u8; 32]; // plain SHA-256 +pub fn grant_locator(id: &CommunityId, member: &[u8; 32]) -> [u8; 32]; +pub fn banlist_locator(id: &CommunityId) -> [u8; 32]; +pub fn pins_locator(id: &CommunityId, channel: &ChannelId) -> [u8; 32]; +pub fn invite_links_locator(id: &CommunityId, creator: &[u8; 32]) -> [u8; 32]; +pub fn recipient_locator(rotator: &[u8; 32], recipient: &[u8; 32], scope: &[u8; 32], epoch: Epoch) -> [u8; 32]; +pub fn invite_bundle_key(token: &[u8; 16]) -> [u8; 32]; // raw hkdf32 output; used as a NIP-44 conversation key +``` + +Rules that must be enforced by construction, not by convention: + +- Hex is lowercase everywhere; pubkeys are x-only hex, never bech32; tag numbers are decimal with no leading zeros (`"4"`, never `04`/`+4`). +- The epoch field is *omitted*, not zeroed, for labels with no epoch (`concord/dissolved`, locators, `concord/community`). +- `scalar_normalize` retries by appending a counter byte to the same `info`, starting at `0`. +- Labels and commitments are append-only. A test asserts every label is unique and that the label table matches Appendix A.6 exactly. + +**Golden vectors.** `derive.rs` carries a `#[cfg(test)]` block pinning every derivation output, seeded from the independent Python vectors published by the Vector implementation (channel/control/control-signer/guestbook at epoch 0 and at `0x0102030405060708`, dissolved, all four locators, invite key, community id, epoch commitment). One vector is missing upstream — `pins_locator` — so we mint it from our own implementation and pin it, flagged in the test as self-referential. Changing any pinned value means the wire format changed. + +## 7. Stream codec (`stream.rs`) + +```rust +pub const KIND_WRAP: u16 = 1059; +pub const KIND_WRAP_EPHEMERAL: u16 = 21059; +pub const KIND_SEAL_ENCRYPTED: u16 = 20013; +pub const KIND_SEAL_PLAINTEXT: u16 = 20014; +pub const NIP44_MAX_PLAINTEXT: usize = 65_535; + +pub fn seal_content(rumor: &UnsignedEvent, form: SealForm, group: &GroupKey) -> Result; +pub fn build_seal(rumor: &UnsignedEvent, form: SealForm, group: &GroupKey, author: &Keys) -> Result; +pub fn wrap_seal(seal: &Event, group: &GroupKey, wrap_kind: u16, at: Timestamp, extra: &[Tag]) -> Result<(Event, Keys), StreamError>; + +pub fn open_wrap(wrap: &Event, group: &GroupKey) -> Result; +pub fn open_wrap_at(wrap: &Event, address: &PublicKey, conversation: &ConversationKey, verify_wrap_sig: bool) -> Result; + +pub fn channel_binding_tags(channel: &ChannelId, epoch: Epoch) -> Vec; +pub fn check_channel_binding(rumor: &UnsignedEvent, channel: &ChannelId, epoch: Epoch) -> Result<(), StreamError>; +pub fn build_rumor(kind: u16, author: PublicKey, content: &str, tags: Vec, at_ms: u64) -> UnsignedEvent; // appends ["ms", n] +pub fn resolve_ms_strict(rumor: &UnsignedEvent) -> Result; +``` + +Design points that are easy to get wrong: + +- The wrap is signed by the **stream key** with a random ephemeral `p` tag — NIP-59 reversed. `extra` is how the caller mirrors a NIP-40 expiration onto the wrap. +- The seal is signed by the **real author** and carries `created_at` equal to the rumor's. It is never published bare. +- Control plane **must** use the plaintext seal; chat, guestbook and rekey planes **must** use the encrypted one. Each plane asserts its own form at both ends. +- The control plane is a write-restricted stream: the wrap key derives from `control_root` while the content is encrypted under the `community_root`-derived conversation key. `open_wrap_at` takes the two halves separately for this reason. +- Open order: kind → address match → wrap signature (only when `verify_wrap_sig`) → NIP-44 open → seal kind → seal signature → rumor parse → `rumor.pubkey == seal.pubkey` → recompute the rumor id and reject a mismatch → strict `ms` resolve. +- Enforce the 65 535-byte NIP-44 cap at every nesting layer before publishing. +- Do not verify the wrap signature for ordinary planes: every reader holds the group key, so the signature proves nothing. It matters only for the restricted control plane and for rekeys. +- The ephemeral wrap keypair is returned to the caller so a client may NIP-09-scrub its own wrap later. + +## 8. Planes, state and folds + +### 8.1 Editions and authority (`edition.rs`, `control.rs`) + +```rust +pub const EDITION_LABEL: &[u8] = b"vector-community/v1/edition"; // frozen, cross-client + +pub fn edition_hash(entity: &[u8; 32], version: u64, prev: Option<&[u8; 32]>, content: &[u8]) -> [u8; 32]; +pub struct ParsedEdition { author: PublicKey, vsk: String, entity: [u8; 32], version: u64, + prev: Option<[u8; 32]>, content: String, self_hash: [u8; 32] }; +pub fn parse_edition(rumor: &UnsignedEvent) -> Result; + +pub struct FoldResult { pub head: Option, pub gap: bool, pub anchored: bool } +pub fn fold(editions: &[EditionMeta], floor: u64, floor_hash: Option<&[u8; 32]>) -> FoldResult; +pub fn bootstrap_head(editions: &[EditionMeta], floor: u64) -> Option; +``` + +- Tag grammar: `["vsk", sub]`, `["eid", hex32]`, `["ev", decimal]`, `["ep", hex32]`, `["vac", eid, version, hash]`. Duplicates of any of the five reject the edition; `ev` must pass a decimal check before parsing. +- Tie-break at equal version is the lower **inner rumor id**, never `created_at`. +- `gap` is a safety signal: a tracking client (already holds the floor) fails closed for that entity and refetches; a bootstrapping client (floor 0) may accept the highest authority-verified head, which is what makes compaction re-wrapping work. +- Entity coordinates are `vsk 0` → `community_id`, `1` → `role_id`, `2` → `channel_id`, `3` → `grant_locator`, `4` → `banlist_locator`, `8` → `invite_links_locator`, `11` → `pins_locator`. All derive from `community_id` only, so a refounding re-wraps heads verbatim. + +```rust +pub const P_MANAGE_ROLES: u64 = 1 << 0; // …bit table from CORD-04 §3, frozen; retired bits are burned +pub struct CommunityRoles { roles: BTreeMap<[u8; 32], Role>, grants: BTreeMap } +impl CommunityRoles { + pub fn permissions_of(&self, member: &PublicKey) -> u64; // union of role bits + pub fn position_of(&self, member: &PublicKey, owner: &PublicKey) -> u32; + pub fn is_authorized(&self, actor: &PublicKey, owner: &PublicKey, bit: u64) -> bool; + pub fn is_authorized_in(&self, actor: &PublicKey, owner: &PublicKey, channel: &ChannelId, bit: u64) -> bool; + pub fn outranks(&self, actor: &PublicKey, owner: &PublicKey, target_position: u32) -> bool; + pub fn can_act_on(&self, actor: &PublicKey, owner: &PublicKey, target: &PublicKey, bit: u64) -> bool; + pub fn is_staff(&self, member: &PublicKey, owner: &PublicKey) -> bool; // the six control bits, CORD-04 §3 +} +``` + +Authority rules to encode once and test hard: + +- The owner is position 0, derived from `community_id`, and is never removable. +- No edition may claim a `position` at or above its own signer's, including the owner: no Role may claim 0. +- The actor must hold the required bit **and strictly outrank** the target. Equal cannot act on equal. +- A `vac` citation is a sync floor, not a verdict: block until the cited Grant version is folded, verify its hash, then judge against the *current* roster. +- A staff-making Grant carries `control_wrap`, a NIP-44 pairwise ciphertext of `epoch_be[8] ‖ control_root[32]`, and is adopted **only if it derives to the `control_pk` the member already holds** for the named epoch. +- Banlist is one replaced entity; mutations carry a re-heal step (re-fold after publish, re-apply if the addition lost the tiebreak). + +### 8.2 Communities, channels, metadata + +`CommunityMetadata` carries `name` (≤ 64 bytes), `description` (≤ 10 000 bytes), `relays` (truncated on read and write to 5), `icon` and `banner` as encrypted-blob pointers (`{url, key, nonce, hash}`), `message_expiration`, and the optional `custom` object. `ChannelMetadata` carries `name`, `private`, optional `voice`, `deleted`, optional `custom`. + +Every content struct uses `#[serde(flatten)] extra: serde_json::Map` and round-trips unknown fields. A name edit by an older client must not wipe another client's `custom` keys. Round-trip discipline gets its own test. + +Channel keying follows CORD-03 §1: a public channel derives from `community_root` at the base epoch, a private one from its own random key at its own epoch. Public→private is a rekey at the next channel epoch (monotonic, never reset); private→public derives from the base again and the prior private history stays unreadable to later joiners. + +### 8.3 Guestbook and member list (`guestbook.rs`) + +```rust +pub enum GuestbookEntry { + Join { member: PublicKey, at_ms: u64, invited_by: Option<(String, String)> }, + Leave { member: PublicKey, at_ms: u64 }, + Kick { actor: PublicKey, target: PublicKey, at_ms: u64, citation: Option }, + Snapshot { refounder: PublicKey, members: Vec, snapshot_id: [u8; 32], chunk: (u32, u32), at_ms: u64 }, +} + +pub fn coalesce(events: &[GuestbookEvent], now_ms: u64, snapshot_authority: Option<&PublicKey>, + can_kick: impl Fn(&PublicKey, &PublicKey, Option<&AuthorityCitation>) -> bool) + -> BTreeMap; + +pub fn complete_memberlist(coalesced: &BTreeMap, + observed: &BTreeMap, // author → newest ms published + banned: &BTreeSet, banned_at: &BTreeMap, + refound: Option<&Refound>) -> BTreeSet; +``` + +- Entries dated more than an hour ahead of local time are dropped. An `ms` outside `0..999` drops the entry rather than being interpreted. +- Coalesce is per npub, one final state each, by millisecond time, ties on the lower inner rumor id. +- A Kick counts only when its signer holds `KICK` and outranks the target. +- A Snapshot counts only from the npub whose Refounding minted the epoch. There is deliberately no owner fallback. +- The member list is `coalesced Joined ∪ observed authors − banlist`, and observation counts **forward only** (an author re-enters on activity newer than their latest Leave/Kick/Ban). A Grant holder with `ms == 0` is present by construction. + +### 8.4 Chat plane (`chat.rs`) + +```rust +pub struct ChatMessage { + pub id: EventId, // recomputed rumor id + pub author: PublicKey, + pub channel: ChannelId, + pub epoch: Epoch, + pub kind: Kind, // 9 | 1111 | 3302 | 1740 | 15 + pub content: String, + pub media: Vec, + pub mentions: Vec, + pub reply_to: Option, // lowercase `e`/`q` + pub thread_root: Option, // uppercase `E` for 1111 + pub at_ms: u64, + pub expiration: Option, + pub edited_at: Option, // folded from 3302 + pub deleted: bool, // folded from 5 + pub reactions: BTreeMap, +} +``` + +Sends funnel through one function so the rules cannot drift: + +```rust +fn publish_chat(store, client, community, channel, epoch, group, rumor, at_ms, ephemeral) -> Task, Error>>; +``` + +It builds the seal + wrap, mirrors the NIP-40 tag onto the wrap for durable kinds, publishes via `send_event(..).to(relays)`, retains the ephemeral wrap key for later NIP-09 scrubbing, and locally echoes its own wrap through the same ingest path so send-then-read works without waiting on a relay round-trip. + +Disappearing messages (CORD-08) live here: `message_expiration` is read from the folded metadata, `["expiration", created_at + t]` is attached to every durable Chat rumor and to the wrap, kinds 5 and 1740 are exempt, ingest refuses an already-expired rumor, a periodic sweep purges stored ones, and the kind 1740 timer notice renders only when its author holds `MANAGE_METADATA`. + +### 8.5 Invites (`invite.rs`) + +```rust +pub struct CommunityInvite { community_id, owner, owner_salt, community_root, root_epoch, + control_pk: Option, channels: Vec, + relays: Vec, name: String, icon: Option, + expires_at: Option, creator_npub: Option, label: Option, + extra: Map } +impl CommunityInvite { pub fn validate(&self) -> Result<(), InviteError>; // recompute community_id + pub fn expired(&self, now_ms: u64) -> bool; } + +pub fn build_bundle(token: &[u8; 16], link_signer: &Keys, invite: &CommunityInvite) -> Result; // 33301, d = "" +pub fn build_revocation(link_signer: &Keys) -> Result; // vsk 9 +pub fn parse_link(input: &str) -> Result; +pub fn encode_fragment(relays: &[RelayUrl], token: &[u8; 16]) -> String; // version byte 4, flags, ≤ 3 relays, base64url +pub fn decode_fragment(fragment: &str) -> Result<(Vec, [u8; 16]), InviteError>; +pub fn build_direct_invite(receiver: &PublicKey, invite: &CommunityInvite, signer: &UniversalSigner) -> Task>; // 3313 rumor → 13 seal → k-tagged 1059 +``` + +The link rides `naddr` (`Nip19Coordinate` for kind 33301, link signer, empty `d`) in the path and the token + bootstrap relays in the fragment. A fragment is never sent to a server. The bundle is decrypted with `invite_bundle_key(token)`, and the joiner must recompute `community_id` from `owner` + `owner_salt`. + +Bounds before allocation: reject a bundle with more than 256 channels, truncate the relay list to 5, refuse an expired one. + +### 8.6 Rekeys and refoundings (`rekey.rs`) + +```rust +pub enum RekeyScope { Channel(ChannelId), Base } +pub fn encode_blob_plaintext(scope, epoch, new_root, control_pk, control_root) -> Vec; // 72 | 104 | 136 bytes +pub fn parse_blob_plaintext(bytes: &[u8], scope, epoch) -> Result; +pub fn build_rekey_rumor(rotator, scope, new_epoch, prev_epoch, prev_commit, blobs, chunk) -> UnsignedEvent; +pub fn plan_refounding(fold, removed: &[PublicKey]) -> Result; +pub fn compact(fold, epoch, new_control_root, ...) -> Vec; // re-wrap heads verbatim, plaintext seals preserved +``` + +- The subscription for rekeys is precomputed from the *next* epoch's address, per private channel and once for the base. +- A receiver accepts a key only after: locating its blob, decrypting with the rotator↔recipient conversation key, checking the bound `scope` and `epoch` inside the plaintext, and matching `prevcommit` against the key it currently holds. +- Only after holding **all** `n` chunks of one `(rotator, newepoch, prevcommit)` set, with none containing its locator, may a client conclude it was removed. +- Send cap 80 blobs per event, accept cap 120 (Vector's documented erratum: the CORD-01 double envelope pushes 120 blobs past a 64 KB relay limit). Record the reason in a comment so nobody "fixes" it back. +- Compacted control heads are re-wrapped with their original signature intact, which is exactly why the control plane uses the plaintext seal. +- Two concurrent refoundings converge on the lexicographically lowest new base key; the heal is down-only. +- Authority: a channel rekey needs `MANAGE_CHANNELS`, a refounding needs `BAN`, and in both the rotator must strictly outrank every removed target. Holding a key is never authority. + +Dissolution (CORD-02 §9) also lives here: a chainless, owner-signed `vsk 10` tombstone at `dissolved_group_key(id)`, plaintext-sealed, and a verifier **must** refuse any tombstone whose `eid` is not the community's own id (including the all-zero placeholder — accepting it lets an owner's genuine tombstone for one community be re-wrapped at another of theirs and kill it permanently). On sight the community is sealed read-only: subscriptions halt, nothing new is honored, existing history stays readable, and a member's delete of their own message is still honored. + +## 9. Storage (`store.rs`) + +Three layers, no new storage engine: + +1. **Raw wraps** (kind 1059) are persisted automatically by the SDK's relay pool when a subscription or fetch matches a filter. Nothing to write. +2. **Opened rumors** are cached locally as NIP-78 `Kind::ApplicationSpecificData` events signed by a session-local keypair, exactly like `chat::set_rumor`. Tags: `["d", rumor_id]` (replace key), `["c", channel_hex]`, `["p", author]`, `["k", kind]`, `["e", wrap_id]`, `["t", "concord"]`. Contents are the rumor JSON. + - The `c`/`t` keys deliberately differ from chat's `r` key so the two message namespaces can never collide in one database. + - The read path dedupes by rumor id and keeps the newest `created_at`, because the local signing key changes per session. +3. **Community state** — one local document per community, `Kind::ApplicationSpecificData` with `["d", "concord/"]`: + +```rust +pub struct CommunityState { + pub id: CommunityId, + pub owner: PublicKey, + pub owner_salt: [u8; 32], + pub community_root: [u8; 32], + pub root_epoch: Epoch, + pub control_root: Option<[u8; 32]>, // present iff the holder is staff + pub control_pks: BTreeMap, + pub channels: Vec, // id, key, epoch, name, private + pub epoch_keys: Vec<([u8; 32], Epoch, [u8; 32])>, // (scope, epoch, key) — the history backfill index + pub relays: Vec, + pub heads: BTreeMap<[u8; 32], (u64, [u8; 32], EventId)>, // entity → (version, self_hash, inner id) + pub guestbook: Vec, + pub observed: BTreeMap, + pub banned: BTreeSet, + pub dissolved: bool, + pub added_at_ms: u64, +} +``` + +Writes are debounced (a fold head changes on every edition); reads load once at init. + +**Decision, stated for the record:** this document stores raw community keys unencrypted in a plaintext local database. That matches the existing posture — `chat` already caches decrypted message rumors in the same LMDB. If that posture ever changes, the state document is the one to wrap with NIP-44-to-self, since it is the only local artifact holding keys. + +History queries: + +```rust +pub async fn query_messages(&self, channel: &ChannelId, until: Option, limit: usize) -> Result, Error>; +pub async fn backfill(&self, plane_authors: &[PublicKey], relays: &[RelayUrl], until: Option, limit: usize) -> Result, Error>; +``` + +`query_messages` reads the local cache (`Filter::new().kind(ApplicationSpecificData).custom_tag(LOWERCASE_C, channel_hex)`); `backfill` pages relays newest-first with `until`, deduplicating by wrap id and stepping past same-second walls. + +## 10. Sync engine and GPUI conventions + +`ConcordRegistry` mirrors `ChatRegistry`'s shape exactly: a foreground GPUI entity holding `Entity` handles, a `flume` signal bus, one background notification listener, one foreground consumer, and task slots that are cleared when the signer changes. + +**Subscription.** Community relays come from the folded metadata. `init`/`join` add them to the client (`client.add_relay(url).and_connect()`), then: + +```rust +let filter = Filter::new() + .kinds([Kind::GiftWrap, Kind::Custom(KIND_WRAP_EPHEMERAL)]) + .authors(plane_authors) // guestbook, control signer, all held channel planes, + // rekey addresses for epoch + 1, dissolved address + .since(Timestamp::from_secs(now - FRESH_WINDOW)) + .limit(0); // live tail only; history comes from backfill +client.subscribe(target).with_id(SubscriptionId::new(CONCORD_SUB)).await?; +``` + +Targeted subscribe against the community relays, with a pool-wide subscribe as the fallback path. Rebuild idempotently whenever a plane's address changes (join, channel added, rekey folded). + +**Routing.** `dispatch` matches on the `subscription_id` carried in `RelayMessage::Event`, dedupes by wrap id (both subscriptions and several relays deliver the same wrap), then recognises the plane by **wrap author** against the derived addresses it holds — never by trial decryption. Recognition order: held channel planes → guestbook → control signer → rekey addresses → dissolved. + +**Ingest pipeline.** Unwrap, verify and fold all happen inside `cx.background_spawn`, never on the foreground thread: secp256k1 verification per edition and per seal is far too expensive for the UI thread. + +```rust +enum Signal { + Chat { community: CommunityId, channel: ChannelId, message: Box }, + Control { community: CommunityId, heads: Vec, roster: Box }, + Guestbook { community: CommunityId, members: BTreeSet }, + Rekey { community: CommunityId, scope: RekeyScope, epoch: Epoch }, + Dissolved(CommunityId), + Eose(SubscriptionId), + Error(ConcordError), +} +``` + +The consumer is `cx.spawn(async move |this, cx| { while let Ok(signal) = rx.recv_async().await { this.update(cx, |this, cx| this.apply(signal, cx))?; } })`, which updates entities and calls `cx.notify()`. + +Rules taken from the project guidelines: + +- Crypto, folding, database queries and network I/O only in `cx.background_spawn`. +- Foreground tasks are `cx.spawn` with `this.update(cx, ..)`; any entity update happens there, and the inner `cx` is always used. +- Tasks are stored in fields (`tasks`, `listener`, `consumer`) so they are cancelled on signer change and dropped with the registry. `detach()` only for genuinely fire-and-forget work such as the local state save. +- Long-running paging is bounded by explicit page and step caps, not by unbounded loops. +- Every fallible path returns `Result` and surfaces through `ConcordEvent::Error`; nothing is silently swallowed. + +**Registry API.** + +```rust +pub fn init(window: &mut Window, cx: &mut App); +pub struct ConcordRegistry { /* … */ } +impl ConcordRegistry { + pub fn global(cx: &App) -> Entity; + pub fn loading(&self) -> bool; + pub fn communities(&self) -> Vec>; + pub fn community(&self, id: &CommunityId, cx: &App) -> Option>; + pub fn find(&self, query: &str, cx: &App) -> Vec>; + + pub fn create(&mut self, params: CommunityParams, cx: &mut Context) -> Task>; + pub fn join(&mut self, link: &str, cx: &mut Context) -> Task>; + pub fn accept_direct_invite(&mut self, rumor: &UnsignedEvent, cx: &mut Context) -> Task>; + pub fn leave(&mut self, id: &CommunityId, cx: &mut Context); + pub fn discard_invite(&mut self, id: &CommunityId, cx: &mut Context); + pub fn refresh(&mut self, id: &CommunityId, cx: &mut Context); + pub fn shutdown(&mut self, cx: &mut Context); // halt subscriptions, keep our own state +} +``` + +**Community API** (`Entity`, `EventEmitter`): + +```rust +pub fn id(&self) -> CommunityId; +pub fn owner(&self) -> PublicKey; +pub fn name(&self) -> SharedString; pub fn description(&self) -> Option; +pub fn icon(&self) -> Option; +pub fn relays(&self) -> Vec; +pub fn epoch(&self) -> Epoch; +pub fn dissolved(&self) -> bool; +pub fn channels(&self) -> Vec>; +pub fn channel(&self, id: &ChannelId, cx: &App) -> Option>; +pub fn members(&self) -> BTreeSet; +pub fn banned(&self) -> BTreeSet; +pub fn roles(&self) -> &CommunityRoles; +pub fn permissions(&self, member: &PublicKey) -> u64; +pub fn is_staff(&self, member: &PublicKey) -> bool; +pub fn message_expiration(&self) -> Option; + +// authority actions — each returns a publish task and nothing optimistic +pub fn set_metadata(&mut self, meta: CommunityMetadata, cx: &mut Context) -> Task>; +pub fn create_channel(&mut self, name: &str, private: bool, cx: &mut Context) -> Task>; +pub fn edit_channel(&mut self, id: &ChannelId, meta: ChannelMetadata, cx: &mut Context) -> Task>; +pub fn create_role(&mut self, role: Role, cx: &mut Context) -> Task>; +pub fn assign_roles(&mut self, member: &PublicKey, roles: &[[u8; 32]], cx: &mut Context) -> Task>; +pub fn ban(&mut self, members: &[PublicKey], cx: &mut Context) -> Task>; +pub fn unban(&mut self, members: &[PublicKey], cx: &mut Context) -> Task>; +pub fn kick(&mut self, member: &PublicKey, cx: &mut Context) -> Task>; +pub fn rekey_channel(&mut self, id: &ChannelId, removed: &[PublicKey], cx: &mut Context) -> Task>; +pub fn refound(&mut self, removed: &[PublicKey], cx: &mut Context) -> Task>; +pub fn dissolve(&mut self, cx: &mut Context) -> Task>; +pub fn create_invite(&mut self, params: InviteParams, cx: &mut Context) -> Task>; +pub fn revoke_invite(&mut self, token: &[u8; 16], cx: &mut Context) -> Task>; +pub fn direct_invite(&mut self, receiver: &PublicKey, cx: &mut Context) -> Task>; +pub fn save_community_list(&mut self, cx: &mut Context) -> Task>; // kind 13302, multi-device sync +``` + +**Channel API** (`Entity`): `id`, `name`, `private`, `epoch`, `deleted`, plus + +```rust +pub fn messages(&self, until: Option, limit: usize, cx: &App) -> Task, Error>>; +pub fn send(&self, content: &str, reply_to: Option, cx: &App) -> Task, Error>>; +pub fn send_file(&self, file: FileAttachment, reply_to: Option, cx: &App) -> Task, Error>>; +pub fn edit(&self, id: EventId, content: &str, cx: &App) -> Task, Error>>; +pub fn delete(&self, id: EventId, cx: &App) -> Task, Error>>; +pub fn react(&self, id: EventId, emoji: &str, cx: &App) -> Task, Error>>; +pub fn typing(&self, cx: &App) -> Task>; // kind 23311, ephemeral +pub fn pin(&self, id: EventId, cx: &App) -> Task>; // vsk 11, PIN_MESSAGES +``` + +`CommunityEvent` and `ChannelEvent` mirror `ChatEvent`: one variant per thing the UI has to react to (`Updated`, `Members`, `Added`, `Removed`, `Dissolved`, `Error`, plus channel-level `Incoming`, `Reload`). + +## 11. Integration with existing crates + +1. **`crates/chat/src/lib.rs` — required fix.** `handle_notifications` currently treats *every* kind 1059 event as a NIP-17 wrap for the current user and pushes an unwrap failure into the trash. Concord wraps are kind 1059 with an ephemeral `p` tag, so they would flood the trash and leak error toasts. Route by `subscription_id` from `RelayMessage::Event` against `sub_id1`/`sub_id2`, and drop the `if rumor.tags.is_empty()` heuristic once the real recipient check is in place. +2. **`desktop/src/main.rs` and `web/src/lib.rs`** — add `concord::init(window, cx)` after `chat::init(window, cx)`. +3. **`Cargo.toml`** — add `hkdf = "0.12"` to `[workspace.dependencies]`; add the crate to `desktop` and `web` dependencies. No other workspace changes. +4. **No changes** to `state`, `person`, `device`, `settings`, `common`, or `ui`. + +## 12. Security invariants to test, not to assume + +Each of these has burned a real implementation, or is a documented cross-client trap: + +- Recompute every rumor id and reject a claimed mismatch; never trust an embedded `id`. +- Require `rumor.pubkey == seal.pubkey`. +- Require the plaintext seal form on Control and the encrypted form on Chat/Guestbook/Rekey — a strict reader must drop a mis-sealed edition rather than fold a chain a later compaction would fork. +- Check `channel` **and** `epoch` against the plane whose key opened the wrap; reject duplicates of either tag. +- Reject duplicate `vsk`/`eid`/`ev`/`ep`/`vac` tags; require decimal-with-no-leading-zeros on every numeric tag. +- Refuse a tombstone whose `eid` is not this community's id. +- Adopt a `control_root` from a Grant only if it derives to the `control_pk` held for that epoch; adopt a rekey blob only if its bound plaintext matches the scope and epoch and its `prevcommit` matches the key currently held. +- Never conclude removal from a partial rekey chunk set. +- Drop guestbook entries more than an hour in the future; treat an out-of-range `ms` as malformed, not as an interpretation opportunity. +- Never honour a Snapshot from anyone but the refounder of that epoch. +- Refuse to write a Pin List from a list the writer could not read. +- Enforce the NIP-44 65 535-byte cap at every layer before publishing, and the 5-relay / 256-channel / 50-membership / 100-roles / 64-roles-per-member / 500-banlist / 25-pin caps at their ingest and write points. +- Lowercase hex only; x-only pubkeys only; no version tag anywhere. + +## 13. Milestones + +| # | Deliverable | Done when | +| --- | --- | --- | +| M0 | Crate skeleton, `derive.rs`, golden vectors, workspace wiring | `cargo test -p concord` pins every derivation from an independent vector set; all labels match Appendix A.6 | +| M1 | `stream.rs` + `store.rs` | seal/wrap/open round-trips for both seal forms; malformed inputs rejected in the documented order; local cache reads back after a restart | +| M2 | `edition.rs` + `control.rs` genesis | a community is created and published; its two genesis wraps open at a second client sharing the keys; edition hash matches the cross-client vector | +| M3 | Control fold + roster + metadata/channels | fold tests for chains, gaps, downgrade refusal, fork tiebreak, compaction dangle; metadata and channel edits visible to a second client | +| M4 | Chat plane | send/receive/edit/delete/reaction across two identities; history pages backwards across an epoch boundary; binding checks reject a foreign channel/epoch | +| M5 | Guestbook + member list + moderation | join/leave/kick/ban converge to the same memberlist on both clients; every authority rule from §8.1 has a negative test | +| M6 | Invites + Community List | link mint → fetch → join round-trips; revoked link refuses; direct invite lands in the recipient's giftwrap inbox via the `k` tag; a second device reconstructs membership from 13302 | +| M7 | Rekeys + refounding + dissolution | a removed member stops reading after a rekey; continuity and race rules tested; a tombstone seals the community and a foreign-id tombstone is refused | +| M8 | Pins + disappearing messages + hardening | pins verify from a keyless reader's view; expiry is refused at ingest and purged by the sweep; the audit of §12 is complete with a test per bullet | + +Ordering is deliberately dependency-first: each milestone is usable on its own, and nothing in M2+ depends on a later milestone. + +## 14. Open questions and risks + +1. **Community List kind.** CORD-02 §8 specifies `13302`, replaceable. Vector has retired it in favour of fragmented `33302`, because a replaceable kind holds one event per pubkey and so cannot shard past the NIP-44 size cap. We implement `13302` per spec, enforce the 50-membership cap and pre-publish size check, and treat `33302` as an interop follow-up. Confirm with Armada before writing the multi-device code. +2. **NIP-42 for stream-authored REQs.** Relays that gate kind 1059 by author (for example `ditto-relay`'s `AUTH_KINDS`) need an AUTH event signed by that plane's derived key. `nostr-sdk`'s `Authenticator` is per-client and returns one identity, while a Concord client holds many plane keys — so this cannot be solved by swapping the authenticator. Options: contribute a per-REQ auth hook upstream, or accept that such relays are unsupported and prefer relays without the gate. Decide before M7; the default is "documented limitation" plus a relay-capability check. +3. **`invite_bundle_key`.** Appendix A.6 says the label "yields the public-invite decrypt key" without stating whether that is the raw HKDF output used as a NIP-44 conversation key or the `conv_key` of a normalized keypair. The reference implementation uses the raw output. Pin a vector and verify against Armada early — this one decides whether links open at all. +4. **Missing golden vector for `pins_locator`.** Upstream publishes none. Ours will be self-referential; flag it in the test. +5. **Relay set.** Up to 5 recommended, and both reads and writes fan out across them. Coop's client is a gossip client with `no_background_refresh`, so community relays must be added explicitly and re-added on metadata change. +6. **Local plaintext state.** §9 records the decision. Revisit only if the local database stops being treated as trusted. +7. **Was a `community_id` ever hashed into a tag?** No — it must never appear on the wire. A lint-style test should assert it appears in no published event other than inside an invite bundle and a direct invite. + +## 15. Test strategy + +- **Unit, pure:** derivations against golden vectors, edition hash, fold, coalesce, memberlist, blob codec, caps and rejection paths. These need no GPUI context and should be exhaustive — they are where cross-client divergence is caught. +- **Integration, GPUI:** `TestAppContext` with two registries sharing an in-memory database, driving wraps through the ingest path; timeouts and delays use `cx.background_executor().timer(..)` per the project guidelines, never `smol::Timer`. +- **Round-trip:** every builder paired with its parser, asserting the parse produces the identical structure, including unknown-field round-tripping on all content types. +- **Negative:** every bullet in §12 gets a test that constructs the hostile input and asserts the drop. -- 2.54.0 From 5f2a5d7a37b588624764858e8774d7113404841f Mon Sep 17 00:00:00 2001 From: Ren Amamiya Date: Wed, 16 Sep 2026 16:21:38 +0700 Subject: [PATCH 02/12] add concord crate and basic cryptography --- Cargo.lock | 12 + Cargo.toml | 1 + PLAN.md | 69 ++++-- crates/concord/Cargo.toml | 14 ++ crates/concord/src/derive.rs | 454 +++++++++++++++++++++++++++++++++++ crates/concord/src/lib.rs | 110 +++++++++ 6 files changed, 643 insertions(+), 17 deletions(-) create mode 100644 crates/concord/Cargo.toml create mode 100644 crates/concord/src/derive.rs create mode 100644 crates/concord/src/lib.rs diff --git a/Cargo.lock b/Cargo.lock index bebad170..fc9c70f2 100644 --- a/Cargo.lock +++ b/Cargo.lock @@ -1302,6 +1302,18 @@ version = "0.4.33" source = "registry+https://github.com/rust-lang/crates.io-index" checksum = "6e8ccc4ea9f6acc32d102c0f6d471d11d913ad15f20c04de743374861fa1d414" +[[package]] +name = "concord" +version = "1.0.2" +dependencies = [ + "anyhow", + "data-encoding", + "hkdf", + "nostr", + "nostr-sdk", + "sha2 0.10.9", +] + [[package]] name = "concurrent-queue" version = "2.5.0" diff --git a/Cargo.toml b/Cargo.toml index a50022e0..3eac7717 100644 --- a/Cargo.toml +++ b/Cargo.toml @@ -31,6 +31,7 @@ nostr = { git = "https://github.com/rust-nostr/nostr", features = [ "nip59", "ni aes-gcm = "0.10" sha2 = "0.10" data-encoding = "2" +hkdf = "0.12" # Others anyhow = "1.0.44" diff --git a/PLAN.md b/PLAN.md index 048a6a01..fbda8e66 100644 --- a/PLAN.md +++ b/PLAN.md @@ -59,7 +59,16 @@ Verified against the pinned revision (`nostr` 0.45.4 / `nostr-sdk` 0.45.2, git ` **Not needed.** `secp256k1` (use `nostr::SecretKey::from_slice` + `Keys::new`), `base64` (use `data_encoding::BASE64`, already a workspace dep), `bech32` (NIP-19 is in the SDK), any new storage engine (the client's LMDB database is enough), any new HTTP client. -**Add one dependency:** `hkdf = "0.12"` (already in `Cargo.lock` transitively). Add it to `[workspace.dependencies]` and to the new crate. `sha2` is already a workspace dep. +**Add one dependency now:** `hkdf = "0.12"` (already in `Cargo.lock` transitively). Add it to `[workspace.dependencies]` and to the new crate. `sha2` is already a workspace dep. **Two more at M8:** the Pin List's per-message key disclosure needs `chacha20 = "0.9"` (already in the tree because we enable nostr's `nip44`, which is where `chacha20` comes from) and `hmac = "0.12"` (already in the tree via `hkdf`) — see §14.8. Zero new crates, all three are direct-dependency lines only. + +**Why not depend on Vector's crates.** `vector-core` (published, MIT) holds the only other Rust Concord implementation, in `src/community/v2/*`. It is not reusable as a dependency, and the "reuse their crypto" argument does not hold: + +- **No shared types.** It exact-pins the nostr family (`nostr = "=0.45.1"`, `nostr-sdk = "=0.45.1"`, `nostr-connect = "=0.45.1"`, `nostr-blossom = "=0.45.0"`) with the note that a caret range would let a consumer resolve a mixed set, while we track git master (`b230cec`, 0.45.4 / 0.45.2). A registry 0.45.1 and a git 0.45.4 cannot unify, so a build linking both carries two `nostr` crates whose `Event`/`Keys`/`PublicKey`/`Client` are unrelated types. +- **Not wasm-buildable.** `rusqlite` (bundled C SQLite), `libc`, `rustls`, `reqwest`, `image`, `bip39`, and a `tokio` `net` + `rt-multi-thread` requirement; `VectorCore::init` installs a process-global rustls provider and raises the fd limit. Coop's `web` target is wasm32. +- **It is an application core, not a Concord library.** 80k+ lines over 111 files, built on process-global singletons (`state::STATE`, `MY_SECRET_KEY`, one app-data dir, one live account, `traits::set_event_emitter`) and its own SQLite schema, relay pool and blocking `listen()` loop. Adopting it means handing it the nsec and letting it own the client, the database and the event loop — replacing `state`, `chat` and `person` rather than reusing a component. Its `login` stores raw secret-key bytes in that global vault, so an account whose key lives in a signer cannot drive it. +- **There is no cryptography to share.** Both implementations call the same audited crates — `hkdf`, `sha2`, nostr's secp256k1 keypair, and nostr's NIP-44 v2. Vector's comment on that same dependency is "audited RustCrypto crate rather than a hand-rolled construction". Our `derive.rs` has no curve arithmetic, AEAD or randomness of its own: it holds the frozen `info` layout and label table, which are the wire format, not a primitive. + +So Vector's crates earn their place as an **oracle, not a dependency**: the golden vectors in `derive.rs` are their published data, produced by an independent implementation. ## 4. Crate layout @@ -96,6 +105,7 @@ pub struct Epoch(pub u64); pub struct GroupKey { keys: Keys, conversation: ConversationKey } impl GroupKey { pub fn pk(&self) -> PublicKey; + pub fn pk_hex(&self) -> String; // lowercase; Debug prints only this, never key material pub fn keys(&self) -> &Keys; pub fn conversation(&self) -> &ConversationKey; } @@ -117,22 +127,25 @@ Ordering everywhere uses `at_ms`, never `created_at`, and ties break on the lowe ## 6. Frozen derivations (`derive.rs`) +Implemented and pinned in `crates/concord/src/derive.rs`. + ```rust fn build_info(label: &str, id: &[u8; 32], epoch: Option) -> Vec; // label ‖ 0x00 ‖ id[32] ‖ epoch_be[8]? fn hkdf32(ikm: &[u8], info: &[u8]) -> [u8; 32]; // HKDF-SHA256, zero-length salt, L = 32 -fn hkdf_to_secret_key(ikm: &[u8], info: &[u8]) -> SecretKey; // A.3 scalar_normalize, counter from 0 +fn hkdf_to_secret_key(ikm: &[u8], info: &[u8]) -> Result; // A.3 scalar_normalize, counter from 0 -fn group_key(label: &str, secret: &[u8], id: &[u8; 32], epoch: Option) -> GroupKey; +fn group_key(label: &str, secret: &[u8], id: &[u8; 32], epoch: Option) -> Result; -pub fn channel_group_key(secret: &[u8; 32], channel: &ChannelId, epoch: Epoch) -> GroupKey; -pub fn control_group_key(root: &[u8; 32], id: &CommunityId, epoch: Epoch) -> GroupKey; // read key -pub fn control_signer_group_key(root: &[u8; 32], id: &CommunityId, epoch: Epoch) -> GroupKey; // write key -pub fn guestbook_group_key(root: &[u8; 32], id: &CommunityId, epoch: Epoch) -> GroupKey; -pub fn channel_rekey_group_key(root: &[u8; 32], channel: &ChannelId, epoch: Epoch) -> GroupKey; -pub fn base_rekey_group_key(root: &[u8; 32], id: &CommunityId, epoch: Epoch) -> GroupKey; -pub fn dissolved_group_key(id: &CommunityId) -> GroupKey; // no epoch field +pub fn channel_group_key(secret: &[u8; 32], channel: &ChannelId, epoch: Epoch) -> Result; +pub fn control_group_key(root: &[u8; 32], id: &CommunityId, epoch: Epoch) -> Result; // read key +pub fn control_signer_group_key(root: &[u8; 32], id: &CommunityId, epoch: Epoch) -> Result; // write key +pub fn guestbook_group_key(root: &[u8; 32], id: &CommunityId, epoch: Epoch) -> Result; +pub fn channel_rekey_group_key(root: &[u8; 32], channel: &ChannelId, epoch: Epoch) -> Result; +pub fn base_rekey_group_key(root: &[u8; 32], id: &CommunityId, epoch: Epoch) -> Result; +pub fn dissolved_group_key(id: &CommunityId) -> Result; // no epoch field pub fn community_id_of(owner_xonly: &[u8; 32], owner_salt: &[u8; 32]) -> CommunityId; // plain SHA-256 +pub fn verify_community_id(id: &CommunityId, owner_xonly: &[u8; 32], owner_salt: &[u8; 32]) -> bool; pub fn epoch_key_commitment(epoch: Epoch, key: &[u8; 32]) -> [u8; 32]; // plain SHA-256 pub fn grant_locator(id: &CommunityId, member: &[u8; 32]) -> [u8; 32]; pub fn banlist_locator(id: &CommunityId) -> [u8; 32]; @@ -140,16 +153,37 @@ pub fn pins_locator(id: &CommunityId, channel: &ChannelId) -> [u8; 32]; pub fn invite_links_locator(id: &CommunityId, creator: &[u8; 32]) -> [u8; 32]; pub fn recipient_locator(rotator: &[u8; 32], recipient: &[u8; 32], scope: &[u8; 32], epoch: Epoch) -> [u8; 32]; pub fn invite_bundle_key(token: &[u8; 16]) -> [u8; 32]; // raw hkdf32 output; used as a NIP-44 conversation key +pub fn clear_memo(); // drop memoised keys on signer change ``` +Appendix A.6, as implemented — `ikm` / `id` / `epoch`. The id is *always* present (all-zeroes where a label has no meaningful one); the epoch is the only omittable field. + +| Label | ikm | id | epoch | +| --- | --- | --- | --- | +| `concord/channel` | channel key or `community_root` | `channel_id` | yes | +| `concord/control` | `community_root` | `community_id` | yes | +| `concord/control-signer` | `control_root` | `community_id` | yes | +| `concord/rekey-pseudonym` | prior `community_root` | `channel_id` | new epoch | +| `concord/base-rekey-pseudonym` | prior `community_root` | `community_id` | new epoch | +| `concord/recipient-pseudonym` | `rotator_xonly ‖ recipient_xonly` (64 B) | scope id | new epoch | +| `concord/guestbook` | `community_root` | `community_id` | yes | +| `concord/dissolved` | `community_id` | zeroes | — | +| `concord/grant` | `community_id` | member x-only | — | +| `concord/banlist` | `community_id` | zeroes | — | +| `concord/pins` | `community_id` | `channel_id` | — | +| `concord/invite-links` | `community_id` | creator x-only | — | +| `concord/invite-key` | token (16 B) | zeroes | — | + +The CORD-07 `concord/voice-*` labels and the retired `concord/invite-locator` / `concord/invite-signer` are reserved and listed here only: they are underived, and the table stays append-only. Every label a derivation does use has a distinct pinned output, so a duplicated label cannot pass the vectors. + Rules that must be enforced by construction, not by convention: - Hex is lowercase everywhere; pubkeys are x-only hex, never bech32; tag numbers are decimal with no leading zeros (`"4"`, never `04`/`+4`). -- The epoch field is *omitted*, not zeroed, for labels with no epoch (`concord/dissolved`, locators, `concord/community`). -- `scalar_normalize` retries by appending a counter byte to the same `info`, starting at `0`. -- Labels and commitments are append-only. A test asserts every label is unique and that the label table matches Appendix A.6 exactly. +- The epoch field is *omitted*, not zeroed, for labels with no epoch; a test asserts `dissolved_group_key` differs from the same derivation with `Some(0)`. +- `scalar_normalize` retries by appending a counter byte to the same `info`, starting at `0`, and reports exhaustion instead of panicking — so plane keys return `Result`. +- Group keys are memoised by a digest of their inputs, so no deriving secret is a map key, bounded at 1024 entries. -**Golden vectors.** `derive.rs` carries a `#[cfg(test)]` block pinning every derivation output, seeded from the independent Python vectors published by the Vector implementation (channel/control/control-signer/guestbook at epoch 0 and at `0x0102030405060708`, dissolved, all four locators, invite key, community id, epoch commitment). One vector is missing upstream — `pins_locator` — so we mint it from our own implementation and pin it, flagged in the test as self-referential. Changing any pinned value means the wire format changed. +**Golden vectors.** `derive.rs` pins all 18 published vectors (the seed and `pk` for channel, control, control-signer and guestbook; both keyed labels at epoch `0` and at `0x0102030405060708`; both rekey labels at epoch 1; dissolved; all four locators; the invite key; the community id; the epoch-key commitment), cross-checked against an independent Python implementation (RFC 5869 HKDF plus pure-integer secp256k1) before being frozen. One vector is missing upstream — `pins_locator` — so we mint it from our own implementation and pin it, flagged in the test as self-referential. Changing any pinned value means the wire format changed. ## 7. Stream codec (`stream.rs`) @@ -545,12 +579,13 @@ Ordering is deliberately dependency-first: each milestone is usable on its own, ## 14. Open questions and risks 1. **Community List kind.** CORD-02 §8 specifies `13302`, replaceable. Vector has retired it in favour of fragmented `33302`, because a replaceable kind holds one event per pubkey and so cannot shard past the NIP-44 size cap. We implement `13302` per spec, enforce the 50-membership cap and pre-publish size check, and treat `33302` as an interop follow-up. Confirm with Armada before writing the multi-device code. -2. **NIP-42 for stream-authored REQs.** Relays that gate kind 1059 by author (for example `ditto-relay`'s `AUTH_KINDS`) need an AUTH event signed by that plane's derived key. `nostr-sdk`'s `Authenticator` is per-client and returns one identity, while a Concord client holds many plane keys — so this cannot be solved by swapping the authenticator. Options: contribute a per-REQ auth hook upstream, or accept that such relays are unsupported and prefer relays without the gate. Decide before M7; the default is "documented limitation" plus a relay-capability check. -3. **`invite_bundle_key`.** Appendix A.6 says the label "yields the public-invite decrypt key" without stating whether that is the raw HKDF output used as a NIP-44 conversation key or the `conv_key` of a normalized keypair. The reference implementation uses the raw output. Pin a vector and verify against Armada early — this one decides whether links open at all. -4. **Missing golden vector for `pins_locator`.** Upstream publishes none. Ours will be self-referential; flag it in the test. +2. **NIP-42 for stream-authored REQs.** Relays that gate kind 1059 by author (for example `ditto-relay`'s `AUTH_KINDS`) need an AUTH event signed by that plane's derived key. `nostr-sdk`'s `Authenticator` is per-client and returns one identity, while a Concord client holds many plane keys — so this cannot be solved by swapping the authenticator. Vector's answer is a dedicated stream-auth responder installed on the client (`community/v2/streamauth`), primed before any relay interaction, which retains the relay's challenge so plane keys registering later can still answer it: a gating relay challenges once per connection and will not re-challenge an authed one, so a responder attached later never gets the chance. Read that module before deciding; the alternative options remain a per-REQ auth hook upstream or documenting the limitation. +3. **`invite_bundle_key` — resolved in M0.** Appendix A.6 was read in full: the raw HKDF output *is* the NIP-44 conversation key, and the derivation is now pinned by a vector. +4. **`pins_locator` has no upstream vector.** Resolved in M0 by minting one from our own implementation and flagging it self-referential in the test. 5. **Relay set.** Up to 5 recommended, and both reads and writes fan out across them. Coop's client is a gossip client with `no_background_refresh`, so community relays must be added explicitly and re-added on metadata change. 6. **Local plaintext state.** §9 records the decision. Revisit only if the local database stops being treated as trusted. 7. **Was a `community_id` ever hashed into a tag?** No — it must never appear on the wire. A lint-style test should assert it appears in no published event other than inside an invite bundle and a direct invite. +8. **The Pin List's message-key disclosure has no public API (M8).** CORD-04 §7 Pins let a keyless reader verify a disclosure, which means revealing one message's NIP-44 keys rather than the plane's conversation key. `nostr`'s `nip44::v2::get_message_keys(conversation_key, nonce)` is a private `fn`, and both public entry points (`encrypt_to_bytes_with_nonce`, `decrypt_to_bytes`) take the whole conversation key — so the expansion has to be reproduced as `hkdf::expand_into(conversation_key, nonce, 76 bytes)` plus ChaCha20 and an HMAC-SHA256, exactly as Vector does, and round-tripped against nostr's own `encrypt` in a test. Read CORD-04 §7 in full at M8 before writing it: the reproduction is only worth it once the exact verification the pin must support is settled, and the alternative is contributing a `pub` message-key accessor upstream (we already track git master, so a patch branch or an upstream PR is viable and strictly better than a reproduction we must keep in sync). ## 15. Test strategy diff --git a/crates/concord/Cargo.toml b/crates/concord/Cargo.toml new file mode 100644 index 00000000..dbc2af0a --- /dev/null +++ b/crates/concord/Cargo.toml @@ -0,0 +1,14 @@ +[package] +name = "concord" +version.workspace = true +edition.workspace = true +publish.workspace = true + +[dependencies] +nostr.workspace = true +nostr-sdk.workspace = true + +hkdf.workspace = true +sha2.workspace = true +data-encoding.workspace = true +anyhow.workspace = true diff --git a/crates/concord/src/derive.rs b/crates/concord/src/derive.rs new file mode 100644 index 00000000..02ec00a6 --- /dev/null +++ b/crates/concord/src/derive.rs @@ -0,0 +1,454 @@ +use std::collections::HashMap; +use std::sync::{LazyLock, Mutex, PoisonError}; + +use anyhow::{Result, bail}; +use hkdf::Hkdf; +use nostr::nips::nip44::v2::ConversationKey; +use nostr_sdk::prelude::{Keys, PublicKey, SecretKey}; +use sha2::{Digest, Sha256}; + +use crate::{ChannelId, CommunityId, Epoch}; + +pub const TOKEN_LEN: usize = 16; + +const LABEL_CHANNEL: &str = "concord/channel"; +const LABEL_CONTROL: &str = "concord/control"; +const LABEL_CONTROL_SIGNER: &str = "concord/control-signer"; +const LABEL_REKEY_PSEUDONYM: &str = "concord/rekey-pseudonym"; +const LABEL_BASE_REKEY_PSEUDONYM: &str = "concord/base-rekey-pseudonym"; +const LABEL_RECIPIENT_PSEUDONYM: &str = "concord/recipient-pseudonym"; +const LABEL_GUESTBOOK: &str = "concord/guestbook"; +const LABEL_DISSOLVED: &str = "concord/dissolved"; +const LABEL_GRANT: &str = "concord/grant"; +const LABEL_BANLIST: &str = "concord/banlist"; +const LABEL_PINS: &str = "concord/pins"; +const LABEL_INVITE_LINKS: &str = "concord/invite-links"; +const LABEL_INVITE_KEY: &str = "concord/invite-key"; + +const LABEL_COMMUNITY: &str = "concord/community"; +const LABEL_EPOCH_COMMITMENT: &str = "concord/epoch-key-commitment"; + +const ZERO32: [u8; 32] = [0u8; 32]; + +fn build_info(label: &str, id32: &[u8; 32], epoch: Option) -> Vec { + let mut info = Vec::with_capacity(label.len() + 1 + 32 + 8); + info.extend_from_slice(label.as_bytes()); + info.push(0x00); + info.extend_from_slice(id32); + + if let Some(epoch) = epoch { + info.extend_from_slice(&epoch.to_be_bytes()); + } + + info +} + +fn hkdf32(ikm: &[u8], info: &[u8]) -> [u8; 32] { + let mut okm = [0u8; 32]; + Hkdf::::new(None, ikm) + .expand(info, &mut okm) + .expect("expanding HKDF to 32 bytes is below the 255*32 ceiling"); + okm +} + +fn hkdf_to_secret_key(ikm: &[u8], base_info: &[u8]) -> Result { + if let Ok(secret_key) = SecretKey::from_slice(&hkdf32(ikm, base_info)) { + return Ok(secret_key); + } + + for counter in 0u8..=u8::MAX { + let mut info = Vec::with_capacity(base_info.len() + 1); + info.extend_from_slice(base_info); + info.push(counter); + + if let Ok(secret_key) = SecretKey::from_slice(&hkdf32(ikm, &info)) { + return Ok(secret_key); + } + } + + bail!("seed stayed out of the secp256k1 scalar range across all 256 counters") +} + +#[derive(Clone)] +pub struct GroupKey { + keys: Keys, + conversation: ConversationKey, +} + +impl GroupKey { + fn derive(label: &str, secret: &[u8], id32: &[u8; 32], epoch: Option) -> Result { + let key = memo_key(label, secret, id32, epoch); + + if let Some(hit) = lock_memo().get(&key) { + return Ok(hit.clone()); + } + + let info = build_info(label, id32, epoch); + let secret_key = hkdf_to_secret_key(secret, &info)?; + let keys = Keys::new(secret_key); + let conversation = ConversationKey::derive(keys.secret_key(), &keys.public_key())?; + let group_key = Self { keys, conversation }; + + let mut memo = lock_memo(); + + if memo.len() >= 1024 { + memo.clear(); + } + + memo.insert(key, group_key.clone()); + + Ok(group_key) + } + + pub fn pk(&self) -> PublicKey { + self.keys.public_key() + } + + pub fn pk_hex(&self) -> String { + self.keys.public_key().to_hex() + } + + pub fn keys(&self) -> &Keys { + &self.keys + } + + pub fn conversation(&self) -> &ConversationKey { + &self.conversation + } +} + +impl std::fmt::Debug for GroupKey { + fn fmt(&self, f: &mut std::fmt::Formatter<'_>) -> std::fmt::Result { + f.debug_struct("GroupKey") + .field("pk", &self.pk_hex()) + .finish() + } +} + +static MEMO: LazyLock>> = LazyLock::new(Default::default); + +fn lock_memo() -> std::sync::MutexGuard<'static, HashMap<[u8; 32], GroupKey>> { + MEMO.lock().unwrap_or_else(PoisonError::into_inner) +} + +pub fn clear_memo() { + lock_memo().clear() +} + +fn memo_key(label: &str, secret: &[u8], id32: &[u8; 32], epoch: Option) -> [u8; 32] { + let mut hasher = Sha256::new(); + hasher.update(label.as_bytes()); + hasher.update([0x00]); + hasher.update(secret); + hasher.update(id32); + hasher.update(epoch.unwrap_or(u64::MAX).to_be_bytes()); + hasher.update([epoch.is_some() as u8]); + hasher.finalize().into() +} + +/// `secret` is the `community_root` for a public channel. +pub fn channel_group_key(secret: &[u8; 32], channel: &ChannelId, epoch: Epoch) -> Result { + GroupKey::derive(LABEL_CHANNEL, secret, channel.as_bytes(), Some(epoch.0)) +} + +/// The plane's read key: its conversation key encrypts the wraps for every member. +pub fn control_group_key( + community_root: &[u8; 32], + community_id: &CommunityId, + epoch: Epoch, +) -> Result { + GroupKey::derive( + LABEL_CONTROL, + community_root, + community_id.as_bytes(), + Some(epoch.0), + ) +} + +/// The plane's address and wrap signer, held only by staff. +/// Wraps still encrypt under [`control_group_key`]. +pub fn control_signer_group_key( + control_root: &[u8; 32], + community_id: &CommunityId, + epoch: Epoch, +) -> Result { + GroupKey::derive( + LABEL_CONTROL_SIGNER, + control_root, + community_id.as_bytes(), + Some(epoch.0), + ) +} + +/// Member-writable, unlike the Control Plane: +/// a join or a leave is each member's own word. +pub fn guestbook_group_key( + community_root: &[u8; 32], + community_id: &CommunityId, + epoch: Epoch, +) -> Result { + GroupKey::derive( + LABEL_GUESTBOOK, + community_root, + community_id.as_bytes(), + Some(epoch.0), + ) +} + +/// Keyed by the prior `community_root` rather than the channel key, +/// so any retained member recovers any epoch's rekey without a ratchet. +pub fn channel_rekey_group_key( + prior_root: &[u8; 32], + channel: &ChannelId, + new_epoch: Epoch, +) -> Result { + GroupKey::derive( + LABEL_REKEY_PSEUDONYM, + prior_root, + channel.as_bytes(), + Some(new_epoch.0), + ) +} + +/// Keyed by the prior `community_root`: the base has no stable key above it +pub fn base_rekey_group_key( + prior_root: &[u8; 32], + community_id: &CommunityId, + new_epoch: Epoch, +) -> Result { + GroupKey::derive( + LABEL_BASE_REKEY_PSEUDONYM, + prior_root, + community_id.as_bytes(), + Some(new_epoch.0), + ) +} + +/// Keyed by the `community_id` alone, so every member past or present resolves +/// the same address and a Refounding cannot strand the grave. +pub fn dissolved_group_key(community_id: &CommunityId) -> Result { + GroupKey::derive(LABEL_DISSOLVED, community_id.as_bytes(), &ZERO32, None) +} + +/// A plain SHA-256 commitment +pub fn community_id_of(owner_xonly: &[u8; 32], owner_salt: &[u8; 32]) -> CommunityId { + let mut hasher = Sha256::new(); + hasher.update(LABEL_COMMUNITY.as_bytes()); + hasher.update(owner_xonly); + hasher.update(owner_salt); + CommunityId::from_bytes(hasher.finalize().into()) +} + +pub fn verify_community_id( + community_id: &CommunityId, + owner_xonly: &[u8; 32], + owner_salt: &[u8; 32], +) -> bool { + community_id_of(owner_xonly, owner_salt) == *community_id +} + +/// The continuity a rekey blob must satisfy against the key currently held. +pub fn epoch_key_commitment(previous_epoch: Epoch, previous_key: &[u8; 32]) -> [u8; 32] { + let mut hasher = Sha256::new(); + hasher.update(LABEL_EPOCH_COMMITMENT.as_bytes()); + hasher.update(previous_epoch.0.to_be_bytes()); + hasher.update(previous_key); + hasher.finalize().into() +} + +/// Bound to the `community_id`, so a member's Grant coordinate survives every refounding. +pub fn grant_locator(community_id: &CommunityId, member_xonly: &[u8; 32]) -> [u8; 32] { + hkdf32( + community_id.as_bytes(), + &build_info(LABEL_GRANT, member_xonly, None), + ) +} + +pub fn banlist_locator(community_id: &CommunityId) -> [u8; 32] { + hkdf32( + community_id.as_bytes(), + &build_info(LABEL_BANLIST, &ZERO32, None), + ) +} + +pub fn pins_locator(community_id: &CommunityId, channel: &ChannelId) -> [u8; 32] { + hkdf32( + community_id.as_bytes(), + &build_info(LABEL_PINS, channel.as_bytes(), None), + ) +} + +/// Bound to the creator, so each creator owns exactly their own registry. +pub fn invite_links_locator(community_id: &CommunityId, creator_xonly: &[u8; 32]) -> [u8; 32] { + hkdf32( + community_id.as_bytes(), + &build_info(LABEL_INVITE_LINKS, creator_xonly, None), + ) +} + +/// Built from public inputs only, so a locator match proves nothing about authenticity +pub fn recipient_locator( + rotator_xonly: &[u8; 32], + recipient_xonly: &[u8; 32], + scope_id: &[u8; 32], + new_epoch: Epoch, +) -> [u8; 32] { + let mut ikm = [0u8; 64]; + ikm[..32].copy_from_slice(rotator_xonly); + ikm[32..].copy_from_slice(recipient_xonly); + hkdf32( + &ikm, + &build_info(LABEL_RECIPIENT_PSEUDONYM, scope_id, Some(new_epoch.0)), + ) +} + +/// The raw output is the NIP-44 conversation key (CORD-05 §2). +pub fn invite_bundle_key(token: &[u8; TOKEN_LEN]) -> [u8; 32] { + hkdf32(token, &build_info(LABEL_INVITE_KEY, &ZERO32, None)) +} + +#[cfg(test)] +mod tests { + use super::*; + + const CHANNEL_E0_SEED: &str = + "1a99a5958bf9fcc5336e6e19db42aabf36ffbfa12f38a1d5fbde2ae383ed751b"; + const CHANNEL_E0_PK: &str = "7a5c5dff759a63f1fc2779864487432bae3d1ea72c4ffabd39f4c1fdaf62097a"; + const CHANNEL_EMULTI_PK: &str = + "f20c7d192cc87615d7341e86f38f85303f4708b40232d4fea521ab8217767391"; + const CONTROL_E0_PK: &str = "c43df20bf4d6eeaea5149619662ffe9b211f31e11bb4a59f56b6e906f702d46f"; + const CONTROL_SIGNER_E0_SEED: &str = + "c4a3e8354d95137132087356412b67b53e025d127d45de45cff9ecf45b0c24f6"; + const CONTROL_SIGNER_E0_PK: &str = + "718aef388257f3fd9f1bfae5cf2cbd0594a2ffc31adb5c1fe22c502c046acaee"; + const CONTROL_SIGNER_EMULTI_PK: &str = + "e27235cc13be2f9ad65648e01ff2b63402846469c8638b5386c625688194ec7d"; + const GUESTBOOK_E0_PK: &str = + "ad09de582026fa7a052db18bb5827fa24c15e929d59aadcc91efb8508f5368ad"; + const CHANNEL_REKEY_E1_PK: &str = + "7c55cdb957e9db2b4800d687b2a07d3f7066b1a35824a1e86ba871f55e87e8b5"; + const BASE_REKEY_E1_PK: &str = + "fb2fa44fba66ba15595f784255a1cb569531db8784432ac0e4fe838498dd9dea"; + const DISSOLVED_PK: &str = "4d3d55d88fdf9d9c2089651e5cbb0dfa93b6b9b10cdcb2319b0dce1a1398096a"; + const GRANT_LOCATOR: &str = "fd2f88cc7f1eb8d7d862c91dc22afe700c358d1845158b3f353b769ce4898e35"; + const BANLIST_LOCATOR: &str = + "88089214afae6d3c412fd817ada44d6df4d485a53565646471e74476397693c9"; + const INVITE_LINKS_LOCATOR: &str = + "f4ae29994165767bac23e8dce630f81b926d2c8aa150e5cbf0bdf75865e8379a"; + const RECIPIENT_LOCATOR: &str = + "342deb400e191f0f52c81f27600934552550beb85aa9bf169f02d0e7f826cf74"; + const INVITE_KEY: &str = "94bf8b0d89e579ddaeccf8d9db3f5de5c86a1259c597f2560ff0120173bc5e1f"; + const COMMUNITY_ID: &str = "2b790bd59df98bdc52092b74ebd6933a89ef8eaeecc9030861cbdeae7c814c46"; + const EPOCH_COMMITMENT: &str = + "3e6d6a3c9973c16d1ca7c5602d36979927c55c21a7e2c840f883af3f047e80a4"; + const PINS_LOCATOR: &str = "3b4529395a35c981ed409b588af3c4cd3081992958a485347356a173c3146c52"; + const EPOCH_MULTI: u64 = 0x0102030405060708; + + /// `0x00..0x1f` / `0xff..0xe0` / `0x11` x32 — the inputs every vector uses. + fn secret() -> [u8; 32] { + let mut key = [0u8; 32]; + for (index, byte) in key.iter_mut().enumerate() { + *byte = index as u8; + } + key + } + + fn id32() -> [u8; 32] { + let mut id = [0u8; 32]; + for (index, byte) in id.iter_mut().enumerate() { + *byte = 255 - index as u8; + } + id + } + + fn hex(bytes: &[u8]) -> String { + data_encoding::HEXLOWER.encode(bytes) + } + + #[test] + fn golden_vectors() { + let secret = secret(); + let id = id32(); + let alt = [0x11u8; 32]; + let community_id = CommunityId::from_bytes(id); + let channel = ChannelId::from_bytes(id); + + let channel_e0 = channel_group_key(&secret, &channel, Epoch(0)).expect("derives"); + assert_eq!( + hex(channel_e0.keys().secret_key().as_secret_bytes()), + CHANNEL_E0_SEED + ); + assert_eq!(channel_e0.pk_hex(), CHANNEL_E0_PK); + assert_eq!( + channel_group_key(&secret, &channel, Epoch(EPOCH_MULTI)) + .expect("derives") + .pk_hex(), + CHANNEL_EMULTI_PK + ); + + assert_eq!( + control_group_key(&secret, &community_id, Epoch(0)) + .expect("derives") + .pk_hex(), + CONTROL_E0_PK + ); + + let signer = control_signer_group_key(&secret, &community_id, Epoch(0)).expect("derives"); + assert_eq!( + hex(signer.keys().secret_key().as_secret_bytes()), + CONTROL_SIGNER_E0_SEED + ); + assert_eq!(signer.pk_hex(), CONTROL_SIGNER_E0_PK); + assert_eq!( + control_signer_group_key(&secret, &community_id, Epoch(EPOCH_MULTI)) + .expect("derives") + .pk_hex(), + CONTROL_SIGNER_EMULTI_PK + ); + + assert_eq!( + guestbook_group_key(&secret, &community_id, Epoch(0)) + .expect("derives") + .pk_hex(), + GUESTBOOK_E0_PK + ); + + assert_eq!( + channel_rekey_group_key(&secret, &channel, Epoch(1)) + .expect("derives") + .pk_hex(), + CHANNEL_REKEY_E1_PK + ); + assert_eq!( + base_rekey_group_key(&secret, &community_id, Epoch(1)) + .expect("derives") + .pk_hex(), + BASE_REKEY_E1_PK + ); + assert_eq!( + dissolved_group_key(&community_id) + .expect("derives") + .pk_hex(), + DISSOLVED_PK + ); + + assert_eq!(hex(&grant_locator(&community_id, &alt)), GRANT_LOCATOR); + assert_eq!(hex(&banlist_locator(&community_id)), BANLIST_LOCATOR); + assert_eq!( + hex(&invite_links_locator(&community_id, &alt)), + INVITE_LINKS_LOCATOR + ); + assert_eq!(hex(&pins_locator(&community_id, &channel)), PINS_LOCATOR); + assert_eq!( + hex(&recipient_locator(&secret, &alt, &id, Epoch(3))), + RECIPIENT_LOCATOR + ); + assert_eq!(hex(&invite_bundle_key(&[0x07u8; TOKEN_LEN])), INVITE_KEY); + + assert_eq!(hex(community_id_of(&secret, &alt).as_bytes()), COMMUNITY_ID); + assert_eq!( + hex(&epoch_key_commitment(Epoch(2), &secret)), + EPOCH_COMMITMENT + ); + } +} diff --git a/crates/concord/src/lib.rs b/crates/concord/src/lib.rs new file mode 100644 index 00000000..0a6dc121 --- /dev/null +++ b/crates/concord/src/lib.rs @@ -0,0 +1,110 @@ +pub mod derive; + +use std::fmt; +use std::str::FromStr; + +use anyhow::{Result, anyhow, bail}; +use data_encoding::HEXLOWER; +pub use derive::GroupKey; + +macro_rules! hex_id { + ($(#[$meta:meta])* $name:ident) => { + $(#[$meta])* + #[derive(Clone, Copy, PartialEq, Eq, PartialOrd, Ord, Hash)] + pub struct $name([u8; 32]); + + impl $name { + pub fn from_bytes(bytes: [u8; 32]) -> Self { + Self(bytes) + } + + pub fn as_bytes(&self) -> &[u8; 32] { + &self.0 + } + + pub fn to_hex(&self) -> String { + HEXLOWER.encode(&self.0) + } + } + + impl From<[u8; 32]> for $name { + fn from(bytes: [u8; 32]) -> Self { + Self(bytes) + } + } + + impl fmt::Display for $name { + fn fmt(&self, f: &mut fmt::Formatter<'_>) -> fmt::Result { + f.write_str(&self.to_hex()) + } + } + + impl fmt::Debug for $name { + fn fmt(&self, f: &mut fmt::Formatter<'_>) -> fmt::Result { + write!(f, "{}({})", stringify!($name), self.to_hex()) + } + } + + impl FromStr for $name { + type Err = anyhow::Error; + + fn from_str(value: &str) -> Result { + Ok(Self(decode_hex_32(value)?)) + } + } + }; +} + +hex_id! { + /// A Community's permanent identity: a self-certifying commitment to its + /// owner's key. It travels inside invites and is itself never on the wire + /// (CORD-02 §1). + CommunityId +} + +hex_id! { + /// A Channel's identity within its Community (CORD-03). + ChannelId +} + +/// A key-rotation counter attached to each Community key. +/// +/// It bumps only on a Rekey, a membership change where somebody is removed. +#[derive(Clone, Copy, PartialEq, Eq, PartialOrd, Ord, Hash, Debug, Default)] +pub struct Epoch(pub u64); + +impl From for Epoch { + fn from(value: u64) -> Self { + Self(value) + } +} + +impl From for u64 { + fn from(value: Epoch) -> Self { + value.0 + } +} + +impl fmt::Display for Epoch { + fn fmt(&self, f: &mut fmt::Formatter<'_>) -> fmt::Result { + write!(f, "{}", self.0) + } +} + +/// Uppercase and other non-canonical spellings are rejected. +fn decode_hex_32(value: &str) -> Result<[u8; 32]> { + let bytes = HEXLOWER + .decode(value.as_bytes()) + .map_err(|error| anyhow!("invalid hex: {error}"))?; + + let decoded: [u8; 32] = bytes + .as_slice() + .try_into() + .map_err(|_| anyhow!("expected 32 bytes, got {}", bytes.len()))?; + + if HEXLOWER.encode(&decoded) != value { + bail!("hex must be lowercase and canonical"); + } + + Ok(decoded) +} -- 2.54.0 From 66f75ad1059c13e6a65717a854690d7535ed11de Mon Sep 17 00:00:00 2001 From: Ren Amamiya Date: Wed, 16 Sep 2026 16:57:04 +0700 Subject: [PATCH 03/12] add basic concord backend --- Cargo.lock | 4 + Cargo.toml | 2 + PLAN.md | 88 +++-- crates/concord/Cargo.toml | 6 + crates/concord/src/lib.rs | 2 + crates/concord/src/store.rs | 156 +++++++++ crates/concord/src/stream.rs | 660 +++++++++++++++++++++++++++++++++++ 7 files changed, 891 insertions(+), 27 deletions(-) create mode 100644 crates/concord/src/store.rs create mode 100644 crates/concord/src/stream.rs diff --git a/Cargo.lock b/Cargo.lock index fc9c70f2..6e03af56 100644 --- a/Cargo.lock +++ b/Cargo.lock @@ -1310,8 +1310,12 @@ dependencies = [ "data-encoding", "hkdf", "nostr", + "nostr-memory", "nostr-sdk", + "rand 0.10.2", + "serde_json", "sha2 0.10.9", + "smol", ] [[package]] diff --git a/Cargo.toml b/Cargo.toml index 3eac7717..39b40fd4 100644 --- a/Cargo.toml +++ b/Cargo.toml @@ -32,6 +32,8 @@ aes-gcm = "0.10" sha2 = "0.10" data-encoding = "2" hkdf = "0.12" +# Pinned to the instance `nostr-sdk` already builds, so NIP-44 nonces share it +rand = { version = "0.10", default-features = false, features = [ "std", "sys_rng" ] } # Others anyhow = "1.0.44" diff --git a/PLAN.md b/PLAN.md index fbda8e66..e40af3c2 100644 --- a/PLAN.md +++ b/PLAN.md @@ -46,7 +46,7 @@ Verified against the pinned revision (`nostr` 0.45.4 / `nostr-sdk` 0.45.2, git ` | NIP-44 under a signer | `nip44::{encrypt, decrypt}` (already wrapped by `state::UniversalSigner`) | | Event id recomputation | `EventId::compute(pubkey, created_at, kind, tags, content)`, `UnsignedEvent::compute_id` | | Event (de)serialization | `Event::{from_json, as_json, verify}`, `UnsignedEvent::from_json` | -| Building events | `EventBuilder::new(..).tags(..).custom_created_at(..)`, `FinalizeEvent::finalize(&Keys)` | +| Building events | `EventBuilder::new(..).tags(..).custom_created_at(..)`, `FinalizeEvent::finalize(&Keys)`; `UnsignedEvent::new(..)` for rumors, whose tags are the author's contract and must not be normalized | | Tags | `Tag::{custom, identifier, public_key, expiration}`, `Tags`, `SingleLetterTag` | | Kinds | `Kind::GiftWrap` (1059), `Kind::Custom(21059|20013|20014|3308|…)`, `Kind::is_ephemeral` | | Publish | `Client::send_event(&event).to(relays).ack_policy(AckPolicy::none())` | @@ -59,16 +59,16 @@ Verified against the pinned revision (`nostr` 0.45.4 / `nostr-sdk` 0.45.2, git ` **Not needed.** `secp256k1` (use `nostr::SecretKey::from_slice` + `Keys::new`), `base64` (use `data_encoding::BASE64`, already a workspace dep), `bech32` (NIP-19 is in the SDK), any new storage engine (the client's LMDB database is enough), any new HTTP client. -**Add one dependency now:** `hkdf = "0.12"` (already in `Cargo.lock` transitively). Add it to `[workspace.dependencies]` and to the new crate. `sha2` is already a workspace dep. **Two more at M8:** the Pin List's per-message key disclosure needs `chacha20 = "0.9"` (already in the tree because we enable nostr's `nip44`, which is where `chacha20` comes from) and `hmac = "0.12"` (already in the tree via `hkdf`) — see §14.8. Zero new crates, all three are direct-dependency lines only. +**Dependencies added so far:** `hkdf = "0.12"` at M0 (already in `Cargo.lock` transitively) and `rand = "0.10"` at M1 for the NIP-44 nonce, pinned to the exact instance `nostr` already builds (`default-features = false`, features `std` + `sys_rng`) so `nostr`'s `os-rng` and ours unify on one `rand`/`getrandom`. `sha2` and `data-encoding` were already workspace deps. **Two more at M8:** the Pin List's per-message key disclosure needs `chacha20 = "0.9"` (already in the tree because we enable nostr's `nip44`, which is where `chacha20` comes from) and `hmac = "0.12"` (already in the tree via `hkdf`) — see §14.8. Zero new crates so far, and all of these are direct-dependency lines only. **Why not depend on Vector's crates.** `vector-core` (published, MIT) holds the only other Rust Concord implementation, in `src/community/v2/*`. It is not reusable as a dependency, and the "reuse their crypto" argument does not hold: - **No shared types.** It exact-pins the nostr family (`nostr = "=0.45.1"`, `nostr-sdk = "=0.45.1"`, `nostr-connect = "=0.45.1"`, `nostr-blossom = "=0.45.0"`) with the note that a caret range would let a consumer resolve a mixed set, while we track git master (`b230cec`, 0.45.4 / 0.45.2). A registry 0.45.1 and a git 0.45.4 cannot unify, so a build linking both carries two `nostr` crates whose `Event`/`Keys`/`PublicKey`/`Client` are unrelated types. - **Not wasm-buildable.** `rusqlite` (bundled C SQLite), `libc`, `rustls`, `reqwest`, `image`, `bip39`, and a `tokio` `net` + `rt-multi-thread` requirement; `VectorCore::init` installs a process-global rustls provider and raises the fd limit. Coop's `web` target is wasm32. - **It is an application core, not a Concord library.** 80k+ lines over 111 files, built on process-global singletons (`state::STATE`, `MY_SECRET_KEY`, one app-data dir, one live account, `traits::set_event_emitter`) and its own SQLite schema, relay pool and blocking `listen()` loop. Adopting it means handing it the nsec and letting it own the client, the database and the event loop — replacing `state`, `chat` and `person` rather than reusing a component. Its `login` stores raw secret-key bytes in that global vault, so an account whose key lives in a signer cannot drive it. -- **There is no cryptography to share.** Both implementations call the same audited crates — `hkdf`, `sha2`, nostr's secp256k1 keypair, and nostr's NIP-44 v2. Vector's comment on that same dependency is "audited RustCrypto crate rather than a hand-rolled construction". Our `derive.rs` has no curve arithmetic, AEAD or randomness of its own: it holds the frozen `info` layout and label table, which are the wire format, not a primitive. +- **There is no cryptography to share.** Both implementations call the same audited crates — `hkdf`, `sha2`, nostr's secp256k1 keypair, and nostr's NIP-44 v2. Vector's comment on that same dependency is "audited RustCrypto crate rather than a hand-rolled construction". Confirmed in M1 by reading `community/cipher.rs`: it is a ~20-line wrapper that draws an OS nonce, calls `nostr::nip44::v2::encrypt_to_bytes_with_nonce`, and base64s the result — which is precisely what `stream.rs` does. Their `stream.rs` likewise calls `nostr_sdk::prelude::nip44::v2::{decrypt_to_bytes, ConversationKey}` directly. Our `derive.rs` holds the frozen `info` layout and label table, and `stream.rs` the seal/wrap ordering; both are wire format, not primitives. -So Vector's crates earn their place as an **oracle, not a dependency**: the golden vectors in `derive.rs` are their published data, produced by an independent implementation. +So Vector's crates earn their place as an **oracle, not a dependency**: the golden vectors in `derive.rs` are their published data, and their `community/v2/stream.rs` was diffed against our §7 before the codec was written. It agrees on every wire detail, and contributed the `ms` first-wins rule, the Control Plane's no-`ms` rumor shape and the `rewrap_seal` contract. ## 4. Crate layout @@ -91,7 +91,9 @@ crates/concord/ `Community` and `Channel` GPUI entities live in `src/lib.rs` next to the registry — they are the public surface, not a separate concern. Ten modules, each with real content; no single-fn files. -Dependencies: `common`, `state`, `person`, `device`, `settings`, `gpui`, `nostr` (for `nip44` features), `nostr-sdk`, `hkdf`, `sha2`, `serde`, `serde_json`, `anyhow`, `flume`, `smallvec`, `itertools`, `futures`, `log`, `instant`. Everything under `cfg(not(target_arch = "wasm32"))` follows the `chat`/`state` split so the crate still builds for web. +Dependencies: `common`, `state`, `person`, `device`, `settings`, `gpui`, `nostr` (for `nip44` features), `nostr-sdk`, `hkdf`, `sha2`, `data-encoding`, `rand`, `serde`, `serde_json`, `anyhow`, `flume`, `smallvec`, `itertools`, `futures`, `log`, `instant`. Everything under `cfg(not(target_arch = "wasm32"))` follows the `chat`/`state` split so the crate still builds for web. + +Declare only what a milestone actually uses. As of M1 the crate depends on `nostr`, `nostr-sdk`, `hkdf`, `sha2`, `data-encoding`, `rand`, `anyhow` (plus `nostr-memory`, `serde_json`, `smol` for tests). `rand` is pinned to the `0.10.2` instance `nostr` already builds and shares its `getrandom`, which the `web` crate already enables `wasm_js` on — so no new package and no new wasm obligation. ## 5. Core types @@ -185,7 +187,7 @@ Rules that must be enforced by construction, not by convention: **Golden vectors.** `derive.rs` pins all 18 published vectors (the seed and `pk` for channel, control, control-signer and guestbook; both keyed labels at epoch `0` and at `0x0102030405060708`; both rekey labels at epoch 1; dissolved; all four locators; the invite key; the community id; the epoch-key commitment), cross-checked against an independent Python implementation (RFC 5869 HKDF plus pure-integer secp256k1) before being frozen. One vector is missing upstream — `pins_locator` — so we mint it from our own implementation and pin it, flagged in the test as self-referential. Changing any pinned value means the wire format changed. -## 7. Stream codec (`stream.rs`) +## 7. Stream codec (`stream.rs`) — implemented in M1 ```rust pub const KIND_WRAP: u16 = 1059; @@ -194,29 +196,56 @@ pub const KIND_SEAL_ENCRYPTED: u16 = 20013; pub const KIND_SEAL_PLAINTEXT: u16 = 20014; pub const NIP44_MAX_PLAINTEXT: usize = 65_535; +pub enum SealForm { Encrypted, Plaintext } + +pub struct OpenedStream { + pub rumor_id: EventId, + pub author: PublicKey, + pub seal_form: SealForm, + pub seal: Event, + pub wrapper_id: EventId, + pub at_ms: u64, + pub rumor: UnsignedEvent, +} + +pub fn split_ms(at_ms: u64) -> (u64, u16); +pub fn resolve_ms_strict(rumor: &UnsignedEvent) -> Result; + pub fn seal_content(rumor: &UnsignedEvent, form: SealForm, group: &GroupKey) -> Result; pub fn build_seal(rumor: &UnsignedEvent, form: SealForm, group: &GroupKey, author: &Keys) -> Result; pub fn wrap_seal(seal: &Event, group: &GroupKey, wrap_kind: u16, at: Timestamp, extra: &[Tag]) -> Result<(Event, Keys), StreamError>; +pub fn rewrap_seal(seal: &Event, new_group: &GroupKey, at: Timestamp) -> Result<(Event, Keys), StreamError>; pub fn open_wrap(wrap: &Event, group: &GroupKey) -> Result; -pub fn open_wrap_at(wrap: &Event, address: &PublicKey, conversation: &ConversationKey, verify_wrap_sig: bool) -> Result; +pub fn open_wrap_at(wrap: &Event, address: &PublicKey, conversation: &ConversationKey, verify_wrap_signature: bool) -> Result; + +pub fn build_rumor_ms(kind: u16, author: PublicKey, content: &str, tags: Vec, at_ms: u64) -> UnsignedEvent; +pub fn build_rumor_secs(kind: u16, author: PublicKey, content: &str, tags: Vec, at_secs: u64) -> UnsignedEvent; pub fn channel_binding_tags(channel: &ChannelId, epoch: Epoch) -> Vec; pub fn check_channel_binding(rumor: &UnsignedEvent, channel: &ChannelId, epoch: Epoch) -> Result<(), StreamError>; -pub fn build_rumor(kind: u16, author: PublicKey, content: &str, tags: Vec, at_ms: u64) -> UnsignedEvent; // appends ["ms", n] -pub fn resolve_ms_strict(rumor: &UnsignedEvent) -> Result; ``` +`StreamError` is a typed enum, not `anyhow`: the caller has to tell a drop from a fatal, and M1's acceptance criterion is that rejections happen in the documented order. + +Refinements against the draft this plan opened with, decided after diffing Vector's `crates/vector-core/src/community/v2/stream.rs` (Concord has no crate of its own there, and `envelope.rs` does not exist): + +- `build_rumor` became `build_rumor_ms` plus `build_rumor_secs`. The Control Plane edition carries **no** `ms` tag, because editions fold by version, not by time. +- `rewrap_seal` was added to the codec. Without it the plaintext-seal carry-forward has no expression, and M7's compaction is its only caller. +- NIP-44 is reached through `nostr`'s own `nip44::v2::{encrypt_to_bytes_with_nonce, decrypt_to_bytes, ConversationKey}`, with a fresh OS nonce per message and `data_encoding::BASE64` for carriage. This is exactly what Vector does; there is no cryptography of theirs to reuse. + Design points that are easy to get wrong: - The wrap is signed by the **stream key** with a random ephemeral `p` tag — NIP-59 reversed. `extra` is how the caller mirrors a NIP-40 expiration onto the wrap. - The seal is signed by the **real author** and carries `created_at` equal to the rumor's. It is never published bare. - Control plane **must** use the plaintext seal; chat, guestbook and rekey planes **must** use the encrypted one. Each plane asserts its own form at both ends. - The control plane is a write-restricted stream: the wrap key derives from `control_root` while the content is encrypted under the `community_root`-derived conversation key. `open_wrap_at` takes the two halves separately for this reason. -- Open order: kind → address match → wrap signature (only when `verify_wrap_sig`) → NIP-44 open → seal kind → seal signature → rumor parse → `rumor.pubkey == seal.pubkey` → recompute the rumor id and reject a mismatch → strict `ms` resolve. +- Open order: kind → address match → wrap signature (only when `verify_wrap_signature`) → NIP-44 open → seal kind → seal signature → rumor parse → `rumor.pubkey == seal.pubkey` → recompute the rumor id and reject a mismatch → strict `ms` resolve. - Enforce the 65 535-byte NIP-44 cap at every nesting layer before publishing. - Do not verify the wrap signature for ordinary planes: every reader holds the group key, so the signature proves nothing. It matters only for the restricted control plane and for rekeys. - The ephemeral wrap keypair is returned to the caller so a client may NIP-09-scrub its own wrap later. +- **A duplicate `ms` tag takes the first value; it is not rejected.** Rejecting made Vector and Armada disagree on whether the event *exists*, and because `ms` orders messages that divergence reached membership. `ms` is the publisher's own value, so conceding a second tag grants an attacker no reach a single one did not. A present-but-valueless `ms`, or one that is not a lone canonical decimal in `0..=999`, is `BadMs` and the event is dropped, never clamped — `u64::from_str` alone would accept a leading `+`, a second encoding a strict peer rejects, so the digit check comes first. +- A binding tag that names the same key twice is rejected outright, since first-match would then be the reader's choice rather than the author's; a valueless tag counts as absent, so a true absence reports `MissingTag`. ## 8. Planes, state and folds @@ -373,14 +402,26 @@ pub fn compact(fold, epoch, new_control_root, ...) -> Vec; // re-wrap h Dissolution (CORD-02 §9) also lives here: a chainless, owner-signed `vsk 10` tombstone at `dissolved_group_key(id)`, plaintext-sealed, and a verifier **must** refuse any tombstone whose `eid` is not the community's own id (including the all-zero placeholder — accepting it lets an owner's genuine tombstone for one community be re-wrapped at another of theirs and kill it permanently). On sight the community is sealed read-only: subscriptions halt, nothing new is honored, existing history stays readable, and a member's delete of their own message is still honored. -## 9. Storage (`store.rs`) +## 9. Storage (`store.rs`) — local layer implemented in M1 Three layers, no new storage engine: 1. **Raw wraps** (kind 1059) are persisted automatically by the SDK's relay pool when a subscription or fetch matches a filter. Nothing to write. 2. **Opened rumors** are cached locally as NIP-78 `Kind::ApplicationSpecificData` events signed by a session-local keypair, exactly like `chat::set_rumor`. Tags: `["d", rumor_id]` (replace key), `["c", channel_hex]`, `["p", author]`, `["k", kind]`, `["e", wrap_id]`, `["t", "concord"]`. Contents are the rumor JSON. - The `c`/`t` keys deliberately differ from chat's `r` key so the two message namespaces can never collide in one database. - - The read path dedupes by rumor id and keeps the newest `created_at`, because the local signing key changes per session. + - `created_at` is the **message's own second** (from `at_ms`), not the wall clock. Otherwise `until` and the ordering would page on cache time rather than message time. + - The read path dedupes by rumor id and keeps the newest `created_at`, because the local signing key changes per session and each session leaves its own copy. The query therefore carries no filter `limit` — every copy has to be in hand before they can be collapsed — and the cap is applied to the deduplicated result instead. + - The layer takes `&dyn NostrDatabase`, not `&Client`: it is local-only, which keeps it testable without a relay or a GPUI context. + +```rust +pub async fn cache_rumor(database: &dyn NostrDatabase, channel: &ChannelId, opened: &OpenedStream) -> Result<()>; +pub async fn query_rumors(database: &dyn NostrDatabase, channel: &ChannelId, until: Option, limit: usize) -> Result>; +``` + +`query_rumors` returns `UnsignedEvent`, not `Event`: the cached payload *is* a rumor, which is also what `OpenedStream` carries, so the caller never has to re-parse. + +**Deferred to M4:** the relay-paging `backfill`. It is network history paging whose "step past the same-second wall" policy belongs with the sync engine, and M1 has no subscription to test it against. + 3. **Community state** — one local document per community, `Kind::ApplicationSpecificData` with `["d", "concord/"]`: ```rust @@ -404,19 +445,10 @@ pub struct CommunityState { } ``` -Writes are debounced (a fold head changes on every edition); reads load once at init. +Writes are debounced (a fold head changes on every edition); reads load once at init. This layer lands in M2, once `Community` exists to hold it. **Decision, stated for the record:** this document stores raw community keys unencrypted in a plaintext local database. That matches the existing posture — `chat` already caches decrypted message rumors in the same LMDB. If that posture ever changes, the state document is the one to wrap with NIP-44-to-self, since it is the only local artifact holding keys. -History queries: - -```rust -pub async fn query_messages(&self, channel: &ChannelId, until: Option, limit: usize) -> Result, Error>; -pub async fn backfill(&self, plane_authors: &[PublicKey], relays: &[RelayUrl], until: Option, limit: usize) -> Result, Error>; -``` - -`query_messages` reads the local cache (`Filter::new().kind(ApplicationSpecificData).custom_tag(LOWERCASE_C, channel_hex)`); `backfill` pages relays newest-first with `until`, deduplicating by wrap id and stepping past same-second walls. - ## 10. Sync engine and GPUI conventions `ConcordRegistry` mirrors `ChatRegistry`'s shape exactly: a foreground GPUI entity holding `Entity` handles, a `flume` signal bus, one background notification listener, one foreground consumer, and task slots that are cleared when the signer changes. @@ -537,7 +569,7 @@ pub fn pin(&self, id: EventId, cx: &App) -> Task>; // vsk 11, ## 11. Integration with existing crates -1. **`crates/chat/src/lib.rs` — required fix.** `handle_notifications` currently treats *every* kind 1059 event as a NIP-17 wrap for the current user and pushes an unwrap failure into the trash. Concord wraps are kind 1059 with an ephemeral `p` tag, so they would flood the trash and leak error toasts. Route by `subscription_id` from `RelayMessage::Event` against `sub_id1`/`sub_id2`, and drop the `if rumor.tags.is_empty()` heuristic once the real recipient check is in place. +1. **`crates/chat/src/lib.rs` — required fix, applied in M2.** `handle_notifications` currently treats *every* kind 1059 event as a NIP-59 gift wrap for the current user and pushes an unwrap failure into the trash. Concord wraps are kind 1059 with an ephemeral `p` tag, so they would flood the trash and leak error toasts. Route by `subscription_id` from `RelayMessage::Event` against `sub_id1`/`sub_id2`, and drop the `if rumor.tags.is_empty()` recipient heuristic. M1 did not need it because nothing subscribes yet. 2. **`desktop/src/main.rs` and `web/src/lib.rs`** — add `concord::init(window, cx)` after `chat::init(window, cx)`. 3. **`Cargo.toml`** — add `hkdf = "0.12"` to `[workspace.dependencies]`; add the crate to `desktop` and `web` dependencies. No other workspace changes. 4. **No changes** to `state`, `person`, `device`, `settings`, `common`, or `ui`. @@ -550,7 +582,7 @@ Each of these has burned a real implementation, or is a documented cross-client - Require `rumor.pubkey == seal.pubkey`. - Require the plaintext seal form on Control and the encrypted form on Chat/Guestbook/Rekey — a strict reader must drop a mis-sealed edition rather than fold a chain a later compaction would fork. - Check `channel` **and** `epoch` against the plane whose key opened the wrap; reject duplicates of either tag. -- Reject duplicate `vsk`/`eid`/`ev`/`ep`/`vac` tags; require decimal-with-no-leading-zeros on every numeric tag. +- Reject duplicate `vsk`/`eid`/`ev`/`ep`/`vac` tags; require decimal-with-no-leading-zeros on every numeric tag. The one exception is `ms`, which takes its first value rather than erroring — see §7 for why rejecting it reached membership. - Refuse a tombstone whose `eid` is not this community's id. - Adopt a `control_root` from a Grant only if it derives to the `control_pk` held for that epoch; adopt a rekey blob only if its bound plaintext matches the scope and epoch and its `prevcommit` matches the key currently held. - Never conclude removal from a partial rekey chunk set. @@ -564,11 +596,11 @@ Each of these has burned a real implementation, or is a documented cross-client | # | Deliverable | Done when | | --- | --- | --- | -| M0 | Crate skeleton, `derive.rs`, golden vectors, workspace wiring | `cargo test -p concord` pins every derivation from an independent vector set; all labels match Appendix A.6 | -| M1 | `stream.rs` + `store.rs` | seal/wrap/open round-trips for both seal forms; malformed inputs rejected in the documented order; local cache reads back after a restart | +| M0 | Crate skeleton, `derive.rs`, golden vectors, workspace wiring | ✅ `cargo test -p concord` pins every derivation; all labels match Appendix A.6 | +| M1 | `stream.rs` + `store.rs` | ✅ seal/wrap/open round-trips for both seal forms; hostile wraps rejected in the documented order; the local cache reads back with the group key gone | | M2 | `edition.rs` + `control.rs` genesis | a community is created and published; its two genesis wraps open at a second client sharing the keys; edition hash matches the cross-client vector | | M3 | Control fold + roster + metadata/channels | fold tests for chains, gaps, downgrade refusal, fork tiebreak, compaction dangle; metadata and channel edits visible to a second client | -| M4 | Chat plane | send/receive/edit/delete/reaction across two identities; history pages backwards across an epoch boundary; binding checks reject a foreign channel/epoch | +| M4 | Chat plane | send/receive/edit/delete/reaction across two identities; history pages backwards across an epoch boundary (relay `backfill` lands here); binding checks reject a foreign channel/epoch | | M5 | Guestbook + member list + moderation | join/leave/kick/ban converge to the same memberlist on both clients; every authority rule from §8.1 has a negative test | | M6 | Invites + Community List | link mint → fetch → join round-trips; revoked link refuses; direct invite lands in the recipient's giftwrap inbox via the `k` tag; a second device reconstructs membership from 13302 | | M7 | Rekeys + refounding + dissolution | a removed member stops reading after a rekey; continuity and race rules tested; a tombstone seals the community and a foreign-id tombstone is refused | @@ -576,6 +608,8 @@ Each of these has burned a real implementation, or is a documented cross-client Ordering is deliberately dependency-first: each milestone is usable on its own, and nothing in M2+ depends on a later milestone. +M1 closed with `cargo test -p concord` (5 tests), `cargo clippy -p concord --all-targets`, and `cargo fmt -p concord --check` all clean. `rand` was added to the workspace pinned to the same `0.10.2` instance `nostr` already builds, so `Cargo.lock` gained no package. + ## 14. Open questions and risks 1. **Community List kind.** CORD-02 §8 specifies `13302`, replaceable. Vector has retired it in favour of fragmented `33302`, because a replaceable kind holds one event per pubkey and so cannot shard past the NIP-44 size cap. We implement `13302` per spec, enforce the 50-membership cap and pre-publish size check, and treat `33302` as an interop follow-up. Confirm with Armada before writing the multi-device code. diff --git a/crates/concord/Cargo.toml b/crates/concord/Cargo.toml index dbc2af0a..cb663be4 100644 --- a/crates/concord/Cargo.toml +++ b/crates/concord/Cargo.toml @@ -11,4 +11,10 @@ nostr-sdk.workspace = true hkdf.workspace = true sha2.workspace = true data-encoding.workspace = true +rand.workspace = true anyhow.workspace = true + +[dev-dependencies] +nostr-memory.workspace = true +serde_json.workspace = true +smol.workspace = true diff --git a/crates/concord/src/lib.rs b/crates/concord/src/lib.rs index 0a6dc121..82b4110e 100644 --- a/crates/concord/src/lib.rs +++ b/crates/concord/src/lib.rs @@ -1,4 +1,6 @@ pub mod derive; +pub mod store; +pub mod stream; use std::fmt; use std::str::FromStr; diff --git a/crates/concord/src/store.rs b/crates/concord/src/store.rs new file mode 100644 index 00000000..85ec7dc6 --- /dev/null +++ b/crates/concord/src/store.rs @@ -0,0 +1,156 @@ +use std::collections::BTreeMap; +use std::sync::LazyLock; + +use anyhow::{Result, anyhow}; +use nostr_sdk::prelude::*; + +use crate::ChannelId; +use crate::stream::OpenedStream; + +static LOCAL_KEYS: LazyLock = LazyLock::new(Keys::generate); + +const CHANNEL_TAG: SingleLetterTag = SingleLetterTag::LOWERCASE_C; +const MARK_TAG: SingleLetterTag = SingleLetterTag::LOWERCASE_T; +const MARK_VALUE: &str = "concord"; +const WRAP_TAG: &str = "e"; +const KIND_TAG: &str = "k"; + +pub async fn cache_rumor( + database: &dyn NostrDatabase, + channel: &ChannelId, + opened: &OpenedStream, +) -> Result<()> { + let tags = vec![ + Tag::identifier(opened.rumor_id), + Tag::custom(KIND_TAG, [opened.rumor.kind.to_string()]), + Tag::custom(WRAP_TAG, [opened.wrapper_id.to_string()]), + Tag::custom(MARK_TAG.as_str(), [MARK_VALUE]), + Tag::custom(CHANNEL_TAG.as_str(), [channel.to_hex()]), + Tag::public_key(opened.author), + ]; + let at = Timestamp::from_secs(opened.at_ms / 1000); + let event = EventBuilder::new(Kind::ApplicationSpecificData, opened.rumor.as_json()) + .tags(tags) + .custom_created_at(at) + .finalize_async(&*LOCAL_KEYS) + .await?; + + database.save_event(&event).await?; + + Ok(()) +} + +/// Read a channel's cached rumors. +pub async fn query_rumors( + database: &dyn NostrDatabase, + channel: &ChannelId, + until: Option, + limit: usize, +) -> Result> { + let mut filter = Filter::new() + .kind(Kind::ApplicationSpecificData) + .custom_tag(MARK_TAG, MARK_VALUE) + .custom_tag(CHANNEL_TAG, channel.to_hex()); + + if let Some(until) = until { + filter = filter.until(until); + } + + let mut newest: BTreeMap = BTreeMap::new(); + for event in database.query(filter).await? { + let Some(rumor_id) = event.tags.identifier() else { + continue; + }; + + match newest.get(&rumor_id) { + Some(existing) if existing.created_at >= event.created_at => {} + _ => { + newest.insert(rumor_id, event); + } + } + } + + let mut events: Vec = newest.into_values().collect(); + events.sort_by_key(|event| std::cmp::Reverse(event.created_at)); + events.truncate(limit); + + let mut rumors = Vec::with_capacity(events.len()); + for event in events { + let rumor = UnsignedEvent::from_json(event.content) + .map_err(|error| anyhow!("cached rumor is not a valid event: {error}"))?; + rumors.push(rumor); + } + + Ok(rumors) +} + +#[cfg(test)] +mod tests { + use nostr_memory::MemoryDatabase; + + use super::*; + use crate::Epoch; + use crate::derive::channel_group_key; + use crate::stream::{ + KIND_WRAP, SealForm, build_rumor_ms, build_seal, channel_binding_tags, open_wrap, wrap_seal, + }; + + const SECRET: [u8; 32] = [0x07u8; 32]; + + #[test] + fn rumors_read_back_after_a_restart() { + let database = MemoryDatabase::unbounded(); + let channel = ChannelId::from_bytes([0xabu8; 32]); + let author = Keys::generate(); + + smol::block_on(async { + let group = channel_group_key(&SECRET, &channel, Epoch(0)).expect("derives"); + + for (content, at_ms) in [("first", 1_000_000u64), ("second", 2_000_000)] { + let rumor = build_rumor_ms( + 9, + author.public_key(), + content, + channel_binding_tags(&channel, Epoch(0)), + at_ms, + ); + let seal = build_seal(&rumor, SealForm::Encrypted, &group, &author).expect("seals"); + let (wrap, _) = wrap_seal( + &seal, + &group, + KIND_WRAP, + Timestamp::from_secs(at_ms / 1000), + &[], + ) + .expect("wraps"); + + let opened = open_wrap(&wrap, &group).expect("opens"); + cache_rumor(&database, &channel, &opened) + .await + .expect("caches"); + } + + // The group key is gone; only the local cache stands in for it. + let rumors = query_rumors(&database, &channel, None, 10) + .await + .expect("queries"); + assert_eq!(rumors.len(), 2, "both messages come back"); + assert_eq!(rumors[0].content, "second", "newest first"); + assert_eq!(rumors[1].content, "first"); + + // A page boundary in message time, not in cache time. + let until = Timestamp::from_secs(1_500); + let page = query_rumors(&database, &channel, Some(until), 10) + .await + .expect("queries"); + assert_eq!(page.len(), 1); + assert_eq!(page[0].content, "first"); + + let capped = query_rumors(&database, &channel, None, 1) + .await + .expect("queries"); + assert_eq!(capped.len(), 1); + assert_eq!(capped[0].content, "second"); + }); + } +} diff --git a/crates/concord/src/stream.rs b/crates/concord/src/stream.rs new file mode 100644 index 00000000..040f77e4 --- /dev/null +++ b/crates/concord/src/stream.rs @@ -0,0 +1,660 @@ +use std::fmt; + +use data_encoding::BASE64; +use nostr::nips::nip44::v2::{ConversationKey, decrypt_to_bytes, encrypt_to_bytes_with_nonce}; +use nostr_sdk::prelude::{ + Event, EventBuilder, EventId, FinalizeEvent, Keys, Kind, PublicKey, Tag, Timestamp, + UnsignedEvent, +}; +use rand::TryRng as _; +use rand::rngs::SysRng; + +use crate::derive::GroupKey; +use crate::{ChannelId, Epoch}; + +pub const KIND_WRAP: u16 = 1059; +pub const KIND_WRAP_EPHEMERAL: u16 = 21059; +pub const KIND_SEAL_ENCRYPTED: u16 = 20013; +pub const KIND_SEAL_PLAINTEXT: u16 = 20014; +pub const NIP44_MAX_PLAINTEXT: usize = 65_535; + +const TAG_MS: &str = "ms"; +const TAG_CHANNEL: &str = "channel"; +const TAG_EPOCH: &str = "epoch"; + +#[derive(Debug, Clone, Copy, PartialEq, Eq)] +pub enum SealForm { + Encrypted, + Plaintext, +} + +impl SealForm { + pub fn kind(self) -> u16 { + match self { + SealForm::Encrypted => KIND_SEAL_ENCRYPTED, + SealForm::Plaintext => KIND_SEAL_PLAINTEXT, + } + } + + fn from_kind(kind: u16) -> Option { + match kind { + KIND_SEAL_ENCRYPTED => Some(SealForm::Encrypted), + KIND_SEAL_PLAINTEXT => Some(SealForm::Plaintext), + _ => None, + } + } +} + +#[derive(Debug)] +pub enum StreamError { + Sign(String), + Encrypt(String), + Decrypt(String), + Parse(String), + Oversize(usize), + BadWrapKind(u16), + WrongStream, + BadWrapSignature, + BadSealKind(u16), + BadSealSignature, + AuthorMismatch, + BadRumorId, + BadMs, + ChannelMismatch, + EpochMismatch, + MissingTag(&'static str), + DuplicateTag(&'static str), + NotRewrappable, +} + +impl fmt::Display for StreamError { + fn fmt(&self, f: &mut fmt::Formatter<'_>) -> fmt::Result { + match self { + StreamError::Sign(error) => write!(f, "sign: {error}"), + StreamError::Encrypt(error) => write!(f, "encrypt: {error}"), + StreamError::Decrypt(error) => write!(f, "decrypt: {error}"), + StreamError::Parse(error) => write!(f, "parse: {error}"), + StreamError::Oversize(len) => write!(f, "plaintext {len} bytes exceeds NIP-44 cap"), + StreamError::BadWrapKind(kind) => write!(f, "not a wrap kind: {kind}"), + StreamError::WrongStream => write!(f, "wrap author is not this stream"), + StreamError::BadWrapSignature => write!(f, "restricted wrap signature invalid"), + StreamError::BadSealKind(kind) => write!(f, "not a seal kind: {kind}"), + StreamError::BadSealSignature => write!(f, "seal signature invalid"), + StreamError::AuthorMismatch => write!(f, "rumor pubkey != seal pubkey"), + StreamError::BadRumorId => write!(f, "rumor id != computed hash"), + StreamError::BadMs => write!(f, "ms is not a canonical decimal in 0..=999"), + StreamError::ChannelMismatch => write!(f, "channel binding mismatch"), + StreamError::EpochMismatch => write!(f, "epoch binding mismatch"), + StreamError::MissingTag(name) => write!(f, "missing rumor tag: {name}"), + StreamError::DuplicateTag(name) => write!(f, "duplicate rumor tag: {name}"), + StreamError::NotRewrappable => write!(f, "only plaintext seals survive re-wrapping"), + } + } +} + +impl std::error::Error for StreamError {} + +#[derive(Debug, Clone)] +pub struct OpenedStream { + pub rumor_id: EventId, + pub author: PublicKey, + pub seal_form: SealForm, + pub seal: Event, + pub wrapper_id: EventId, + pub at_ms: u64, + pub rumor: UnsignedEvent, +} + +pub fn split_ms(at_ms: u64) -> (u64, u16) { + (at_ms / 1000, (at_ms % 1000) as u16) +} + +/// Build a rumor carrying a full epoch-ms time: `created_at` +/// holds the seconds and an `["ms", 0..=999]` tag the remainder. +pub fn build_rumor_ms( + kind: u16, + author: PublicKey, + content: &str, + mut tags: Vec, + at_ms: u64, +) -> UnsignedEvent { + let (seconds, offset) = split_ms(at_ms); + tags.push(Tag::custom(TAG_MS, [offset.to_string()])); + build_rumor_secs(kind, author, content, tags, seconds) +} + +/// Build a rumor with a plain seconds timestamp and no `ms` tag. +pub fn build_rumor_secs( + kind: u16, + author: PublicKey, + content: &str, + tags: Vec, + at_secs: u64, +) -> UnsignedEvent { + let mut rumor = UnsignedEvent::new( + author, + Timestamp::from_secs(at_secs), + Kind::Custom(kind), + tags, + content, + ); + rumor.ensure_id(); + rumor +} + +/// Resolve a rumor's true millisecond time. +pub fn resolve_ms_strict(rumor: &UnsignedEvent) -> Result { + let seconds = rumor.created_at.as_secs().saturating_mul(1000); + let mut tag: Option> = None; + + for candidate in rumor.tags.iter() { + let fields = candidate.as_slice(); + if fields.first().map(String::as_str) == Some(TAG_MS) { + tag = Some(fields.get(1).cloned()); + break; + } + } + + let Some(raw) = tag else { + return Ok(seconds); + }; + let raw = raw.ok_or(StreamError::BadMs)?; + + if raw.is_empty() || !raw.bytes().all(|byte| byte.is_ascii_digit()) { + return Err(StreamError::BadMs); + } + + let offset: u64 = raw.parse().map_err(|_| StreamError::BadMs)?; + + if offset > 999 || (raw.len() > 1 && raw.starts_with('0')) { + return Err(StreamError::BadMs); + } + + Ok(seconds.saturating_add(offset)) +} + +pub fn seal_content( + rumor: &UnsignedEvent, + form: SealForm, + group: &GroupKey, +) -> Result { + let json = rumor.as_json(); + check_plaintext_cap(json.len())?; + + match form { + SealForm::Plaintext => Ok(json), + SealForm::Encrypted => Ok(BASE64.encode(&encrypt(group.conversation(), json.as_bytes())?)), + } +} + +pub fn build_seal( + rumor: &UnsignedEvent, + form: SealForm, + group: &GroupKey, + author: &Keys, +) -> Result { + let content = seal_content(rumor, form, group)?; + EventBuilder::new(Kind::Custom(form.kind()), content) + .custom_created_at(rumor.created_at) + .finalize(author) + .map_err(|error| StreamError::Sign(error.to_string())) +} + +pub fn wrap_seal( + seal: &Event, + group: &GroupKey, + wrap_kind: u16, + at: Timestamp, + extra: &[Tag], +) -> Result<(Event, Keys), StreamError> { + if wrap_kind != KIND_WRAP && wrap_kind != KIND_WRAP_EPHEMERAL { + return Err(StreamError::BadWrapKind(wrap_kind)); + } + + let json = seal.as_json(); + check_plaintext_cap(json.len())?; + + let content = BASE64.encode(&encrypt(group.conversation(), json.as_bytes())?); + let ephemeral = Keys::generate(); + + let mut tags = vec![Tag::public_key(ephemeral.public_key())]; + tags.extend_from_slice(extra); + + let wrap = EventBuilder::new(Kind::Custom(wrap_kind), content) + .tags(tags) + .custom_created_at(at) + .finalize(group.keys()) + .map_err(|error| StreamError::Sign(error.to_string()))?; + + Ok((wrap, ephemeral)) +} + +pub fn rewrap_seal( + seal: &Event, + new_group: &GroupKey, + at: Timestamp, +) -> Result<(Event, Keys), StreamError> { + if seal.kind.as_u16() != KIND_SEAL_PLAINTEXT { + return Err(StreamError::NotRewrappable); + } + wrap_seal(seal, new_group, KIND_WRAP, at, &[]) +} + +pub fn open_wrap(wrap: &Event, group: &GroupKey) -> Result { + open_wrap_at(wrap, &group.pk(), group.conversation(), false) +} + +/// Open and verify a wrap against a stream read view: the address to check and +/// the conversation key that opens the wraps, with no signing secret required. +pub fn open_wrap_at( + wrap: &Event, + address: &PublicKey, + conversation: &ConversationKey, + verify_wrap_signature: bool, +) -> Result { + let wrap_kind = wrap.kind.as_u16(); + + if wrap_kind != KIND_WRAP && wrap_kind != KIND_WRAP_EPHEMERAL { + return Err(StreamError::BadWrapKind(wrap_kind)); + } + + if wrap.pubkey != *address { + return Err(StreamError::WrongStream); + } + + if verify_wrap_signature && wrap.verify().is_err() { + return Err(StreamError::BadWrapSignature); + } + + let seal: Event = Event::from_json(decode_content(conversation, &wrap.content)?) + .map_err(|error| StreamError::Parse(error.to_string()))?; + let seal_kind = seal.kind.as_u16(); + let seal_form = SealForm::from_kind(seal_kind).ok_or(StreamError::BadSealKind(seal_kind))?; + seal.verify().map_err(|_| StreamError::BadSealSignature)?; + + let rumor_json = match seal_form { + SealForm::Plaintext => seal.content.clone(), + SealForm::Encrypted => decode_content(conversation, &seal.content)?, + }; + + let mut rumor: UnsignedEvent = UnsignedEvent::from_json(rumor_json.as_bytes()) + .map_err(|error| StreamError::Parse(error.to_string()))?; + + if rumor.pubkey != seal.pubkey { + return Err(StreamError::AuthorMismatch); + } + + let computed = rumor.compute_id(); + if let Some(claimed) = rumor.id + && claimed != computed + { + return Err(StreamError::BadRumorId); + } + rumor.id = Some(computed); + + let at_ms = resolve_ms_strict(&rumor)?; + + Ok(OpenedStream { + rumor_id: computed, + author: seal.pubkey, + seal_form, + seal, + wrapper_id: wrap.id, + at_ms, + rumor, + }) +} + +pub fn channel_binding_tags(channel: &ChannelId, epoch: Epoch) -> Vec { + vec![ + Tag::custom(TAG_CHANNEL, [channel.to_hex()]), + Tag::custom(TAG_EPOCH, [epoch.0.to_string()]), + ] +} + +pub fn check_channel_binding( + rumor: &UnsignedEvent, + channel: &ChannelId, + epoch: Epoch, +) -> Result<(), StreamError> { + match unique_tag(rumor, TAG_CHANNEL)? { + Some(value) if value == channel.to_hex() => {} + Some(_) => return Err(StreamError::ChannelMismatch), + None => return Err(StreamError::MissingTag(TAG_CHANNEL)), + } + + match unique_tag(rumor, TAG_EPOCH)? { + Some(value) if value == epoch.0.to_string() => {} + Some(_) => return Err(StreamError::EpochMismatch), + None => return Err(StreamError::MissingTag(TAG_EPOCH)), + } + + Ok(()) +} + +fn encrypt(conversation: &ConversationKey, plaintext: &[u8]) -> Result, StreamError> { + let mut nonce = [0u8; 32]; + + SysRng + .try_fill_bytes(&mut nonce) + .map_err(|error| StreamError::Encrypt(error.to_string()))?; + + encrypt_to_bytes_with_nonce(conversation, plaintext, nonce) + .map_err(|error| StreamError::Encrypt(error.to_string())) +} + +fn decode_content(conversation: &ConversationKey, content: &str) -> Result { + let payload = BASE64 + .decode(content.as_bytes()) + .map_err(|error| StreamError::Decrypt(error.to_string()))?; + + let plaintext = decrypt_to_bytes(conversation, &payload) + .map_err(|error| StreamError::Decrypt(error.to_string()))?; + + String::from_utf8(plaintext).map_err(|error| StreamError::Parse(error.to_string())) +} + +fn check_plaintext_cap(len: usize) -> Result<(), StreamError> { + if len > NIP44_MAX_PLAINTEXT { + return Err(StreamError::Oversize(len)); + } + + Ok(()) +} + +fn unique_tag(rumor: &UnsignedEvent, name: &'static str) -> Result, StreamError> { + let mut found: Option = None; + + for tag in rumor.tags.iter() { + let fields = tag.as_slice(); + if fields.len() >= 2 && fields[0] == name { + if found.is_some() { + return Err(StreamError::DuplicateTag(name)); + } + found = Some(fields[1].clone()); + } + } + + Ok(found) +} + +#[cfg(test)] +mod tests { + use super::*; + use crate::derive::channel_group_key; + + const SECRET: [u8; 32] = [0x07u8; 32]; + const OTHER_SECRET: [u8; 32] = [0x08u8; 32]; + + fn channel() -> ChannelId { + ChannelId::from_bytes([0xabu8; 32]) + } + + fn group(epoch: u64) -> GroupKey { + channel_group_key(&SECRET, &channel(), Epoch(epoch)).expect("derives") + } + + fn wrapper_p_tag(wrap: &Event) -> Option { + wrap.tags + .iter() + .find(|tag| tag.as_slice().first().map(String::as_str) == Some("p")) + .and_then(|tag| tag.as_slice().get(1).cloned()) + } + + fn bound_rumor(content: &str, author: PublicKey, at_ms: u64) -> UnsignedEvent { + build_rumor_ms( + 9, + author, + content, + channel_binding_tags(&channel(), Epoch(0)), + at_ms, + ) + } + + fn sealed(rumor: &UnsignedEvent, form: SealForm, author: &Keys) -> Event { + build_seal(rumor, form, &group(0), author).expect("seals") + } + + fn wrapped(seal: &Event, kind: u16, at_secs: u64) -> Event { + wrap_seal(seal, &group(0), kind, Timestamp::from_secs(at_secs), &[]) + .expect("wraps") + .0 + } + + fn encrypted_wrap(content: &str, author: &Keys, at_ms: u64, kind: u16) -> Event { + let rumor = bound_rumor(content, author.public_key(), at_ms); + wrapped( + &sealed(&rumor, SealForm::Encrypted, author), + kind, + at_ms / 1000, + ) + } + + #[test] + fn both_seal_forms_round_trip() { + let author = Keys::generate(); + let at_ms = 1_686_840_217_417; + let wrap = encrypted_wrap("Hey chat!", &author, at_ms, KIND_WRAP); + + assert_eq!(wrap.kind, Kind::GiftWrap, "the durable wrap is kind 1059"); + assert_eq!(wrap.pubkey, group(0).pk(), "the stream key signs the wrap"); + + let opened = open_wrap(&wrap, &group(0)).expect("opens"); + assert_eq!(opened.author, author.public_key()); + assert_eq!(opened.rumor.content, "Hey chat!"); + assert_eq!(opened.rumor_id, opened.rumor.id.expect("id is set")); + assert_eq!(opened.wrapper_id, wrap.id); + assert_eq!(opened.at_ms, at_ms); + assert_eq!(opened.seal_form, SealForm::Encrypted); + check_channel_binding(&opened.rumor, &channel(), Epoch(0)).expect("binding holds"); + + // The wrap's `p` tag must identify neither the stream nor the author. + let p = wrapper_p_tag(&wrap).expect("the wrap carries a p tag"); + assert_ne!(p, group(0).pk_hex()); + assert_ne!(p, author.public_key().to_hex()); + + // Ephemeral actions ride the same structure at a kind relays must drop. + let typing = encrypted_wrap("typing", &author, 5_000, KIND_WRAP_EPHEMERAL); + assert_eq!(typing.kind.as_u16(), 21059); + assert_eq!( + open_wrap(&typing, &group(0)).expect("opens").rumor.content, + "typing" + ); + + // The plaintext form carries the rumor's bytes verbatim, which is what + // lets a compaction re-wrap the signed edition into a later epoch. + let edition = build_rumor_secs( + 3308, + author.public_key(), + "an edition", + vec![], + 1_700_000_000, + ); + let seal = sealed(&edition, SealForm::Plaintext, &author); + assert_eq!(seal.content, edition.as_json(), "the rumor rides verbatim"); + + let opened = open_wrap(&wrapped(&seal, KIND_WRAP, 1), &group(0)).expect("opens"); + assert_eq!(opened.seal_form, SealForm::Plaintext); + + let (rewrapped, _) = + rewrap_seal(&opened.seal, &group(1), Timestamp::from_secs(2)).expect("rewraps"); + let reopened = open_wrap(&rewrapped, &group(1)).expect("opens"); + assert_eq!(reopened.rumor_id, opened.rumor_id, "the rumor id survives"); + assert_eq!(reopened.author, author.public_key()); + assert_eq!( + reopened.seal.sig, opened.seal.sig, + "the signature rides whole" + ); + assert_ne!(reopened.wrapper_id, opened.wrapper_id); + + assert!(matches!( + rewrap_seal( + &sealed(&edition, SealForm::Encrypted, &author), + &group(1), + Timestamp::from_secs(2) + ), + Err(StreamError::NotRewrappable) + )); + } + + #[test] + fn hostile_wraps_are_dropped_in_order() { + let author = Keys::generate(); + let impostor = Keys::generate(); + + // Kind and address are settled before any decryption is attempted. + let mut wrong_kind = encrypted_wrap("x", &author, 1_000, KIND_WRAP); + wrong_kind.kind = Kind::Custom(1058); + assert!(matches!( + open_wrap(&wrong_kind, &group(0)), + Err(StreamError::BadWrapKind(1058)) + )); + + let foreign = channel_group_key(&OTHER_SECRET, &channel(), Epoch(0)).expect("derives"); + let wrap = encrypted_wrap("x", &author, 1_000, KIND_WRAP); + assert!(matches!( + open_wrap(&wrap, &foreign), + Err(StreamError::WrongStream) + )); + + // A flipped ciphertext byte fails the NIP-44 MAC. + let mut payload = BASE64 + .decode(wrap.content.as_bytes()) + .expect("content is base64"); + payload[40] ^= 0x01; + let mut tampered = wrap.clone(); + tampered.content = BASE64.encode(&payload); + assert!(matches!( + open_wrap(&tampered, &group(0)), + Err(StreamError::Decrypt(_)) + )); + + // A seal claiming an author it holds no signature for. + let seal = sealed( + &bound_rumor("spoof", author.public_key(), 1_000), + SealForm::Encrypted, + &impostor, + ); + let mut swapped: serde_json::Value = serde_json::from_str(&seal.as_json()).expect("json"); + swapped["pubkey"] = serde_json::Value::String(author.public_key().to_hex()); + let seal = Event::from_json(swapped.to_string()).expect("a swapped pubkey still parses"); + assert!(matches!( + open_wrap(&wrapped(&seal, KIND_WRAP, 1), &group(0)), + Err(StreamError::BadSealSignature) + )); + + // A seal that does not vouch for the rumor's author. + let seal = sealed( + &bound_rumor("spoof", impostor.public_key(), 1_000), + SealForm::Encrypted, + &author, + ); + assert!(matches!( + open_wrap(&wrapped(&seal, KIND_WRAP, 1), &group(0)), + Err(StreamError::AuthorMismatch) + )); + + // A claimed id the rumor's own bytes do not hash to. The plaintext seal + // smuggles the forgery through verbatim. + let rumor = bound_rumor("real", author.public_key(), 1_000); + let mut forged: serde_json::Value = serde_json::from_str(&rumor.as_json()).expect("json"); + forged["id"] = serde_json::Value::String("00".repeat(32)); + let seal = EventBuilder::new(Kind::Custom(KIND_SEAL_PLAINTEXT), forged.to_string()) + .custom_created_at(rumor.created_at) + .finalize(&author) + .expect("seals"); + assert!(matches!( + open_wrap(&wrapped(&seal, KIND_WRAP, 1), &group(0)), + Err(StreamError::BadRumorId) + )); + + // Binding splices: another channel, another epoch, a duplicate or none. + let doubled = vec![channel_binding_tags(&channel(), Epoch(0)); 2].concat(); + let rumor = bound_rumor("x", author.public_key(), 1_000); + + assert!(matches!( + check_channel_binding(&rumor, &ChannelId::from_bytes([0xcdu8; 32]), Epoch(0)), + Err(StreamError::ChannelMismatch) + )); + + assert!(matches!( + check_channel_binding(&rumor, &channel(), Epoch(1)), + Err(StreamError::EpochMismatch) + )); + + let duplicate = build_rumor_ms(9, author.public_key(), "x", doubled, 1_000); + assert!(matches!( + check_channel_binding(&duplicate, &channel(), Epoch(0)), + Err(StreamError::DuplicateTag(_)) + )); + + let unbound = build_rumor_ms(9, author.public_key(), "x", vec![], 1_000); + assert!(matches!( + check_channel_binding(&unbound, &channel(), Epoch(0)), + Err(StreamError::MissingTag(_)) + )); + + let oversize = build_rumor_ms( + 9, + author.public_key(), + &"x".repeat(NIP44_MAX_PLAINTEXT + 1), + vec![], + 1_000, + ); + assert!(matches!( + seal_content(&oversize, SealForm::Encrypted, &group(0)), + Err(StreamError::Oversize(_)) + )); + } + + #[test] + fn ms_is_a_drop_gate() { + let author = Keys::generate(); + + let absent = build_rumor_secs(9, author.public_key(), "x", vec![], 1_000); + assert_eq!(resolve_ms_strict(&absent).expect("resolves"), 1_000_000); + + let highest = build_rumor_ms(9, author.public_key(), "x", vec![], 1_000_999); + assert_eq!(resolve_ms_strict(&highest).expect("resolves"), 1_000_999); + + for malformed in ["1000", "007", "abc", "+5", ""] { + let rumor = build_rumor_secs( + 9, + author.public_key(), + "x", + vec![Tag::custom(TAG_MS, [malformed.to_string()])], + 1_000, + ); + assert!( + matches!(resolve_ms_strict(&rumor), Err(StreamError::BadMs)), + "{malformed:?} must be malformed" + ); + } + + // Present but valueless is malformed, not an offset-0 default. + let valueless = build_rumor_secs( + 9, + author.public_key(), + "x", + vec![Tag::custom(TAG_MS, Vec::::new())], + 1_000, + ); + assert!(matches!( + resolve_ms_strict(&valueless), + Err(StreamError::BadMs) + )); + + // A valued duplicate takes the first, matching Armada. + let repeated = build_rumor_secs( + 9, + author.public_key(), + "x", + vec![ + Tag::custom(TAG_MS, ["1".to_string()]), + Tag::custom(TAG_MS, ["2".to_string()]), + ], + 1_000, + ); + assert_eq!(resolve_ms_strict(&repeated).expect("resolves"), 1_000_001); + } +} -- 2.54.0 From d926c1e3ea441a9af38649acffa057a992b8985c Mon Sep 17 00:00:00 2001 From: Ren Amamiya Date: Wed, 16 Sep 2026 17:35:33 +0700 Subject: [PATCH 04/12] update concord backend --- Cargo.lock | 1 + PLAN.md | 52 +++-- crates/concord/Cargo.toml | 3 +- crates/concord/src/control.rs | 300 ++++++++++++++++++++++++++ crates/concord/src/edition.rs | 392 ++++++++++++++++++++++++++++++++++ crates/concord/src/lib.rs | 36 +++- crates/concord/src/store.rs | 147 ++++++++++++- crates/concord/src/stream.rs | 29 ++- 8 files changed, 929 insertions(+), 31 deletions(-) create mode 100644 crates/concord/src/control.rs create mode 100644 crates/concord/src/edition.rs diff --git a/Cargo.lock b/Cargo.lock index 6e03af56..de364b01 100644 --- a/Cargo.lock +++ b/Cargo.lock @@ -1313,6 +1313,7 @@ dependencies = [ "nostr-memory", "nostr-sdk", "rand 0.10.2", + "serde", "serde_json", "sha2 0.10.9", "smol", diff --git a/PLAN.md b/PLAN.md index e40af3c2..ec6529b6 100644 --- a/PLAN.md +++ b/PLAN.md @@ -93,7 +93,7 @@ crates/concord/ Dependencies: `common`, `state`, `person`, `device`, `settings`, `gpui`, `nostr` (for `nip44` features), `nostr-sdk`, `hkdf`, `sha2`, `data-encoding`, `rand`, `serde`, `serde_json`, `anyhow`, `flume`, `smallvec`, `itertools`, `futures`, `log`, `instant`. Everything under `cfg(not(target_arch = "wasm32"))` follows the `chat`/`state` split so the crate still builds for web. -Declare only what a milestone actually uses. As of M1 the crate depends on `nostr`, `nostr-sdk`, `hkdf`, `sha2`, `data-encoding`, `rand`, `anyhow` (plus `nostr-memory`, `serde_json`, `smol` for tests). `rand` is pinned to the `0.10.2` instance `nostr` already builds and shares its `getrandom`, which the `web` crate already enables `wasm_js` on — so no new package and no new wasm obligation. +Declare only what a milestone actually uses. As of M2 the crate depends on `nostr`, `nostr-sdk`, `hkdf`, `sha2`, `data-encoding`, `rand`, `serde`, `serde_json`, `anyhow` (plus `nostr-memory` and `smol` for tests). `rand` is pinned to the `0.10.2` instance `nostr` already builds and shares its `getrandom`, which the `web` crate already enables `wasm_js` on — so no new package and no new wasm obligation. `serde`/`serde_json` were already in the graph via `nostr`; promoting `serde_json` from dev to main for the metadata content types added no package either, only the `concord → serde` edge. ## 5. Core types @@ -252,22 +252,33 @@ Design points that are easy to get wrong: ### 8.1 Editions and authority (`edition.rs`, `control.rs`) ```rust -pub const EDITION_LABEL: &[u8] = b"vector-community/v1/edition"; // frozen, cross-client +pub const EDITION_LABEL: &[u8] = b"vector-community/v1/edition"; // frozen, cross-client (27 bytes) +// sha256( u64be(len(label)) ‖ label ‖ entity[32] ‖ u64be(version) +// ‖ flag[1] ‖ prev[32] ‖ u64be(len(content)) ‖ content ) +// `prev` is always 33 bytes: 0x01 ‖ hash, or 0x00 ‖ zeroes when absent. +// The hash commits to no actor: identity enters only via the rumor id. pub fn edition_hash(entity: &[u8; 32], version: u64, prev: Option<&[u8; 32]>, content: &[u8]) -> [u8; 32]; -pub struct ParsedEdition { author: PublicKey, vsk: String, entity: [u8; 32], version: u64, - prev: Option<[u8; 32]>, content: String, self_hash: [u8; 32] }; +pub struct ParsedEdition { author: PublicKey, subkind: String, entity: [u8; 32], version: u64, + prev: Option<[u8; 32]>, citation: Option, + content: String, self_hash: [u8; 32], rumor_id: EventId }; pub fn parse_edition(rumor: &UnsignedEvent) -> Result; pub struct FoldResult { pub head: Option, pub gap: bool, pub anchored: bool } pub fn fold(editions: &[EditionMeta], floor: u64, floor_hash: Option<&[u8; 32]>) -> FoldResult; -pub fn bootstrap_head(editions: &[EditionMeta], floor: u64) -> Option; +pub fn bootstrap_head(editions: &[EditionMeta]) -> Option; // highest, contiguity ignored ``` -- Tag grammar: `["vsk", sub]`, `["eid", hex32]`, `["ev", decimal]`, `["ep", hex32]`, `["vac", eid, version, hash]`. Duplicates of any of the five reject the edition; `ev` must pass a decimal check before parsing. -- Tie-break at equal version is the lower **inner rumor id**, never `created_at`. -- `gap` is a safety signal: a tracking client (already holds the floor) fails closed for that entity and refetches; a bootstrapping client (floor 0) may accept the highest authority-verified head, which is what makes compaction re-wrapping work. -- Entity coordinates are `vsk 0` → `community_id`, `1` → `role_id`, `2` → `channel_id`, `3` → `grant_locator`, `4` → `banlist_locator`, `8` → `invite_links_locator`, `11` → `pins_locator`. All derive from `community_id` only, so a refounding re-wraps heads verbatim. +- Tag grammar: `["vsk", sub]`, `["eid", hex32]`, `["ev", decimal]`, `["ep", hex32]`, `["vac", eid, version, hash]`. Duplicates of any of the five reject the edition; `ev` must pass a decimal check before parsing. A version of `0` parses and then reads as a gap — the rule lives in the fold, not the parser. +- **Versions start at 1, not 0** (CORD-04 §1: "climbs from 1"). Genesis is `(version 1, prev None)` for both entities. +- **The edition hash is not the signature.** The actor's Schnorr signature covers the kind-20014 plaintext seal; `edition_hash` is a separate SHA-256 used only for chaining (`ep`, `vac`). `content` is the rumor's content string byte-verbatim, never re-serialized, which is what lets compaction re-wrap a head and preserve its hash. +- The domain label is `vector-community/v1/edition`, not a `concord/…` label. Inconsistent with Appendix A.6, frozen anyway — do not "fix" it. +- Tie-break at equal version is the lower **inner rumor id** (the kind-3308 rumor), never the outer wrap id and never `created_at`. Only one of the two implementations that must agree applies to a wrap, so the inner id is the only stable choice. +- `gap` is a safety signal: a tracking client (already holds the floor) fails closed for that entity and refetches; a bootstrapping client (floor 0) may accept the highest authority-verified head, which is what makes compaction re-wrapping work. `bootstrap_head` therefore takes no floor: it *is* the floor-zero path. +- **Owner anchoring is not in the fold.** `fold` is a pure function of chain shape; authority is a pre-filter the caller applies to the candidate set before folding. `community_id` proves the owner, and `is_authorized` short-circuits `owner == actor`, so the owner needs no Grant entity at all. +- Entity coordinates are `vsk 0` → `community_id`, `1` → `role_id`, `2` → `channel_id`, `3` → `grant_locator`, `4` → `banlist_locator`, `8` → `invite_links_locator`, `11` → `pins_locator`. `5` is reserved, `6`/`9` belong to the 33301 invite marker, `7` is retired. All derive from `community_id` only, so a refounding re-wraps heads verbatim. +- The Control Plane is **plaintext-seal only**. A 20013-encrypted control edition is rejected, because compaction re-wraps a signed plaintext seal byte-verbatim into the new epoch; accepting an encrypted one would let a later compaction fork the chain. +- **Genesis is exactly two owner-signed editions** — community metadata (`vsk 0`, `eid = community_id`) and one public `#general` channel (`vsk 2`, fresh random `channel_id`) — at epoch 0, version 1, no `ep`, no `vac`. No default roles, no scaffolding, and no Grant for the owner. Secrets minted: `owner_salt`, `community_root`, `control_root` (deliberately not derived from `community_id`). ```rust pub const P_MANAGE_ROLES: u64 = 1 << 0; // …bit table from CORD-04 §3, frozen; retired bits are burned @@ -432,20 +443,17 @@ pub struct CommunityState { pub community_root: [u8; 32], pub root_epoch: Epoch, pub control_root: Option<[u8; 32]>, // present iff the holder is staff - pub control_pks: BTreeMap, - pub channels: Vec, // id, key, epoch, name, private - pub epoch_keys: Vec<([u8; 32], Epoch, [u8; 32])>, // (scope, epoch, key) — the history backfill index + pub control_pks: BTreeMap, // epoch → the plane's signer address + pub channels: Vec, // id, name, private, epoch pub relays: Vec, - pub heads: BTreeMap<[u8; 32], (u64, [u8; 32], EventId)>, // entity → (version, self_hash, inner id) - pub guestbook: Vec, - pub observed: BTreeMap, - pub banned: BTreeSet, - pub dissolved: bool, + pub heads: Vec, // entity, version, self_hash, inner id pub added_at_ms: u64, } ``` -Writes are debounced (a fold head changes on every edition); reads load once at init. This layer lands in M2, once `Community` exists to hold it. +Landed in M2 with exactly the fields genesis can populate: `save_state`/`load_state` and `CommunityState::from_genesis`. Three fields the plan sketched are deliberately absent until something can fill them — `epoch_keys` (needs rekeys, M7), and `guestbook`/`observed`/`banned`/`dissolved` (need the guestbook, M5). `control_pks` keyed by `u64` rather than `Epoch` and `heads` as a `Vec` rather than a `BTreeMap<[u8; 32], _>`, because serde_json cannot use a byte-array map key. + +Writes are debounced (a fold head changes on every edition); reads load once at init. **Decision, stated for the record:** this document stores raw community keys unencrypted in a plaintext local database. That matches the existing posture — `chat` already caches decrypted message rumors in the same LMDB. If that posture ever changes, the state document is the one to wrap with NIP-44-to-self, since it is the only local artifact holding keys. @@ -569,7 +577,7 @@ pub fn pin(&self, id: EventId, cx: &App) -> Task>; // vsk 11, ## 11. Integration with existing crates -1. **`crates/chat/src/lib.rs` — required fix, applied in M2.** `handle_notifications` currently treats *every* kind 1059 event as a NIP-59 gift wrap for the current user and pushes an unwrap failure into the trash. Concord wraps are kind 1059 with an ephemeral `p` tag, so they would flood the trash and leak error toasts. Route by `subscription_id` from `RelayMessage::Event` against `sub_id1`/`sub_id2`, and drop the `if rumor.tags.is_empty()` recipient heuristic. M1 did not need it because nothing subscribes yet. +1. **`crates/chat/src/lib.rs` — required fix, moved from M2 to the milestone that first subscribes.** `handle_notifications` currently treats *every* kind 1059 event as a NIP-59 gift wrap for the current user and pushes an unwrap failure into the trash. Concord wraps are kind 1059 with an ephemeral `p` tag, so they would flood the trash and leak error toasts. Route by `subscription_id` from `RelayMessage::Event` against `sub_id1`/`sub_id2`, and drop the `if rumor.tags.is_empty()` recipient heuristic. **M2 did not apply it**: the crate has no subscription and no `ConcordRegistry` yet, so no concord wrap can reach that handler and the change would be untestable. It lands with the sync engine (§10), as does the `concord::init` wiring in `desktop` and `web`. 2. **`desktop/src/main.rs` and `web/src/lib.rs`** — add `concord::init(window, cx)` after `chat::init(window, cx)`. 3. **`Cargo.toml`** — add `hkdf = "0.12"` to `[workspace.dependencies]`; add the crate to `desktop` and `web` dependencies. No other workspace changes. 4. **No changes** to `state`, `person`, `device`, `settings`, `common`, or `ui`. @@ -598,7 +606,7 @@ Each of these has burned a real implementation, or is a documented cross-client | --- | --- | --- | | M0 | Crate skeleton, `derive.rs`, golden vectors, workspace wiring | ✅ `cargo test -p concord` pins every derivation; all labels match Appendix A.6 | | M1 | `stream.rs` + `store.rs` | ✅ seal/wrap/open round-trips for both seal forms; hostile wraps rejected in the documented order; the local cache reads back with the group key gone | -| M2 | `edition.rs` + `control.rs` genesis | a community is created and published; its two genesis wraps open at a second client sharing the keys; edition hash matches the cross-client vector | +| M2 | `edition.rs` + `control.rs` genesis + `store.rs` state document | ✅ `cargo test -p concord` (7 tests): `edition_hash` reproduces the cross-client vector `2daf42e6…`, and a community minted by one holder has both genesis wraps open for a second holder holding only the invite keys, folding to version 1 | | M3 | Control fold + roster + metadata/channels | fold tests for chains, gaps, downgrade refusal, fork tiebreak, compaction dangle; metadata and channel edits visible to a second client | | M4 | Chat plane | send/receive/edit/delete/reaction across two identities; history pages backwards across an epoch boundary (relay `backfill` lands here); binding checks reject a foreign channel/epoch | | M5 | Guestbook + member list + moderation | join/leave/kick/ban converge to the same memberlist on both clients; every authority rule from §8.1 has a negative test | @@ -610,6 +618,10 @@ Ordering is deliberately dependency-first: each milestone is usable on its own, M1 closed with `cargo test -p concord` (5 tests), `cargo clippy -p concord --all-targets`, and `cargo fmt -p concord --check` all clean. `rand` was added to the workspace pinned to the same `0.10.2` instance `nostr` already builds, so `Cargo.lock` gained no package. +M2 closed the same way at 7 tests, with `serde` added to the crate's dependencies (`serde_json` promoted from dev to main for the metadata content types) — `Cargo.lock` gained no package again, only the `concord → serde` edge. + +**M2's "created and published" is verified offline**: "published" is the two wraps existing and being openable by the invite keys, not a relay round-trip. There is no registry to publish through until §10, and a relay test would be testing the SDK, not the protocol. + ## 14. Open questions and risks 1. **Community List kind.** CORD-02 §8 specifies `13302`, replaceable. Vector has retired it in favour of fragmented `33302`, because a replaceable kind holds one event per pubkey and so cannot shard past the NIP-44 size cap. We implement `13302` per spec, enforce the 50-membership cap and pre-publish size check, and treat `33302` as an interop follow-up. Confirm with Armada before writing the multi-device code. diff --git a/crates/concord/Cargo.toml b/crates/concord/Cargo.toml index cb663be4..29d21505 100644 --- a/crates/concord/Cargo.toml +++ b/crates/concord/Cargo.toml @@ -12,9 +12,10 @@ hkdf.workspace = true sha2.workspace = true data-encoding.workspace = true rand.workspace = true +serde.workspace = true +serde_json.workspace = true anyhow.workspace = true [dev-dependencies] nostr-memory.workspace = true -serde_json.workspace = true smol.workspace = true diff --git a/crates/concord/src/control.rs b/crates/concord/src/control.rs new file mode 100644 index 00000000..70daa75e --- /dev/null +++ b/crates/concord/src/control.rs @@ -0,0 +1,300 @@ +use anyhow::{Result, bail}; +use nostr_sdk::prelude::{Event, Keys, PublicKey, Timestamp, UnsignedEvent}; +use serde::{Deserialize, Serialize}; +use serde_json::{Map, Value}; + +use crate::derive::{ + community_id_of, control_group_key, control_signer_group_key, verify_community_id, +}; +use crate::edition::{EditionFields, ParsedEdition, build_edition, parse_edition, vsk}; +use crate::stream::{KIND_WRAP, SealForm, build_seal, open_wrap_at, wrap_seal_with}; +use crate::{ChannelId, CommunityId, Epoch, GroupKey, random_32}; + +pub const MAX_NAME_BYTES: usize = 64; +pub const MAX_DESCRIPTION_BYTES: usize = 10_000; +pub const MAX_RELAYS: usize = 5; + +pub const GENERAL_CHANNEL: &str = "general"; +pub const ROOT_EPOCH: Epoch = Epoch(0); + +/// The first edition every entity starts at. Genuinely 1, not 0: a version of +/// 0 is what the fold treats as a gap. +const GENESIS_VERSION: u64 = 1; + +type Extra = Map; + +#[derive(Debug, Clone, Default, PartialEq, Serialize, Deserialize)] +pub struct ImageRef { + pub url: String, + pub key: String, + pub nonce: String, + pub hash: String, + #[serde(flatten)] + pub extra: Extra, +} + +#[derive(Debug, Clone, Default, PartialEq, Serialize, Deserialize)] +pub struct CommunityMetadata { + pub name: String, + #[serde(default, skip_serializing_if = "Option::is_none")] + pub description: Option, + #[serde(default, skip_serializing_if = "Vec::is_empty")] + pub relays: Vec, + #[serde(default, skip_serializing_if = "Option::is_none")] + pub icon: Option, + #[serde(default, skip_serializing_if = "Option::is_none")] + pub banner: Option, + #[serde(default, skip_serializing_if = "Option::is_none")] + pub custom: Option, + #[serde(flatten)] + pub extra: Extra, +} + +#[derive(Debug, Clone, Default, PartialEq, Serialize, Deserialize)] +pub struct ChannelMetadata { + pub name: String, + pub private: bool, + #[serde(default, skip_serializing_if = "Option::is_none")] + pub voice: Option, + #[serde(default, skip_serializing_if = "Option::is_none")] + pub deleted: Option, + #[serde(default, skip_serializing_if = "Option::is_none")] + pub custom: Option, + #[serde(flatten)] + pub extra: Extra, +} + +/// A community's permanent identity. +#[derive(Debug, Clone, PartialEq, Eq)] +pub struct CommunityIdentity { + pub community_id: CommunityId, + pub owner: PublicKey, + pub owner_salt: [u8; 32], +} + +impl CommunityIdentity { + pub fn verify(&self) -> bool { + verify_community_id(&self.community_id, &self.owner.to_bytes(), &self.owner_salt) + } +} + +/// Everything a creation mints: +/// +/// - Identity +/// - Roots an invite will carry +/// - First channel +/// - Two genesis wraps to publish +#[derive(Debug, Clone)] +pub struct CommunityGenesis { + pub identity: CommunityIdentity, + pub community_root: [u8; 32], + pub control_root: [u8; 32], + pub channel_id: ChannelId, + pub wraps: Vec, +} + +/// Mints a community and signs its two genesis editions. +pub fn genesis( + owner: &Keys, + metadata: &CommunityMetadata, + at_secs: u64, +) -> Result { + let mut metadata = metadata.clone(); + + if metadata.name.len() > MAX_NAME_BYTES { + bail!("community name exceeds {MAX_NAME_BYTES} bytes"); + } + + if metadata + .description + .as_ref() + .is_some_and(|description| description.len() > MAX_DESCRIPTION_BYTES) + { + bail!("community description exceeds {MAX_DESCRIPTION_BYTES} bytes"); + } + + metadata.relays.truncate(MAX_RELAYS); + + let owner_salt = random_32()?; + let identity = CommunityIdentity { + community_id: community_id_of(&owner.public_key().to_bytes(), &owner_salt), + owner: owner.public_key(), + owner_salt, + }; + + let community_root = random_32()?; + let control_root = random_32()?; + let channel_id = ChannelId::from_bytes(random_32()?); + + let read = control_group_key(&community_root, &identity.community_id, ROOT_EPOCH)?; + let signer = control_signer_group_key(&control_root, &identity.community_id, ROOT_EPOCH)?; + + let metadata_content = serde_json::to_string(&metadata)?; + let channel_content = serde_json::to_string(&ChannelMetadata { + name: GENERAL_CHANNEL.to_owned(), + private: false, + ..ChannelMetadata::default() + })?; + + let editions = [ + build_edition(EditionFields { + author: identity.owner, + subkind: vsk::COMMUNITY_METADATA, + entity: *identity.community_id.as_bytes(), + version: GENESIS_VERSION, + prev: None, + citation: None, + content: &metadata_content, + at_secs, + }), + build_edition(EditionFields { + author: identity.owner, + subkind: vsk::CHANNEL_METADATA, + entity: *channel_id.as_bytes(), + version: GENESIS_VERSION, + prev: None, + citation: None, + content: &channel_content, + at_secs, + }), + ]; + + let mut wraps = Vec::with_capacity(editions.len()); + + for edition in &editions { + wraps.push(seal_edition(edition, owner, &read, &signer, at_secs)?); + } + + Ok(CommunityGenesis { + identity, + community_root, + control_root, + channel_id, + wraps, + }) +} + +/// Opens a Control Plane wrap from its reading key alone. +pub fn open_edition( + wrap: &Event, + read: &GroupKey, + address: &PublicKey, + verify_wrap_signature: bool, +) -> Result { + let opened = open_wrap_at(wrap, address, read.conversation(), verify_wrap_signature)?; + + if opened.seal_form != SealForm::Plaintext { + bail!("control editions require a plaintext seal"); + } + + Ok(parse_edition(&opened.rumor)?) +} + +fn seal_edition( + edition: &UnsignedEvent, + owner: &Keys, + read: &GroupKey, + signer: &GroupKey, + at_secs: u64, +) -> Result { + let seal = build_seal(edition, SealForm::Plaintext, read, owner)?; + + let (wrap, _) = wrap_seal_with( + &seal, + read.conversation(), + signer.keys(), + KIND_WRAP, + Timestamp::from_secs(at_secs), + &[], + )?; + + Ok(wrap) +} + +#[cfg(test)] +mod tests { + use nostr_memory::MemoryDatabase; + + use super::*; + use crate::edition::{EditionMeta, fold}; + use crate::store::{CommunityState, load_state, save_state}; + + #[test] + fn genesis_reopens_for_a_second_holder() { + let owner = Keys::generate(); + let metadata = CommunityMetadata { + name: "coop".to_owned(), + relays: vec!["wss://relay.example".to_owned()], + ..CommunityMetadata::default() + }; + let at_secs = 1_700_000_000; + + let minted = genesis(&owner, &metadata, at_secs).expect("mints"); + assert!(minted.identity.verify(), "identity is self-certifying"); + + // The second client holds only what an invite hands over: the roots, + // the community id and the owner salt. + let read = control_group_key( + &minted.community_root, + &minted.identity.community_id, + ROOT_EPOCH, + ) + .expect("derives"); + let address = control_signer_group_key( + &minted.control_root, + &minted.identity.community_id, + ROOT_EPOCH, + ) + .expect("derives") + .pk(); + + let mut editions = Vec::new(); + for wrap in &minted.wraps { + editions.push(open_edition(wrap, &read, &address, true).expect("opens")); + } + + assert_eq!(editions.len(), 2); + + let community = &editions[0]; + assert_eq!(community.subkind, vsk::COMMUNITY_METADATA); + assert_eq!(community.entity, *minted.identity.community_id.as_bytes()); + assert_eq!(community.author, owner.public_key()); + assert_eq!((community.version, community.prev), (1, None)); + assert_eq!( + serde_json::from_str::(&community.content) + .expect("parses") + .name, + "coop" + ); + + let channel = &editions[1]; + assert_eq!(channel.subkind, vsk::CHANNEL_METADATA); + assert_eq!(channel.entity, *minted.channel_id.as_bytes()); + + for edition in &editions { + let folded = fold(&[EditionMeta::from(edition)], 0, None); + assert_eq!(folded.head, Some(0)); + assert!( + folded.anchored && !folded.gap, + "genesis anchors at its floor" + ); + } + + let state = + CommunityState::from_genesis(&minted, &editions, at_secs * 1_000).expect("projects"); + + smol::block_on(async { + let database = MemoryDatabase::unbounded(); + save_state(&database, &state).await.expect("saves"); + let loaded = load_state(&database, &minted.identity.community_id) + .await + .expect("loads") + .expect("present"); + + assert_eq!(loaded.community_root, minted.community_root); + assert_eq!(loaded.control_root, Some(minted.control_root)); + assert_eq!(loaded.channels.len(), 1); + assert_eq!(loaded.heads.len(), 2); + }); + } +} diff --git a/crates/concord/src/edition.rs b/crates/concord/src/edition.rs new file mode 100644 index 00000000..a758b6cd --- /dev/null +++ b/crates/concord/src/edition.rs @@ -0,0 +1,392 @@ +use std::collections::BTreeMap; +use std::fmt; + +use data_encoding::HEXLOWER; +use nostr_sdk::prelude::{EventId, PublicKey, Tag, UnsignedEvent}; +use sha2::{Digest, Sha256}; + +use crate::decode_hex_32; +use crate::stream::build_rumor_secs; + +pub const KIND_CONTROL: u16 = 3308; + +const EDITION_LABEL: &[u8] = b"vector-community/v1/edition"; + +/// Entity types an edition can address (CORD-02 Appendix B). +pub mod vsk { + pub const COMMUNITY_METADATA: &str = "0"; + pub const ROLE: &str = "1"; + pub const CHANNEL_METADATA: &str = "2"; + pub const GRANT: &str = "3"; + pub const BANLIST: &str = "4"; + pub const INVITE_LIVE: &str = "6"; + pub const INVITE_LINKS: &str = "8"; + pub const INVITE_REVOKED: &str = "9"; + pub const DISSOLVED: &str = "10"; + pub const PINS: &str = "11"; +} + +const TAG_SUBKIND: &str = "vsk"; +const TAG_ENTITY: &str = "eid"; +const TAG_VERSION: &str = "ev"; +const TAG_PREV: &str = "ep"; +const TAG_CITATION: &str = "vac"; + +#[derive(Debug)] +pub enum EditionError { + BadKind(u16), + BadField(&'static str), + Duplicate(&'static str), + Missing(&'static str), +} + +impl fmt::Display for EditionError { + fn fmt(&self, f: &mut fmt::Formatter<'_>) -> fmt::Result { + match self { + EditionError::BadKind(kind) => write!(f, "not an edition kind: {kind}"), + EditionError::BadField(name) => write!(f, "malformed edition field: {name}"), + EditionError::Duplicate(name) => write!(f, "duplicate edition field: {name}"), + EditionError::Missing(name) => write!(f, "missing edition field: {name}"), + } + } +} + +impl std::error::Error for EditionError {} + +/// A `vac` citation: the Grant edition an actor claims rank under, pinned by +/// coordinate, version and hash. It is a sync floor, not the verdict. +#[derive(Debug, Clone, Copy, PartialEq, Eq)] +pub struct AuthorityCitation { + pub entity: [u8; 32], + pub version: u64, + pub hash: [u8; 32], +} + +#[derive(Debug, Clone)] +pub struct ParsedEdition { + pub author: PublicKey, + pub subkind: String, + pub entity: [u8; 32], + pub version: u64, + pub prev: Option<[u8; 32]>, + pub citation: Option, + pub content: String, + pub self_hash: [u8; 32], + pub rumor_id: EventId, +} + +pub struct EditionFields<'a> { + pub author: PublicKey, + pub subkind: &'a str, + pub entity: [u8; 32], + pub version: u64, + pub prev: Option<[u8; 32]>, + pub citation: Option, + pub content: &'a str, + pub at_secs: u64, +} + +fn signing_bytes( + entity: &[u8; 32], + version: u64, + prev: Option<&[u8; 32]>, + content: &[u8], +) -> Vec { + let mut bytes = + Vec::with_capacity(8 + EDITION_LABEL.len() + 32 + 8 + 1 + 32 + 8 + content.len()); + + bytes.extend_from_slice(&(EDITION_LABEL.len() as u64).to_be_bytes()); + bytes.extend_from_slice(EDITION_LABEL); + bytes.extend_from_slice(entity); + bytes.extend_from_slice(&version.to_be_bytes()); + + match prev { + Some(prev) => { + bytes.push(1); + bytes.extend_from_slice(prev); + } + None => { + bytes.push(0); + bytes.extend_from_slice(&[0u8; 32]); + } + } + + bytes.extend_from_slice(&(content.len() as u64).to_be_bytes()); + bytes.extend_from_slice(content); + bytes +} + +pub fn edition_hash( + entity: &[u8; 32], + version: u64, + prev: Option<&[u8; 32]>, + content: &[u8], +) -> [u8; 32] { + Sha256::digest(signing_bytes(entity, version, prev, content)).into() +} + +pub fn build_edition(fields: EditionFields<'_>) -> UnsignedEvent { + let mut tags = vec![ + Tag::custom(TAG_SUBKIND, [fields.subkind]), + Tag::custom(TAG_ENTITY, [HEXLOWER.encode(&fields.entity)]), + Tag::custom(TAG_VERSION, [fields.version.to_string()]), + ]; + + if let Some(prev) = fields.prev { + tags.push(Tag::custom(TAG_PREV, [HEXLOWER.encode(&prev)])); + } + + if let Some(citation) = fields.citation { + tags.push(Tag::custom( + TAG_CITATION, + [ + HEXLOWER.encode(&citation.entity), + citation.version.to_string(), + HEXLOWER.encode(&citation.hash), + ], + )); + } + + build_rumor_secs( + KIND_CONTROL, + fields.author, + fields.content, + tags, + fields.at_secs, + ) +} + +pub fn parse_edition(rumor: &UnsignedEvent) -> Result { + let kind = rumor.kind.as_u16(); + + if kind != KIND_CONTROL { + return Err(EditionError::BadKind(kind)); + } + + let subkind = value(rumor, TAG_SUBKIND)? + .ok_or(EditionError::Missing(TAG_SUBKIND))? + .to_owned(); + + if canonical_decimal(&subkind).is_none() { + return Err(EditionError::BadField(TAG_SUBKIND)); + } + + let entity = hex32( + value(rumor, TAG_ENTITY)?.ok_or(EditionError::Missing(TAG_ENTITY))?, + TAG_ENTITY, + )?; + + let version = + canonical_decimal(value(rumor, TAG_VERSION)?.ok_or(EditionError::Missing(TAG_VERSION))?) + .ok_or(EditionError::BadField(TAG_VERSION))?; + + let prev = match value(rumor, TAG_PREV)? { + Some(raw) => Some(hex32(raw, TAG_PREV)?), + None => None, + }; + + let citation = match fields(rumor, TAG_CITATION)? { + Some(fields) if fields.len() == 4 => Some(AuthorityCitation { + entity: hex32(&fields[1], TAG_CITATION)?, + version: canonical_decimal(&fields[2]).ok_or(EditionError::BadField(TAG_CITATION))?, + hash: hex32(&fields[3], TAG_CITATION)?, + }), + Some(_) => return Err(EditionError::BadField(TAG_CITATION)), + None => None, + }; + + let self_hash = edition_hash(&entity, version, prev.as_ref(), rumor.content.as_bytes()); + + Ok(ParsedEdition { + author: rumor.pubkey, + subkind, + entity, + version, + prev, + citation, + content: rumor.content.clone(), + self_hash, + rumor_id: rumor.id.unwrap_or_else(|| rumor.compute_id()), + }) +} + +#[derive(Debug, Clone, Copy, PartialEq, Eq)] +pub struct EditionMeta { + pub version: u64, + pub self_hash: [u8; 32], + pub prev: Option<[u8; 32]>, + pub tiebreak_id: EventId, +} + +impl From<&ParsedEdition> for EditionMeta { + fn from(edition: &ParsedEdition) -> Self { + Self { + version: edition.version, + self_hash: edition.self_hash, + prev: edition.prev, + tiebreak_id: edition.rumor_id, + } + } +} + +#[derive(Debug, Clone, Copy, PartialEq, Eq, Default)] +pub struct FoldResult { + pub head: Option, + pub gap: bool, + pub anchored: bool, +} + +/// The highest version whose chain is intact, given a held floor. +pub fn fold(editions: &[EditionMeta], floor: u64, floor_hash: Option<&[u8; 32]>) -> FoldResult { + let mut by_version: BTreeMap = BTreeMap::new(); + + for (index, edition) in editions.iter().enumerate() { + if edition.version < floor { + continue; + } + + match by_version.get(&edition.version) { + Some(¤t) if editions[current].tiebreak_id <= edition.tiebreak_id => {} + _ => { + by_version.insert(edition.version, index); + } + } + } + + let Some((&lowest_version, &lowest_index)) = by_version.first_key_value() else { + return FoldResult::default(); + }; + + let lowest = editions[lowest_index]; + + let anchored = if floor == 0 { + lowest_version == 1 && lowest.prev.is_none() + } else if lowest_version == floor { + floor_hash == Some(&lowest.self_hash) + } else if lowest_version == floor + 1 { + floor_hash.is_some() && lowest.prev.as_ref() == floor_hash + } else { + false + }; + + let mut head = Some(lowest_index); + let mut gap = !anchored; + let mut previous_version = lowest_version; + let mut previous_hash = lowest.self_hash; + + for (&version, &index) in by_version.range(lowest_version + 1..) { + let edition = editions[index]; + + if version == previous_version + 1 && edition.prev == Some(previous_hash) { + head = Some(index); + previous_version = version; + previous_hash = edition.self_hash; + } else { + gap = true; + break; + } + } + + FoldResult { + head, + gap, + anchored, + } +} + +/// The highest version overall, ignoring contiguity. +pub fn bootstrap_head(editions: &[EditionMeta]) -> Option { + editions + .iter() + .enumerate() + .reduce(|(best_index, best), (index, candidate)| { + let supersedes = candidate.version > best.version + || (candidate.version == best.version && candidate.tiebreak_id < best.tiebreak_id); + + if supersedes { + (index, candidate) + } else { + (best_index, best) + } + }) + .map(|(index, _)| index) +} + +fn canonical_decimal(raw: &str) -> Option { + if raw.is_empty() || !raw.bytes().all(|byte| byte.is_ascii_digit()) { + return None; + } + + if raw.len() > 1 && raw.starts_with('0') { + return None; + } + + raw.parse().ok() +} + +fn hex32(raw: &str, name: &'static str) -> Result<[u8; 32], EditionError> { + decode_hex_32(raw).map_err(|_| EditionError::BadField(name)) +} + +fn fields<'a>( + rumor: &'a UnsignedEvent, + name: &'static str, +) -> Result, EditionError> { + let mut found: Option<&[String]> = None; + + for tag in rumor.tags.iter() { + let tag_fields = tag.as_slice(); + + if tag_fields.first().map(String::as_str) != Some(name) { + continue; + } + + if found.is_some() { + return Err(EditionError::Duplicate(name)); + } + + found = Some(tag_fields); + } + + Ok(found) +} + +fn value<'a>( + rumor: &'a UnsignedEvent, + name: &'static str, +) -> Result, EditionError> { + match fields(rumor, name)? { + Some(fields) if fields.len() == 2 => Ok(Some(fields[1].as_str())), + Some(_) => Err(EditionError::BadField(name)), + None => Ok(None), + } +} + +#[cfg(test)] +mod tests { + use super::*; + + #[test] + fn edition_hash_matches_the_cross_client_vector() { + let entity = [0x11u8; 32]; + + assert_eq!( + HEXLOWER.encode(&edition_hash(&entity, 1, None, b"hello")), + "2daf42e65a6bc259a4c99fac6df754a5d3d92310607cf13e2a1e8c94d42f6303" + ); + + // The golden vector only exercises the absent-prev encoding; pin the + // present-prev branch structurally so a swapped flag stays visible. + let bytes = signing_bytes(&entity, 1, Some(&entity), b"hello"); + assert_eq!( + bytes.len(), + 8 + EDITION_LABEL.len() + 32 + 8 + 1 + 32 + 8 + 5 + ); + assert_eq!(&bytes[8..8 + EDITION_LABEL.len()], EDITION_LABEL); + assert_eq!( + bytes[8 + EDITION_LABEL.len() + 32..][..8], + 1u64.to_be_bytes() + ); + assert_eq!(bytes[8 + EDITION_LABEL.len() + 32 + 8], 1); + } +} diff --git a/crates/concord/src/lib.rs b/crates/concord/src/lib.rs index 82b4110e..481493fa 100644 --- a/crates/concord/src/lib.rs +++ b/crates/concord/src/lib.rs @@ -1,4 +1,6 @@ +pub mod control; pub mod derive; +pub mod edition; pub mod store; pub mod stream; @@ -8,6 +10,9 @@ use std::str::FromStr; use anyhow::{Result, anyhow, bail}; use data_encoding::HEXLOWER; pub use derive::GroupKey; +use rand::TryRng as _; +use rand::rngs::SysRng; +use serde::{Deserialize, Deserializer, Serialize, Serializer}; macro_rules! hex_id { ($(#[$meta:meta])* $name:ident) => { @@ -54,6 +59,19 @@ macro_rules! hex_id { Ok(Self(decode_hex_32(value)?)) } } + + impl Serialize for $name { + fn serialize(&self, serializer: S) -> Result { + serializer.serialize_str(&self.to_hex()) + } + } + + impl<'de> Deserialize<'de> for $name { + fn deserialize>(deserializer: D) -> Result { + let value = String::deserialize(deserializer)?; + value.parse().map_err(serde::de::Error::custom) + } + } }; } @@ -72,7 +90,9 @@ hex_id! { /// A key-rotation counter attached to each Community key. /// /// It bumps only on a Rekey, a membership change where somebody is removed. -#[derive(Clone, Copy, PartialEq, Eq, PartialOrd, Ord, Hash, Debug, Default)] +#[derive( + Clone, Copy, PartialEq, Eq, PartialOrd, Ord, Hash, Debug, Default, Serialize, Deserialize, +)] pub struct Epoch(pub u64); impl From for Epoch { @@ -94,7 +114,7 @@ impl fmt::Display for Epoch { } /// Uppercase and other non-canonical spellings are rejected. -fn decode_hex_32(value: &str) -> Result<[u8; 32]> { +pub(crate) fn decode_hex_32(value: &str) -> Result<[u8; 32]> { let bytes = HEXLOWER .decode(value.as_bytes()) .map_err(|error| anyhow!("invalid hex: {error}"))?; @@ -110,3 +130,15 @@ fn decode_hex_32(value: &str) -> Result<[u8; 32]> { Ok(decoded) } + +pub(crate) fn fill_random(bytes: &mut [u8]) -> Result<()> { + SysRng + .try_fill_bytes(bytes) + .map_err(|error| anyhow!("os rng: {error}")) +} + +pub(crate) fn random_32() -> Result<[u8; 32]> { + let mut bytes = [0u8; 32]; + fill_random(&mut bytes)?; + Ok(bytes) +} diff --git a/crates/concord/src/store.rs b/crates/concord/src/store.rs index 85ec7dc6..95f0aeb1 100644 --- a/crates/concord/src/store.rs +++ b/crates/concord/src/store.rs @@ -3,9 +3,13 @@ use std::sync::LazyLock; use anyhow::{Result, anyhow}; use nostr_sdk::prelude::*; +use serde::{Deserialize, Serialize}; -use crate::ChannelId; +use crate::control::{ChannelMetadata, CommunityGenesis, CommunityMetadata, ROOT_EPOCH}; +use crate::derive::control_signer_group_key; +use crate::edition::{ParsedEdition, vsk}; use crate::stream::OpenedStream; +use crate::{ChannelId, CommunityId, Epoch}; static LOCAL_KEYS: LazyLock = LazyLock::new(Keys::generate); @@ -14,6 +18,7 @@ const MARK_TAG: SingleLetterTag = SingleLetterTag::LOWERCASE_T; const MARK_VALUE: &str = "concord"; const WRAP_TAG: &str = "e"; const KIND_TAG: &str = "k"; +const STATE_PREFIX: &str = "concord/"; pub async fn cache_rumor( database: &dyn NostrDatabase, @@ -84,6 +89,146 @@ pub async fn query_rumors( Ok(rumors) } +#[derive(Debug, Clone, PartialEq, Eq, Serialize, Deserialize)] +pub struct EntityHead { + pub entity: [u8; 32], + pub version: u64, + pub self_hash: [u8; 32], + pub rumor_id: EventId, +} + +#[derive(Debug, Clone, PartialEq, Eq, Serialize, Deserialize)] +pub struct ChannelKeyRef { + pub id: ChannelId, + pub name: String, + pub private: bool, + pub epoch: Epoch, +} + +/// One local document per community, keyed by `concord/`. +#[derive(Debug, Clone, PartialEq, Serialize, Deserialize)] +pub struct CommunityState { + pub id: CommunityId, + pub owner: PublicKey, + pub owner_salt: [u8; 32], + pub community_root: [u8; 32], + pub root_epoch: Epoch, + #[serde(default, skip_serializing_if = "Option::is_none")] + pub control_root: Option<[u8; 32]>, + #[serde(default, skip_serializing_if = "BTreeMap::is_empty")] + pub control_pks: BTreeMap, + #[serde(default, skip_serializing_if = "Vec::is_empty")] + pub channels: Vec, + pub relays: Vec, + #[serde(default, skip_serializing_if = "Vec::is_empty")] + pub heads: Vec, + pub added_at_ms: u64, +} + +impl CommunityState { + pub fn from_genesis( + genesis: &CommunityGenesis, + editions: &[ParsedEdition], + added_at_ms: u64, + ) -> Result { + let mut channels = Vec::new(); + let mut heads = Vec::with_capacity(editions.len()); + let mut relays = Vec::new(); + + for edition in editions { + heads.push(EntityHead { + entity: edition.entity, + version: edition.version, + self_hash: edition.self_hash, + rumor_id: edition.rumor_id, + }); + + match edition.subkind.as_str() { + vsk::COMMUNITY_METADATA => { + let metadata: CommunityMetadata = serde_json::from_str(&edition.content)?; + relays.extend( + metadata + .relays + .iter() + .filter_map(|relay| RelayUrl::parse(relay).ok()), + ); + } + vsk::CHANNEL_METADATA => { + let metadata: ChannelMetadata = serde_json::from_str(&edition.content)?; + channels.push(ChannelKeyRef { + id: ChannelId::from_bytes(edition.entity), + name: metadata.name, + private: metadata.private, + epoch: ROOT_EPOCH, + }); + } + _ => {} + } + } + + let control_pks = BTreeMap::from([( + ROOT_EPOCH.0, + control_signer_group_key( + &genesis.control_root, + &genesis.identity.community_id, + ROOT_EPOCH, + )? + .pk(), + )]); + + Ok(Self { + id: genesis.identity.community_id, + owner: genesis.identity.owner, + owner_salt: genesis.identity.owner_salt, + community_root: genesis.community_root, + root_epoch: ROOT_EPOCH, + control_root: Some(genesis.control_root), + control_pks, + channels, + relays, + heads, + added_at_ms, + }) + } + + pub fn identifier(&self) -> String { + state_identifier(&self.id) + } +} + +fn state_identifier(id: &CommunityId) -> String { + format!("{STATE_PREFIX}{}", id.to_hex()) +} + +pub async fn save_state(database: &D, state: &CommunityState) -> Result<()> +where + D: NostrDatabase, +{ + let event = EventBuilder::new(Kind::ApplicationSpecificData, serde_json::to_string(state)?) + .tags([Tag::identifier(state.identifier())]) + .finalize_async(&*LOCAL_KEYS) + .await?; + + database.save_event(&event).await?; + + Ok(()) +} + +pub async fn load_state(database: &D, id: &CommunityId) -> Result> +where + D: NostrDatabase, +{ + let filter = Filter::new() + .kind(Kind::ApplicationSpecificData) + .identifier(state_identifier(id)) + .limit(1); + + match database.query(filter).await?.into_iter().next() { + Some(event) => Ok(Some(serde_json::from_str(&event.content)?)), + None => Ok(None), + } +} + #[cfg(test)] mod tests { use nostr_memory::MemoryDatabase; diff --git a/crates/concord/src/stream.rs b/crates/concord/src/stream.rs index 040f77e4..2a04120d 100644 --- a/crates/concord/src/stream.rs +++ b/crates/concord/src/stream.rs @@ -6,8 +6,6 @@ use nostr_sdk::prelude::{ Event, EventBuilder, EventId, FinalizeEvent, Keys, Kind, PublicKey, Tag, Timestamp, UnsignedEvent, }; -use rand::TryRng as _; -use rand::rngs::SysRng; use crate::derive::GroupKey; use crate::{ChannelId, Epoch}; @@ -206,6 +204,25 @@ pub fn wrap_seal( wrap_kind: u16, at: Timestamp, extra: &[Tag], +) -> Result<(Event, Keys), StreamError> { + wrap_seal_with( + seal, + group.conversation(), + group.keys(), + wrap_kind, + at, + extra, + ) +} + +/// Signs with `signer` while encrypting under `conversation`. +pub fn wrap_seal_with( + seal: &Event, + conversation: &ConversationKey, + signer: &Keys, + wrap_kind: u16, + at: Timestamp, + extra: &[Tag], ) -> Result<(Event, Keys), StreamError> { if wrap_kind != KIND_WRAP && wrap_kind != KIND_WRAP_EPHEMERAL { return Err(StreamError::BadWrapKind(wrap_kind)); @@ -214,7 +231,7 @@ pub fn wrap_seal( let json = seal.as_json(); check_plaintext_cap(json.len())?; - let content = BASE64.encode(&encrypt(group.conversation(), json.as_bytes())?); + let content = BASE64.encode(&encrypt(conversation, json.as_bytes())?); let ephemeral = Keys::generate(); let mut tags = vec![Tag::public_key(ephemeral.public_key())]; @@ -223,7 +240,7 @@ pub fn wrap_seal( let wrap = EventBuilder::new(Kind::Custom(wrap_kind), content) .tags(tags) .custom_created_at(at) - .finalize(group.keys()) + .finalize(signer) .map_err(|error| StreamError::Sign(error.to_string()))?; Ok((wrap, ephemeral)) @@ -335,9 +352,7 @@ pub fn check_channel_binding( fn encrypt(conversation: &ConversationKey, plaintext: &[u8]) -> Result, StreamError> { let mut nonce = [0u8; 32]; - SysRng - .try_fill_bytes(&mut nonce) - .map_err(|error| StreamError::Encrypt(error.to_string()))?; + crate::fill_random(&mut nonce).map_err(|error| StreamError::Encrypt(error.to_string()))?; encrypt_to_bytes_with_nonce(conversation, plaintext, nonce) .map_err(|error| StreamError::Encrypt(error.to_string())) -- 2.54.0 From 4329385abeec9994efd6e6aec9707ec7c7a74d55 Mon Sep 17 00:00:00 2001 From: Ren Amamiya Date: Wed, 16 Sep 2026 20:05:51 +0700 Subject: [PATCH 05/12] add control fold and roster --- PLAN.md | 161 ++++- crates/concord/src/control.rs | 630 +++++++++++++++++-- crates/concord/src/derive.rs | 7 +- crates/concord/src/edition.rs | 167 +++++ crates/concord/src/lib.rs | 19 +- crates/concord/src/roles.rs | 1083 +++++++++++++++++++++++++++++++++ crates/concord/src/store.rs | 58 +- crates/concord/src/stream.rs | 2 - 8 files changed, 2016 insertions(+), 111 deletions(-) create mode 100644 crates/concord/src/roles.rs diff --git a/PLAN.md b/PLAN.md index ec6529b6..9b0d3947 100644 --- a/PLAN.md +++ b/PLAN.md @@ -80,8 +80,9 @@ crates/concord/ src/lib.rs init, ConcordRegistry, ConcordEvent, signal bus, subscriptions, ingest pipeline src/derive.rs frozen HKDF / group_key / locators / commitments + golden vectors src/stream.rs CORD-01: seal + wrap + open (SealForm, OpenedStream), channel/epoch binding - src/edition.rs CORD-04 §1: canonical signing bytes, edition hash, parse, fold - src/control.rs control plane view, genesis, content types, roster fold, authority checks + src/edition.rs CORD-04 §1: edition hash, parse, chain fold, floor-aware head selection + src/control.rs control plane: genesis, content types, the control fold, the edition writer + src/roles.rs CORD-04 §2–§4: permissions, roles, grants, banlist, delegation fixpoint src/guestbook.rs CORD-02 §5: join/leave/kick/snapshot, coalesce, complete memberlist src/chat.rs CORD-03: channel plane — message/edit/delete/reaction builders + message view src/invite.rs CORD-05: bundle, link, registry, Invite List, Direct Invite @@ -89,11 +90,11 @@ crates/concord/ src/store.rs local persistence + opened-rumor cache + history queries ``` -`Community` and `Channel` GPUI entities live in `src/lib.rs` next to the registry — they are the public surface, not a separate concern. Ten modules, each with real content; no single-fn files. +`Community` and `Channel` GPUI entities live in `src/lib.rs` next to the registry — they are the public surface, not a separate concern. Eleven modules, each with real content; no single-fn files. Dependencies: `common`, `state`, `person`, `device`, `settings`, `gpui`, `nostr` (for `nip44` features), `nostr-sdk`, `hkdf`, `sha2`, `data-encoding`, `rand`, `serde`, `serde_json`, `anyhow`, `flume`, `smallvec`, `itertools`, `futures`, `log`, `instant`. Everything under `cfg(not(target_arch = "wasm32"))` follows the `chat`/`state` split so the crate still builds for web. -Declare only what a milestone actually uses. As of M2 the crate depends on `nostr`, `nostr-sdk`, `hkdf`, `sha2`, `data-encoding`, `rand`, `serde`, `serde_json`, `anyhow` (plus `nostr-memory` and `smol` for tests). `rand` is pinned to the `0.10.2` instance `nostr` already builds and shares its `getrandom`, which the `web` crate already enables `wasm_js` on — so no new package and no new wasm obligation. `serde`/`serde_json` were already in the graph via `nostr`; promoting `serde_json` from dev to main for the metadata content types added no package either, only the `concord → serde` edge. +Declare only what a milestone actually uses. As of M3 the crate depends on `nostr`, `nostr-sdk`, `hkdf`, `sha2`, `data-encoding`, `rand`, `serde`, `serde_json`, `anyhow` (plus `nostr-memory` and `smol` for tests). `rand` is pinned to the `0.10.2` instance `nostr` already builds and shares its `getrandom`, which the `web` crate already enables `wasm_js` on — so no new package and no new wasm obligation. `serde`/`serde_json` were already in the graph via `nostr`; promoting `serde_json` from dev to main for the metadata content types added no package either, only the `concord → serde` edge. M3 added no dependency of its own. ## 5. Core types @@ -249,7 +250,7 @@ Design points that are easy to get wrong: ## 8. Planes, state and folds -### 8.1 Editions and authority (`edition.rs`, `control.rs`) +### 8.1 Editions, authority and the control fold (`edition.rs`, `roles.rs`) ```rust pub const EDITION_LABEL: &[u8] = b"vector-community/v1/edition"; // frozen, cross-client (27 bytes) @@ -267,6 +268,13 @@ pub fn parse_edition(rumor: &UnsignedEvent) -> Result, pub gap: bool, pub anchored: bool } pub fn fold(editions: &[EditionMeta], floor: u64, floor_hash: Option<&[u8; 32]>) -> FoldResult; pub fn bootstrap_head(editions: &[EditionMeta]) -> Option; // highest, contiguity ignored + +// One entity's committed head, and the refuse-downgrade floor a later fold is judged +// against. +pub struct EntityHead { entity: [u8; 32], version: u64, self_hash: [u8; 32], rumor_id: EventId } +pub type Floors = BTreeMap<[u8; 32], EntityHead>; +pub struct HeadSelection { pub head: Option, pub gap: bool } +pub fn fold_head(editions: &[EditionMeta], floor: Option<&EntityHead>) -> HeadSelection; ``` - Tag grammar: `["vsk", sub]`, `["eid", hex32]`, `["ev", decimal]`, `["ep", hex32]`, `["vac", eid, version, hash]`. Duplicates of any of the five reject the edition; `ev` must pass a decimal check before parsing. A version of `0` parses and then reads as a gap — the rule lives in the fold, not the parser. @@ -274,42 +282,121 @@ pub fn bootstrap_head(editions: &[EditionMeta]) -> Option; // highest, - **The edition hash is not the signature.** The actor's Schnorr signature covers the kind-20014 plaintext seal; `edition_hash` is a separate SHA-256 used only for chaining (`ep`, `vac`). `content` is the rumor's content string byte-verbatim, never re-serialized, which is what lets compaction re-wrap a head and preserve its hash. - The domain label is `vector-community/v1/edition`, not a `concord/…` label. Inconsistent with Appendix A.6, frozen anyway — do not "fix" it. - Tie-break at equal version is the lower **inner rumor id** (the kind-3308 rumor), never the outer wrap id and never `created_at`. Only one of the two implementations that must agree applies to a wrap, so the inner id is the only stable choice. -- `gap` is a safety signal: a tracking client (already holds the floor) fails closed for that entity and refetches; a bootstrapping client (floor 0) may accept the highest authority-verified head, which is what makes compaction re-wrapping work. `bootstrap_head` therefore takes no floor: it *is* the floor-zero path. +- `gap` is a safety signal: a tracking client (already holds the floor) fails closed for that entity and refetches; a bootstrapping client (floor 0) may accept the highest authority-verified head, which is what makes compaction re-wrapping work. `bootstrap_head` therefore takes no floor: it *is* the floor-zero path. `fold_head` is the composition — floor 0 takes `bootstrap_head`; under a held floor the chain-anchored head wins and any upper gap is reported; everything below the floor is a stale relay, not a gap; and a head detached from the floor converges a same-version fork to its lower rumor id when that is genuinely earlier than what we hold, else fails closed as withholding. - **Owner anchoring is not in the fold.** `fold` is a pure function of chain shape; authority is a pre-filter the caller applies to the candidate set before folding. `community_id` proves the owner, and `is_authorized` short-circuits `owner == actor`, so the owner needs no Grant entity at all. - Entity coordinates are `vsk 0` → `community_id`, `1` → `role_id`, `2` → `channel_id`, `3` → `grant_locator`, `4` → `banlist_locator`, `8` → `invite_links_locator`, `11` → `pins_locator`. `5` is reserved, `6`/`9` belong to the 33301 invite marker, `7` is retired. All derive from `community_id` only, so a refounding re-wraps heads verbatim. - The Control Plane is **plaintext-seal only**. A 20013-encrypted control edition is rejected, because compaction re-wraps a signed plaintext seal byte-verbatim into the new epoch; accepting an encrypted one would let a later compaction fork the chain. - **Genesis is exactly two owner-signed editions** — community metadata (`vsk 0`, `eid = community_id`) and one public `#general` channel (`vsk 2`, fresh random `channel_id`) — at epoch 0, version 1, no `ep`, no `vac`. No default roles, no scaffolding, and no Grant for the owner. Secrets minted: `owner_salt`, `community_root`, `control_root` (deliberately not derived from `community_id`). ```rust -pub const P_MANAGE_ROLES: u64 = 1 << 0; // …bit table from CORD-04 §3, frozen; retired bits are burned -pub struct CommunityRoles { roles: BTreeMap<[u8; 32], Role>, grants: BTreeMap } -impl CommunityRoles { - pub fn permissions_of(&self, member: &PublicKey) -> u64; // union of role bits - pub fn position_of(&self, member: &PublicKey, owner: &PublicKey) -> u32; - pub fn is_authorized(&self, actor: &PublicKey, owner: &PublicKey, bit: u64) -> bool; - pub fn is_authorized_in(&self, actor: &PublicKey, owner: &PublicKey, channel: &ChannelId, bit: u64) -> bool; - pub fn outranks(&self, actor: &PublicKey, owner: &PublicKey, target_position: u32) -> bool; - pub fn can_act_on(&self, actor: &PublicKey, owner: &PublicKey, target: &PublicKey, bit: u64) -> bool; - pub fn is_staff(&self, member: &PublicKey, owner: &PublicKey) -> bool; // the six control bits, CORD-04 §3 +// CORD-04 §3, frozen. 1<<7 was MANAGE_INVITES and is burned, never reassigned. +// MANAGE_ROLES 1<<0 · MANAGE_CHANNELS 1<<1 · MANAGE_METADATA 1<<2 · KICK 1<<3 · +// BAN 1<<4 · MANAGE_MESSAGES 1<<5 · CREATE_INVITE 1<<6 · VIEW_AUDIT_LOG 1<<8 · +// MENTION_EVERYONE 1<<9 · PIN_MESSAGES 1<<11 · reserved: MANAGE_EMOJI 1<<10, MANAGE_EVENTS 1<<12 +pub struct Permissions(pub u64); +impl Permissions { + pub const STAFF_MASK: u64; // MANAGE_ROLES|MANAGE_CHANNELS|MANAGE_METADATA|BAN|CREATE_INVITE|PIN_MESSAGES + pub fn contains(self, bits: u64) -> bool; + pub fn union(self, other: Self) -> Self; + pub fn is_staff(self) -> bool; } + +pub enum RoleScope { Server, Channel(ChannelId) } // {"kind":"server"} / {"kind":"channel","channel_id":…} +pub struct Role { role_id: RoleId, name: String, position: u32, permissions: Permissions, + scope: RoleScope, color: u32, extra: Extra } +pub struct Grant { member: PublicKey, role_ids: Vec, control_wrap: Option, extra: Extra } + +pub struct CommunityRoles { roles: BTreeMap, grants: BTreeMap } +impl CommunityRoles { + pub fn role(&self, role_id: &RoleId) -> Option<&Role>; + pub fn roles_of(&self, member: &PublicKey) -> impl Iterator; + pub fn effective_permissions(&self, member: &PublicKey) -> Permissions; // union of granted role bits + pub fn has_permission(&self, member: &PublicKey, bits: u64) -> bool; + pub fn highest_position(&self, member: &PublicKey) -> Option; // lowest position they hold + pub fn is_authorized(&self, actor, owner, permission: u64) -> bool; // owner == actor → true + pub fn outranks(&self, actor, owner, target_position: u32) -> bool; // strict `<` + pub fn can_act_on_position(&self, actor, owner, target_position: u32, permission: u64) -> bool; + pub fn can_act_on_member(&self, actor, owner, target: &PublicKey, permission: u64) -> bool; + pub fn is_staff(&self, member, owner) -> bool; +} + +// The delegation fixpoint. Content is parsed once, up front: the fixpoint revisits +// every candidate on each pass. +pub enum AuthorityContent { Role(Role), Grant(Grant), Banlist(Vec) } +pub struct AuthorityEdition { entity: [u8; 32], meta: EditionMeta, author: PublicKey, + citation: Option, content: AuthorityContent } +impl AuthorityEdition { + pub fn parse(edition: &ParsedEdition, community_id: &CommunityId) -> Option; +} +pub struct Roster { roles: CommunityRoles, banned: BTreeSet, floors: Floors, gapped: bool } +pub fn fold_roster(owner, community_id, editions: &[AuthorityEdition], floors: &Floors, + held_bans: &BTreeSet) -> Roster; +pub fn citation_ok(owner, community_id, author, citation: Option<&AuthorityCitation>, + floors: &Floors) -> bool; ``` -Authority rules to encode once and test hard: +Authority rules as implemented: -- The owner is position 0, derived from `community_id`, and is never removable. -- No edition may claim a `position` at or above its own signer's, including the owner: no Role may claim 0. +- The owner is position 0, proven by `community_id`, supreme, unremovable, and **not a Role**: no Role may claim position 0, and every gate short-circuits `owner == actor`. The owner therefore needs no Grant and cites nothing. +- A member's rank is the **lowest** position among their Roles; a roleless member sits at `u32::MAX`. Two Roles may share a position (peers, neither acts on the other); display tie-breaks on the lower `role_id`. - The actor must hold the required bit **and strictly outrank** the target. Equal cannot act on equal. -- A `vac` citation is a sync floor, not a verdict: block until the cited Grant version is folded, verify its hash, then judge against the *current* roster. -- A staff-making Grant carries `control_wrap`, a NIP-44 pairwise ciphertext of `epoch_be[8] ‖ control_root[32]`, and is adopted **only if it derives to the `control_pk` the member already holds** for the named epoch. -- Banlist is one replaced entity; mutations carry a re-heal step (re-fold after publish, re-apply if the addition lost the tiebreak). +- `AuthorityEdition::parse` drops, rather than repairs: a `role_id` that is not its own coordinate, a `position` of 0, a Grant whose `member` does not hash to its entity, a `vsk 4` at a coordinate that is not this community's banlist locator, malformed JSON, and any `vsk` this type does not own. A Grant's `role_ids` truncate at 64 on read. +- **Refuse-downgrade**: an edition below the persisted floor for its entity is never a candidate. +- The fold is a **Jacobi fixed point** — authority propagates one delegation level per pass, bounded by `2 × (entities) + 8`. Convergence compares the roster only, not the heads. Cross-pass state is exactly the accepted roster plus its heads, and `citation_ok` reads the *previous* pass's heads, so the first pass sees none. +- **Roles** replay each entity's versions **ascending**, one winner per version group, `admissible` collecting the winners that pass. Gates, in order: not banned; `can_act_on_position(author, owner, position, MANAGE_ROLES)`; if a predecessor was admitted, the same call against *its* position; then the citation. The highest admissible version wins. Replaying ascending is what makes the second gate work: without it an admin at position 5 republishes a position-1 role at position 9, every check passes since 9 is beneath them, and a role that outranked them ends up beneath them along with everyone holding it. +- **Grants** take no version-group replay: the first candidate in vector order clearing every gate wins. Role references resolve **partially** — the resolvable subset is carried and the rest fold in on a later pass — because all-or-nothing resolution deadlocks the ordinary growth path (an admin creates a role, the owner grants it to them, and neither can go first, collapsing the entire roster including the owner's own grants). The final gate ranks every resolved position *and* the member. +- A **citation** that cannot be resolved parks the edition; a missing one is tolerated only where the rank gates carry the weight. For a Role that is everywhere. For a Grant it is not: a revoke names no position, so its rank test is vacuous — hence an uncited Grant may add authority but **never remove** it. +- The **banlist** is folded after a preliminary roster, since a ban only exists once someone authorized to place it does. Its head is the highest edition whose author currently holds `BAN` and does not already sit in the held banlist; each entry is kept only if that author strictly outranks the target, and the list caps at 500. **Withholding retains the held list** rather than un-banning nobody on a relay's word. The final roster is then re-folded with the banned set excluded, so a banned admin loses their authority in the same pass. +- A staff-making Grant carries `control_wrap`, a NIP-44 pairwise ciphertext of `epoch_be[8] ‖ control_root[32]`, adopted **only if it derives to the `control_pk` the member already holds** for the named epoch. Delivery, never authority. +- Caps: **100 Roles per community**, by the 100 lowest `role_id`, applied *after* authorization so forged low ids cannot evict a real role; the grants then shed the dropped ids. **64 Roles per member**, at parse. **500 banlist entries**, at fold. + +**Two deliberate divergences from the reference implementation** (Vector is an oracle, not a specification): + +1. **The role fork winner.** Vector's role branch walks version groups with `.iter().rev()`, taking each group's *highest* inner id, while its own adjacent comment says forks break on the lowest and the rest of its codebase (`fold_head`, `version::fold`, its invite-registry test) does use the lowest. No test in Vector pins the branch. We implement **lowest inner id**, per CORD-04 §1 — and our tests pin it. +2. **Banlist candidates must be `vsk 4`.** Vector collects banlist candidates from every edition sitting at the banlist locator regardless of `vsk`, so a `vsk 1` forged there can win the banlist head, parse to an empty list, and clear the ban. We require `vsk::BANLIST` for a candidate at all. + +One reference limitation we **reproduce and do not fix** (recorded here rather than silently diverging): the grant rank gate reads the *previous* pass's roster, so a mid-rank `MANAGE_ROLES` holder who cites a real, folded grant of their own can revoke a higher-ranked member whose authority is still propagating. Fixing it means resolving a Grant's target rank against the same pass, which changes the fixpoint's convergence argument. Revisit only with a spec amendment. ### 8.2 Communities, channels, metadata -`CommunityMetadata` carries `name` (≤ 64 bytes), `description` (≤ 10 000 bytes), `relays` (truncated on read and write to 5), `icon` and `banner` as encrypted-blob pointers (`{url, key, nonce, hash}`), `message_expiration`, and the optional `custom` object. `ChannelMetadata` carries `name`, `private`, optional `voice`, `deleted`, optional `custom`. +`CommunityMetadata` carries `name` (≤ 64 bytes), `description` (≤ 10 000 bytes), `relays` (truncated on read and write to 5), `icon` and `banner` as encrypted-blob pointers (`{url, key, nonce, hash}`), and the optional `custom` object. `ChannelMetadata` carries `name`, `private`, optional `voice`, `deleted`, optional `custom`. Every content struct carries `#[serde(flatten)] extra`, so a field one client does not model still round-trips and a rename by an older client cannot wipe another client's `custom` keys. -Every content struct uses `#[serde(flatten)] extra: serde_json::Map` and round-trips unknown fields. A name edit by an older client must not wipe another client's `custom` keys. Round-trip discipline gets its own test. +The Control Plane's whole projection is one call: -Channel keying follows CORD-03 §1: a public channel derives from `community_root` at the base epoch, a private one from its own random key at its own epoch. Public→private is a rekey at the next channel epoch (monotonic, never reset); private→public derives from the base again and the prior private history stays unreadable to later joiners. +```rust +pub struct ControlFold { + pub roles: CommunityRoles, + pub banned: BTreeSet, + pub community: Option, + pub channels: BTreeMap, + pub floors: Floors, + pub gapped: bool, +} +pub fn fold_control(owner: &PublicKey, community_id: &CommunityId, editions: &[ParsedEdition], + floors: &Floors, held_bans: &BTreeSet) -> ControlFold; +``` + +- The roster is folded first, and `vsk 0` / `vsk 2` are then judged against it: the head of each entity is the highest edition whose author *currently* holds `MANAGE_METADATA` / `MANAGE_CHANNELS`, is not banned, and either is the owner or cites their own folded Grant. Pre-filtering before the chain fold is what stops a demoted admin's later, higher-version edition from being the head. +- A `vsk 2` whose entity is the community's own id is excluded, and a `vsk 0` at any other coordinate with it: the floor row keys on the entity alone, so the two would otherwise share and corrupt one chain. +- `None` means "this client saw no authorized edition", never "the value is gone": a caller keeps what it holds rather than walking the community backwards. That is also how a withheld or downgraded entity reads. +- A `deleted` channel is reported as metadata with `deleted: true`; the policy of dropping it belongs to the store. + +Writes go through one primitive, so every edition names the head it supersedes and a client cannot silently fork a chain it cannot see: + +```rust +pub struct Edition<'a> { subkind: &'a str, entity: [u8; 32], content: &'a str, + head: Option<&'a EntityHead>, citation: Option } +pub struct ControlWriter { pub author: PublicKey, pub read: GroupKey, pub signer: GroupKey } +impl ControlWriter { + pub fn publish(&self, keys: &Keys, edition: Edition<'_>, at_secs: u64) -> Result<(Event, EntityHead)>; + pub fn set_community_metadata(&self, keys, community_id, metadata, head, at_secs) -> Result<(Event, EntityHead)>; + pub fn set_channel_metadata(&self, keys, channel, metadata, head, at_secs) -> Result<(Event, EntityHead)>; +} +``` + +`keys` is the acting member's own signer: the seal carries their signature, while the wrap is signed by the plane's published `control_pk`. Roles, grants and banlists ride the same `publish`, and their wrappers land with the moderation API (M5). A remote signer (NIP-46) is not yet plumbed — `publish` takes `&Keys`, not a `NostrSigner`. + +Channel keying follows CORD-03 §1: a public channel derives from `community_root` at the base epoch, a private one from its own random key at its own epoch. Public→private is a rekey at the next channel epoch (monotonic, never reset); private→public derives from the base again and the prior private history stays unreadable to later joiners. The store applies only the public declaration and the deletion: the public-to-private flip is ignored until the convert flow (key mint plus cursor rebase) lands, and a channel this client holds no key for is not added at all — it arrives with the invite that carries the key. ### 8.3 Guestbook and member list (`guestbook.rs`) @@ -413,7 +500,7 @@ pub fn compact(fold, epoch, new_control_root, ...) -> Vec; // re-wrap h Dissolution (CORD-02 §9) also lives here: a chainless, owner-signed `vsk 10` tombstone at `dissolved_group_key(id)`, plaintext-sealed, and a verifier **must** refuse any tombstone whose `eid` is not the community's own id (including the all-zero placeholder — accepting it lets an owner's genuine tombstone for one community be re-wrapped at another of theirs and kill it permanently). On sight the community is sealed read-only: subscriptions halt, nothing new is honored, existing history stays readable, and a member's delete of their own message is still honored. -## 9. Storage (`store.rs`) — local layer implemented in M1 +## 9. Storage (`store.rs`) — local layer implemented in M1, state document in M2, fold bridge in M3 Three layers, no new storage engine: @@ -453,6 +540,17 @@ pub struct CommunityState { Landed in M2 with exactly the fields genesis can populate: `save_state`/`load_state` and `CommunityState::from_genesis`. Three fields the plan sketched are deliberately absent until something can fill them — `epoch_keys` (needs rekeys, M7), and `guestbook`/`observed`/`banned`/`dissolved` (need the guestbook, M5). `control_pks` keyed by `u64` rather than `Epoch` and `heads` as a `Vec` rather than a `BTreeMap<[u8; 32], _>`, because serde_json cannot use a byte-array map key. +M3 added the two bridges between this document and the fold: + +```rust +impl CommunityState { + pub fn floors(&self) -> Floors; // the fold's input + pub fn apply_fold(&mut self, fold: &ControlFold); // the fold's output +} +``` + +`apply_fold` merges channels rather than replacing them, so a locally-held key survives a metadata edit. `banned` is not yet persisted here: `fold_control` takes the held list as an argument and returns the folded one, and the field lands with the moderation API (M5) that first writes it. + Writes are debounced (a fold head changes on every edition); reads load once at init. **Decision, stated for the record:** this document stores raw community keys unencrypted in a plaintext local database. That matches the existing posture — `chat` already caches decrypted message rumors in the same LMDB. If that posture ever changes, the state document is the one to wrap with NIP-44-to-self, since it is the only local artifact holding keys. @@ -577,7 +675,7 @@ pub fn pin(&self, id: EventId, cx: &App) -> Task>; // vsk 11, ## 11. Integration with existing crates -1. **`crates/chat/src/lib.rs` — required fix, moved from M2 to the milestone that first subscribes.** `handle_notifications` currently treats *every* kind 1059 event as a NIP-59 gift wrap for the current user and pushes an unwrap failure into the trash. Concord wraps are kind 1059 with an ephemeral `p` tag, so they would flood the trash and leak error toasts. Route by `subscription_id` from `RelayMessage::Event` against `sub_id1`/`sub_id2`, and drop the `if rumor.tags.is_empty()` recipient heuristic. **M2 did not apply it**: the crate has no subscription and no `ConcordRegistry` yet, so no concord wrap can reach that handler and the change would be untestable. It lands with the sync engine (§10), as does the `concord::init` wiring in `desktop` and `web`. +1. **`crates/chat/src/lib.rs` — required fix, moved from M2 to the milestone that first subscribes.** `handle_notifications` currently treats *every* kind 1059 event as a NIP-59 gift wrap for the current user and pushes an unwrap failure into the trash. Concord wraps are kind 1059 with an ephemeral `p` tag, so they would flood the trash and leak error toasts. Route by `subscription_id` from `RelayMessage::Event` against `sub_id1`/`sub_id2`, and drop the `if rumor.tags.is_empty()` recipient heuristic. **M2 and M3 did not apply it**: the crate has no subscription and no `ConcordRegistry` yet, so no concord wrap can reach that handler and the change would be untestable. It lands with the sync engine (§10), as does the `concord::init` wiring in `desktop` and `web`. 2. **`desktop/src/main.rs` and `web/src/lib.rs`** — add `concord::init(window, cx)` after `chat::init(window, cx)`. 3. **`Cargo.toml`** — add `hkdf = "0.12"` to `[workspace.dependencies]`; add the crate to `desktop` and `web` dependencies. No other workspace changes. 4. **No changes** to `state`, `person`, `device`, `settings`, `common`, or `ui`. @@ -599,6 +697,7 @@ Each of these has burned a real implementation, or is a documented cross-client - Refuse to write a Pin List from a list the writer could not read. - Enforce the NIP-44 65 535-byte cap at every layer before publishing, and the 5-relay / 256-channel / 50-membership / 100-roles / 64-roles-per-member / 500-banlist / 25-pin caps at their ingest and write points. - Lowercase hex only; x-only pubkeys only; no version tag anywhere. +- **Enforced in M3:** a Role's `role_id` is its own coordinate and never 0; a Grant's `member` hashes to its coordinate; a `vsk 4` sits at this community's banlist locator; a banned npub's editions are dropped and a grant naming them carries no rank; a revocation carries a citation; the 100-role cap keeps the lowest ids *after* authorization; a below-floor edition is never a candidate. Still owed to M8's audit: the byte caps, the 256-channel and 25-pin caps, and the write-side counterparts. ## 13. Milestones @@ -607,7 +706,7 @@ Each of these has burned a real implementation, or is a documented cross-client | M0 | Crate skeleton, `derive.rs`, golden vectors, workspace wiring | ✅ `cargo test -p concord` pins every derivation; all labels match Appendix A.6 | | M1 | `stream.rs` + `store.rs` | ✅ seal/wrap/open round-trips for both seal forms; hostile wraps rejected in the documented order; the local cache reads back with the group key gone | | M2 | `edition.rs` + `control.rs` genesis + `store.rs` state document | ✅ `cargo test -p concord` (7 tests): `edition_hash` reproduces the cross-client vector `2daf42e6…`, and a community minted by one holder has both genesis wraps open for a second holder holding only the invite keys, folding to version 1 | -| M3 | Control fold + roster + metadata/channels | fold tests for chains, gaps, downgrade refusal, fork tiebreak, compaction dangle; metadata and channel edits visible to a second client | +| M3 | Control fold + roster + metadata/channels | ✅ `cargo test -p concord` (15 tests): the chain fold, its gaps, fork tiebreak, downgrade refusal and compaction dangle are pinned; the delegation fixpoint resolves outward from the owner and refuses escalation, an unauthorized higher version, rank inversion by republish and an uncited revoke; a community minted by one holder has its metadata and channel edits fold for a second holder from the invite keys alone | | M4 | Chat plane | send/receive/edit/delete/reaction across two identities; history pages backwards across an epoch boundary (relay `backfill` lands here); binding checks reject a foreign channel/epoch | | M5 | Guestbook + member list + moderation | join/leave/kick/ban converge to the same memberlist on both clients; every authority rule from §8.1 has a negative test | | M6 | Invites + Community List | link mint → fetch → join round-trips; revoked link refuses; direct invite lands in the recipient's giftwrap inbox via the `k` tag; a second device reconstructs membership from 13302 | @@ -620,6 +719,10 @@ M1 closed with `cargo test -p concord` (5 tests), `cargo clippy -p concord --all M2 closed the same way at 7 tests, with `serde` added to the crate's dependencies (`serde_json` promoted from dev to main for the metadata content types) — `Cargo.lock` gained no package again, only the `concord → serde` edge. +M3 closed at 15 tests with no dependency change at all, and `Cargo.lock` untouched. New: `src/roles.rs` (permissions, Role/Grant/banlist content, `CommunityRoles`, the delegation fixpoint) and, in `src/control.rs`, `ControlFold` / `fold_control`, the metadata-and-channel fold, `ControlWriter` and its `Edition` input. `EntityHead` and `Floors` moved from `store.rs` into `edition.rs`, where `fold_head` now composes `fold` and `bootstrap_head` for the floor-aware case. + +What M3 still defers, and to what: the **sync engine's paging** driven by `ControlFold.gapped` and the **`chat::handle_notifications` routing fix** (both §10, together with the `concord::init` wiring — no concord wrap can reach that handler until the subscription exists); the **persisted banlist** and `CommunityState.banned` (M5, with the moderation API that writes it); the **role/grant/banlist write wrappers** (M5 — `ControlWriter::publish` already carries them, only the convenience surface is pending); and the **NIP-46 remote signer**, since `publish` takes `&Keys` rather than a `NostrSigner`. + **M2's "created and published" is verified offline**: "published" is the two wraps existing and being openable by the invite keys, not a relay round-trip. There is no registry to publish through until §10, and a relay test would be testing the SDK, not the protocol. ## 14. Open questions and risks @@ -632,6 +735,8 @@ M2 closed the same way at 7 tests, with `serde` added to the crate's dependencie 6. **Local plaintext state.** §9 records the decision. Revisit only if the local database stops being treated as trusted. 7. **Was a `community_id` ever hashed into a tag?** No — it must never appear on the wire. A lint-style test should assert it appears in no published event other than inside an invite bundle and a direct invite. 8. **The Pin List's message-key disclosure has no public API (M8).** CORD-04 §7 Pins let a keyless reader verify a disclosure, which means revealing one message's NIP-44 keys rather than the plane's conversation key. `nostr`'s `nip44::v2::get_message_keys(conversation_key, nonce)` is a private `fn`, and both public entry points (`encrypt_to_bytes_with_nonce`, `decrypt_to_bytes`) take the whole conversation key — so the expansion has to be reproduced as `hkdf::expand_into(conversation_key, nonce, 76 bytes)` plus ChaCha20 and an HMAC-SHA256, exactly as Vector does, and round-tripped against nostr's own `encrypt` in a test. Read CORD-04 §7 in full at M8 before writing it: the reproduction is only worth it once the exact verification the pin must support is settled, and the alternative is contributing a `pub` message-key accessor upstream (we already track git master, so a patch branch or an upstream PR is viable and strictly better than a reproduction we must keep in sync). +9. **A remote signer is not plumbed.** `ControlWriter::publish` and `stream`'s seal builders all take `&Keys`. NIP-46 is a stated Vector feature (§10's task slots are already cleared on signer change), but making the writers async over a `NostrSigner` is a change to every builder, so it should be one deliberate pass rather than a patch per milestone. Nothing in M4–M8 depends on it except the UX of using a remote signer at all. +10. **The fold is not incremental.** `fold_control` re-parses and re-folds the whole control edition window on every call, and each fold is up to `2 × entities + 8` passes. That is fine at the caps the spec sets (100 roles, 400-odd grants) and it is the simplest thing that is correct, but if the sync engine ends up calling it per event rather than per batch, the candidate maps and their parse belong in a cache keyed by edition id. Measure before optimizing. ## 15. Test strategy diff --git a/crates/concord/src/control.rs b/crates/concord/src/control.rs index 70daa75e..30b56e53 100644 --- a/crates/concord/src/control.rs +++ b/crates/concord/src/control.rs @@ -1,14 +1,21 @@ +use std::collections::{BTreeMap, BTreeSet}; + use anyhow::{Result, bail}; use nostr_sdk::prelude::{Event, Keys, PublicKey, Timestamp, UnsignedEvent}; use serde::{Deserialize, Serialize}; -use serde_json::{Map, Value}; use crate::derive::{ community_id_of, control_group_key, control_signer_group_key, verify_community_id, }; -use crate::edition::{EditionFields, ParsedEdition, build_edition, parse_edition, vsk}; +use crate::edition::{ + AuthorityCitation, EditionFields, EditionMeta, EntityHead, Floors, ParsedEdition, + build_edition, fold_head, parse_edition, vsk, +}; +use crate::roles::{ + AuthorityEdition, CommunityRoles, Permissions, Roster, citation_ok, fold_roster, +}; use crate::stream::{KIND_WRAP, SealForm, build_seal, open_wrap_at, wrap_seal_with}; -use crate::{ChannelId, CommunityId, Epoch, GroupKey, random_32}; +use crate::{ChannelId, CommunityId, Epoch, Extra, GroupKey, random_32}; pub const MAX_NAME_BYTES: usize = 64; pub const MAX_DESCRIPTION_BYTES: usize = 10_000; @@ -17,12 +24,8 @@ pub const MAX_RELAYS: usize = 5; pub const GENERAL_CHANNEL: &str = "general"; pub const ROOT_EPOCH: Epoch = Epoch(0); -/// The first edition every entity starts at. Genuinely 1, not 0: a version of -/// 0 is what the fold treats as a gap. const GENESIS_VERSION: u64 = 1; -type Extra = Map; - #[derive(Debug, Clone, Default, PartialEq, Serialize, Deserialize)] pub struct ImageRef { pub url: String, @@ -64,7 +67,6 @@ pub struct ChannelMetadata { pub extra: Extra, } -/// A community's permanent identity. #[derive(Debug, Clone, PartialEq, Eq)] pub struct CommunityIdentity { pub community_id: CommunityId, @@ -78,12 +80,6 @@ impl CommunityIdentity { } } -/// Everything a creation mints: -/// -/// - Identity -/// - Roots an invite will carry -/// - First channel -/// - Two genesis wraps to publish #[derive(Debug, Clone)] pub struct CommunityGenesis { pub identity: CommunityIdentity, @@ -93,29 +89,14 @@ pub struct CommunityGenesis { pub wraps: Vec, } -/// Mints a community and signs its two genesis editions. pub fn genesis( owner: &Keys, metadata: &CommunityMetadata, at_secs: u64, ) -> Result { - let mut metadata = metadata.clone(); - - if metadata.name.len() > MAX_NAME_BYTES { - bail!("community name exceeds {MAX_NAME_BYTES} bytes"); - } - - if metadata - .description - .as_ref() - .is_some_and(|description| description.len() > MAX_DESCRIPTION_BYTES) - { - bail!("community description exceeds {MAX_DESCRIPTION_BYTES} bytes"); - } - - metadata.relays.truncate(MAX_RELAYS); - + let metadata_content = encode_metadata(metadata)?; let owner_salt = random_32()?; + let identity = CommunityIdentity { community_id: community_id_of(&owner.public_key().to_bytes(), &owner_salt), owner: owner.public_key(), @@ -129,7 +110,6 @@ pub fn genesis( let read = control_group_key(&community_root, &identity.community_id, ROOT_EPOCH)?; let signer = control_signer_group_key(&control_root, &identity.community_id, ROOT_EPOCH)?; - let metadata_content = serde_json::to_string(&metadata)?; let channel_content = serde_json::to_string(&ChannelMetadata { name: GENERAL_CHANNEL.to_owned(), private: false, @@ -190,6 +170,277 @@ pub fn open_edition( Ok(parse_edition(&opened.rumor)?) } +/// Appends editions to entity chains. +pub struct ControlWriter { + pub author: PublicKey, + pub read: GroupKey, + pub signer: GroupKey, +} + +pub struct Edition<'a> { + pub subkind: &'a str, + pub entity: [u8; 32], + pub content: &'a str, + /// The head this edition supersedes. + /// + /// `None` starts the chain. + pub head: Option<&'a EntityHead>, + pub citation: Option, +} + +impl ControlWriter { + pub fn publish( + &self, + keys: &Keys, + edition: Edition<'_>, + at_secs: u64, + ) -> Result<(Event, EntityHead)> { + let rumor = build_edition(EditionFields { + author: self.author, + subkind: edition.subkind, + entity: edition.entity, + version: edition + .head + .map_or(GENESIS_VERSION, |head| head.version + 1), + prev: edition.head.map(|head| head.self_hash), + citation: edition.citation, + content: edition.content, + at_secs, + }); + + let parsed = parse_edition(&rumor)?; + let wrap = seal_edition(&rumor, keys, &self.read, &self.signer, at_secs)?; + + Ok((wrap, EntityHead::from(&parsed))) + } + + pub fn set_community_metadata( + &self, + keys: &Keys, + community_id: &CommunityId, + metadata: &CommunityMetadata, + head: Option<&EntityHead>, + at_secs: u64, + ) -> Result<(Event, EntityHead)> { + let content = encode_metadata(metadata)?; + + self.publish( + keys, + Edition { + subkind: vsk::COMMUNITY_METADATA, + entity: *community_id.as_bytes(), + content: &content, + head, + citation: None, + }, + at_secs, + ) + } + + pub fn set_channel_metadata( + &self, + keys: &Keys, + channel: &ChannelId, + metadata: &ChannelMetadata, + head: Option<&EntityHead>, + at_secs: u64, + ) -> Result<(Event, EntityHead)> { + let content = serde_json::to_string(metadata)?; + + self.publish( + keys, + Edition { + subkind: vsk::CHANNEL_METADATA, + entity: *channel.as_bytes(), + content: &content, + head, + citation: None, + }, + at_secs, + ) + } +} + +fn encode_metadata(metadata: &CommunityMetadata) -> Result { + if metadata.name.len() > MAX_NAME_BYTES { + bail!("community name exceeds {MAX_NAME_BYTES} bytes"); + } + + if metadata + .description + .as_ref() + .is_some_and(|description| description.len() > MAX_DESCRIPTION_BYTES) + { + bail!("community description exceeds {MAX_DESCRIPTION_BYTES} bytes"); + } + + let mut metadata = metadata.clone(); + metadata.relays.truncate(MAX_RELAYS); + + Ok(serde_json::to_string(&metadata)?) +} + +#[derive(Debug, Clone, Default)] +pub struct ControlFold { + pub roles: CommunityRoles, + pub banned: BTreeSet, + pub community: Option, + pub channels: BTreeMap, + pub floors: Floors, + pub gapped: bool, +} + +pub fn fold_control( + owner: &PublicKey, + community_id: &CommunityId, + editions: &[ParsedEdition], + floors: &Floors, + held_bans: &BTreeSet, +) -> ControlFold { + let authority: Vec = editions + .iter() + .filter_map(|edition| AuthorityEdition::parse(edition, community_id)) + .collect(); + + let roster = fold_roster(owner, community_id, &authority, floors, held_bans); + let metadata = fold_metadata(owner, community_id, editions, &roster, floors); + + let mut floors = roster.floors; + floors.extend(metadata.floors); + + ControlFold { + roles: roster.roles, + banned: roster.banned, + community: metadata.community, + channels: metadata.channels, + floors, + gapped: roster.gapped || metadata.gapped, + } +} + +#[derive(Debug, Default)] +struct MetadataFold { + community: Option, + channels: BTreeMap, + floors: Floors, + gapped: bool, +} + +fn fold_metadata( + owner: &PublicKey, + community_id: &CommunityId, + editions: &[ParsedEdition], + roster: &Roster, + floors: &Floors, +) -> MetadataFold { + let judge = Judge { + owner, + community_id, + roster, + floors, + }; + let community_entity = *community_id.as_bytes(); + let mut community: Vec<&ParsedEdition> = Vec::new(); + let mut channels: BTreeMap<[u8; 32], Vec<&ParsedEdition>> = BTreeMap::new(); + + for edition in editions { + match edition.subkind.as_str() { + // A channel addressed at the community's own coordinate would share, and + // corrupt, the metadata chain's floor. + vsk::COMMUNITY_METADATA if edition.entity == community_entity => { + community.push(edition) + } + vsk::CHANNEL_METADATA if edition.entity != community_entity => { + channels.entry(edition.entity).or_default().push(edition); + } + _ => {} + } + } + + let mut fold = MetadataFold::default(); + + if let Some(head) = authorized_head( + &judge, + community_entity, + &community, + Permissions::MANAGE_METADATA, + &mut fold.gapped, + ) { + fold.community = serde_json::from_str(&head.content).ok(); + fold.floors.insert(head.entity, EntityHead::from(head)); + } + + for (entity, candidates) in &channels { + let Some(head) = authorized_head( + &judge, + *entity, + candidates, + Permissions::MANAGE_CHANNELS, + &mut fold.gapped, + ) else { + continue; + }; + + fold.floors.insert(*entity, EntityHead::from(head)); + + if let Ok(metadata) = serde_json::from_str::(&head.content) { + fold.channels + .insert(ChannelId::from_bytes(*entity), metadata); + } + } + + fold +} + +struct Judge<'a> { + owner: &'a PublicKey, + community_id: &'a CommunityId, + roster: &'a Roster, + floors: &'a Floors, +} + +fn authorized_head<'a>( + judge: &Judge<'_>, + entity: [u8; 32], + candidates: &[&'a ParsedEdition], + permission: u64, + gapped: &mut bool, +) -> Option<&'a ParsedEdition> { + let authorized: Vec<&ParsedEdition> = candidates + .iter() + .copied() + .filter(|edition| { + // A banned npub's edits are dropped even while a grant naming them still carries the bit. + !judge.roster.banned.contains(&edition.author) + && judge + .roster + .roles + .is_authorized(&edition.author, judge.owner, permission) + && citation_ok( + judge.owner, + judge.community_id, + &edition.author, + edition.citation.as_ref(), + &judge.roster.floors, + ) + }) + .collect(); + + if authorized.is_empty() { + return None; + } + + let metas: Vec = authorized + .iter() + .map(|edition| EditionMeta::from(*edition)) + .collect(); + + let selection = fold_head(&metas, judge.floors.get(&entity)); + *gapped |= selection.gap; + + selection.head.map(|index| authorized[index]) +} + fn seal_edition( edition: &UnsignedEvent, owner: &Keys, @@ -216,42 +467,53 @@ mod tests { use nostr_memory::MemoryDatabase; use super::*; - use crate::edition::{EditionMeta, fold}; + use crate::derive::grant_locator; + use crate::edition::fold; + use crate::roles::{Grant, Role, RoleScope}; use crate::store::{CommunityState, load_state, save_state}; + use crate::{Extra, RoleId}; + + const AT: u64 = 1_700_000_000; + + fn holder(minted: &CommunityGenesis) -> (GroupKey, GroupKey) { + let community_id = minted.identity.community_id; + + ( + control_group_key(&minted.community_root, &community_id, ROOT_EPOCH).expect("derives"), + control_signer_group_key(&minted.control_root, &community_id, ROOT_EPOCH) + .expect("derives"), + ) + } + + fn open_all(wraps: &[Event], read: &GroupKey, address: &PublicKey) -> Vec { + wraps + .iter() + .map(|wrap| open_edition(wrap, read, address, true).expect("opens")) + .collect() + } + + fn metadata(name: &str) -> CommunityMetadata { + CommunityMetadata { + name: name.to_owned(), + ..CommunityMetadata::default() + } + } #[test] fn genesis_reopens_for_a_second_holder() { let owner = Keys::generate(); - let metadata = CommunityMetadata { + let community_metadata = CommunityMetadata { name: "coop".to_owned(), relays: vec!["wss://relay.example".to_owned()], ..CommunityMetadata::default() }; - let at_secs = 1_700_000_000; - let minted = genesis(&owner, &metadata, at_secs).expect("mints"); + let minted = genesis(&owner, &community_metadata, AT).expect("mints"); assert!(minted.identity.verify(), "identity is self-certifying"); - // The second client holds only what an invite hands over: the roots, - // the community id and the owner salt. - let read = control_group_key( - &minted.community_root, - &minted.identity.community_id, - ROOT_EPOCH, - ) - .expect("derives"); - let address = control_signer_group_key( - &minted.control_root, - &minted.identity.community_id, - ROOT_EPOCH, - ) - .expect("derives") - .pk(); - - let mut editions = Vec::new(); - for wrap in &minted.wraps { - editions.push(open_edition(wrap, &read, &address, true).expect("opens")); - } + // Only what an invite hands over: the roots, the community id and the owner salt. + let (read, signer) = holder(&minted); + let editions = open_all(&minted.wraps, &read, &signer.pk()); assert_eq!(editions.len(), 2); @@ -280,8 +542,7 @@ mod tests { ); } - let state = - CommunityState::from_genesis(&minted, &editions, at_secs * 1_000).expect("projects"); + let state = CommunityState::from_genesis(&minted, &editions, AT * 1_000).expect("projects"); smol::block_on(async { let database = MemoryDatabase::unbounded(); @@ -297,4 +558,257 @@ mod tests { assert_eq!(loaded.heads.len(), 2); }); } + + #[test] + fn metadata_and_channel_edits_reach_a_second_client() { + let owner = Keys::generate(); + let minted = genesis(&owner, &metadata("coop"), AT).expect("mints"); + let community_id = minted.identity.community_id; + let owner_pk = owner.public_key(); + let (read, signer) = holder(&minted); + + let genesis_editions = open_all(&minted.wraps, &read, &signer.pk()); + let roster = fold_control( + &owner_pk, + &community_id, + &genesis_editions, + &Floors::new(), + &BTreeSet::new(), + ); + assert_eq!( + roster.community.as_ref().map(|meta| meta.name.as_str()), + Some("coop") + ); + + let writer = ControlWriter { + author: owner_pk, + read: read.clone(), + signer: signer.clone(), + }; + let community_head = roster.floors.get(community_id.as_bytes()).expect("head"); + let channel_head = roster + .floors + .get(minted.channel_id.as_bytes()) + .expect("head"); + + let (community_wrap, _) = writer + .set_community_metadata( + &owner, + &community_id, + &CommunityMetadata { + relays: vec!["wss://relay.example".to_owned()], + ..metadata("coop two") + }, + Some(community_head), + AT + 1, + ) + .expect("publishes"); + let (channel_wrap, _) = writer + .set_channel_metadata( + &owner, + &minted.channel_id, + &ChannelMetadata { + name: "lobby".to_owned(), + private: false, + ..ChannelMetadata::default() + }, + Some(channel_head), + AT + 2, + ) + .expect("publishes"); + + let mut edited = genesis_editions.clone(); + edited.extend(open_all( + &[community_wrap, channel_wrap], + &read, + &signer.pk(), + )); + + let folded = fold_control( + &owner_pk, + &community_id, + &edited, + &Floors::new(), + &BTreeSet::new(), + ); + assert_eq!( + folded.community.as_ref().map(|meta| meta.name.as_str()), + Some("coop two") + ); + assert_eq!( + folded + .channels + .get(&minted.channel_id) + .map(|channel| channel.name.as_str()), + Some("lobby") + ); + + // A relay serving only the editions a client already folded past must not walk + // the community backwards. + let stale = fold_control( + &owner_pk, + &community_id, + &genesis_editions, + &folded.floors, + &BTreeSet::new(), + ); + assert!(stale.community.is_none()); + assert!(stale.channels.is_empty()); + + let mut state = + CommunityState::from_genesis(&minted, &genesis_editions, AT * 1_000).expect("projects"); + state.apply_fold(&folded); + assert_eq!(state.channels.len(), 1); + assert_eq!(state.channels[0].name, "lobby"); + assert_eq!(state.relays.len(), 1); + } + + #[test] + fn a_delegated_member_edits_metadata_only_under_its_own_grant() { + let owner = Keys::generate(); + let member = Keys::generate(); + let minted = genesis(&owner, &metadata("coop"), AT).expect("mints"); + let community_id = minted.identity.community_id; + let owner_pk = owner.public_key(); + let (read, signer) = holder(&minted); + + let writer = ControlWriter { + author: owner_pk, + read: read.clone(), + signer: signer.clone(), + }; + let role_id = RoleId::from_bytes([0x07; 32]); + let role = Role { + role_id, + name: "Mod".to_owned(), + position: 1, + permissions: Permissions(Permissions::MANAGE_METADATA), + scope: RoleScope::Server, + color: 0, + extra: Extra::default(), + }; + + let (role_wrap, _) = writer + .publish( + &owner, + Edition { + subkind: vsk::ROLE, + entity: *role_id.as_bytes(), + content: &role.to_content().expect("serializes"), + head: None, + citation: None, + }, + AT + 1, + ) + .expect("publishes"); + let (grant_wrap, _) = writer + .publish( + &owner, + Edition { + subkind: vsk::GRANT, + entity: grant_locator(&community_id, &member.public_key().to_bytes()), + content: &Grant { + member: member.public_key(), + role_ids: vec![role_id], + control_wrap: None, + extra: Extra::default(), + } + .to_content() + .expect("serializes"), + head: None, + citation: None, + }, + AT + 2, + ) + .expect("publishes"); + + let mut base = open_all(&minted.wraps, &read, &signer.pk()); + base.extend(open_all(&[role_wrap, grant_wrap], &read, &signer.pk())); + + let roster = fold_control( + &owner_pk, + &community_id, + &base, + &Floors::new(), + &BTreeSet::new(), + ); + assert!(roster.roles.is_staff(&member.public_key(), &owner_pk)); + + let grant = roster + .floors + .get(&grant_locator( + &community_id, + &member.public_key().to_bytes(), + )) + .expect("the member's grant folded"); + let head = roster.floors.get(community_id.as_bytes()).expect("head"); + + // The member seals with their own keys and wraps with the staff write key. + let member_writer = ControlWriter { + author: member.public_key(), + read, + signer: signer.clone(), + }; + let content = serde_json::to_string(&metadata("coop by mod")).expect("serializes"); + + let (uncited, _) = member_writer + .publish( + &member, + Edition { + subkind: vsk::COMMUNITY_METADATA, + entity: *community_id.as_bytes(), + content: &content, + head: Some(head), + citation: None, + }, + AT + 3, + ) + .expect("publishes"); + let (cited, _) = member_writer + .publish( + &member, + Edition { + subkind: vsk::COMMUNITY_METADATA, + entity: *community_id.as_bytes(), + content: &content, + head: Some(head), + citation: Some(AuthorityCitation { + entity: grant.entity, + version: grant.version, + hash: grant.self_hash, + }), + }, + AT + 4, + ) + .expect("publishes"); + + // Uncited, the edit claims an authority the member never showed. + let mut forged = base.clone(); + forged.extend(open_all(&[uncited], &member_writer.read, &signer.pk())); + let folded = fold_control( + &owner_pk, + &community_id, + &forged, + &Floors::new(), + &BTreeSet::new(), + ); + assert_eq!( + folded.community.as_ref().map(|meta| meta.name.as_str()), + Some("coop") + ); + + let mut edited_editions = base; + edited_editions.extend(open_all(&[cited], &member_writer.read, &signer.pk())); + let folded = fold_control( + &owner_pk, + &community_id, + &edited_editions, + &Floors::new(), + &BTreeSet::new(), + ); + assert_eq!( + folded.community.as_ref().map(|meta| meta.name.as_str()), + Some("coop by mod") + ); + } } diff --git a/crates/concord/src/derive.rs b/crates/concord/src/derive.rs index 02ec00a6..374bb342 100644 --- a/crates/concord/src/derive.rs +++ b/crates/concord/src/derive.rs @@ -181,7 +181,8 @@ pub fn control_signer_group_key( } /// Member-writable, unlike the Control Plane: -/// a join or a leave is each member's own word. +/// +/// - A join or a leave is each member's own word. pub fn guestbook_group_key( community_root: &[u8; 32], community_id: &CommunityId, @@ -210,7 +211,6 @@ pub fn channel_rekey_group_key( ) } -/// Keyed by the prior `community_root`: the base has no stable key above it pub fn base_rekey_group_key( prior_root: &[u8; 32], community_id: &CommunityId, @@ -224,13 +224,10 @@ pub fn base_rekey_group_key( ) } -/// Keyed by the `community_id` alone, so every member past or present resolves -/// the same address and a Refounding cannot strand the grave. pub fn dissolved_group_key(community_id: &CommunityId) -> Result { GroupKey::derive(LABEL_DISSOLVED, community_id.as_bytes(), &ZERO32, None) } -/// A plain SHA-256 commitment pub fn community_id_of(owner_xonly: &[u8; 32], owner_salt: &[u8; 32]) -> CommunityId { let mut hasher = Sha256::new(); hasher.update(LABEL_COMMUNITY.as_bytes()); diff --git a/crates/concord/src/edition.rs b/crates/concord/src/edition.rs index a758b6cd..16295ba9 100644 --- a/crates/concord/src/edition.rs +++ b/crates/concord/src/edition.rs @@ -3,6 +3,7 @@ use std::fmt; use data_encoding::HEXLOWER; use nostr_sdk::prelude::{EventId, PublicKey, Tag, UnsignedEvent}; +use serde::{Deserialize, Serialize}; use sha2::{Digest, Sha256}; use crate::decode_hex_32; @@ -312,6 +313,92 @@ pub fn bootstrap_head(editions: &[EditionMeta]) -> Option { .map(|(index, _)| index) } +#[derive(Debug, Clone, Copy, PartialEq, Eq, Default)] +pub struct HeadSelection { + pub head: Option, + pub gap: bool, +} + +/// The head to prefer for one entity, given what this client already committed to. +pub fn fold_head(editions: &[EditionMeta], floor: Option<&EntityHead>) -> HeadSelection { + let Some(floor) = floor else { + return HeadSelection { + head: bootstrap_head(editions), + gap: false, + }; + }; + + let anchored = fold(editions, floor.version, Some(&floor.self_hash)); + + if anchored.anchored { + return HeadSelection { + head: anchored.head, + gap: anchored.gap, + }; + } + + if anchored.head.is_none() && !anchored.gap { + return HeadSelection::default(); + } + + let fork = editions + .iter() + .enumerate() + .filter(|(_, edition)| edition.version == floor.version) + .min_by_key(|(_, edition)| edition.tiebreak_id); + + let winner = match fork { + Some((_, edition)) + if edition.self_hash != floor.self_hash && edition.tiebreak_id < floor.rumor_id => + { + edition.self_hash + } + _ => { + return HeadSelection { + head: None, + gap: true, + }; + } + }; + + let refolded = fold(editions, floor.version, Some(&winner)); + + if refolded.anchored { + HeadSelection { + head: refolded.head, + gap: refolded.gap, + } + } else { + HeadSelection { + head: None, + gap: true, + } + } +} + +/// A committed head, and the refuse-downgrade floor a later fold is judged against. +#[derive(Debug, Clone, PartialEq, Eq, Serialize, Deserialize)] +pub struct EntityHead { + pub entity: [u8; 32], + pub version: u64, + pub self_hash: [u8; 32], + pub rumor_id: EventId, +} + +impl From<&ParsedEdition> for EntityHead { + fn from(edition: &ParsedEdition) -> Self { + Self { + entity: edition.entity, + version: edition.version, + self_hash: edition.self_hash, + rumor_id: edition.rumor_id, + } + } +} + +/// Every entity's committed head, keyed by coordinate. +pub type Floors = BTreeMap<[u8; 32], EntityHead>; + fn canonical_decimal(raw: &str) -> Option { if raw.is_empty() || !raw.bytes().all(|byte| byte.is_ascii_digit()) { return None; @@ -366,6 +453,86 @@ fn value<'a>( mod tests { use super::*; + fn meta(version: u64, prev: Option<[u8; 32]>, hash: u8, tiebreak: u8) -> EditionMeta { + EditionMeta { + version, + self_hash: [hash; 32], + prev, + tiebreak_id: EventId::from_byte_array([tiebreak; 32]), + } + } + + fn head(version: u64, hash: u8, rumor: u8) -> EntityHead { + EntityHead { + entity: [0x11; 32], + version, + self_hash: [hash; 32], + rumor_id: EventId::from_byte_array([rumor; 32]), + } + } + + #[test] + fn fold_picks_the_head_from_the_chain_and_the_floor() { + let chain = [ + meta(1, None, 0xa1, 1), + meta(2, Some([0xa1; 32]), 0xa2, 2), + meta(3, Some([0xa2; 32]), 0xa3, 3), + ]; + + let folded = fold(&chain, 0, None); + assert_eq!(folded.head, Some(2)); + assert!(!folded.gap && folded.anchored); + + // A missing link stops the walk at the last contiguous edition. + let gapped = fold(&[chain[0], chain[2]], 0, None); + assert_eq!(gapped.head, Some(0)); + assert!(gapped.gap && gapped.anchored); + + // Everything below the held floor is a stale relay, not a gap. + let stale = fold(&chain[..2], 3, Some(&[0xa3; 32])); + assert_eq!(stale.head, None); + assert!(!stale.gap && !stale.anchored); + + // A fork at a version breaks on the lower inner rumor id, and the chain resumes. + let fork = [meta(1, None, 0xb1, 9), meta(1, None, 0xa1, 1)]; + assert_eq!( + fold(&fork, 0, None).head, + Some(1), + "the lower rumor id wins" + ); + let forked = [fork[0], fork[1], chain[1], chain[2]]; + assert_eq!(fold(&forked, 0, None).head, Some(3)); + + // A re-wrap onto the head we hold is the legitimate case; one whose `prev` no + // longer resolves is a withholding. + let rewrapped = meta(5, Some([0x99; 32]), 0xc5, 5); + assert_eq!( + fold_head(&[rewrapped], Some(&head(4, 0x99, 4))).head, + Some(0) + ); + let dangling = meta(5, Some([0x88; 32]), 0xc5, 5); + let refused = fold_head(&[dangling], Some(&head(4, 0x99, 4))); + assert_eq!(refused.head, None); + assert!(refused.gap); + + // A bootstrap takes it anyway: a compaction would leave a joiner with nothing. + assert_eq!(bootstrap_head(&[dangling]), Some(0)); + assert_eq!(fold_head(&[dangling], None).head, Some(0)); + + // A fork at the floor's own version converges to the lower rumor id when that is + // genuinely earlier than what we hold, and the chain above it re-anchors. + let forked = [ + meta(2, Some([0xa1; 32]), 0xb2, 3), + meta(3, Some([0xb2; 32]), 0xb3, 4), + ]; + let converged = fold_head(&forked, Some(&head(2, 0xaa, 9))); + assert_eq!(converged.head, Some(1)); + assert!(!converged.gap); + + // A fork that is not earlier than the held head is refused. + assert_eq!(fold_head(&forked, Some(&head(2, 0xaa, 2))).head, None); + } + #[test] fn edition_hash_matches_the_cross_client_vector() { let entity = [0x11u8; 32]; diff --git a/crates/concord/src/lib.rs b/crates/concord/src/lib.rs index 481493fa..0e6f96ba 100644 --- a/crates/concord/src/lib.rs +++ b/crates/concord/src/lib.rs @@ -1,6 +1,7 @@ pub mod control; pub mod derive; pub mod edition; +pub mod roles; pub mod store; pub mod stream; @@ -14,6 +15,9 @@ use rand::TryRng as _; use rand::rngs::SysRng; use serde::{Deserialize, Deserializer, Serialize, Serializer}; +/// Unknown fields a content struct does not model, so a republish cannot wipe them. +pub(crate) type Extra = serde_json::Map; + macro_rules! hex_id { ($(#[$meta:meta])* $name:ident) => { $(#[$meta])* @@ -76,20 +80,21 @@ macro_rules! hex_id { } hex_id! { - /// A Community's permanent identity: a self-certifying commitment to its - /// owner's key. It travels inside invites and is itself never on the wire - /// (CORD-02 §1). + /// A self-certifying commitment to the owner's key, carried inside invites and + /// never on the wire. CommunityId } hex_id! { - /// A Channel's identity within its Community (CORD-03). ChannelId } -/// A key-rotation counter attached to each Community key. -/// -/// It bumps only on a Rekey, a membership change where somebody is removed. +hex_id! { + /// Both a Role's entity coordinate and the field it repeats in its own content. + RoleId +} + +/// A key-rotation counter; it bumps only on a Rekey that removes somebody. #[derive( Clone, Copy, PartialEq, Eq, PartialOrd, Ord, Hash, Debug, Default, Serialize, Deserialize, )] diff --git a/crates/concord/src/roles.rs b/crates/concord/src/roles.rs new file mode 100644 index 00000000..c9b38d32 --- /dev/null +++ b/crates/concord/src/roles.rs @@ -0,0 +1,1083 @@ +use std::collections::{BTreeMap, BTreeSet, HashSet}; + +use anyhow::Result; +use nostr_sdk::prelude::PublicKey; +use serde::{Deserialize, Deserializer, Serialize, Serializer}; + +use crate::derive::{banlist_locator, grant_locator}; +use crate::edition::{ + AuthorityCitation, EditionMeta, EntityHead, Floors, ParsedEdition, fold_head, vsk, +}; +use crate::{ChannelId, CommunityId, Extra, RoleId, decode_hex_32}; + +pub const MAX_ROLES_PER_COMMUNITY: usize = 100; +pub const MAX_ROLES_PER_MEMBER: usize = 64; +pub const MAX_BANLIST: usize = 500; + +#[derive(Debug, Clone, Copy, PartialEq, Eq, Default, PartialOrd, Ord, Hash)] +pub struct Permissions(pub u64); + +impl Permissions { + pub const BAN: u64 = 1 << 4; + pub const CREATE_INVITE: u64 = 1 << 6; + pub const KICK: u64 = 1 << 3; + pub const MANAGE_CHANNELS: u64 = 1 << 1; + pub const MANAGE_MESSAGES: u64 = 1 << 5; + pub const MANAGE_METADATA: u64 = 1 << 2; + pub const MANAGE_ROLES: u64 = 1 << 0; + pub const MENTION_EVERYONE: u64 = 1 << 9; + pub const PIN_MESSAGES: u64 = 1 << 11; + pub const STAFF_MASK: u64 = Self::MANAGE_ROLES + | Self::MANAGE_CHANNELS + | Self::MANAGE_METADATA + | Self::BAN + | Self::CREATE_INVITE + | Self::PIN_MESSAGES; + pub const VIEW_AUDIT_LOG: u64 = 1 << 8; + + pub fn contains(self, bits: u64) -> bool { + self.0 & bits == bits + } + + pub fn union(self, other: Self) -> Self { + Self(self.0 | other.0) + } + + pub fn is_staff(self) -> bool { + self.0 & Self::STAFF_MASK != 0 + } +} + +impl Serialize for Permissions { + fn serialize(&self, serializer: S) -> Result { + serializer.serialize_str(&self.0.to_string()) + } +} + +impl<'de> Deserialize<'de> for Permissions { + fn deserialize>(deserializer: D) -> Result { + #[derive(Deserialize)] + #[serde(untagged)] + enum Raw { + String(String), + Number(u64), + } + + match Raw::deserialize(deserializer)? { + Raw::Number(bits) => Ok(Self(bits)), + Raw::String(bits) => { + if bits.is_empty() || !bits.bytes().all(|byte| byte.is_ascii_digit()) { + return Err(serde::de::Error::custom( + "permissions must be a decimal string", + )); + } + + bits.parse().map(Self).map_err(serde::de::Error::custom) + } + } + } +} + +#[derive(Debug, Clone, PartialEq, Eq, Serialize, Deserialize)] +#[serde(rename_all = "snake_case", tag = "kind", content = "channel_id")] +pub enum RoleScope { + Server, + Channel(ChannelId), +} + +#[derive(Debug, Clone, PartialEq, Serialize, Deserialize)] +pub struct Role { + pub role_id: RoleId, + pub name: String, + /// 0 belongs to the owner. + pub position: u32, + pub permissions: Permissions, + pub scope: RoleScope, + #[serde(default)] + pub color: u32, + #[serde(flatten)] + pub extra: Extra, +} + +impl Role { + pub fn parse(content: &str) -> Option { + serde_json::from_str(content).ok() + } + + pub fn to_content(&self) -> Result { + Ok(serde_json::to_string(self)?) + } +} + +#[derive(Debug, Clone, PartialEq, Serialize, Deserialize)] +pub struct Grant { + pub member: PublicKey, + /// Empty is a revoke. + #[serde(default)] + pub role_ids: Vec, + #[serde(default, skip_serializing_if = "Option::is_none")] + pub control_wrap: Option, + #[serde(flatten)] + pub extra: Extra, +} + +impl Grant { + pub fn parse(content: &str) -> Option { + serde_json::from_str(content).ok() + } + + pub fn to_content(&self) -> Result { + Ok(serde_json::to_string(self)?) + } +} + +pub fn parse_banlist(content: &str) -> Option> { + let entries: Vec = serde_json::from_str(content).ok()?; + let mut banned = Vec::with_capacity(entries.len()); + + for entry in &entries { + banned.push(PublicKey::from_slice(&decode_hex_32(entry).ok()?).ok()?); + } + + Some(banned) +} + +/// The graph aggregated from the folded Role and Grant editions; not a wire document. +#[derive(Debug, Clone, Default, PartialEq)] +pub struct CommunityRoles { + roles: BTreeMap, + grants: BTreeMap, +} + +impl CommunityRoles { + pub fn role(&self, role_id: &RoleId) -> Option<&Role> { + self.roles.get(role_id) + } + + pub fn roles(&self) -> impl Iterator { + self.roles.values() + } + + pub fn grants(&self) -> impl Iterator { + self.grants.values() + } + + pub fn is_empty(&self) -> bool { + self.roles.is_empty() && self.grants.is_empty() + } + + pub fn roles_of<'a>(&'a self, member: &PublicKey) -> impl Iterator + 'a { + self.grants + .get(member) + .into_iter() + .flat_map(|grant| grant.role_ids.iter()) + .filter_map(|role_id| self.roles.get(role_id)) + } + + pub fn effective_permissions(&self, member: &PublicKey) -> Permissions { + self.roles_of(member) + .fold(Permissions::default(), |total, role| { + total.union(role.permissions) + }) + } + + pub fn has_permission(&self, member: &PublicKey, bits: u64) -> bool { + self.effective_permissions(member).contains(bits) + } + + pub fn highest_position(&self, member: &PublicKey) -> Option { + self.roles_of(member).map(|role| role.position).min() + } + + pub fn is_authorized(&self, actor: &PublicKey, owner: &PublicKey, permission: u64) -> bool { + actor == owner || self.has_permission(actor, permission) + } + + /// Strictly outranks: equal cannot act on equal. + pub fn outranks(&self, actor: &PublicKey, owner: &PublicKey, target_position: u32) -> bool { + if actor == owner { + return true; + } + + match self.highest_position(actor) { + Some(position) => position < target_position, + None => false, + } + } + + pub fn can_act_on_position( + &self, + actor: &PublicKey, + owner: &PublicKey, + target_position: u32, + permission: u64, + ) -> bool { + if actor == owner { + return true; + } + + self.has_permission(actor, permission) && self.outranks(actor, owner, target_position) + } + + pub fn can_act_on_member( + &self, + actor: &PublicKey, + owner: &PublicKey, + target: &PublicKey, + permission: u64, + ) -> bool { + if target == owner { + return false; + } + + self.can_act_on_position( + actor, + owner, + self.highest_position(target).unwrap_or(u32::MAX), + permission, + ) + } + + pub fn is_staff(&self, member: &PublicKey, owner: &PublicKey) -> bool { + member == owner || self.effective_permissions(member).is_staff() + } + + fn cap_roles(&mut self) { + if self.roles.len() <= MAX_ROLES_PER_COMMUNITY { + return; + } + + let Some(threshold) = self.roles.keys().nth(MAX_ROLES_PER_COMMUNITY).copied() else { + return; + }; + + self.roles.split_off(&threshold); + + let roles = &self.roles; + self.grants.retain(|_, grant| { + grant.role_ids.retain(|role_id| roles.contains_key(role_id)); + !grant.role_ids.is_empty() + }); + } +} + +#[derive(Debug, Clone)] +pub enum AuthorityContent { + Role(Role), + Grant(Grant), + Banlist(Vec), +} + +#[derive(Debug, Clone)] +pub struct AuthorityEdition { + pub entity: [u8; 32], + pub meta: EditionMeta, + pub author: PublicKey, + pub citation: Option, + pub content: AuthorityContent, +} + +impl AuthorityEdition { + /// `None` for anything the fold should drop rather than repair. + pub fn parse(edition: &ParsedEdition, community_id: &CommunityId) -> Option { + let content = match edition.subkind.as_str() { + vsk::ROLE => { + let role = Role::parse(&edition.content)?; + + if *role.role_id.as_bytes() != edition.entity || role.position == 0 { + return None; + } + + AuthorityContent::Role(role) + } + vsk::GRANT => { + let mut grant = Grant::parse(&edition.content)?; + + if grant_locator(community_id, &grant.member.to_bytes()) != edition.entity { + return None; + } + + grant.role_ids.truncate(MAX_ROLES_PER_MEMBER); + + AuthorityContent::Grant(grant) + } + vsk::BANLIST => { + if edition.entity != banlist_locator(community_id) { + return None; + } + + AuthorityContent::Banlist(parse_banlist(&edition.content)?) + } + _ => return None, + }; + + Some(Self { + entity: edition.entity, + meta: EditionMeta::from(edition), + author: edition.author, + citation: edition.citation, + content, + }) + } + + pub fn head(&self) -> EntityHead { + EntityHead { + entity: self.entity, + version: self.meta.version, + self_hash: self.meta.self_hash, + rumor_id: self.meta.tiebreak_id, + } + } +} + +#[derive(Debug, Clone, Default)] +pub struct Roster { + pub roles: CommunityRoles, + pub banned: BTreeSet, + /// The role, grant and banlist heads this fold settled. + pub floors: Floors, + pub gapped: bool, +} + +pub fn fold_roster( + owner: &PublicKey, + community_id: &CommunityId, + editions: &[AuthorityEdition], + floors: &Floors, + held_bans: &BTreeSet, +) -> Roster { + let banlist = banlist_locator(community_id); + let mut role_candidates: BTreeMap<[u8; 32], Vec<&AuthorityEdition>> = BTreeMap::new(); + let mut grant_candidates: BTreeMap<[u8; 32], Vec<&AuthorityEdition>> = BTreeMap::new(); + let mut banlist_candidates: Vec<&AuthorityEdition> = Vec::new(); + + for edition in editions { + if edition.meta.version < floors.get(&edition.entity).map_or(0, |floor| floor.version) { + continue; + } + + match &edition.content { + AuthorityContent::Role(_) => { + role_candidates + .entry(edition.entity) + .or_default() + .push(edition); + } + AuthorityContent::Grant(_) => { + grant_candidates + .entry(edition.entity) + .or_default() + .push(edition); + } + AuthorityContent::Banlist(_) if edition.entity == banlist => { + banlist_candidates.push(edition); + } + AuthorityContent::Banlist(_) => {} + } + } + + for candidates in role_candidates + .values_mut() + .chain(grant_candidates.values_mut()) + { + rank(candidates); + } + + rank(&mut banlist_candidates); + + let mut gapped = false; + for (entity, candidates) in role_candidates.iter().chain(&grant_candidates) { + gapped |= entity_gapped(entity, candidates, floors); + } + gapped |= entity_gapped(&banlist, &banlist_candidates, floors); + + let preliminary = select_authorized( + owner, + community_id, + &role_candidates, + &grant_candidates, + &BTreeSet::new(), + ); + + let (banned, banlist_head, banlist_gapped) = fold_banlist( + owner, + community_id, + &banlist_candidates, + &preliminary.roles, + floors, + held_bans, + ); + gapped |= banlist_gapped; + + let mut selection = select_authorized( + owner, + community_id, + &role_candidates, + &grant_candidates, + &banned, + ); + selection.roles.cap_roles(); + + let mut floors = selection.floors; + if let Some(head) = banlist_head { + floors.insert(head.entity, head); + } + + Roster { + roles: selection.roles, + banned, + floors, + gapped, + } +} + +pub fn citation_ok( + owner: &PublicKey, + community_id: &CommunityId, + author: &PublicKey, + citation: Option<&AuthorityCitation>, + floors: &Floors, +) -> bool { + if author == owner { + return true; + } + + let Some(citation) = citation else { + return false; + }; + + let grant = grant_locator(community_id, &author.to_bytes()); + + if citation.entity != grant { + return false; + } + + match floors.get(&grant) { + Some(head) if head.version > citation.version => true, + Some(head) if head.version == citation.version => head.self_hash == citation.hash, + _ => false, + } +} + +fn rank(candidates: &mut Vec<&AuthorityEdition>) { + candidates.sort_by(|a, b| { + b.meta + .version + .cmp(&a.meta.version) + .then(a.meta.tiebreak_id.cmp(&b.meta.tiebreak_id)) + }); +} + +fn entity_gapped(entity: &[u8; 32], candidates: &[&AuthorityEdition], floors: &Floors) -> bool { + let metas: Vec = candidates.iter().map(|candidate| candidate.meta).collect(); + + fold_head(&metas, floors.get(entity)).gap +} + +#[derive(Debug, Default)] +struct Selection { + roles: CommunityRoles, + floors: Floors, +} + +fn select_authorized( + owner: &PublicKey, + community_id: &CommunityId, + role_candidates: &BTreeMap<[u8; 32], Vec<&AuthorityEdition>>, + grant_candidates: &BTreeMap<[u8; 32], Vec<&AuthorityEdition>>, + excluded: &BTreeSet, +) -> Selection { + let fixpoint = Fixpoint { + owner, + community_id, + excluded, + roles: role_candidates, + grants: grant_candidates, + }; + + let bound = 2 * (role_candidates.len() + grant_candidates.len()) + 8; + let mut accepted = Selection::default(); + + for _ in 0..bound { + let next = fixpoint.pass(&accepted); + + if next.roles == accepted.roles { + return next; + } + + accepted = next; + } + + accepted +} + +struct Fixpoint<'a> { + owner: &'a PublicKey, + community_id: &'a CommunityId, + excluded: &'a BTreeSet, + roles: &'a BTreeMap<[u8; 32], Vec<&'a AuthorityEdition>>, + grants: &'a BTreeMap<[u8; 32], Vec<&'a AuthorityEdition>>, +} + +impl Fixpoint<'_> { + fn pass(&self, accepted: &Selection) -> Selection { + let mut next = Selection::default(); + + for candidates in self.roles.values() { + self.select_role(candidates, accepted, &mut next); + } + + for candidates in self.grants.values() { + self.select_grant(candidates, accepted, &mut next); + } + + next + } + + fn select_role( + &self, + candidates: &[&AuthorityEdition], + accepted: &Selection, + next: &mut Selection, + ) { + let mut admissible: HashSet<[u8; 32]> = HashSet::new(); + let mut standing: Option = None; + let mut end = candidates.len(); + + while end > 0 { + let version = candidates[end - 1].meta.version; + let mut start = end; + + while start > 0 && candidates[start - 1].meta.version == version { + start -= 1; + } + + // One winner per version, so fork siblings cannot sidestep the gates. + for candidate in &candidates[start..end] { + let AuthorityContent::Role(role) = &candidate.content else { + continue; + }; + + if self.excluded.contains(&candidate.author) { + continue; + } + + if !accepted.roles.can_act_on_position( + &candidate.author, + self.owner, + role.position, + Permissions::MANAGE_ROLES, + ) { + continue; + } + + if let Some(previous) = standing + && !accepted.roles.can_act_on_position( + &candidate.author, + self.owner, + previous, + Permissions::MANAGE_ROLES, + ) + { + continue; + } + + // An unresolvable citation parks the edition; an absent one declared no floor to wait for. + if candidate.citation.is_some() + && !citation_ok( + self.owner, + self.community_id, + &candidate.author, + candidate.citation.as_ref(), + &accepted.floors, + ) + { + continue; + } + + admissible.insert(candidate.meta.self_hash); + standing = Some(role.position); + break; + } + + end = start; + } + + for candidate in candidates { + let AuthorityContent::Role(role) = &candidate.content else { + continue; + }; + + if !admissible.contains(&candidate.meta.self_hash) { + continue; + } + + next.roles.roles.insert(role.role_id, role.clone()); + next.floors.insert(candidate.entity, candidate.head()); + break; + } + } + + fn select_grant( + &self, + candidates: &[&AuthorityEdition], + accepted: &Selection, + next: &mut Selection, + ) { + for candidate in candidates { + let AuthorityContent::Grant(grant) = &candidate.content else { + continue; + }; + + if self.excluded.contains(&candidate.author) || self.excluded.contains(&grant.member) { + continue; + } + + let resolved: Vec<(&RoleId, u32)> = grant + .role_ids + .iter() + .filter_map(|role_id| { + accepted + .roles + .role(role_id) + .map(|role| (role_id, role.position)) + }) + .collect(); + + if resolved.is_empty() && !grant.role_ids.is_empty() { + continue; + } + + let cited = citation_ok( + self.owner, + self.community_id, + &candidate.author, + candidate.citation.as_ref(), + &accepted.floors, + ); + + if !cited && candidate.citation.is_some() { + continue; + } + + if !cited && resolved.is_empty() { + continue; + } + + let ranks_every_role = resolved.iter().all(|(_, position)| { + accepted.roles.can_act_on_position( + &candidate.author, + self.owner, + *position, + Permissions::MANAGE_ROLES, + ) + }); + + if !ranks_every_role + || !accepted.roles.can_act_on_member( + &candidate.author, + self.owner, + &grant.member, + Permissions::MANAGE_ROLES, + ) + { + continue; + } + + // A revoke still advances the floor; it just does not belong in the roster. + next.floors.insert(candidate.entity, candidate.head()); + + if !resolved.is_empty() { + let mut resolved_grant = grant.clone(); + resolved_grant.role_ids = resolved.iter().map(|(role_id, _)| **role_id).collect(); + next.roles.grants.insert(grant.member, resolved_grant); + } + + break; + } + } +} + +fn fold_banlist( + owner: &PublicKey, + community_id: &CommunityId, + candidates: &[&AuthorityEdition], + roster: &CommunityRoles, + floors: &Floors, + held_bans: &BTreeSet, +) -> (BTreeSet, Option, bool) { + let authorized: Vec<&AuthorityEdition> = candidates + .iter() + .copied() + .filter(|candidate| { + !held_bans.contains(&candidate.author) + && roster.is_authorized(&candidate.author, owner, Permissions::BAN) + && citation_ok( + owner, + community_id, + &candidate.author, + candidate.citation.as_ref(), + floors, + ) + }) + .collect(); + + if authorized.is_empty() { + return (held_bans.clone(), None, false); + } + + let entity = banlist_locator(community_id); + let metas: Vec = authorized.iter().map(|candidate| candidate.meta).collect(); + let selection = fold_head(&metas, floors.get(&entity)); + + let Some(index) = selection.head else { + return (held_bans.clone(), None, selection.gap); + }; + + let head = authorized[index]; + + let AuthorityContent::Banlist(entries) = &head.content else { + return (held_bans.clone(), None, selection.gap); + }; + + let banned: BTreeSet = entries + .iter() + .filter(|target| roster.can_act_on_member(&head.author, owner, target, Permissions::BAN)) + .take(MAX_BANLIST) + .copied() + .collect(); + + (banned, Some(head.head()), selection.gap) +} + +#[cfg(test)] +mod tests { + use nostr_sdk::prelude::Keys; + + use super::*; + use crate::edition::{EditionFields, build_edition, parse_edition}; + + const COMMUNITY: [u8; 32] = [0xc0; 32]; + const AT: u64 = 1_700_000_000; + + fn community_id() -> CommunityId { + CommunityId::from_bytes(COMMUNITY) + } + + fn role(entity: [u8; 32], position: u32) -> Role { + Role { + role_id: RoleId::from_bytes(entity), + name: "Mod".to_owned(), + position, + permissions: Permissions(Permissions::MANAGE_ROLES | Permissions::MANAGE_METADATA), + scope: RoleScope::Server, + color: 0, + extra: Extra::default(), + } + } + + fn edition( + author: &PublicKey, + subkind: &str, + entity: [u8; 32], + content: &str, + version: u64, + citation: Option, + ) -> AuthorityEdition { + let rumor = build_edition(EditionFields { + author: *author, + subkind, + entity, + version, + prev: None, + citation, + content, + at_secs: AT, + }); + + AuthorityEdition::parse(&parse_edition(&rumor).expect("parses"), &community_id()) + .expect("recognizes") + } + + fn role_edition(author: &PublicKey, id: u8, position: u32, version: u64) -> AuthorityEdition { + let entity = [id; 32]; + let content = role(entity, position).to_content().expect("serializes"); + + edition(author, vsk::ROLE, entity, &content, version, None) + } + + fn grant_edition( + author: &PublicKey, + member: &PublicKey, + roles: &[u8], + version: u64, + citation: Option, + ) -> AuthorityEdition { + let content = Grant { + member: *member, + role_ids: roles + .iter() + .map(|id| RoleId::from_bytes([*id; 32])) + .collect(), + control_wrap: None, + extra: Extra::default(), + } + .to_content() + .expect("serializes"); + + edition( + author, + vsk::GRANT, + grant_locator(&community_id(), &member.to_bytes()), + &content, + version, + citation, + ) + } + + fn fold( + owner: &PublicKey, + editions: &[AuthorityEdition], + banned: &BTreeSet, + ) -> Roster { + fold_roster(owner, &community_id(), editions, &Floors::new(), banned) + } + + fn position(roster: &Roster, id: u8) -> Option { + roster + .roles + .role(&RoleId::from_bytes([id; 32])) + .map(|role| role.position) + } + + #[test] + fn permission_bits_are_frozen() { + assert_eq!(Permissions::MANAGE_ROLES, 1); + assert_eq!(Permissions::MANAGE_CHANNELS, 2); + assert_eq!(Permissions::MANAGE_METADATA, 4); + assert_eq!(Permissions::KICK, 8); + assert_eq!(Permissions::BAN, 16); + assert_eq!(Permissions::MANAGE_MESSAGES, 32); + assert_eq!(Permissions::CREATE_INVITE, 64); + assert_eq!(Permissions::VIEW_AUDIT_LOG, 256); + assert_eq!(Permissions::MENTION_EVERYONE, 512); + assert_eq!(Permissions::PIN_MESSAGES, 2048); + assert_eq!(Permissions::STAFF_MASK, 1 | 2 | 4 | 16 | 64 | 2048); + } + + #[test] + fn role_content_round_trips_with_the_permissions_as_a_decimal_string() { + let id = [0x01; 32]; + let content = role(id, 2).to_content().expect("serializes"); + + assert!( + content.contains("\"permissions\":\"5\""), + "permissions ride as a string: {content}" + ); + assert!( + content.contains("\"scope\":{\"kind\":\"server\"}"), + "{content}" + ); + assert_eq!(Role::parse(&content).expect("parses").position, 2); + + let legacy = content.replace("\"permissions\":\"5\"", "\"permissions\":5"); + assert_eq!( + Role::parse(&legacy).expect("parses").permissions, + Permissions(Permissions::MANAGE_ROLES | Permissions::MANAGE_METADATA) + ); + assert!(Role::parse(&content.replace("\"5\"", "\"+5\"")).is_none()); + + let scoped = Role { + scope: RoleScope::Channel(ChannelId::from_bytes([0x0a; 32])), + ..role(id, 2) + }; + assert!( + scoped + .to_content() + .expect("serializes") + .contains("\"scope\":{\"kind\":\"channel\",\"channel_id\":") + ); + } + + #[test] + fn authority_resolves_outward_from_the_owner_and_refuses_escalation() { + let owner = Keys::generate(); + let admin = Keys::generate(); + let member = Keys::generate(); + let stranger = Keys::generate(); + + let editions = vec![ + role_edition(&owner.public_key(), 0x01, 1, 1), + grant_edition(&owner.public_key(), &admin.public_key(), &[0x01], 1, None), + grant_edition(&owner.public_key(), &member.public_key(), &[0x01], 1, None), + ]; + let roster = fold(&owner.public_key(), &editions, &BTreeSet::new()); + + assert!( + roster + .roles + .is_staff(&admin.public_key(), &owner.public_key()) + ); + assert!( + roster + .roles + .is_staff(&member.public_key(), &owner.public_key()) + ); + assert!( + !roster + .roles + .is_staff(&stranger.public_key(), &owner.public_key()) + ); + assert!(roster.roles.is_authorized( + &owner.public_key(), + &owner.public_key(), + Permissions::MANAGE_METADATA + )); + + // Equal cannot act on equal, and a roleless member outranks nobody. + assert!(!roster.roles.can_act_on_position( + &admin.public_key(), + &owner.public_key(), + 1, + Permissions::MANAGE_ROLES + )); + assert!(roster.roles.can_act_on_position( + &admin.public_key(), + &owner.public_key(), + 2, + Permissions::MANAGE_ROLES + )); + assert!(!roster.roles.can_act_on_member( + &admin.public_key(), + &owner.public_key(), + &member.public_key(), + Permissions::BAN + )); + + let banned = fold( + &owner.public_key(), + &editions, + &BTreeSet::from([admin.public_key()]), + ); + assert!( + !banned + .roles + .is_staff(&admin.public_key(), &owner.public_key()) + ); + assert!( + banned + .roles + .is_staff(&member.public_key(), &owner.public_key()) + ); + + // An unauthorized higher version is dropped, not allowed to vanish the entity. + let roster = fold( + &owner.public_key(), + &[ + role_edition(&stranger.public_key(), 0x02, 9, 9), + role_edition(&owner.public_key(), 0x02, 3, 1), + ], + &BTreeSet::new(), + ); + assert_eq!(position(&roster, 0x02), Some(3)); + + // Nor may an admin republish a role out from under its holders. + let roster = fold( + &owner.public_key(), + &[ + role_edition(&admin.public_key(), 0x03, 9, 2), + role_edition(&owner.public_key(), 0x03, 1, 1), + ], + &BTreeSet::new(), + ); + assert_eq!(position(&roster, 0x03), Some(1)); + } + + #[test] + fn an_uncited_revocation_strips_nothing_while_a_cited_one_does() { + let owner = Keys::generate(); + let moderator = Keys::generate(); + let member = Keys::generate(); + + let editions = vec![ + role_edition(&owner.public_key(), 0x01, 5, 1), + role_edition(&owner.public_key(), 0x02, 9, 1), + grant_edition( + &owner.public_key(), + &moderator.public_key(), + &[0x01], + 1, + None, + ), + grant_edition(&owner.public_key(), &member.public_key(), &[0x02], 1, None), + ]; + + let roster = fold(&owner.public_key(), &editions, &BTreeSet::new()); + assert!(roster.roles.roles_of(&member.public_key()).next().is_some()); + assert!(roster.roles.can_act_on_member( + &moderator.public_key(), + &owner.public_key(), + &member.public_key(), + Permissions::MANAGE_ROLES + )); + + let revoke = |citation| { + grant_edition( + &moderator.public_key(), + &member.public_key(), + &[], + 2, + citation, + ) + }; + + // A revoke names no position to rank-check. Uncited it is only a stranger's + // word, and strips nothing. + let stripped = fold( + &owner.public_key(), + &[editions.clone(), vec![revoke(None)]].concat(), + &BTreeSet::new(), + ); + assert!( + stripped + .roles + .roles_of(&member.public_key()) + .next() + .is_some() + ); + + // Cited against the grant it acts under, the same revoke lands. + let head = roster + .floors + .get(&grant_locator( + &community_id(), + &moderator.public_key().to_bytes(), + )) + .expect("the moderator's own grant folded"); + let stripped = fold( + &owner.public_key(), + &[ + editions, + vec![revoke(Some(AuthorityCitation { + entity: head.entity, + version: head.version, + hash: head.self_hash, + }))], + ] + .concat(), + &BTreeSet::new(), + ); + assert!( + stripped + .roles + .roles_of(&member.public_key()) + .next() + .is_none() + ); + assert!( + stripped + .roles + .is_staff(&moderator.public_key(), &owner.public_key()) + ); + } +} diff --git a/crates/concord/src/store.rs b/crates/concord/src/store.rs index 95f0aeb1..0348f839 100644 --- a/crates/concord/src/store.rs +++ b/crates/concord/src/store.rs @@ -5,9 +5,11 @@ use anyhow::{Result, anyhow}; use nostr_sdk::prelude::*; use serde::{Deserialize, Serialize}; -use crate::control::{ChannelMetadata, CommunityGenesis, CommunityMetadata, ROOT_EPOCH}; +use crate::control::{ + ChannelMetadata, CommunityGenesis, CommunityMetadata, ControlFold, ROOT_EPOCH, +}; use crate::derive::control_signer_group_key; -use crate::edition::{ParsedEdition, vsk}; +use crate::edition::{EntityHead, Floors, ParsedEdition, vsk}; use crate::stream::OpenedStream; use crate::{ChannelId, CommunityId, Epoch}; @@ -45,7 +47,6 @@ pub async fn cache_rumor( Ok(()) } -/// Read a channel's cached rumors. pub async fn query_rumors( database: &dyn NostrDatabase, channel: &ChannelId, @@ -89,14 +90,6 @@ pub async fn query_rumors( Ok(rumors) } -#[derive(Debug, Clone, PartialEq, Eq, Serialize, Deserialize)] -pub struct EntityHead { - pub entity: [u8; 32], - pub version: u64, - pub self_hash: [u8; 32], - pub rumor_id: EventId, -} - #[derive(Debug, Clone, PartialEq, Eq, Serialize, Deserialize)] pub struct ChannelKeyRef { pub id: ChannelId, @@ -194,6 +187,49 @@ impl CommunityState { pub fn identifier(&self) -> String { state_identifier(&self.id) } + + pub fn floors(&self) -> Floors { + self.heads + .iter() + .map(|head| (head.entity, head.clone())) + .collect() + } + + pub fn apply_fold(&mut self, fold: &ControlFold) { + self.heads = fold.floors.values().cloned().collect(); + + if let Some(community) = &fold.community { + self.relays = community + .relays + .iter() + .filter_map(|relay| RelayUrl::parse(relay).ok()) + .collect(); + } + + for (id, metadata) in &fold.channels { + if metadata.deleted.unwrap_or(false) { + self.channels.retain(|channel| channel.id != *id); + continue; + } + + match self.channels.iter_mut().find(|channel| channel.id == *id) { + Some(channel) => { + channel.name = metadata.name.clone(); + + if !metadata.private { + channel.private = false; + } + } + None if !metadata.private => self.channels.push(ChannelKeyRef { + id: *id, + name: metadata.name.clone(), + private: false, + epoch: self.root_epoch, + }), + None => {} + } + } + } } fn state_identifier(id: &CommunityId) -> String { diff --git a/crates/concord/src/stream.rs b/crates/concord/src/stream.rs index 2a04120d..f1817c07 100644 --- a/crates/concord/src/stream.rs +++ b/crates/concord/src/stream.rs @@ -140,7 +140,6 @@ pub fn build_rumor_secs( rumor } -/// Resolve a rumor's true millisecond time. pub fn resolve_ms_strict(rumor: &UnsignedEvent) -> Result { let seconds = rumor.created_at.as_secs().saturating_mul(1000); let mut tag: Option> = None; @@ -215,7 +214,6 @@ pub fn wrap_seal( ) } -/// Signs with `signer` while encrypting under `conversation`. pub fn wrap_seal_with( seal: &Event, conversation: &ConversationKey, -- 2.54.0 From d1b83fdc33ca6fe696bf0aa2cc26e15171be9888 Mon Sep 17 00:00:00 2001 From: Ren Amamiya Date: Wed, 16 Sep 2026 20:46:22 +0700 Subject: [PATCH 06/12] add chat plane --- PLAN.md | 112 ++++- crates/concord/src/chat.rs | 854 ++++++++++++++++++++++++++++++++++ crates/concord/src/edition.rs | 2 +- crates/concord/src/lib.rs | 1 + crates/concord/src/store.rs | 170 ++++++- 5 files changed, 1111 insertions(+), 28 deletions(-) create mode 100644 crates/concord/src/chat.rs diff --git a/PLAN.md b/PLAN.md index 9b0d3947..25110cf7 100644 --- a/PLAN.md +++ b/PLAN.md @@ -424,37 +424,80 @@ pub fn complete_memberlist(coalesced: &BTreeMap, - A Snapshot counts only from the npub whose Refounding minted the epoch. There is deliberately no owner fallback. - The member list is `coalesced Joined ∪ observed authors − banlist`, and observation counts **forward only** (an author re-enters on activity newer than their latest Leave/Kick/Ban). A Grant holder with `ms == 0` is present by construction. -### 8.4 Chat plane (`chat.rs`) +### 8.4 Chat plane (`chat.rs`) — implemented in M4 + +Kinds (CORD-02 Appendix B): `9` message, `1111` NIP-22 comment, `7` NIP-25 reaction, +`5` NIP-09 delete, `3302` edit, `3310` WebXDC peer signal, `23311` ephemeral typing. ```rust +pub struct ChatRumor { id, author, kind, channel, epoch, at_ms, content, + expiration: Option, action: ChatAction } +pub enum ChatAction { + Message { reply_to: Option, thread_root: Option }, + Reaction { target: EventId, emoji: String }, + Edit { target: EventId, content: String }, + Delete { target: EventId, target_kind: Option }, + Typing, + Opaque, +} +pub struct ReplyRef { id: EventId, author: Option } +pub struct Target { reply: ReplyRef, kind: u16 } // the wire commits the target's kind + +pub fn build_message(author, channel, epoch, content, quote: Option<&ReplyRef>, at_ms) -> UnsignedEvent; +pub fn build_comment(author, channel, epoch, content, parent: &Target, root: Option<&Target>, at_ms) -> UnsignedEvent; +pub fn build_reaction(author, channel, epoch, target: &Target, emoji: &str, at_ms) -> UnsignedEvent; +pub fn build_edit(author, channel, epoch, target: EventId, content: &str, at_ms) -> UnsignedEvent; +pub fn build_delete(author, channel, epoch, target: EventId, target_kind: Option, at_ms) -> UnsignedEvent; +pub fn build_typing(author, channel, epoch, at_ms) -> UnsignedEvent; + +pub fn seal_rumor(rumor: &UnsignedEvent, group: &GroupKey, author: &Keys, ephemeral: bool) + -> Result<(Event, Keys), ChatError>; +pub fn open(wrap: &Event, group: &GroupKey, channel: &ChannelId, epoch: Epoch) + -> Result<(OpenedStream, ChatRumor), ChatError>; +pub fn plane_keys(held: &[(Epoch, [u8; 32])], channel: &ChannelId) -> Result>; +pub fn fold(rumors: &[ChatRumor]) -> Vec; + pub struct ChatMessage { - pub id: EventId, // recomputed rumor id + pub id: EventId, pub author: PublicKey, pub channel: ChannelId, pub epoch: Epoch, - pub kind: Kind, // 9 | 1111 | 3302 | 1740 | 15 + pub kind: Kind, pub content: String, - pub media: Vec, - pub mentions: Vec, - pub reply_to: Option, // lowercase `e`/`q` - pub thread_root: Option, // uppercase `E` for 1111 + pub reply_to: Option, // a kind 9's `q`, or a comment's lowercase `e` + pub thread_root: Option, // a comment's uppercase `E` pub at_ms: u64, pub expiration: Option, - pub edited_at: Option, // folded from 3302 - pub deleted: bool, // folded from 5 + pub edited_at: Option, + pub deleted: bool, pub reactions: BTreeMap, } ``` -Sends funnel through one function so the rules cannot drift: - -```rust -fn publish_chat(store, client, community, channel, epoch, group, rumor, at_ms, ephemeral) -> Task, Error>>; -``` - -It builds the seal + wrap, mirrors the NIP-40 tag onto the wrap for durable kinds, publishes via `send_event(..).to(relays)`, retains the ephemeral wrap key for later NIP-09 scrubbing, and locally echoes its own wrap through the same ingest path so send-then-read works without waiting on a relay round-trip. - -Disappearing messages (CORD-08) live here: `message_expiration` is read from the folded metadata, `["expiration", created_at + t]` is attached to every durable Chat rumor and to the wrap, kinds 5 and 1740 are exempt, ingest refuses an already-expired rumor, a periodic sweep purges stored ones, and the kind 1740 timer notice renders only when its author holds `MANAGE_METADATA`. +- `open` returns the `OpenedStream` next to the typed rumor because the two halves go + different ways: the caller caches the envelope, and folds the rumor. +- Ordering is `(at_ms, id)` everywhere, ties on the lower inner rumor id. The fold emits + newest first; mutations replay ascending on `(at_ms, Reverse(id))` so the last one + applied wins — the highest `at_ms` and, between equal ones, the lower id. A deletion is + terminal: a later edit never revives it. +- **M4 honors a delete only from the message's own author.** A moderator delete (a `vac` + citation under `MANAGE_MESSAGES`) needs the roster, so the builder's `citation`, the + fold's `can_delete` predicate and its tests land with M5. Failing closed here loses a + moderator's reach, never a member's authorship. +- `kind 15` is coop's own file-message convention, outside the CORD registry — it is + accepted on the read side so a second coop device's files are not dropped, and + `send_file` lands with the registry. +- A reference's author slot is a SHOULD on the wire, so it is optional. NIP-25 nonetheless + makes `p` a requirement, so a builder must be handed a `Target` built from the message it + acts on and never one whose author is empty, or a peer that requires the tag drops the result. +- `ms` orders a page but cannot page within one: a relay's `until` filter is second-granular, + so the cursor step below is what has to cope with a boundary second. +- `seal_rumor` gates the kind at publish and mirrors a NIP-40 `expiration` onto the wrap + (CORD-08 §2). The timer's policy — ingest refusal, the sweep, kind 1740 — is M8. +- **`media` and `mentions` are deliberately absent.** Both are pure post-processing of + `content` by `common` (`extract_and_remove_media_urls`, `NostrParser`) and both return a + gpui type, and a protocol crate does not take a UI dependency for a derived field. They + land with the first consumer that renders them. ### 8.5 Invites (`invite.rs`) @@ -514,11 +557,21 @@ Three layers, no new storage engine: ```rust pub async fn cache_rumor(database: &dyn NostrDatabase, channel: &ChannelId, opened: &OpenedStream) -> Result<()>; pub async fn query_rumors(database: &dyn NostrDatabase, channel: &ChannelId, until: Option, limit: usize) -> Result>; +pub async fn backfill(client: &Client, database: &dyn NostrDatabase, channel: &ChannelId, + held: &[(Epoch, [u8; 32])], until: Option, limit: usize) + -> Result>; ``` `query_rumors` returns `UnsignedEvent`, not `Event`: the cached payload *is* a rumor, which is also what `OpenedStream` carries, so the caller never has to re-parse. -**Deferred to M4:** the relay-paging `backfill`. It is network history paging whose "step past the same-second wall" policy belongs with the sync engine, and M1 has no subscription to test it against. +**Landed in M4:** `backfill` — newest-first relay paging across every held epoch. It derives +every held epoch's plane key once, fetches `kinds [1059, 21059]` by all of those addresses in +one filter with an inclusive `until`, opens each wrap against the plane whose address it +carries, caches it, and pages until the page is short of the limit, adds nothing new, or the +cursor cannot advance. That last case is real: `until` has second granularity, so a page that +begins and ends inside one boundary second has nowhere left to step and its remainder stays +unreachable until a relay serves it. Capped at `MAX_PAGES` so a relay that only ever repeats +itself cannot loop a client forever. 3. **Community state** — one local document per community, `Kind::ApplicationSpecificData` with `["d", "concord/"]`: @@ -673,6 +726,11 @@ pub fn pin(&self, id: EventId, cx: &App) -> Task>; // vsk 11, `CommunityEvent` and `ChannelEvent` mirror `ChatEvent`: one variant per thing the UI has to react to (`Updated`, `Members`, `Added`, `Removed`, `Dissolved`, `Error`, plus channel-level `Incoming`, `Reload`). +Every send funnels through one function so the rules cannot drift: it seals and wraps the +rumor, mirrors any NIP-40 `expiration` onto the wrap, publishes via `send_event(..).to(relays)`, +retains the ephemeral wrap key for a later NIP-09 scrub, and echoes its own wrap through the +same ingest path so send-then-read never waits on a relay round-trip. + ## 11. Integration with existing crates 1. **`crates/chat/src/lib.rs` — required fix, moved from M2 to the milestone that first subscribes.** `handle_notifications` currently treats *every* kind 1059 event as a NIP-59 gift wrap for the current user and pushes an unwrap failure into the trash. Concord wraps are kind 1059 with an ephemeral `p` tag, so they would flood the trash and leak error toasts. Route by `subscription_id` from `RelayMessage::Event` against `sub_id1`/`sub_id2`, and drop the `if rumor.tags.is_empty()` recipient heuristic. **M2 and M3 did not apply it**: the crate has no subscription and no `ConcordRegistry` yet, so no concord wrap can reach that handler and the change would be untestable. It lands with the sync engine (§10), as does the `concord::init` wiring in `desktop` and `web`. @@ -697,7 +755,9 @@ Each of these has burned a real implementation, or is a documented cross-client - Refuse to write a Pin List from a list the writer could not read. - Enforce the NIP-44 65 535-byte cap at every layer before publishing, and the 5-relay / 256-channel / 50-membership / 100-roles / 64-roles-per-member / 500-banlist / 25-pin caps at their ingest and write points. - Lowercase hex only; x-only pubkeys only; no version tag anywhere. -- **Enforced in M3:** a Role's `role_id` is its own coordinate and never 0; a Grant's `member` hashes to its coordinate; a `vsk 4` sits at this community's banlist locator; a banned npub's editions are dropped and a grant naming them carries no rank; a revocation carries a citation; the 100-role cap keeps the lowest ids *after* authorization; a below-floor edition is never a candidate. Still owed to M8's audit: the byte caps, the 256-channel and 25-pin caps, and the write-side counterparts. +- **Enforced in M3:** a Role's `role_id` is its own coordinate and never 0; a Grant's `member` hashes to its coordinate; a `vsk 4` sits at this community's banlist locator; a banned npub's editions are dropped and a grant naming them carries no rank; a revocation carries a citation; the 100-role cap keeps the lowest ids *after* authorization; a below-floor edition is never a candidate. +- **Enforced in M4:** the chat plane's encrypted-seal requirement, at both publish and open; `channel` **and** `epoch` strict-equal to the plane whose key opened the wrap; a retired or unregistered rumor kind rejected on both sides; a target bearing tag that appears twice rejected outright; and a delete honored only from the message's own author. +- Still owed to M8's audit: the byte caps, the 256-channel and 25-pin caps, the guestbook's future-clock and Snapshot rules, and the write-side counterparts. ## 13. Milestones @@ -706,23 +766,27 @@ Each of these has burned a real implementation, or is a documented cross-client | M0 | Crate skeleton, `derive.rs`, golden vectors, workspace wiring | ✅ `cargo test -p concord` pins every derivation; all labels match Appendix A.6 | | M1 | `stream.rs` + `store.rs` | ✅ seal/wrap/open round-trips for both seal forms; hostile wraps rejected in the documented order; the local cache reads back with the group key gone | | M2 | `edition.rs` + `control.rs` genesis + `store.rs` state document | ✅ `cargo test -p concord` (7 tests): `edition_hash` reproduces the cross-client vector `2daf42e6…`, and a community minted by one holder has both genesis wraps open for a second holder holding only the invite keys, folding to version 1 | -| M3 | Control fold + roster + metadata/channels | ✅ `cargo test -p concord` (15 tests): the chain fold, its gaps, fork tiebreak, downgrade refusal and compaction dangle are pinned; the delegation fixpoint resolves outward from the owner and refuses escalation, an unauthorized higher version, rank inversion by republish and an uncited revoke; a community minted by one holder has its metadata and channel edits fold for a second holder from the invite keys alone | -| M4 | Chat plane | send/receive/edit/delete/reaction across two identities; history pages backwards across an epoch boundary (relay `backfill` lands here); binding checks reject a foreign channel/epoch | +| M3 | Control fold + roster + metadata/channels | ✅ `cargo test -p concord` (14 tests): the chain fold, its gaps, fork tiebreak, downgrade refusal and compaction dangle are pinned; the delegation fixpoint resolves outward from the owner and refuses escalation, an unauthorized higher version, rank inversion by republish and an uncited revoke; a community minted by one holder has its metadata and channel edits fold for a second holder from the invite keys alone | +| M4 | Chat plane | ✅ `cargo test -p concord` (19 tests): a second holder folds a message's reactions, its author's edit and its author's delete, and ignores an edit or a delete from anybody else; a comment's root and parent survive the wire; a foreign channel, a replayed epoch, a plaintext seal, a retired kind and a duplicated target are each rejected; and history pages backwards across a rekey in order | | M5 | Guestbook + member list + moderation | join/leave/kick/ban converge to the same memberlist on both clients; every authority rule from §8.1 has a negative test | | M6 | Invites + Community List | link mint → fetch → join round-trips; revoked link refuses; direct invite lands in the recipient's giftwrap inbox via the `k` tag; a second device reconstructs membership from 13302 | | M7 | Rekeys + refounding + dissolution | a removed member stops reading after a rekey; continuity and race rules tested; a tombstone seals the community and a foreign-id tombstone is refused | | M8 | Pins + disappearing messages + hardening | pins verify from a keyless reader's view; expiry is refused at ingest and purged by the sweep; the audit of §12 is complete with a test per bullet | -Ordering is deliberately dependency-first: each milestone is usable on its own, and nothing in M2+ depends on a later milestone. +Ordering is deliberately dependency-first: each milestone is usable on its own, and nothing in M2+ depends on a later milestone. The sync engine and its GPUI wiring (§10, §11) have no row of their own because they are cross-cutting: every plane they consume has to exist first, so they follow M8, and they are the milestone that applies §11's `chat::handle_notifications` routing fix. M1 closed with `cargo test -p concord` (5 tests), `cargo clippy -p concord --all-targets`, and `cargo fmt -p concord --check` all clean. `rand` was added to the workspace pinned to the same `0.10.2` instance `nostr` already builds, so `Cargo.lock` gained no package. M2 closed the same way at 7 tests, with `serde` added to the crate's dependencies (`serde_json` promoted from dev to main for the metadata content types) — `Cargo.lock` gained no package again, only the `concord → serde` edge. -M3 closed at 15 tests with no dependency change at all, and `Cargo.lock` untouched. New: `src/roles.rs` (permissions, Role/Grant/banlist content, `CommunityRoles`, the delegation fixpoint) and, in `src/control.rs`, `ControlFold` / `fold_control`, the metadata-and-channel fold, `ControlWriter` and its `Edition` input. `EntityHead` and `Floors` moved from `store.rs` into `edition.rs`, where `fold_head` now composes `fold` and `bootstrap_head` for the floor-aware case. +M3 closed at 14 tests with no dependency change at all, and `Cargo.lock` untouched. New: `src/roles.rs` (permissions, Role/Grant/banlist content, `CommunityRoles`, the delegation fixpoint) and, in `src/control.rs`, `ControlFold` / `fold_control`, the metadata-and-channel fold, `ControlWriter` and its `Edition` input. `EntityHead` and `Floors` moved from `store.rs` into `edition.rs`, where `fold_head` now composes `fold` and `bootstrap_head` for the floor-aware case. + +M4 closed at 19 tests, again with no dependency change and `Cargo.lock` untouched — relay paging is `Client::fetch_events` over the same `NostrDatabase` the cache already used, so nothing new was needed. New: `src/chat.rs` (the whole channel plane) and, in `src/store.rs`, `backfill` plus the pure `advance` page step it is built from, which is what the paging test drives instead of a socket. `edition::canonical_decimal` became `pub(crate)` so the chat tag grammar shares one decimal check. What M3 still defers, and to what: the **sync engine's paging** driven by `ControlFold.gapped` and the **`chat::handle_notifications` routing fix** (both §10, together with the `concord::init` wiring — no concord wrap can reach that handler until the subscription exists); the **persisted banlist** and `CommunityState.banned` (M5, with the moderation API that writes it); the **role/grant/banlist write wrappers** (M5 — `ControlWriter::publish` already carries them, only the convenience surface is pending); and the **NIP-46 remote signer**, since `publish` takes `&Keys` rather than a `NostrSigner`. +What M4 still defers, and to what: **moderator deletes and the `can_delete` predicate** (M5 — M4 honors a delete only from the message's own author, so a moderator's reach is missing rather than forged); **`media`/`mentions`** on `ChatMessage` and **`send_file`** (the registry/UI milestone — the first needs a gpui type and the second needs the blob-upload path); and **the timer's policy** under the `expiration` tag that `seal_rumor` already mirrors (M8). + **M2's "created and published" is verified offline**: "published" is the two wraps existing and being openable by the invite keys, not a relay round-trip. There is no registry to publish through until §10, and a relay test would be testing the SDK, not the protocol. ## 14. Open questions and risks diff --git a/crates/concord/src/chat.rs b/crates/concord/src/chat.rs new file mode 100644 index 00000000..067bcccf --- /dev/null +++ b/crates/concord/src/chat.rs @@ -0,0 +1,854 @@ +use std::cmp::Reverse; +use std::collections::BTreeMap; +use std::fmt; + +use anyhow::Result; +use nostr_sdk::prelude::*; + +use crate::derive::channel_group_key; +use crate::edition::canonical_decimal; +use crate::stream::{ + KIND_WRAP, KIND_WRAP_EPHEMERAL, OpenedStream, SealForm, StreamError, build_rumor_ms, + build_seal, channel_binding_tags, check_channel_binding, open_wrap, resolve_ms_strict, + wrap_seal, +}; +use crate::{ChannelId, Epoch, GroupKey, decode_hex_32}; + +pub const KIND_MESSAGE: u16 = 9; +pub const KIND_COMMENT: u16 = 1111; +pub const KIND_REACTION: u16 = 7; +pub const KIND_DELETE: u16 = 5; +pub const KIND_EDIT: u16 = 3302; +pub const KIND_FILE: u16 = 15; +pub const KIND_WEBXDC: u16 = 3310; +pub const KIND_TYPING: u16 = 23311; + +const TAG_QUOTE: &str = "q"; +const TAG_TARGET: &str = "e"; +const TAG_TARGET_KIND: &str = "k"; +const TAG_ROOT: &str = "E"; +const TAG_ROOT_KIND: &str = "K"; +const TAG_ROOT_AUTHOR: &str = "P"; +const TAG_TARGET_AUTHOR: &str = "p"; +const TAG_EXPIRATION: &str = "expiration"; + +#[derive(Debug)] +pub enum ChatError { + Stream(StreamError), + NotEncryptedSealed, + UnknownKind(u16), + MissingTag(&'static str), + DuplicateTag(&'static str), + BadTag(&'static str), +} + +impl fmt::Display for ChatError { + fn fmt(&self, f: &mut fmt::Formatter<'_>) -> fmt::Result { + match self { + ChatError::Stream(error) => write!(f, "stream: {error}"), + ChatError::NotEncryptedSealed => write!(f, "chat rumor must ride an encrypted seal"), + ChatError::UnknownKind(kind) => write!(f, "not a chat rumor kind: {kind}"), + ChatError::MissingTag(name) => write!(f, "missing chat tag: {name}"), + ChatError::DuplicateTag(name) => write!(f, "duplicate chat tag: {name}"), + ChatError::BadTag(name) => write!(f, "malformed chat tag: {name}"), + } + } +} + +impl std::error::Error for ChatError {} + +impl From for ChatError { + fn from(error: StreamError) -> Self { + ChatError::Stream(error) + } +} + +/// A chat event another chat event refers to: a quote, a comment's parent, a +/// reaction's target. The author slot is a SHOULD on the wire, so it is optional. +#[derive(Debug, Clone, Copy, PartialEq, Eq)] +pub struct ReplyRef { + pub id: EventId, + pub author: Option, +} + +/// A reference that also names the referenced event's kind, which a comment +/// (`K`/`k`) and a reaction (`k`) must commit on the wire. +#[derive(Debug, Clone, Copy, PartialEq, Eq)] +pub struct Target { + pub reply: ReplyRef, + pub kind: u16, +} + +#[derive(Debug, Clone)] +pub enum ChatAction { + Message { + reply_to: Option, + thread_root: Option, + }, + Reaction { + target: EventId, + emoji: String, + }, + Edit { + target: EventId, + content: String, + }, + Delete { + target: EventId, + target_kind: Option, + }, + Typing, + Opaque, +} + +#[derive(Debug, Clone)] +pub struct ChatRumor { + pub id: EventId, + pub author: PublicKey, + pub kind: Kind, + pub channel: ChannelId, + pub epoch: Epoch, + pub at_ms: u64, + pub content: String, + pub expiration: Option, + pub action: ChatAction, +} + +/// A channel's timeline row, with every edit, delete and reaction folded in. +#[derive(Debug, Clone, PartialEq, Eq)] +pub struct ChatMessage { + pub id: EventId, + pub author: PublicKey, + pub channel: ChannelId, + pub epoch: Epoch, + pub kind: Kind, + pub content: String, + pub reply_to: Option, + pub thread_root: Option, + pub at_ms: u64, + pub expiration: Option, + pub edited_at: Option, + pub deleted: bool, + pub reactions: BTreeMap, +} + +pub fn build_message( + author: PublicKey, + channel: &ChannelId, + epoch: Epoch, + content: &str, + quote: Option<&ReplyRef>, + at_ms: u64, +) -> UnsignedEvent { + let mut tags = channel_binding_tags(channel, epoch); + + if let Some(quote) = quote { + tags.push(reply_tag(TAG_QUOTE, quote)); + } + + build_rumor_ms(KIND_MESSAGE, author, content, tags, at_ms) +} + +/// A NIP-22 comment. `parent` is the immediate parent and `root` the thread's +/// immutable root; `None` means the parent is itself the root. +pub fn build_comment( + author: PublicKey, + channel: &ChannelId, + epoch: Epoch, + content: &str, + parent: &Target, + root: Option<&Target>, + at_ms: u64, +) -> UnsignedEvent { + let root = root.unwrap_or(parent); + let mut tags = channel_binding_tags(channel, epoch); + + tags.push(Tag::custom(TAG_ROOT_KIND, [root.kind.to_string()])); + tags.push(reply_tag(TAG_ROOT, &root.reply)); + if let Some(root_author) = root.reply.author { + tags.push(Tag::custom(TAG_ROOT_AUTHOR, [root_author.to_hex()])); + } + + tags.push(Tag::custom(TAG_TARGET_KIND, [parent.kind.to_string()])); + tags.push(reply_tag(TAG_TARGET, &parent.reply)); + if let Some(parent_author) = parent.reply.author { + tags.push(Tag::custom(TAG_TARGET_AUTHOR, [parent_author.to_hex()])); + } + + build_rumor_ms(KIND_COMMENT, author, content, tags, at_ms) +} + +pub fn build_reaction( + author: PublicKey, + channel: &ChannelId, + epoch: Epoch, + target: &Target, + emoji: &str, + at_ms: u64, +) -> UnsignedEvent { + let mut tags = channel_binding_tags(channel, epoch); + + tags.push(Tag::custom(TAG_TARGET, [target.reply.id.to_hex()])); + if let Some(target_author) = target.reply.author { + tags.push(Tag::custom(TAG_TARGET_AUTHOR, [target_author.to_hex()])); + } + tags.push(Tag::custom(TAG_TARGET_KIND, [target.kind.to_string()])); + + build_rumor_ms(KIND_REACTION, author, emoji, tags, at_ms) +} + +pub fn build_edit( + author: PublicKey, + channel: &ChannelId, + epoch: Epoch, + target: EventId, + content: &str, + at_ms: u64, +) -> UnsignedEvent { + let mut tags = channel_binding_tags(channel, epoch); + tags.push(Tag::custom(TAG_TARGET, [target.to_hex()])); + + build_rumor_ms(KIND_EDIT, author, content, tags, at_ms) +} + +pub fn build_delete( + author: PublicKey, + channel: &ChannelId, + epoch: Epoch, + target: EventId, + target_kind: Option, + at_ms: u64, +) -> UnsignedEvent { + let mut tags = channel_binding_tags(channel, epoch); + tags.push(Tag::custom(TAG_TARGET, [target.to_hex()])); + + if let Some(target_kind) = target_kind { + tags.push(Tag::custom(TAG_TARGET_KIND, [target_kind.to_string()])); + } + + build_rumor_ms(KIND_DELETE, author, "", tags, at_ms) +} + +pub fn build_typing( + author: PublicKey, + channel: &ChannelId, + epoch: Epoch, + at_ms: u64, +) -> UnsignedEvent { + build_rumor_ms( + KIND_TYPING, + author, + "", + channel_binding_tags(channel, epoch), + at_ms, + ) +} + +/// Seals a chat rumor and wraps it at the channel's address. `ephemeral` picks +/// the 21059 wrap, which relays must not store. +pub fn seal_rumor( + rumor: &UnsignedEvent, + group: &GroupKey, + author: &Keys, + ephemeral: bool, +) -> Result<(Event, Keys), ChatError> { + let kind = rumor.kind.as_u16(); + + if !is_chat_kind(kind) { + return Err(ChatError::UnknownKind(kind)); + } + + let seal = build_seal(rumor, SealForm::Encrypted, group, author)?; + let wrap_kind = if ephemeral { + KIND_WRAP_EPHEMERAL + } else { + KIND_WRAP + }; + + // CORD-08 §2: a NIP-40 expiration rides the wrap as well, so relays drop the + // stored event on schedule; the inner copy is what drives a local purge. + let expiration: Vec = rumor + .tags + .iter() + .filter(|tag| tag.as_slice().first().map(String::as_str) == Some(TAG_EXPIRATION)) + .cloned() + .collect(); + + Ok(wrap_seal( + &seal, + group, + wrap_kind, + rumor.created_at, + &expiration, + )?) +} + +/// Opens a wrap against the plane whose key is tried. The channel and epoch the +/// rumor claims must both be the ones that opened it, so a keyholder of two +/// planes cannot re-seal a rumor elsewhere or replay it across an epoch. +pub fn open( + wrap: &Event, + group: &GroupKey, + channel: &ChannelId, + epoch: Epoch, +) -> Result<(OpenedStream, ChatRumor), ChatError> { + let opened = open_wrap(wrap, group)?; + + if opened.seal_form != SealForm::Encrypted { + return Err(ChatError::NotEncryptedSealed); + } + + check_channel_binding(&opened.rumor, channel, epoch)?; + + let chat = typed(&opened.rumor, channel, epoch)?; + + Ok((opened, chat)) +} + +/// Every epoch's group key for one channel. `secret` is whatever feeds the +/// channel at that epoch: the `community_root` for a public one, its own key +/// for a private one. +pub fn plane_keys( + held: &[(Epoch, [u8; 32])], + channel: &ChannelId, +) -> Result> { + held.iter() + .map(|(epoch, secret)| Ok((*epoch, channel_group_key(secret, channel, *epoch)?))) + .collect() +} + +/// Folds the chat plane into timeline rows, newest first. A delete is honored +/// only from the message's own author, and a deletion is terminal: an edit or a +/// reaction arriving later never revives it. +pub fn fold(rumors: &[ChatRumor]) -> Vec { + let mut order: Vec = (0..rumors.len()).collect(); + order.sort_by_key(|&index| (rumors[index].at_ms, rumors[index].id)); + + let mut messages: Vec = Vec::new(); + let mut slot: BTreeMap = BTreeMap::new(); + + for index in order { + let rumor = &rumors[index]; + + let ChatAction::Message { + reply_to, + thread_root, + } = &rumor.action + else { + continue; + }; + + slot.insert(rumor.id, messages.len()); + messages.push(ChatMessage { + id: rumor.id, + author: rumor.author, + channel: rumor.channel, + epoch: rumor.epoch, + kind: rumor.kind, + content: rumor.content.clone(), + reply_to: reply_to.map(|reply| reply.id), + thread_root: thread_root.map(|reply| reply.id), + at_ms: rumor.at_ms, + expiration: rumor.expiration, + edited_at: None, + deleted: false, + reactions: BTreeMap::new(), + }); + } + + // Mutations replay so the last one applied is the winner: the highest + // `at_ms` and, between equal ones, the lower inner rumor id. + let mut mutations: Vec = (0..rumors.len()).collect(); + mutations.sort_by_key(|&index| (rumors[index].at_ms, Reverse(rumors[index].id))); + + for index in mutations { + let rumor = &rumors[index]; + + match &rumor.action { + ChatAction::Edit { target, content } => { + let Some(&slot) = slot.get(target) else { + continue; + }; + let message = &mut messages[slot]; + + if message.deleted || message.author != rumor.author { + continue; + } + + message.content = content.clone(); + message.edited_at = Some(rumor.at_ms); + } + ChatAction::Delete { target, .. } => { + let Some(&slot) = slot.get(target) else { + continue; + }; + + if messages[slot].author == rumor.author { + messages[slot].deleted = true; + } + } + ChatAction::Reaction { target, emoji } => { + let Some(&slot) = slot.get(target) else { + continue; + }; + + messages[slot].reactions.insert(rumor.author, emoji.clone()); + } + ChatAction::Message { .. } | ChatAction::Typing | ChatAction::Opaque => {} + } + } + + messages.sort_by_key(|message| (Reverse(message.at_ms), message.id)); + + messages +} + +fn is_chat_kind(kind: u16) -> bool { + matches!( + kind, + KIND_MESSAGE + | KIND_COMMENT + | KIND_REACTION + | KIND_DELETE + | KIND_EDIT + | KIND_FILE + | KIND_WEBXDC + | KIND_TYPING + ) +} + +fn typed(rumor: &UnsignedEvent, channel: &ChannelId, epoch: Epoch) -> Result { + Ok(ChatRumor { + id: rumor.id.unwrap_or_else(|| rumor.compute_id()), + author: rumor.pubkey, + kind: rumor.kind, + channel: *channel, + epoch, + at_ms: resolve_ms_strict(rumor)?, + content: rumor.content.clone(), + expiration: expiration_of(rumor)?, + action: action_of(rumor)?, + }) +} + +fn action_of(rumor: &UnsignedEvent) -> Result { + let kind = rumor.kind.as_u16(); + + match kind { + KIND_MESSAGE | KIND_FILE => Ok(ChatAction::Message { + reply_to: optional_reply(rumor, TAG_QUOTE)?, + thread_root: None, + }), + KIND_COMMENT => Ok(ChatAction::Message { + reply_to: optional_reply(rumor, TAG_TARGET)?, + thread_root: optional_reply(rumor, TAG_ROOT)?, + }), + KIND_REACTION => Ok(ChatAction::Reaction { + target: required_id(rumor, TAG_TARGET)?, + emoji: rumor.content.clone(), + }), + KIND_EDIT => Ok(ChatAction::Edit { + target: required_id(rumor, TAG_TARGET)?, + content: rumor.content.clone(), + }), + KIND_DELETE => Ok(ChatAction::Delete { + target: required_id(rumor, TAG_TARGET)?, + target_kind: optional_kind(rumor, TAG_TARGET_KIND)?, + }), + KIND_TYPING => Ok(ChatAction::Typing), + KIND_WEBXDC => Ok(ChatAction::Opaque), + other => Err(ChatError::UnknownKind(other)), + } +} + +fn optional_reply( + rumor: &UnsignedEvent, + name: &'static str, +) -> Result, ChatError> { + let Some(fields) = tag(rumor, name)? else { + return Ok(None); + }; + + // NIP-C7 `q` and NIP-22 `E`/`e` put a relay hint at index 2 and the + // referenced author at index 3, which is a SHOULD, so absent reads as unknown. + let author = match fields.get(3).map(String::as_str) { + Some(hex) if !hex.is_empty() => Some(pubkey(hex, name)?), + _ => None, + }; + + Ok(Some(ReplyRef { + id: hex_id(fields, name)?, + author, + })) +} + +fn required_id(rumor: &UnsignedEvent, name: &'static str) -> Result { + let fields = tag(rumor, name)?.ok_or(ChatError::MissingTag(name))?; + hex_id(fields, name) +} + +fn optional_kind(rumor: &UnsignedEvent, name: &'static str) -> Result, ChatError> { + let Some(fields) = tag(rumor, name)? else { + return Ok(None); + }; + + let raw = value(fields, name)?; + let kind = canonical_decimal(raw).ok_or(ChatError::BadTag(name))?; + + u16::try_from(kind) + .map(Some) + .map_err(|_| ChatError::BadTag(name)) +} + +fn expiration_of(rumor: &UnsignedEvent) -> Result, ChatError> { + let Some(fields) = tag(rumor, TAG_EXPIRATION)? else { + return Ok(None); + }; + + let seconds = canonical_decimal(value(fields, TAG_EXPIRATION)?) + .ok_or(ChatError::BadTag(TAG_EXPIRATION))?; + + Ok(Some(Timestamp::from_secs(seconds))) +} + +fn reply_tag(name: &str, reply: &ReplyRef) -> Tag { + Tag::custom( + name, + [ + reply.id.to_hex(), + String::new(), + reply + .author + .map(|author| author.to_hex()) + .unwrap_or_default(), + ], + ) +} + +fn tag<'a>( + rumor: &'a UnsignedEvent, + name: &'static str, +) -> Result, ChatError> { + let mut found: Option<&[String]> = None; + + for candidate in rumor.tags.iter() { + let fields = candidate.as_slice(); + + if fields.first().map(String::as_str) != Some(name) { + continue; + } + + if found.is_some() { + return Err(ChatError::DuplicateTag(name)); + } + + found = Some(fields); + } + + Ok(found) +} + +fn value<'a>(fields: &'a [String], name: &'static str) -> Result<&'a str, ChatError> { + fields + .get(1) + .map(String::as_str) + .ok_or(ChatError::BadTag(name)) +} + +fn hex_id(fields: &[String], name: &'static str) -> Result { + let bytes = decode_hex_32(value(fields, name)?).map_err(|_| ChatError::BadTag(name))?; + + EventId::from_slice(&bytes).map_err(|_| ChatError::BadTag(name)) +} + +fn pubkey(hex: &str, name: &'static str) -> Result { + let bytes = decode_hex_32(hex).map_err(|_| ChatError::BadTag(name))?; + + PublicKey::from_slice(&bytes).map_err(|_| ChatError::BadTag(name)) +} + +#[cfg(test)] +mod tests { + use super::*; + + const SECRET: [u8; 32] = [0x2du8; 32]; + const AT: u64 = 1_700_000_000_417; + + fn channel() -> ChannelId { + ChannelId::from_bytes([0x9cu8; 32]) + } + + fn group() -> GroupKey { + channel_group_key(&SECRET, &channel(), Epoch(0)).expect("derives") + } + + fn sealed(rumor: &UnsignedEvent, group: &GroupKey, author: &Keys) -> Event { + seal_rumor(rumor, group, author, false).expect("seals").0 + } + + fn read(rumor: &UnsignedEvent, group: &GroupKey, author: &Keys, epoch: Epoch) -> ChatRumor { + open(&sealed(rumor, group, author), group, &channel(), epoch) + .expect("opens") + .1 + } + + fn target(id: EventId, author: &Keys) -> Target { + Target { + reply: ReplyRef { + id, + author: Some(author.public_key()), + }, + kind: KIND_MESSAGE, + } + } + + #[test] + fn a_second_holder_folds_edits_reactions_and_a_self_delete() { + let alice = Keys::generate(); + let carol = Keys::generate(); + let group = group(); + + let message = build_message(alice.public_key(), &channel(), Epoch(0), "hello", None, AT); + let id = message.compute_id(); + + let rumors = vec![ + read(&message, &group, &alice, Epoch(0)), + read( + &build_reaction( + carol.public_key(), + &channel(), + Epoch(0), + &target(id, &alice), + "🔥", + AT + 1_000, + ), + &group, + &carol, + Epoch(0), + ), + read( + &build_edit( + alice.public_key(), + &channel(), + Epoch(0), + id, + "hello (fixed)", + AT + 2_000, + ), + &group, + &alice, + Epoch(0), + ), + read( + &build_delete( + alice.public_key(), + &channel(), + Epoch(0), + id, + Some(KIND_MESSAGE), + AT + 3_000, + ), + &group, + &alice, + Epoch(0), + ), + ]; + + let folded = fold(&rumors); + + assert_eq!(folded.len(), 1); + assert_eq!(folded[0].id, id); + assert_eq!(folded[0].content, "hello (fixed)"); + assert_eq!(folded[0].edited_at, Some(AT + 2_000)); + assert_eq!( + folded[0].reactions.get(&carol.public_key()), + Some(&"🔥".to_owned()) + ); + assert!(folded[0].deleted); + } + + #[test] + fn an_edit_or_delete_from_another_author_is_ignored() { + let alice = Keys::generate(); + let bob = Keys::generate(); + let group = group(); + + let message = build_message(alice.public_key(), &channel(), Epoch(0), "hello", None, AT); + let id = message.compute_id(); + + let rumors = vec![ + read(&message, &group, &alice, Epoch(0)), + read( + &build_edit( + bob.public_key(), + &channel(), + Epoch(0), + id, + "mine now", + AT + 1_000, + ), + &group, + &bob, + Epoch(0), + ), + read( + &build_delete( + bob.public_key(), + &channel(), + Epoch(0), + id, + Some(KIND_MESSAGE), + AT + 2_000, + ), + &group, + &bob, + Epoch(0), + ), + ]; + + let folded = fold(&rumors); + + assert_eq!(folded.len(), 1); + assert_eq!(folded[0].content, "hello"); + assert_eq!(folded[0].edited_at, None); + assert!(!folded[0].deleted); + } + + #[test] + fn a_comment_carries_its_root_and_its_parent() { + let alice = Keys::generate(); + let bob = Keys::generate(); + let group = group(); + + let root = build_message(alice.public_key(), &channel(), Epoch(0), "root", None, AT); + let root_id = root.compute_id(); + let parent = build_message( + bob.public_key(), + &channel(), + Epoch(0), + "parent", + None, + AT + 1_000, + ); + let parent_id = parent.compute_id(); + + let comment = build_comment( + alice.public_key(), + &channel(), + Epoch(0), + "deep", + &target(parent_id, &bob), + Some(&target(root_id, &alice)), + AT + 2_000, + ); + + assert!(comment.tags.iter().any(|tag| tag.as_slice() == ["K", "9"])); + assert!( + comment + .tags + .iter() + .any(|tag| { tag.as_slice()[0] == "E" && tag.as_slice()[1] == root_id.to_hex() }) + ); + assert!( + comment + .tags + .iter() + .any(|tag| { tag.as_slice()[0] == "e" && tag.as_slice()[1] == parent_id.to_hex() }) + ); + + let rumor = read(&comment, &group, &alice, Epoch(0)); + let ChatAction::Message { + reply_to, + thread_root, + } = &rumor.action + else { + panic!("a comment is a message row") + }; + + assert_eq!(reply_to.map(|reply| reply.id), Some(parent_id)); + assert_eq!(thread_root.map(|root| root.id), Some(root_id)); + } + + #[test] + fn a_rumor_bound_to_another_channel_or_epoch_is_rejected() { + let alice = Keys::generate(); + let group = group(); + + let plain = build_message(alice.public_key(), &channel(), Epoch(0), "hello", None, AT); + assert!( + open( + &sealed(&plain, &group, &alice), + &group, + &channel(), + Epoch(0) + ) + .is_ok() + ); + + // The keyholder re-addresses their own rumor: the binding is judged + // against the plane whose key opened the wrap, never the rumor's claim. + let elsewhere = ChannelId::from_bytes([0xeeu8; 32]); + assert!(matches!( + open( + &sealed(&plain, &group, &alice), + &group, + &elsewhere, + Epoch(0) + ), + Err(ChatError::Stream(StreamError::ChannelMismatch)) + )); + + let stale = build_message(alice.public_key(), &channel(), Epoch(1), "stale", None, AT); + assert!(matches!( + open( + &sealed(&stale, &group, &alice), + &group, + &channel(), + Epoch(0) + ), + Err(ChatError::Stream(StreamError::EpochMismatch)) + )); + + // Chat is encrypted-seal only (CORD-02 §5), and a retired kind is not a + // chat rumor however well-formed it looks. + let seal = build_seal(&plain, SealForm::Plaintext, &group, &alice).expect("seals"); + let (wrap, _) = wrap_seal( + &seal, + &group, + KIND_WRAP, + Timestamp::from_secs(AT / 1000), + &[], + ) + .expect("wraps"); + assert!(matches!( + open(&wrap, &group, &channel(), Epoch(0)), + Err(ChatError::NotEncryptedSealed) + )); + + let ghost = build_rumor_ms( + 3300, + alice.public_key(), + "v1 ghost", + channel_binding_tags(&channel(), Epoch(0)), + AT, + ); + assert!(matches!( + seal_rumor(&ghost, &group, &alice, false), + Err(ChatError::UnknownKind(3300)) + )); + + let mut tags = channel_binding_tags(&channel(), Epoch(0)); + tags.push(Tag::custom(TAG_TARGET, ["ab".repeat(32)])); + tags.push(Tag::custom(TAG_TARGET, ["cd".repeat(32)])); + let ambiguous = build_rumor_ms(KIND_DELETE, alice.public_key(), "", tags, AT); + assert!(matches!( + open( + &sealed(&ambiguous, &group, &alice), + &group, + &channel(), + Epoch(0) + ), + Err(ChatError::DuplicateTag(TAG_TARGET)) + )); + } +} diff --git a/crates/concord/src/edition.rs b/crates/concord/src/edition.rs index 16295ba9..83236b0d 100644 --- a/crates/concord/src/edition.rs +++ b/crates/concord/src/edition.rs @@ -399,7 +399,7 @@ impl From<&ParsedEdition> for EntityHead { /// Every entity's committed head, keyed by coordinate. pub type Floors = BTreeMap<[u8; 32], EntityHead>; -fn canonical_decimal(raw: &str) -> Option { +pub(crate) fn canonical_decimal(raw: &str) -> Option { if raw.is_empty() || !raw.bytes().all(|byte| byte.is_ascii_digit()) { return None; } diff --git a/crates/concord/src/lib.rs b/crates/concord/src/lib.rs index 0e6f96ba..4f38a0dd 100644 --- a/crates/concord/src/lib.rs +++ b/crates/concord/src/lib.rs @@ -1,3 +1,4 @@ +pub mod chat; pub mod control; pub mod derive; pub mod edition; diff --git a/crates/concord/src/store.rs b/crates/concord/src/store.rs index 0348f839..8d31818e 100644 --- a/crates/concord/src/store.rs +++ b/crates/concord/src/store.rs @@ -1,20 +1,23 @@ -use std::collections::BTreeMap; +use std::cmp::Reverse; +use std::collections::{BTreeMap, BTreeSet}; use std::sync::LazyLock; use anyhow::{Result, anyhow}; use nostr_sdk::prelude::*; use serde::{Deserialize, Serialize}; +use crate::chat::{self, ChatRumor, plane_keys}; use crate::control::{ ChannelMetadata, CommunityGenesis, CommunityMetadata, ControlFold, ROOT_EPOCH, }; use crate::derive::control_signer_group_key; use crate::edition::{EntityHead, Floors, ParsedEdition, vsk}; -use crate::stream::OpenedStream; -use crate::{ChannelId, CommunityId, Epoch}; +use crate::stream::{KIND_WRAP_EPHEMERAL, OpenedStream}; +use crate::{ChannelId, CommunityId, Epoch, GroupKey}; static LOCAL_KEYS: LazyLock = LazyLock::new(Keys::generate); +const MAX_PAGES: usize = 8; const CHANNEL_TAG: SingleLetterTag = SingleLetterTag::LOWERCASE_C; const MARK_TAG: SingleLetterTag = SingleLetterTag::LOWERCASE_T; const MARK_VALUE: &str = "concord"; @@ -265,18 +268,179 @@ where } } +pub async fn backfill( + client: &Client, + database: &dyn NostrDatabase, + channel: &ChannelId, + held: &[(Epoch, [u8; 32])], + until: Option, + limit: usize, +) -> Result> { + let planes = plane_keys(held, channel)?; + let authors: Vec = planes.iter().map(|(_, group)| group.pk()).collect(); + + let mut cursor = until; + let mut seen: BTreeSet = BTreeSet::new(); + let mut found: Vec = Vec::new(); + + for _ in 0..MAX_PAGES { + let page = fetch_page(client, &authors, cursor, limit).await?; + + if page.is_empty() { + break; + } + + let (fresh, next) = advance(&page, &planes, channel, cursor, limit, &mut seen); + + for (opened, rumor) in fresh { + cache_rumor(database, channel, &opened).await?; + found.push(rumor); + } + + match next { + Some(next) => cursor = Some(next), + None => break, + } + } + + found.sort_by_key(|rumor| (Reverse(rumor.at_ms), rumor.id)); + found.truncate(limit); + + Ok(found) +} + +fn advance( + page: &BTreeSet, + planes: &[(Epoch, GroupKey)], + channel: &ChannelId, + cursor: Option, + limit: usize, + seen: &mut BTreeSet, +) -> (Vec<(OpenedStream, ChatRumor)>, Option) { + let mut fresh = Vec::new(); + + for wrap in page { + let Some((epoch, group)) = planes.iter().find(|(_, group)| group.pk() == wrap.pubkey) + else { + continue; + }; + + let Ok((opened, rumor)) = chat::open(wrap, group, channel, *epoch) else { + continue; + }; + + if seen.insert(rumor.id) { + fresh.push((opened, rumor)); + } + } + + if fresh.is_empty() || page.len() < limit { + return (fresh, None); + } + + let oldest = page.iter().map(|event| event.created_at).min(); + + match oldest { + Some(oldest) if cursor != Some(oldest) => (fresh, Some(oldest)), + _ => (fresh, None), + } +} + +async fn fetch_page( + client: &Client, + authors: &[PublicKey], + until: Option, + limit: usize, +) -> Result> { + let mut filter = Filter::new() + .kinds([Kind::GiftWrap, Kind::Custom(KIND_WRAP_EPHEMERAL)]) + .authors(authors.iter().copied()) + .limit(limit); + + if let Some(until) = until { + filter = filter.until(until); + } + + Ok(client.fetch_events(filter).await?) +} + #[cfg(test)] mod tests { use nostr_memory::MemoryDatabase; use super::*; use crate::Epoch; + use crate::chat::{build_message, seal_rumor}; use crate::derive::channel_group_key; use crate::stream::{ KIND_WRAP, SealForm, build_rumor_ms, build_seal, channel_binding_tags, open_wrap, wrap_seal, }; const SECRET: [u8; 32] = [0x07u8; 32]; + const NEXT_SECRET: [u8; 32] = [0x11u8; 32]; + + /// What a relay does with an inclusive `until` and a `limit`. + fn serve_page( + relay: &BTreeSet, + cursor: Option, + limit: usize, + ) -> BTreeSet { + let mut events: Vec = relay + .iter() + .filter(|event| cursor.is_none_or(|cursor| event.created_at <= cursor)) + .cloned() + .collect(); + + events.sort_by_key(|event| Reverse(event.created_at)); + events.truncate(limit); + events.into_iter().collect() + } + + #[test] + fn history_pages_back_across_a_rekey() { + let channel = ChannelId::from_bytes([0x9cu8; 32]); + let author = Keys::generate(); + let held = [(Epoch(0), SECRET), (Epoch(1), NEXT_SECRET)]; + let planes = plane_keys(&held, &channel).expect("derives"); + + // Three messages a second apart: a page boundary falls between each. + let base = 1_700_000_000_000; + let mut relay: BTreeSet = BTreeSet::new(); + + for (content, secret, epoch, at_ms) in [ + ("before the rekey", &SECRET, Epoch(0), base), + ("still before", &SECRET, Epoch(0), base + 1_000), + ("after the rekey", &NEXT_SECRET, Epoch(1), base + 2_000), + ] { + let group = channel_group_key(secret, &channel, epoch).expect("derives"); + let rumor = build_message(author.public_key(), &channel, epoch, content, None, at_ms); + relay.insert(seal_rumor(&rumor, &group, &author, false).expect("seals").0); + } + + let mut seen = BTreeSet::new(); + let mut found = Vec::new(); + let mut cursor = None; + + for _ in 0..3 { + let page = serve_page(&relay, cursor, 2); + let (fresh, next) = advance(&page, &planes, &channel, cursor, 2, &mut seen); + + found.extend(fresh.into_iter().map(|(_, rumor)| rumor)); + + match next { + Some(next) => cursor = Some(next), + None => break, + } + } + + found.sort_by_key(|rumor| (Reverse(rumor.at_ms), rumor.id)); + + let contents: Vec<&str> = found.iter().map(|rumor| rumor.content.as_str()).collect(); + assert_eq!( + contents, + ["after the rekey", "still before", "before the rekey"] + ); + } #[test] fn rumors_read_back_after_a_restart() { -- 2.54.0 From ea9abae5543bfb6d80e1baf5a3b4f0fc230cee5a Mon Sep 17 00:00:00 2001 From: Ren Amamiya Date: Thu, 17 Sep 2026 07:30:38 +0700 Subject: [PATCH 07/12] add guestbook and moderation --- PLAN.md | 114 +++- crates/concord/src/chat.rs | 119 ++++- crates/concord/src/control.rs | 85 ++- crates/concord/src/edition.rs | 41 +- crates/concord/src/guestbook.rs | 890 ++++++++++++++++++++++++++++++++ crates/concord/src/lib.rs | 1 + crates/concord/src/store.rs | 4 + 7 files changed, 1196 insertions(+), 58 deletions(-) create mode 100644 crates/concord/src/guestbook.rs diff --git a/PLAN.md b/PLAN.md index 25110cf7..22f691dd 100644 --- a/PLAN.md +++ b/PLAN.md @@ -389,40 +389,91 @@ pub struct Edition<'a> { subkind: &'a str, entity: [u8; 32], content: &'a str, pub struct ControlWriter { pub author: PublicKey, pub read: GroupKey, pub signer: GroupKey } impl ControlWriter { pub fn publish(&self, keys: &Keys, edition: Edition<'_>, at_secs: u64) -> Result<(Event, EntityHead)>; - pub fn set_community_metadata(&self, keys, community_id, metadata, head, at_secs) -> Result<(Event, EntityHead)>; - pub fn set_channel_metadata(&self, keys, channel, metadata, head, at_secs) -> Result<(Event, EntityHead)>; + pub fn set_community_metadata(&self, keys, community_id, metadata, head, + citation: Option, at_secs) -> Result<(Event, EntityHead)>; + pub fn set_channel_metadata(&self, keys, channel, metadata, head, citation, at_secs) -> Result<(Event, EntityHead)>; + pub fn set_role(&self, keys, role: &Role, head, citation, at_secs) -> Result<(Event, EntityHead)>; + pub fn set_grant(&self, keys, community_id, grant: &Grant, head, citation, at_secs) -> Result<(Event, EntityHead)>; + pub fn set_banlist(&self, keys, community_id, banned: &BTreeSet, head, citation, at_secs) + -> Result<(Event, EntityHead)>; } ``` -`keys` is the acting member's own signer: the seal carries their signature, while the wrap is signed by the plane's published `control_pk`. Roles, grants and banlists ride the same `publish`, and their wrappers land with the moderation API (M5). A remote signer (NIP-46) is not yet plumbed — `publish` takes `&Keys`, not a `NostrSigner`. +`keys` is the acting member's own signer: the seal carries their signature, while the wrap is signed by the plane's published `control_pk`. **Every wrapper takes the citation** (M5): a delegated admin edits metadata, roles, grants and the banlist only under a `vac`, so a wrapper that hardcoded `None` would be an owner-only API. Only `publish` is usable without one. A remote signer (NIP-46) is not yet plumbed — every entry point takes `&Keys`, not a `NostrSigner`. Channel keying follows CORD-03 §1: a public channel derives from `community_root` at the base epoch, a private one from its own random key at its own epoch. Public→private is a rekey at the next channel epoch (monotonic, never reset); private→public derives from the base again and the prior private history stays unreadable to later joiners. The store applies only the public declaration and the deletion: the public-to-private flip is ignored until the convert flow (key mint plus cursor rebase) lands, and a channel this client holds no key for is not added at all — it arrives with the invite that carries the key. -### 8.3 Guestbook and member list (`guestbook.rs`) +### 8.3 Guestbook and member list (`guestbook.rs`) — implemented in M5 ```rust +pub const KIND_JOIN_LEAVE: u16 = 3306; +pub const KIND_KICK: u16 = 3309; +pub const KIND_SNAPSHOT: u16 = 3312; +pub const MAX_SNAPSHOT_CHUNK: usize = 400; +pub const MAX_FUTURE_SKEW_MS: u64 = 60 * 60 * 1000; + pub enum GuestbookEntry { Join { member: PublicKey, at_ms: u64, invited_by: Option<(String, String)> }, Leave { member: PublicKey, at_ms: u64 }, Kick { actor: PublicKey, target: PublicKey, at_ms: u64, citation: Option }, Snapshot { refounder: PublicKey, members: Vec, snapshot_id: [u8; 32], chunk: (u32, u32), at_ms: u64 }, } +pub struct GuestbookRumor { pub id: EventId, pub author: PublicKey, pub kind: Kind, pub at_ms: u64, + pub entry: GuestbookEntry } +pub enum MemberState { + Joined { at_ms: u64, invited_by: Option<(String, String)> }, + Left { at_ms: u64 }, + Kicked { at_ms: u64, actor: PublicKey }, +} -pub fn coalesce(events: &[GuestbookEvent], now_ms: u64, snapshot_authority: Option<&PublicKey>, +pub fn build_join(member: PublicKey, invited_by: Option<(&str, &str)>, at_ms: u64) -> UnsignedEvent; +pub fn build_leave(member: PublicKey, at_ms: u64) -> UnsignedEvent; +pub fn build_kick(actor, target: &PublicKey, citation: Option<&AuthorityCitation>, at_ms) -> UnsignedEvent; +pub fn build_snapshot_chunks(refounder, members: &[PublicKey], snapshot_id: [u8; 32], at_ms) + -> Vec; +pub fn seal_rumor(rumor: &UnsignedEvent, group: &GroupKey, author: &Keys) + -> Result<(Event, Keys), GuestbookError>; +pub fn open(wrap: &Event, group: &GroupKey) + -> Result<(OpenedStream, GuestbookRumor), GuestbookError>; + +pub fn coalesce(rumors: &[GuestbookRumor], now_ms: u64, snapshot_authority: Option<&PublicKey>, can_kick: impl Fn(&PublicKey, &PublicKey, Option<&AuthorityCitation>) -> bool) -> BTreeMap; pub fn complete_memberlist(coalesced: &BTreeMap, observed: &BTreeMap, // author → newest ms published - banned: &BTreeSet, banned_at: &BTreeMap, - refound: Option<&Refound>) -> BTreeSet; + granted: &BTreeSet, + banned: &BTreeSet, banned_at: &BTreeMap) + -> BTreeSet; ``` -- Entries dated more than an hour ahead of local time are dropped. An `ms` outside `0..999` drops the entry rather than being interpreted. +- The plane is community-wide, and its key already carries the epoch, so a guestbook rumor binds no + `channel`/`epoch` tags and `GuestbookRumor` carries neither. Coalescing spans every held epoch, which + is what lets a snapshot bridge a Refounding. +- Entries dated more than an hour ahead of local time are dropped. That is a coalesce-time check (`now_ms` + is an argument, so it is testable without a clock); the `ms` range is not — `open` inherits + `open_wrap`'s strict resolution, so an `ms` outside `0..999` drops the entry at parse instead. - Coalesce is per npub, one final state each, by millisecond time, ties on the lower inner rumor id. -- A Kick counts only when its signer holds `KICK` and outranks the target. -- A Snapshot counts only from the npub whose Refounding minted the epoch. There is deliberately no owner fallback. -- The member list is `coalesced Joined ∪ observed authors − banlist`, and observation counts **forward only** (an author re-enters on activity newer than their latest Leave/Kick/Ban). A Grant holder with `ms == 0` is present by construction. +- A Join or Leave is self-signed by construction: `member` comes from the rumor's own author, so there is + no wire field that could name somebody else. +- A Kick counts only where `can_kick(actor, target, citation)` admits it — the roster's `KICK` plus a + strictly higher rank plus a resolvable citation, composed by the caller. A Snapshot counts only from the + npub whose Refounding minted the epoch. There is deliberately no owner fallback. +- `invited_by` is an echo of the optional `invite` tag, never authority, so a malformed or repeated tag + costs the label and not the member's own word. +- `build_snapshot_chunks` is the only snapshot builder: 400 members per event, every chunk sharing one id + and one timestamp. A chunk is independently useful, so `coalesce` seeds each chunk's members at that + chunk's own time and never waits for its siblings. +- The member list is `coalesced Joined ∪ observed authors ∪ Grant holders − banlist`. `granted` is the one + addition to the spec's literal formula: a Grant recipient holds keys, so they are a member even with no + Join and no published activity. It counts as an *unjdated* positive, so any dated Leave, Kick or Ban + beats it. Observation counts forward only: an author re-enters on activity newer than their latest + Leave, Kick or Ban. +- `banned_at` is caller-supplied, and an entry missing from it excludes its npub outright — an empty map + therefore means "every ban is terminal", which is the safe reading. The fold does not yet expose the + banlist head's timestamp, so that plumbing belongs with the registry rather than here. +- The earlier sketch's `refound: Option<&Refound>` argument is dropped: nothing mints a Refound until M7, + and a parameter no caller can fill is a guess. It returns with the refounding that produces one. ### 8.4 Chat plane (`chat.rs`) — implemented in M4 @@ -436,7 +487,7 @@ pub enum ChatAction { Message { reply_to: Option, thread_root: Option }, Reaction { target: EventId, emoji: String }, Edit { target: EventId, content: String }, - Delete { target: EventId, target_kind: Option }, + Delete { target: EventId, target_kind: Option, citation: Option }, Typing, Opaque, } @@ -447,7 +498,8 @@ pub fn build_message(author, channel, epoch, content, quote: Option<&ReplyRef>, pub fn build_comment(author, channel, epoch, content, parent: &Target, root: Option<&Target>, at_ms) -> UnsignedEvent; pub fn build_reaction(author, channel, epoch, target: &Target, emoji: &str, at_ms) -> UnsignedEvent; pub fn build_edit(author, channel, epoch, target: EventId, content: &str, at_ms) -> UnsignedEvent; -pub fn build_delete(author, channel, epoch, target: EventId, target_kind: Option, at_ms) -> UnsignedEvent; +pub fn build_delete(author, channel, epoch, target: EventId, target_kind: Option, + citation: Option<&AuthorityCitation>, at_ms) -> UnsignedEvent; pub fn build_typing(author, channel, epoch, at_ms) -> UnsignedEvent; pub fn seal_rumor(rumor: &UnsignedEvent, group: &GroupKey, author: &Keys, ephemeral: bool) @@ -455,7 +507,9 @@ pub fn seal_rumor(rumor: &UnsignedEvent, group: &GroupKey, author: &Keys, epheme pub fn open(wrap: &Event, group: &GroupKey, channel: &ChannelId, epoch: Epoch) -> Result<(OpenedStream, ChatRumor), ChatError>; pub fn plane_keys(held: &[(Epoch, [u8; 32])], channel: &ChannelId) -> Result>; -pub fn fold(rumors: &[ChatRumor]) -> Vec; +pub fn fold(rumors: &[ChatRumor], + can_delete: impl Fn(&PublicKey, Option<&AuthorityCitation>, &PublicKey) -> bool) + -> Vec; pub struct ChatMessage { pub id: EventId, @@ -480,10 +534,12 @@ pub struct ChatMessage { newest first; mutations replay ascending on `(at_ms, Reverse(id))` so the last one applied wins — the highest `at_ms` and, between equal ones, the lower id. A deletion is terminal: a later edit never revives it. -- **M4 honors a delete only from the message's own author.** A moderator delete (a `vac` - citation under `MANAGE_MESSAGES`) needs the roster, so the builder's `citation`, the - fold's `can_delete` predicate and its tests land with M5. Failing closed here loses a - moderator's reach, never a member's authorship. +- **Since M5 a delete is honored from the message's own author unconditionally, or from anyone + `can_delete` admits.** That predicate is `(actor, citation, target_author)`, composed by the caller + from the roster: a resolvable `vac` citation, `MANAGE_MESSAGES`, and a strictly higher rank than the + author — the delete is the one Chat-plane authority action, which is why the builder takes a citation + and the fold takes a gate. A self-delete never consults the predicate, matching CORD-02 §9's carve-out + that a member's erasure of their own words survives even Dissolution. - `kind 15` is coop's own file-message convention, outside the CORD registry — it is accepted on the read side so a second coop device's files are not dropped, and `send_file` lands with the registry. @@ -543,7 +599,7 @@ pub fn compact(fold, epoch, new_control_root, ...) -> Vec; // re-wrap h Dissolution (CORD-02 §9) also lives here: a chainless, owner-signed `vsk 10` tombstone at `dissolved_group_key(id)`, plaintext-sealed, and a verifier **must** refuse any tombstone whose `eid` is not the community's own id (including the all-zero placeholder — accepting it lets an owner's genuine tombstone for one community be re-wrapped at another of theirs and kill it permanently). On sight the community is sealed read-only: subscriptions halt, nothing new is honored, existing history stays readable, and a member's delete of their own message is still honored. -## 9. Storage (`store.rs`) — local layer implemented in M1, state document in M2, fold bridge in M3 +## 9. Storage (`store.rs`) — local layer implemented in M1, state document in M2, fold bridge in M3, banlist in M5 Three layers, no new storage engine: @@ -587,11 +643,12 @@ pub struct CommunityState { pub channels: Vec, // id, name, private, epoch pub relays: Vec, pub heads: Vec, // entity, version, self_hash, inner id + pub banned: BTreeSet, // the held banlist, fed back into the next fold pub added_at_ms: u64, } ``` -Landed in M2 with exactly the fields genesis can populate: `save_state`/`load_state` and `CommunityState::from_genesis`. Three fields the plan sketched are deliberately absent until something can fill them — `epoch_keys` (needs rekeys, M7), and `guestbook`/`observed`/`banned`/`dissolved` (need the guestbook, M5). `control_pks` keyed by `u64` rather than `Epoch` and `heads` as a `Vec` rather than a `BTreeMap<[u8; 32], _>`, because serde_json cannot use a byte-array map key. +Landed in M2 with exactly the fields genesis can populate: `save_state`/`load_state` and `CommunityState::from_genesis`. Three fields the plan sketched are deliberately absent until something can fill them — `epoch_keys` (needs rekeys, M7), `dissolved` (needs the tombstone, M7), and `observed`/`guestbook` (need the guestbook ingest of §10). `control_pks` keyed by `u64` rather than `Epoch` and `heads` as a `Vec` rather than a `BTreeMap<[u8; 32], _>`, because serde_json cannot use a byte-array map key. `banned` is a `BTreeSet` — serde has no such problem with a `Vec`-like sequence. M3 added the two bridges between this document and the fold: @@ -602,7 +659,7 @@ impl CommunityState { } ``` -`apply_fold` merges channels rather than replacing them, so a locally-held key survives a metadata edit. `banned` is not yet persisted here: `fold_control` takes the held list as an argument and returns the folded one, and the field lands with the moderation API (M5) that first writes it. +`apply_fold` merges channels rather than replacing them, so a locally-held key survives a metadata edit, and **assigns `banned` wholesale** (M5): the fold already retains a withheld list, so its output is the authority and a caller must not merge it by hand. `floors()` and `banned` are the two inputs the next `fold_control` call needs, which makes the state document a fold cache rather than a second source of truth. Writes are debounced (a fold head changes on every edition); reads load once at init. @@ -756,8 +813,9 @@ Each of these has burned a real implementation, or is a documented cross-client - Enforce the NIP-44 65 535-byte cap at every layer before publishing, and the 5-relay / 256-channel / 50-membership / 100-roles / 64-roles-per-member / 500-banlist / 25-pin caps at their ingest and write points. - Lowercase hex only; x-only pubkeys only; no version tag anywhere. - **Enforced in M3:** a Role's `role_id` is its own coordinate and never 0; a Grant's `member` hashes to its coordinate; a `vsk 4` sits at this community's banlist locator; a banned npub's editions are dropped and a grant naming them carries no rank; a revocation carries a citation; the 100-role cap keeps the lowest ids *after* authorization; a below-floor edition is never a candidate. -- **Enforced in M4:** the chat plane's encrypted-seal requirement, at both publish and open; `channel` **and** `epoch` strict-equal to the plane whose key opened the wrap; a retired or unregistered rumor kind rejected on both sides; a target bearing tag that appears twice rejected outright; and a delete honored only from the message's own author. -- Still owed to M8's audit: the byte caps, the 256-channel and 25-pin caps, the guestbook's future-clock and Snapshot rules, and the write-side counterparts. +- **Enforced in M4:** the chat plane's encrypted-seal requirement, at both publish and open; `channel` **and** `epoch` strict-equal to the plane whose key opened the wrap; a retired or unregistered rumor kind rejected on both sides; a target bearing tag that appears twice rejected outright; and a delete from anybody other than the message's own author refused by the caller's gate. +- **Enforced in M5:** a Kick counts only where the roster admits its actor under `KICK` with a strictly higher rank and a resolvable citation; a Snapshot counts only from the epoch's refounder, with no owner fallback; a guestbook entry more than an hour ahead is dropped, and an out-of-range `ms` or a non-verb `3306` entry is malformed, not interpreted; a duplicated `vac` is rejected outright; and a delete is honored from its target's author unconditionally, or from another actor only where `can_delete` admits them. +- Still owed to M8's audit: the byte caps, the 256-channel and 25-pin caps, and the write-side counterparts of the caps the folds already apply. ## 13. Milestones @@ -768,7 +826,7 @@ Each of these has burned a real implementation, or is a documented cross-client | M2 | `edition.rs` + `control.rs` genesis + `store.rs` state document | ✅ `cargo test -p concord` (7 tests): `edition_hash` reproduces the cross-client vector `2daf42e6…`, and a community minted by one holder has both genesis wraps open for a second holder holding only the invite keys, folding to version 1 | | M3 | Control fold + roster + metadata/channels | ✅ `cargo test -p concord` (14 tests): the chain fold, its gaps, fork tiebreak, downgrade refusal and compaction dangle are pinned; the delegation fixpoint resolves outward from the owner and refuses escalation, an unauthorized higher version, rank inversion by republish and an uncited revoke; a community minted by one holder has its metadata and channel edits fold for a second holder from the invite keys alone | | M4 | Chat plane | ✅ `cargo test -p concord` (19 tests): a second holder folds a message's reactions, its author's edit and its author's delete, and ignores an edit or a delete from anybody else; a comment's root and parent survive the wire; a foreign channel, a replayed epoch, a plaintext seal, a retired kind and a duplicated target are each rejected; and history pages backwards across a rekey in order | -| M5 | Guestbook + member list + moderation | join/leave/kick/ban converge to the same memberlist on both clients; every authority rule from §8.1 has a negative test | +| M5 | Guestbook + member list + moderation | ✅ `cargo test -p concord` (23 tests): a second holder folds joins, leaves, a cited kick and a chunked snapshot into one memberlist in either arrival order, with a ban and a Grant deciding the edges; an uncited, unranked or owner-directed kick and a foreign snapshot are dropped; a future-dated entry, a malformed `ms`, a non-verb `3306`, a duplicated `vac` and a bad snapshot chunk are each refused; and a moderator delete lands only under a citation the roster admits, while a self-delete never asks | | M6 | Invites + Community List | link mint → fetch → join round-trips; revoked link refuses; direct invite lands in the recipient's giftwrap inbox via the `k` tag; a second device reconstructs membership from 13302 | | M7 | Rekeys + refounding + dissolution | a removed member stops reading after a rekey; continuity and race rules tested; a tombstone seals the community and a foreign-id tombstone is refused | | M8 | Pins + disappearing messages + hardening | pins verify from a keyless reader's view; expiry is refused at ingest and purged by the sweep; the audit of §12 is complete with a test per bullet | @@ -783,9 +841,13 @@ M3 closed at 14 tests with no dependency change at all, and `Cargo.lock` untouch M4 closed at 19 tests, again with no dependency change and `Cargo.lock` untouched — relay paging is `Client::fetch_events` over the same `NostrDatabase` the cache already used, so nothing new was needed. New: `src/chat.rs` (the whole channel plane) and, in `src/store.rs`, `backfill` plus the pure `advance` page step it is built from, which is what the paging test drives instead of a socket. `edition::canonical_decimal` became `pub(crate)` so the chat tag grammar shares one decimal check. -What M3 still defers, and to what: the **sync engine's paging** driven by `ControlFold.gapped` and the **`chat::handle_notifications` routing fix** (both §10, together with the `concord::init` wiring — no concord wrap can reach that handler until the subscription exists); the **persisted banlist** and `CommunityState.banned` (M5, with the moderation API that writes it); the **role/grant/banlist write wrappers** (M5 — `ControlWriter::publish` already carries them, only the convenience surface is pending); and the **NIP-46 remote signer**, since `publish` takes `&Keys` rather than a `NostrSigner`. +What M3 still defers, and to what: the **sync engine's paging** driven by `ControlFold.gapped` and the **`chat::handle_notifications` routing fix** (both §10, together with the `concord::init` wiring — no concord wrap can reach that handler until the subscription exists); and the **NIP-46 remote signer**, since every writer takes `&Keys` rather than a `NostrSigner`. Its persisted banlist and its role/grant/banlist write wrappers both landed in M5. -What M4 still defers, and to what: **moderator deletes and the `can_delete` predicate** (M5 — M4 honors a delete only from the message's own author, so a moderator's reach is missing rather than forged); **`media`/`mentions`** on `ChatMessage` and **`send_file`** (the registry/UI milestone — the first needs a gpui type and the second needs the blob-upload path); and **the timer's policy** under the `expiration` tag that `seal_rumor` already mirrors (M8). +What M4 still defers, and to what: **`media`/`mentions`** on `ChatMessage` and **`send_file`** (the registry/UI milestone — the first needs a gpui type and the second needs the blob-upload path); and **the timer's policy** under the `expiration` tag that `seal_rumor` already mirrors (M8). Its moderator delete landed in M5. + +M5 closed at 23 tests, again with no dependency change and `Cargo.lock` untouched. New: `src/guestbook.rs` (the whole membership plane — three rumor codecs, the coalesce, the memberlist), a `vac` citation on a kind-5 delete plus the fold's `can_delete` gate, `ControlWriter::set_role`/`set_grant`/`set_banlist`, and `CommunityState.banned`. Two shared helpers moved into `edition.rs` — `citation_tag` and `citation_from` — so the control, chat and guestbook grammars parse one `vac`. Every metadata/role/grant/banlist wrapper now takes its citation, which is a fix rather than an addition: M3's wrappers hardcoded `None` and so were owner-only, which the delegated-metadata test had been working around with `publish`. + +What M5 still defers, and to what: the **guestbook's fetch and ingest path** — `coalesce` is fed wraps by its caller, so the plane has no `backfill` twin of `chat::backfill` until §10's sync engine needs one, and no `CommunityState.observed` to persist what it would learn; the **banlist head's timestamp** that `complete_memberlist`'s `banned_at` wants (the fold does not surface it, so an empty map means "every ban is terminal" until the registry plumbs it); and the **`Refound` seed** argument to `complete_memberlist`, which waits for M7 to mint one. **M2's "created and published" is verified offline**: "published" is the two wraps existing and being openable by the invite keys, not a relay round-trip. There is no registry to publish through until §10, and a relay test would be testing the SDK, not the protocol. diff --git a/crates/concord/src/chat.rs b/crates/concord/src/chat.rs index 067bcccf..0d9a8a1d 100644 --- a/crates/concord/src/chat.rs +++ b/crates/concord/src/chat.rs @@ -6,7 +6,9 @@ use anyhow::Result; use nostr_sdk::prelude::*; use crate::derive::channel_group_key; -use crate::edition::canonical_decimal; +use crate::edition::{ + AuthorityCitation, TAG_CITATION, canonical_decimal, citation_from, citation_tag, +}; use crate::stream::{ KIND_WRAP, KIND_WRAP_EPHEMERAL, OpenedStream, SealForm, StreamError, build_rumor_ms, build_seal, channel_binding_tags, check_channel_binding, open_wrap, resolve_ms_strict, @@ -96,6 +98,7 @@ pub enum ChatAction { Delete { target: EventId, target_kind: Option, + citation: Option, }, Typing, Opaque, @@ -217,6 +220,7 @@ pub fn build_delete( epoch: Epoch, target: EventId, target_kind: Option, + citation: Option<&AuthorityCitation>, at_ms: u64, ) -> UnsignedEvent { let mut tags = channel_binding_tags(channel, epoch); @@ -226,6 +230,10 @@ pub fn build_delete( tags.push(Tag::custom(TAG_TARGET_KIND, [target_kind.to_string()])); } + if let Some(citation) = citation { + tags.push(citation_tag(citation)); + } + build_rumor_ms(KIND_DELETE, author, "", tags, at_ms) } @@ -317,10 +325,10 @@ pub fn plane_keys( .collect() } -/// Folds the chat plane into timeline rows, newest first. A delete is honored -/// only from the message's own author, and a deletion is terminal: an edit or a -/// reaction arriving later never revives it. -pub fn fold(rumors: &[ChatRumor]) -> Vec { +pub fn fold( + rumors: &[ChatRumor], + can_delete: impl Fn(&PublicKey, Option<&AuthorityCitation>, &PublicKey) -> bool, +) -> Vec { let mut order: Vec = (0..rumors.len()).collect(); order.sort_by_key(|&index| (rumors[index].at_ms, rumors[index].id)); @@ -356,8 +364,6 @@ pub fn fold(rumors: &[ChatRumor]) -> Vec { }); } - // Mutations replay so the last one applied is the winner: the highest - // `at_ms` and, between equal ones, the lower inner rumor id. let mut mutations: Vec = (0..rumors.len()).collect(); mutations.sort_by_key(|&index| (rumors[index].at_ms, Reverse(rumors[index].id))); @@ -378,12 +384,16 @@ pub fn fold(rumors: &[ChatRumor]) -> Vec { message.content = content.clone(); message.edited_at = Some(rumor.at_ms); } - ChatAction::Delete { target, .. } => { + ChatAction::Delete { + target, citation, .. + } => { let Some(&slot) = slot.get(target) else { continue; }; - if messages[slot].author == rumor.author { + let author = messages[slot].author; + + if author == rumor.author || can_delete(&rumor.author, citation.as_ref(), &author) { messages[slot].deleted = true; } } @@ -454,6 +464,7 @@ fn action_of(rumor: &UnsignedEvent) -> Result { KIND_DELETE => Ok(ChatAction::Delete { target: required_id(rumor, TAG_TARGET)?, target_kind: optional_kind(rumor, TAG_TARGET_KIND)?, + citation: optional_citation(rumor)?, }), KIND_TYPING => Ok(ChatAction::Typing), KIND_WEBXDC => Ok(ChatAction::Opaque), @@ -469,8 +480,7 @@ fn optional_reply( return Ok(None); }; - // NIP-C7 `q` and NIP-22 `E`/`e` put a relay hint at index 2 and the - // referenced author at index 3, which is a SHOULD, so absent reads as unknown. + // NIP-C7 `q` and NIP-22 `E`/`e` put a relay hint at index 2 and the referenced author at index 3. let author = match fields.get(3).map(String::as_str) { Some(hex) if !hex.is_empty() => Some(pubkey(hex, name)?), _ => None, @@ -500,6 +510,16 @@ fn optional_kind(rumor: &UnsignedEvent, name: &'static str) -> Result Result, ChatError> { + let Some(fields) = tag(rumor, TAG_CITATION)? else { + return Ok(None); + }; + + citation_from(fields) + .map(Some) + .ok_or(ChatError::BadTag(TAG_CITATION)) +} + fn expiration_of(rumor: &UnsignedEvent) -> Result, ChatError> { let Some(fields) = tag(rumor, TAG_EXPIRATION)? else { return Ok(None); @@ -646,6 +666,7 @@ mod tests { Epoch(0), id, Some(KIND_MESSAGE), + None, AT + 3_000, ), &group, @@ -654,7 +675,7 @@ mod tests { ), ]; - let folded = fold(&rumors); + let folded = fold(&rumors, |_, _, _| false); assert_eq!(folded.len(), 1); assert_eq!(folded[0].id, id); @@ -698,6 +719,7 @@ mod tests { Epoch(0), id, Some(KIND_MESSAGE), + None, AT + 2_000, ), &group, @@ -706,7 +728,7 @@ mod tests { ), ]; - let folded = fold(&rumors); + let folded = fold(&rumors, |_, _, _| false); assert_eq!(folded.len(), 1); assert_eq!(folded[0].content, "hello"); @@ -851,4 +873,75 @@ mod tests { Err(ChatError::DuplicateTag(TAG_TARGET)) )); } + + #[test] + fn a_moderator_delete_needs_the_roster_and_a_citation() { + let alice = Keys::generate(); + let moderator = Keys::generate(); + let peer = Keys::generate(); + let group = group(); + + let message = read( + &build_message(alice.public_key(), &channel(), Epoch(0), "hello", None, AT), + &group, + &alice, + Epoch(0), + ); + let id = message.id; + let citation = AuthorityCitation { + entity: [0x33; 32], + version: 1, + hash: [0x44; 32], + }; + + let delete = |author: &Keys, citation: Option<&AuthorityCitation>| { + read( + &build_delete( + author.public_key(), + &channel(), + Epoch(0), + id, + Some(KIND_MESSAGE), + citation, + AT + 1_000, + ), + &group, + author, + Epoch(0), + ) + }; + + let can_delete = + |actor: &PublicKey, citation: Option<&AuthorityCitation>, author: &PublicKey| { + actor != author && citation.is_some() && actor == &moderator.public_key() + }; + + let cited = vec![message.clone(), delete(&moderator, Some(&citation))]; + assert!(matches!( + &cited[1].action, + ChatAction::Delete { citation: Some(parsed), .. } if *parsed == citation + )); + assert!( + fold(&cited, can_delete)[0].deleted, + "a cited moderator delete lands" + ); + + let uncited = vec![message.clone(), delete(&moderator, None)]; + assert!( + !fold(&uncited, can_delete)[0].deleted, + "an uncited delete names no rank" + ); + + let peer_delete = vec![message.clone(), delete(&peer, Some(&citation))]; + assert!( + !fold(&peer_delete, can_delete)[0].deleted, + "a peer's delete is not authority" + ); + + let own = vec![message.clone(), delete(&alice, None)]; + assert!( + fold(&own, |_, _, _| false)[0].deleted, + "a self-delete never consults the predicate" + ); + } } diff --git a/crates/concord/src/control.rs b/crates/concord/src/control.rs index 30b56e53..2a6f900d 100644 --- a/crates/concord/src/control.rs +++ b/crates/concord/src/control.rs @@ -5,14 +5,15 @@ use nostr_sdk::prelude::{Event, Keys, PublicKey, Timestamp, UnsignedEvent}; use serde::{Deserialize, Serialize}; use crate::derive::{ - community_id_of, control_group_key, control_signer_group_key, verify_community_id, + banlist_locator, community_id_of, control_group_key, control_signer_group_key, grant_locator, + verify_community_id, }; use crate::edition::{ AuthorityCitation, EditionFields, EditionMeta, EntityHead, Floors, ParsedEdition, build_edition, fold_head, parse_edition, vsk, }; use crate::roles::{ - AuthorityEdition, CommunityRoles, Permissions, Roster, citation_ok, fold_roster, + AuthorityEdition, CommunityRoles, Grant, Permissions, Role, Roster, citation_ok, fold_roster, }; use crate::stream::{KIND_WRAP, SealForm, build_seal, open_wrap_at, wrap_seal_with}; use crate::{ChannelId, CommunityId, Epoch, Extra, GroupKey, random_32}; @@ -220,6 +221,7 @@ impl ControlWriter { community_id: &CommunityId, metadata: &CommunityMetadata, head: Option<&EntityHead>, + citation: Option, at_secs: u64, ) -> Result<(Event, EntityHead)> { let content = encode_metadata(metadata)?; @@ -231,7 +233,7 @@ impl ControlWriter { entity: *community_id.as_bytes(), content: &content, head, - citation: None, + citation, }, at_secs, ) @@ -243,6 +245,7 @@ impl ControlWriter { channel: &ChannelId, metadata: &ChannelMetadata, head: Option<&EntityHead>, + citation: Option, at_secs: u64, ) -> Result<(Event, EntityHead)> { let content = serde_json::to_string(metadata)?; @@ -254,7 +257,79 @@ impl ControlWriter { entity: *channel.as_bytes(), content: &content, head, - citation: None, + citation, + }, + at_secs, + ) + } + + pub fn set_role( + &self, + keys: &Keys, + role: &Role, + head: Option<&EntityHead>, + citation: Option, + at_secs: u64, + ) -> Result<(Event, EntityHead)> { + let content = role.to_content()?; + + self.publish( + keys, + Edition { + subkind: vsk::ROLE, + entity: *role.role_id.as_bytes(), + content: &content, + head, + citation, + }, + at_secs, + ) + } + + pub fn set_grant( + &self, + keys: &Keys, + community_id: &CommunityId, + grant: &Grant, + head: Option<&EntityHead>, + citation: Option, + at_secs: u64, + ) -> Result<(Event, EntityHead)> { + let content = grant.to_content()?; + + self.publish( + keys, + Edition { + subkind: vsk::GRANT, + entity: grant_locator(community_id, &grant.member.to_bytes()), + content: &content, + head, + citation, + }, + at_secs, + ) + } + + pub fn set_banlist( + &self, + keys: &Keys, + community_id: &CommunityId, + banned: &BTreeSet, + head: Option<&EntityHead>, + citation: Option, + at_secs: u64, + ) -> Result<(Event, EntityHead)> { + let entries: Vec = banned.iter().map(PublicKey::to_hex).collect(); + let content = serde_json::to_string(&entries)?; + + self.publish( + keys, + Edition { + subkind: vsk::BANLIST, + entity: banlist_locator(community_id), + content: &content, + head, + citation, }, at_secs, ) @@ -600,6 +675,7 @@ mod tests { ..metadata("coop two") }, Some(community_head), + None, AT + 1, ) .expect("publishes"); @@ -613,6 +689,7 @@ mod tests { ..ChannelMetadata::default() }, Some(channel_head), + None, AT + 2, ) .expect("publishes"); diff --git a/crates/concord/src/edition.rs b/crates/concord/src/edition.rs index 83236b0d..cb5fed73 100644 --- a/crates/concord/src/edition.rs +++ b/crates/concord/src/edition.rs @@ -31,7 +31,7 @@ const TAG_SUBKIND: &str = "vsk"; const TAG_ENTITY: &str = "eid"; const TAG_VERSION: &str = "ev"; const TAG_PREV: &str = "ep"; -const TAG_CITATION: &str = "vac"; +pub const TAG_CITATION: &str = "vac"; #[derive(Debug)] pub enum EditionError { @@ -126,6 +126,29 @@ pub fn edition_hash( Sha256::digest(signing_bytes(entity, version, prev, content)).into() } +pub fn citation_tag(citation: &AuthorityCitation) -> Tag { + Tag::custom( + TAG_CITATION, + [ + HEXLOWER.encode(&citation.entity), + citation.version.to_string(), + HEXLOWER.encode(&citation.hash), + ], + ) +} + +pub fn citation_from(fields: &[String]) -> Option { + if fields.len() != 4 { + return None; + } + + Some(AuthorityCitation { + entity: hex32(&fields[1], TAG_CITATION).ok()?, + version: canonical_decimal(&fields[2])?, + hash: hex32(&fields[3], TAG_CITATION).ok()?, + }) +} + pub fn build_edition(fields: EditionFields<'_>) -> UnsignedEvent { let mut tags = vec![ Tag::custom(TAG_SUBKIND, [fields.subkind]), @@ -138,14 +161,7 @@ pub fn build_edition(fields: EditionFields<'_>) -> UnsignedEvent { } if let Some(citation) = fields.citation { - tags.push(Tag::custom( - TAG_CITATION, - [ - HEXLOWER.encode(&citation.entity), - citation.version.to_string(), - HEXLOWER.encode(&citation.hash), - ], - )); + tags.push(citation_tag(&citation)); } build_rumor_secs( @@ -187,12 +203,7 @@ pub fn parse_edition(rumor: &UnsignedEvent) -> Result Some(AuthorityCitation { - entity: hex32(&fields[1], TAG_CITATION)?, - version: canonical_decimal(&fields[2]).ok_or(EditionError::BadField(TAG_CITATION))?, - hash: hex32(&fields[3], TAG_CITATION)?, - }), - Some(_) => return Err(EditionError::BadField(TAG_CITATION)), + Some(fields) => Some(citation_from(fields).ok_or(EditionError::BadField(TAG_CITATION))?), None => None, }; diff --git a/crates/concord/src/guestbook.rs b/crates/concord/src/guestbook.rs new file mode 100644 index 00000000..95a7db88 --- /dev/null +++ b/crates/concord/src/guestbook.rs @@ -0,0 +1,890 @@ +use std::cmp::Reverse; +use std::collections::{BTreeMap, BTreeSet}; +use std::fmt; + +use anyhow::Result; +use data_encoding::HEXLOWER; +use nostr_sdk::prelude::*; + +use crate::edition::{ + AuthorityCitation, TAG_CITATION, canonical_decimal, citation_from, citation_tag, +}; +use crate::stream::{ + KIND_WRAP, OpenedStream, SealForm, StreamError, build_rumor_ms, build_seal, open_wrap, + wrap_seal, +}; +use crate::{GroupKey, decode_hex_32}; + +pub const KIND_JOIN_LEAVE: u16 = 3306; +pub const KIND_KICK: u16 = 3309; +pub const KIND_SNAPSHOT: u16 = 3312; + +pub const MAX_SNAPSHOT_CHUNK: usize = 400; +pub const MAX_FUTURE_SKEW_MS: u64 = 60 * 60 * 1000; + +const TAG_INVITE: &str = "invite"; +const TAG_TARGET: &str = "p"; +const TAG_SNAP: &str = "snap"; +const TAG_CONTENT: &str = "content"; +const CONTENT_JOIN: &str = "join"; +const CONTENT_LEAVE: &str = "leave"; + +#[derive(Debug)] +pub enum GuestbookError { + Stream(StreamError), + NotEncryptedSealed, + UnknownKind(u16), + MissingTag(&'static str), + DuplicateTag(&'static str), + BadTag(&'static str), +} + +impl fmt::Display for GuestbookError { + fn fmt(&self, f: &mut fmt::Formatter<'_>) -> fmt::Result { + match self { + GuestbookError::Stream(error) => write!(f, "stream: {error}"), + GuestbookError::NotEncryptedSealed => { + write!(f, "guestbook rumor must ride an encrypted seal") + } + GuestbookError::UnknownKind(kind) => { + write!(f, "not a guestbook rumor kind: {kind}") + } + GuestbookError::MissingTag(name) => write!(f, "missing guestbook tag: {name}"), + GuestbookError::DuplicateTag(name) => write!(f, "duplicate guestbook tag: {name}"), + GuestbookError::BadTag(name) => write!(f, "malformed guestbook tag: {name}"), + } + } +} + +impl std::error::Error for GuestbookError {} + +impl From for GuestbookError { + fn from(error: StreamError) -> Self { + GuestbookError::Stream(error) + } +} + +#[derive(Debug, Clone, PartialEq, Eq)] +pub enum GuestbookEntry { + Join { + member: PublicKey, + at_ms: u64, + /// The `(creator, label)` an invite attributed the join to. + invited_by: Option<(String, String)>, + }, + Leave { + member: PublicKey, + at_ms: u64, + }, + Kick { + actor: PublicKey, + target: PublicKey, + at_ms: u64, + citation: Option, + }, + Snapshot { + refounder: PublicKey, + members: Vec, + snapshot_id: [u8; 32], + chunk: (u32, u32), + at_ms: u64, + }, +} + +#[derive(Debug, Clone, PartialEq, Eq)] +pub struct GuestbookRumor { + pub id: EventId, + pub author: PublicKey, + pub kind: Kind, + pub at_ms: u64, + pub entry: GuestbookEntry, +} + +#[derive(Debug, Clone, PartialEq, Eq)] +pub enum MemberState { + Joined { + at_ms: u64, + invited_by: Option<(String, String)>, + }, + Left { + at_ms: u64, + }, + Kicked { + at_ms: u64, + actor: PublicKey, + }, +} + +pub fn build_join( + member: PublicKey, + invited_by: Option<(&str, &str)>, + at_ms: u64, +) -> UnsignedEvent { + let mut tags = Vec::new(); + + if let Some((creator, label)) = invited_by { + tags.push(Tag::custom(TAG_INVITE, [creator, label])); + } + + build_rumor_ms(KIND_JOIN_LEAVE, member, CONTENT_JOIN, tags, at_ms) +} + +pub fn build_leave(member: PublicKey, at_ms: u64) -> UnsignedEvent { + build_rumor_ms(KIND_JOIN_LEAVE, member, CONTENT_LEAVE, Vec::new(), at_ms) +} + +pub fn build_kick( + actor: PublicKey, + target: &PublicKey, + citation: Option<&AuthorityCitation>, + at_ms: u64, +) -> UnsignedEvent { + let mut tags = vec![Tag::custom(TAG_TARGET, [target.to_hex()])]; + + if let Some(citation) = citation { + tags.push(citation_tag(citation)); + } + + build_rumor_ms(KIND_KICK, actor, "", tags, at_ms) +} + +pub fn build_snapshot_chunks( + refounder: PublicKey, + members: &[PublicKey], + snapshot_id: [u8; 32], + at_ms: u64, +) -> Vec { + let chunks: Vec<&[PublicKey]> = members.chunks(MAX_SNAPSHOT_CHUNK).collect(); + let total = chunks.len() as u32; + + chunks + .iter() + .enumerate() + .map(|(index, chunk)| { + let hex: Vec = chunk.iter().map(PublicKey::to_hex).collect(); + let content = format!( + "[{}]", + hex.iter() + .map(|member| format!("\"{member}\"")) + .collect::>() + .join(",") + ); + let tags = vec![Tag::custom( + TAG_SNAP, + [ + HEXLOWER.encode(&snapshot_id), + (index as u32 + 1).to_string(), + total.to_string(), + ], + )]; + + build_rumor_ms(KIND_SNAPSHOT, refounder, &content, tags, at_ms) + }) + .collect() +} + +pub fn seal_rumor( + rumor: &UnsignedEvent, + group: &GroupKey, + author: &Keys, +) -> Result<(Event, Keys), GuestbookError> { + let kind = rumor.kind.as_u16(); + + if !is_guestbook_kind(kind) { + return Err(GuestbookError::UnknownKind(kind)); + } + + let seal = build_seal(rumor, SealForm::Encrypted, group, author)?; + + Ok(wrap_seal(&seal, group, KIND_WRAP, rumor.created_at, &[])?) +} + +pub fn open( + wrap: &Event, + group: &GroupKey, +) -> Result<(OpenedStream, GuestbookRumor), GuestbookError> { + let opened = open_wrap(wrap, group)?; + + if opened.seal_form != SealForm::Encrypted { + return Err(GuestbookError::NotEncryptedSealed); + } + + let entry = entry_of(&opened)?; + let rumor = GuestbookRumor { + id: opened.rumor_id, + author: opened.author, + kind: opened.rumor.kind, + at_ms: opened.at_ms, + entry, + }; + + Ok((opened, rumor)) +} + +pub fn coalesce( + rumors: &[GuestbookRumor], + now_ms: u64, + snapshot_authority: Option<&PublicKey>, + can_kick: impl Fn(&PublicKey, &PublicKey, Option<&AuthorityCitation>) -> bool, +) -> BTreeMap { + let mut states: BTreeMap, MemberState)> = BTreeMap::new(); + let horizon = now_ms.saturating_add(MAX_FUTURE_SKEW_MS); + + for rumor in rumors { + if rumor.at_ms > horizon { + continue; + } + + match &rumor.entry { + GuestbookEntry::Join { + member, + at_ms, + invited_by, + } => offer( + &mut states, + *member, + *at_ms, + rumor.id, + MemberState::Joined { + at_ms: *at_ms, + invited_by: invited_by.clone(), + }, + ), + GuestbookEntry::Leave { member, at_ms } => offer( + &mut states, + *member, + *at_ms, + rumor.id, + MemberState::Left { at_ms: *at_ms }, + ), + GuestbookEntry::Kick { + actor, + target, + at_ms, + citation, + } => { + if !can_kick(actor, target, citation.as_ref()) { + continue; + } + + offer( + &mut states, + *target, + *at_ms, + rumor.id, + MemberState::Kicked { + at_ms: *at_ms, + actor: *actor, + }, + ); + } + GuestbookEntry::Snapshot { + refounder, + members, + at_ms, + .. + } => { + if snapshot_authority != Some(refounder) { + continue; + } + + for member in members { + offer( + &mut states, + *member, + *at_ms, + rumor.id, + MemberState::Joined { + at_ms: *at_ms, + invited_by: None, + }, + ); + } + } + } + } + + states + .into_iter() + .map(|(member, (_, _, state))| (member, state)) + .collect() +} + +pub fn complete_memberlist( + coalesced: &BTreeMap, + observed: &BTreeMap, + granted: &BTreeSet, + banned: &BTreeSet, + banned_at: &BTreeMap, +) -> BTreeSet { + let mut candidates: BTreeSet<&PublicKey> = coalesced.keys().collect(); + candidates.extend(observed.keys()); + candidates.extend(granted.iter()); + + let mut members = BTreeSet::new(); + + for member in candidates { + let mut inclusion = observed.get(member).copied(); + + if let Some(state) = coalesced.get(member) { + match state { + MemberState::Joined { at_ms, .. } => { + inclusion = Some(inclusion.map_or(*at_ms, |seen| seen.max(*at_ms))); + } + MemberState::Left { .. } | MemberState::Kicked { .. } => {} + } + } + + if inclusion.is_none() && granted.contains(member) { + inclusion = Some(0); + } + + let mut exclusion = match coalesced.get(member) { + Some(MemberState::Left { at_ms }) | Some(MemberState::Kicked { at_ms, .. }) => { + Some(*at_ms) + } + _ => None, + }; + + if banned.contains(member) { + exclusion = Some(match banned_at.get(member) { + Some(at_ms) => exclusion.map_or(*at_ms, |seen| seen.max(*at_ms)), + None => u64::MAX, + }); + } + + if let Some(inclusion) = inclusion + && exclusion.is_none_or(|exclusion| inclusion > exclusion) + { + members.insert(*member); + } + } + + members +} + +fn offer( + states: &mut BTreeMap, MemberState)>, + member: PublicKey, + at_ms: u64, + id: EventId, + state: MemberState, +) { + let candidate = (at_ms, Reverse(id)); + + if let Some(existing) = states.get(&member) + && (existing.0, existing.1) >= candidate + { + return; + } + + states.insert(member, (at_ms, Reverse(id), state)); +} + +fn is_guestbook_kind(kind: u16) -> bool { + matches!(kind, KIND_JOIN_LEAVE | KIND_KICK | KIND_SNAPSHOT) +} + +fn entry_of(opened: &OpenedStream) -> Result { + let rumor = &opened.rumor; + let author = opened.author; + let at_ms = opened.at_ms; + + match rumor.kind.as_u16() { + KIND_JOIN_LEAVE => match rumor.content.as_str() { + CONTENT_JOIN => Ok(GuestbookEntry::Join { + member: author, + at_ms, + invited_by: invite_of(rumor), + }), + CONTENT_LEAVE => Ok(GuestbookEntry::Leave { + member: author, + at_ms, + }), + _ => Err(GuestbookError::BadTag(TAG_CONTENT)), + }, + KIND_KICK => Ok(GuestbookEntry::Kick { + actor: author, + target: tagged_pubkey(rumor, TAG_TARGET)?, + at_ms, + citation: optional_citation(rumor)?, + }), + KIND_SNAPSHOT => { + let (snapshot_id, chunk) = snapshot_of(rumor)?; + let members = members_of(&rumor.content)?; + + Ok(GuestbookEntry::Snapshot { + refounder: author, + members, + snapshot_id, + chunk, + at_ms, + }) + } + other => Err(GuestbookError::UnknownKind(other)), + } +} + +fn invite_of(rumor: &UnsignedEvent) -> Option<(String, String)> { + rumor.tags.iter().find_map(|candidate| { + let fields = candidate.as_slice(); + + (fields.len() >= 3 && fields[0] == TAG_INVITE) + .then(|| (fields[1].clone(), fields[2].clone())) + }) +} + +fn members_of(content: &str) -> Result, GuestbookError> { + let entries: Vec = + serde_json::from_str(content).map_err(|_| GuestbookError::BadTag(TAG_CONTENT))?; + + if entries.len() > MAX_SNAPSHOT_CHUNK { + return Err(GuestbookError::BadTag(TAG_SNAP)); + } + + entries + .iter() + .map(|entry| pubkey(entry, TAG_CONTENT)) + .collect() +} + +fn snapshot_of(rumor: &UnsignedEvent) -> Result<([u8; 32], (u32, u32)), GuestbookError> { + let fields = required(rumor, TAG_SNAP)?; + + if fields.len() != 4 { + return Err(GuestbookError::BadTag(TAG_SNAP)); + } + + let snapshot_id = decode_hex_32(&fields[1]).map_err(|_| GuestbookError::BadTag(TAG_SNAP))?; + let index = decimal(&fields[2])?; + let total = decimal(&fields[3])?; + + if index == 0 || index > total { + return Err(GuestbookError::BadTag(TAG_SNAP)); + } + + Ok((snapshot_id, (index, total))) +} + +fn optional_citation(rumor: &UnsignedEvent) -> Result, GuestbookError> { + let Some(fields) = tag(rumor, TAG_CITATION)? else { + return Ok(None); + }; + + citation_from(fields) + .map(Some) + .ok_or(GuestbookError::BadTag(TAG_CITATION)) +} + +fn decimal(raw: &str) -> Result { + canonical_decimal(raw) + .and_then(|value| u32::try_from(value).ok()) + .ok_or(GuestbookError::BadTag(TAG_SNAP)) +} + +fn required<'a>( + rumor: &'a UnsignedEvent, + name: &'static str, +) -> Result<&'a [String], GuestbookError> { + tag(rumor, name)?.ok_or(GuestbookError::MissingTag(name)) +} + +fn tagged_pubkey(rumor: &UnsignedEvent, name: &'static str) -> Result { + pubkey(value(required(rumor, name)?, name)?, name) +} + +fn tag<'a>( + rumor: &'a UnsignedEvent, + name: &'static str, +) -> Result, GuestbookError> { + let mut found: Option<&[String]> = None; + + for candidate in rumor.tags.iter() { + let fields = candidate.as_slice(); + + if fields.first().map(String::as_str) != Some(name) { + continue; + } + + if found.is_some() { + return Err(GuestbookError::DuplicateTag(name)); + } + + found = Some(fields); + } + + Ok(found) +} + +fn value<'a>(fields: &'a [String], name: &'static str) -> Result<&'a str, GuestbookError> { + fields + .get(1) + .map(String::as_str) + .ok_or(GuestbookError::BadTag(name)) +} + +fn pubkey(hex: &str, name: &'static str) -> Result { + let bytes = decode_hex_32(hex).map_err(|_| GuestbookError::BadTag(name))?; + + PublicKey::from_slice(&bytes).map_err(|_| GuestbookError::BadTag(name)) +} + +#[cfg(test)] +mod tests { + use super::*; + use crate::derive::guestbook_group_key; + use crate::stream::build_rumor_secs; + use crate::{CommunityId, Epoch}; + + const ROOT: [u8; 32] = [0x5au8; 32]; + const AT: u64 = 1_700_000_000_000; + + fn community() -> CommunityId { + CommunityId::from_bytes([0x11u8; 32]) + } + + fn group() -> GroupKey { + guestbook_group_key(&ROOT, &community(), Epoch(0)).expect("derives") + } + + fn citation() -> AuthorityCitation { + AuthorityCitation { + entity: [0x33u8; 32], + version: 1, + hash: [0x44u8; 32], + } + } + + fn publish(rumor: &UnsignedEvent, author: &Keys) -> GuestbookRumor { + let wrap = seal_rumor(rumor, &group(), author).expect("seals").0; + + open(&wrap, &group()).expect("opens").1 + } + + #[test] + fn join_leave_kick_and_snapshot_converge_to_one_memberlist() { + let alice = Keys::generate(); + let bob = Keys::generate(); + let carol = Keys::generate(); + let dave = Keys::generate(); + let frank = Keys::generate(); + let grace = Keys::generate(); + let owner = Keys::generate(); + + let survivors: Vec = (0..401).map(|_| Keys::generate().public_key()).collect(); + + let mut rumors = vec![ + publish( + &build_join( + alice.public_key(), + Some((&"ab".repeat(32), "Reddit")), + AT + 1_000, + ), + &alice, + ), + publish(&build_join(bob.public_key(), None, AT + 2_000), &bob), + publish(&build_leave(bob.public_key(), AT + 3_000), &bob), + publish(&build_join(dave.public_key(), None, AT + 4_000), &dave), + publish( + &build_kick( + carol.public_key(), + &dave.public_key(), + Some(&citation()), + AT + 5_000, + ), + &carol, + ), + publish(&build_join(frank.public_key(), None, AT + 7_000), &frank), + ]; + + let snapshot_id = "77".repeat(32); + let chunks = + build_snapshot_chunks(carol.public_key(), &survivors, [0x77u8; 32], AT + 6_000); + assert_eq!(chunks.len(), 2, "401 survivors chunk into two events"); + for (index, chunk) in chunks.iter().enumerate() { + assert!(chunk.tags.iter().any(|tag| tag.as_slice() + == [ + TAG_SNAP, + snapshot_id.as_str(), + &(index + 1).to_string(), + "2" + ])); + rumors.push(publish(chunk, &carol)); + } + + let can_kick = + |actor: &PublicKey, target: &PublicKey, citation: Option<&AuthorityCitation>| { + citation.is_some() && actor == &carol.public_key() && target != &owner.public_key() + }; + + let states = coalesce(&rumors, AT + 8_000, Some(&carol.public_key()), can_kick); + + assert_eq!( + states.get(&alice.public_key()), + Some(&MemberState::Joined { + at_ms: AT + 1_000, + invited_by: Some(("ab".repeat(32), "Reddit".to_owned())), + }) + ); + assert_eq!( + states.get(&bob.public_key()), + Some(&MemberState::Left { at_ms: AT + 3_000 }) + ); + assert_eq!( + states.get(&dave.public_key()), + Some(&MemberState::Kicked { + at_ms: AT + 5_000, + actor: carol.public_key(), + }) + ); + assert!( + survivors + .iter() + .all(|member| matches!(states.get(member), Some(MemberState::Joined { .. }))), + "every chunk seeds its own members" + ); + + let reversed: Vec = rumors.iter().rev().cloned().collect(); + assert_eq!( + coalesce(&reversed, AT + 8_000, Some(&carol.public_key()), can_kick), + states, + "arrival order cannot change the fold" + ); + + let observed = BTreeMap::from([ + (bob.public_key(), AT + 9_000), + (carol.public_key(), AT + 5_000), + ]); + let granted = BTreeSet::from([grace.public_key()]); + let banned = BTreeSet::from([frank.public_key()]); + let banned_at = BTreeMap::from([(frank.public_key(), AT + 8_000)]); + + let members = complete_memberlist(&states, &observed, &granted, &banned, &banned_at); + + let mut expected = BTreeSet::from([ + alice.public_key(), + bob.public_key(), + carol.public_key(), + grace.public_key(), + ]); + expected.extend(survivors.iter().copied()); + + assert_eq!(members, expected); + assert!( + !members.contains(&dave.public_key()), + "a kicked member is out" + ); + assert!( + !members.contains(&frank.public_key()), + "a ban wins over a later join" + ); + } + + #[test] + fn a_kick_or_snapshot_without_authority_is_dropped() { + let moderator = Keys::generate(); + let outsider = Keys::generate(); + let owner = Keys::generate(); + let kicked = Keys::generate(); + let uncited = Keys::generate(); + let unranked = Keys::generate(); + let refounder = Keys::generate(); + let impostor = Keys::generate(); + let seeded = Keys::generate(); + let smuggled = Keys::generate(); + + let can_kick = |actor: &PublicKey, + target: &PublicKey, + citation: Option<&AuthorityCitation>| { + citation.is_some() && actor == &moderator.public_key() && target != &owner.public_key() + }; + + let rumors = vec![ + publish( + &build_kick( + moderator.public_key(), + &kicked.public_key(), + Some(&citation()), + AT, + ), + &moderator, + ), + publish( + &build_kick(moderator.public_key(), &uncited.public_key(), None, AT), + &moderator, + ), + publish( + &build_kick( + outsider.public_key(), + &unranked.public_key(), + Some(&citation()), + AT, + ), + &outsider, + ), + publish( + &build_kick( + moderator.public_key(), + &owner.public_key(), + Some(&citation()), + AT, + ), + &moderator, + ), + ]; + + let states = coalesce(&rumors, AT + 1_000, None, can_kick); + + assert_eq!( + states.get(&kicked.public_key()), + Some(&MemberState::Kicked { + at_ms: AT, + actor: moderator.public_key(), + }) + ); + assert!( + !states.contains_key(&uncited.public_key()), + "a kick cites the Grant it acts under" + ); + assert!( + !states.contains_key(&unranked.public_key()), + "a kick needs KICK" + ); + assert!( + !states.contains_key(&owner.public_key()), + "nobody kicks the owner" + ); + + let by_refounder = build_snapshot_chunks( + refounder.public_key(), + &[seeded.public_key()], + [0x77u8; 32], + AT, + ) + .remove(0); + let by_impostor = build_snapshot_chunks( + impostor.public_key(), + &[smuggled.public_key()], + [0x88u8; 32], + AT, + ) + .remove(0); + + for authority in [None, Some(refounder.public_key())] { + let states = coalesce( + &[ + publish(&by_refounder, &refounder), + publish(&by_impostor, &impostor), + ], + AT + 1_000, + authority.as_ref(), + |_, _, _| true, + ); + + assert_eq!( + states.contains_key(&seeded.public_key()), + authority.is_some(), + "only the epoch's refounder seeds, and there is no owner fallback" + ); + assert!( + !states.contains_key(&smuggled.public_key()), + "a foreign snapshot never seeds" + ); + } + } + + #[test] + fn a_future_entry_a_bad_ms_and_a_malformed_snapshot_are_dropped() { + let member = Keys::generate(); + let moderator = Keys::generate(); + let target = Keys::generate(); + + let future = publish( + &build_join(member.public_key(), None, AT + MAX_FUTURE_SKEW_MS + 1), + &member, + ); + let horizon = publish( + &build_join(member.public_key(), None, AT + MAX_FUTURE_SKEW_MS), + &member, + ); + assert!( + coalesce(&[future], AT, None, |_, _, _| true).is_empty(), + "an entry more than an hour ahead is dropped" + ); + assert_eq!( + coalesce(&[horizon], AT, None, |_, _, _| true).len(), + 1, + "the horizon itself is skew, not forgery" + ); + + let bad_ms = build_rumor_secs( + KIND_JOIN_LEAVE, + member.public_key(), + CONTENT_JOIN, + vec![Tag::custom("ms", ["1000"])], + AT / 1000, + ); + assert!(matches!( + open( + &seal_rumor(&bad_ms, &group(), &member).expect("seals").0, + &group() + ), + Err(GuestbookError::Stream(StreamError::BadMs)) + )); + + let bad_verb = build_rumor_ms(KIND_JOIN_LEAVE, member.public_key(), "maybe", vec![], AT); + assert!(matches!( + open( + &seal_rumor(&bad_verb, &group(), &member).expect("seals").0, + &group() + ), + Err(GuestbookError::BadTag(TAG_CONTENT)) + )); + + let ambiguous = build_rumor_ms( + KIND_KICK, + moderator.public_key(), + "", + vec![ + Tag::custom(TAG_TARGET, [target.public_key().to_hex()]), + citation_tag(&citation()), + citation_tag(&citation()), + ], + AT, + ); + assert!(matches!( + open( + &seal_rumor(&ambiguous, &group(), &moderator) + .expect("seals") + .0, + &group() + ), + Err(GuestbookError::DuplicateTag(TAG_CITATION)) + )); + + for fields in [ + vec![snapshot_id(), "0".to_owned(), "2".to_owned()], + vec![snapshot_id(), "3".to_owned(), "2".to_owned()], + vec![snapshot_id(), "1".to_owned()], + ] { + let rumor = build_rumor_ms( + KIND_SNAPSHOT, + moderator.public_key(), + "[]", + vec![Tag::custom(TAG_SNAP, fields)], + AT, + ); + assert!(matches!( + open( + &seal_rumor(&rumor, &group(), &moderator).expect("seals").0, + &group() + ), + Err(GuestbookError::BadTag(TAG_SNAP)) + )); + } + } + + fn snapshot_id() -> String { + "ab".repeat(32) + } +} diff --git a/crates/concord/src/lib.rs b/crates/concord/src/lib.rs index 4f38a0dd..088dadf6 100644 --- a/crates/concord/src/lib.rs +++ b/crates/concord/src/lib.rs @@ -2,6 +2,7 @@ pub mod chat; pub mod control; pub mod derive; pub mod edition; +pub mod guestbook; pub mod roles; pub mod store; pub mod stream; diff --git a/crates/concord/src/store.rs b/crates/concord/src/store.rs index 8d31818e..21c1bc2b 100644 --- a/crates/concord/src/store.rs +++ b/crates/concord/src/store.rs @@ -118,6 +118,8 @@ pub struct CommunityState { pub relays: Vec, #[serde(default, skip_serializing_if = "Vec::is_empty")] pub heads: Vec, + #[serde(default, skip_serializing_if = "BTreeSet::is_empty")] + pub banned: BTreeSet, pub added_at_ms: u64, } @@ -183,6 +185,7 @@ impl CommunityState { channels, relays, heads, + banned: BTreeSet::new(), added_at_ms, }) } @@ -200,6 +203,7 @@ impl CommunityState { pub fn apply_fold(&mut self, fold: &ControlFold) { self.heads = fold.floors.values().cloned().collect(); + self.banned = fold.banned.clone(); if let Some(community) = &fold.community { self.relays = community -- 2.54.0 From 79a4dd387deda1ea6ae5512c1c3ab240740700d4 Mon Sep 17 00:00:00 2001 From: Ren Amamiya Date: Thu, 17 Sep 2026 07:49:07 +0700 Subject: [PATCH 08/12] add invites and community list --- PLAN.md | 158 ++++++- crates/concord/src/edition.rs | 7 +- crates/concord/src/invite.rs | 810 ++++++++++++++++++++++++++++++++++ crates/concord/src/lib.rs | 2 + crates/concord/src/list.rs | 480 ++++++++++++++++++++ crates/concord/src/stream.rs | 22 +- 6 files changed, 1449 insertions(+), 30 deletions(-) create mode 100644 crates/concord/src/invite.rs create mode 100644 crates/concord/src/list.rs diff --git a/PLAN.md b/PLAN.md index 22f691dd..bf8c6968 100644 --- a/PLAN.md +++ b/PLAN.md @@ -85,7 +85,8 @@ crates/concord/ src/roles.rs CORD-04 §2–§4: permissions, roles, grants, banlist, delegation fixpoint src/guestbook.rs CORD-02 §5: join/leave/kick/snapshot, coalesce, complete memberlist src/chat.rs CORD-03: channel plane — message/edit/delete/reaction builders + message view - src/invite.rs CORD-05: bundle, link, registry, Invite List, Direct Invite + src/invite.rs CORD-05 §1–§3, §6: bundle, link (naddr + fragment), Direct Invite + src/list.rs CORD-02 §8: the Community List — join material, merge, to-self envelope src/rekey.rs CORD-06: blob codec, continuity, refounding, compaction, dissolution src/store.rs local persistence + opened-rumor cache + history queries ``` @@ -555,30 +556,142 @@ pub struct ChatMessage { gpui type, and a protocol crate does not take a UI dependency for a derived field. They land with the first consumer that renders them. -### 8.5 Invites (`invite.rs`) +### 8.5 Invites (`invite.rs`) — implemented in M6 ```rust -pub struct CommunityInvite { community_id, owner, owner_salt, community_root, root_epoch, - control_pk: Option, channels: Vec, - relays: Vec, name: String, icon: Option, - expires_at: Option, creator_npub: Option, label: Option, - extra: Map } -impl CommunityInvite { pub fn validate(&self) -> Result<(), InviteError>; // recompute community_id - pub fn expired(&self, now_ms: u64) -> bool; } +pub const KIND_BUNDLE: u16 = 33301; +pub const KIND_DIRECT_INVITE: u16 = 3313; +pub const FRAGMENT_VERSION: u8 = 4; // the dictionary generation +pub const MAX_BUNDLE_CHANNELS: usize = 256; +pub const MAX_BOOTSTRAP_RELAYS: usize = 3; +pub const MAX_BUNDLE_EPOCH: u64 = 1 << 40; // attacker-set, so bound it -pub fn build_bundle(token: &[u8; 16], link_signer: &Keys, invite: &CommunityInvite) -> Result; // 33301, d = "" -pub fn build_revocation(link_signer: &Keys) -> Result; // vsk 9 +pub struct ChannelGrant { id: ChannelId, key: Option, epoch: Epoch, name: String, extra } +pub struct CommunityInvite { community_id: CommunityId, owner: PublicKey, owner_salt: String, + community_root: String, root_epoch: Epoch, + control_pk: Option, channels: Vec, + relays: Vec, name: String, icon: Option, + expires_at: Option, creator_npub: Option, + label: Option, extra } +impl CommunityInvite { + pub fn from_bundle_json(json: &str) -> Result; // bound, truncate, validate + pub fn validate(&self) -> Result<(), InviteError>; // recompute community_id + pub fn expired(&self, now_ms: u64) -> bool; +} +pub enum BundleState { Live(Box), Revoked } + +pub fn build_bundle_event(link_signer: &Keys, invite: &CommunityInvite, bundle_key: &[u8; 32]) + -> Result; +pub fn build_revocation(link_signer: &Keys) -> Result; +pub fn parse_bundle_event(event: &Event, expected_signer: &PublicKey, bundle_key: &[u8; 32]) + -> Result; + +pub fn stock_relays() -> Vec; +pub fn encode_fragment(token: &[u8; 16], relays: &[String]) -> Result; +pub fn decode_fragment(fragment: &str) -> Result<([u8; 16], Vec), InviteError>; +pub fn bundle_naddr(link_signer: &PublicKey) -> Result; +pub fn build_invite_url(base: &str, link_signer, token, relays) -> Result; +pub struct ParsedInviteLink { link_signer: PublicKey, token: [u8; 16], + bootstrap_relays: Vec, naddr: String } pub fn parse_link(input: &str) -> Result; -pub fn encode_fragment(relays: &[RelayUrl], token: &[u8; 16]) -> String; // version byte 4, flags, ≤ 3 relays, base64url -pub fn decode_fragment(fragment: &str) -> Result<(Vec, [u8; 16]), InviteError>; -pub fn build_direct_invite(receiver: &PublicKey, invite: &CommunityInvite, signer: &UniversalSigner) -> Task>; // 3313 rumor → 13 seal → k-tagged 1059 + +pub fn build_direct_invite(inviter: &Keys, recipient: &PublicKey, invite: &CommunityInvite) + -> Result; +pub fn unwrap_direct_invite(wrap: &Event, recipient: &Keys) + -> Result<(PublicKey, CommunityInvite), InviteError>; ``` -The link rides `naddr` (`Nip19Coordinate` for kind 33301, link signer, empty `d`) in the path and the token + bootstrap relays in the fragment. A fragment is never sent to a server. The bundle is decrypted with `invite_bundle_key(token)`, and the joiner must recompute `community_id` from `owner` + `owner_salt`. +- The link is `…/invite/#`: `Nip19Coordinate` for `(33301, link_signer, "")` in +the path, and `[version][flags][relays?][token:16]` base64url-no-pad in the fragment, which is +never sent to a server. `parse_link` also accepts the domain-agnostic bare `#`. +- **The fragment's byte layout is frozen and golden-tested** against hand-computed base64url: the +stock set is flag `0x01` with zero relay bytes (and exempt from the 3-relay cap, which applies to +explicit entries), otherwise a count then per-relay dictionary id, `0x00 len host` for a +`wss://`-implied literal, or `0xff len url` verbatim. A version this client will not decode is +fatal in **both** directions, since a legacy link would be decoded against the wrong dictionary. +- The bundle is sealed with `invite_bundle_key(token)` used **directly** as the NIP-44 conversation +key (`ConversationKey::new`, not an ECDH pair) — the one place in the protocol where that is so. +- Trust is the `community_id`, which `validate` recomputes from `owner` + `owner_salt`. The bundle +is attacker-reached input, so it is bounded *before* it is used: an over-count of channels, an +epoch past the ceiling, and a secret that is not 32 bytes of hex are all refused up front. +`expires_at` is deliberately not part of `validate`: a parked invite still renders past expiry, +only joining refuses. +- `channels` and `relays` default to empty rather than being required: a bundle vending no channel +keys omits the field entirely, and a required `Vec` would turn a stale read into a join failure. +A channel `key` is `Option`, because a public channel derives from `community_root` and never +carries one. +- The bundle's `name` is a preview, not authority — the Control fold is — so it is not bounded here. +`relays` **is** truncated to the community cap, because a hostile list is a connect storm. +- `parse_bundle_event` re-checks the author, the empty `d` (an absent `d` is that coordinate, so +absent and empty are both accepted and a non-empty one is fatal), the signature, and the `vsk` +marker before it decrypts anything. `vsk 6` is live, `vsk 9` is the tombstone. +- The Direct Invite is a **standard** NIP-59 giftwrap, built by `nip59::GiftWrapBuilder` and read +by `nip59::UnwrappedGift::from_gift_wrap`, both from nostr: the builder does the ephemeral wrap and +the tweaked timestamps, and the reader verifies the wrap, the seal's signature, and the rumor/seal +author bind, which is exactly the gate set the reference implementation hand-rolls. Everything left +is the rumor kind and the bundle's own validation. The wrap carries `["k","3313"]` so a recipient +can index their invites without decrypting their whole giftwrap inbox, and mirrors `expires_at` +as a NIP-40 tag in seconds. +- The earlier sketch's `signer: &UniversalSigner` is `&Keys`, matching every other builder in the +crate: NIP-46 is one deliberate pass, not a per-milestone patch (see §14.9). -Bounds before allocation: reject a bundle with more than 256 channels, truncate the relay list to 5, refuse an expired one. +**The Invite List (13303) and the Registry (vsk 8) are deferred to M7**, together and for the same +reason: nothing in M6 consumes them. A link's signer is held by its caller, so minting, refreshing +and revoking need no document, and a Registry write whose fold does not exist is dead wire. M7 is +where both become load-bearing — the Registry's aggregate is the Public/Private source of truth and +retiring the last live link is what triggers a Refounding. -### 8.6 Rekeys and refoundings (`rekey.rs`) +### 8.6 The Community List (`list.rs`) — implemented in M6 + +The member's own memberships, `13302` replaceable, NIP-44-encrypted to self. + +```rust +pub const KIND_COMMUNITY_LIST: u16 = 13302; +pub const MAX_MEMBERSHIPS: usize = 50; + +pub struct JoinMaterial { community_id: CommunityId, owner: PublicKey, owner_salt: String, + community_root: String, root_epoch: Epoch, + control_pk: Option, control_root: Option, + channels: Vec, relays: Vec, name: String, extra } +pub struct CommunityListEntry { community_id: CommunityId, seed: JoinMaterial, + current: JoinMaterial, added_at: u64, extra } +pub struct Tombstone { community_id: CommunityId, removed_at: u64, extra } +pub struct CommunityList { entries: Vec, tombstones: Vec, extra } +impl CommunityList { + pub fn is_live(&self, community_id: &CommunityId) -> bool; + pub fn fits(&self) -> Result<(), ListError>; // the write gate +} + +pub fn join_material(invite: &CommunityInvite, control_root: Option<&[u8; 32]>) -> JoinMaterial; +pub fn merge(held: CommunityList, incoming: CommunityList) -> CommunityList; +pub fn build_list_event(keys: &Keys, list: &CommunityList) -> Result; +pub fn parse_list_event(keys: &Keys, event: &Event) -> Result; +``` + +- Join material is the bundle's **membership subset**: the link-only fields (icon, expiry, label, +creator) are dropped, and `control_root` is added when the holder is staff, since no bundle carries +it. `join_material` is the one conversion between the two documents. +- The two snapshots solve opposite problems and merge the same way: `seed` keeps the **lower** +`root_epoch` and `current` keeps the **higher**, each anchoring an end of the history so a fresh +device needs no epoch-by-epoch walk. +- An epoch tie breaks on the **lowest canonical bytes of the whole snapshot** — a total order, so +two devices never flap competing republishes. `extra` maps union on the same principle (the lower +canonical value wins a key clash), which keeps the merge order-independent while still preserving +unknown fields. +- `added_at` merges to the newest and `removed_at` likewise, so a re-join legitimately resurrects a +membership while a stale device's republish can never re-add a tombstoned id. **A tombstoned entry +stays in the document** — pruning it would make the merge depend on gossip order — and +`is_live` is what reads the newest of the two timestamps. +- `parse_list_event`'s failure is deliberately meaningful: the caller must read it as "no news" and +merge whatever does arrive, never clobber a populated local list with an absence. +- `fits` is the write gate: over the membership cap or over the NIP-44 plaintext cap, the build is +refused rather than truncating memberships to make it fit. The *join* gate that refuses a 51st +membership is the registry's, and the byte-level cap audit is M8's. +- `13302` is implemented per spec. Vector has retired it for fragmented `33302` (a replaceable kind +holds one event per pubkey, so it cannot shard past the size cap); that remains an interop +follow-up, recorded in §14.1. + +### 8.7 Rekeys and refoundings (`rekey.rs`) ```rust pub enum RekeyScope { Channel(ChannelId), Base } @@ -815,6 +928,7 @@ Each of these has burned a real implementation, or is a documented cross-client - **Enforced in M3:** a Role's `role_id` is its own coordinate and never 0; a Grant's `member` hashes to its coordinate; a `vsk 4` sits at this community's banlist locator; a banned npub's editions are dropped and a grant naming them carries no rank; a revocation carries a citation; the 100-role cap keeps the lowest ids *after* authorization; a below-floor edition is never a candidate. - **Enforced in M4:** the chat plane's encrypted-seal requirement, at both publish and open; `channel` **and** `epoch` strict-equal to the plane whose key opened the wrap; a retired or unregistered rumor kind rejected on both sides; a target bearing tag that appears twice rejected outright; and a delete from anybody other than the message's own author refused by the caller's gate. - **Enforced in M5:** a Kick counts only where the roster admits its actor under `KICK` with a strictly higher rank and a resolvable citation; a Snapshot counts only from the epoch's refounder, with no owner fallback; a guestbook entry more than an hour ahead is dropped, and an out-of-range `ms` or a non-verb `3306` entry is malformed, not interpreted; a duplicated `vac` is rejected outright; and a delete is honored from its target's author unconditionally, or from another actor only where `can_delete` admits them. +- **Enforced in M6:** an invite fragment whose version is not this one is refused in either direction, as is one with a bad count or trailing bytes, and its encoding caps bootstrap relays at three; a bundle past the channel cap, past the epoch ceiling, or carrying a secret that is not 32 bytes of hex is refused before it is used, and one whose `owner` + `owner_salt` does not reproduce its `community_id` is refused outright; a bundle event off its coordinate, off its author, or unsigned is refused, and a tombstone at the coordinate reads as revoked; a Direct Invite's wrap, its seal signature and its rumor/seal author bind are all verified before the bundle is even parsed; and a Community List refuses to build past its membership cap or the NIP-44 plaintext cap. - Still owed to M8's audit: the byte caps, the 256-channel and 25-pin caps, and the write-side counterparts of the caps the folds already apply. ## 13. Milestones @@ -827,7 +941,7 @@ Each of these has burned a real implementation, or is a documented cross-client | M3 | Control fold + roster + metadata/channels | ✅ `cargo test -p concord` (14 tests): the chain fold, its gaps, fork tiebreak, downgrade refusal and compaction dangle are pinned; the delegation fixpoint resolves outward from the owner and refuses escalation, an unauthorized higher version, rank inversion by republish and an uncited revoke; a community minted by one holder has its metadata and channel edits fold for a second holder from the invite keys alone | | M4 | Chat plane | ✅ `cargo test -p concord` (19 tests): a second holder folds a message's reactions, its author's edit and its author's delete, and ignores an edit or a delete from anybody else; a comment's root and parent survive the wire; a foreign channel, a replayed epoch, a plaintext seal, a retired kind and a duplicated target are each rejected; and history pages backwards across a rekey in order | | M5 | Guestbook + member list + moderation | ✅ `cargo test -p concord` (23 tests): a second holder folds joins, leaves, a cited kick and a chunked snapshot into one memberlist in either arrival order, with a ban and a Grant deciding the edges; an uncited, unranked or owner-directed kick and a foreign snapshot are dropped; a future-dated entry, a malformed `ms`, a non-verb `3306`, a duplicated `vac` and a bad snapshot chunk are each refused; and a moderator delete lands only under a citation the roster admits, while a self-delete never asks | -| M6 | Invites + Community List | link mint → fetch → join round-trips; revoked link refuses; direct invite lands in the recipient's giftwrap inbox via the `k` tag; a second device reconstructs membership from 13302 | +| M6 | Invites + Community List | ✅ `cargo test -p concord` (32 tests): the fragment's byte layout is pinned by golden base64url for the stock set, a dictionary mix and a verbatim literal, with a wrong version in either direction, trailing bytes and an over-cap count each fatal; a link round-trips as a full URL and as a bare naddr, and refuses a non-invite; a bundle round-trips while a revocation tombstone reads as revoked, and a wrong token, a squatter's author, a foreign `d`, a forged owner, a malformed secret and an over-cap channel count are each refused; a Direct Invite round-trips to its verified inviter and refuses a stranger's keys and a non-invite rumor; and the Community List keeps the earlier seed and the later current in either merge order, refuses to resurrect a tombstoned id until a newer join outruns it, and rebuilds on a second device with unknown fields intact while refusing an over-cap or oversized list | | M7 | Rekeys + refounding + dissolution | a removed member stops reading after a rekey; continuity and race rules tested; a tombstone seals the community and a foreign-id tombstone is refused | | M8 | Pins + disappearing messages + hardening | pins verify from a keyless reader's view; expiry is refused at ingest and purged by the sweep; the audit of §12 is complete with a test per bullet | @@ -849,11 +963,15 @@ M5 closed at 23 tests, again with no dependency change and `Cargo.lock` untouche What M5 still defers, and to what: the **guestbook's fetch and ingest path** — `coalesce` is fed wraps by its caller, so the plane has no `backfill` twin of `chat::backfill` until §10's sync engine needs one, and no `CommunityState.observed` to persist what it would learn; the **banlist head's timestamp** that `complete_memberlist`'s `banned_at` wants (the fold does not surface it, so an empty map means "every ban is terminal" until the registry plumbs it); and the **`Refound` seed** argument to `complete_memberlist`, which waits for M7 to mint one. +M6 closed at 32 tests, again with no dependency change and `Cargo.lock` untouched — the fragment codec is `data_encoding::BASE64URL_NOPAD` and the Direct Invite is nostr's own `nip59` builder and unwrapper, so nothing new was needed. New: `src/invite.rs` (bundle, link, fragment, Direct Invite) and `src/list.rs` (the Community List). In `stream.rs`, the seal and open primitives were factored out of `seal_content`/`decode_content` as `seal_bytes`/`open_bytes`, so the bundle's raw-token key and the List's to-self envelope share the crate's one NIP-44 shape instead of re-implementing it twice; `edition::TAG_SUBKIND` became public so the invite sub-kind tags do not restate the string. + +What M6 still defers, and to what: the **Invite List (13303) and the Registry (`vsk 8`)**, both to M7 and for the same reason — nothing in M6 consumes them, and a Registry write whose fold does not exist is dead wire, while M7's refounding is what reads the Registry's aggregate as the Public/Private source of truth (see §8.5); the **join gate that refuses a 51st membership** (the registry's, since `fits` protects the write rather than the add); and the byte-level cap audit, which is M8's. + **M2's "created and published" is verified offline**: "published" is the two wraps existing and being openable by the invite keys, not a relay round-trip. There is no registry to publish through until §10, and a relay test would be testing the SDK, not the protocol. ## 14. Open questions and risks -1. **Community List kind.** CORD-02 §8 specifies `13302`, replaceable. Vector has retired it in favour of fragmented `33302`, because a replaceable kind holds one event per pubkey and so cannot shard past the NIP-44 size cap. We implement `13302` per spec, enforce the 50-membership cap and pre-publish size check, and treat `33302` as an interop follow-up. Confirm with Armada before writing the multi-device code. +1. **Community List kind.** CORD-02 §8 specifies `13302`, replaceable. Vector has retired it in favour of fragmented `33302`, because a replaceable kind holds one event per pubkey and so cannot shard past the NIP-44 size cap. **M6 implements `13302` per spec**, with the 50-membership cap and the pre-publish size check as the write gate (`list::CommunityList::fits`); `33302` remains an interop follow-up, so a coop member's list is invisible to a Vector device until it lands. Confirm the sharded form with Armada before writing it. 2. **NIP-42 for stream-authored REQs.** Relays that gate kind 1059 by author (for example `ditto-relay`'s `AUTH_KINDS`) need an AUTH event signed by that plane's derived key. `nostr-sdk`'s `Authenticator` is per-client and returns one identity, while a Concord client holds many plane keys — so this cannot be solved by swapping the authenticator. Vector's answer is a dedicated stream-auth responder installed on the client (`community/v2/streamauth`), primed before any relay interaction, which retains the relay's challenge so plane keys registering later can still answer it: a gating relay challenges once per connection and will not re-challenge an authed one, so a responder attached later never gets the chance. Read that module before deciding; the alternative options remain a per-REQ auth hook upstream or documenting the limitation. 3. **`invite_bundle_key` — resolved in M0.** Appendix A.6 was read in full: the raw HKDF output *is* the NIP-44 conversation key, and the derivation is now pinned by a vector. 4. **`pins_locator` has no upstream vector.** Resolved in M0 by minting one from our own implementation and flagging it self-referential in the test. @@ -861,7 +979,7 @@ What M5 still defers, and to what: the **guestbook's fetch and ingest path** — 6. **Local plaintext state.** §9 records the decision. Revisit only if the local database stops being treated as trusted. 7. **Was a `community_id` ever hashed into a tag?** No — it must never appear on the wire. A lint-style test should assert it appears in no published event other than inside an invite bundle and a direct invite. 8. **The Pin List's message-key disclosure has no public API (M8).** CORD-04 §7 Pins let a keyless reader verify a disclosure, which means revealing one message's NIP-44 keys rather than the plane's conversation key. `nostr`'s `nip44::v2::get_message_keys(conversation_key, nonce)` is a private `fn`, and both public entry points (`encrypt_to_bytes_with_nonce`, `decrypt_to_bytes`) take the whole conversation key — so the expansion has to be reproduced as `hkdf::expand_into(conversation_key, nonce, 76 bytes)` plus ChaCha20 and an HMAC-SHA256, exactly as Vector does, and round-tripped against nostr's own `encrypt` in a test. Read CORD-04 §7 in full at M8 before writing it: the reproduction is only worth it once the exact verification the pin must support is settled, and the alternative is contributing a `pub` message-key accessor upstream (we already track git master, so a patch branch or an upstream PR is viable and strictly better than a reproduction we must keep in sync). -9. **A remote signer is not plumbed.** `ControlWriter::publish` and `stream`'s seal builders all take `&Keys`. NIP-46 is a stated Vector feature (§10's task slots are already cleared on signer change), but making the writers async over a `NostrSigner` is a change to every builder, so it should be one deliberate pass rather than a patch per milestone. Nothing in M4–M8 depends on it except the UX of using a remote signer at all. +9. **A remote signer is not plumbed.** `ControlWriter::publish`, `stream`'s seal builders and the invite builders all take `&Keys`. NIP-46 is a stated Vector feature (§10's task slots are already cleared on signer change), but making the writers async over a `NostrSigner` is a change to every builder, so it should be one deliberate pass rather than a patch per milestone. Nothing in M4–M8 depends on it except the UX of using a remote signer at all. 10. **The fold is not incremental.** `fold_control` re-parses and re-folds the whole control edition window on every call, and each fold is up to `2 × entities + 8` passes. That is fine at the caps the spec sets (100 roles, 400-odd grants) and it is the simplest thing that is correct, but if the sync engine ends up calling it per event rather than per batch, the candidate maps and their parse belong in a cache keyed by edition id. Measure before optimizing. ## 15. Test strategy diff --git a/crates/concord/src/edition.rs b/crates/concord/src/edition.rs index cb5fed73..c5e070f4 100644 --- a/crates/concord/src/edition.rs +++ b/crates/concord/src/edition.rs @@ -13,7 +13,7 @@ pub const KIND_CONTROL: u16 = 3308; const EDITION_LABEL: &[u8] = b"vector-community/v1/edition"; -/// Entity types an edition can address (CORD-02 Appendix B). +/// Entity types an edition can address. pub mod vsk { pub const COMMUNITY_METADATA: &str = "0"; pub const ROLE: &str = "1"; @@ -27,11 +27,12 @@ pub mod vsk { pub const PINS: &str = "11"; } -const TAG_SUBKIND: &str = "vsk"; +pub const TAG_SUBKIND: &str = "vsk"; +pub const TAG_CITATION: &str = "vac"; + const TAG_ENTITY: &str = "eid"; const TAG_VERSION: &str = "ev"; const TAG_PREV: &str = "ep"; -pub const TAG_CITATION: &str = "vac"; #[derive(Debug)] pub enum EditionError { diff --git a/crates/concord/src/invite.rs b/crates/concord/src/invite.rs new file mode 100644 index 00000000..9c8a0b57 --- /dev/null +++ b/crates/concord/src/invite.rs @@ -0,0 +1,810 @@ +use std::fmt; + +use data_encoding::BASE64URL_NOPAD; +use nostr::nips::nip01::Coordinate; +use nostr::nips::nip19::{Nip19, Nip19Coordinate}; +use nostr::nips::nip44::v2::ConversationKey; +use nostr::nips::nip59::{GiftWrapBuilder, UnwrappedGift}; +use nostr_sdk::prelude::*; +use serde::{Deserialize, Serialize}; + +use crate::control::{ImageRef, MAX_RELAYS}; +use crate::derive::{TOKEN_LEN, verify_community_id}; +use crate::edition::{TAG_SUBKIND, vsk}; +use crate::stream::{self, StreamError}; +use crate::{ChannelId, CommunityId, Epoch, Extra, decode_hex_32}; + +pub const KIND_BUNDLE: u16 = 33301; +pub const KIND_DIRECT_INVITE: u16 = 3313; +pub const FRAGMENT_VERSION: u8 = 4; +pub const MAX_BUNDLE_CHANNELS: usize = 256; +pub const MAX_BOOTSTRAP_RELAYS: usize = 3; +pub const MAX_BUNDLE_EPOCH: u64 = 1 << 40; + +const FLAG_STOCK_SET: u8 = 0x01; +const INVITE_PATH: &str = "/invite/"; +const TAG_IDENTIFIER: &str = "d"; +const TAG_EXPIRATION: &str = "expiration"; + +const RELAY_DICT: [&str; 4] = [ + "wss://jskitty.com/nostr", + "wss://asia.vectorapp.io/nostr", + "wss://relay.ditto.pub", + "wss://relay.dreamith.to", +]; + +#[derive(Debug)] +pub enum InviteError { + Stream(StreamError), + Json(String), + BadHex(&'static str), + TooManyChannels(usize), + EpochTooLarge(u64), + OwnerMismatch, + BadFragment(&'static str), + BadVersion(u8), + BadLink(&'static str), + BadEvent(&'static str), + Crypto(String), +} + +impl fmt::Display for InviteError { + fn fmt(&self, f: &mut fmt::Formatter<'_>) -> fmt::Result { + match self { + InviteError::Stream(error) => write!(f, "stream: {error}"), + InviteError::Json(error) => write!(f, "json: {error}"), + InviteError::BadHex(field) => write!(f, "{field} is not 32-byte lowercase hex"), + InviteError::TooManyChannels(count) => { + write!( + f, + "bundle carries {count} channels (cap {MAX_BUNDLE_CHANNELS})" + ) + } + InviteError::EpochTooLarge(epoch) => write!(f, "epoch {epoch} out of range"), + InviteError::OwnerMismatch => { + write!(f, "bundle owner does not reproduce its community_id") + } + InviteError::BadFragment(why) => write!(f, "bad invite fragment: {why}"), + InviteError::BadVersion(version) => { + write!(f, "unsupported invite fragment version {version}") + } + InviteError::BadLink(why) => write!(f, "bad invite link: {why}"), + InviteError::BadEvent(why) => write!(f, "bad invite bundle event: {why}"), + InviteError::Crypto(error) => write!(f, "crypto: {error}"), + } + } +} + +impl std::error::Error for InviteError {} + +impl From for InviteError { + fn from(error: StreamError) -> Self { + InviteError::Stream(error) + } +} + +#[derive(Debug, Clone, PartialEq, Serialize, Deserialize)] +pub struct ChannelGrant { + pub id: ChannelId, + #[serde(default, skip_serializing_if = "Option::is_none")] + pub key: Option, + pub epoch: Epoch, + #[serde(default)] + pub name: String, + #[serde(flatten)] + pub extra: Extra, +} + +#[derive(Debug, Clone, PartialEq, Serialize, Deserialize)] +pub struct CommunityInvite { + pub community_id: CommunityId, + pub owner: PublicKey, + pub owner_salt: String, + pub community_root: String, + pub root_epoch: Epoch, + #[serde(default, skip_serializing_if = "Option::is_none")] + pub control_pk: Option, + #[serde(default, skip_serializing_if = "Vec::is_empty")] + pub channels: Vec, + #[serde(default, skip_serializing_if = "Vec::is_empty")] + pub relays: Vec, + pub name: String, + #[serde(default, skip_serializing_if = "Option::is_none")] + pub icon: Option, + /// Unix **ms**: past it the preview still renders, joining refuses. + #[serde(default, skip_serializing_if = "Option::is_none")] + pub expires_at: Option, + #[serde(default, skip_serializing_if = "Option::is_none")] + pub creator_npub: Option, + #[serde(default, skip_serializing_if = "Option::is_none")] + pub label: Option, + #[serde(flatten)] + pub extra: Extra, +} + +impl CommunityInvite { + /// Parse, bound and validate a decrypted bundle, whichever lane carried it. + pub fn from_bundle_json(json: &str) -> Result { + let mut invite: Self = + serde_json::from_str(json).map_err(|error| InviteError::Json(error.to_string()))?; + + if invite.channels.len() > MAX_BUNDLE_CHANNELS { + return Err(InviteError::TooManyChannels(invite.channels.len())); + } + + invite.relays.truncate(MAX_RELAYS); + invite.validate()?; + + Ok(invite) + } + + pub fn validate(&self) -> Result<(), InviteError> { + if self.channels.len() > MAX_BUNDLE_CHANNELS { + return Err(InviteError::TooManyChannels(self.channels.len())); + } + + for epoch in std::iter::once(self.root_epoch).chain(self.channels.iter().map(|c| c.epoch)) { + if epoch.0 > MAX_BUNDLE_EPOCH { + return Err(InviteError::EpochTooLarge(epoch.0)); + } + } + + let owner_salt = hex32(&self.owner_salt, "owner_salt")?; + hex32(&self.community_root, "community_root")?; + + for channel in &self.channels { + if let Some(key) = &channel.key { + hex32(key, "channel key")?; + } + } + + if !verify_community_id(&self.community_id, &self.owner.to_bytes(), &owner_salt) { + return Err(InviteError::OwnerMismatch); + } + + Ok(()) + } + + pub fn expired(&self, now_ms: u64) -> bool { + self.expires_at.is_some_and(|expires| now_ms > expires) + } +} + +#[derive(Debug, Clone)] +pub enum BundleState { + Live(Box), + Revoked, +} + +pub fn build_bundle_event( + link_signer: &Keys, + invite: &CommunityInvite, + bundle_key: &[u8; 32], +) -> Result { + invite.validate()?; + + let json = serde_json::to_string(invite).map_err(json_error)?; + let content = seal_bundle(bundle_key, &json)?; + + EventBuilder::new(Kind::Custom(KIND_BUNDLE), content) + .tags([empty_identifier(), subkind_tag(vsk::INVITE_LIVE)]) + .finalize(link_signer) + .map_err(crypto_error) +} + +pub fn build_revocation(link_signer: &Keys) -> Result { + EventBuilder::new(Kind::Custom(KIND_BUNDLE), "") + .tags([empty_identifier(), subkind_tag(vsk::INVITE_REVOKED)]) + .finalize(link_signer) + .map_err(crypto_error) +} + +pub fn parse_bundle_event( + event: &Event, + expected_signer: &PublicKey, + bundle_key: &[u8; 32], +) -> Result { + if event.kind.as_u16() != KIND_BUNDLE { + return Err(InviteError::BadEvent("wrong kind")); + } + + if event.pubkey != *expected_signer { + return Err(InviteError::BadEvent("author is not the link signer")); + } + + if first_tag(event, TAG_IDENTIFIER).is_some_and(|identifier| !identifier.is_empty()) { + return Err(InviteError::BadEvent( + "bundle is not at the link's coordinate", + )); + } + + event + .verify() + .map_err(|_| InviteError::BadEvent("signature invalid"))?; + + match first_tag(event, TAG_SUBKIND).as_deref() { + Some(vsk::INVITE_REVOKED) => return Ok(BundleState::Revoked), + Some(vsk::INVITE_LIVE) => {} + _ => return Err(InviteError::BadEvent("unknown or missing bundle marker")), + } + + let json = open_bundle(bundle_key, &event.content)?; + + Ok(BundleState::Live(Box::new( + CommunityInvite::from_bundle_json(&json)?, + ))) +} + +pub fn stock_relays() -> Vec { + RELAY_DICT.iter().map(|relay| relay.to_string()).collect() +} + +pub fn encode_fragment(token: &[u8; TOKEN_LEN], relays: &[String]) -> Result { + let stock = relays == RELAY_DICT; + + let mut bytes = Vec::with_capacity(2 + TOKEN_LEN + relays.len() * 8); + bytes.push(FRAGMENT_VERSION); + + if stock { + bytes.push(FLAG_STOCK_SET); + } else { + bytes.push(0x00); + + let bounded = &relays[..relays.len().min(MAX_BOOTSTRAP_RELAYS)]; + bytes.push(bounded.len() as u8); + + for relay in bounded { + match dict_id(relay) { + Some(id) => bytes.push(id), + None => { + let (lead, literal) = match relay.strip_prefix("wss://") { + Some(host) => (0x00, host), + None => (0xff, relay.as_str()), + }; + + if literal.len() > u8::MAX as usize { + return Err(InviteError::BadFragment("relay too long")); + } + + bytes.extend_from_slice(&[lead, literal.len() as u8]); + bytes.extend_from_slice(literal.as_bytes()); + } + } + } + } + + bytes.extend_from_slice(token); + + Ok(BASE64URL_NOPAD.encode(&bytes)) +} + +pub fn decode_fragment(fragment: &str) -> Result<([u8; TOKEN_LEN], Vec), InviteError> { + let bytes = BASE64URL_NOPAD + .decode(fragment.trim().as_bytes()) + .map_err(|_| InviteError::BadFragment("not base64url"))?; + + let version = *bytes.first().ok_or(InviteError::BadFragment("truncated"))?; + + if version != FRAGMENT_VERSION { + return Err(InviteError::BadVersion(version)); + } + + let flags = *bytes.get(1).ok_or(InviteError::BadFragment("truncated"))?; + + let mut offset = 2; + let mut relays = Vec::new(); + + if flags & FLAG_STOCK_SET != 0 { + relays = stock_relays(); + } else { + let count = *bytes + .get(offset) + .ok_or(InviteError::BadFragment("truncated"))? as usize; + offset += 1; + + if count > MAX_BOOTSTRAP_RELAYS { + return Err(InviteError::BadFragment("too many bootstrap relays")); + } + + for _ in 0..count { + let lead = *bytes + .get(offset) + .ok_or(InviteError::BadFragment("truncated"))?; + offset += 1; + + if (1..=254).contains(&lead) { + if let Some(url) = dict_url(lead) { + relays.push(url.to_string()); + } + + continue; + } + + let len = *bytes + .get(offset) + .ok_or(InviteError::BadFragment("truncated"))? as usize; + offset += 1; + + let end = offset + .checked_add(len) + .ok_or(InviteError::BadFragment("truncated"))?; + + let raw = bytes + .get(offset..end) + .ok_or(InviteError::BadFragment("truncated"))?; + + let text = std::str::from_utf8(raw) + .map_err(|_| InviteError::BadFragment("relay is not utf8"))?; + + relays.push(match lead { + 0x00 => format!("wss://{text}"), + 0xff => text.to_string(), + _ => return Err(InviteError::BadFragment("unknown relay lead byte")), + }); + + offset = end; + } + } + + let end = offset + .checked_add(TOKEN_LEN) + .ok_or(InviteError::BadFragment("truncated"))?; + + let raw = bytes + .get(offset..end) + .ok_or(InviteError::BadFragment("truncated"))?; + + if end != bytes.len() { + return Err(InviteError::BadFragment("trailing bytes")); + } + + let mut token = [0u8; TOKEN_LEN]; + token.copy_from_slice(raw); + + Ok((token, relays)) +} + +pub fn bundle_naddr(link_signer: &PublicKey) -> Result { + let coordinate = Coordinate { + kind: Kind::Custom(KIND_BUNDLE), + public_key: *link_signer, + identifier: String::new(), + }; + + Nip19::Coordinate(Nip19Coordinate { + coordinate, + relays: Vec::new(), + }) + .to_bech32() + .map_err(|_| InviteError::BadLink("invalid naddr")) +} + +pub fn build_invite_url( + base: &str, + link_signer: &PublicKey, + token: &[u8; TOKEN_LEN], + relays: &[String], +) -> Result { + let naddr = bundle_naddr(link_signer)?; + let fragment = encode_fragment(token, relays)?; + + Ok(format!( + "{}{INVITE_PATH}{naddr}#{fragment}", + base.trim_end_matches('/') + )) +} + +#[derive(Debug, Clone)] +pub struct ParsedInviteLink { + /// The bundle coordinate's author. + pub link_signer: PublicKey, + pub token: [u8; TOKEN_LEN], + pub bootstrap_relays: Vec, + /// The bare naddr as it appeared in the link, for the fetch. + pub naddr: String, +} + +pub fn parse_link(input: &str) -> Result { + let (locator, fragment) = input + .trim() + .split_once('#') + .ok_or(InviteError::BadLink("no fragment"))?; + + if fragment.is_empty() { + return Err(InviteError::BadLink("empty fragment")); + } + + let naddr = match locator.find(INVITE_PATH) { + Some(index) => locator[index + INVITE_PATH.len()..].trim_end_matches('/'), + None => locator.trim_start_matches("nostr:"), + }; + + let link_signer = signer_from_naddr(naddr)?; + let (token, bootstrap_relays) = decode_fragment(fragment)?; + + Ok(ParsedInviteLink { + link_signer, + token, + bootstrap_relays, + naddr: naddr.to_string(), + }) +} + +pub fn build_direct_invite( + inviter: &Keys, + recipient: &PublicKey, + invite: &CommunityInvite, +) -> Result { + invite.validate()?; + + let json = serde_json::to_string(invite).map_err(json_error)?; + let rumor = EventBuilder::new(Kind::Custom(KIND_DIRECT_INVITE), json) + .finalize_unsigned(inviter.public_key()); + + let mut tags = vec![Tag::custom("k", [KIND_DIRECT_INVITE.to_string()])]; + + if let Some(expires_at) = invite.expires_at { + tags.push(Tag::custom( + TAG_EXPIRATION, + [(expires_at / 1000).to_string()], + )); + } + + GiftWrapBuilder::new(*recipient, rumor) + .extra_tags(tags) + .finalize(inviter) + .map_err(crypto_error) +} + +pub fn unwrap_direct_invite( + wrap: &Event, + recipient: &Keys, +) -> Result<(PublicKey, CommunityInvite), InviteError> { + let unwrapped = UnwrappedGift::from_gift_wrap(recipient, wrap).map_err(crypto_error)?; + + if unwrapped.rumor.kind.as_u16() != KIND_DIRECT_INVITE { + return Err(InviteError::BadEvent("rumor is not a direct invite")); + } + + Ok(( + unwrapped.sender, + CommunityInvite::from_bundle_json(&unwrapped.rumor.content)?, + )) +} + +fn seal_bundle(bundle_key: &[u8; 32], json: &str) -> Result { + Ok(stream::seal_bytes( + &ConversationKey::new(*bundle_key), + json.as_bytes(), + )?) +} + +fn open_bundle(bundle_key: &[u8; 32], content: &str) -> Result { + let plaintext = stream::open_bytes(&ConversationKey::new(*bundle_key), content)?; + + String::from_utf8(plaintext).map_err(|_| InviteError::BadFragment("bundle is not utf8")) +} + +fn signer_from_naddr(naddr: &str) -> Result { + match Nip19::from_bech32(naddr.trim_start_matches("nostr:")) { + Ok(Nip19::Coordinate(coordinate)) + if coordinate.coordinate.kind.as_u16() == KIND_BUNDLE + && coordinate.coordinate.identifier.is_empty() => + { + Ok(coordinate.coordinate.public_key) + } + _ => Err(InviteError::BadLink( + "naddr is not an invite-bundle coordinate", + )), + } +} + +fn hex32(value: &str, field: &'static str) -> Result<[u8; 32], InviteError> { + decode_hex_32(value).map_err(|_| InviteError::BadHex(field)) +} + +fn dict_id(relay: &str) -> Option { + RELAY_DICT + .iter() + .position(|known| *known == relay) + .map(|index| index as u8 + 1) +} + +fn dict_url(id: u8) -> Option<&'static str> { + RELAY_DICT.get(id.checked_sub(1)? as usize).copied() +} + +fn empty_identifier() -> Tag { + Tag::identifier("") +} + +fn subkind_tag(value: &str) -> Tag { + Tag::custom(TAG_SUBKIND, [value]) +} + +fn first_tag(event: &Event, name: &str) -> Option { + event.tags.iter().find_map(|tag| { + let fields = tag.as_slice(); + + (fields.len() >= 2 && fields[0] == name).then(|| fields[1].clone()) + }) +} + +fn json_error(error: serde_json::Error) -> InviteError { + InviteError::Json(error.to_string()) +} + +fn crypto_error(error: impl fmt::Display) -> InviteError { + InviteError::Crypto(error.to_string()) +} + +#[cfg(test)] +mod tests { + use data_encoding::HEXLOWER; + + use super::*; + use crate::derive::{community_id_of, invite_bundle_key}; + + const SALT: [u8; 32] = [0x33u8; 32]; + + fn bundle() -> CommunityInvite { + let owner = Keys::generate(); + + CommunityInvite { + community_id: community_id_of(&owner.public_key().to_bytes(), &SALT), + owner: owner.public_key(), + owner_salt: HEXLOWER.encode(&SALT), + community_root: "44".repeat(32), + root_epoch: Epoch(0), + control_pk: None, + channels: vec![ChannelGrant { + id: ChannelId::from_bytes([0x9cu8; 32]), + key: Some("55".repeat(32)), + epoch: Epoch(1), + name: "lounge".to_owned(), + extra: Extra::default(), + }], + relays: vec!["wss://relay.example".to_owned()], + name: "Test community".to_owned(), + icon: None, + expires_at: None, + creator_npub: None, + label: None, + extra: Extra::default(), + } + } + + fn token16() -> [u8; TOKEN_LEN] { + std::array::from_fn(|i| i as u8) + } + + #[test] + fn fragment_goldens_pin_the_wire_layout() { + let token = token16(); + + // [04 version][01 stock flag][token 00..0f] + let stock = encode_fragment(&token, &stock_relays()).expect("encodes"); + assert_eq!(stock, "BAEAAQIDBAUGBwgJCgsMDQ4P"); + assert_eq!( + decode_fragment(&stock).expect("decodes"), + (token, stock_relays()) + ); + + // [04][00 flags][02 count][02 dict-id][04 dict-id][token 00..0f] + let mixed = vec![RELAY_DICT[1].to_owned(), RELAY_DICT[3].to_owned()]; + let encoded = encode_fragment(&token, &mixed).expect("encodes"); + assert_eq!(encoded, "BAACAgQAAQIDBAUGBwgJCgsMDQ4P"); + assert_eq!(decode_fragment(&encoded).expect("decodes"), (token, mixed)); + + // [04][00][01 count][ff verbatim lead][06 len]["ws://h"][token 00..0f] + let verbatim = vec!["ws://h".to_owned()]; + let encoded = encode_fragment(&token, &verbatim).expect("encodes"); + assert_eq!(encoded, "BAAB_wZ3czovL2gAAQIDBAUGBwgJCgsMDQ4P"); + assert_eq!( + decode_fragment(&encoded).expect("decodes"), + (token, verbatim) + ); + } + + #[test] + fn a_fragment_is_strict_about_framing_and_counts() { + let token = token16(); + + for version in [3u8, 5] { + let mut bytes = vec![version, FLAG_STOCK_SET]; + bytes.extend_from_slice(&token); + let encoded = BASE64URL_NOPAD.encode(&bytes); + assert!( + matches!(decode_fragment(&encoded), Err(InviteError::BadVersion(v)) if v == version), + "a legacy and a future version are both refused" + ); + } + + let mut trailing = vec![FRAGMENT_VERSION, FLAG_STOCK_SET]; + trailing.extend_from_slice(&token); + trailing.push(0xff); + assert!(matches!( + decode_fragment(&BASE64URL_NOPAD.encode(&trailing)), + Err(InviteError::BadFragment(_)) + )); + + let mut over = vec![FRAGMENT_VERSION, 0x00, 0x04, 1, 2, 3, 4]; + over.extend_from_slice(&token); + assert!(matches!( + decode_fragment(&BASE64URL_NOPAD.encode(&over)), + Err(InviteError::BadFragment(_)) + )); + + // An unknown dictionary id is skipped, not fatal, so the dictionary can grow. + let mut unknown = vec![FRAGMENT_VERSION, 0x00, 0x01, 200]; + unknown.extend_from_slice(&token); + let (decoded, relays) = + decode_fragment(&BASE64URL_NOPAD.encode(&unknown)).expect("decodes"); + assert_eq!(decoded, token); + assert!(relays.is_empty()); + + let relays: Vec = (0..4).map(|i| format!("wss://r{i}.example")).collect(); + let (_, capped) = + decode_fragment(&encode_fragment(&token, &relays).expect("encodes")).expect("decodes"); + assert_eq!(capped.len(), MAX_BOOTSTRAP_RELAYS); + } + + #[test] + fn a_link_round_trips_and_refuses_a_non_invite() { + let link_signer = Keys::generate(); + let token = token16(); + let relays = vec!["wss://a.example".to_owned()]; + + let url = build_invite_url( + "https://vectorapp.io/", + &link_signer.public_key(), + &token, + &relays, + ) + .expect("builds"); + + let parsed = parse_link(&url).expect("parses"); + assert_eq!(parsed.link_signer, link_signer.public_key()); + assert_eq!(parsed.token, token); + assert_eq!(parsed.bootstrap_relays, relays); + + let fragment = url.split('#').nth(1).expect("carries a fragment"); + let bare = format!("{}#{fragment}", parsed.naddr); + let reparsed = parse_link(&bare).expect("parses the domain-agnostic form"); + assert_eq!(reparsed.link_signer, link_signer.public_key()); + assert_eq!(reparsed.token, token); + + assert!( + parse_link("https://x/invite/#frag").is_err(), + "the naddr is not optional" + ); + assert!( + parse_link("wss://relay.example.com").is_err(), + "nor the fragment" + ); + } + + #[test] + fn a_bundle_round_trips_while_a_revocation_reads_as_revoked() { + let invite = bundle(); + let link_signer = Keys::generate(); + let key = invite_bundle_key(&[7u8; TOKEN_LEN]); + + let event = build_bundle_event(&link_signer, &invite, &key).expect("builds"); + assert_eq!(event.pubkey, link_signer.public_key()); + + match parse_bundle_event(&event, &link_signer.public_key(), &key).expect("parses") { + BundleState::Live(opened) => { + assert_eq!(opened.community_id, invite.community_id); + assert_eq!(opened.channels.len(), 1); + } + BundleState::Revoked => panic!("expected a live bundle"), + } + + let revocation = build_revocation(&link_signer).expect("builds"); + assert!(matches!( + parse_bundle_event(&revocation, &link_signer.public_key(), &key), + Ok(BundleState::Revoked) + )); + + // The token is the only way in, and a squatter is a different coordinate. + assert!( + parse_bundle_event( + &event, + &link_signer.public_key(), + &invite_bundle_key(&[8u8; TOKEN_LEN]) + ) + .is_err() + ); + let squatter = Keys::generate(); + assert!(matches!( + parse_bundle_event(&event, &squatter.public_key(), &key), + Err(InviteError::BadEvent(_)) + )); + } + + #[test] + fn a_bundle_off_its_coordinate_or_off_its_owner_is_refused() { + let invite = bundle(); + let link_signer = Keys::generate(); + let key = invite_bundle_key(&[9u8; TOKEN_LEN]); + let json = serde_json::to_string(&invite).expect("serializes"); + let content = seal_bundle(&key, &json).expect("seals"); + + // The fetch filters on the author, so the empty `d` is pinned here: a + // signature-valid event of the same author at another `d` is not the bundle. + let elsewhere = EventBuilder::new(Kind::Custom(KIND_BUNDLE), content) + .tags([Tag::identifier("elsewhere"), subkind_tag(vsk::INVITE_LIVE)]) + .finalize(&link_signer) + .expect("signs"); + assert!(matches!( + parse_bundle_event(&elsewhere, &link_signer.public_key(), &key), + Err(InviteError::BadEvent(_)) + )); + + let mut forged = bundle(); + forged.owner = Keys::generate().public_key(); + assert!(matches!(forged.validate(), Err(InviteError::OwnerMismatch))); + assert!(matches!( + build_bundle_event(&link_signer, &forged, &key), + Err(InviteError::OwnerMismatch) + )); + + let mut malformed = bundle(); + malformed.community_root = "not hex".to_owned(); + assert!(matches!(malformed.validate(), Err(InviteError::BadHex(_)))); + + let mut crowded = bundle(); + crowded.channels = (0..=MAX_BUNDLE_CHANNELS) + .map(|_| ChannelGrant { + id: ChannelId::from_bytes([0x01; 32]), + key: None, + epoch: Epoch(0), + name: String::new(), + extra: Extra::default(), + }) + .collect(); + assert!(matches!( + crowded.validate(), + Err(InviteError::TooManyChannels(n)) if n == MAX_BUNDLE_CHANNELS + 1 + )); + } + + #[test] + fn a_direct_invite_round_trips_and_refuses_a_foreign_rumor() { + let inviter = Keys::generate(); + let recipient = Keys::generate(); + let invite = bundle(); + + let wrap = build_direct_invite(&inviter, &recipient.public_key(), &invite).expect("builds"); + assert_eq!(wrap.kind, Kind::GiftWrap); + assert_ne!( + wrap.pubkey, + inviter.public_key(), + "the wrap author is ephemeral" + ); + assert!( + wrap.tags.iter().any(|tag| tag.as_slice() == ["k", "3313"]), + "the k tag is what makes an invite indexable" + ); + + let (sender, opened) = unwrap_direct_invite(&wrap, &recipient).expect("unwraps"); + assert_eq!(sender, inviter.public_key()); + assert_eq!(opened.community_id, invite.community_id); + + // Somebody else's wrap is not ours to open... + let stranger = Keys::generate(); + assert!(unwrap_direct_invite(&wrap, &stranger).is_err()); + + // ...and a wrap that opens to some other kind is not an invite. + let rumor = EventBuilder::new(Kind::Custom(crate::chat::KIND_MESSAGE), "hello") + .finalize_unsigned(recipient.public_key()); + let wrap = GiftWrapBuilder::new(recipient.public_key(), rumor) + .finalize(&recipient) + .expect("wraps"); + assert!(matches!( + unwrap_direct_invite(&wrap, &recipient), + Err(InviteError::BadEvent(_)) + )); + } +} diff --git a/crates/concord/src/lib.rs b/crates/concord/src/lib.rs index 088dadf6..da4bb3cd 100644 --- a/crates/concord/src/lib.rs +++ b/crates/concord/src/lib.rs @@ -3,6 +3,8 @@ pub mod control; pub mod derive; pub mod edition; pub mod guestbook; +pub mod invite; +pub mod list; pub mod roles; pub mod store; pub mod stream; diff --git a/crates/concord/src/list.rs b/crates/concord/src/list.rs new file mode 100644 index 00000000..0c291440 --- /dev/null +++ b/crates/concord/src/list.rs @@ -0,0 +1,480 @@ +use std::collections::BTreeMap; +use std::collections::btree_map::Entry; +use std::fmt; + +use nostr::nips::nip44::{self, Version}; +use nostr_sdk::prelude::*; +use serde::{Deserialize, Serialize}; + +use crate::invite::{ChannelGrant, CommunityInvite}; +use crate::stream::NIP44_MAX_PLAINTEXT; +use crate::{CommunityId, Epoch, Extra}; + +pub const KIND_COMMUNITY_LIST: u16 = 13302; +pub const MAX_MEMBERSHIPS: usize = 50; + +#[derive(Debug)] +pub enum ListError { + Kind(u16), + Crypto(String), + Json(String), + TooManyMemberships(usize), + Oversize(usize), +} + +impl fmt::Display for ListError { + fn fmt(&self, f: &mut fmt::Formatter<'_>) -> fmt::Result { + match self { + ListError::Kind(kind) => write!(f, "not a community list kind: {kind}"), + ListError::Crypto(error) => write!(f, "crypto: {error}"), + ListError::Json(error) => write!(f, "json: {error}"), + ListError::TooManyMemberships(count) => { + write!( + f, + "list carries {count} memberships (cap {MAX_MEMBERSHIPS})" + ) + } + ListError::Oversize(len) => { + write!(f, "list is {len} bytes (cap {NIP44_MAX_PLAINTEXT})") + } + } + } +} + +impl std::error::Error for ListError {} + +/// A membership's keys. +#[derive(Debug, Clone, PartialEq, Serialize, Deserialize)] +pub struct JoinMaterial { + pub community_id: CommunityId, + pub owner: PublicKey, + pub owner_salt: String, + pub community_root: String, + pub root_epoch: Epoch, + #[serde(default, skip_serializing_if = "Option::is_none")] + pub control_pk: Option, + /// Present only when the holder is staff. + #[serde(default, skip_serializing_if = "Option::is_none")] + pub control_root: Option, + #[serde(default, skip_serializing_if = "Vec::is_empty")] + pub channels: Vec, + #[serde(default, skip_serializing_if = "Vec::is_empty")] + pub relays: Vec, + pub name: String, + #[serde(flatten)] + pub extra: Extra, +} + +#[derive(Debug, Clone, PartialEq, Serialize, Deserialize)] +pub struct CommunityListEntry { + pub community_id: CommunityId, + pub seed: JoinMaterial, + pub current: JoinMaterial, + pub added_at: u64, + #[serde(flatten)] + pub extra: Extra, +} + +#[derive(Debug, Clone, PartialEq, Serialize, Deserialize)] +pub struct Tombstone { + pub community_id: CommunityId, + pub removed_at: u64, + #[serde(flatten)] + pub extra: Extra, +} + +/// A member's own memberships. +#[derive(Debug, Clone, Default, PartialEq, Serialize, Deserialize)] +pub struct CommunityList { + #[serde(default, skip_serializing_if = "Vec::is_empty")] + pub entries: Vec, + #[serde(default, skip_serializing_if = "Vec::is_empty")] + pub tombstones: Vec, + #[serde(flatten)] + pub extra: Extra, +} + +impl CommunityList { + pub fn is_live(&self, community_id: &CommunityId) -> bool { + let added = self + .entries + .iter() + .find(|entry| entry.community_id == *community_id) + .map(|entry| entry.added_at); + + match added { + None => false, + Some(added) => self + .tombstones + .iter() + .find(|tombstone| tombstone.community_id == *community_id) + .is_none_or(|tombstone| added > tombstone.removed_at), + } + } + + pub fn fits(&self) -> Result<(), ListError> { + if self.entries.len() > MAX_MEMBERSHIPS { + return Err(ListError::TooManyMemberships(self.entries.len())); + } + + let json = serde_json::to_string(self).map_err(json_error)?; + + if json.len() > NIP44_MAX_PLAINTEXT { + return Err(ListError::Oversize(json.len())); + } + + Ok(()) + } +} + +pub fn join_material(invite: &CommunityInvite, control_root: Option<&[u8; 32]>) -> JoinMaterial { + JoinMaterial { + community_id: invite.community_id, + owner: invite.owner, + owner_salt: invite.owner_salt.clone(), + community_root: invite.community_root.clone(), + root_epoch: invite.root_epoch, + control_pk: invite.control_pk, + control_root: control_root.map(|key| data_encoding::HEXLOWER.encode(key)), + channels: invite.channels.clone(), + relays: invite.relays.clone(), + name: invite.name.clone(), + extra: Extra::default(), + } +} + +pub fn merge(held: CommunityList, incoming: CommunityList) -> CommunityList { + let mut entries: BTreeMap = BTreeMap::new(); + + for entry in held.entries.into_iter().chain(incoming.entries) { + match entries.entry(entry.community_id) { + Entry::Vacant(slot) => { + slot.insert(entry); + } + Entry::Occupied(mut slot) => merge_entry(slot.get_mut(), entry), + } + } + + let mut tombstones: BTreeMap = BTreeMap::new(); + + for tombstone in held.tombstones.into_iter().chain(incoming.tombstones) { + match tombstones.entry(tombstone.community_id) { + Entry::Vacant(slot) => { + slot.insert(tombstone); + } + Entry::Occupied(mut slot) => { + let held = slot.get_mut(); + held.removed_at = held.removed_at.max(tombstone.removed_at); + union(&mut held.extra, tombstone.extra); + } + } + } + + let mut extra = held.extra; + union(&mut extra, incoming.extra); + + CommunityList { + entries: entries.into_values().collect(), + tombstones: tombstones.into_values().collect(), + extra, + } +} + +pub fn build_list_event(keys: &Keys, list: &CommunityList) -> Result { + list.fits()?; + + let json = serde_json::to_string(list).map_err(json_error)?; + let content = nip44::encrypt( + keys.secret_key(), + &keys.public_key(), + json.as_bytes(), + Version::V2, + ) + .map_err(crypto_error)?; + + EventBuilder::new(Kind::Custom(KIND_COMMUNITY_LIST), content) + .finalize(keys) + .map_err(crypto_error) +} + +pub fn parse_list_event(keys: &Keys, event: &Event) -> Result { + if event.kind.as_u16() != KIND_COMMUNITY_LIST { + return Err(ListError::Kind(event.kind.as_u16())); + } + + let json = nip44::decrypt(keys.secret_key(), &keys.public_key(), &event.content) + .map_err(crypto_error)?; + + serde_json::from_str(&json).map_err(json_error) +} + +#[derive(Clone, Copy, PartialEq, Eq)] +enum Snapshot { + Seed, + Current, +} + +fn merge_entry(held: &mut CommunityListEntry, incoming: CommunityListEntry) { + held.added_at = held.added_at.max(incoming.added_at); + held.seed = pick(&held.seed, &incoming.seed, Snapshot::Seed).clone(); + held.current = pick(&held.current, &incoming.current, Snapshot::Current).clone(); + union(&mut held.extra, incoming.extra); +} + +fn pick<'a>( + held: &'a JoinMaterial, + incoming: &'a JoinMaterial, + which: Snapshot, +) -> &'a JoinMaterial { + let preferred = match which { + Snapshot::Seed => incoming.root_epoch < held.root_epoch, + Snapshot::Current => incoming.root_epoch > held.root_epoch, + }; + + if preferred { + return incoming; + } + + if incoming.root_epoch == held.root_epoch && canonical(incoming) < canonical(held) { + return incoming; + } + + held +} + +fn union(into: &mut Extra, other: Extra) { + for (key, value) in other { + let replace = match into.get(&key) { + Some(existing) => canonical(&value) < canonical(existing), + None => true, + }; + + if replace { + into.insert(key, value); + } + } +} + +fn canonical(value: &T) -> String { + serde_json::to_string(value).unwrap_or_default() +} + +fn json_error(error: serde_json::Error) -> ListError { + ListError::Json(error.to_string()) +} + +fn crypto_error(error: impl fmt::Display) -> ListError { + ListError::Crypto(error.to_string()) +} + +#[cfg(test)] +mod tests { + use super::*; + + fn id(byte: u8) -> CommunityId { + CommunityId::from_bytes([byte; 32]) + } + + fn material( + community_id: CommunityId, + owner: PublicKey, + name: &str, + epoch: u64, + ) -> JoinMaterial { + JoinMaterial { + community_id, + owner, + owner_salt: "33".repeat(32), + community_root: "44".repeat(32), + root_epoch: Epoch(epoch), + control_pk: None, + control_root: None, + channels: vec![], + relays: vec!["wss://relay.example".to_owned()], + name: name.to_owned(), + extra: Extra::default(), + } + } + + fn entry( + community_id: CommunityId, + seed: JoinMaterial, + current: JoinMaterial, + added_at: u64, + ) -> CommunityListEntry { + CommunityListEntry { + community_id, + seed, + current, + added_at, + extra: Extra::default(), + } + } + + fn list(entries: Vec) -> CommunityList { + CommunityList { + entries, + ..Default::default() + } + } + + fn removal(community_id: CommunityId, removed_at: u64) -> CommunityList { + CommunityList { + tombstones: vec![Tombstone { + community_id, + removed_at, + extra: Extra::default(), + }], + ..Default::default() + } + } + + #[test] + fn merge_keeps_the_earlier_seed_and_the_later_current_either_way_round() { + let owner = Keys::generate().public_key(); + let older = material(id(0x11), owner, "Room", 1); + let newer = material(id(0x11), owner, "Room", 3); + + let a = list(vec![entry(id(0x11), older.clone(), newer.clone(), 5_000)]); + let b = list(vec![entry(id(0x11), newer, older, 5_000)]); + + for merged in [merge(a.clone(), b.clone()), merge(b, a)] { + let merged = merged.entries.first().expect("one membership"); + assert_eq!( + merged.seed.root_epoch, + Epoch(1), + "seed anchors the earliest epoch held" + ); + assert_eq!(merged.current.root_epoch, Epoch(3)); + assert_eq!(merged.added_at, 5_000); + } + + // An epoch tie breaks on the whole snapshot's bytes, and does so for both + // orders, so two devices never flap competing republishes. + let alpha = material(id(0x11), owner, "Alpha", 2); + let beta = material(id(0x11), owner, "Beta", 2); + let a = list(vec![entry(id(0x11), alpha.clone(), alpha, 1)]); + let b = list(vec![entry(id(0x11), beta.clone(), beta, 1)]); + + let first = merge(a.clone(), b.clone()); + assert_eq!(first, merge(b, a)); + assert_eq!(first.entries[0].current.name, "Alpha"); + } + + #[test] + fn a_tombstone_is_terminal_until_a_newer_join_outruns_it() { + let owner = Keys::generate().public_key(); + let joined = entry( + id(0x11), + material(id(0x11), owner, "Room", 0), + material(id(0x11), owner, "Room", 0), + 5_000, + ); + + let left = merge(list(vec![joined.clone()]), removal(id(0x11), 6_000)); + assert!(!left.is_live(&id(0x11))); + assert_eq!( + left.entries.len(), + 1, + "a retired entry stays in the document" + ); + + // A stale device re-merging the entry cannot resurrect it. + assert!(!merge(left.clone(), list(vec![joined.clone()])).is_live(&id(0x11))); + + // A re-join genuinely newer than the removal does. + let rejoined = list(vec![entry( + id(0x11), + material(id(0x11), owner, "Room", 0), + material(id(0x11), owner, "Room", 0), + 7_000, + )]); + let live = merge(left, rejoined); + assert!(live.is_live(&id(0x11))); + + // And the older removal is not re-applied on top of it. + assert!(merge(live, removal(id(0x11), 6_000)).is_live(&id(0x11))); + } + + #[test] + fn a_second_device_reconstructs_membership_from_13302() { + let me = Keys::generate(); + let owner = Keys::generate().public_key(); + let mine = CommunityList { + entries: vec![ + entry( + id(0x11), + material(id(0x11), owner, "Room", 1), + material(id(0x11), owner, "Room", 4), + AT, + ), + entry( + id(0x22), + material(id(0x22), owner, "Other", 0), + material(id(0x22), owner, "Other", 0), + AT + 1, + ), + ], + tombstones: vec![Tombstone { + community_id: id(0x33), + removed_at: AT, + extra: Extra::default(), + }], + extra: Extra::default(), + }; + + let event = build_list_event(&me, &mine).expect("builds"); + assert_eq!(event.kind, Kind::Custom(KIND_COMMUNITY_LIST)); + assert_eq!(parse_list_event(&me, &event).expect("parses"), mine); + assert!( + !parse_list_event(&me, &event) + .expect("parses") + .is_live(&id(0x33)) + ); + + // Only the member's own keys open it, and an unreadable list is "no news". + let stranger = Keys::generate(); + assert!(parse_list_event(&stranger, &event).is_err()); + + // Unknown fields survive the round trip, so a republish cannot wipe them. + let mut held = mine.clone(); + held.extra + .insert("future".to_owned(), serde_json::json!({"deep": [1, 2]})); + held.entries[0] + .current + .extra + .insert("held_roots".to_owned(), serde_json::json!([{"epoch": 1}])); + let rebuilt = + parse_list_event(&me, &build_list_event(&me, &held).expect("builds")).expect("parses"); + assert_eq!(rebuilt, held); + + // The write gate refuses an over-cap or oversized List before publishing. + let crowded = list( + (0..=MAX_MEMBERSHIPS) + .map(|index| { + let community_id = CommunityId::from_bytes([index as u8; 32]); + entry( + community_id, + material(community_id, owner, "Room", 0), + material(community_id, owner, "Room", 0), + AT, + ) + }) + .collect(), + ); + assert!(matches!( + build_list_event(&me, &crowded), + Err(ListError::TooManyMemberships(n)) if n == MAX_MEMBERSHIPS + 1 + )); + + let oversized = list(vec![entry( + id(0x11), + material(id(0x11), owner, &"x".repeat(NIP44_MAX_PLAINTEXT), 0), + material(id(0x11), owner, "Room", 0), + AT, + )]); + assert!(matches!(oversized.fits(), Err(ListError::Oversize(_)))); + } + + const AT: u64 = 1_719_800_000_000; +} diff --git a/crates/concord/src/stream.rs b/crates/concord/src/stream.rs index f1817c07..13fe9b3a 100644 --- a/crates/concord/src/stream.rs +++ b/crates/concord/src/stream.rs @@ -180,10 +180,23 @@ pub fn seal_content( match form { SealForm::Plaintext => Ok(json), - SealForm::Encrypted => Ok(BASE64.encode(&encrypt(group.conversation(), json.as_bytes())?)), + SealForm::Encrypted => seal_bytes(group.conversation(), json.as_bytes()), } } +pub fn seal_bytes(conversation: &ConversationKey, plaintext: &[u8]) -> Result { + Ok(BASE64.encode(&encrypt(conversation, plaintext)?)) +} + +pub fn open_bytes(conversation: &ConversationKey, content: &str) -> Result, StreamError> { + let payload = BASE64 + .decode(content.as_bytes()) + .map_err(|error| StreamError::Decrypt(error.to_string()))?; + + decrypt_to_bytes(conversation, &payload) + .map_err(|error| StreamError::Decrypt(error.to_string())) +} + pub fn build_seal( rumor: &UnsignedEvent, form: SealForm, @@ -357,12 +370,7 @@ fn encrypt(conversation: &ConversationKey, plaintext: &[u8]) -> Result, } fn decode_content(conversation: &ConversationKey, content: &str) -> Result { - let payload = BASE64 - .decode(content.as_bytes()) - .map_err(|error| StreamError::Decrypt(error.to_string()))?; - - let plaintext = decrypt_to_bytes(conversation, &payload) - .map_err(|error| StreamError::Decrypt(error.to_string()))?; + let plaintext = open_bytes(conversation, content)?; String::from_utf8(plaintext).map_err(|error| StreamError::Parse(error.to_string())) } -- 2.54.0 From ecd08273ebcc9f337c712e3af85a7260b2d92d17 Mon Sep 17 00:00:00 2001 From: Ren Amamiya Date: Thu, 17 Sep 2026 08:16:41 +0700 Subject: [PATCH 09/12] add rekeys, refounding and dissolution --- PLAN.md | 159 +++- crates/concord/src/control.rs | 90 +- crates/concord/src/invite.rs | 174 +++- crates/concord/src/lib.rs | 1 + crates/concord/src/list.rs | 6 +- crates/concord/src/rekey.rs | 1634 +++++++++++++++++++++++++++++++++ crates/concord/src/store.rs | 3 + crates/concord/src/stream.rs | 12 +- 8 files changed, 2035 insertions(+), 44 deletions(-) create mode 100644 crates/concord/src/rekey.rs diff --git a/PLAN.md b/PLAN.md index bf8c6968..668d36b9 100644 --- a/PLAN.md +++ b/PLAN.md @@ -556,7 +556,7 @@ pub struct ChatMessage { gpui type, and a protocol crate does not take a UI dependency for a derived field. They land with the first consumer that renders them. -### 8.5 Invites (`invite.rs`) — implemented in M6 +### 8.5 Invites (`invite.rs`) — bundle and Direct Invite in M6, Invite List in M7, Registry in M7 ```rust pub const KIND_BUNDLE: u16 = 33301; @@ -635,11 +635,41 @@ as a NIP-40 tag in seconds. - The earlier sketch's `signer: &UniversalSigner` is `&Keys`, matching every other builder in the crate: NIP-46 is one deliberate pass, not a per-milestone patch (see §14.9). -**The Invite List (13303) and the Registry (vsk 8) are deferred to M7**, together and for the same -reason: nothing in M6 consumes them. A link's signer is held by its caller, so minting, refreshing -and revoking need no document, and a Registry write whose fold does not exist is dead wire. M7 is -where both become load-bearing — the Registry's aggregate is the Public/Private source of truth and -retiring the last live link is what triggers a Refounding. +**The Invite List (`13303`) and the Registry (`vsk 8`) landed in M7**, deferred from M6 for one reason: +nothing in M6 consumed them. A link's signer is held by its caller, so minting, refreshing and revoking +need no document, and a Registry write whose fold does not exist is dead wire. + +```rust +pub const KIND_INVITE_LIST: u16 = 13303; +pub const MAX_INVITE_ENTRIES: usize = 64; + +pub struct InviteEntry { token: String, signer_sk: String, community_id: CommunityId, url: String, + label: Option, created_at: u64, expires_at: Option, extra } +pub struct InviteTombstone { token: String, community_id: CommunityId, extra } +pub struct InviteList { entries: Vec, tombstones: Vec, extra } +impl InviteList { + pub fn is_live(&self, token: &str) -> bool; + pub fn fits(&self) -> Result<(), InviteError>; // the write gate +} + +pub fn merge_invite_lists(held: InviteList, incoming: InviteList) -> InviteList; +pub fn build_invite_list(keys: &Keys, list: &InviteList) -> Result; +pub fn parse_invite_list(keys: &Keys, event: &Event) -> Result; +``` + +- The creator's private bookkeeping: `token` is the link's unlock secret **and** its merge key, and +`signer_sk` is the `link_signer` secret that refreshing or retiring the bundle needs. An entry is +immutable once minted, so a divergent pair is settled on the lowest canonical bytes — a total order, so +two devices never flap. Tombstones union and beat an entry **terminally**, so a stale device can never +resurrect a revoked link. Like the Community List it is NIP-44-to-self at a replaceable kind, and +`fits` refuses to build past the entry cap or the NIP-44 plaintext cap. +- The Registry is its member-facing shadow: a Control Plane entity (`vsk 8`) at +`invite_links_locator(community_id, creator)` whose content is the live links' **coordinates only** — +never a token, URL or signing secret — so members can see that links exist without being able to use one. +`ControlFold.registries` holds one set per creator, honored only under `CREATE_INVITE`, and +`ControlFold::is_public` reads their aggregate: non-empty means a live link exists and the community is +Public. Retiring the last live link empties it, and that flip is what a Refounding seals (§8.7). +`ControlWriter::set_registry` is the write side. ### 8.6 The Community List (`list.rs`) — implemented in M6 @@ -691,26 +721,103 @@ membership is the registry's, and the byte-level cap audit is M8's. holds one event per pubkey, so it cannot shard past the size cap); that remains an interop follow-up, recorded in §14.1. -### 8.7 Rekeys and refoundings (`rekey.rs`) +### 8.7 Rekeys, refoundings and dissolution (`rekey.rs`) — implemented in M7 ```rust +pub const KIND_REKEY: u16 = 3303; +pub const MAX_REKEY_BLOBS_PER_EVENT: usize = 80; +pub const MAX_REKEY_BLOBS_RECEIVED: usize = 120; +pub const MAX_REKEY_EPOCH: u64 = 1 << 40; + pub enum RekeyScope { Channel(ChannelId), Base } -pub fn encode_blob_plaintext(scope, epoch, new_root, control_pk, control_root) -> Vec; // 72 | 104 | 136 bytes -pub fn parse_blob_plaintext(bytes: &[u8], scope, epoch) -> Result; -pub fn build_rekey_rumor(rotator, scope, new_epoch, prev_epoch, prev_commit, blobs, chunk) -> UnsignedEvent; -pub fn plan_refounding(fold, removed: &[PublicKey]) -> Result; -pub fn compact(fold, epoch, new_control_root, ...) -> Vec; // re-wrap heads verbatim, plaintext seals preserved +pub struct RekeyBlob { locator: String, wrapped: String } +pub struct KeyDelivery { new_key: [u8; 32], control_pk: Option<[u8; 32]>, control_root: Option<[u8; 32]> } +pub enum Continuity { Extends, Gap, Fork } +pub struct RekeyChunk { rotator, scope, new_epoch, prev_epoch, prev_commit, chunk: (u32, u32), + blobs, citation, severed } +pub struct Rotation { rotator, scope, new_epoch, prev_epoch, prev_commit, blobs, declared, + held, severed, citation } +pub struct Refounding { epoch: Epoch, new_root: [u8; 32], new_control_root: [u8; 32] } + +pub fn encode_blob_plaintext(scope, epoch, new_key, control_pk, control_root) -> Result, RekeyError>; +pub fn parse_blob_plaintext(bytes, scope, epoch, community_id) -> Result; +pub fn rekey_group(scope, addressing_root, community_id, new_epoch) -> Result; +pub fn blob_locator(rotator, recipient, scope, epoch) -> String; +pub fn build_blob(rotator: &Keys, recipient, scope, epoch, new_key, control_pk, control_root) + -> Result; +pub fn open_blob(recipient: &Keys, rotator, scope, epoch, blob, community_id) + -> Result; +pub fn find_my_blobs<'a>(blobs: &'a [RekeyBlob], rotator, me, scope, epoch) + -> impl Iterator; + +pub fn build_rekey_rumor(rotator, scope, new_epoch, prev_epoch, prev_commit, blobs, chunk, + citation, severed, at_secs) -> Result; +pub fn build_rekey_chunks(rotator: &Keys, group, scope, new_epoch, prev_epoch, prev_commit, blobs, + citation, severed, at_secs) -> Result, RekeyError>; +pub fn parse_rekey_chunk(opened: &OpenedStream) -> Result; + +pub fn collect_rotations(chunks: &[RekeyChunk]) -> Vec; +pub fn am_i_removed(rotation: &Rotation, me: &PublicKey) -> Option; +pub fn rekey_authorized(roles, owner, rotator, permission, removed) -> bool; +pub fn fork_winner(held: Option<&[u8; 32]>, candidates: &[[u8; 32]]) -> Option; + +pub fn plan_refounding(epoch: Epoch) -> Result; +pub fn compact(seals: &[Event], read: &GroupKey, signer: &GroupKey, at_secs: u64) + -> Result, RekeyError>; + +pub struct DissolvedTombstone { owner: PublicKey } +pub fn dissolved_tombstone_rumor(owner, community_id, at_secs) -> UnsignedEvent; +pub fn seal_dissolved(rumor, community_id, owner: &Keys, at_secs) -> Result; +pub fn open_dissolved(wrap, community_id) -> Result; +pub fn verify_dissolved(wrap, identity: &CommunityIdentity) -> bool; ``` -- The subscription for rekeys is precomputed from the *next* epoch's address, per private channel and once for the base. -- A receiver accepts a key only after: locating its blob, decrypting with the rotator↔recipient conversation key, checking the bound `scope` and `epoch` inside the plaintext, and matching `prevcommit` against the key it currently holds. -- Only after holding **all** `n` chunks of one `(rotator, newepoch, prevcommit)` set, with none containing its locator, may a client conclude it was removed. -- Send cap 80 blobs per event, accept cap 120 (Vector's documented erratum: the CORD-01 double envelope pushes 120 blobs past a 64 KB relay limit). Record the reason in a comment so nobody "fixes" it back. -- Compacted control heads are re-wrapped with their original signature intact, which is exactly why the control plane uses the plaintext seal. -- Two concurrent refoundings converge on the lexicographically lowest new base key; the heal is down-only. -- Authority: a channel rekey needs `MANAGE_CHANNELS`, a refounding needs `BAN`, and in both the rotator must strictly outrank every removed target. Holding a key is never authority. +- The subscription for rekeys is precomputed from the *next* epoch's address, per private channel +and once for the base. That is a `Filter` and belongs to the sync engine (§10), not here; `rekey_group` +is what makes it derivable, and keeps a channel scope from being addressed at the base derivation. +- A receiver accepts a key only after: locating its blob, decrypting with the rotator↔recipient +conversation key, checking the bound `scope` and `epoch` **inside** the plaintext, and matching +`prevcommit` against the key it currently holds. The locator is deliberately *not* gated on open: +it derives from public keys alone (NIP-46 bunker parity) and so proves nothing, and the pairwise +decrypt plus the bound check are the whole gate. +- Only after holding **all** `n` chunks of one `(rotator, newepoch, prevcommit)` set, with none +containing its locator, may a client conclude it was removed. `collect_rotations` **unions** the blobs +of two chunks claiming one index rather than keeping the first: a catch-up chunk can legitimately +re-claim a slot, and a recipient dropped from the union would read as removed, which deletes the +community locally. `severed` is an OR across chunks for the same reason in reverse. +- Send cap 80 blobs per event, accept cap 120. The 80 is an erratum this reproduces: the CORD-01 double +envelope costs two NIP-44 base64 expansions, so 120 blobs measure ~77 KB and a 64 KB relay refuses +them. The accept cap stays at the spec's 120 so a peer at the spec limit still parses. +- `parse_blob_plaintext` takes the `community_id`, which the earlier sketch did not: the 104/136-byte +base forms carry the next epoch's Control Plane keys, and the 136-byte secret must derive to the +`control_pk` beside it — a community- and epoch-bound check. A **width past 136** is a form this client +predates, so it degrades rather than refusing: the frozen 72-byte prefix yields the root (membership and +every chat plane survive), the appended pair is kept when it verifies, and the rest freezes. Widths +between the defined forms fit no extension and stay malformed. +- Authority: a channel rekey needs `MANAGE_CHANNELS`, a refounding needs `BAN`, and in both the rotator +must strictly outrank every removed target. `rekey_authorized` is that whole rule — one permission check +plus `can_act_on_member` per target — so it is testable without minting a rotation, and an empty removed +set (a hygienic rotation, or a flip to Private) needs only the permission. Holding a key is never authority. +- Two concurrent refoundings converge on the lexicographically lowest new base key, and the heal is +down-only. `fork_winner` folds both into one call: the lowest candidate, returned only when it strictly +lowers a key already held, so a flaky fetch cannot re-fork a settled epoch. +- `Refounding` owns the epoch and the freshly minted pair and derives all three coordinates from them +(`read`, `signer`, and the signer's pk), so a caller cannot pair a root with the wrong epoch. +`plan_refounding` deliberately takes neither the fold nor the removed set: the authority gate is +`rekey_authorized`, one job each, and the pair travels in the base blobs. +- Compacted control heads are re-wrapped with their original signature intact, which is exactly why the +control plane uses the plaintext seal. `compact` is `stream::rewrap_seal` over the held seals, whose +signature grew a separate `signer` group in M7 — a split epoch reads under the rolled root but wraps as +the new control signer, so one group could not express it. +- Nothing in coop **mints** `severed` yet: it is a receiver-side rule, and the Server-Severing flow that +would set it awaits the invite-link wiring of §10. -Dissolution (CORD-02 §9) also lives here: a chainless, owner-signed `vsk 10` tombstone at `dissolved_group_key(id)`, plaintext-sealed, and a verifier **must** refuse any tombstone whose `eid` is not the community's own id (including the all-zero placeholder — accepting it lets an owner's genuine tombstone for one community be re-wrapped at another of theirs and kill it permanently). On sight the community is sealed read-only: subscriptions halt, nothing new is honored, existing history stays readable, and a member's delete of their own message is still honored. +Dissolution (CORD-02 §9) also lives here: a chainless, owner-signed `vsk 10` tombstone at +`dissolved_group_key(id)`, plaintext-sealed, and a verifier **must** refuse any tombstone whose `eid` is +not the community's own id (including the all-zero placeholder — accepting it lets an owner's genuine +tombstone for one community be re-wrapped at another of theirs and kill it permanently). On sight the +community is sealed read-only: `CommunityState.dissolved` records it, subscriptions halt, nothing new is +honored, existing history stays readable, and a member's delete of their own message is still honored. ## 9. Storage (`store.rs`) — local layer implemented in M1, state document in M2, fold bridge in M3, banlist in M5 @@ -761,7 +868,7 @@ pub struct CommunityState { } ``` -Landed in M2 with exactly the fields genesis can populate: `save_state`/`load_state` and `CommunityState::from_genesis`. Three fields the plan sketched are deliberately absent until something can fill them — `epoch_keys` (needs rekeys, M7), `dissolved` (needs the tombstone, M7), and `observed`/`guestbook` (need the guestbook ingest of §10). `control_pks` keyed by `u64` rather than `Epoch` and `heads` as a `Vec` rather than a `BTreeMap<[u8; 32], _>`, because serde_json cannot use a byte-array map key. `banned` is a `BTreeSet` — serde has no such problem with a `Vec`-like sequence. +Landed in M2 with exactly the fields genesis can populate: `save_state`/`load_state` and `CommunityState::from_genesis`. `dissolved` landed in M7, set from a verified tombstone. Two fields the plan sketched are still absent: `epoch_keys`, because `ChannelKeyRef` carries no key for it to hold (§14.11), and `observed`/`guestbook`, which need the guestbook ingest of §10. `control_pks` keyed by `u64` rather than `Epoch` and `heads` as a `Vec` rather than a `BTreeMap<[u8; 32], _>`, because serde_json cannot use a byte-array map key. `banned` is a `BTreeSet` — serde has no such problem with a `Vec`-like sequence. M3 added the two bridges between this document and the fold: @@ -929,6 +1036,7 @@ Each of these has burned a real implementation, or is a documented cross-client - **Enforced in M4:** the chat plane's encrypted-seal requirement, at both publish and open; `channel` **and** `epoch` strict-equal to the plane whose key opened the wrap; a retired or unregistered rumor kind rejected on both sides; a target bearing tag that appears twice rejected outright; and a delete from anybody other than the message's own author refused by the caller's gate. - **Enforced in M5:** a Kick counts only where the roster admits its actor under `KICK` with a strictly higher rank and a resolvable citation; a Snapshot counts only from the epoch's refounder, with no owner fallback; a guestbook entry more than an hour ahead is dropped, and an out-of-range `ms` or a non-verb `3306` entry is malformed, not interpreted; a duplicated `vac` is rejected outright; and a delete is honored from its target's author unconditionally, or from another actor only where `can_delete` admits them. - **Enforced in M6:** an invite fragment whose version is not this one is refused in either direction, as is one with a bad count or trailing bytes, and its encoding caps bootstrap relays at three; a bundle past the channel cap, past the epoch ceiling, or carrying a secret that is not 32 bytes of hex is refused before it is used, and one whose `owner` + `owner_salt` does not reproduce its `community_id` is refused outright; a bundle event off its coordinate, off its author, or unsigned is refused, and a tombstone at the coordinate reads as revoked; a Direct Invite's wrap, its seal signature and its rumor/seal author bind are all verified before the bundle is even parsed; and a Community List refuses to build past its membership cap or the NIP-44 plaintext cap. +- **Enforced in M7:** a blob's bound scope and epoch are checked inside the ciphertext, so a channel blob cannot be opened under the base scope or under another epoch, and a 136-byte base blob whose secret does not derive to the pk beside it is refused whole rather than adopting a split control plane; the locator is never gated, because it derives from public keys and proves nothing; a removal is never concluded from a partial chunk set, and two chunks claiming one index union their blobs rather than letting the loser's recipients read as removed; a rotation needs its permission and must strictly outrank every target, so a rotator holding the prior root or a demoted staffer holding the `control_root` is dropped; a plaintext-sealed rekey is refused; and a tombstone is refused unless its signed `eid` is this community's own id — the all-zero placeholder and a sibling community of the same owner included. - Still owed to M8's audit: the byte caps, the 256-channel and 25-pin caps, and the write-side counterparts of the caps the folds already apply. ## 13. Milestones @@ -942,7 +1050,7 @@ Each of these has burned a real implementation, or is a documented cross-client | M4 | Chat plane | ✅ `cargo test -p concord` (19 tests): a second holder folds a message's reactions, its author's edit and its author's delete, and ignores an edit or a delete from anybody else; a comment's root and parent survive the wire; a foreign channel, a replayed epoch, a plaintext seal, a retired kind and a duplicated target are each rejected; and history pages backwards across a rekey in order | | M5 | Guestbook + member list + moderation | ✅ `cargo test -p concord` (23 tests): a second holder folds joins, leaves, a cited kick and a chunked snapshot into one memberlist in either arrival order, with a ban and a Grant deciding the edges; an uncited, unranked or owner-directed kick and a foreign snapshot are dropped; a future-dated entry, a malformed `ms`, a non-verb `3306`, a duplicated `vac` and a bad snapshot chunk are each refused; and a moderator delete lands only under a citation the roster admits, while a self-delete never asks | | M6 | Invites + Community List | ✅ `cargo test -p concord` (32 tests): the fragment's byte layout is pinned by golden base64url for the stock set, a dictionary mix and a verbatim literal, with a wrong version in either direction, trailing bytes and an over-cap count each fatal; a link round-trips as a full URL and as a bare naddr, and refuses a non-invite; a bundle round-trips while a revocation tombstone reads as revoked, and a wrong token, a squatter's author, a foreign `d`, a forged owner, a malformed secret and an over-cap channel count are each refused; a Direct Invite round-trips to its verified inviter and refuses a stranger's keys and a non-invite rumor; and the Community List keeps the earlier seed and the later current in either merge order, refuses to resurrect a tombstoned id until a newer join outruns it, and rebuilds on a second device with unknown fields intact while refusing an over-cap or oversized list | -| M7 | Rekeys + refounding + dissolution | a removed member stops reading after a rekey; continuity and race rules tested; a tombstone seals the community and a foreign-id tombstone is refused | +| M7 | Rekeys + refounding + dissolution | ✅ `cargo test -p concord` (40 tests): a blob's bound scope and epoch are checked inside the ciphertext, so a channel blob cannot be opened under the base scope or under another epoch, and a staff secret that does not derive to the pk beside it refuses the whole blob; a removal is concluded only from a complete chunk set, and two chunks claiming one index union rather than drop a recipient's blob; continuity extends, gaps and forks, and the fork winner is the lowest key adopted only when it strictly lowers one already held; a rotation needs its permission and must strictly outrank every target, so holding a key is never authority; a full 80-blob chunk fits a 64 KB relay event and one more splits; compaction carries a settled head across a refounding with the original author's signature intact; and an owner's tombstone seals the community while an impostor's, the spec's all-zero `eid` and one re-wrapped from another community of the same owner are each refused | | M8 | Pins + disappearing messages + hardening | pins verify from a keyless reader's view; expiry is refused at ingest and purged by the sweep; the audit of §12 is complete with a test per bullet | Ordering is deliberately dependency-first: each milestone is usable on its own, and nothing in M2+ depends on a later milestone. The sync engine and its GPUI wiring (§10, §11) have no row of their own because they are cross-cutting: every plane they consume has to exist first, so they follow M8, and they are the milestone that applies §11's `chat::handle_notifications` routing fix. @@ -965,7 +1073,11 @@ What M5 still defers, and to what: the **guestbook's fetch and ingest path** — M6 closed at 32 tests, again with no dependency change and `Cargo.lock` untouched — the fragment codec is `data_encoding::BASE64URL_NOPAD` and the Direct Invite is nostr's own `nip59` builder and unwrapper, so nothing new was needed. New: `src/invite.rs` (bundle, link, fragment, Direct Invite) and `src/list.rs` (the Community List). In `stream.rs`, the seal and open primitives were factored out of `seal_content`/`decode_content` as `seal_bytes`/`open_bytes`, so the bundle's raw-token key and the List's to-self envelope share the crate's one NIP-44 shape instead of re-implementing it twice; `edition::TAG_SUBKIND` became public so the invite sub-kind tags do not restate the string. -What M6 still defers, and to what: the **Invite List (13303) and the Registry (`vsk 8`)**, both to M7 and for the same reason — nothing in M6 consumes them, and a Registry write whose fold does not exist is dead wire, while M7's refounding is what reads the Registry's aggregate as the Public/Private source of truth (see §8.5); the **join gate that refuses a 51st membership** (the registry's, since `fits` protects the write rather than the add); and the byte-level cap audit, which is M8's. +What M6 still defers, and to what: the **Invite List (13303) and the Registry (`vsk 8`)** — both landed in M7, as planned; the **join gate that refuses a 51st membership** (the registry's, since `fits` protects the write rather than the add); and the byte-level cap audit, which is M8's. + +M7 closed at 40 tests, again with no dependency change and `Cargo.lock` untouched. New: `src/rekey.rs` (the blob atom, the 3303 chunk set and its collection, continuity and the fork winner, the authority gate, refounding planning and compaction, and dissolution). The Invite List landed in `src/invite.rs` beside the bundle it bookkeeps, and the Registry became a Control Plane entity: `ControlFold.registries` keyed by creator, folded under `CREATE_INVITE`, its aggregate exposed as `ControlFold::is_public`, with `ControlWriter::set_registry` as the write side. `invite_links_locator` and M6's revocation machinery needed no change. In `stream.rs`, `rewrap_seal` gained a separate `signer` group, because a split epoch reads under the rolled root but wraps as the new control signer. In `store.rs`, `CommunityState.dissolved` records the seal; `list.rs`'s `canonical`/`union` became `pub(crate)` so the Invite List's merge shares them rather than restating the same total order. + +What M7 still defers, and to what: **epoch key retention** — the blob atom delivers every plane key a refounding mints, but nothing can persist one, because `ChannelKeyRef` is `{id, name, private, epoch}` with no key field (§14.11); the **rekey subscription**, precomputed from the next epoch's address for every held private channel plus the base, which is a `Filter` and belongs with the sync engine (§10); the **Server-Severing flow** that would set `severed`, which awaits the invite-link wiring; and the **`Refound` seed** to `complete_memberlist`, which M7 can now mint but whose consumer is still the guestbook ingest of §10. **M2's "created and published" is verified offline**: "published" is the two wraps existing and being openable by the invite keys, not a relay round-trip. There is no registry to publish through until §10, and a relay test would be testing the SDK, not the protocol. @@ -981,6 +1093,7 @@ What M6 still defers, and to what: the **Invite List (13303) and the Registry (` 8. **The Pin List's message-key disclosure has no public API (M8).** CORD-04 §7 Pins let a keyless reader verify a disclosure, which means revealing one message's NIP-44 keys rather than the plane's conversation key. `nostr`'s `nip44::v2::get_message_keys(conversation_key, nonce)` is a private `fn`, and both public entry points (`encrypt_to_bytes_with_nonce`, `decrypt_to_bytes`) take the whole conversation key — so the expansion has to be reproduced as `hkdf::expand_into(conversation_key, nonce, 76 bytes)` plus ChaCha20 and an HMAC-SHA256, exactly as Vector does, and round-tripped against nostr's own `encrypt` in a test. Read CORD-04 §7 in full at M8 before writing it: the reproduction is only worth it once the exact verification the pin must support is settled, and the alternative is contributing a `pub` message-key accessor upstream (we already track git master, so a patch branch or an upstream PR is viable and strictly better than a reproduction we must keep in sync). 9. **A remote signer is not plumbed.** `ControlWriter::publish`, `stream`'s seal builders and the invite builders all take `&Keys`. NIP-46 is a stated Vector feature (§10's task slots are already cleared on signer change), but making the writers async over a `NostrSigner` is a change to every builder, so it should be one deliberate pass rather than a patch per milestone. Nothing in M4–M8 depends on it except the UX of using a remote signer at all. 10. **The fold is not incremental.** `fold_control` re-parses and re-folds the whole control edition window on every call, and each fold is up to `2 × entities + 8` passes. That is fine at the caps the spec sets (100 roles, 400-odd grants) and it is the simplest thing that is correct, but if the sync engine ends up calling it per event rather than per batch, the candidate maps and their parse belong in a cache keyed by edition id. Measure before optimizing. +11. **No plane key can be persisted (found in M7).** The rekey blob atom hands a receiver every key a rotation mints — the next `community_root`, the `control_root`, and a private channel's fresh key — and `CommunityState` has nowhere to put any of them: `ChannelKeyRef` is `{id, name, private, epoch}`, and the state's only root fields are the *current* `community_root`/`root_epoch`. So a client can verify a rotation and still lose it on restart, and it cannot read history written under a prior root or a prior channel epoch. M7 therefore left `epoch_keys` out rather than add a field with no key to hold. The fix is a schema change — `ChannelKeyRef` gains the key and its retired `priors`, and the state gains a root-per-epoch map — and it belongs with the sync engine that reads them back, since it changes `apply_fold` and the `13302` join material together. Armada already carries `priors` for exactly this reason. ## 15. Test strategy diff --git a/crates/concord/src/control.rs b/crates/concord/src/control.rs index 2a6f900d..6cf9da8a 100644 --- a/crates/concord/src/control.rs +++ b/crates/concord/src/control.rs @@ -6,7 +6,7 @@ use serde::{Deserialize, Serialize}; use crate::derive::{ banlist_locator, community_id_of, control_group_key, control_signer_group_key, grant_locator, - verify_community_id, + invite_links_locator, verify_community_id, }; use crate::edition::{ AuthorityCitation, EditionFields, EditionMeta, EntityHead, Floors, ParsedEdition, @@ -21,6 +21,7 @@ use crate::{ChannelId, CommunityId, Epoch, Extra, GroupKey, random_32}; pub const MAX_NAME_BYTES: usize = 64; pub const MAX_DESCRIPTION_BYTES: usize = 10_000; pub const MAX_RELAYS: usize = 5; +pub const MAX_REGISTRY_LINKS: usize = 64; pub const GENERAL_CHANNEL: &str = "general"; pub const ROOT_EPOCH: Epoch = Epoch(0); @@ -334,6 +335,37 @@ impl ControlWriter { at_secs, ) } + + #[allow(clippy::too_many_arguments)] + pub fn set_registry( + &self, + keys: &Keys, + community_id: &CommunityId, + creator: &PublicKey, + links: &[PublicKey], + head: Option<&EntityHead>, + citation: Option, + at_secs: u64, + ) -> Result<(Event, EntityHead)> { + let entries: Vec = links + .iter() + .take(MAX_REGISTRY_LINKS) + .map(PublicKey::to_hex) + .collect(); + let content = serde_json::to_string(&entries)?; + + self.publish( + keys, + Edition { + subkind: vsk::INVITE_LINKS, + entity: invite_links_locator(community_id, &creator.to_bytes()), + content: &content, + head, + citation, + }, + at_secs, + ) + } } fn encode_metadata(metadata: &CommunityMetadata) -> Result { @@ -361,10 +393,18 @@ pub struct ControlFold { pub banned: BTreeSet, pub community: Option, pub channels: BTreeMap, + /// Each creator's live link-signer set. + pub registries: BTreeMap>, pub floors: Floors, pub gapped: bool, } +impl ControlFold { + pub fn is_public(&self) -> bool { + self.registries.values().any(|links| !links.is_empty()) + } +} + pub fn fold_control( owner: &PublicKey, community_id: &CommunityId, @@ -388,6 +428,7 @@ pub fn fold_control( banned: roster.banned, community: metadata.community, channels: metadata.channels, + registries: metadata.registries, floors, gapped: roster.gapped || metadata.gapped, } @@ -397,6 +438,7 @@ pub fn fold_control( struct MetadataFold { community: Option, channels: BTreeMap, + registries: BTreeMap>, floors: Floors, gapped: bool, } @@ -464,9 +506,55 @@ fn fold_metadata( } } + fold.registries = fold_registries(&judge, editions, &mut fold.floors, &mut fold.gapped); + fold } +fn fold_registries( + judge: &Judge<'_>, + editions: &[ParsedEdition], + floors: &mut Floors, + gapped: &mut bool, +) -> BTreeMap> { + let mut candidates: BTreeMap<[u8; 32], Vec<&ParsedEdition>> = BTreeMap::new(); + + for edition in editions { + if edition.subkind == vsk::INVITE_LINKS + && invite_links_locator(judge.community_id, &edition.author.to_bytes()) + == edition.entity + { + candidates.entry(edition.entity).or_default().push(edition); + } + } + + let mut registries = BTreeMap::new(); + + for (entity, group) in &candidates { + let Some(head) = authorized_head(judge, *entity, group, Permissions::CREATE_INVITE, gapped) + else { + continue; + }; + + floors.insert(*entity, EntityHead::from(head)); + + let Ok(links) = serde_json::from_str::>(&head.content) else { + continue; + }; + + registries.insert( + head.author, + links + .iter() + .filter_map(|link| PublicKey::from_hex(link).ok()) + .take(MAX_REGISTRY_LINKS) + .collect(), + ); + } + + registries +} + struct Judge<'a> { owner: &'a PublicKey, community_id: &'a CommunityId, diff --git a/crates/concord/src/invite.rs b/crates/concord/src/invite.rs index 9c8a0b57..022cd0a9 100644 --- a/crates/concord/src/invite.rs +++ b/crates/concord/src/invite.rs @@ -1,9 +1,12 @@ +use std::collections::BTreeMap; +use std::collections::btree_map::Entry; use std::fmt; use data_encoding::BASE64URL_NOPAD; use nostr::nips::nip01::Coordinate; use nostr::nips::nip19::{Nip19, Nip19Coordinate}; use nostr::nips::nip44::v2::ConversationKey; +use nostr::nips::nip44::{self, Version}; use nostr::nips::nip59::{GiftWrapBuilder, UnwrappedGift}; use nostr_sdk::prelude::*; use serde::{Deserialize, Serialize}; @@ -11,15 +14,18 @@ use serde::{Deserialize, Serialize}; use crate::control::{ImageRef, MAX_RELAYS}; use crate::derive::{TOKEN_LEN, verify_community_id}; use crate::edition::{TAG_SUBKIND, vsk}; -use crate::stream::{self, StreamError}; +use crate::list::{canonical, union}; +use crate::stream::{self, NIP44_MAX_PLAINTEXT, StreamError}; use crate::{ChannelId, CommunityId, Epoch, Extra, decode_hex_32}; pub const KIND_BUNDLE: u16 = 33301; +pub const KIND_INVITE_LIST: u16 = 13303; pub const KIND_DIRECT_INVITE: u16 = 3313; pub const FRAGMENT_VERSION: u8 = 4; pub const MAX_BUNDLE_CHANNELS: usize = 256; pub const MAX_BOOTSTRAP_RELAYS: usize = 3; pub const MAX_BUNDLE_EPOCH: u64 = 1 << 40; +pub const MAX_INVITE_ENTRIES: usize = 64; const FLAG_STOCK_SET: u8 = 0x01; const INVITE_PATH: &str = "/invite/"; @@ -39,6 +45,9 @@ pub enum InviteError { Json(String), BadHex(&'static str), TooManyChannels(usize), + TooManyInvites(usize), + Oversize(usize), + Kind(u16), EpochTooLarge(u64), OwnerMismatch, BadFragment(&'static str), @@ -60,6 +69,16 @@ impl fmt::Display for InviteError { "bundle carries {count} channels (cap {MAX_BUNDLE_CHANNELS})" ) } + InviteError::TooManyInvites(count) => { + write!( + f, + "invite list carries {count} entries (cap {MAX_INVITE_ENTRIES})" + ) + } + InviteError::Oversize(len) => { + write!(f, "invite list is {len} bytes (cap {NIP44_MAX_PLAINTEXT})") + } + InviteError::Kind(kind) => write!(f, "not an invite list kind: {kind}"), InviteError::EpochTooLarge(epoch) => write!(f, "epoch {epoch} out of range"), InviteError::OwnerMismatch => { write!(f, "bundle owner does not reproduce its community_id") @@ -123,7 +142,6 @@ pub struct CommunityInvite { } impl CommunityInvite { - /// Parse, bound and validate a decrypted bundle, whichever lane carried it. pub fn from_bundle_json(json: &str) -> Result { let mut invite: Self = serde_json::from_str(json).map_err(|error| InviteError::Json(error.to_string()))?; @@ -472,6 +490,149 @@ pub fn unwrap_direct_invite( )) } +#[derive(Debug, Clone, PartialEq, Serialize, Deserialize)] +pub struct InviteEntry { + /// The link's unlock secret, and its merge key. + pub token: String, + /// The `link_signer` secret: refreshing or retiring the bundle needs it. + pub signer_sk: String, + pub community_id: CommunityId, + pub url: String, + #[serde(default, skip_serializing_if = "Option::is_none")] + pub label: Option, + pub created_at: u64, + #[serde(default, skip_serializing_if = "Option::is_none")] + pub expires_at: Option, + #[serde(flatten)] + pub extra: Extra, +} + +#[derive(Debug, Clone, PartialEq, Serialize, Deserialize)] +pub struct InviteTombstone { + pub token: String, + pub community_id: CommunityId, + #[serde(flatten)] + pub extra: Extra, +} + +/// A creator's own link bookkeeping. +#[derive(Debug, Clone, Default, PartialEq, Serialize, Deserialize)] +pub struct InviteList { + #[serde(default, skip_serializing_if = "Vec::is_empty")] + pub entries: Vec, + #[serde(default, skip_serializing_if = "Vec::is_empty")] + pub tombstones: Vec, + #[serde(flatten)] + pub extra: Extra, +} + +impl InviteList { + /// A tombstone beats an entry terminally, so a stale device can never resurrect a revoked link. + pub fn is_live(&self, token: &str) -> bool { + self.entries.iter().any(|entry| entry.token == token) + && !self + .tombstones + .iter() + .any(|tombstone| tombstone.token == token) + } + + pub fn fits(&self) -> Result<(), InviteError> { + if self.entries.len() > MAX_INVITE_ENTRIES { + return Err(InviteError::TooManyInvites(self.entries.len())); + } + + let json = serde_json::to_string(self).map_err(json_error)?; + + if json.len() > NIP44_MAX_PLAINTEXT { + return Err(InviteError::Oversize(json.len())); + } + + Ok(()) + } +} + +pub fn merge_invite_lists(held: InviteList, incoming: InviteList) -> InviteList { + let mut entries: BTreeMap = BTreeMap::new(); + + for entry in held.entries.into_iter().chain(incoming.entries) { + match entries.entry(entry.token.clone()) { + Entry::Vacant(slot) => { + slot.insert(entry); + } + Entry::Occupied(mut slot) => { + let merged = merge_entry(slot.get(), &entry); + *slot.get_mut() = merged; + } + } + } + + let mut tombstones: BTreeMap = BTreeMap::new(); + + for tombstone in held.tombstones.into_iter().chain(incoming.tombstones) { + match tombstones.entry(tombstone.token.clone()) { + Entry::Vacant(slot) => { + slot.insert(tombstone); + } + Entry::Occupied(mut slot) => { + if canonical(&tombstone) < canonical(slot.get()) { + *slot.get_mut() = tombstone; + } + } + } + } + + let mut extra = held.extra; + union(&mut extra, incoming.extra); + + InviteList { + entries: entries.into_values().collect(), + tombstones: tombstones.into_values().collect(), + extra, + } +} + +pub fn build_invite_list(keys: &Keys, list: &InviteList) -> Result { + list.fits()?; + + let json = serde_json::to_string(list).map_err(json_error)?; + let content = nip44::encrypt( + keys.secret_key(), + &keys.public_key(), + json.as_bytes(), + Version::V2, + ) + .map_err(crypto_error)?; + + EventBuilder::new(Kind::Custom(KIND_INVITE_LIST), content) + .finalize(keys) + .map_err(crypto_error) +} + +pub fn parse_invite_list(keys: &Keys, event: &Event) -> Result { + if event.kind.as_u16() != KIND_INVITE_LIST { + return Err(InviteError::Kind(event.kind.as_u16())); + } + + let json = nip44::decrypt(keys.secret_key(), &keys.public_key(), &event.content) + .map_err(crypto_error)?; + + serde_json::from_str(&json).map_err(json_error) +} + +/// An entry is immutable once minted, so two copies should agree. +fn merge_entry(held: &InviteEntry, incoming: &InviteEntry) -> InviteEntry { + let (winner, loser) = if canonical(incoming) < canonical(held) { + (incoming, held) + } else { + (held, incoming) + }; + + let mut merged = winner.clone(); + union(&mut merged.extra, loser.extra.clone()); + + merged +} + fn seal_bundle(bundle_key: &[u8; 32], json: &str) -> Result { Ok(stream::seal_bytes( &ConversationKey::new(*bundle_key), @@ -642,11 +803,6 @@ mod tests { decode_fragment(&BASE64URL_NOPAD.encode(&unknown)).expect("decodes"); assert_eq!(decoded, token); assert!(relays.is_empty()); - - let relays: Vec = (0..4).map(|i| format!("wss://r{i}.example")).collect(); - let (_, capped) = - decode_fragment(&encode_fragment(&token, &relays).expect("encodes")).expect("decodes"); - assert_eq!(capped.len(), MAX_BOOTSTRAP_RELAYS); } #[test] @@ -678,10 +834,6 @@ mod tests { parse_link("https://x/invite/#frag").is_err(), "the naddr is not optional" ); - assert!( - parse_link("wss://relay.example.com").is_err(), - "nor the fragment" - ); } #[test] diff --git a/crates/concord/src/lib.rs b/crates/concord/src/lib.rs index da4bb3cd..c4f72bb6 100644 --- a/crates/concord/src/lib.rs +++ b/crates/concord/src/lib.rs @@ -5,6 +5,7 @@ pub mod edition; pub mod guestbook; pub mod invite; pub mod list; +pub mod rekey; pub mod roles; pub mod store; pub mod stream; diff --git a/crates/concord/src/list.rs b/crates/concord/src/list.rs index 0c291440..bb8eb86d 100644 --- a/crates/concord/src/list.rs +++ b/crates/concord/src/list.rs @@ -43,7 +43,6 @@ impl fmt::Display for ListError { impl std::error::Error for ListError {} -/// A membership's keys. #[derive(Debug, Clone, PartialEq, Serialize, Deserialize)] pub struct JoinMaterial { pub community_id: CommunityId, @@ -83,7 +82,6 @@ pub struct Tombstone { pub extra: Extra, } -/// A member's own memberships. #[derive(Debug, Clone, Default, PartialEq, Serialize, Deserialize)] pub struct CommunityList { #[serde(default, skip_serializing_if = "Vec::is_empty")] @@ -242,7 +240,7 @@ fn pick<'a>( held } -fn union(into: &mut Extra, other: Extra) { +pub(crate) fn union(into: &mut Extra, other: Extra) { for (key, value) in other { let replace = match into.get(&key) { Some(existing) => canonical(&value) < canonical(existing), @@ -255,7 +253,7 @@ fn union(into: &mut Extra, other: Extra) { } } -fn canonical(value: &T) -> String { +pub(crate) fn canonical(value: &T) -> String { serde_json::to_string(value).unwrap_or_default() } diff --git a/crates/concord/src/rekey.rs b/crates/concord/src/rekey.rs new file mode 100644 index 00000000..d16e3b53 --- /dev/null +++ b/crates/concord/src/rekey.rs @@ -0,0 +1,1634 @@ +use std::collections::{BTreeMap, BTreeSet}; +use std::fmt; + +use anyhow::Result; +use data_encoding::HEXLOWER; +use nostr::nips::nip44::v2::ConversationKey; +use nostr_sdk::prelude::{Event, Keys, PublicKey, SecretKey, Tag, Timestamp, UnsignedEvent}; +use serde::{Deserialize, Serialize}; + +use crate::control::CommunityIdentity; +use crate::derive::{ + base_rekey_group_key, channel_rekey_group_key, control_group_key, control_signer_group_key, + dissolved_group_key, epoch_key_commitment, recipient_locator, +}; +use crate::edition::{ + AuthorityCitation, KIND_CONTROL, TAG_SUBKIND, canonical_decimal, citation_from, citation_tag, + vsk, +}; +use crate::roles::CommunityRoles; +use crate::stream::{self, KIND_SEAL_PLAINTEXT, OpenedStream, SealForm, StreamError}; +use crate::{ChannelId, CommunityId, Epoch, GroupKey, random_32}; + +pub const KIND_REKEY: u16 = 3303; +/// The send cap. A rekey rides the CORD-01 double envelope, so each blob costs two +/// NIP-44 base64 expansions: 120 blobs measure ~77 KB and a 64 KB relay refuses +/// them, while 80 measure ~55 KB. CORD-06 states 120 — an erratum this reproduces. +pub const MAX_REKEY_BLOBS_PER_EVENT: usize = 80; +/// The accept cap stays at the spec's 120, above the send cap, so a chunk minted by +/// another client at the spec limit still parses. +pub const MAX_REKEY_BLOBS_RECEIVED: usize = 120; +pub const MAX_REKEY_EPOCH: u64 = 1 << 40; + +const TAG_SCOPE: &str = "scope"; +const TAG_NEW_EPOCH: &str = "newepoch"; +const TAG_PREV_EPOCH: &str = "prevepoch"; +const TAG_PREV_COMMIT: &str = "prevcommit"; +const TAG_CHUNK: &str = "chunk"; +const TAG_SEVER: &str = "sever"; +const TAG_EID: &str = "eid"; + +const CHANNEL_BLOB_LEN: usize = 72; +const MEMBER_BASE_BLOB_LEN: usize = 104; +const STAFF_BASE_BLOB_LEN: usize = 136; + +#[derive(Debug)] +pub enum RekeyError { + Stream(StreamError), + Crypto(String), + Json(String), + BadBlobLength(usize), + BadBaseBlobWidth(usize), + ControlPairMismatch, + MisplacedControlKey, + ScopeSplice, + EpochSplice, + NotARekey(u16), + NotADissolution, + BadTag(&'static str), + NonMonotonicEpoch, + EpochTooLarge(u64), + BadChunkIndex, + TooManyBlobs(usize), +} + +impl fmt::Display for RekeyError { + fn fmt(&self, f: &mut fmt::Formatter<'_>) -> fmt::Result { + match self { + RekeyError::Stream(error) => write!(f, "stream: {error}"), + RekeyError::Crypto(error) => write!(f, "crypto: {error}"), + RekeyError::Json(error) => write!(f, "json: {error}"), + RekeyError::BadBlobLength(len) => { + write!(f, "rekey blob plaintext is {len} bytes, expected 72") + } + RekeyError::BadBaseBlobWidth(len) => write!( + f, + "base rekey blob plaintext is {len} bytes, expected 72, 104 or 136" + ), + RekeyError::ControlPairMismatch => { + write!( + f, + "base rekey blob control_root does not derive to its control_pk" + ) + } + RekeyError::MisplacedControlKey => { + write!(f, "rekey blob carries a control key at the wrong width") + } + RekeyError::ScopeSplice => write!(f, "rekey blob scope does not match its coordinate"), + RekeyError::EpochSplice => write!(f, "rekey blob epoch does not match its coordinate"), + RekeyError::NotARekey(kind) => write!(f, "kind {kind} is not a rekey"), + RekeyError::NotADissolution => write!(f, "not a dissolution tombstone"), + RekeyError::BadTag(name) => write!(f, "missing, repeated or malformed tag: {name}"), + RekeyError::NonMonotonicEpoch => write!(f, "a rotation must advance the epoch"), + RekeyError::EpochTooLarge(epoch) => write!(f, "epoch {epoch} out of range"), + RekeyError::BadChunkIndex => write!(f, "rekey chunk index out of range"), + RekeyError::TooManyBlobs(count) => { + write!(f, "rekey carries {count} blobs, over the cap") + } + } + } +} + +impl std::error::Error for RekeyError {} + +impl From for RekeyError { + fn from(error: StreamError) -> Self { + RekeyError::Stream(error) + } +} + +/// What a rotation rotates: one private channel, or the whole community base. +#[derive(Debug, Clone, Copy, PartialEq, Eq)] +pub enum RekeyScope { + Channel(ChannelId), + Base, +} + +impl RekeyScope { + /// The all-zero sentinel addresses the base; a channel id is random, so it + /// never collides. The value is stamped inside every blob's ciphertext. + pub fn id32(self) -> [u8; 32] { + match self { + RekeyScope::Channel(channel) => *channel.as_bytes(), + RekeyScope::Base => [0u8; 32], + } + } + + fn to_hex(self) -> String { + HEXLOWER.encode(&self.id32()) + } + + fn from_hex(raw: &str) -> Option { + let bytes = crate::decode_hex_32(raw).ok()?; + + if bytes == [0u8; 32] { + return Some(RekeyScope::Base); + } + + Some(RekeyScope::Channel(ChannelId::from_bytes(bytes))) + } +} + +#[derive(Debug, Clone, PartialEq, Eq, Serialize, Deserialize)] +pub struct RekeyBlob { + pub locator: String, + pub wrapped: String, +} + +/// The plaintext a blob delivered. The width declared which fields ride. +#[derive(Debug, Clone, Copy, PartialEq, Eq)] +pub struct KeyDelivery { + pub new_key: [u8; 32], + pub control_pk: Option<[u8; 32]>, + pub control_root: Option<[u8; 32]>, +} + +pub fn encode_blob_plaintext( + scope: RekeyScope, + epoch: Epoch, + new_key: &[u8; 32], + control_pk: Option<&[u8; 32]>, + control_root: Option<&[u8; 32]>, +) -> Result, RekeyError> { + if control_root.is_some() && control_pk.is_none() { + return Err(RekeyError::MisplacedControlKey); + } + + let mut bytes = Vec::with_capacity(STAFF_BASE_BLOB_LEN); + bytes.extend_from_slice(&scope.id32()); + bytes.extend_from_slice(&epoch.0.to_be_bytes()); + bytes.extend_from_slice(new_key); + + match (scope, control_pk) { + (RekeyScope::Base, Some(control_pk)) => { + bytes.extend_from_slice(control_pk); + + if let Some(control_root) = control_root { + bytes.extend_from_slice(control_root); + } + } + (RekeyScope::Base | RekeyScope::Channel(_), None) => {} + (RekeyScope::Channel(_), Some(_)) => return Err(RekeyError::MisplacedControlKey), + } + + Ok(bytes) +} + +pub fn parse_blob_plaintext( + bytes: &[u8], + scope: RekeyScope, + epoch: Epoch, + community_id: &CommunityId, +) -> Result { + if bytes.len() < CHANNEL_BLOB_LEN { + return Err(RekeyError::BadBlobLength(bytes.len())); + } + + if bytes[..32] != scope.id32() { + return Err(RekeyError::ScopeSplice); + } + + let mut epoch_be = [0u8; 8]; + epoch_be.copy_from_slice(&bytes[32..40]); + + if u64::from_be_bytes(epoch_be) != epoch.0 { + return Err(RekeyError::EpochSplice); + } + + let mut new_key = [0u8; 32]; + new_key.copy_from_slice(&bytes[40..CHANNEL_BLOB_LEN]); + + if let RekeyScope::Channel(_) = scope { + if bytes.len() != CHANNEL_BLOB_LEN { + return Err(RekeyError::BadBlobLength(bytes.len())); + } + + return Ok(KeyDelivery { + new_key, + control_pk: None, + control_root: None, + }); + } + + let width = bytes.len(); + + if width == CHANNEL_BLOB_LEN { + return Ok(KeyDelivery { + new_key, + control_pk: None, + control_root: None, + }); + } + + if width != MEMBER_BASE_BLOB_LEN && width != STAFF_BASE_BLOB_LEN && width < STAFF_BASE_BLOB_LEN + { + return Err(RekeyError::BadBaseBlobWidth(width)); + } + + let mut control_pk = [0u8; 32]; + control_pk.copy_from_slice(&bytes[CHANNEL_BLOB_LEN..MEMBER_BASE_BLOB_LEN]); + + if width == MEMBER_BASE_BLOB_LEN { + return Ok(KeyDelivery { + new_key, + control_pk: Some(control_pk), + control_root: None, + }); + } + + let mut control_root = [0u8; 32]; + control_root.copy_from_slice(&bytes[MEMBER_BASE_BLOB_LEN..STAFF_BASE_BLOB_LEN]); + + if control_signer_group_key(&control_root, community_id, epoch) + .map_err(crypto_error)? + .pk() + .to_bytes() + != control_pk + { + if width == STAFF_BASE_BLOB_LEN { + return Err(RekeyError::ControlPairMismatch); + } + + // A width past 136 is a form this client predates. Refusing it would park + // the member at the old epoch, so the frozen prefix and the appended fields + // that still verify are kept and the rest freezes. + return Ok(KeyDelivery { + new_key, + control_pk: Some(control_pk), + control_root: None, + }); + } + + Ok(KeyDelivery { + new_key, + control_pk: Some(control_pk), + control_root: Some(control_root), + }) +} + +/// The rekey plane's address for a scope. A standalone channel rotation rides the +/// current root; one forced by a removal rides the prior root beside the base +/// rotation, which is exactly what lets a base-fork loser still open it. +pub fn rekey_group( + scope: RekeyScope, + addressing_root: &[u8; 32], + community_id: &CommunityId, + new_epoch: Epoch, +) -> Result { + match scope { + RekeyScope::Channel(channel) => { + channel_rekey_group_key(addressing_root, &channel, new_epoch) + } + RekeyScope::Base => base_rekey_group_key(addressing_root, community_id, new_epoch), + } +} + +pub fn blob_locator( + rotator: &PublicKey, + recipient: &PublicKey, + scope: RekeyScope, + epoch: Epoch, +) -> String { + HEXLOWER.encode(&recipient_locator( + &rotator.to_bytes(), + &recipient.to_bytes(), + &scope.id32(), + epoch, + )) +} + +pub fn build_blob( + rotator: &Keys, + recipient: &PublicKey, + scope: RekeyScope, + epoch: Epoch, + new_key: &[u8; 32], + control_pk: Option<&[u8; 32]>, + control_root: Option<&[u8; 32]>, +) -> Result { + let plaintext = encode_blob_plaintext(scope, epoch, new_key, control_pk, control_root)?; + + Ok(RekeyBlob { + locator: blob_locator(&rotator.public_key(), recipient, scope, epoch), + wrapped: seal_to(rotator.secret_key(), recipient, &plaintext)?, + }) +} + +/// The locator is public and authenticates nothing, so it is not gated here: +/// the pairwise decrypt, plus the scope and epoch bound inside the ciphertext, +/// are the whole gate. +pub fn open_blob( + recipient: &Keys, + rotator: &PublicKey, + scope: RekeyScope, + epoch: Epoch, + blob: &RekeyBlob, + community_id: &CommunityId, +) -> Result { + let conversation = + ConversationKey::derive(recipient.secret_key(), rotator).map_err(crypto_error)?; + let plaintext = stream::open_bytes(&conversation, &blob.wrapped)?; + + parse_blob_plaintext(&plaintext, scope, epoch, community_id) +} + +/// Every blob at my locator. Anyone can publish a blob at mine, since the locator +/// is public, so the caller tries each and adopts the first that opens. +pub fn find_my_blobs<'a>( + blobs: &'a [RekeyBlob], + rotator: &PublicKey, + me: &PublicKey, + scope: RekeyScope, + epoch: Epoch, +) -> impl Iterator { + let wanted = blob_locator(rotator, me, scope, epoch); + + blobs.iter().filter(move |blob| blob.locator == wanted) +} + +fn seal_to( + secret: &SecretKey, + recipient: &PublicKey, + plaintext: &[u8], +) -> Result { + let conversation = ConversationKey::derive(secret, recipient).map_err(crypto_error)?; + + Ok(stream::seal_bytes(&conversation, plaintext)?) +} + +#[derive(Debug, Clone)] +pub struct RekeyChunk { + pub rotator: PublicKey, + pub scope: RekeyScope, + pub new_epoch: Epoch, + pub prev_epoch: Epoch, + pub prev_commit: [u8; 32], + pub chunk: (u32, u32), + pub blobs: Vec, + pub citation: Option, + pub severed: bool, +} + +/// The key that groups the chunks of one rotation. Two rotators racing the same +/// epoch, or one rotator over two channels, never alias. +pub type RotationKey = ([u8; 32], [u8; 32], u64, [u8; 32]); + +impl RekeyChunk { + pub fn correlation(&self) -> RotationKey { + ( + self.rotator.to_bytes(), + self.scope.id32(), + self.new_epoch.0, + self.prev_commit, + ) + } +} + +#[derive(Debug, Clone)] +pub struct Rotation { + pub rotator: PublicKey, + pub scope: RekeyScope, + pub new_epoch: Epoch, + pub prev_epoch: Epoch, + pub prev_commit: [u8; 32], + pub blobs: Vec, + pub declared: u32, + pub held: BTreeSet, + /// OR across chunks: an extension minted without the marker must not launder + /// a severed rotation back into an ordinary one. + pub severed: bool, + pub citation: Option, +} + +impl Rotation { + /// Every declared index held. A missing chunk is never a removal. + pub fn is_complete(&self) -> bool { + self.declared >= 1 && (1..=self.declared).all(|index| self.held.contains(&index)) + } + + pub fn continuity(&self, held_epoch: Epoch, held_key: &[u8; 32]) -> Continuity { + continuity(self.prev_epoch, &self.prev_commit, held_epoch, held_key) + } +} + +#[derive(Debug, Clone, Copy, PartialEq, Eq)] +pub enum Continuity { + Extends, + Gap, + Fork, +} + +pub fn collect_rotations(chunks: &[RekeyChunk]) -> Vec { + let mut by_key: BTreeMap = BTreeMap::new(); + + for chunk in chunks { + let rotation = by_key + .entry(chunk.correlation()) + .or_insert_with(|| Rotation { + rotator: chunk.rotator, + scope: chunk.scope, + new_epoch: chunk.new_epoch, + prev_epoch: chunk.prev_epoch, + prev_commit: chunk.prev_commit, + blobs: Vec::new(), + declared: chunk.chunk.1, + held: BTreeSet::new(), + severed: chunk.severed, + citation: chunk.citation, + }); + + rotation.severed |= chunk.severed; + rotation.held.insert(chunk.chunk.0); + + // A union, never first-wins: two chunks can claim one index after a + // catch-up, and a recipient dropped from the union reads as removed. + for blob in &chunk.blobs { + if !rotation.blobs.iter().any(|held| held == blob) { + rotation.blobs.push(blob.clone()); + } + } + } + + by_key.into_values().collect() +} + +/// `None` until every chunk is held: an incomplete set is never a removal. +pub fn am_i_removed(rotation: &Rotation, me: &PublicKey) -> Option { + if !rotation.is_complete() { + return None; + } + + Some( + find_my_blobs( + &rotation.blobs, + &rotation.rotator, + me, + rotation.scope, + rotation.new_epoch, + ) + .next() + .is_none(), + ) +} + +fn continuity( + prev_epoch: Epoch, + prev_commit: &[u8; 32], + held_epoch: Epoch, + held_key: &[u8; 32], +) -> Continuity { + if prev_epoch.0 == held_epoch.0 { + return if epoch_key_commitment(held_epoch, held_key) == *prev_commit { + Continuity::Extends + } else { + Continuity::Fork + }; + } + + if prev_epoch.0 > held_epoch.0 { + Continuity::Gap + } else { + Continuity::Fork + } +} + +/// The winner among concurrent rotations at one continuity point: the lowest key, +/// adopted only when it strictly lowers a key already held. A settled epoch heals +/// down and never re-forks upward. +pub fn fork_winner(held: Option<&[u8; 32]>, candidates: &[[u8; 32]]) -> Option { + let (index, winner) = candidates.iter().enumerate().min_by_key(|(_, key)| **key)?; + + match held { + Some(held) if winner >= held => None, + _ => Some(index), + } +} + +/// Holding a key is never authority, so a rotation is honored only from an actor +/// with the permission who strictly outranks every target it removes. +pub fn rekey_authorized( + roles: &CommunityRoles, + owner: &PublicKey, + rotator: &PublicKey, + permission: u64, + removed: &[PublicKey], +) -> bool { + roles.is_authorized(rotator, owner, permission) + && removed + .iter() + .all(|target| roles.can_act_on_member(rotator, owner, target, permission)) +} + +#[derive(Debug, Clone, Copy, PartialEq, Eq)] +pub struct Refounding { + pub epoch: Epoch, + pub new_root: [u8; 32], + pub new_control_root: [u8; 32], +} + +impl Refounding { + pub fn read(&self, community_id: &CommunityId) -> Result { + control_group_key(&self.new_root, community_id, self.epoch) + } + + pub fn signer(&self, community_id: &CommunityId) -> Result { + control_signer_group_key(&self.new_control_root, community_id, self.epoch) + } +} + +pub fn plan_refounding(epoch: Epoch) -> Result { + Ok(Refounding { + epoch, + new_root: random_32()?, + new_control_root: random_32()?, + }) +} + +/// Carries the settled heads across a refounding. The control plane is +/// plaintext-sealed precisely so this preserves the original authors' signatures +/// instead of re-signing a snapshot as the refounder. +pub fn compact( + seals: &[Event], + read: &GroupKey, + signer: &GroupKey, + at_secs: u64, +) -> Result, RekeyError> { + let at = Timestamp::from_secs(at_secs); + let mut wraps = Vec::with_capacity(seals.len()); + + for seal in seals { + wraps.push(stream::rewrap_seal(seal, read, signer, at)?.0); + } + + Ok(wraps) +} + +#[allow(clippy::too_many_arguments)] +pub fn build_rekey_rumor( + rotator: PublicKey, + scope: RekeyScope, + new_epoch: Epoch, + prev_epoch: Epoch, + prev_commit: &[u8; 32], + blobs: &[RekeyBlob], + chunk: (u32, u32), + citation: Option<&AuthorityCitation>, + severed: bool, + at_secs: u64, +) -> Result { + if new_epoch.0 <= prev_epoch.0 { + return Err(RekeyError::NonMonotonicEpoch); + } + + if new_epoch.0 > MAX_REKEY_EPOCH { + return Err(RekeyError::EpochTooLarge(new_epoch.0)); + } + + if chunk.1 < 1 || chunk.0 < 1 || chunk.0 > chunk.1 { + return Err(RekeyError::BadChunkIndex); + } + + if blobs.len() > MAX_REKEY_BLOBS_PER_EVENT { + return Err(RekeyError::TooManyBlobs(blobs.len())); + } + + let content = serde_json::to_string(blobs).map_err(json_error)?; + + let mut tags = vec![ + Tag::custom(TAG_SCOPE, [scope.to_hex()]), + Tag::custom(TAG_NEW_EPOCH, [new_epoch.0.to_string()]), + Tag::custom(TAG_PREV_EPOCH, [prev_epoch.0.to_string()]), + Tag::custom(TAG_PREV_COMMIT, [HEXLOWER.encode(prev_commit)]), + Tag::custom(TAG_CHUNK, [chunk.0.to_string(), chunk.1.to_string()]), + ]; + + if let Some(citation) = citation { + tags.push(citation_tag(citation)); + } + + if severed { + tags.push(Tag::custom(TAG_SEVER, ["1"])); + } + + Ok(stream::build_rumor_secs( + KIND_REKEY, rotator, &content, tags, at_secs, + )) +} + +#[allow(clippy::too_many_arguments)] +pub fn build_rekey_chunks( + rotator: &Keys, + group: &GroupKey, + scope: RekeyScope, + new_epoch: Epoch, + prev_epoch: Epoch, + prev_commit: &[u8; 32], + blobs: &[RekeyBlob], + citation: Option<&AuthorityCitation>, + severed: bool, + at_secs: u64, +) -> Result, RekeyError> { + let mut groups: Vec<&[RekeyBlob]> = blobs.chunks(MAX_REKEY_BLOBS_PER_EVENT).collect(); + + if groups.is_empty() { + groups.push(&[]); + } + + let total = groups.len() as u32; + let mut chunks = Vec::with_capacity(groups.len()); + + for (index, group_blobs) in groups.into_iter().enumerate() { + let rumor = build_rekey_rumor( + rotator.public_key(), + scope, + new_epoch, + prev_epoch, + prev_commit, + group_blobs, + (index as u32 + 1, total), + citation, + severed, + at_secs, + )?; + + let seal = stream::build_seal(&rumor, SealForm::Encrypted, group, rotator)?; + let (wrap, _) = stream::wrap_seal( + &seal, + group, + stream::KIND_WRAP, + Timestamp::from_secs(at_secs), + &[], + )?; + + chunks.push(wrap); + } + + Ok(chunks) +} + +pub fn parse_rekey_chunk(opened: &OpenedStream) -> Result { + if opened.seal_form != SealForm::Encrypted { + // A plaintext-sealed rekey would be a public artifact anyone could lift. + return Err(RekeyError::Stream(StreamError::BadSealKind( + KIND_SEAL_PLAINTEXT, + ))); + } + + let rumor = &opened.rumor; + + if rumor.kind.as_u16() != KIND_REKEY { + return Err(RekeyError::NotARekey(rumor.kind.as_u16())); + } + + let scope = tag(rumor, TAG_SCOPE)? + .and_then(|fields| fields.get(1)) + .and_then(|raw| RekeyScope::from_hex(raw)) + .ok_or(RekeyError::BadTag(TAG_SCOPE))?; + + let new_epoch = Epoch(decimal(rumor, TAG_NEW_EPOCH)?); + let prev_epoch = Epoch(decimal(rumor, TAG_PREV_EPOCH)?); + + if new_epoch.0 <= prev_epoch.0 { + return Err(RekeyError::NonMonotonicEpoch); + } + + if new_epoch.0 > MAX_REKEY_EPOCH { + return Err(RekeyError::EpochTooLarge(new_epoch.0)); + } + + let prev_commit = tag(rumor, TAG_PREV_COMMIT)? + .and_then(|fields| fields.get(1)) + .and_then(|raw| crate::decode_hex_32(raw).ok()) + .ok_or(RekeyError::BadTag(TAG_PREV_COMMIT))?; + + let blobs: Vec = + serde_json::from_str(&rumor.content).map_err(|_| RekeyError::BadTag("blobs"))?; + + if blobs.len() > MAX_REKEY_BLOBS_RECEIVED { + return Err(RekeyError::TooManyBlobs(blobs.len())); + } + + let severed = match tag(rumor, TAG_SEVER)? { + None => false, + Some(fields) if fields.get(1).map(String::as_str) == Some("1") => true, + Some(_) => return Err(RekeyError::BadTag(TAG_SEVER)), + }; + + Ok(RekeyChunk { + rotator: opened.author, + scope, + new_epoch, + prev_epoch, + prev_commit, + chunk: parse_chunk(rumor)?, + blobs, + citation: tag(rumor, crate::edition::TAG_CITATION)?.and_then(citation_from), + severed, + }) +} + +#[derive(Debug, Clone, Copy, PartialEq, Eq)] +pub struct DissolvedTombstone { + pub owner: PublicKey, +} + +/// The `eid` commits the community, deliberately diverging from the all-zero +/// placeholder CORD-02 §9 shows: the dissolved address derives from the public +/// `community_id`, so a zero binding lets an owner's genuine tombstone for one of +/// their communities be re-wrapped at another and kill it. +pub fn dissolved_tombstone_rumor( + owner: PublicKey, + community_id: &CommunityId, + at_secs: u64, +) -> UnsignedEvent { + stream::build_rumor_secs( + KIND_CONTROL, + owner, + "", + vec![ + Tag::custom(TAG_SUBKIND, [vsk::DISSOLVED]), + Tag::custom(TAG_EID, [HEXLOWER.encode(community_id.as_bytes())]), + ], + at_secs, + ) +} + +pub fn seal_dissolved( + rumor: &UnsignedEvent, + community_id: &CommunityId, + owner: &Keys, + at_secs: u64, +) -> Result { + let group = dissolved_group_key(community_id).map_err(crypto_error)?; + let seal = stream::build_seal(rumor, SealForm::Plaintext, &group, owner)?; + let (wrap, _) = stream::wrap_seal( + &seal, + &group, + stream::KIND_WRAP, + Timestamp::from_secs(at_secs), + &[], + )?; + + Ok(wrap) +} + +/// Proves the seal signature and the tombstone shape, but not the owner. +pub fn open_dissolved( + wrap: &Event, + community_id: &CommunityId, +) -> Result { + let group = dissolved_group_key(community_id).map_err(crypto_error)?; + let opened = stream::open_wrap(wrap, &group)?; + + if !is_tombstone(&opened.rumor, community_id) { + return Err(RekeyError::NotADissolution); + } + + Ok(DissolvedTombstone { + owner: opened.author, + }) +} + +/// Fail-closed: an unverifiable or foreign-signed tombstone is not death. +pub fn verify_dissolved(wrap: &Event, identity: &CommunityIdentity) -> bool { + if !identity.verify() { + return false; + } + + matches!( + open_dissolved(wrap, &identity.community_id), + Ok(tombstone) if tombstone.owner == identity.owner + ) +} + +fn is_tombstone(rumor: &UnsignedEvent, community_id: &CommunityId) -> bool { + let eid = HEXLOWER.encode(community_id.as_bytes()); + + rumor.kind.as_u16() == KIND_CONTROL + && value(rumor, TAG_SUBKIND) == Some(vsk::DISSOLVED) + && value(rumor, TAG_EID) == Some(eid.as_str()) +} + +fn value<'a>(rumor: &'a UnsignedEvent, name: &'static str) -> Option<&'a str> { + tag(rumor, name) + .ok() + .flatten() + .and_then(|fields| fields.get(1)) + .map(String::as_str) +} + +fn tag<'a>( + rumor: &'a UnsignedEvent, + name: &'static str, +) -> Result, RekeyError> { + let mut found: Option<&[String]> = None; + + for candidate in rumor.tags.iter() { + let fields = candidate.as_slice(); + + if fields.first().map(String::as_str) != Some(name) { + continue; + } + + if found.is_some() { + return Err(RekeyError::BadTag(name)); + } + + found = Some(fields); + } + + Ok(found) +} + +fn decimal(rumor: &UnsignedEvent, name: &'static str) -> Result { + tag(rumor, name)? + .and_then(|fields| fields.get(1)) + .and_then(|raw| canonical_decimal(raw)) + .ok_or(RekeyError::BadTag(name)) +} + +fn parse_chunk(rumor: &UnsignedEvent) -> Result<(u32, u32), RekeyError> { + let fields = tag(rumor, TAG_CHUNK)?.ok_or(RekeyError::BadTag(TAG_CHUNK))?; + + let index = as_u32(fields.get(1), TAG_CHUNK)?; + let total = as_u32(fields.get(2), TAG_CHUNK)?; + + if total < 1 || index < 1 || index > total { + return Err(RekeyError::BadChunkIndex); + } + + Ok((index, total)) +} + +fn as_u32(raw: Option<&String>, name: &'static str) -> Result { + let value = raw + .and_then(|raw| canonical_decimal(raw)) + .ok_or(RekeyError::BadTag(name))?; + + u32::try_from(value).map_err(|_| RekeyError::BadTag(name)) +} + +fn json_error(error: serde_json::Error) -> RekeyError { + RekeyError::Json(error.to_string()) +} + +fn crypto_error(error: impl fmt::Display) -> RekeyError { + RekeyError::Crypto(error.to_string()) +} + +#[cfg(test)] +mod tests { + use std::collections::BTreeSet; + + use super::*; + use crate::control::{ + CommunityMetadata, ControlWriter, Edition, ROOT_EPOCH, fold_control, genesis, open_edition, + }; + use crate::derive::{community_id_of, grant_locator}; + use crate::edition::{EditionFields, Floors, build_edition}; + use crate::roles::{Grant, Permissions, Role, RoleScope}; + use crate::stream::KIND_WRAP; + use crate::{Extra, RoleId}; + + const AT: u64 = 1_700_000_000; + const ROOT: [u8; 32] = [0x55; 32]; + const PRIOR_KEY: [u8; 32] = [0xEE; 32]; + + fn channel() -> ChannelId { + ChannelId::from_bytes([0x42; 32]) + } + + fn community() -> CommunityId { + CommunityId::from_bytes([0x77; 32]) + } + + fn chunk_at( + rotator: &Keys, + scope: RekeyScope, + new_epoch: u64, + prev_epoch: u64, + prev_key: &[u8; 32], + blobs: Vec, + chunk: (u32, u32), + ) -> RekeyChunk { + RekeyChunk { + rotator: rotator.public_key(), + scope, + new_epoch: Epoch(new_epoch), + prev_epoch: Epoch(prev_epoch), + prev_commit: epoch_key_commitment(Epoch(prev_epoch), prev_key), + chunk, + blobs, + citation: None, + severed: false, + } + } + + #[test] + fn a_blob_binds_its_scope_and_the_width_declares_the_base_form() { + let rotator = Keys::generate(); + let recipient = Keys::generate(); + let community_id = community(); + let epoch = Epoch(3); + let key = [0xABu8; 32]; + let scope = RekeyScope::Channel(channel()); + + let open = |keys: &Keys, scope: RekeyScope, epoch: Epoch, blob: &RekeyBlob| { + open_blob( + keys, + &rotator.public_key(), + scope, + epoch, + blob, + &community_id, + ) + }; + let blob = build_blob( + &rotator, + &recipient.public_key(), + scope, + epoch, + &key, + None, + None, + ) + .expect("builds"); + + assert_eq!( + blob.locator, + blob_locator(&rotator.public_key(), &recipient.public_key(), scope, epoch) + ); + + let delivery = open(&recipient, scope, epoch, &blob).expect("opens"); + assert_eq!(delivery.new_key, key); + assert_eq!(delivery.control_pk, None); + + // The locator is a public lookup index, so an outsider computes it and + // still cannot open: the pairwise decrypt is the gate. + let outsider = Keys::generate(); + assert!(open(&outsider, scope, epoch, &blob).is_err()); + + assert!(matches!( + open(&recipient, RekeyScope::Base, epoch, &blob), + Err(RekeyError::ScopeSplice) + )); + assert!(matches!( + open(&recipient, scope, Epoch(4), &blob), + Err(RekeyError::EpochSplice) + )); + + let control_root = [0x5Cu8; 32]; + let control_pk = control_signer_group_key(&control_root, &community_id, epoch) + .expect("derives") + .pk() + .to_bytes(); + + let base = |pk: Option<&[u8; 32]>, root: Option<&[u8; 32]>| { + build_blob( + &rotator, + &recipient.public_key(), + RekeyScope::Base, + epoch, + &key, + pk, + root, + ) + .expect("builds") + }; + + for (pk, root, expected_pk, expected_root) in [ + (None, None, None, None), + (Some(&control_pk), None, Some(control_pk), None), + ( + Some(&control_pk), + Some(&control_root), + Some(control_pk), + Some(control_root), + ), + ] { + let delivery = + open(&recipient, RekeyScope::Base, epoch, &base(pk, root)).expect("opens"); + + assert_eq!(delivery.new_key, key); + assert_eq!(delivery.control_pk, expected_pk); + assert_eq!(delivery.control_root, expected_root); + } + + // A staff secret that does not derive to the pk beside it refuses the whole + // blob, rather than adopting a control plane split from its readers. + let forged = encode_blob_plaintext( + RekeyScope::Base, + epoch, + &key, + Some(&control_pk), + Some(&[0x11; 32]), + ) + .expect("encodes"); + assert!(matches!( + parse_blob_plaintext(&forged, RekeyScope::Base, epoch, &community_id), + Err(RekeyError::ControlPairMismatch) + )); + + // A width between the defined forms fits no append-only extension. + let staff = encode_blob_plaintext( + RekeyScope::Base, + epoch, + &key, + Some(&control_pk), + Some(&control_root), + ) + .expect("encodes"); + + for width in [73usize, 105, 135] { + let mut bytes = staff.clone(); + bytes.truncate(width); + + assert!(matches!( + parse_blob_plaintext(&bytes, RekeyScope::Base, epoch, &community_id), + Err(RekeyError::BadBaseBlobWidth(len)) if len == width + )); + } + + assert!(matches!( + encode_blob_plaintext(RekeyScope::Base, epoch, &key, None, Some(&control_root)), + Err(RekeyError::MisplacedControlKey) + )); + } + + #[test] + fn a_rekey_round_trips_and_only_a_complete_rotation_concludes_a_removal() { + let rotator = Keys::generate(); + let me = Keys::generate(); + let other = Keys::generate(); + let scope = RekeyScope::Channel(channel()); + let epoch = Epoch(1); + let community_id = community(); + + let blob_for = |recipient: &Keys, key: [u8; 32]| { + build_blob( + &rotator, + &recipient.public_key(), + scope, + epoch, + &key, + None, + None, + ) + .expect("builds") + }; + let mine = blob_for(&me, [0xAA; 32]); + let theirs = blob_for(&other, [0xBB; 32]); + + let group = rekey_group(scope, &ROOT, &community_id, epoch).expect("derives"); + let prior_commit = epoch_key_commitment(Epoch(0), &PRIOR_KEY); + let chunks = build_rekey_chunks( + &rotator, + &group, + scope, + epoch, + Epoch(0), + &prior_commit, + &[mine.clone(), theirs.clone()], + None, + false, + AT, + ) + .expect("builds"); + assert_eq!(chunks.len(), 1); + + let opened = stream::open_wrap(&chunks[0], &group).expect("opens"); + let chunk = parse_rekey_chunk(&opened).expect("parses"); + assert_eq!( + chunk.rotator, + rotator.public_key(), + "the seal names the rotator" + ); + assert_eq!(chunk.scope, scope); + assert_eq!((chunk.new_epoch, chunk.prev_epoch), (epoch, Epoch(0))); + assert_eq!(chunk.prev_commit, prior_commit); + assert_eq!( + chunk.prev_commit, + epoch_key_commitment(Epoch(0), &PRIOR_KEY) + ); + assert_eq!(chunk.chunk, (1, 1)); + assert_eq!(chunk.blobs, vec![mine.clone(), theirs.clone()]); + + // One of two chunks held, and it lacks my blob: not answerable yet. + let first = chunk_at( + &rotator, + scope, + 1, + 0, + &PRIOR_KEY, + vec![theirs.clone()], + (1, 2), + ); + let rotations = collect_rotations(std::slice::from_ref(&first)); + assert!(!rotations[0].is_complete()); + assert_eq!(am_i_removed(&rotations[0], &me.public_key()), None); + + // The second arrives with my blob: complete, and I am retained. + let second = chunk_at( + &rotator, + scope, + 1, + 0, + &PRIOR_KEY, + vec![mine.clone()], + (2, 2), + ); + let rotations = collect_rotations(&[first, second]); + assert!(rotations[0].is_complete()); + assert_eq!(am_i_removed(&rotations[0], &me.public_key()), Some(false)); + + let located = find_my_blobs( + &rotations[0].blobs, + &rotator.public_key(), + &me.public_key(), + scope, + epoch, + ) + .next() + .expect("located"); + assert_eq!( + open_blob( + &me, + &rotator.public_key(), + scope, + epoch, + located, + &community_id + ) + .expect("opens") + .new_key, + [0xAA; 32] + ); + + // A complete rotation carrying only someone else's blob is a removal. + let alone = chunk_at( + &rotator, + scope, + 1, + 0, + &PRIOR_KEY, + vec![theirs.clone()], + (1, 1), + ); + assert_eq!( + am_i_removed(&collect_rotations(&[alone])[0], &me.public_key()), + Some(true) + ); + + // Two chunks claiming one index union their blobs. Dropping the loser's + // blobs would delete its recipients from the union, and a recipient with no + // blob reads as removed. + let reclaim = chunk_at(&rotator, scope, 1, 0, &PRIOR_KEY, vec![theirs], (1, 1)); + let original = chunk_at(&rotator, scope, 1, 0, &PRIOR_KEY, vec![mine], (1, 1)); + + for order in [ + vec![original.clone(), reclaim.clone()], + vec![reclaim, original], + ] { + let rotations = collect_rotations(&order); + assert_eq!(rotations.len(), 1); + assert_eq!( + am_i_removed(&rotations[0], &me.public_key()), + Some(false), + "an index collision must never fabricate a removal" + ); + } + } + + #[test] + fn a_severed_rotation_is_not_laundered_by_an_unmarked_sibling() { + let rotator = Keys::generate(); + let scope = RekeyScope::Base; + let mut marked = chunk_at(&rotator, scope, 2, 1, &PRIOR_KEY, vec![], (1, 2)); + marked.severed = true; + let unmarked = chunk_at(&rotator, scope, 2, 1, &PRIOR_KEY, vec![], (2, 2)); + + for order in [ + vec![marked.clone(), unmarked.clone()], + vec![unmarked, marked], + ] { + let rotations = collect_rotations(&order); + assert_eq!(rotations.len(), 1); + assert!(rotations[0].severed); + assert!(rotations[0].is_complete()); + } + + // Two rotators racing one epoch, and one rotator over two scopes, never merge. + let other = Keys::generate(); + let rotations = collect_rotations(&[ + chunk_at(&rotator, RekeyScope::Base, 2, 1, &PRIOR_KEY, vec![], (1, 1)), + chunk_at(&other, RekeyScope::Base, 2, 1, &PRIOR_KEY, vec![], (1, 1)), + chunk_at( + &rotator, + RekeyScope::Channel(channel()), + 2, + 1, + &PRIOR_KEY, + vec![], + (1, 1), + ), + ]); + assert_eq!(rotations.len(), 3); + } + + #[test] + fn continuity_gaps_and_forks_and_the_winner_heals_only_downward() { + let rotator = Keys::generate(); + let held = [0x33u8; 32]; + + let extends = chunk_at(&rotator, RekeyScope::Base, 3, 2, &held, vec![], (1, 1)); + assert_eq!( + collect_rotations(&[extends])[0].continuity(Epoch(2), &held), + Continuity::Extends + ); + + let ahead = chunk_at(&rotator, RekeyScope::Base, 5, 4, &held, vec![], (1, 1)); + assert_eq!( + collect_rotations(&[ahead])[0].continuity(Epoch(2), &held), + Continuity::Gap + ); + + // The same epoch with a different prior key, and a rotation older than where + // I am, are both forks. + let forked = chunk_at( + &rotator, + RekeyScope::Base, + 3, + 2, + &[0x99; 32], + vec![], + (1, 1), + ); + assert_eq!( + collect_rotations(&[forked])[0].continuity(Epoch(2), &held), + Continuity::Fork + ); + + let stale = chunk_at(&rotator, RekeyScope::Base, 2, 1, &held, vec![], (1, 1)); + assert_eq!( + collect_rotations(&[stale])[0].continuity(Epoch(2), &held), + Continuity::Fork + ); + + // The lowest key wins, and only when it strictly lowers a key already held, + // so a settled epoch re-converges down and never re-forks upward. + let candidates = [[0x03u8; 32], [0x01u8; 32], [0x02u8; 32]]; + assert_eq!(fork_winner(None, &candidates), Some(1)); + assert_eq!(fork_winner(Some(&[0x05u8; 32]), &candidates), Some(1)); + assert_eq!(fork_winner(Some(&[0x00u8; 32]), &candidates), None); + assert_eq!(fork_winner(None, &[]), None); + } + + fn publish_authority( + writer: &ControlWriter, + owner: &Keys, + subkind: &'static str, + entity: [u8; 32], + content: String, + at_secs: u64, + ) -> Event { + writer + .publish( + owner, + Edition { + subkind, + entity, + content: &content, + head: None, + citation: None, + }, + at_secs, + ) + .expect("publishes") + .0 + } + + #[test] + fn a_rotation_needs_the_permission_and_must_strictly_outrank_every_target() { + let owner = Keys::generate(); + let minted = genesis(&owner, &CommunityMetadata::default(), AT).expect("mints"); + let community_id = minted.identity.community_id; + let read = + control_group_key(&minted.community_root, &community_id, ROOT_EPOCH).expect("derives"); + let signer = control_signer_group_key(&minted.control_root, &community_id, ROOT_EPOCH) + .expect("derives"); + let writer = ControlWriter { + author: owner.public_key(), + read: read.clone(), + signer: signer.clone(), + }; + + let senior = Keys::generate(); + let junior = Keys::generate(); + let target = Keys::generate(); + let mut wraps = Vec::new(); + + for (index, position, member, permissions) in [ + ( + 0u64, + 1u32, + &senior, + Permissions::MANAGE_CHANNELS | Permissions::BAN, + ), + (1, 3, &junior, Permissions::MANAGE_CHANNELS), + ] { + let role_id = RoleId::from_bytes([index as u8 + 1; 32]); + let role = Role { + role_id, + name: "role".to_owned(), + position, + permissions: Permissions(permissions), + scope: RoleScope::Server, + color: 0, + extra: Extra::default(), + }; + let grant = Grant { + member: member.public_key(), + role_ids: vec![role_id], + control_wrap: None, + extra: Extra::default(), + }; + + wraps.push(publish_authority( + &writer, + &owner, + vsk::ROLE, + *role_id.as_bytes(), + role.to_content().expect("serializes"), + AT + index, + )); + wraps.push(publish_authority( + &writer, + &owner, + vsk::GRANT, + grant_locator(&community_id, &member.public_key().to_bytes()), + grant.to_content().expect("serializes"), + AT + 10 + index, + )); + } + + let editions: Vec<_> = minted + .wraps + .iter() + .chain(wraps.iter()) + .map(|wrap| open_edition(wrap, &read, &signer.pk(), true).expect("opens")) + .collect(); + + let roles = fold_control( + &owner.public_key(), + &community_id, + &editions, + &Floors::new(), + &BTreeSet::new(), + ) + .roles; + let owner_pk = owner.public_key(); + let authorized = |rotator: &PublicKey, permission: u64, removed: &[PublicKey]| { + rekey_authorized(&roles, &owner_pk, rotator, permission, removed) + }; + let targets = [target.public_key()]; + + // The owner needs no grant. + assert!(authorized(&owner_pk, Permissions::BAN, &targets)); + + // The senior may refound; the junior holds only MANAGE_CHANNELS. + assert!(authorized(&senior.public_key(), Permissions::BAN, &targets)); + assert!(authorized( + &junior.public_key(), + Permissions::MANAGE_CHANNELS, + &targets + )); + assert!(!authorized( + &junior.public_key(), + Permissions::BAN, + &targets + )); + + // Strictly outrank: equal or above cannot rotate the other out. + assert!(!authorized( + &junior.public_key(), + Permissions::MANAGE_CHANNELS, + &[senior.public_key()] + )); + + // Holding a key is never authority. + assert!(!authorized( + &target.public_key(), + Permissions::MANAGE_CHANNELS, + &[] + )); + } + + #[test] + fn a_full_send_chunk_stays_within_a_relay_event() { + let rotator = Keys::generate(); + let community_id = community(); + let epoch = Epoch(1); + let scope = RekeyScope::Base; + let group = rekey_group(scope, &ROOT, &community_id, epoch).expect("derives"); + + let blobs: Vec = (0..MAX_REKEY_BLOBS_PER_EVENT) + .map(|_| { + let member = Keys::generate(); + + build_blob( + &rotator, + &member.public_key(), + scope, + epoch, + &[0xCD; 32], + None, + None, + ) + .expect("builds") + }) + .collect(); + + let chunks = build_rekey_chunks( + &rotator, + &group, + scope, + epoch, + Epoch(0), + &PRIOR_KEY, + &blobs, + None, + false, + AT, + ) + .expect("builds"); + + assert_eq!(chunks.len(), 1, "a full send chunk is one event"); + assert!( + chunks[0].as_json().len() <= 65_536, + "a full chunk must fit a 64 KB relay event" + ); + + let mut over = blobs; + over.push(RekeyBlob { + locator: "aa".repeat(32), + wrapped: "x".to_owned(), + }); + + let chunks = build_rekey_chunks( + &rotator, + &group, + scope, + epoch, + Epoch(0), + &PRIOR_KEY, + &over, + None, + false, + AT, + ) + .expect("builds"); + + assert_eq!(chunks.len(), 2, "one over the cap splits across two events"); + } + + #[test] + fn compaction_carries_a_settled_head_with_its_signature_intact() { + let owner = Keys::generate(); + let community_id = community(); + let prior_read = control_group_key(&[0x01; 32], &community_id, Epoch(0)).expect("derives"); + + let rumor = build_edition(EditionFields { + author: owner.public_key(), + subkind: vsk::COMMUNITY_METADATA, + entity: *community_id.as_bytes(), + version: 1, + prev: None, + citation: None, + content: "{}", + at_secs: AT, + }); + let seal = + stream::build_seal(&rumor, SealForm::Plaintext, &prior_read, &owner).expect("seals"); + + let refounding = plan_refounding(Epoch(1)).expect("plans"); + let read = refounding.read(&community_id).expect("derives"); + let signer = refounding.signer(&community_id).expect("derives"); + assert_ne!(refounding.new_root, refounding.new_control_root); + + let compacted = + compact(std::slice::from_ref(&seal), &read, &signer, AT + 1).expect("compacts"); + assert_eq!(compacted.len(), 1); + + let reopened = stream::open_wrap_at(&compacted[0], &signer.pk(), read.conversation(), true) + .expect("opens"); + assert_eq!( + reopened.seal.sig, seal.sig, + "the refounder re-signs nothing: the original author's signature rides the new wrap" + ); + assert_eq!(reopened.rumor_id, rumor.id.expect("has an id")); + assert_eq!(reopened.author, owner.public_key()); + + // Only a plaintext seal can be carried forward. + let encrypted = + stream::build_seal(&rumor, SealForm::Encrypted, &prior_read, &owner).expect("seals"); + assert!(matches!( + compact(&[encrypted], &read, &signer, AT + 1), + Err(RekeyError::Stream(StreamError::NotRewrappable)) + )); + } + + #[test] + fn a_foreign_eid_tombstone_cannot_seal_a_community() { + let owner = Keys::generate(); + let salt = [0x33u8; 32]; + let community_id = community_id_of(&owner.public_key().to_bytes(), &salt); + let identity = CommunityIdentity { + community_id, + owner: owner.public_key(), + owner_salt: salt, + }; + + let rumor = dissolved_tombstone_rumor(owner.public_key(), &community_id, AT); + let wrap = seal_dissolved(&rumor, &community_id, &owner, AT).expect("seals"); + + assert!(verify_dissolved(&wrap, &identity)); + assert_eq!( + open_dissolved(&wrap, &community_id).expect("opens").owner, + owner.public_key() + ); + + // Anyone holding the community id finds the address, but only the committed + // owner's signature counts. + let impostor = Keys::generate(); + let forged = seal_dissolved( + &dissolved_tombstone_rumor(impostor.public_key(), &community_id, AT), + &community_id, + &impostor, + AT, + ) + .expect("seals"); + assert!(!verify_dissolved(&forged, &identity)); + + // The spec's all-zero `eid` is refused: it would let one owner's genuine + // tombstone be re-wrapped at another of their communities and kill it. + let zeroed = seal_dissolved( + &stream::build_rumor_secs( + KIND_CONTROL, + owner.public_key(), + "", + vec![ + Tag::custom(TAG_SUBKIND, [vsk::DISSOLVED]), + Tag::custom(TAG_EID, ["00".repeat(32)]), + ], + AT, + ), + &community_id, + &owner, + AT, + ) + .expect("seals"); + assert!(matches!( + open_dissolved(&zeroed, &community_id), + Err(RekeyError::NotADissolution) + )); + assert!(!verify_dissolved(&zeroed, &identity)); + + // One owner, two communities: lifting X's seal to Y's public address keeps + // the signed `eid` naming X, so Y is not sealed. + let other_salt = [0x44u8; 32]; + let other_id = community_id_of(&owner.public_key().to_bytes(), &other_salt); + let other_identity = CommunityIdentity { + community_id: other_id, + owner: owner.public_key(), + owner_salt: other_salt, + }; + let seal = stream::open_wrap(&wrap, &dissolved_group_key(&community_id).expect("derives")) + .expect("opens") + .seal; + let replayed = stream::wrap_seal( + &seal, + &dissolved_group_key(&other_id).expect("derives"), + KIND_WRAP, + Timestamp::from_secs(AT + 1), + &[], + ) + .expect("rewraps") + .0; + + assert!(matches!( + open_dissolved(&replayed, &other_id), + Err(RekeyError::NotADissolution) + )); + assert!(!verify_dissolved(&replayed, &other_identity)); + } +} diff --git a/crates/concord/src/store.rs b/crates/concord/src/store.rs index 21c1bc2b..131cf315 100644 --- a/crates/concord/src/store.rs +++ b/crates/concord/src/store.rs @@ -120,6 +120,8 @@ pub struct CommunityState { pub heads: Vec, #[serde(default, skip_serializing_if = "BTreeSet::is_empty")] pub banned: BTreeSet, + #[serde(default)] + pub dissolved: bool, pub added_at_ms: u64, } @@ -186,6 +188,7 @@ impl CommunityState { relays, heads, banned: BTreeSet::new(), + dissolved: false, added_at_ms, }) } diff --git a/crates/concord/src/stream.rs b/crates/concord/src/stream.rs index 13fe9b3a..d5b4b903 100644 --- a/crates/concord/src/stream.rs +++ b/crates/concord/src/stream.rs @@ -259,21 +259,21 @@ pub fn wrap_seal_with( pub fn rewrap_seal( seal: &Event, - new_group: &GroupKey, + read: &GroupKey, + signer: &GroupKey, at: Timestamp, ) -> Result<(Event, Keys), StreamError> { if seal.kind.as_u16() != KIND_SEAL_PLAINTEXT { return Err(StreamError::NotRewrappable); } - wrap_seal(seal, new_group, KIND_WRAP, at, &[]) + + wrap_seal_with(seal, read.conversation(), signer.keys(), KIND_WRAP, at, &[]) } pub fn open_wrap(wrap: &Event, group: &GroupKey) -> Result { open_wrap_at(wrap, &group.pk(), group.conversation(), false) } -/// Open and verify a wrap against a stream read view: the address to check and -/// the conversation key that opens the wraps, with no signing secret required. pub fn open_wrap_at( wrap: &Event, address: &PublicKey, @@ -498,7 +498,8 @@ mod tests { assert_eq!(opened.seal_form, SealForm::Plaintext); let (rewrapped, _) = - rewrap_seal(&opened.seal, &group(1), Timestamp::from_secs(2)).expect("rewraps"); + rewrap_seal(&opened.seal, &group(1), &group(1), Timestamp::from_secs(2)) + .expect("rewraps"); let reopened = open_wrap(&rewrapped, &group(1)).expect("opens"); assert_eq!(reopened.rumor_id, opened.rumor_id, "the rumor id survives"); assert_eq!(reopened.author, author.public_key()); @@ -512,6 +513,7 @@ mod tests { rewrap_seal( &sealed(&edition, SealForm::Encrypted, &author), &group(1), + &group(1), Timestamp::from_secs(2) ), Err(StreamError::NotRewrappable) -- 2.54.0 From fd39be0edafa6691fb061ece44ae5a424f6b7552 Mon Sep 17 00:00:00 2001 From: Ren Amamiya Date: Thu, 17 Sep 2026 10:35:41 +0700 Subject: [PATCH 10/12] add pins, disappearing messages and hardenin --- Cargo.lock | 2 + Cargo.toml | 3 + PLAN.md | 151 ++++++- crates/concord/Cargo.toml | 2 + crates/concord/src/chat.rs | 275 +++++++++++- crates/concord/src/control.rs | 301 ++++++++++++- crates/concord/src/lib.rs | 9 +- crates/concord/src/pins.rs | 825 ++++++++++++++++++++++++++++++++++ crates/concord/src/rekey.rs | 42 +- crates/concord/src/roles.rs | 6 +- crates/concord/src/store.rs | 150 ++++++- crates/concord/src/stream.rs | 1 + 12 files changed, 1677 insertions(+), 90 deletions(-) create mode 100644 crates/concord/src/pins.rs diff --git a/Cargo.lock b/Cargo.lock index de364b01..7dce1c28 100644 --- a/Cargo.lock +++ b/Cargo.lock @@ -1307,8 +1307,10 @@ name = "concord" version = "1.0.2" dependencies = [ "anyhow", + "chacha20 0.9.1", "data-encoding", "hkdf", + "hmac 0.12.1", "nostr", "nostr-memory", "nostr-sdk", diff --git a/Cargo.toml b/Cargo.toml index 39b40fd4..391680ca 100644 --- a/Cargo.toml +++ b/Cargo.toml @@ -32,6 +32,9 @@ aes-gcm = "0.10" sha2 = "0.10" data-encoding = "2" hkdf = "0.12" +# Pinned to the instances `nostr` already builds: the NIP-44 message-key disclosure +chacha20 = "0.9" +hmac = "0.12" # Pinned to the instance `nostr-sdk` already builds, so NIP-44 nonces share it rand = { version = "0.10", default-features = false, features = [ "std", "sys_rng" ] } diff --git a/PLAN.md b/PLAN.md index 668d36b9..7f882c02 100644 --- a/PLAN.md +++ b/PLAN.md @@ -16,7 +16,7 @@ Scope of this plan: **backend + public Rust API**. No views, no widgets, no copy - Any UI work. - CORD-07 audio/video. Reserve `23313`, the `concord/voice-*` labels and the `voice` metadata flag so nothing else claims them, and implement nothing. -- Pins (CORD-04 §7) ship in the last milestone; the design accounts for `vsk 11` early so the fold is not retrofitted. +- Pins (CORD-04 §7) and disappearing messages (CORD-08) land in M8, the last milestone; the design accounted for `vsk 11` and `PIN_MESSAGES` from M3 so neither fold is retrofitted. - Cross-client interop testing (Vector/Armada/Grimoire). Tracked as follow-up work, not blocking. ## 2. Sources of truth @@ -59,7 +59,7 @@ Verified against the pinned revision (`nostr` 0.45.4 / `nostr-sdk` 0.45.2, git ` **Not needed.** `secp256k1` (use `nostr::SecretKey::from_slice` + `Keys::new`), `base64` (use `data_encoding::BASE64`, already a workspace dep), `bech32` (NIP-19 is in the SDK), any new storage engine (the client's LMDB database is enough), any new HTTP client. -**Dependencies added so far:** `hkdf = "0.12"` at M0 (already in `Cargo.lock` transitively) and `rand = "0.10"` at M1 for the NIP-44 nonce, pinned to the exact instance `nostr` already builds (`default-features = false`, features `std` + `sys_rng`) so `nostr`'s `os-rng` and ours unify on one `rand`/`getrandom`. `sha2` and `data-encoding` were already workspace deps. **Two more at M8:** the Pin List's per-message key disclosure needs `chacha20 = "0.9"` (already in the tree because we enable nostr's `nip44`, which is where `chacha20` comes from) and `hmac = "0.12"` (already in the tree via `hkdf`) — see §14.8. Zero new crates so far, and all of these are direct-dependency lines only. +**Dependencies added so far:** `hkdf = "0.12"` at M0 and `rand = "0.10"` at M1 for the NIP-44 nonce, pinned to the exact instance `nostr` already builds (`default-features = false`, features `std` + `sys_rng`) so `nostr`'s `os-rng` and ours unify on one `rand`/`getrandom`; `sha2` and `data-encoding` were already workspace deps. M8 added the Pin List's two, read off the crate graph rather than chosen: `chacha20 = "0.9"` (the instance nostr's `nip44` builds) and `hmac = "0.12"` (the instance `hkdf` builds) — see §14.8. **No new package has ever been added**, and M8's pair changed `Cargo.lock` only by the two `concord` edges; every addition is a direct-dependency line on something already compiled. **Why not depend on Vector's crates.** `vector-core` (published, MIT) holds the only other Rust Concord implementation, in `src/community/v2/*`. It is not reusable as a dependency, and the "reuse their crypto" argument does not hold: @@ -87,6 +87,7 @@ crates/concord/ src/chat.rs CORD-03: channel plane — message/edit/delete/reaction builders + message view src/invite.rs CORD-05 §1–§3, §6: bundle, link (naddr + fragment), Direct Invite src/list.rs CORD-02 §8: the Community List — join material, merge, to-self envelope + src/pins.rs CORD-04 §7: the Pin List, the NIP-44 key disclosure, the two content forms src/rekey.rs CORD-06: blob codec, continuity, refounding, compaction, dissolution src/store.rs local persistence + opened-rumor cache + history queries ``` @@ -360,7 +361,7 @@ One reference limitation we **reproduce and do not fix** (recorded here rather t ### 8.2 Communities, channels, metadata -`CommunityMetadata` carries `name` (≤ 64 bytes), `description` (≤ 10 000 bytes), `relays` (truncated on read and write to 5), `icon` and `banner` as encrypted-blob pointers (`{url, key, nonce, hash}`), and the optional `custom` object. `ChannelMetadata` carries `name`, `private`, optional `voice`, `deleted`, optional `custom`. Every content struct carries `#[serde(flatten)] extra`, so a field one client does not model still round-trips and a rename by an older client cannot wipe another client's `custom` keys. +`CommunityMetadata` carries `name` (≤ 64 bytes), `description` (≤ 10 000 bytes), `relays` (truncated on read and write to 5), `icon` and `banner` as encrypted-blob pointers (`{url, key, nonce, hash}`), the optional `custom` object, and `message_expiration` (CORD-08's timer, in seconds, §8.8). `ChannelMetadata` carries `name`, `private`, optional `voice`, `deleted`, optional `custom`. Every content struct carries `#[serde(flatten)] extra`, so a field one client does not model still round-trips and a rename by an older client cannot wipe another client's `custom` keys. The Control Plane's whole projection is one call: @@ -370,11 +371,15 @@ pub struct ControlFold { pub banned: BTreeSet, pub community: Option, pub channels: BTreeMap, + pub registries: BTreeMap>, // vsk 8, keyed by creator + pub pins: BTreeMap<[u8; 32], String>, // vsk 11 content, keyed by locator (§8.8) pub floors: Floors, pub gapped: bool, } pub fn fold_control(owner: &PublicKey, community_id: &CommunityId, editions: &[ParsedEdition], floors: &Floors, held_bans: &BTreeSet) -> ControlFold; +impl ControlFold { pub fn is_public(&self) -> bool; + pub fn pin_content(&self, community_id, channel) -> Option<&str>; } ``` - The roster is folded first, and `vsk 0` / `vsk 2` are then judged against it: the head of each entity is the highest edition whose author *currently* holds `MANAGE_METADATA` / `MANAGE_CHANNELS`, is not banned, and either is the owner or cites their own folded Grant. Pre-filtering before the chain fold is what stops a demoted admin's later, higher-version edition from being the head. @@ -397,6 +402,10 @@ impl ControlWriter { pub fn set_grant(&self, keys, community_id, grant: &Grant, head, citation, at_secs) -> Result<(Event, EntityHead)>; pub fn set_banlist(&self, keys, community_id, banned: &BTreeSet, head, citation, at_secs) -> Result<(Event, EntityHead)>; + pub fn set_registry(&self, keys, community_id, creator, links: &[PublicKey], head, citation, at_secs) + -> Result<(Event, EntityHead)>; + pub fn set_pin_list(&self, keys, community_id, channel, content: &str, head, citation, at_secs) + -> Result<(Event, EntityHead)>; } ``` @@ -479,7 +488,8 @@ pub fn complete_memberlist(coalesced: &BTreeMap, ### 8.4 Chat plane (`chat.rs`) — implemented in M4 Kinds (CORD-02 Appendix B): `9` message, `1111` NIP-22 comment, `7` NIP-25 reaction, -`5` NIP-09 delete, `3302` edit, `3310` WebXDC peer signal, `23311` ephemeral typing. +`5` NIP-09 delete, `3302` edit, `1740` timer notice (M8), `3310` WebXDC peer signal, +`23311` ephemeral typing. ```rust pub struct ChatRumor { id, author, kind, channel, epoch, at_ms, content, @@ -490,17 +500,19 @@ pub enum ChatAction { Edit { target: EventId, content: String }, Delete { target: EventId, target_kind: Option, citation: Option }, Typing, + TimerNotice { seconds: u64 }, Opaque, } pub struct ReplyRef { id: EventId, author: Option } pub struct Target { reply: ReplyRef, kind: u16 } // the wire commits the target's kind -pub fn build_message(author, channel, epoch, content, quote: Option<&ReplyRef>, at_ms) -> UnsignedEvent; -pub fn build_comment(author, channel, epoch, content, parent: &Target, root: Option<&Target>, at_ms) -> UnsignedEvent; -pub fn build_reaction(author, channel, epoch, target: &Target, emoji: &str, at_ms) -> UnsignedEvent; -pub fn build_edit(author, channel, epoch, target: EventId, content: &str, at_ms) -> UnsignedEvent; +pub fn build_message(author, channel, epoch, content, quote: Option<&ReplyRef>, at_ms, timer: Option) -> UnsignedEvent; +pub fn build_comment(author, channel, epoch, content, parent: &Target, root: Option<&Target>, at_ms, timer) -> UnsignedEvent; +pub fn build_reaction(author, channel, epoch, target: &Target, emoji: &str, at_ms, timer) -> UnsignedEvent; +pub fn build_edit(author, channel, epoch, target: EventId, content: &str, at_ms, timer) -> UnsignedEvent; pub fn build_delete(author, channel, epoch, target: EventId, target_kind: Option, citation: Option<&AuthorityCitation>, at_ms) -> UnsignedEvent; +pub fn build_timer_notice(author, channel, epoch, seconds: u64, at_ms) -> UnsignedEvent; pub fn build_typing(author, channel, epoch, at_ms) -> UnsignedEvent; pub fn seal_rumor(rumor: &UnsignedEvent, group: &GroupKey, author: &Keys, ephemeral: bool) @@ -508,7 +520,9 @@ pub fn seal_rumor(rumor: &UnsignedEvent, group: &GroupKey, author: &Keys, epheme pub fn open(wrap: &Event, group: &GroupKey, channel: &ChannelId, epoch: Epoch) -> Result<(OpenedStream, ChatRumor), ChatError>; pub fn plane_keys(held: &[(Epoch, [u8; 32])], channel: &ChannelId) -> Result>; -pub fn fold(rumors: &[ChatRumor], +pub fn expiration_of(rumor: &UnsignedEvent) -> Result, ChatError>; +pub fn expired(rumor: &ChatRumor, now: Timestamp) -> bool; +pub fn fold(rumors: &[ChatRumor], now: Timestamp, can_delete: impl Fn(&PublicKey, Option<&AuthorityCitation>, &PublicKey) -> bool) -> Vec; @@ -550,7 +564,9 @@ pub struct ChatMessage { - `ms` orders a page but cannot page within one: a relay's `until` filter is second-granular, so the cursor step below is what has to cope with a boundary second. - `seal_rumor` gates the kind at publish and mirrors a NIP-40 `expiration` onto the wrap - (CORD-08 §2). The timer's policy — ingest refusal, the sweep, kind 1740 — is M8. + (CORD-08 §2), so a NIP-40 relay drops the ciphertext itself. Since M8 the durable builders + *attach* the tag from the folded timer, `fold` drops an expired rumor before displaying it, and + a delete or a timer notice carrying the tag is refused outright — see §8.8. - **`media` and `mentions` are deliberately absent.** Both are pure post-processing of `content` by `common` (`extract_and_remove_media_urls`, `NostrParser`) and both return a gpui type, and a protocol crate does not take a UI dependency for a derived field. They @@ -819,6 +835,96 @@ tombstone for one community be re-wrapped at another of theirs and kill it perma community is sealed read-only: `CommunityState.dissolved` records it, subscriptions halt, nothing new is honored, existing history stays readable, and a member's delete of their own message is still honored. +### 8.8 Pins and disappearing messages (`pins.rs`, `chat.rs`, `store.rs`) — implemented in M8 + +**Pins (CORD-04 §7).** One Pin List per Channel: `vsk 11` at `pins_locator(community_id, channel)`, +derived from the `community_id`, so it survives every refounding and a fresh joiner derives the same +coordinate. A pin does not quote a message, it *proves* one — the entry carries the original kind-20013 +seal verbatim plus that message's 76-byte NIP-44 key disclosure, so a reader holding no history and no +old keys still verifies author, words, Channel and signed time. + +```rust +pub const PIN_MAX_ENTRIES: usize = 25; +pub const PIN_MAX_CONTENT_BYTES: usize = 32_768; +pub const MESSAGE_KEYS_BYTES: usize = 76; + +pub struct MessageKeys { /* chacha_key[32] ‖ chacha_nonce[12] ‖ hmac_key[32] */ } +pub struct PinEditBundle { seal: Event, keys: String } +pub struct PinEntry { seal: Event, keys: String, wrap: Option, edit: Option, extra } +pub struct VerifiedPin { rumor_id, author, kind, content, tags, epoch, at_ms, created_at, wrap, edited, entry } +pub struct ReadPinList { entries: Vec, sealed: bool } +pub enum PinError { NotEncryptedSeal, BadPayload, Unverifiable, Unreadable, TooManyEntries, Oversize, Seal, Encode } + +pub fn build_entry(opened: &OpenedStream, group: &GroupKey, channel: &ChannelId) -> Result; +pub fn build_edit_bundle(edit: &OpenedStream, group, original: &VerifiedPin, channel) -> Result; +pub fn with_proven_edit(entry: &PinEntry, edit: &OpenedStream, group, channel) -> PinEntry; +pub fn verify_entry(entry: &PinEntry, channel: &ChannelId) -> Option; +pub fn read_list(content: &str, unseal: impl Fn(Epoch) -> Option) -> ReadPinList; +pub fn publishable(read: &ReadPinList, private: bool, group: &GroupKey, epoch: Epoch) -> Result; +pub fn killed_by(pin: &VerifiedPin, delete: &ChatRumor) -> bool; +``` + +- **The disclosure is a reproduction, not a re-use.** `nostr`'s `nip44::v2::get_message_keys` is a private +`fn` and both public entry points take the whole conversation key, so the expansion is rebuilt from the +same audited primitives: `hkdf::Hkdf::from_prk(conversation_key).expand(nonce, 76)`, split, then +HMAC-SHA256 over `nonce ‖ ciphertext` — compared in constant time, so the verify path is never a MAC +oracle — ChaCha20, and NIP-44's padding check. `MessageKeys::to_hex`/`from_hex` are the wire form +(lowercase-canonical), and one test round-trips the whole thing against nostr's own +`encrypt_to_bytes_with_nonce`, which is what keeps the copy honest. +- **The whole verification**, in order: the seal is kind 20013 and verifies → the entry's disclosure +decodes → MAC → decrypt → unpad → `UnsignedEvent::from_json` → the rumor's `pubkey` equals the seal's +(NIP-59's impersonation check) → the kind is `9` or `1111` → `channel` strict-equal to this list's +Channel → a canonical `epoch` → `verify_id()` and a *recomputed* identity. An Edit bundle failing any of +its own steps is dropped alone; the pin survives it. +- **Two deliberate divergences from Vector**, both stricter: the `epoch` tag is required (CORD-03 §3 makes +it mandatory and `chat::open` already enforces it) where Vector ignores it, and an out-of-range `ms` is +malformed rather than read as `0`, matching this crate's own reader. A duplicated `channel` tag resolves +to the first, as Vector does, because the rumor is author-signed and the lenient reading is not +exploitable. +- **Two self-describing content forms**, never signalled by the fold. Public is `{ "entries": [...] }`; +private is `{ "epoch": "", "sealed": seal_bytes(...) }`, whose sealed plaintext is that same +`{ "entries": [...] }`. A reader accepts either form regardless of its metadata fold; a writer must use +the form matching the Channel's folded type. +- **The caps are content-level, never chain-level.** `read_list` yields an EMPTY list for oversize +content, an over-cap count, a malformed envelope or a failed decrypt, and `sealed: true` — darkness, not +violation — when the named epoch's key is missing. The edition itself always folds, so every client walks +the same chain. +- **`publishable` is the only write path**, and it refuses a dark list outright: an unreadable entry and +an absent one are indistinguishable, so re-forming would silently drop every entry the writer cannot see. +That is the write half of §12's "never publish from a list you could not read". +- The Edit bundle is the same five steps with kind `3302` substituted plus the fold's own two rules — the +proven author equals the original's, and the `e` tag names the original's recomputed rumor id — so a +keyless reader reaches the verdict a keyed reader reaches by folding. At most one, ever: a later Edit +*replaces* it, since edits target the original and never each other. +- Deferred to §10: §7's "a private→public conversion MUST NOT mechanically re-form the list" has no +encoding — "mechanically" and "deliberately" are the same bytes — so it is a duty of the conversion flow +(§14.13); the re-heal and deletion-omission writes (`killed_by` exists, the re-fold-and-republish loop +does not); and the automatic Edit refresh. + +**Disappearing messages (CORD-08).** + +- `CommunityMetadata.message_expiration: Option` is the timer in seconds, with a lenient +deserializer: absent, `0`, a string, a float or any other garbage all read as `None`, and garbage never +poisons the rest of the entity. The write side normalizes `Some(0)` away. +- The timer is community state, so the tag rides the **signed rumor**: `build_message`, `build_comment`, +`build_reaction` and `build_edit` take `timer: Option` and attach +`["expiration", created_at + timer]` from their own `at_ms`, which is what makes a change +non-retroactive. `seal_rumor` had mirrored the tag onto the wrap since M4, so a NIP-40 relay deletes the +ciphertext itself. +- **Two kinds are exempt**, and a rumor carrying the tag is refused with `ChatError::ExemptExpiration`: +a delete (its target may outlive it) and the timer notice (the policy must not be erased by the policy). +- **The timer notice** is kind `1740`: `build_timer_notice` writes it, `ChatAction::TimerNotice { seconds }` +reads it with a canonical `timer` tag, and the fold emits it as a row of its own, like any message. +Whether its author may be believed about policy is `MANAGE_METADATA` in the roster — the registry's call, +not the fold's, and `roles.is_authorized(&author, &owner, Permissions::MANAGE_METADATA)` already answers +it. +- **Enforcement lives in two places, both keyed on the rumor's own signed tag.** `chat::fold(rumors, now, +can_delete)` drops an expired rumor, so it is never displayed whatever the ingest path. `store::cache_rumor` +refuses to store one that has already expired (returning whether it kept it) and `backfill` both skips the +cache and drops it from the page. `store::purge_expired` is the sweep: it re-reads each cached row's +rumor, collects the expired ids and physically deletes them, because hiding is not disappearing and the +local store is the artifact a seized device surrenders. A malformed tag expires nothing. + ## 9. Storage (`store.rs`) — local layer implemented in M1, state document in M2, fold bridge in M3, banlist in M5 Three layers, no new storage engine: @@ -831,13 +937,18 @@ Three layers, no new storage engine: - The layer takes `&dyn NostrDatabase`, not `&Client`: it is local-only, which keeps it testable without a relay or a GPUI context. ```rust -pub async fn cache_rumor(database: &dyn NostrDatabase, channel: &ChannelId, opened: &OpenedStream) -> Result<()>; +pub async fn cache_rumor(database: &dyn NostrDatabase, channel: &ChannelId, opened: &OpenedStream) -> Result; pub async fn query_rumors(database: &dyn NostrDatabase, channel: &ChannelId, until: Option, limit: usize) -> Result>; +pub async fn purge_expired(database: &dyn NostrDatabase, channel: &ChannelId, now: Timestamp) -> Result; pub async fn backfill(client: &Client, database: &dyn NostrDatabase, channel: &ChannelId, held: &[(Epoch, [u8; 32])], until: Option, limit: usize) -> Result>; ``` +`cache_rumor` returned `()` until M8, when it gained the CORD-08 §3 ingest rule and with it a reason to +report: `false` means an already-expired rumor was refused and nothing was written. `cache_rumor` and +`purge_expired` are the two halves of the timer's storage policy; §8.8 has the rest. + `query_rumors` returns `UnsignedEvent`, not `Event`: the cached payload *is* a rumor, which is also what `OpenedStream` carries, so the caller never has to re-parse. **Landed in M4:** `backfill` — newest-first relay paging across every held epoch. It derives @@ -1030,14 +1141,16 @@ Each of these has burned a real implementation, or is a documented cross-client - Drop guestbook entries more than an hour in the future; treat an out-of-range `ms` as malformed, not as an interpretation opportunity. - Never honour a Snapshot from anyone but the refounder of that epoch. - Refuse to write a Pin List from a list the writer could not read. -- Enforce the NIP-44 65 535-byte cap at every layer before publishing, and the 5-relay / 256-channel / 50-membership / 100-roles / 64-roles-per-member / 500-banlist / 25-pin caps at their ingest and write points. +- Enforce the NIP-44 65 535-byte cap at every layer before publishing, and the 5-relay / 50-membership / 100-role / 64-role-per-member / 500-banlist / 25-pin caps at their ingest and write points. - Lowercase hex only; x-only pubkeys only; no version tag anywhere. - **Enforced in M3:** a Role's `role_id` is its own coordinate and never 0; a Grant's `member` hashes to its coordinate; a `vsk 4` sits at this community's banlist locator; a banned npub's editions are dropped and a grant naming them carries no rank; a revocation carries a citation; the 100-role cap keeps the lowest ids *after* authorization; a below-floor edition is never a candidate. - **Enforced in M4:** the chat plane's encrypted-seal requirement, at both publish and open; `channel` **and** `epoch` strict-equal to the plane whose key opened the wrap; a retired or unregistered rumor kind rejected on both sides; a target bearing tag that appears twice rejected outright; and a delete from anybody other than the message's own author refused by the caller's gate. - **Enforced in M5:** a Kick counts only where the roster admits its actor under `KICK` with a strictly higher rank and a resolvable citation; a Snapshot counts only from the epoch's refounder, with no owner fallback; a guestbook entry more than an hour ahead is dropped, and an out-of-range `ms` or a non-verb `3306` entry is malformed, not interpreted; a duplicated `vac` is rejected outright; and a delete is honored from its target's author unconditionally, or from another actor only where `can_delete` admits them. - **Enforced in M6:** an invite fragment whose version is not this one is refused in either direction, as is one with a bad count or trailing bytes, and its encoding caps bootstrap relays at three; a bundle past the channel cap, past the epoch ceiling, or carrying a secret that is not 32 bytes of hex is refused before it is used, and one whose `owner` + `owner_salt` does not reproduce its `community_id` is refused outright; a bundle event off its coordinate, off its author, or unsigned is refused, and a tombstone at the coordinate reads as revoked; a Direct Invite's wrap, its seal signature and its rumor/seal author bind are all verified before the bundle is even parsed; and a Community List refuses to build past its membership cap or the NIP-44 plaintext cap. - **Enforced in M7:** a blob's bound scope and epoch are checked inside the ciphertext, so a channel blob cannot be opened under the base scope or under another epoch, and a 136-byte base blob whose secret does not derive to the pk beside it is refused whole rather than adopting a split control plane; the locator is never gated, because it derives from public keys and proves nothing; a removal is never concluded from a partial chunk set, and two chunks claiming one index union their blobs rather than letting the loser's recipients read as removed; a rotation needs its permission and must strictly outrank every target, so a rotator holding the prior root or a demoted staffer holding the `control_root` is dropped; a plaintext-sealed rekey is refused; and a tombstone is refused unless its signed `eid` is this community's own id — the all-zero placeholder and a sibling community of the same owner included. -- Still owed to M8's audit: the byte caps, the 256-channel and 25-pin caps, and the write-side counterparts of the caps the folds already apply. +- **Enforced in M8:** a plaintext seal carries no payload to disclose, so it cannot be pinned; a pin whose disclosure does not open its own seal is refused before it costs list budget, and a built entry is run through the same verification every reader applies; a `channel` or `epoch` bound to another Channel, a claimed rumor id that is not its own, another author's Edit, and a delete from anybody but the pin's own author are each refused; a list past 25 entries or 32 768 content bytes is refused on the write side and reads as EMPTY on every reader's; a sealed list that cannot be opened reads as darkness and never as an empty public one, and a writer can never re-form a list it could not read; a delete and a timer notice may not carry an `expiration` and a malformed `timer` is refused; an already-expired rumor is refused at ingest, dropped from every fold, and purged by the sweep. +- **The byte caps, closed out:** the NIP-44 plaintext cap is now checked inside `seal_bytes` as well as `seal_content`, so every raw envelope — the invite bundle, the sealed pin form, the List's to-self document — is inside it before it is published. 25 pins / 32 768 bytes; 100 roles and 64 roles per member, the member's cap also refusing the write; 500 banlist entries refused on the write side as well as capped in the fold; the 64-byte name cap refused on every community, channel and role write; 5 relays truncated on read as well as on write; 50 memberships in `list::fits`. The 256 in the old wording was the *invite bundle's* channel cap (M6), not a community-wide one — the spec states no community channel count, so none was invented. +- **Still owed:** the `vac`-carrying pin write and the 100-role write gate both need the roster, so both are the registry's (§10), as is the private→public re-form refusal (§14.13). ## 13. Milestones @@ -1051,7 +1164,7 @@ Each of these has burned a real implementation, or is a documented cross-client | M5 | Guestbook + member list + moderation | ✅ `cargo test -p concord` (23 tests): a second holder folds joins, leaves, a cited kick and a chunked snapshot into one memberlist in either arrival order, with a ban and a Grant deciding the edges; an uncited, unranked or owner-directed kick and a foreign snapshot are dropped; a future-dated entry, a malformed `ms`, a non-verb `3306`, a duplicated `vac` and a bad snapshot chunk are each refused; and a moderator delete lands only under a citation the roster admits, while a self-delete never asks | | M6 | Invites + Community List | ✅ `cargo test -p concord` (32 tests): the fragment's byte layout is pinned by golden base64url for the stock set, a dictionary mix and a verbatim literal, with a wrong version in either direction, trailing bytes and an over-cap count each fatal; a link round-trips as a full URL and as a bare naddr, and refuses a non-invite; a bundle round-trips while a revocation tombstone reads as revoked, and a wrong token, a squatter's author, a foreign `d`, a forged owner, a malformed secret and an over-cap channel count are each refused; a Direct Invite round-trips to its verified inviter and refuses a stranger's keys and a non-invite rumor; and the Community List keeps the earlier seed and the later current in either merge order, refuses to resurrect a tombstoned id until a newer join outruns it, and rebuilds on a second device with unknown fields intact while refusing an over-cap or oversized list | | M7 | Rekeys + refounding + dissolution | ✅ `cargo test -p concord` (40 tests): a blob's bound scope and epoch are checked inside the ciphertext, so a channel blob cannot be opened under the base scope or under another epoch, and a staff secret that does not derive to the pk beside it refuses the whole blob; a removal is concluded only from a complete chunk set, and two chunks claiming one index union rather than drop a recipient's blob; continuity extends, gaps and forks, and the fork winner is the lowest key adopted only when it strictly lowers one already held; a rotation needs its permission and must strictly outrank every target, so holding a key is never authority; a full 80-blob chunk fits a 64 KB relay event and one more splits; compaction carries a settled head across a refounding with the original author's signature intact; and an owner's tombstone seals the community while an impostor's, the spec's all-zero `eid` and one re-wrapped from another community of the same owner are each refused | -| M8 | Pins + disappearing messages + hardening | pins verify from a keyless reader's view; expiry is refused at ingest and purged by the sweep; the audit of §12 is complete with a test per bullet | +| M8 | Pins + disappearing messages + hardening | ✅ `cargo test -p concord` (49 tests): a disclosed 76-byte expansion opens exactly the message it was derived from and nothing else, pinned by a round-trip against nostr's own encrypt; a built entry proves its author and its words while tampered keys, a re-signed seal, a claimed id that is not its own and a Channel it does not belong to are all refused; a proven Edit replaces the words and a stranger's never attaches; both list forms round-trip with a sealed one dark without its key and lit with it, 26 entries refusing to build and reading back as empty, and a writer never re-forms a list it could not read; a Pin List folds under a derived coordinate for a second client while a neighbouring Channel reads none; a timer tag rides a durable rumor and its wrap while a delete and a notice carrying it are refused; the metadata timer is set, off and garbage without poisoning the entity; an expired rumor is refused at ingest and purged by the sweep while an untimed one never is; and the banlist, grant, channel-name and relay caps hold on the way out | Ordering is deliberately dependency-first: each milestone is usable on its own, and nothing in M2+ depends on a later milestone. The sync engine and its GPUI wiring (§10, §11) have no row of their own because they are cross-cutting: every plane they consume has to exist first, so they follow M8, and they are the milestone that applies §11's `chat::handle_notifications` routing fix. @@ -1077,8 +1190,12 @@ What M6 still defers, and to what: the **Invite List (13303) and the Registry (` M7 closed at 40 tests, again with no dependency change and `Cargo.lock` untouched. New: `src/rekey.rs` (the blob atom, the 3303 chunk set and its collection, continuity and the fork winner, the authority gate, refounding planning and compaction, and dissolution). The Invite List landed in `src/invite.rs` beside the bundle it bookkeeps, and the Registry became a Control Plane entity: `ControlFold.registries` keyed by creator, folded under `CREATE_INVITE`, its aggregate exposed as `ControlFold::is_public`, with `ControlWriter::set_registry` as the write side. `invite_links_locator` and M6's revocation machinery needed no change. In `stream.rs`, `rewrap_seal` gained a separate `signer` group, because a split epoch reads under the rolled root but wraps as the new control signer. In `store.rs`, `CommunityState.dissolved` records the seal; `list.rs`'s `canonical`/`union` became `pub(crate)` so the Invite List's merge shares them rather than restating the same total order. +M8 closed at 49 tests. New: `src/pins.rs` (the disclosure primitive, the entry codec and its full verification, the Edit bundle, both content forms and their caps) and, in `control.rs`, `ControlFold.pins` keyed by locator with `pin_content`/`set_pin_list`, plus the metadata timer and the write-side caps. `chat.rs` gained kind 1740, `ChatAction::TimerNotice`, the `timer` argument on the four durable builders, the exemption refusal, `expired`, and a `now` parameter on `fold`; `store.rs` gained the ingest refusal and `purge_expired`; `stream.rs` gained the plaintext-cap check inside `seal_bytes`; `roles.rs` gained the 64-role write gate; and `lib.rs`'s `decode_hex_32` was generalized into `decode_hex_lower::` so the 76-byte disclosure shares the one canonical-hex check rather than restating it. This is the first milestone that touched `Cargo.lock`: two direct-dependency edges (`chacha20`, `hmac`), no new package (§3). Evidence for one line of §12 that was wrong: the 256-channel cap does not exist in the spec — the 256 was `invite::MAX_BUNDLE_CHANNELS` all along. + What M7 still defers, and to what: **epoch key retention** — the blob atom delivers every plane key a refounding mints, but nothing can persist one, because `ChannelKeyRef` is `{id, name, private, epoch}` with no key field (§14.11); the **rekey subscription**, precomputed from the next epoch's address for every held private channel plus the base, which is a `Filter` and belongs with the sync engine (§10); the **Server-Severing flow** that would set `severed`, which awaits the invite-link wiring; and the **`Refound` seed** to `complete_memberlist`, which M7 can now mint but whose consumer is still the guestbook ingest of §10. +What M8 still defers, and to what: the **`vac`-carrying pin write, the 100-role write gate and the timer notice's roster gate** — all three need the roster, so they are the registry's (§10); the **private→public re-form refusal**, which cannot be encoded at the list level (§14.13); the **re-heal and deletion-omission pin writes** and the automatic Edit refresh, which are pins §7 behaviours a curator flow drives; and everything §10 already owed — the sync engine, the `chat::handle_notifications` routing fix, and the §14.11 key-retention schema change. + **M2's "created and published" is verified offline**: "published" is the two wraps existing and being openable by the invite keys, not a relay round-trip. There is no registry to publish through until §10, and a relay test would be testing the SDK, not the protocol. ## 14. Open questions and risks @@ -1090,14 +1207,16 @@ What M7 still defers, and to what: **epoch key retention** — the blob atom del 5. **Relay set.** Up to 5 recommended, and both reads and writes fan out across them. Coop's client is a gossip client with `no_background_refresh`, so community relays must be added explicitly and re-added on metadata change. 6. **Local plaintext state.** §9 records the decision. Revisit only if the local database stops being treated as trusted. 7. **Was a `community_id` ever hashed into a tag?** No — it must never appear on the wire. A lint-style test should assert it appears in no published event other than inside an invite bundle and a direct invite. -8. **The Pin List's message-key disclosure has no public API (M8).** CORD-04 §7 Pins let a keyless reader verify a disclosure, which means revealing one message's NIP-44 keys rather than the plane's conversation key. `nostr`'s `nip44::v2::get_message_keys(conversation_key, nonce)` is a private `fn`, and both public entry points (`encrypt_to_bytes_with_nonce`, `decrypt_to_bytes`) take the whole conversation key — so the expansion has to be reproduced as `hkdf::expand_into(conversation_key, nonce, 76 bytes)` plus ChaCha20 and an HMAC-SHA256, exactly as Vector does, and round-tripped against nostr's own `encrypt` in a test. Read CORD-04 §7 in full at M8 before writing it: the reproduction is only worth it once the exact verification the pin must support is settled, and the alternative is contributing a `pub` message-key accessor upstream (we already track git master, so a patch branch or an upstream PR is viable and strictly better than a reproduction we must keep in sync). +8. **The Pin List's message-key disclosure had no public API — resolved in M8.** `nostr`'s `nip44::v2::get_message_keys` is a private `fn`, so the expansion is reproduced in `pins.rs` over the same audited primitives — `hkdf`'s `from_prk`/`expand`, `hmac`'s constant-time `verify_slice`, `chacha20` — as two direct-dependency lines on packages already in the graph (§3), pinned by a round-trip against nostr's own `encrypt_to_bytes_with_nonce`. That test is the whole contract: if the reproduction ever drifts, it fails loudly rather than silently unverifying every pin. A `pub` accessor upstream remains strictly better than a copy we must keep in sync and is still a viable PR, but it is now an improvement rather than a blocker. 9. **A remote signer is not plumbed.** `ControlWriter::publish`, `stream`'s seal builders and the invite builders all take `&Keys`. NIP-46 is a stated Vector feature (§10's task slots are already cleared on signer change), but making the writers async over a `NostrSigner` is a change to every builder, so it should be one deliberate pass rather than a patch per milestone. Nothing in M4–M8 depends on it except the UX of using a remote signer at all. 10. **The fold is not incremental.** `fold_control` re-parses and re-folds the whole control edition window on every call, and each fold is up to `2 × entities + 8` passes. That is fine at the caps the spec sets (100 roles, 400-odd grants) and it is the simplest thing that is correct, but if the sync engine ends up calling it per event rather than per batch, the candidate maps and their parse belong in a cache keyed by edition id. Measure before optimizing. 11. **No plane key can be persisted (found in M7).** The rekey blob atom hands a receiver every key a rotation mints — the next `community_root`, the `control_root`, and a private channel's fresh key — and `CommunityState` has nowhere to put any of them: `ChannelKeyRef` is `{id, name, private, epoch}`, and the state's only root fields are the *current* `community_root`/`root_epoch`. So a client can verify a rotation and still lose it on restart, and it cannot read history written under a prior root or a prior channel epoch. M7 therefore left `epoch_keys` out rather than add a field with no key to hold. The fix is a schema change — `ChannelKeyRef` gains the key and its retired `priors`, and the state gains a root-per-epoch map — and it belongs with the sync engine that reads them back, since it changes `apply_fold` and the `13302` join material together. Armada already carries `priors` for exactly this reason. +12. **`message_expiration` normalization is lossy, deliberately.** A `0`, a float or a string all fold to `None`, and a republish of that fold writes the field away rather than carrying the garbage through. CORD-08 §1 says malformed means off and a reader must not guess, so the value is uninterpretable by construction — but every other content struct in this crate round-trips bytes it does not understand via `extra`, and this field does not. Nothing depends on the distinction yet; giving an uninterpretable timer an `extra` slot is not worth it until something does. +13. **The private→public Pin List rule has no encoding.** §7 says a list MUST NOT be *mechanically* re-formed across a private→public conversion, because the pre-switch entries are private-era and a re-form republishes them community-wide — but "mechanically" and "deliberately" are the same bytes, so no reader-side check can tell them apart and `pins::publishable` does not pretend to. It refuses only what it can prove: a list it could not read. The refusal is therefore a duty of the conversion flow (§10), which must not auto-republish on a `private: false` metadata change. ## 15. Test strategy -- **Unit, pure:** derivations against golden vectors, edition hash, fold, coalesce, memberlist, blob codec, caps and rejection paths. These need no GPUI context and should be exhaustive — they are where cross-client divergence is caught. +- **Unit, pure:** derivations against golden vectors, edition hash, fold, coalesce, memberlist, blob codec, the NIP-44 disclosure, caps and rejection paths. These need no GPUI context and should be exhaustive — they are where cross-client divergence is caught. - **Integration, GPUI:** `TestAppContext` with two registries sharing an in-memory database, driving wraps through the ingest path; timeouts and delays use `cx.background_executor().timer(..)` per the project guidelines, never `smol::Timer`. - **Round-trip:** every builder paired with its parser, asserting the parse produces the identical structure, including unknown-field round-tripping on all content types. - **Negative:** every bullet in §12 gets a test that constructs the hostile input and asserts the drop. diff --git a/crates/concord/Cargo.toml b/crates/concord/Cargo.toml index 29d21505..009a86e1 100644 --- a/crates/concord/Cargo.toml +++ b/crates/concord/Cargo.toml @@ -10,6 +10,8 @@ nostr-sdk.workspace = true hkdf.workspace = true sha2.workspace = true +chacha20.workspace = true +hmac.workspace = true data-encoding.workspace = true rand.workspace = true serde.workspace = true diff --git a/crates/concord/src/chat.rs b/crates/concord/src/chat.rs index 0d9a8a1d..b6106287 100644 --- a/crates/concord/src/chat.rs +++ b/crates/concord/src/chat.rs @@ -22,6 +22,7 @@ pub const KIND_REACTION: u16 = 7; pub const KIND_DELETE: u16 = 5; pub const KIND_EDIT: u16 = 3302; pub const KIND_FILE: u16 = 15; +pub const KIND_TIMER_NOTICE: u16 = 1740; pub const KIND_WEBXDC: u16 = 3310; pub const KIND_TYPING: u16 = 23311; @@ -33,6 +34,7 @@ const TAG_ROOT_KIND: &str = "K"; const TAG_ROOT_AUTHOR: &str = "P"; const TAG_TARGET_AUTHOR: &str = "p"; const TAG_EXPIRATION: &str = "expiration"; +const TAG_TIMER: &str = "timer"; #[derive(Debug)] pub enum ChatError { @@ -42,6 +44,9 @@ pub enum ChatError { MissingTag(&'static str), DuplicateTag(&'static str), BadTag(&'static str), + /// A delete is a tombstone and a timer notice documents the policy, so + /// neither may be erased by the policy it carries. + ExemptExpiration, } impl fmt::Display for ChatError { @@ -53,6 +58,9 @@ impl fmt::Display for ChatError { ChatError::MissingTag(name) => write!(f, "missing chat tag: {name}"), ChatError::DuplicateTag(name) => write!(f, "duplicate chat tag: {name}"), ChatError::BadTag(name) => write!(f, "malformed chat tag: {name}"), + ChatError::ExemptExpiration => { + write!(f, "a delete or timer notice must not carry an expiration") + } } } } @@ -102,6 +110,9 @@ pub enum ChatAction { }, Typing, Opaque, + TimerNotice { + seconds: u64, + }, } #[derive(Debug, Clone)] @@ -142,6 +153,7 @@ pub fn build_message( content: &str, quote: Option<&ReplyRef>, at_ms: u64, + timer: Option, ) -> UnsignedEvent { let mut tags = channel_binding_tags(channel, epoch); @@ -149,11 +161,14 @@ pub fn build_message( tags.push(reply_tag(TAG_QUOTE, quote)); } + tags.extend(expiration_tag(at_ms, timer)); + build_rumor_ms(KIND_MESSAGE, author, content, tags, at_ms) } /// A NIP-22 comment. `parent` is the immediate parent and `root` the thread's /// immutable root; `None` means the parent is itself the root. +#[allow(clippy::too_many_arguments)] pub fn build_comment( author: PublicKey, channel: &ChannelId, @@ -162,6 +177,7 @@ pub fn build_comment( parent: &Target, root: Option<&Target>, at_ms: u64, + timer: Option, ) -> UnsignedEvent { let root = root.unwrap_or(parent); let mut tags = channel_binding_tags(channel, epoch); @@ -178,6 +194,8 @@ pub fn build_comment( tags.push(Tag::custom(TAG_TARGET_AUTHOR, [parent_author.to_hex()])); } + tags.extend(expiration_tag(at_ms, timer)); + build_rumor_ms(KIND_COMMENT, author, content, tags, at_ms) } @@ -188,6 +206,7 @@ pub fn build_reaction( target: &Target, emoji: &str, at_ms: u64, + timer: Option, ) -> UnsignedEvent { let mut tags = channel_binding_tags(channel, epoch); @@ -197,6 +216,8 @@ pub fn build_reaction( } tags.push(Tag::custom(TAG_TARGET_KIND, [target.kind.to_string()])); + tags.extend(expiration_tag(at_ms, timer)); + build_rumor_ms(KIND_REACTION, author, emoji, tags, at_ms) } @@ -207,13 +228,37 @@ pub fn build_edit( target: EventId, content: &str, at_ms: u64, + timer: Option, ) -> UnsignedEvent { let mut tags = channel_binding_tags(channel, epoch); tags.push(Tag::custom(TAG_TARGET, [target.to_hex()])); + tags.extend(expiration_tag(at_ms, timer)); + build_rumor_ms(KIND_EDIT, author, content, tags, at_ms) } +/// CORD-08 §4: an informational row in the timeline, gated by the roster rather +/// than by the fold, so it is built like any other chat rumor. +pub fn build_timer_notice( + author: PublicKey, + channel: &ChannelId, + epoch: Epoch, + seconds: u64, + at_ms: u64, +) -> UnsignedEvent { + let mut tags = channel_binding_tags(channel, epoch); + tags.push(Tag::custom(TAG_TIMER, [seconds.to_string()])); + + build_rumor_ms(KIND_TIMER_NOTICE, author, "", tags, at_ms) +} + +/// The tag is derived from the rumor's own signed `created_at`, so a later +/// metadata edit can never reach back into history. +fn expiration_tag(at_ms: u64, timer: Option) -> Option { + timer.map(|timer| Tag::custom(TAG_EXPIRATION, [(at_ms / 1000 + timer).to_string()])) +} + pub fn build_delete( author: PublicKey, channel: &ChannelId, @@ -327,6 +372,7 @@ pub fn plane_keys( pub fn fold( rumors: &[ChatRumor], + now: Timestamp, can_delete: impl Fn(&PublicKey, Option<&AuthorityCitation>, &PublicKey) -> bool, ) -> Vec { let mut order: Vec = (0..rumors.len()).collect(); @@ -338,12 +384,20 @@ pub fn fold( for index in order { let rumor = &rumors[index]; - let ChatAction::Message { - reply_to, - thread_root, - } = &rumor.action - else { + if expired(rumor, now) { continue; + } + + let (reply_to, thread_root) = match &rumor.action { + ChatAction::Message { + reply_to, + thread_root, + } => ( + reply_to.map(|reply| reply.id), + thread_root.map(|reply| reply.id), + ), + ChatAction::TimerNotice { .. } => (None, None), + _ => continue, }; slot.insert(rumor.id, messages.len()); @@ -354,8 +408,8 @@ pub fn fold( epoch: rumor.epoch, kind: rumor.kind, content: rumor.content.clone(), - reply_to: reply_to.map(|reply| reply.id), - thread_root: thread_root.map(|reply| reply.id), + reply_to, + thread_root, at_ms: rumor.at_ms, expiration: rumor.expiration, edited_at: None, @@ -404,7 +458,10 @@ pub fn fold( messages[slot].reactions.insert(rumor.author, emoji.clone()); } - ChatAction::Message { .. } | ChatAction::Typing | ChatAction::Opaque => {} + ChatAction::Message { .. } + | ChatAction::Typing + | ChatAction::TimerNotice { .. } + | ChatAction::Opaque => {} } } @@ -413,6 +470,11 @@ pub fn fold( messages } +/// CORD-08 §3: an expired rumor is never displayed, whatever its ingest path. +pub fn expired(rumor: &ChatRumor, now: Timestamp) -> bool { + rumor.expiration.is_some_and(|expiration| expiration <= now) +} + fn is_chat_kind(kind: u16) -> bool { matches!( kind, @@ -422,12 +484,19 @@ fn is_chat_kind(kind: u16) -> bool { | KIND_DELETE | KIND_EDIT | KIND_FILE + | KIND_TIMER_NOTICE | KIND_WEBXDC | KIND_TYPING ) } fn typed(rumor: &UnsignedEvent, channel: &ChannelId, epoch: Epoch) -> Result { + let expiration = expiration_of(rumor)?; + + if expiration.is_some() && matches!(rumor.kind.as_u16(), KIND_DELETE | KIND_TIMER_NOTICE) { + return Err(ChatError::ExemptExpiration); + } + Ok(ChatRumor { id: rumor.id.unwrap_or_else(|| rumor.compute_id()), author: rumor.pubkey, @@ -436,7 +505,7 @@ fn typed(rumor: &UnsignedEvent, channel: &ChannelId, epoch: Epoch) -> Result Result { citation: optional_citation(rumor)?, }), KIND_TYPING => Ok(ChatAction::Typing), + KIND_TIMER_NOTICE => Ok(ChatAction::TimerNotice { + seconds: timer_of(rumor)?, + }), KIND_WEBXDC => Ok(ChatAction::Opaque), other => Err(ChatError::UnknownKind(other)), } } +fn timer_of(rumor: &UnsignedEvent) -> Result { + let fields = tag(rumor, TAG_TIMER)?.ok_or(ChatError::MissingTag(TAG_TIMER))?; + + canonical_decimal(value(fields, TAG_TIMER)?).ok_or(ChatError::BadTag(TAG_TIMER)) +} + fn optional_reply( rumor: &UnsignedEvent, name: &'static str, @@ -520,7 +598,7 @@ fn optional_citation(rumor: &UnsignedEvent) -> Result, .ok_or(ChatError::BadTag(TAG_CITATION)) } -fn expiration_of(rumor: &UnsignedEvent) -> Result, ChatError> { +pub fn expiration_of(rumor: &UnsignedEvent) -> Result, ChatError> { let Some(fields) = tag(rumor, TAG_EXPIRATION)? else { return Ok(None); }; @@ -594,6 +672,11 @@ mod tests { const SECRET: [u8; 32] = [0x2du8; 32]; const AT: u64 = 1_700_000_000_417; + /// Well past every timestamp these tests use. + fn now() -> Timestamp { + Timestamp::from_secs(2_000_000_000) + } + fn channel() -> ChannelId { ChannelId::from_bytes([0x9cu8; 32]) } @@ -628,7 +711,15 @@ mod tests { let carol = Keys::generate(); let group = group(); - let message = build_message(alice.public_key(), &channel(), Epoch(0), "hello", None, AT); + let message = build_message( + alice.public_key(), + &channel(), + Epoch(0), + "hello", + None, + AT, + None, + ); let id = message.compute_id(); let rumors = vec![ @@ -641,6 +732,7 @@ mod tests { &target(id, &alice), "🔥", AT + 1_000, + None, ), &group, &carol, @@ -654,6 +746,7 @@ mod tests { id, "hello (fixed)", AT + 2_000, + None, ), &group, &alice, @@ -675,7 +768,7 @@ mod tests { ), ]; - let folded = fold(&rumors, |_, _, _| false); + let folded = fold(&rumors, now(), |_, _, _| false); assert_eq!(folded.len(), 1); assert_eq!(folded[0].id, id); @@ -694,7 +787,15 @@ mod tests { let bob = Keys::generate(); let group = group(); - let message = build_message(alice.public_key(), &channel(), Epoch(0), "hello", None, AT); + let message = build_message( + alice.public_key(), + &channel(), + Epoch(0), + "hello", + None, + AT, + None, + ); let id = message.compute_id(); let rumors = vec![ @@ -707,6 +808,7 @@ mod tests { id, "mine now", AT + 1_000, + None, ), &group, &bob, @@ -728,7 +830,7 @@ mod tests { ), ]; - let folded = fold(&rumors, |_, _, _| false); + let folded = fold(&rumors, now(), |_, _, _| false); assert_eq!(folded.len(), 1); assert_eq!(folded[0].content, "hello"); @@ -742,7 +844,15 @@ mod tests { let bob = Keys::generate(); let group = group(); - let root = build_message(alice.public_key(), &channel(), Epoch(0), "root", None, AT); + let root = build_message( + alice.public_key(), + &channel(), + Epoch(0), + "root", + None, + AT, + None, + ); let root_id = root.compute_id(); let parent = build_message( bob.public_key(), @@ -751,6 +861,7 @@ mod tests { "parent", None, AT + 1_000, + None, ); let parent_id = parent.compute_id(); @@ -762,6 +873,7 @@ mod tests { &target(parent_id, &bob), Some(&target(root_id, &alice)), AT + 2_000, + None, ); assert!(comment.tags.iter().any(|tag| tag.as_slice() == ["K", "9"])); @@ -796,7 +908,15 @@ mod tests { let alice = Keys::generate(); let group = group(); - let plain = build_message(alice.public_key(), &channel(), Epoch(0), "hello", None, AT); + let plain = build_message( + alice.public_key(), + &channel(), + Epoch(0), + "hello", + None, + AT, + None, + ); assert!( open( &sealed(&plain, &group, &alice), @@ -820,7 +940,15 @@ mod tests { Err(ChatError::Stream(StreamError::ChannelMismatch)) )); - let stale = build_message(alice.public_key(), &channel(), Epoch(1), "stale", None, AT); + let stale = build_message( + alice.public_key(), + &channel(), + Epoch(1), + "stale", + None, + AT, + None, + ); assert!(matches!( open( &sealed(&stale, &group, &alice), @@ -882,7 +1010,15 @@ mod tests { let group = group(); let message = read( - &build_message(alice.public_key(), &channel(), Epoch(0), "hello", None, AT), + &build_message( + alice.public_key(), + &channel(), + Epoch(0), + "hello", + None, + AT, + None, + ), &group, &alice, Epoch(0), @@ -922,26 +1058,121 @@ mod tests { ChatAction::Delete { citation: Some(parsed), .. } if *parsed == citation )); assert!( - fold(&cited, can_delete)[0].deleted, + fold(&cited, now(), can_delete)[0].deleted, "a cited moderator delete lands" ); let uncited = vec![message.clone(), delete(&moderator, None)]; assert!( - !fold(&uncited, can_delete)[0].deleted, + !fold(&uncited, now(), can_delete)[0].deleted, "an uncited delete names no rank" ); let peer_delete = vec![message.clone(), delete(&peer, Some(&citation))]; assert!( - !fold(&peer_delete, can_delete)[0].deleted, + !fold(&peer_delete, now(), can_delete)[0].deleted, "a peer's delete is not authority" ); let own = vec![message.clone(), delete(&alice, None)]; assert!( - fold(&own, |_, _, _| false)[0].deleted, + fold(&own, now(), |_, _, _| false)[0].deleted, "a self-delete never consults the predicate" ); } + + #[test] + fn a_timer_rides_durable_rumors_and_expiry_gates_the_fold() { + let alice = Keys::generate(); + let group = group(); + let expires = (AT / 1000 + 60).to_string(); + + // Computed from the signed `created_at`, and mirrored onto the wrap so + // relays drop the ciphertext too. + let message = build_message( + alice.public_key(), + &channel(), + Epoch(0), + "tick", + None, + AT, + Some(60), + ); + assert!( + message + .tags + .iter() + .any(|tag| tag.as_slice() == [TAG_EXPIRATION, expires.as_str()]) + ); + assert!( + sealed(&message, &group, &alice) + .tags + .iter() + .any(|tag| tag.as_slice() == [TAG_EXPIRATION, expires.as_str()]) + ); + + let live = read(&message, &group, &alice, Epoch(0)); + assert_eq!(live.expiration, Some(Timestamp::from_secs(AT / 1000 + 60))); + assert!(!expired(&live, Timestamp::from_secs(AT / 1000 + 59))); + assert!(expired(&live, Timestamp::from_secs(AT / 1000 + 60))); + assert_eq!( + fold( + std::slice::from_ref(&live), + Timestamp::from_secs(AT / 1000 + 59), + |_, _, _| false + ) + .len(), + 1 + ); + assert_eq!( + fold(&[live], Timestamp::from_secs(AT / 1000 + 60), |_, _, _| { + false + }) + .len(), + 0 + ); + + // A delete is a tombstone and a notice documents the policy, so neither + // may be erased by the policy it carries. + let mut expiring = channel_binding_tags(&channel(), Epoch(0)); + expiring.push(Tag::custom(TAG_EXPIRATION, ["1"])); + expiring.push(Tag::custom(TAG_TARGET, ["ab".repeat(32)])); + + for kind in [KIND_DELETE, KIND_TIMER_NOTICE] { + let rumor = build_rumor_ms(kind, alice.public_key(), "", expiring.clone(), AT); + assert!(matches!( + open( + &sealed(&rumor, &group, &alice), + &group, + &channel(), + Epoch(0) + ), + Err(ChatError::ExemptExpiration) + )); + } + + // A notice is a row of its own; whether its author may be believed + // about policy is the roster's call, not the fold's. + let notice = build_timer_notice(alice.public_key(), &channel(), Epoch(0), 3_600, AT); + let folded = fold( + &[read(¬ice, &group, &alice, Epoch(0))], + now(), + |_, _, _| false, + ); + assert_eq!(folded.len(), 1); + assert_eq!(folded[0].kind, Kind::Custom(KIND_TIMER_NOTICE)); + + let mut malformed = channel_binding_tags(&channel(), Epoch(0)); + malformed.push(Tag::custom(TAG_TIMER, ["060"])); + let rumor = build_rumor_ms(KIND_TIMER_NOTICE, alice.public_key(), "", malformed, AT); + assert!(matches!( + open( + &sealed(&rumor, &group, &alice), + &group, + &channel(), + Epoch(0) + ), + Err(ChatError::BadTag(TAG_TIMER)) + )); + } } diff --git a/crates/concord/src/control.rs b/crates/concord/src/control.rs index 6cf9da8a..323c8e64 100644 --- a/crates/concord/src/control.rs +++ b/crates/concord/src/control.rs @@ -6,14 +6,15 @@ use serde::{Deserialize, Serialize}; use crate::derive::{ banlist_locator, community_id_of, control_group_key, control_signer_group_key, grant_locator, - invite_links_locator, verify_community_id, + invite_links_locator, pins_locator, verify_community_id, }; use crate::edition::{ AuthorityCitation, EditionFields, EditionMeta, EntityHead, Floors, ParsedEdition, build_edition, fold_head, parse_edition, vsk, }; use crate::roles::{ - AuthorityEdition, CommunityRoles, Grant, Permissions, Role, Roster, citation_ok, fold_roster, + AuthorityEdition, CommunityRoles, Grant, MAX_BANLIST, Permissions, Role, Roster, citation_ok, + fold_roster, }; use crate::stream::{KIND_WRAP, SealForm, build_seal, open_wrap_at, wrap_seal_with}; use crate::{ChannelId, CommunityId, Epoch, Extra, GroupKey, random_32}; @@ -49,12 +50,32 @@ pub struct CommunityMetadata { pub icon: Option, #[serde(default, skip_serializing_if = "Option::is_none")] pub banner: Option, + /// CORD-08's disappearing-messages timer, in seconds. + #[serde( + default, + deserialize_with = "timer_seconds", + skip_serializing_if = "Option::is_none" + )] + pub message_expiration: Option, #[serde(default, skip_serializing_if = "Option::is_none")] pub custom: Option, #[serde(flatten)] pub extra: Extra, } +/// CORD-08 §1: absent, `0` and malformed all mean off, and a reader must not +/// guess a default from garbage. +fn timer_seconds<'de, D>(deserializer: D) -> Result, D::Error> +where + D: serde::Deserializer<'de>, +{ + let value = Option::::deserialize(deserializer)?; + + Ok(value + .and_then(|value| value.as_u64()) + .filter(|seconds| *seconds > 0)) +} + #[derive(Debug, Clone, Default, PartialEq, Serialize, Deserialize)] pub struct ChannelMetadata { pub name: String, @@ -249,6 +270,10 @@ impl ControlWriter { citation: Option, at_secs: u64, ) -> Result<(Event, EntityHead)> { + if metadata.name.len() > MAX_NAME_BYTES { + bail!("channel name exceeds {MAX_NAME_BYTES} bytes"); + } + let content = serde_json::to_string(metadata)?; self.publish( @@ -272,6 +297,10 @@ impl ControlWriter { citation: Option, at_secs: u64, ) -> Result<(Event, EntityHead)> { + if role.name.len() > MAX_NAME_BYTES { + bail!("role name exceeds {MAX_NAME_BYTES} bytes"); + } + let content = role.to_content()?; self.publish( @@ -320,6 +349,10 @@ impl ControlWriter { citation: Option, at_secs: u64, ) -> Result<(Event, EntityHead)> { + if banned.len() > MAX_BANLIST { + bail!("banlist exceeds {MAX_BANLIST} entries"); + } + let entries: Vec = banned.iter().map(PublicKey::to_hex).collect(); let content = serde_json::to_string(&entries)?; @@ -366,6 +399,32 @@ impl ControlWriter { at_secs, ) } + + /// `content` is the whole Pin List, in whichever of CORD-04 §7's two + /// self-describing forms the Channel's folded type calls for. + #[allow(clippy::too_many_arguments)] + pub fn set_pin_list( + &self, + keys: &Keys, + community_id: &CommunityId, + channel: &ChannelId, + content: &str, + head: Option<&EntityHead>, + citation: Option, + at_secs: u64, + ) -> Result<(Event, EntityHead)> { + self.publish( + keys, + Edition { + subkind: vsk::PINS, + entity: pins_locator(community_id, channel), + content, + head, + citation, + }, + at_secs, + ) + } } fn encode_metadata(metadata: &CommunityMetadata) -> Result { @@ -383,6 +442,7 @@ fn encode_metadata(metadata: &CommunityMetadata) -> Result { let mut metadata = metadata.clone(); metadata.relays.truncate(MAX_RELAYS); + metadata.message_expiration = metadata.message_expiration.filter(|seconds| *seconds > 0); Ok(serde_json::to_string(&metadata)?) } @@ -395,6 +455,9 @@ pub struct ControlFold { pub channels: BTreeMap, /// Each creator's live link-signer set. pub registries: BTreeMap>, + /// Head content per `pins_locator`: a Pin List is addressed by a one-way + /// coordinate, so a fold cannot name the Channel it belongs to. + pub pins: BTreeMap<[u8; 32], String>, pub floors: Floors, pub gapped: bool, } @@ -403,6 +466,12 @@ impl ControlFold { pub fn is_public(&self) -> bool { self.registries.values().any(|links| !links.is_empty()) } + + pub fn pin_content(&self, community_id: &CommunityId, channel: &ChannelId) -> Option<&str> { + self.pins + .get(&pins_locator(community_id, channel)) + .map(String::as_str) + } } pub fn fold_control( @@ -429,6 +498,7 @@ pub fn fold_control( community: metadata.community, channels: metadata.channels, registries: metadata.registries, + pins: metadata.pins, floors, gapped: roster.gapped || metadata.gapped, } @@ -439,6 +509,7 @@ struct MetadataFold { community: Option, channels: BTreeMap, registries: BTreeMap>, + pins: BTreeMap<[u8; 32], String>, floors: Floors, gapped: bool, } @@ -483,7 +554,14 @@ fn fold_metadata( Permissions::MANAGE_METADATA, &mut fold.gapped, ) { - fold.community = serde_json::from_str(&head.content).ok(); + fold.community = serde_json::from_str::(&head.content) + .ok() + .map(|mut metadata| { + // Up to 5 relays is a recommendation, so a longer set is + // truncated rather than refused, on read as well as on write. + metadata.relays.truncate(MAX_RELAYS); + metadata + }); fold.floors.insert(head.entity, EntityHead::from(head)); } @@ -507,10 +585,44 @@ fn fold_metadata( } fold.registries = fold_registries(&judge, editions, &mut fold.floors, &mut fold.gapped); + fold.pins = fold_pins(&judge, editions, &mut fold.floors, &mut fold.gapped); fold } +/// A Pin List's coordinate derives one-way, so unlike the banlist, a grant or a +/// registry there is nothing to check the `eid` against: an edition at an +/// unknown coordinate is simply never read. Its content is stored verbatim, +/// because a violating list still folds but reads as empty (CORD-04 §7). +fn fold_pins( + judge: &Judge<'_>, + editions: &[ParsedEdition], + floors: &mut Floors, + gapped: &mut bool, +) -> BTreeMap<[u8; 32], String> { + let mut candidates: BTreeMap<[u8; 32], Vec<&ParsedEdition>> = BTreeMap::new(); + + for edition in editions { + if edition.subkind == vsk::PINS { + candidates.entry(edition.entity).or_default().push(edition); + } + } + + let mut pins = BTreeMap::new(); + + for (entity, group) in &candidates { + let Some(head) = authorized_head(judge, *entity, group, Permissions::PIN_MESSAGES, gapped) + else { + continue; + }; + + floors.insert(*entity, EntityHead::from(head)); + pins.insert(*entity, head.content.clone()); + } + + pins +} + fn fold_registries( judge: &Judge<'_>, editions: &[ParsedEdition], @@ -630,11 +742,12 @@ mod tests { use nostr_memory::MemoryDatabase; use super::*; - use crate::derive::grant_locator; + use crate::chat::{self, build_message, seal_rumor}; + use crate::derive::{channel_group_key, grant_locator}; use crate::edition::fold; - use crate::roles::{Grant, Role, RoleScope}; + use crate::roles::{Grant, MAX_BANLIST, MAX_ROLES_PER_MEMBER, Role, RoleScope}; use crate::store::{CommunityState, load_state, save_state}; - use crate::{Extra, RoleId}; + use crate::{Extra, RoleId, pins}; const AT: u64 = 1_700_000_000; @@ -976,4 +1089,180 @@ mod tests { Some("coop by mod") ); } + + #[test] + fn a_pin_list_folds_under_its_coordinate_for_a_second_client() { + let owner = Keys::generate(); + let minted = genesis(&owner, &metadata("coop"), AT).expect("mints"); + let community_id = minted.identity.community_id; + let owner_pk = owner.public_key(); + let (read, signer) = holder(&minted); + + let channel = minted.channel_id; + let group = + channel_group_key(&minted.community_root, &channel, ROOT_EPOCH).expect("derives"); + let author = Keys::generate(); + + let rumor = build_message( + author.public_key(), + &channel, + ROOT_EPOCH, + "pin me", + None, + AT * 1_000, + None, + ); + let (wrap, _) = seal_rumor(&rumor, &group, &author, false).expect("seals"); + let opened = chat::open(&wrap, &group, &channel, ROOT_EPOCH) + .expect("opens") + .0; + let entry = pins::build_entry(&opened, &group, &channel).expect("pins"); + + let content = pins::publishable( + &pins::ReadPinList { + entries: vec![entry], + sealed: false, + }, + false, + &group, + ROOT_EPOCH, + ) + .expect("publishes"); + + let writer = ControlWriter { + author: owner_pk, + read: read.clone(), + signer: signer.clone(), + }; + let (pin_wrap, _) = writer + .set_pin_list( + &owner, + &community_id, + &channel, + &content, + None, + None, + AT + 1, + ) + .expect("publishes"); + + let mut editions = open_all(&minted.wraps, &read, &signer.pk()); + editions.extend(open_all(&[pin_wrap], &read, &signer.pk())); + + let folded = fold_control( + &owner_pk, + &community_id, + &editions, + &Floors::new(), + &BTreeSet::new(), + ); + + // The coordinate derives one-way, so the list is found by naming the Channel. + let found = pins::read_list( + folded + .pin_content(&community_id, &channel) + .expect("the list folds"), + |_| None, + ); + assert_eq!(found.entries.len(), 1); + assert_eq!( + pins::verify_entry(&found.entries[0], &channel) + .expect("verifies") + .content, + "pin me" + ); + + let other = ChannelId::from_bytes([0x77; 32]); + assert!(folded.pin_content(&community_id, &other).is_none()); + } + + #[test] + fn the_timer_is_never_guessed_and_the_write_caps_hold() { + let owner = Keys::generate(); + let minted = genesis(&owner, &metadata("coop"), AT).expect("mints"); + let community_id = minted.identity.community_id; + let owner_pk = owner.public_key(); + let (read, signer) = holder(&minted); + + let writer = ControlWriter { + author: owner_pk, + read, + signer, + }; + let fold = |metadata: &CommunityMetadata| { + fold_control( + &owner_pk, + &community_id, + &open_all( + &[writer + .set_community_metadata(&owner, &community_id, metadata, None, None, AT + 1) + .expect("publishes") + .0], + &writer.read, + &writer.signer.pk(), + ), + &Floors::new(), + &BTreeSet::new(), + ) + .community + .expect("folds") + }; + + let mut timed = metadata("coop"); + timed.message_expiration = Some(2_592_000); + assert_eq!(fold(&timed).message_expiration, Some(2_592_000)); + + // Absent, zero and garbage all mean off, and garbage never poisons the rest. + assert_eq!(fold(&metadata("coop")).message_expiration, None); + + let mut off = metadata("coop"); + off.message_expiration = Some(0); + assert_eq!(fold(&off).message_expiration, None); + + let garbage = serde_json::json!({ + "name": "coop", + "message_expiration": "later", + }) + .to_string(); + let folded: CommunityMetadata = serde_json::from_str(&garbage).expect("parses"); + assert_eq!(folded.name, "coop"); + assert_eq!(folded.message_expiration, None); + + // The caps the folds apply also hold on the way out. + let banned: BTreeSet = (0..=MAX_BANLIST) + .map(|_| Keys::generate().public_key()) + .collect(); + assert!( + writer + .set_banlist(&owner, &community_id, &banned, None, None, AT + 2) + .is_err() + ); + + let grant = Grant { + member: owner_pk, + role_ids: (0..=MAX_ROLES_PER_MEMBER) + .map(|index| RoleId::from_bytes([index as u8; 32])) + .collect(), + control_wrap: None, + extra: Extra::default(), + }; + assert!(grant.to_content().is_err()); + + assert!( + writer + .set_channel_metadata( + &owner, + &minted.channel_id, + &ChannelMetadata { + name: "x".repeat(MAX_NAME_BYTES + 1), + private: false, + ..ChannelMetadata::default() + }, + None, + None, + AT + 3, + ) + .is_err() + ); + } } diff --git a/crates/concord/src/lib.rs b/crates/concord/src/lib.rs index c4f72bb6..2e5057bb 100644 --- a/crates/concord/src/lib.rs +++ b/crates/concord/src/lib.rs @@ -5,6 +5,7 @@ pub mod edition; pub mod guestbook; pub mod invite; pub mod list; +pub mod pins; pub mod rekey; pub mod roles; pub mod store; @@ -125,14 +126,18 @@ impl fmt::Display for Epoch { /// Uppercase and other non-canonical spellings are rejected. pub(crate) fn decode_hex_32(value: &str) -> Result<[u8; 32]> { + decode_hex_lower::<32>(value) +} + +pub(crate) fn decode_hex_lower(value: &str) -> Result<[u8; N]> { let bytes = HEXLOWER .decode(value.as_bytes()) .map_err(|error| anyhow!("invalid hex: {error}"))?; - let decoded: [u8; 32] = bytes + let decoded: [u8; N] = bytes .as_slice() .try_into() - .map_err(|_| anyhow!("expected 32 bytes, got {}", bytes.len()))?; + .map_err(|_| anyhow!("expected {N} bytes, got {}", bytes.len()))?; if HEXLOWER.encode(&decoded) != value { bail!("hex must be lowercase and canonical"); diff --git a/crates/concord/src/pins.rs b/crates/concord/src/pins.rs new file mode 100644 index 00000000..91c774ce --- /dev/null +++ b/crates/concord/src/pins.rs @@ -0,0 +1,825 @@ +use std::fmt; + +use chacha20::ChaCha20; +use chacha20::cipher::{KeyIvInit, StreamCipher}; +use data_encoding::{BASE64, HEXLOWER}; +use hkdf::Hkdf; +use hmac::{Hmac, Mac}; +use nostr_sdk::prelude::*; +use serde::{Deserialize, Serialize}; +use sha2::Sha256; + +use crate::chat::{ChatAction, ChatRumor, KIND_COMMENT, KIND_EDIT, KIND_MESSAGE}; +use crate::edition::canonical_decimal; +use crate::stream::{self, OpenedStream, SealForm, resolve_ms_strict}; +use crate::{ChannelId, Epoch, Extra, GroupKey, decode_hex_lower}; + +pub const PIN_MAX_ENTRIES: usize = 25; +pub const PIN_MAX_CONTENT_BYTES: usize = 32_768; + +/// The serialized disclosure: `chacha_key[32] || chacha_nonce[12] || hmac_key[32]`. +pub const MESSAGE_KEYS_BYTES: usize = 76; + +const TAG_CHANNEL: &str = "channel"; +const TAG_EPOCH: &str = "epoch"; +const TAG_TARGET: &str = "e"; + +#[derive(Debug, Clone, PartialEq, Eq)] +pub enum PinError { + NotEncryptedSeal, + BadPayload, + Unverifiable, + Unreadable, + TooManyEntries, + Oversize(usize), + Seal(String), + Encode(String), +} + +impl fmt::Display for PinError { + fn fmt(&self, f: &mut fmt::Formatter<'_>) -> fmt::Result { + match self { + PinError::NotEncryptedSeal => write!(f, "pin requires an encrypted seal"), + PinError::BadPayload => write!(f, "the seal payload does not open"), + PinError::Unverifiable => write!(f, "the entry would not verify"), + PinError::Unreadable => { + write!(f, "refusing to publish a pin list this client cannot read") + } + PinError::TooManyEntries => write!(f, "pin list exceeds {PIN_MAX_ENTRIES} entries"), + PinError::Oversize(len) => { + write!( + f, + "pin list content is {len} bytes (cap {PIN_MAX_CONTENT_BYTES})" + ) + } + PinError::Seal(error) => write!(f, "seal: {error}"), + PinError::Encode(error) => write!(f, "encode: {error}"), + } + } +} + +impl std::error::Error for PinError {} + +#[derive(Clone, Copy, PartialEq, Eq)] +pub struct MessageKeys { + chacha_key: [u8; 32], + chacha_nonce: [u8; 12], + hmac_key: [u8; 32], +} + +impl fmt::Debug for MessageKeys { + fn fmt(&self, f: &mut fmt::Formatter<'_>) -> fmt::Result { + f.write_str("MessageKeys()") + } +} + +impl MessageKeys { + pub fn to_hex(&self) -> String { + let mut packed = [0u8; MESSAGE_KEYS_BYTES]; + packed[0..32].copy_from_slice(&self.chacha_key); + packed[32..44].copy_from_slice(&self.chacha_nonce); + packed[44..76].copy_from_slice(&self.hmac_key); + HEXLOWER.encode(&packed) + } + + pub fn from_hex(value: &str) -> Option { + let bytes = decode_hex_lower::(value).ok()?; + + Some(Self { + chacha_key: bytes[0..32].try_into().ok()?, + chacha_nonce: bytes[32..44].try_into().ok()?, + hmac_key: bytes[44..76].try_into().ok()?, + }) + } + + fn derive(conversation_key: &[u8; 32], nonce: &[u8]) -> Option { + let hkdf = Hkdf::::from_prk(conversation_key).ok()?; + let mut key_material = [0u8; MESSAGE_KEYS_BYTES]; + hkdf.expand(nonce, &mut key_material).ok()?; + + Some(Self { + chacha_key: key_material[0..32].try_into().ok()?, + chacha_nonce: key_material[32..44].try_into().ok()?, + hmac_key: key_material[44..76].try_into().ok()?, + }) + } +} + +struct Payload { + nonce: [u8; 32], + ciphertext: Vec, + mac: [u8; 32], +} + +fn decode_payload(payload: &str) -> Option { + let data = BASE64.decode(payload.as_bytes()).ok()?; + + if data.len() < 99 || data[0] != 2 { + return None; + } + + let mac_at = data.len() - 32; + + Some(Payload { + nonce: data[1..33].try_into().ok()?, + ciphertext: data[33..mac_at].to_vec(), + mac: data[mac_at..].try_into().ok()?, + }) +} + +fn disclose_keys(payload: &str, conversation_key: &[u8; 32]) -> Option { + let decoded = decode_payload(payload)?; + MessageKeys::derive(conversation_key, &decoded.nonce) +} + +fn open_payload(payload: &str, keys: &MessageKeys) -> Option { + let decoded = decode_payload(payload)?; + + let mut mac = Hmac::::new_from_slice(&keys.hmac_key).ok()?; + mac.update(&decoded.nonce); + mac.update(&decoded.ciphertext); + mac.verify_slice(&decoded.mac).ok()?; + + let mut padded = decoded.ciphertext; + let mut cipher = ChaCha20::new((&keys.chacha_key).into(), (&keys.chacha_nonce).into()); + cipher.apply_keystream(&mut padded); + + unpad(&padded) +} + +fn unpad(padded: &[u8]) -> Option { + let (len, prefix) = plaintext_length(padded)?; + let unpadded = padded.get(prefix..prefix.checked_add(len)?)?; + + if len < 1 || padded.len() != prefix.checked_add(padded_len(len)?)? { + return None; + } + + String::from_utf8(unpadded.to_vec()).ok() +} + +fn plaintext_length(padded: &[u8]) -> Option<(usize, usize)> { + let short = u16::from_be_bytes(padded.get(..2)?.try_into().ok()?); + + if short != 0 { + return Some((short as usize, 2)); + } + + let long = u32::from_be_bytes(padded.get(2..6)?.try_into().ok()?); + + if long < 65_536 { + return None; + } + + Some((long as usize, 6)) +} + +fn padded_len(len: usize) -> Option { + if len < 1 { + return None; + } + + if len <= 32 { + return Some(32); + } + + let next_power = 1usize.checked_shl(usize::BITS - (len - 1).leading_zeros())?; + let chunk = if next_power <= 256 { + 32 + } else { + next_power / 8 + }; + + Some(chunk * ((len - 1) / chunk + 1)) +} + +#[derive(Debug, Clone, PartialEq, Serialize, Deserialize)] +pub struct PinEditBundle { + pub seal: Event, + pub keys: String, +} + +#[derive(Debug, Clone, PartialEq, Serialize, Deserialize)] +pub struct PinEntry { + pub seal: Event, + pub keys: String, + /// An unverifiable locator hint; a mismatch is expected and never fatal. + #[serde(default, skip_serializing_if = "Option::is_none")] + pub wrap: Option, + #[serde(default, skip_serializing_if = "Option::is_none")] + pub edit: Option, + #[serde(flatten)] + pub extra: Extra, +} + +#[derive(Debug, Clone, PartialEq, Eq)] +pub struct EditedContent { + pub content: String, + pub at_ms: u64, +} + +#[derive(Debug, Clone)] +pub struct VerifiedPin { + pub rumor_id: EventId, + pub author: PublicKey, + pub kind: u16, + pub content: String, + pub tags: Tags, + pub epoch: Epoch, + pub at_ms: u64, + pub created_at: u64, + pub wrap: Option, + pub edited: Option, + pub entry: PinEntry, +} + +#[derive(Debug, Clone, Default)] +pub struct ReadPinList { + pub entries: Vec, + pub sealed: bool, +} + +pub fn build_entry( + opened: &OpenedStream, + group: &GroupKey, + channel: &ChannelId, +) -> Result { + let keys = disclosed_keys(opened, group)?; + + let entry = PinEntry { + seal: opened.seal.clone(), + keys: keys.to_hex(), + wrap: Some(opened.wrapper_id.to_hex()), + edit: None, + extra: Extra::default(), + }; + + if verify_entry(&entry, channel).is_none() { + return Err(PinError::Unverifiable); + } + + Ok(entry) +} + +pub fn build_edit_bundle( + edit: &OpenedStream, + group: &GroupKey, + original: &VerifiedPin, + channel: &ChannelId, +) -> Result { + let bundle = PinEditBundle { + seal: edit.seal.clone(), + keys: disclosed_keys(edit, group)?.to_hex(), + }; + + if verify_edit_bundle(&bundle, &original.author, &original.rumor_id, channel).is_none() { + return Err(PinError::Unverifiable); + } + + Ok(bundle) +} + +pub fn with_proven_edit( + entry: &PinEntry, + edit: &OpenedStream, + group: &GroupKey, + channel: &ChannelId, +) -> PinEntry { + let Some(original) = verify_entry(entry, channel) else { + return entry.clone(); + }; + + let Ok(bundle) = build_edit_bundle(edit, group, &original, channel) else { + return entry.clone(); + }; + + let mut refreshed = entry.clone(); + refreshed.edit = Some(bundle); + refreshed +} + +fn disclosed_keys(opened: &OpenedStream, group: &GroupKey) -> Result { + if opened.seal_form != SealForm::Encrypted { + return Err(PinError::NotEncryptedSeal); + } + + let conversation: [u8; 32] = group + .conversation() + .as_bytes() + .try_into() + .map_err(|_| PinError::BadPayload)?; + + let keys = disclose_keys(&opened.seal.content, &conversation).ok_or(PinError::BadPayload)?; + + if open_payload(&opened.seal.content, &keys).is_none() { + return Err(PinError::BadPayload); + } + + Ok(keys) +} + +pub fn verify_entry(entry: &PinEntry, channel: &ChannelId) -> Option { + let seal = &entry.seal; + + if seal.kind.as_u16() != stream::KIND_SEAL_ENCRYPTED || seal.verify().is_err() { + return None; + } + + let keys = MessageKeys::from_hex(&entry.keys)?; + let plaintext = open_payload(&seal.content, &keys)?; + let rumor = UnsignedEvent::from_json(&plaintext).ok()?; + + // NIP-59's impersonation check: the renderer shows the rumor's fields. + if rumor.pubkey != seal.pubkey { + return None; + } + + let kind = rumor.kind.as_u16(); + + if kind != KIND_MESSAGE && kind != KIND_COMMENT { + return None; + } + + // CORD-01's binding, restated for a path that decrypts no wrap: without + // this, a private Channel's keyholder could pin its messages into a public + // list, disclosing them community-wide with proof. + if tag_value(&rumor, TAG_CHANNEL)? != channel.to_hex() { + return None; + } + + let epoch = Epoch(canonical_decimal(tag_value(&rumor, TAG_EPOCH)?)?); + + // Every reader recomputes the identity; a claimed `id` is never trusted. + rumor.verify_id().ok()?; + let rumor_id = rumor.compute_id(); + + let edited = entry + .edit + .as_ref() + .and_then(|bundle| verify_edit_bundle(bundle, &rumor.pubkey, &rumor_id, channel)); + + Some(VerifiedPin { + author: rumor.pubkey, + content: edited + .as_ref() + .map_or_else(|| rumor.content.clone(), |edited| edited.content.clone()), + epoch, + at_ms: resolve_ms_strict(&rumor).ok()?, + created_at: rumor.created_at.as_secs(), + tags: rumor.tags.clone(), + wrap: entry.wrap.clone(), + edited, + entry: entry.clone(), + kind, + rumor_id, + }) +} + +fn verify_edit_bundle( + bundle: &PinEditBundle, + original_author: &PublicKey, + original_id: &EventId, + channel: &ChannelId, +) -> Option { + let seal = &bundle.seal; + + // Nobody else may revise another member's words, and this is checkable + // before any crypto. + if seal.kind.as_u16() != stream::KIND_SEAL_ENCRYPTED || seal.pubkey != *original_author { + return None; + } + + if seal.verify().is_err() { + return None; + } + + let keys = MessageKeys::from_hex(&bundle.keys)?; + let plaintext = open_payload(&seal.content, &keys)?; + let rumor = UnsignedEvent::from_json(&plaintext).ok()?; + + if rumor.pubkey != seal.pubkey || rumor.kind.as_u16() != KIND_EDIT { + return None; + } + + if tag_value(&rumor, TAG_CHANNEL)? != channel.to_hex() { + return None; + } + + if tag_value(&rumor, TAG_TARGET)? != original_id.to_hex() { + return None; + } + + rumor.verify_id().ok()?; + + Some(EditedContent { + content: rumor.content.clone(), + at_ms: resolve_ms_strict(&rumor).ok()?, + }) +} + +fn tag_value<'a>(rumor: &'a UnsignedEvent, name: &str) -> Option<&'a str> { + rumor + .tags + .iter() + .find(|tag| tag.as_slice().first().map(String::as_str) == Some(name)) + .and_then(|tag| tag.as_slice().get(1)) + .map(String::as_str) +} + +#[derive(Serialize, Deserialize)] +struct PlainForm { + entries: Vec, +} + +pub fn publishable( + read: &ReadPinList, + private: bool, + group: &GroupKey, + epoch: Epoch, +) -> Result { + if read.sealed { + return Err(PinError::Unreadable); + } + + if private { + serialize_sealed(&read.entries, group, epoch) + } else { + serialize_public(&read.entries) + } +} + +fn serialize_public(entries: &[PinEntry]) -> Result { + let content = encode_form(entries)?; + check_caps(entries.len(), &content)?; + + Ok(content) +} + +fn serialize_sealed( + entries: &[PinEntry], + group: &GroupKey, + epoch: Epoch, +) -> Result { + if entries.len() > PIN_MAX_ENTRIES { + return Err(PinError::TooManyEntries); + } + + let inner = encode_form(entries)?; + let sealed = stream::seal_bytes(group.conversation(), inner.as_bytes()) + .map_err(|error| PinError::Seal(error.to_string()))?; + let content = serde_json::json!({ "epoch": epoch.to_string(), "sealed": sealed }).to_string(); + + check_caps(entries.len(), &content)?; + + Ok(content) +} + +fn encode_form(entries: &[PinEntry]) -> Result { + serde_json::to_string(&PlainForm { + entries: entries.to_vec(), + }) + .map_err(|error| PinError::Encode(error.to_string())) +} + +fn check_caps(count: usize, content: &str) -> Result<(), PinError> { + if count > PIN_MAX_ENTRIES { + return Err(PinError::TooManyEntries); + } + + if content.len() > PIN_MAX_CONTENT_BYTES { + return Err(PinError::Oversize(content.len())); + } + + Ok(()) +} + +pub fn read_list(content: &str, unseal: impl Fn(Epoch) -> Option) -> ReadPinList { + const EMPTY: ReadPinList = ReadPinList { + entries: Vec::new(), + sealed: false, + }; + + if content.len() > PIN_MAX_CONTENT_BYTES { + return EMPTY; + } + + let Ok(value) = serde_json::from_str::(content) else { + return EMPTY; + }; + + if value.get("entries").is_some() { + return match serde_json::from_value::(value) { + Ok(form) if form.entries.len() <= PIN_MAX_ENTRIES => ReadPinList { + entries: form.entries, + sealed: false, + }, + _ => EMPTY, + }; + } + + let (Some(epoch), Some(sealed)) = ( + value.get("epoch").and_then(serde_json::Value::as_str), + value.get("sealed").and_then(serde_json::Value::as_str), + ) else { + return EMPTY; + }; + + let Some(epoch) = canonical_decimal(epoch) else { + return EMPTY; + }; + + let Some(group) = unseal(Epoch(epoch)) else { + return ReadPinList { + sealed: true, + ..EMPTY + }; + }; + + let Ok(inner) = stream::open_bytes(group.conversation(), sealed) else { + return EMPTY; + }; + + let Ok(form) = serde_json::from_slice::(&inner) else { + return EMPTY; + }; + + if form.entries.len() > PIN_MAX_ENTRIES { + return EMPTY; + } + + ReadPinList { + entries: form.entries, + sealed: false, + } +} + +pub fn killed_by(pin: &VerifiedPin, delete: &ChatRumor) -> bool { + delete.author == pin.author + && matches!(&delete.action, ChatAction::Delete { target, .. } if *target == pin.rumor_id) +} + +#[cfg(test)] +mod tests { + use nostr::nips::nip44::v2::{self, ConversationKey}; + + use super::*; + use crate::chat::{ChatRumor, build_delete, build_edit, build_message, open, seal_rumor}; + use crate::derive::channel_group_key; + + const AT_MS: u64 = 1_700_000_000_000; + const SECRET: [u8; 32] = [0x21u8; 32]; + + fn channel() -> ChannelId { + ChannelId::from_bytes([0xabu8; 32]) + } + + fn group() -> GroupKey { + channel_group_key(&SECRET, &channel(), Epoch(0)).expect("derives") + } + + fn conversation() -> ConversationKey { + *group().conversation() + } + + /// A real message through the production seal/open pipeline, as a pinner sees it. + fn sealed_message(author: &Keys, text: &str, at_ms: u64) -> (OpenedStream, ChatRumor) { + let rumor = build_message( + author.public_key(), + &channel(), + Epoch(0), + text, + None, + at_ms, + None, + ); + let (wrap, _) = seal_rumor(&rumor, &group(), author, false).expect("seals"); + + open(&wrap, &group(), &channel(), Epoch(0)).expect("opens") + } + + fn entry_for(author: &Keys, text: &str) -> (PinEntry, OpenedStream) { + let (opened, _) = sealed_message(author, text, AT_MS); + let entry = build_entry(&opened, &group(), &channel()).expect("builds"); + (entry, opened) + } + + fn some(entries: Vec) -> ReadPinList { + ReadPinList { + entries, + sealed: false, + } + } + + /// The load-bearing primitive: the reproduction must open what nostr's own + /// encryption produced, through the disclosure alone. + #[test] + fn a_disclosure_opens_its_message_and_nothing_else() { + let nonce = [0x5au8; 32]; + let disclosure = + MessageKeys::derive(conversation().as_bytes().try_into().expect("32"), &nonce) + .expect("derives"); + + for text in ["a", "hello world", &"padding boundary ".repeat(40)] { + let raw = v2::encrypt_to_bytes_with_nonce(&conversation(), text.as_bytes(), nonce) + .expect("encrypts"); + let payload = BASE64.encode(&raw); + + assert_eq!(open_payload(&payload, &disclosure).as_deref(), Some(text)); + } + + // Another nonce discloses different keys, which open nothing else. + let other = v2::encrypt_to_bytes_with_nonce(&conversation(), b"second", [0x99u8; 32]) + .expect("encrypts"); + assert!(open_payload(&BASE64.encode(&other), &disclosure).is_none()); + + assert_eq!( + MessageKeys::from_hex(&disclosure.to_hex()), + Some(disclosure) + ); + assert!(MessageKeys::from_hex(&disclosure.to_hex().to_uppercase()).is_none()); + } + + #[test] + fn a_built_entry_proves_its_author_and_cannot_cross_channels() { + let author = Keys::generate(); + let (entry, opened) = entry_for(&author, "pin me"); + let verified = verify_entry(&entry, &channel()).expect("verifies"); + + assert_eq!(verified.author, author.public_key()); + assert_eq!(verified.content, "pin me"); + assert_eq!(verified.rumor_id, opened.rumor_id); + assert_eq!(verified.at_ms, AT_MS); + assert_eq!(verified.epoch, Epoch(0)); + + // A keyholder must not be able to pin channel X's message into Y's list. + let foreign = ChannelId::from_bytes([0xcdu8; 32]); + assert!(verify_entry(&entry, &foreign).is_none()); + + // Tampered keys and a re-signed seal both fail. + let mut bad_keys = entry.clone(); + bad_keys.keys = format!("00{}", &entry.keys[2..]); + assert!(verify_entry(&bad_keys, &channel()).is_none()); + + let mut forged = entry.clone(); + forged.seal.pubkey = Keys::generate().public_key(); + assert!(verify_entry(&forged, &channel()).is_none()); + + // A rumor carrying a claimed id that is not its own is refused. + let plaintext = stream::open_bytes(&conversation(), &opened.seal.content).expect("opens"); + let mut value: serde_json::Value = serde_json::from_slice(&plaintext).expect("json"); + value["id"] = serde_json::Value::String("00".repeat(32)); + + let raw = v2::encrypt_to_bytes_with_nonce( + &conversation(), + value.to_string().as_bytes(), + [0x11u8; 32], + ) + .expect("encrypts"); + let content = BASE64.encode(&raw); + let seal = EventBuilder::new(Kind::Custom(stream::KIND_SEAL_ENCRYPTED), &content) + .custom_created_at(opened.seal.created_at) + .finalize(&author) + .expect("signs"); + + let lying = PinEntry { + keys: disclose_keys(&content, conversation().as_bytes().try_into().expect("32")) + .expect("discloses") + .to_hex(), + seal, + wrap: None, + edit: None, + extra: Extra::default(), + }; + assert!(verify_entry(&lying, &channel()).is_none()); + } + + #[test] + fn a_proven_edit_replaces_the_words_and_a_stranger_cannot_revise() { + let author = Keys::generate(); + let (entry, original) = entry_for(&author, "teh typo"); + + let edit = build_edit( + author.public_key(), + &channel(), + Epoch(0), + original.rumor_id, + "the typo, fixed", + AT_MS + 5_000, + None, + ); + let (wrap, _) = seal_rumor(&edit, &group(), &author, false).expect("seals"); + let (edit_opened, _) = open(&wrap, &group(), &channel(), Epoch(0)).expect("opens"); + + let refreshed = with_proven_edit(&entry, &edit_opened, &group(), &channel()); + let verified = verify_entry(&refreshed, &channel()).expect("verifies"); + assert_eq!(verified.content, "the typo, fixed"); + assert_eq!(verified.edited.expect("edited").at_ms, AT_MS + 5_000); + + // A stranger's edit of the same message never attaches. + let stranger = Keys::generate(); + let hijack = build_edit( + stranger.public_key(), + &channel(), + Epoch(0), + original.rumor_id, + "hijacked", + AT_MS + 6_000, + None, + ); + let (wrap, _) = seal_rumor(&hijack, &group(), &stranger, false).expect("seals"); + let (hijack_opened, _) = open(&wrap, &group(), &channel(), Epoch(0)).expect("opens"); + + let unchanged = with_proven_edit(&entry, &hijack_opened, &group(), &channel()); + assert!(unchanged.edit.is_none()); + } + + #[test] + fn both_list_forms_round_trip_and_obey_their_caps() { + let author = Keys::generate(); + let (entry, _) = entry_for(&author, "hello"); + + let public = + publishable(&some(vec![entry.clone()]), false, &group(), Epoch(0)).expect("publishes"); + let read = read_list(&public, |_| None); + assert!(!read.sealed); + assert_eq!(read.entries.len(), 1); + assert!(verify_entry(&read.entries[0], &channel()).is_some()); + + // A sealed list stays dark without its key, lights with it, and a wrong + // key reads empty rather than panicking. + let at_epoch_4 = channel_group_key(&SECRET, &channel(), Epoch(4)).expect("derives"); + let sealed = publishable(&some(vec![entry.clone()]), true, &at_epoch_4, Epoch(4)) + .expect("publishes"); + + let dark = read_list(&sealed, |_| None); + assert!(dark.sealed && dark.entries.is_empty()); + + let lit = read_list(&sealed, |epoch| { + (epoch == Epoch(4)) + .then(|| channel_group_key(&SECRET, &channel(), Epoch(4)).expect("derives")) + }); + assert!(!lit.sealed); + assert!(verify_entry(&lit.entries[0], &channel()).is_some()); + + assert!(read_list(&sealed, |_| Some(group())).entries.is_empty()); + + // 26 entries: the writer refuses, and a hand-built violating edition + // reads as empty rather than forking the chain. + let many = vec![entry; PIN_MAX_ENTRIES + 1]; + assert_eq!( + publishable(&some(many.clone()), false, &group(), Epoch(0)), + Err(PinError::TooManyEntries) + ); + let violating = serde_json::json!({ "entries": many }).to_string(); + assert!(read_list(&violating, |_| None).entries.is_empty()); + + // Garbage never panics and never reads as a list. + for bad in [ + "", + "not json", + "[]", + "42", + r#"{"entries": 7}"#, + r#"{"epoch":"04","sealed":"y"}"#, + ] { + let read = read_list(bad, |_| None); + assert!(read.entries.is_empty() && !read.sealed, "{bad}"); + } + } + + #[test] + fn a_dark_list_is_never_reformed_and_only_the_author_kills_a_pin() { + let author = Keys::generate(); + let (entry, _) = entry_for(&author, "delete me later"); + + let dark = ReadPinList { + entries: vec![entry.clone()], + sealed: true, + }; + assert_eq!( + publishable(&dark, false, &group(), Epoch(0)), + Err(PinError::Unreadable) + ); + + let verified = verify_entry(&entry, &channel()).expect("verifies"); + + for author_keys in [&author, &Keys::generate()] { + let delete = build_delete( + author_keys.public_key(), + &channel(), + Epoch(0), + verified.rumor_id, + Some(KIND_MESSAGE), + None, + AT_MS + 1_000, + ); + let (wrap, _) = seal_rumor(&delete, &group(), author_keys, false).expect("seals"); + let (_, rumor) = open(&wrap, &group(), &channel(), Epoch(0)).expect("opens"); + + assert_eq!( + killed_by(&verified, &rumor), + author_keys.public_key() == author.public_key() + ); + } + } +} diff --git a/crates/concord/src/rekey.rs b/crates/concord/src/rekey.rs index d16e3b53..fd955b3b 100644 --- a/crates/concord/src/rekey.rs +++ b/crates/concord/src/rekey.rs @@ -21,12 +21,7 @@ use crate::stream::{self, KIND_SEAL_PLAINTEXT, OpenedStream, SealForm, StreamErr use crate::{ChannelId, CommunityId, Epoch, GroupKey, random_32}; pub const KIND_REKEY: u16 = 3303; -/// The send cap. A rekey rides the CORD-01 double envelope, so each blob costs two -/// NIP-44 base64 expansions: 120 blobs measure ~77 KB and a 64 KB relay refuses -/// them, while 80 measure ~55 KB. CORD-06 states 120 — an erratum this reproduces. pub const MAX_REKEY_BLOBS_PER_EVENT: usize = 80; -/// The accept cap stays at the spec's 120, above the send cap, so a chunk minted by -/// another client at the spec limit still parses. pub const MAX_REKEY_BLOBS_RECEIVED: usize = 120; pub const MAX_REKEY_EPOCH: u64 = 1 << 40; @@ -115,8 +110,7 @@ pub enum RekeyScope { } impl RekeyScope { - /// The all-zero sentinel addresses the base; a channel id is random, so it - /// never collides. The value is stamped inside every blob's ciphertext. + /// The all-zero sentinel addresses the base; a channel id is random. pub fn id32(self) -> [u8; 32] { match self { RekeyScope::Channel(channel) => *channel.as_bytes(), @@ -259,9 +253,6 @@ pub fn parse_blob_plaintext( return Err(RekeyError::ControlPairMismatch); } - // A width past 136 is a form this client predates. Refusing it would park - // the member at the old epoch, so the frozen prefix and the appended fields - // that still verify are kept and the rest freezes. return Ok(KeyDelivery { new_key, control_pk: Some(control_pk), @@ -276,9 +267,7 @@ pub fn parse_blob_plaintext( }) } -/// The rekey plane's address for a scope. A standalone channel rotation rides the -/// current root; one forced by a removal rides the prior root beside the base -/// rotation, which is exactly what lets a base-fork loser still open it. +/// The rekey plane's address for a scope. pub fn rekey_group( scope: RekeyScope, addressing_root: &[u8; 32], @@ -324,9 +313,6 @@ pub fn build_blob( }) } -/// The locator is public and authenticates nothing, so it is not gated here: -/// the pairwise decrypt, plus the scope and epoch bound inside the ciphertext, -/// are the whole gate. pub fn open_blob( recipient: &Keys, rotator: &PublicKey, @@ -342,8 +328,6 @@ pub fn open_blob( parse_blob_plaintext(&plaintext, scope, epoch, community_id) } -/// Every blob at my locator. Anyone can publish a blob at mine, since the locator -/// is public, so the caller tries each and adopts the first that opens. pub fn find_my_blobs<'a>( blobs: &'a [RekeyBlob], rotator: &PublicKey, @@ -352,7 +336,6 @@ pub fn find_my_blobs<'a>( epoch: Epoch, ) -> impl Iterator { let wanted = blob_locator(rotator, me, scope, epoch); - blobs.iter().filter(move |blob| blob.locator == wanted) } @@ -362,7 +345,6 @@ fn seal_to( plaintext: &[u8], ) -> Result { let conversation = ConversationKey::derive(secret, recipient).map_err(crypto_error)?; - Ok(stream::seal_bytes(&conversation, plaintext)?) } @@ -379,8 +361,7 @@ pub struct RekeyChunk { pub severed: bool, } -/// The key that groups the chunks of one rotation. Two rotators racing the same -/// epoch, or one rotator over two channels, never alias. +/// The key that groups the chunks of one rotation. pub type RotationKey = ([u8; 32], [u8; 32], u64, [u8; 32]); impl RekeyChunk { @@ -404,8 +385,6 @@ pub struct Rotation { pub blobs: Vec, pub declared: u32, pub held: BTreeSet, - /// OR across chunks: an extension minted without the marker must not launder - /// a severed rotation back into an ordinary one. pub severed: bool, pub citation: Option, } @@ -450,8 +429,6 @@ pub fn collect_rotations(chunks: &[RekeyChunk]) -> Vec { rotation.severed |= chunk.severed; rotation.held.insert(chunk.chunk.0); - // A union, never first-wins: two chunks can claim one index after a - // catch-up, and a recipient dropped from the union reads as removed. for blob in &chunk.blobs { if !rotation.blobs.iter().any(|held| held == blob) { rotation.blobs.push(blob.clone()); @@ -502,9 +479,6 @@ fn continuity( } } -/// The winner among concurrent rotations at one continuity point: the lowest key, -/// adopted only when it strictly lowers a key already held. A settled epoch heals -/// down and never re-forks upward. pub fn fork_winner(held: Option<&[u8; 32]>, candidates: &[[u8; 32]]) -> Option { let (index, winner) = candidates.iter().enumerate().min_by_key(|(_, key)| **key)?; @@ -514,8 +488,6 @@ pub fn fork_winner(held: Option<&[u8; 32]>, candidates: &[[u8; 32]]) -> Option Result { }) } -/// Carries the settled heads across a refounding. The control plane is -/// plaintext-sealed precisely so this preserves the original authors' signatures -/// instead of re-signing a snapshot as the refounder. +/// Carries the settled heads across a refounding. pub fn compact( seals: &[Event], read: &GroupKey, @@ -742,10 +712,6 @@ pub struct DissolvedTombstone { pub owner: PublicKey, } -/// The `eid` commits the community, deliberately diverging from the all-zero -/// placeholder CORD-02 §9 shows: the dissolved address derives from the public -/// `community_id`, so a zero binding lets an owner's genuine tombstone for one of -/// their communities be re-wrapped at another and kill it. pub fn dissolved_tombstone_rumor( owner: PublicKey, community_id: &CommunityId, diff --git a/crates/concord/src/roles.rs b/crates/concord/src/roles.rs index c9b38d32..e9d028b9 100644 --- a/crates/concord/src/roles.rs +++ b/crates/concord/src/roles.rs @@ -1,6 +1,6 @@ use std::collections::{BTreeMap, BTreeSet, HashSet}; -use anyhow::Result; +use anyhow::{Result, bail}; use nostr_sdk::prelude::PublicKey; use serde::{Deserialize, Deserializer, Serialize, Serializer}; @@ -127,6 +127,10 @@ impl Grant { } pub fn to_content(&self) -> Result { + if self.role_ids.len() > MAX_ROLES_PER_MEMBER { + bail!("grant exceeds {MAX_ROLES_PER_MEMBER} roles"); + } + Ok(serde_json::to_string(self)?) } } diff --git a/crates/concord/src/store.rs b/crates/concord/src/store.rs index 131cf315..97bd81b9 100644 --- a/crates/concord/src/store.rs +++ b/crates/concord/src/store.rs @@ -25,11 +25,18 @@ const WRAP_TAG: &str = "e"; const KIND_TAG: &str = "k"; const STATE_PREFIX: &str = "concord/"; +/// CORD-08 §3: an already-expired rumor is refused at ingest, never stored. +/// Returns whether the rumor was kept. pub async fn cache_rumor( database: &dyn NostrDatabase, channel: &ChannelId, opened: &OpenedStream, -) -> Result<()> { +) -> Result { + if chat::expiration_of(&opened.rumor)?.is_some_and(|expiration| expiration <= Timestamp::now()) + { + return Ok(false); + } + let tags = vec![ Tag::identifier(opened.rumor_id), Tag::custom(KIND_TAG, [opened.rumor.kind.to_string()]), @@ -47,7 +54,42 @@ pub async fn cache_rumor( database.save_event(&event).await?; - Ok(()) + Ok(true) +} + +pub async fn purge_expired( + database: &dyn NostrDatabase, + channel: &ChannelId, + now: Timestamp, +) -> Result { + let filter = Filter::new() + .kind(Kind::ApplicationSpecificData) + .custom_tag(MARK_TAG, MARK_VALUE) + .custom_tag(CHANNEL_TAG, channel.to_hex()); + + let mut expired = Vec::new(); + + for event in database.query(filter).await? { + let Ok(rumor) = UnsignedEvent::from_json(&event.content) else { + continue; + }; + + let Ok(Some(expiration)) = chat::expiration_of(&rumor) else { + continue; + }; + + if expiration <= now { + expired.push(event.id); + } + } + + let purged = expired.len(); + + if purged > 0 { + database.delete(Filter::new().ids(expired)).await?; + } + + Ok(purged) } pub async fn query_rumors( @@ -300,8 +342,9 @@ pub async fn backfill( let (fresh, next) = advance(&page, &planes, channel, cursor, limit, &mut seen); for (opened, rumor) in fresh { - cache_rumor(database, channel, &opened).await?; - found.push(rumor); + if cache_rumor(database, channel, &opened).await? { + found.push(rumor); + } } match next { @@ -420,7 +463,15 @@ mod tests { ("after the rekey", &NEXT_SECRET, Epoch(1), base + 2_000), ] { let group = channel_group_key(secret, &channel, epoch).expect("derives"); - let rumor = build_message(author.public_key(), &channel, epoch, content, None, at_ms); + let rumor = build_message( + author.public_key(), + &channel, + epoch, + content, + None, + at_ms, + None, + ); relay.insert(seal_rumor(&rumor, &group, &author, false).expect("seals").0); } @@ -505,4 +556,93 @@ mod tests { assert_eq!(capped[0].content, "second"); }); } + + #[test] + fn an_expired_rumor_is_refused_at_ingest_and_purged_by_the_sweep() { + let database = MemoryDatabase::unbounded(); + let channel = ChannelId::from_bytes([0x77u8; 32]); + let author = Keys::generate(); + let group = channel_group_key(&SECRET, &channel, Epoch(0)).expect("derives"); + let now = Timestamp::now().as_secs(); + + smol::block_on(async { + // A live timer is stored; one that already elapsed is refused at ingest. + assert!( + cache( + &database, + &group, + &channel, + &author, + "live", + Some(3_600), + now + ) + .await + ); + assert!( + !cache( + &database, + &group, + &channel, + &author, + "gone", + Some(1), + now - 120 + ) + .await + ); + + let stored = query_rumors(&database, &channel, None, 10) + .await + .expect("queries"); + assert_eq!(stored.len(), 1); + assert_eq!(stored[0].content, "live"); + + // Hiding is not disappearing: the sweep removes the row itself, + // judged on the rumor's own signed tag. + let purged = purge_expired(&database, &channel, Timestamp::from_secs(now + 7_200)) + .await + .expect("sweeps"); + assert_eq!(purged, 1); + assert!( + query_rumors(&database, &channel, None, 10) + .await + .expect("queries") + .is_empty() + ); + + // An untimed rumor is never swept, whatever the clock says. + assert!(cache(&database, &group, &channel, &author, "timeless", None, now).await); + let purged = purge_expired(&database, &channel, Timestamp::from_secs(now + 86_400)) + .await + .expect("sweeps"); + assert_eq!(purged, 0); + }); + } + + async fn cache( + database: &MemoryDatabase, + group: &GroupKey, + channel: &ChannelId, + author: &Keys, + content: &str, + timer: Option, + at_secs: u64, + ) -> bool { + let rumor = build_message( + author.public_key(), + channel, + Epoch(0), + content, + None, + at_secs * 1_000, + timer, + ); + let (wrap, _) = seal_rumor(&rumor, group, author, false).expect("seals"); + let opened = open_wrap(&wrap, group).expect("opens"); + + cache_rumor(database, channel, &opened) + .await + .expect("caches") + } } diff --git a/crates/concord/src/stream.rs b/crates/concord/src/stream.rs index d5b4b903..931c8cde 100644 --- a/crates/concord/src/stream.rs +++ b/crates/concord/src/stream.rs @@ -185,6 +185,7 @@ pub fn seal_content( } pub fn seal_bytes(conversation: &ConversationKey, plaintext: &[u8]) -> Result { + check_plaintext_cap(plaintext.len())?; Ok(BASE64.encode(&encrypt(conversation, plaintext)?)) } -- 2.54.0 From fe2d956d40d8c13da3a882f0f37b0ca9c106586f Mon Sep 17 00:00:00 2001 From: Ren Amamiya Date: Thu, 17 Sep 2026 11:13:13 +0700 Subject: [PATCH 11/12] clean up --- PLAN.md | 1222 --------------------------------- crates/concord/src/chat.rs | 32 +- crates/concord/src/control.rs | 25 +- crates/concord/src/derive.rs | 54 +- crates/concord/src/edition.rs | 27 +- crates/concord/src/invite.rs | 14 +- crates/concord/src/lib.rs | 15 +- crates/concord/src/list.rs | 22 +- crates/concord/src/pins.rs | 22 +- crates/concord/src/rekey.rs | 4 +- crates/concord/src/store.rs | 3 +- crates/concord/src/stream.rs | 20 +- docs/concord-usage.md | 438 ++++++++++++ 13 files changed, 507 insertions(+), 1391 deletions(-) delete mode 100644 PLAN.md create mode 100644 docs/concord-usage.md diff --git a/PLAN.md b/PLAN.md deleted file mode 100644 index 7f882c02..00000000 --- a/PLAN.md +++ /dev/null @@ -1,1222 +0,0 @@ -# Concord support — backend & API plan - -Concord is an encrypted community/channel protocol over Nostr: shared-key "Private Streams" (CORD-01), communities with a self-certifying owner id and three planes (CORD-02), public/private channels (CORD-03), an owner-rooted signed roster (CORD-04), invites (CORD-05), rekeys/refoundings (CORD-06) and disappearing messages (CORD-08). - -Scope of this plan: **backend + public Rust API**. No views, no widgets, no copy. - -## 1. Scope - -**In scope** - -- New `crates/concord` crate: derivations, stream codec, control/chat/guestbook planes, authority fold, ephemeral state, invites, rekeys, dissolution, storage, sync engine. -- Public API for a future UI: `ConcordRegistry` + `Entity` / `Entity` + events, mirroring the shape of `crates/chat`. -- The minimum surgical edits to existing crates required for coexistence (see §11). - -**Out of scope** - -- Any UI work. -- CORD-07 audio/video. Reserve `23313`, the `concord/voice-*` labels and the `voice` metadata flag so nothing else claims them, and implement nothing. -- Pins (CORD-04 §7) and disappearing messages (CORD-08) land in M8, the last milestone; the design accounted for `vsk 11` and `PIN_MESSAGES` from M3 so neither fold is retrofitted. -- Cross-client interop testing (Vector/Armada/Grimoire). Tracked as follow-up work, not blocking. - -## 2. Sources of truth - -| Doc | What we take from it | -| --- | --- | -| CORD-01 | Stream event shape, seal forms 20013/20014, encoding rules, binding, deletions | -| CORD-02 | `community_id`, `community_root`, `control_root`, epochs, 3 planes, metadata, invites, Community List, dissolution | -| CORD-03 | Channel keying, metadata, message kinds, `channel`/`epoch` binding, threads vs quotes | -| CORD-04 | Editions, `vac`, the roster, permission bits, banlist, the three removals, pins | -| CORD-05 | Bundle, link (naddr + fragment), relay dictionary, Invite List, Registry, Direct Invite | -| CORD-06 | Rekey blobs, chunking, `prevcommit` continuity, Refounding, compaction, races | -| CORD-08 | `message_expiration`, NIP-40 tagging, ingest/purge enforcement, timer notice 1740 | - -Appendix A (derivations) and Appendix B (kinds) of CORD-02 are **frozen**: every labeled byte and every kind number is part of the wire format. Treat both as constants with golden-vector tests. - -Reference implementations for cross-checking behaviour (not for copying code): Vector (`crates/vector-core/src/community/v2/*`), Armada, Grimoire. - -## 3. Reuse map — nostr-sdk APIs we build on - -Verified against the pinned revision (`nostr` 0.45.4 / `nostr-sdk` 0.45.2, git `b230cec`). - -| Concord need | Existing API | -| --- | --- | -| NIP-44 under a raw conversation key | `nostr::nips::nip44::v2::{ConversationKey, encrypt_to_bytes_with_nonce, decrypt_to_bytes}` | -| NIP-44 conversation key for a keypair | `ConversationKey::derive(&SecretKey, &PublicKey)` (self-ECDH for streams) | -| NIP-44 under a signer | `nip44::{encrypt, decrypt}` (already wrapped by `state::UniversalSigner`) | -| Event id recomputation | `EventId::compute(pubkey, created_at, kind, tags, content)`, `UnsignedEvent::compute_id` | -| Event (de)serialization | `Event::{from_json, as_json, verify}`, `UnsignedEvent::from_json` | -| Building events | `EventBuilder::new(..).tags(..).custom_created_at(..)`, `FinalizeEvent::finalize(&Keys)`; `UnsignedEvent::new(..)` for rumors, whose tags are the author's contract and must not be normalized | -| Tags | `Tag::{custom, identifier, public_key, expiration}`, `Tags`, `SingleLetterTag` | -| Kinds | `Kind::GiftWrap` (1059), `Kind::Custom(21059|20013|20014|3308|…)`, `Kind::is_ephemeral` | -| Publish | `Client::send_event(&event).to(relays).ack_policy(AckPolicy::none())` | -| Subscribe | `Client::subscribe(target).with_id(..).close_on(..)`, `SubscribeAutoCloseOptions`, `ReqExitPolicy` | -| Backfill | `Client::fetch_events(target)`, `Client::stream_events(target)` | -| Local persistence | `Client::database()` → `query(Filter)`, `save_event(&Event)` | -| Invite link parsing | `Nip19::from_bech32` → `Nip19::Coordinate(Nip19Coordinate)` | -| Signer abstraction | `state::UniversalSigner` (`AsyncSignEvent` + `AsyncNip44`) | -| Relay auth | `nostr_sdk::{Authenticator, SignerAuthenticator}` | - -**Not needed.** `secp256k1` (use `nostr::SecretKey::from_slice` + `Keys::new`), `base64` (use `data_encoding::BASE64`, already a workspace dep), `bech32` (NIP-19 is in the SDK), any new storage engine (the client's LMDB database is enough), any new HTTP client. - -**Dependencies added so far:** `hkdf = "0.12"` at M0 and `rand = "0.10"` at M1 for the NIP-44 nonce, pinned to the exact instance `nostr` already builds (`default-features = false`, features `std` + `sys_rng`) so `nostr`'s `os-rng` and ours unify on one `rand`/`getrandom`; `sha2` and `data-encoding` were already workspace deps. M8 added the Pin List's two, read off the crate graph rather than chosen: `chacha20 = "0.9"` (the instance nostr's `nip44` builds) and `hmac = "0.12"` (the instance `hkdf` builds) — see §14.8. **No new package has ever been added**, and M8's pair changed `Cargo.lock` only by the two `concord` edges; every addition is a direct-dependency line on something already compiled. - -**Why not depend on Vector's crates.** `vector-core` (published, MIT) holds the only other Rust Concord implementation, in `src/community/v2/*`. It is not reusable as a dependency, and the "reuse their crypto" argument does not hold: - -- **No shared types.** It exact-pins the nostr family (`nostr = "=0.45.1"`, `nostr-sdk = "=0.45.1"`, `nostr-connect = "=0.45.1"`, `nostr-blossom = "=0.45.0"`) with the note that a caret range would let a consumer resolve a mixed set, while we track git master (`b230cec`, 0.45.4 / 0.45.2). A registry 0.45.1 and a git 0.45.4 cannot unify, so a build linking both carries two `nostr` crates whose `Event`/`Keys`/`PublicKey`/`Client` are unrelated types. -- **Not wasm-buildable.** `rusqlite` (bundled C SQLite), `libc`, `rustls`, `reqwest`, `image`, `bip39`, and a `tokio` `net` + `rt-multi-thread` requirement; `VectorCore::init` installs a process-global rustls provider and raises the fd limit. Coop's `web` target is wasm32. -- **It is an application core, not a Concord library.** 80k+ lines over 111 files, built on process-global singletons (`state::STATE`, `MY_SECRET_KEY`, one app-data dir, one live account, `traits::set_event_emitter`) and its own SQLite schema, relay pool and blocking `listen()` loop. Adopting it means handing it the nsec and letting it own the client, the database and the event loop — replacing `state`, `chat` and `person` rather than reusing a component. Its `login` stores raw secret-key bytes in that global vault, so an account whose key lives in a signer cannot drive it. -- **There is no cryptography to share.** Both implementations call the same audited crates — `hkdf`, `sha2`, nostr's secp256k1 keypair, and nostr's NIP-44 v2. Vector's comment on that same dependency is "audited RustCrypto crate rather than a hand-rolled construction". Confirmed in M1 by reading `community/cipher.rs`: it is a ~20-line wrapper that draws an OS nonce, calls `nostr::nip44::v2::encrypt_to_bytes_with_nonce`, and base64s the result — which is precisely what `stream.rs` does. Their `stream.rs` likewise calls `nostr_sdk::prelude::nip44::v2::{decrypt_to_bytes, ConversationKey}` directly. Our `derive.rs` holds the frozen `info` layout and label table, and `stream.rs` the seal/wrap ordering; both are wire format, not primitives. - -So Vector's crates earn their place as an **oracle, not a dependency**: the golden vectors in `derive.rs` are their published data, and their `community/v2/stream.rs` was diffed against our §7 before the codec was written. It agrees on every wire detail, and contributed the `ms` first-wins rule, the Control Plane's no-`ms` rumor shape and the `rewrap_seal` contract. - -## 4. Crate layout - -New crate `crates/concord`, picked up automatically by the `crates/*` workspace member glob. - -``` -crates/concord/ - Cargo.toml - src/lib.rs init, ConcordRegistry, ConcordEvent, signal bus, subscriptions, ingest pipeline - src/derive.rs frozen HKDF / group_key / locators / commitments + golden vectors - src/stream.rs CORD-01: seal + wrap + open (SealForm, OpenedStream), channel/epoch binding - src/edition.rs CORD-04 §1: edition hash, parse, chain fold, floor-aware head selection - src/control.rs control plane: genesis, content types, the control fold, the edition writer - src/roles.rs CORD-04 §2–§4: permissions, roles, grants, banlist, delegation fixpoint - src/guestbook.rs CORD-02 §5: join/leave/kick/snapshot, coalesce, complete memberlist - src/chat.rs CORD-03: channel plane — message/edit/delete/reaction builders + message view - src/invite.rs CORD-05 §1–§3, §6: bundle, link (naddr + fragment), Direct Invite - src/list.rs CORD-02 §8: the Community List — join material, merge, to-self envelope - src/pins.rs CORD-04 §7: the Pin List, the NIP-44 key disclosure, the two content forms - src/rekey.rs CORD-06: blob codec, continuity, refounding, compaction, dissolution - src/store.rs local persistence + opened-rumor cache + history queries -``` - -`Community` and `Channel` GPUI entities live in `src/lib.rs` next to the registry — they are the public surface, not a separate concern. Eleven modules, each with real content; no single-fn files. - -Dependencies: `common`, `state`, `person`, `device`, `settings`, `gpui`, `nostr` (for `nip44` features), `nostr-sdk`, `hkdf`, `sha2`, `data-encoding`, `rand`, `serde`, `serde_json`, `anyhow`, `flume`, `smallvec`, `itertools`, `futures`, `log`, `instant`. Everything under `cfg(not(target_arch = "wasm32"))` follows the `chat`/`state` split so the crate still builds for web. - -Declare only what a milestone actually uses. As of M3 the crate depends on `nostr`, `nostr-sdk`, `hkdf`, `sha2`, `data-encoding`, `rand`, `serde`, `serde_json`, `anyhow` (plus `nostr-memory` and `smol` for tests). `rand` is pinned to the `0.10.2` instance `nostr` already builds and shares its `getrandom`, which the `web` crate already enables `wasm_js` on — so no new package and no new wasm obligation. `serde`/`serde_json` were already in the graph via `nostr`; promoting `serde_json` from dev to main for the metadata content types added no package either, only the `concord → serde` edge. M3 added no dependency of its own. - -## 5. Core types - -```rust -pub struct CommunityId([u8; 32]); // sha256 commitment, never on the wire -pub struct ChannelId([u8; 32]); -pub struct Epoch(pub u64); - -/// A derived stream: signing keypair + the self-ECDH conversation key that -/// encrypts the wraps. Memoised in a bounded process-wide cache. -pub struct GroupKey { keys: Keys, conversation: ConversationKey } -impl GroupKey { - pub fn pk(&self) -> PublicKey; - pub fn pk_hex(&self) -> String; // lowercase; Debug prints only this, never key material - pub fn keys(&self) -> &Keys; - pub fn conversation(&self) -> &ConversationKey; -} - -pub enum SealForm { Encrypted, Plaintext } - -pub struct OpenedStream { - pub rumor_id: EventId, - pub author: PublicKey, // the seal's verified pubkey - pub seal_form: SealForm, - pub seal: Event, // retained: compaction re-wraps plaintext seals verbatim - pub wrapper_id: EventId, - pub at_ms: u64, // created_at * 1000 + ms tag - pub rumor: UnsignedEvent, -} -``` - -Ordering everywhere uses `at_ms`, never `created_at`, and ties break on the lower inner rumor id. - -## 6. Frozen derivations (`derive.rs`) - -Implemented and pinned in `crates/concord/src/derive.rs`. - -```rust -fn build_info(label: &str, id: &[u8; 32], epoch: Option) -> Vec; // label ‖ 0x00 ‖ id[32] ‖ epoch_be[8]? -fn hkdf32(ikm: &[u8], info: &[u8]) -> [u8; 32]; // HKDF-SHA256, zero-length salt, L = 32 -fn hkdf_to_secret_key(ikm: &[u8], info: &[u8]) -> Result; // A.3 scalar_normalize, counter from 0 - -fn group_key(label: &str, secret: &[u8], id: &[u8; 32], epoch: Option) -> Result; - -pub fn channel_group_key(secret: &[u8; 32], channel: &ChannelId, epoch: Epoch) -> Result; -pub fn control_group_key(root: &[u8; 32], id: &CommunityId, epoch: Epoch) -> Result; // read key -pub fn control_signer_group_key(root: &[u8; 32], id: &CommunityId, epoch: Epoch) -> Result; // write key -pub fn guestbook_group_key(root: &[u8; 32], id: &CommunityId, epoch: Epoch) -> Result; -pub fn channel_rekey_group_key(root: &[u8; 32], channel: &ChannelId, epoch: Epoch) -> Result; -pub fn base_rekey_group_key(root: &[u8; 32], id: &CommunityId, epoch: Epoch) -> Result; -pub fn dissolved_group_key(id: &CommunityId) -> Result; // no epoch field - -pub fn community_id_of(owner_xonly: &[u8; 32], owner_salt: &[u8; 32]) -> CommunityId; // plain SHA-256 -pub fn verify_community_id(id: &CommunityId, owner_xonly: &[u8; 32], owner_salt: &[u8; 32]) -> bool; -pub fn epoch_key_commitment(epoch: Epoch, key: &[u8; 32]) -> [u8; 32]; // plain SHA-256 -pub fn grant_locator(id: &CommunityId, member: &[u8; 32]) -> [u8; 32]; -pub fn banlist_locator(id: &CommunityId) -> [u8; 32]; -pub fn pins_locator(id: &CommunityId, channel: &ChannelId) -> [u8; 32]; -pub fn invite_links_locator(id: &CommunityId, creator: &[u8; 32]) -> [u8; 32]; -pub fn recipient_locator(rotator: &[u8; 32], recipient: &[u8; 32], scope: &[u8; 32], epoch: Epoch) -> [u8; 32]; -pub fn invite_bundle_key(token: &[u8; 16]) -> [u8; 32]; // raw hkdf32 output; used as a NIP-44 conversation key -pub fn clear_memo(); // drop memoised keys on signer change -``` - -Appendix A.6, as implemented — `ikm` / `id` / `epoch`. The id is *always* present (all-zeroes where a label has no meaningful one); the epoch is the only omittable field. - -| Label | ikm | id | epoch | -| --- | --- | --- | --- | -| `concord/channel` | channel key or `community_root` | `channel_id` | yes | -| `concord/control` | `community_root` | `community_id` | yes | -| `concord/control-signer` | `control_root` | `community_id` | yes | -| `concord/rekey-pseudonym` | prior `community_root` | `channel_id` | new epoch | -| `concord/base-rekey-pseudonym` | prior `community_root` | `community_id` | new epoch | -| `concord/recipient-pseudonym` | `rotator_xonly ‖ recipient_xonly` (64 B) | scope id | new epoch | -| `concord/guestbook` | `community_root` | `community_id` | yes | -| `concord/dissolved` | `community_id` | zeroes | — | -| `concord/grant` | `community_id` | member x-only | — | -| `concord/banlist` | `community_id` | zeroes | — | -| `concord/pins` | `community_id` | `channel_id` | — | -| `concord/invite-links` | `community_id` | creator x-only | — | -| `concord/invite-key` | token (16 B) | zeroes | — | - -The CORD-07 `concord/voice-*` labels and the retired `concord/invite-locator` / `concord/invite-signer` are reserved and listed here only: they are underived, and the table stays append-only. Every label a derivation does use has a distinct pinned output, so a duplicated label cannot pass the vectors. - -Rules that must be enforced by construction, not by convention: - -- Hex is lowercase everywhere; pubkeys are x-only hex, never bech32; tag numbers are decimal with no leading zeros (`"4"`, never `04`/`+4`). -- The epoch field is *omitted*, not zeroed, for labels with no epoch; a test asserts `dissolved_group_key` differs from the same derivation with `Some(0)`. -- `scalar_normalize` retries by appending a counter byte to the same `info`, starting at `0`, and reports exhaustion instead of panicking — so plane keys return `Result`. -- Group keys are memoised by a digest of their inputs, so no deriving secret is a map key, bounded at 1024 entries. - -**Golden vectors.** `derive.rs` pins all 18 published vectors (the seed and `pk` for channel, control, control-signer and guestbook; both keyed labels at epoch `0` and at `0x0102030405060708`; both rekey labels at epoch 1; dissolved; all four locators; the invite key; the community id; the epoch-key commitment), cross-checked against an independent Python implementation (RFC 5869 HKDF plus pure-integer secp256k1) before being frozen. One vector is missing upstream — `pins_locator` — so we mint it from our own implementation and pin it, flagged in the test as self-referential. Changing any pinned value means the wire format changed. - -## 7. Stream codec (`stream.rs`) — implemented in M1 - -```rust -pub const KIND_WRAP: u16 = 1059; -pub const KIND_WRAP_EPHEMERAL: u16 = 21059; -pub const KIND_SEAL_ENCRYPTED: u16 = 20013; -pub const KIND_SEAL_PLAINTEXT: u16 = 20014; -pub const NIP44_MAX_PLAINTEXT: usize = 65_535; - -pub enum SealForm { Encrypted, Plaintext } - -pub struct OpenedStream { - pub rumor_id: EventId, - pub author: PublicKey, - pub seal_form: SealForm, - pub seal: Event, - pub wrapper_id: EventId, - pub at_ms: u64, - pub rumor: UnsignedEvent, -} - -pub fn split_ms(at_ms: u64) -> (u64, u16); -pub fn resolve_ms_strict(rumor: &UnsignedEvent) -> Result; - -pub fn seal_content(rumor: &UnsignedEvent, form: SealForm, group: &GroupKey) -> Result; -pub fn build_seal(rumor: &UnsignedEvent, form: SealForm, group: &GroupKey, author: &Keys) -> Result; -pub fn wrap_seal(seal: &Event, group: &GroupKey, wrap_kind: u16, at: Timestamp, extra: &[Tag]) -> Result<(Event, Keys), StreamError>; -pub fn rewrap_seal(seal: &Event, new_group: &GroupKey, at: Timestamp) -> Result<(Event, Keys), StreamError>; - -pub fn open_wrap(wrap: &Event, group: &GroupKey) -> Result; -pub fn open_wrap_at(wrap: &Event, address: &PublicKey, conversation: &ConversationKey, verify_wrap_signature: bool) -> Result; - -pub fn build_rumor_ms(kind: u16, author: PublicKey, content: &str, tags: Vec, at_ms: u64) -> UnsignedEvent; -pub fn build_rumor_secs(kind: u16, author: PublicKey, content: &str, tags: Vec, at_secs: u64) -> UnsignedEvent; - -pub fn channel_binding_tags(channel: &ChannelId, epoch: Epoch) -> Vec; -pub fn check_channel_binding(rumor: &UnsignedEvent, channel: &ChannelId, epoch: Epoch) -> Result<(), StreamError>; -``` - -`StreamError` is a typed enum, not `anyhow`: the caller has to tell a drop from a fatal, and M1's acceptance criterion is that rejections happen in the documented order. - -Refinements against the draft this plan opened with, decided after diffing Vector's `crates/vector-core/src/community/v2/stream.rs` (Concord has no crate of its own there, and `envelope.rs` does not exist): - -- `build_rumor` became `build_rumor_ms` plus `build_rumor_secs`. The Control Plane edition carries **no** `ms` tag, because editions fold by version, not by time. -- `rewrap_seal` was added to the codec. Without it the plaintext-seal carry-forward has no expression, and M7's compaction is its only caller. -- NIP-44 is reached through `nostr`'s own `nip44::v2::{encrypt_to_bytes_with_nonce, decrypt_to_bytes, ConversationKey}`, with a fresh OS nonce per message and `data_encoding::BASE64` for carriage. This is exactly what Vector does; there is no cryptography of theirs to reuse. - -Design points that are easy to get wrong: - -- The wrap is signed by the **stream key** with a random ephemeral `p` tag — NIP-59 reversed. `extra` is how the caller mirrors a NIP-40 expiration onto the wrap. -- The seal is signed by the **real author** and carries `created_at` equal to the rumor's. It is never published bare. -- Control plane **must** use the plaintext seal; chat, guestbook and rekey planes **must** use the encrypted one. Each plane asserts its own form at both ends. -- The control plane is a write-restricted stream: the wrap key derives from `control_root` while the content is encrypted under the `community_root`-derived conversation key. `open_wrap_at` takes the two halves separately for this reason. -- Open order: kind → address match → wrap signature (only when `verify_wrap_signature`) → NIP-44 open → seal kind → seal signature → rumor parse → `rumor.pubkey == seal.pubkey` → recompute the rumor id and reject a mismatch → strict `ms` resolve. -- Enforce the 65 535-byte NIP-44 cap at every nesting layer before publishing. -- Do not verify the wrap signature for ordinary planes: every reader holds the group key, so the signature proves nothing. It matters only for the restricted control plane and for rekeys. -- The ephemeral wrap keypair is returned to the caller so a client may NIP-09-scrub its own wrap later. -- **A duplicate `ms` tag takes the first value; it is not rejected.** Rejecting made Vector and Armada disagree on whether the event *exists*, and because `ms` orders messages that divergence reached membership. `ms` is the publisher's own value, so conceding a second tag grants an attacker no reach a single one did not. A present-but-valueless `ms`, or one that is not a lone canonical decimal in `0..=999`, is `BadMs` and the event is dropped, never clamped — `u64::from_str` alone would accept a leading `+`, a second encoding a strict peer rejects, so the digit check comes first. -- A binding tag that names the same key twice is rejected outright, since first-match would then be the reader's choice rather than the author's; a valueless tag counts as absent, so a true absence reports `MissingTag`. - -## 8. Planes, state and folds - -### 8.1 Editions, authority and the control fold (`edition.rs`, `roles.rs`) - -```rust -pub const EDITION_LABEL: &[u8] = b"vector-community/v1/edition"; // frozen, cross-client (27 bytes) - -// sha256( u64be(len(label)) ‖ label ‖ entity[32] ‖ u64be(version) -// ‖ flag[1] ‖ prev[32] ‖ u64be(len(content)) ‖ content ) -// `prev` is always 33 bytes: 0x01 ‖ hash, or 0x00 ‖ zeroes when absent. -// The hash commits to no actor: identity enters only via the rumor id. -pub fn edition_hash(entity: &[u8; 32], version: u64, prev: Option<&[u8; 32]>, content: &[u8]) -> [u8; 32]; -pub struct ParsedEdition { author: PublicKey, subkind: String, entity: [u8; 32], version: u64, - prev: Option<[u8; 32]>, citation: Option, - content: String, self_hash: [u8; 32], rumor_id: EventId }; -pub fn parse_edition(rumor: &UnsignedEvent) -> Result; - -pub struct FoldResult { pub head: Option, pub gap: bool, pub anchored: bool } -pub fn fold(editions: &[EditionMeta], floor: u64, floor_hash: Option<&[u8; 32]>) -> FoldResult; -pub fn bootstrap_head(editions: &[EditionMeta]) -> Option; // highest, contiguity ignored - -// One entity's committed head, and the refuse-downgrade floor a later fold is judged -// against. -pub struct EntityHead { entity: [u8; 32], version: u64, self_hash: [u8; 32], rumor_id: EventId } -pub type Floors = BTreeMap<[u8; 32], EntityHead>; -pub struct HeadSelection { pub head: Option, pub gap: bool } -pub fn fold_head(editions: &[EditionMeta], floor: Option<&EntityHead>) -> HeadSelection; -``` - -- Tag grammar: `["vsk", sub]`, `["eid", hex32]`, `["ev", decimal]`, `["ep", hex32]`, `["vac", eid, version, hash]`. Duplicates of any of the five reject the edition; `ev` must pass a decimal check before parsing. A version of `0` parses and then reads as a gap — the rule lives in the fold, not the parser. -- **Versions start at 1, not 0** (CORD-04 §1: "climbs from 1"). Genesis is `(version 1, prev None)` for both entities. -- **The edition hash is not the signature.** The actor's Schnorr signature covers the kind-20014 plaintext seal; `edition_hash` is a separate SHA-256 used only for chaining (`ep`, `vac`). `content` is the rumor's content string byte-verbatim, never re-serialized, which is what lets compaction re-wrap a head and preserve its hash. -- The domain label is `vector-community/v1/edition`, not a `concord/…` label. Inconsistent with Appendix A.6, frozen anyway — do not "fix" it. -- Tie-break at equal version is the lower **inner rumor id** (the kind-3308 rumor), never the outer wrap id and never `created_at`. Only one of the two implementations that must agree applies to a wrap, so the inner id is the only stable choice. -- `gap` is a safety signal: a tracking client (already holds the floor) fails closed for that entity and refetches; a bootstrapping client (floor 0) may accept the highest authority-verified head, which is what makes compaction re-wrapping work. `bootstrap_head` therefore takes no floor: it *is* the floor-zero path. `fold_head` is the composition — floor 0 takes `bootstrap_head`; under a held floor the chain-anchored head wins and any upper gap is reported; everything below the floor is a stale relay, not a gap; and a head detached from the floor converges a same-version fork to its lower rumor id when that is genuinely earlier than what we hold, else fails closed as withholding. -- **Owner anchoring is not in the fold.** `fold` is a pure function of chain shape; authority is a pre-filter the caller applies to the candidate set before folding. `community_id` proves the owner, and `is_authorized` short-circuits `owner == actor`, so the owner needs no Grant entity at all. -- Entity coordinates are `vsk 0` → `community_id`, `1` → `role_id`, `2` → `channel_id`, `3` → `grant_locator`, `4` → `banlist_locator`, `8` → `invite_links_locator`, `11` → `pins_locator`. `5` is reserved, `6`/`9` belong to the 33301 invite marker, `7` is retired. All derive from `community_id` only, so a refounding re-wraps heads verbatim. -- The Control Plane is **plaintext-seal only**. A 20013-encrypted control edition is rejected, because compaction re-wraps a signed plaintext seal byte-verbatim into the new epoch; accepting an encrypted one would let a later compaction fork the chain. -- **Genesis is exactly two owner-signed editions** — community metadata (`vsk 0`, `eid = community_id`) and one public `#general` channel (`vsk 2`, fresh random `channel_id`) — at epoch 0, version 1, no `ep`, no `vac`. No default roles, no scaffolding, and no Grant for the owner. Secrets minted: `owner_salt`, `community_root`, `control_root` (deliberately not derived from `community_id`). - -```rust -// CORD-04 §3, frozen. 1<<7 was MANAGE_INVITES and is burned, never reassigned. -// MANAGE_ROLES 1<<0 · MANAGE_CHANNELS 1<<1 · MANAGE_METADATA 1<<2 · KICK 1<<3 · -// BAN 1<<4 · MANAGE_MESSAGES 1<<5 · CREATE_INVITE 1<<6 · VIEW_AUDIT_LOG 1<<8 · -// MENTION_EVERYONE 1<<9 · PIN_MESSAGES 1<<11 · reserved: MANAGE_EMOJI 1<<10, MANAGE_EVENTS 1<<12 -pub struct Permissions(pub u64); -impl Permissions { - pub const STAFF_MASK: u64; // MANAGE_ROLES|MANAGE_CHANNELS|MANAGE_METADATA|BAN|CREATE_INVITE|PIN_MESSAGES - pub fn contains(self, bits: u64) -> bool; - pub fn union(self, other: Self) -> Self; - pub fn is_staff(self) -> bool; -} - -pub enum RoleScope { Server, Channel(ChannelId) } // {"kind":"server"} / {"kind":"channel","channel_id":…} -pub struct Role { role_id: RoleId, name: String, position: u32, permissions: Permissions, - scope: RoleScope, color: u32, extra: Extra } -pub struct Grant { member: PublicKey, role_ids: Vec, control_wrap: Option, extra: Extra } - -pub struct CommunityRoles { roles: BTreeMap, grants: BTreeMap } -impl CommunityRoles { - pub fn role(&self, role_id: &RoleId) -> Option<&Role>; - pub fn roles_of(&self, member: &PublicKey) -> impl Iterator; - pub fn effective_permissions(&self, member: &PublicKey) -> Permissions; // union of granted role bits - pub fn has_permission(&self, member: &PublicKey, bits: u64) -> bool; - pub fn highest_position(&self, member: &PublicKey) -> Option; // lowest position they hold - pub fn is_authorized(&self, actor, owner, permission: u64) -> bool; // owner == actor → true - pub fn outranks(&self, actor, owner, target_position: u32) -> bool; // strict `<` - pub fn can_act_on_position(&self, actor, owner, target_position: u32, permission: u64) -> bool; - pub fn can_act_on_member(&self, actor, owner, target: &PublicKey, permission: u64) -> bool; - pub fn is_staff(&self, member, owner) -> bool; -} - -// The delegation fixpoint. Content is parsed once, up front: the fixpoint revisits -// every candidate on each pass. -pub enum AuthorityContent { Role(Role), Grant(Grant), Banlist(Vec) } -pub struct AuthorityEdition { entity: [u8; 32], meta: EditionMeta, author: PublicKey, - citation: Option, content: AuthorityContent } -impl AuthorityEdition { - pub fn parse(edition: &ParsedEdition, community_id: &CommunityId) -> Option; -} -pub struct Roster { roles: CommunityRoles, banned: BTreeSet, floors: Floors, gapped: bool } -pub fn fold_roster(owner, community_id, editions: &[AuthorityEdition], floors: &Floors, - held_bans: &BTreeSet) -> Roster; -pub fn citation_ok(owner, community_id, author, citation: Option<&AuthorityCitation>, - floors: &Floors) -> bool; -``` - -Authority rules as implemented: - -- The owner is position 0, proven by `community_id`, supreme, unremovable, and **not a Role**: no Role may claim position 0, and every gate short-circuits `owner == actor`. The owner therefore needs no Grant and cites nothing. -- A member's rank is the **lowest** position among their Roles; a roleless member sits at `u32::MAX`. Two Roles may share a position (peers, neither acts on the other); display tie-breaks on the lower `role_id`. -- The actor must hold the required bit **and strictly outrank** the target. Equal cannot act on equal. -- `AuthorityEdition::parse` drops, rather than repairs: a `role_id` that is not its own coordinate, a `position` of 0, a Grant whose `member` does not hash to its entity, a `vsk 4` at a coordinate that is not this community's banlist locator, malformed JSON, and any `vsk` this type does not own. A Grant's `role_ids` truncate at 64 on read. -- **Refuse-downgrade**: an edition below the persisted floor for its entity is never a candidate. -- The fold is a **Jacobi fixed point** — authority propagates one delegation level per pass, bounded by `2 × (entities) + 8`. Convergence compares the roster only, not the heads. Cross-pass state is exactly the accepted roster plus its heads, and `citation_ok` reads the *previous* pass's heads, so the first pass sees none. -- **Roles** replay each entity's versions **ascending**, one winner per version group, `admissible` collecting the winners that pass. Gates, in order: not banned; `can_act_on_position(author, owner, position, MANAGE_ROLES)`; if a predecessor was admitted, the same call against *its* position; then the citation. The highest admissible version wins. Replaying ascending is what makes the second gate work: without it an admin at position 5 republishes a position-1 role at position 9, every check passes since 9 is beneath them, and a role that outranked them ends up beneath them along with everyone holding it. -- **Grants** take no version-group replay: the first candidate in vector order clearing every gate wins. Role references resolve **partially** — the resolvable subset is carried and the rest fold in on a later pass — because all-or-nothing resolution deadlocks the ordinary growth path (an admin creates a role, the owner grants it to them, and neither can go first, collapsing the entire roster including the owner's own grants). The final gate ranks every resolved position *and* the member. -- A **citation** that cannot be resolved parks the edition; a missing one is tolerated only where the rank gates carry the weight. For a Role that is everywhere. For a Grant it is not: a revoke names no position, so its rank test is vacuous — hence an uncited Grant may add authority but **never remove** it. -- The **banlist** is folded after a preliminary roster, since a ban only exists once someone authorized to place it does. Its head is the highest edition whose author currently holds `BAN` and does not already sit in the held banlist; each entry is kept only if that author strictly outranks the target, and the list caps at 500. **Withholding retains the held list** rather than un-banning nobody on a relay's word. The final roster is then re-folded with the banned set excluded, so a banned admin loses their authority in the same pass. -- A staff-making Grant carries `control_wrap`, a NIP-44 pairwise ciphertext of `epoch_be[8] ‖ control_root[32]`, adopted **only if it derives to the `control_pk` the member already holds** for the named epoch. Delivery, never authority. -- Caps: **100 Roles per community**, by the 100 lowest `role_id`, applied *after* authorization so forged low ids cannot evict a real role; the grants then shed the dropped ids. **64 Roles per member**, at parse. **500 banlist entries**, at fold. - -**Two deliberate divergences from the reference implementation** (Vector is an oracle, not a specification): - -1. **The role fork winner.** Vector's role branch walks version groups with `.iter().rev()`, taking each group's *highest* inner id, while its own adjacent comment says forks break on the lowest and the rest of its codebase (`fold_head`, `version::fold`, its invite-registry test) does use the lowest. No test in Vector pins the branch. We implement **lowest inner id**, per CORD-04 §1 — and our tests pin it. -2. **Banlist candidates must be `vsk 4`.** Vector collects banlist candidates from every edition sitting at the banlist locator regardless of `vsk`, so a `vsk 1` forged there can win the banlist head, parse to an empty list, and clear the ban. We require `vsk::BANLIST` for a candidate at all. - -One reference limitation we **reproduce and do not fix** (recorded here rather than silently diverging): the grant rank gate reads the *previous* pass's roster, so a mid-rank `MANAGE_ROLES` holder who cites a real, folded grant of their own can revoke a higher-ranked member whose authority is still propagating. Fixing it means resolving a Grant's target rank against the same pass, which changes the fixpoint's convergence argument. Revisit only with a spec amendment. - -### 8.2 Communities, channels, metadata - -`CommunityMetadata` carries `name` (≤ 64 bytes), `description` (≤ 10 000 bytes), `relays` (truncated on read and write to 5), `icon` and `banner` as encrypted-blob pointers (`{url, key, nonce, hash}`), the optional `custom` object, and `message_expiration` (CORD-08's timer, in seconds, §8.8). `ChannelMetadata` carries `name`, `private`, optional `voice`, `deleted`, optional `custom`. Every content struct carries `#[serde(flatten)] extra`, so a field one client does not model still round-trips and a rename by an older client cannot wipe another client's `custom` keys. - -The Control Plane's whole projection is one call: - -```rust -pub struct ControlFold { - pub roles: CommunityRoles, - pub banned: BTreeSet, - pub community: Option, - pub channels: BTreeMap, - pub registries: BTreeMap>, // vsk 8, keyed by creator - pub pins: BTreeMap<[u8; 32], String>, // vsk 11 content, keyed by locator (§8.8) - pub floors: Floors, - pub gapped: bool, -} -pub fn fold_control(owner: &PublicKey, community_id: &CommunityId, editions: &[ParsedEdition], - floors: &Floors, held_bans: &BTreeSet) -> ControlFold; -impl ControlFold { pub fn is_public(&self) -> bool; - pub fn pin_content(&self, community_id, channel) -> Option<&str>; } -``` - -- The roster is folded first, and `vsk 0` / `vsk 2` are then judged against it: the head of each entity is the highest edition whose author *currently* holds `MANAGE_METADATA` / `MANAGE_CHANNELS`, is not banned, and either is the owner or cites their own folded Grant. Pre-filtering before the chain fold is what stops a demoted admin's later, higher-version edition from being the head. -- A `vsk 2` whose entity is the community's own id is excluded, and a `vsk 0` at any other coordinate with it: the floor row keys on the entity alone, so the two would otherwise share and corrupt one chain. -- `None` means "this client saw no authorized edition", never "the value is gone": a caller keeps what it holds rather than walking the community backwards. That is also how a withheld or downgraded entity reads. -- A `deleted` channel is reported as metadata with `deleted: true`; the policy of dropping it belongs to the store. - -Writes go through one primitive, so every edition names the head it supersedes and a client cannot silently fork a chain it cannot see: - -```rust -pub struct Edition<'a> { subkind: &'a str, entity: [u8; 32], content: &'a str, - head: Option<&'a EntityHead>, citation: Option } -pub struct ControlWriter { pub author: PublicKey, pub read: GroupKey, pub signer: GroupKey } -impl ControlWriter { - pub fn publish(&self, keys: &Keys, edition: Edition<'_>, at_secs: u64) -> Result<(Event, EntityHead)>; - pub fn set_community_metadata(&self, keys, community_id, metadata, head, - citation: Option, at_secs) -> Result<(Event, EntityHead)>; - pub fn set_channel_metadata(&self, keys, channel, metadata, head, citation, at_secs) -> Result<(Event, EntityHead)>; - pub fn set_role(&self, keys, role: &Role, head, citation, at_secs) -> Result<(Event, EntityHead)>; - pub fn set_grant(&self, keys, community_id, grant: &Grant, head, citation, at_secs) -> Result<(Event, EntityHead)>; - pub fn set_banlist(&self, keys, community_id, banned: &BTreeSet, head, citation, at_secs) - -> Result<(Event, EntityHead)>; - pub fn set_registry(&self, keys, community_id, creator, links: &[PublicKey], head, citation, at_secs) - -> Result<(Event, EntityHead)>; - pub fn set_pin_list(&self, keys, community_id, channel, content: &str, head, citation, at_secs) - -> Result<(Event, EntityHead)>; -} -``` - -`keys` is the acting member's own signer: the seal carries their signature, while the wrap is signed by the plane's published `control_pk`. **Every wrapper takes the citation** (M5): a delegated admin edits metadata, roles, grants and the banlist only under a `vac`, so a wrapper that hardcoded `None` would be an owner-only API. Only `publish` is usable without one. A remote signer (NIP-46) is not yet plumbed — every entry point takes `&Keys`, not a `NostrSigner`. - -Channel keying follows CORD-03 §1: a public channel derives from `community_root` at the base epoch, a private one from its own random key at its own epoch. Public→private is a rekey at the next channel epoch (monotonic, never reset); private→public derives from the base again and the prior private history stays unreadable to later joiners. The store applies only the public declaration and the deletion: the public-to-private flip is ignored until the convert flow (key mint plus cursor rebase) lands, and a channel this client holds no key for is not added at all — it arrives with the invite that carries the key. - -### 8.3 Guestbook and member list (`guestbook.rs`) — implemented in M5 - -```rust -pub const KIND_JOIN_LEAVE: u16 = 3306; -pub const KIND_KICK: u16 = 3309; -pub const KIND_SNAPSHOT: u16 = 3312; -pub const MAX_SNAPSHOT_CHUNK: usize = 400; -pub const MAX_FUTURE_SKEW_MS: u64 = 60 * 60 * 1000; - -pub enum GuestbookEntry { - Join { member: PublicKey, at_ms: u64, invited_by: Option<(String, String)> }, - Leave { member: PublicKey, at_ms: u64 }, - Kick { actor: PublicKey, target: PublicKey, at_ms: u64, citation: Option }, - Snapshot { refounder: PublicKey, members: Vec, snapshot_id: [u8; 32], chunk: (u32, u32), at_ms: u64 }, -} -pub struct GuestbookRumor { pub id: EventId, pub author: PublicKey, pub kind: Kind, pub at_ms: u64, - pub entry: GuestbookEntry } -pub enum MemberState { - Joined { at_ms: u64, invited_by: Option<(String, String)> }, - Left { at_ms: u64 }, - Kicked { at_ms: u64, actor: PublicKey }, -} - -pub fn build_join(member: PublicKey, invited_by: Option<(&str, &str)>, at_ms: u64) -> UnsignedEvent; -pub fn build_leave(member: PublicKey, at_ms: u64) -> UnsignedEvent; -pub fn build_kick(actor, target: &PublicKey, citation: Option<&AuthorityCitation>, at_ms) -> UnsignedEvent; -pub fn build_snapshot_chunks(refounder, members: &[PublicKey], snapshot_id: [u8; 32], at_ms) - -> Vec; -pub fn seal_rumor(rumor: &UnsignedEvent, group: &GroupKey, author: &Keys) - -> Result<(Event, Keys), GuestbookError>; -pub fn open(wrap: &Event, group: &GroupKey) - -> Result<(OpenedStream, GuestbookRumor), GuestbookError>; - -pub fn coalesce(rumors: &[GuestbookRumor], now_ms: u64, snapshot_authority: Option<&PublicKey>, - can_kick: impl Fn(&PublicKey, &PublicKey, Option<&AuthorityCitation>) -> bool) - -> BTreeMap; - -pub fn complete_memberlist(coalesced: &BTreeMap, - observed: &BTreeMap, // author → newest ms published - granted: &BTreeSet, - banned: &BTreeSet, banned_at: &BTreeMap) - -> BTreeSet; -``` - -- The plane is community-wide, and its key already carries the epoch, so a guestbook rumor binds no - `channel`/`epoch` tags and `GuestbookRumor` carries neither. Coalescing spans every held epoch, which - is what lets a snapshot bridge a Refounding. -- Entries dated more than an hour ahead of local time are dropped. That is a coalesce-time check (`now_ms` - is an argument, so it is testable without a clock); the `ms` range is not — `open` inherits - `open_wrap`'s strict resolution, so an `ms` outside `0..999` drops the entry at parse instead. -- Coalesce is per npub, one final state each, by millisecond time, ties on the lower inner rumor id. -- A Join or Leave is self-signed by construction: `member` comes from the rumor's own author, so there is - no wire field that could name somebody else. -- A Kick counts only where `can_kick(actor, target, citation)` admits it — the roster's `KICK` plus a - strictly higher rank plus a resolvable citation, composed by the caller. A Snapshot counts only from the - npub whose Refounding minted the epoch. There is deliberately no owner fallback. -- `invited_by` is an echo of the optional `invite` tag, never authority, so a malformed or repeated tag - costs the label and not the member's own word. -- `build_snapshot_chunks` is the only snapshot builder: 400 members per event, every chunk sharing one id - and one timestamp. A chunk is independently useful, so `coalesce` seeds each chunk's members at that - chunk's own time and never waits for its siblings. -- The member list is `coalesced Joined ∪ observed authors ∪ Grant holders − banlist`. `granted` is the one - addition to the spec's literal formula: a Grant recipient holds keys, so they are a member even with no - Join and no published activity. It counts as an *unjdated* positive, so any dated Leave, Kick or Ban - beats it. Observation counts forward only: an author re-enters on activity newer than their latest - Leave, Kick or Ban. -- `banned_at` is caller-supplied, and an entry missing from it excludes its npub outright — an empty map - therefore means "every ban is terminal", which is the safe reading. The fold does not yet expose the - banlist head's timestamp, so that plumbing belongs with the registry rather than here. -- The earlier sketch's `refound: Option<&Refound>` argument is dropped: nothing mints a Refound until M7, - and a parameter no caller can fill is a guess. It returns with the refounding that produces one. - -### 8.4 Chat plane (`chat.rs`) — implemented in M4 - -Kinds (CORD-02 Appendix B): `9` message, `1111` NIP-22 comment, `7` NIP-25 reaction, -`5` NIP-09 delete, `3302` edit, `1740` timer notice (M8), `3310` WebXDC peer signal, -`23311` ephemeral typing. - -```rust -pub struct ChatRumor { id, author, kind, channel, epoch, at_ms, content, - expiration: Option, action: ChatAction } -pub enum ChatAction { - Message { reply_to: Option, thread_root: Option }, - Reaction { target: EventId, emoji: String }, - Edit { target: EventId, content: String }, - Delete { target: EventId, target_kind: Option, citation: Option }, - Typing, - TimerNotice { seconds: u64 }, - Opaque, -} -pub struct ReplyRef { id: EventId, author: Option } -pub struct Target { reply: ReplyRef, kind: u16 } // the wire commits the target's kind - -pub fn build_message(author, channel, epoch, content, quote: Option<&ReplyRef>, at_ms, timer: Option) -> UnsignedEvent; -pub fn build_comment(author, channel, epoch, content, parent: &Target, root: Option<&Target>, at_ms, timer) -> UnsignedEvent; -pub fn build_reaction(author, channel, epoch, target: &Target, emoji: &str, at_ms, timer) -> UnsignedEvent; -pub fn build_edit(author, channel, epoch, target: EventId, content: &str, at_ms, timer) -> UnsignedEvent; -pub fn build_delete(author, channel, epoch, target: EventId, target_kind: Option, - citation: Option<&AuthorityCitation>, at_ms) -> UnsignedEvent; -pub fn build_timer_notice(author, channel, epoch, seconds: u64, at_ms) -> UnsignedEvent; -pub fn build_typing(author, channel, epoch, at_ms) -> UnsignedEvent; - -pub fn seal_rumor(rumor: &UnsignedEvent, group: &GroupKey, author: &Keys, ephemeral: bool) - -> Result<(Event, Keys), ChatError>; -pub fn open(wrap: &Event, group: &GroupKey, channel: &ChannelId, epoch: Epoch) - -> Result<(OpenedStream, ChatRumor), ChatError>; -pub fn plane_keys(held: &[(Epoch, [u8; 32])], channel: &ChannelId) -> Result>; -pub fn expiration_of(rumor: &UnsignedEvent) -> Result, ChatError>; -pub fn expired(rumor: &ChatRumor, now: Timestamp) -> bool; -pub fn fold(rumors: &[ChatRumor], now: Timestamp, - can_delete: impl Fn(&PublicKey, Option<&AuthorityCitation>, &PublicKey) -> bool) - -> Vec; - -pub struct ChatMessage { - pub id: EventId, - pub author: PublicKey, - pub channel: ChannelId, - pub epoch: Epoch, - pub kind: Kind, - pub content: String, - pub reply_to: Option, // a kind 9's `q`, or a comment's lowercase `e` - pub thread_root: Option, // a comment's uppercase `E` - pub at_ms: u64, - pub expiration: Option, - pub edited_at: Option, - pub deleted: bool, - pub reactions: BTreeMap, -} -``` - -- `open` returns the `OpenedStream` next to the typed rumor because the two halves go - different ways: the caller caches the envelope, and folds the rumor. -- Ordering is `(at_ms, id)` everywhere, ties on the lower inner rumor id. The fold emits - newest first; mutations replay ascending on `(at_ms, Reverse(id))` so the last one - applied wins — the highest `at_ms` and, between equal ones, the lower id. A deletion is - terminal: a later edit never revives it. -- **Since M5 a delete is honored from the message's own author unconditionally, or from anyone - `can_delete` admits.** That predicate is `(actor, citation, target_author)`, composed by the caller - from the roster: a resolvable `vac` citation, `MANAGE_MESSAGES`, and a strictly higher rank than the - author — the delete is the one Chat-plane authority action, which is why the builder takes a citation - and the fold takes a gate. A self-delete never consults the predicate, matching CORD-02 §9's carve-out - that a member's erasure of their own words survives even Dissolution. -- `kind 15` is coop's own file-message convention, outside the CORD registry — it is - accepted on the read side so a second coop device's files are not dropped, and - `send_file` lands with the registry. -- A reference's author slot is a SHOULD on the wire, so it is optional. NIP-25 nonetheless - makes `p` a requirement, so a builder must be handed a `Target` built from the message it - acts on and never one whose author is empty, or a peer that requires the tag drops the result. -- `ms` orders a page but cannot page within one: a relay's `until` filter is second-granular, - so the cursor step below is what has to cope with a boundary second. -- `seal_rumor` gates the kind at publish and mirrors a NIP-40 `expiration` onto the wrap - (CORD-08 §2), so a NIP-40 relay drops the ciphertext itself. Since M8 the durable builders - *attach* the tag from the folded timer, `fold` drops an expired rumor before displaying it, and - a delete or a timer notice carrying the tag is refused outright — see §8.8. -- **`media` and `mentions` are deliberately absent.** Both are pure post-processing of - `content` by `common` (`extract_and_remove_media_urls`, `NostrParser`) and both return a - gpui type, and a protocol crate does not take a UI dependency for a derived field. They - land with the first consumer that renders them. - -### 8.5 Invites (`invite.rs`) — bundle and Direct Invite in M6, Invite List in M7, Registry in M7 - -```rust -pub const KIND_BUNDLE: u16 = 33301; -pub const KIND_DIRECT_INVITE: u16 = 3313; -pub const FRAGMENT_VERSION: u8 = 4; // the dictionary generation -pub const MAX_BUNDLE_CHANNELS: usize = 256; -pub const MAX_BOOTSTRAP_RELAYS: usize = 3; -pub const MAX_BUNDLE_EPOCH: u64 = 1 << 40; // attacker-set, so bound it - -pub struct ChannelGrant { id: ChannelId, key: Option, epoch: Epoch, name: String, extra } -pub struct CommunityInvite { community_id: CommunityId, owner: PublicKey, owner_salt: String, - community_root: String, root_epoch: Epoch, - control_pk: Option, channels: Vec, - relays: Vec, name: String, icon: Option, - expires_at: Option, creator_npub: Option, - label: Option, extra } -impl CommunityInvite { - pub fn from_bundle_json(json: &str) -> Result; // bound, truncate, validate - pub fn validate(&self) -> Result<(), InviteError>; // recompute community_id - pub fn expired(&self, now_ms: u64) -> bool; -} -pub enum BundleState { Live(Box), Revoked } - -pub fn build_bundle_event(link_signer: &Keys, invite: &CommunityInvite, bundle_key: &[u8; 32]) - -> Result; -pub fn build_revocation(link_signer: &Keys) -> Result; -pub fn parse_bundle_event(event: &Event, expected_signer: &PublicKey, bundle_key: &[u8; 32]) - -> Result; - -pub fn stock_relays() -> Vec; -pub fn encode_fragment(token: &[u8; 16], relays: &[String]) -> Result; -pub fn decode_fragment(fragment: &str) -> Result<([u8; 16], Vec), InviteError>; -pub fn bundle_naddr(link_signer: &PublicKey) -> Result; -pub fn build_invite_url(base: &str, link_signer, token, relays) -> Result; -pub struct ParsedInviteLink { link_signer: PublicKey, token: [u8; 16], - bootstrap_relays: Vec, naddr: String } -pub fn parse_link(input: &str) -> Result; - -pub fn build_direct_invite(inviter: &Keys, recipient: &PublicKey, invite: &CommunityInvite) - -> Result; -pub fn unwrap_direct_invite(wrap: &Event, recipient: &Keys) - -> Result<(PublicKey, CommunityInvite), InviteError>; -``` - -- The link is `…/invite/#`: `Nip19Coordinate` for `(33301, link_signer, "")` in -the path, and `[version][flags][relays?][token:16]` base64url-no-pad in the fragment, which is -never sent to a server. `parse_link` also accepts the domain-agnostic bare `#`. -- **The fragment's byte layout is frozen and golden-tested** against hand-computed base64url: the -stock set is flag `0x01` with zero relay bytes (and exempt from the 3-relay cap, which applies to -explicit entries), otherwise a count then per-relay dictionary id, `0x00 len host` for a -`wss://`-implied literal, or `0xff len url` verbatim. A version this client will not decode is -fatal in **both** directions, since a legacy link would be decoded against the wrong dictionary. -- The bundle is sealed with `invite_bundle_key(token)` used **directly** as the NIP-44 conversation -key (`ConversationKey::new`, not an ECDH pair) — the one place in the protocol where that is so. -- Trust is the `community_id`, which `validate` recomputes from `owner` + `owner_salt`. The bundle -is attacker-reached input, so it is bounded *before* it is used: an over-count of channels, an -epoch past the ceiling, and a secret that is not 32 bytes of hex are all refused up front. -`expires_at` is deliberately not part of `validate`: a parked invite still renders past expiry, -only joining refuses. -- `channels` and `relays` default to empty rather than being required: a bundle vending no channel -keys omits the field entirely, and a required `Vec` would turn a stale read into a join failure. -A channel `key` is `Option`, because a public channel derives from `community_root` and never -carries one. -- The bundle's `name` is a preview, not authority — the Control fold is — so it is not bounded here. -`relays` **is** truncated to the community cap, because a hostile list is a connect storm. -- `parse_bundle_event` re-checks the author, the empty `d` (an absent `d` is that coordinate, so -absent and empty are both accepted and a non-empty one is fatal), the signature, and the `vsk` -marker before it decrypts anything. `vsk 6` is live, `vsk 9` is the tombstone. -- The Direct Invite is a **standard** NIP-59 giftwrap, built by `nip59::GiftWrapBuilder` and read -by `nip59::UnwrappedGift::from_gift_wrap`, both from nostr: the builder does the ephemeral wrap and -the tweaked timestamps, and the reader verifies the wrap, the seal's signature, and the rumor/seal -author bind, which is exactly the gate set the reference implementation hand-rolls. Everything left -is the rumor kind and the bundle's own validation. The wrap carries `["k","3313"]` so a recipient -can index their invites without decrypting their whole giftwrap inbox, and mirrors `expires_at` -as a NIP-40 tag in seconds. -- The earlier sketch's `signer: &UniversalSigner` is `&Keys`, matching every other builder in the -crate: NIP-46 is one deliberate pass, not a per-milestone patch (see §14.9). - -**The Invite List (`13303`) and the Registry (`vsk 8`) landed in M7**, deferred from M6 for one reason: -nothing in M6 consumed them. A link's signer is held by its caller, so minting, refreshing and revoking -need no document, and a Registry write whose fold does not exist is dead wire. - -```rust -pub const KIND_INVITE_LIST: u16 = 13303; -pub const MAX_INVITE_ENTRIES: usize = 64; - -pub struct InviteEntry { token: String, signer_sk: String, community_id: CommunityId, url: String, - label: Option, created_at: u64, expires_at: Option, extra } -pub struct InviteTombstone { token: String, community_id: CommunityId, extra } -pub struct InviteList { entries: Vec, tombstones: Vec, extra } -impl InviteList { - pub fn is_live(&self, token: &str) -> bool; - pub fn fits(&self) -> Result<(), InviteError>; // the write gate -} - -pub fn merge_invite_lists(held: InviteList, incoming: InviteList) -> InviteList; -pub fn build_invite_list(keys: &Keys, list: &InviteList) -> Result; -pub fn parse_invite_list(keys: &Keys, event: &Event) -> Result; -``` - -- The creator's private bookkeeping: `token` is the link's unlock secret **and** its merge key, and -`signer_sk` is the `link_signer` secret that refreshing or retiring the bundle needs. An entry is -immutable once minted, so a divergent pair is settled on the lowest canonical bytes — a total order, so -two devices never flap. Tombstones union and beat an entry **terminally**, so a stale device can never -resurrect a revoked link. Like the Community List it is NIP-44-to-self at a replaceable kind, and -`fits` refuses to build past the entry cap or the NIP-44 plaintext cap. -- The Registry is its member-facing shadow: a Control Plane entity (`vsk 8`) at -`invite_links_locator(community_id, creator)` whose content is the live links' **coordinates only** — -never a token, URL or signing secret — so members can see that links exist without being able to use one. -`ControlFold.registries` holds one set per creator, honored only under `CREATE_INVITE`, and -`ControlFold::is_public` reads their aggregate: non-empty means a live link exists and the community is -Public. Retiring the last live link empties it, and that flip is what a Refounding seals (§8.7). -`ControlWriter::set_registry` is the write side. - -### 8.6 The Community List (`list.rs`) — implemented in M6 - -The member's own memberships, `13302` replaceable, NIP-44-encrypted to self. - -```rust -pub const KIND_COMMUNITY_LIST: u16 = 13302; -pub const MAX_MEMBERSHIPS: usize = 50; - -pub struct JoinMaterial { community_id: CommunityId, owner: PublicKey, owner_salt: String, - community_root: String, root_epoch: Epoch, - control_pk: Option, control_root: Option, - channels: Vec, relays: Vec, name: String, extra } -pub struct CommunityListEntry { community_id: CommunityId, seed: JoinMaterial, - current: JoinMaterial, added_at: u64, extra } -pub struct Tombstone { community_id: CommunityId, removed_at: u64, extra } -pub struct CommunityList { entries: Vec, tombstones: Vec, extra } -impl CommunityList { - pub fn is_live(&self, community_id: &CommunityId) -> bool; - pub fn fits(&self) -> Result<(), ListError>; // the write gate -} - -pub fn join_material(invite: &CommunityInvite, control_root: Option<&[u8; 32]>) -> JoinMaterial; -pub fn merge(held: CommunityList, incoming: CommunityList) -> CommunityList; -pub fn build_list_event(keys: &Keys, list: &CommunityList) -> Result; -pub fn parse_list_event(keys: &Keys, event: &Event) -> Result; -``` - -- Join material is the bundle's **membership subset**: the link-only fields (icon, expiry, label, -creator) are dropped, and `control_root` is added when the holder is staff, since no bundle carries -it. `join_material` is the one conversion between the two documents. -- The two snapshots solve opposite problems and merge the same way: `seed` keeps the **lower** -`root_epoch` and `current` keeps the **higher**, each anchoring an end of the history so a fresh -device needs no epoch-by-epoch walk. -- An epoch tie breaks on the **lowest canonical bytes of the whole snapshot** — a total order, so -two devices never flap competing republishes. `extra` maps union on the same principle (the lower -canonical value wins a key clash), which keeps the merge order-independent while still preserving -unknown fields. -- `added_at` merges to the newest and `removed_at` likewise, so a re-join legitimately resurrects a -membership while a stale device's republish can never re-add a tombstoned id. **A tombstoned entry -stays in the document** — pruning it would make the merge depend on gossip order — and -`is_live` is what reads the newest of the two timestamps. -- `parse_list_event`'s failure is deliberately meaningful: the caller must read it as "no news" and -merge whatever does arrive, never clobber a populated local list with an absence. -- `fits` is the write gate: over the membership cap or over the NIP-44 plaintext cap, the build is -refused rather than truncating memberships to make it fit. The *join* gate that refuses a 51st -membership is the registry's, and the byte-level cap audit is M8's. -- `13302` is implemented per spec. Vector has retired it for fragmented `33302` (a replaceable kind -holds one event per pubkey, so it cannot shard past the size cap); that remains an interop -follow-up, recorded in §14.1. - -### 8.7 Rekeys, refoundings and dissolution (`rekey.rs`) — implemented in M7 - -```rust -pub const KIND_REKEY: u16 = 3303; -pub const MAX_REKEY_BLOBS_PER_EVENT: usize = 80; -pub const MAX_REKEY_BLOBS_RECEIVED: usize = 120; -pub const MAX_REKEY_EPOCH: u64 = 1 << 40; - -pub enum RekeyScope { Channel(ChannelId), Base } -pub struct RekeyBlob { locator: String, wrapped: String } -pub struct KeyDelivery { new_key: [u8; 32], control_pk: Option<[u8; 32]>, control_root: Option<[u8; 32]> } -pub enum Continuity { Extends, Gap, Fork } -pub struct RekeyChunk { rotator, scope, new_epoch, prev_epoch, prev_commit, chunk: (u32, u32), - blobs, citation, severed } -pub struct Rotation { rotator, scope, new_epoch, prev_epoch, prev_commit, blobs, declared, - held, severed, citation } -pub struct Refounding { epoch: Epoch, new_root: [u8; 32], new_control_root: [u8; 32] } - -pub fn encode_blob_plaintext(scope, epoch, new_key, control_pk, control_root) -> Result, RekeyError>; -pub fn parse_blob_plaintext(bytes, scope, epoch, community_id) -> Result; -pub fn rekey_group(scope, addressing_root, community_id, new_epoch) -> Result; -pub fn blob_locator(rotator, recipient, scope, epoch) -> String; -pub fn build_blob(rotator: &Keys, recipient, scope, epoch, new_key, control_pk, control_root) - -> Result; -pub fn open_blob(recipient: &Keys, rotator, scope, epoch, blob, community_id) - -> Result; -pub fn find_my_blobs<'a>(blobs: &'a [RekeyBlob], rotator, me, scope, epoch) - -> impl Iterator; - -pub fn build_rekey_rumor(rotator, scope, new_epoch, prev_epoch, prev_commit, blobs, chunk, - citation, severed, at_secs) -> Result; -pub fn build_rekey_chunks(rotator: &Keys, group, scope, new_epoch, prev_epoch, prev_commit, blobs, - citation, severed, at_secs) -> Result, RekeyError>; -pub fn parse_rekey_chunk(opened: &OpenedStream) -> Result; - -pub fn collect_rotations(chunks: &[RekeyChunk]) -> Vec; -pub fn am_i_removed(rotation: &Rotation, me: &PublicKey) -> Option; -pub fn rekey_authorized(roles, owner, rotator, permission, removed) -> bool; -pub fn fork_winner(held: Option<&[u8; 32]>, candidates: &[[u8; 32]]) -> Option; - -pub fn plan_refounding(epoch: Epoch) -> Result; -pub fn compact(seals: &[Event], read: &GroupKey, signer: &GroupKey, at_secs: u64) - -> Result, RekeyError>; - -pub struct DissolvedTombstone { owner: PublicKey } -pub fn dissolved_tombstone_rumor(owner, community_id, at_secs) -> UnsignedEvent; -pub fn seal_dissolved(rumor, community_id, owner: &Keys, at_secs) -> Result; -pub fn open_dissolved(wrap, community_id) -> Result; -pub fn verify_dissolved(wrap, identity: &CommunityIdentity) -> bool; -``` - -- The subscription for rekeys is precomputed from the *next* epoch's address, per private channel -and once for the base. That is a `Filter` and belongs to the sync engine (§10), not here; `rekey_group` -is what makes it derivable, and keeps a channel scope from being addressed at the base derivation. -- A receiver accepts a key only after: locating its blob, decrypting with the rotator↔recipient -conversation key, checking the bound `scope` and `epoch` **inside** the plaintext, and matching -`prevcommit` against the key it currently holds. The locator is deliberately *not* gated on open: -it derives from public keys alone (NIP-46 bunker parity) and so proves nothing, and the pairwise -decrypt plus the bound check are the whole gate. -- Only after holding **all** `n` chunks of one `(rotator, newepoch, prevcommit)` set, with none -containing its locator, may a client conclude it was removed. `collect_rotations` **unions** the blobs -of two chunks claiming one index rather than keeping the first: a catch-up chunk can legitimately -re-claim a slot, and a recipient dropped from the union would read as removed, which deletes the -community locally. `severed` is an OR across chunks for the same reason in reverse. -- Send cap 80 blobs per event, accept cap 120. The 80 is an erratum this reproduces: the CORD-01 double -envelope costs two NIP-44 base64 expansions, so 120 blobs measure ~77 KB and a 64 KB relay refuses -them. The accept cap stays at the spec's 120 so a peer at the spec limit still parses. -- `parse_blob_plaintext` takes the `community_id`, which the earlier sketch did not: the 104/136-byte -base forms carry the next epoch's Control Plane keys, and the 136-byte secret must derive to the -`control_pk` beside it — a community- and epoch-bound check. A **width past 136** is a form this client -predates, so it degrades rather than refusing: the frozen 72-byte prefix yields the root (membership and -every chat plane survive), the appended pair is kept when it verifies, and the rest freezes. Widths -between the defined forms fit no extension and stay malformed. -- Authority: a channel rekey needs `MANAGE_CHANNELS`, a refounding needs `BAN`, and in both the rotator -must strictly outrank every removed target. `rekey_authorized` is that whole rule — one permission check -plus `can_act_on_member` per target — so it is testable without minting a rotation, and an empty removed -set (a hygienic rotation, or a flip to Private) needs only the permission. Holding a key is never authority. -- Two concurrent refoundings converge on the lexicographically lowest new base key, and the heal is -down-only. `fork_winner` folds both into one call: the lowest candidate, returned only when it strictly -lowers a key already held, so a flaky fetch cannot re-fork a settled epoch. -- `Refounding` owns the epoch and the freshly minted pair and derives all three coordinates from them -(`read`, `signer`, and the signer's pk), so a caller cannot pair a root with the wrong epoch. -`plan_refounding` deliberately takes neither the fold nor the removed set: the authority gate is -`rekey_authorized`, one job each, and the pair travels in the base blobs. -- Compacted control heads are re-wrapped with their original signature intact, which is exactly why the -control plane uses the plaintext seal. `compact` is `stream::rewrap_seal` over the held seals, whose -signature grew a separate `signer` group in M7 — a split epoch reads under the rolled root but wraps as -the new control signer, so one group could not express it. -- Nothing in coop **mints** `severed` yet: it is a receiver-side rule, and the Server-Severing flow that -would set it awaits the invite-link wiring of §10. - -Dissolution (CORD-02 §9) also lives here: a chainless, owner-signed `vsk 10` tombstone at -`dissolved_group_key(id)`, plaintext-sealed, and a verifier **must** refuse any tombstone whose `eid` is -not the community's own id (including the all-zero placeholder — accepting it lets an owner's genuine -tombstone for one community be re-wrapped at another of theirs and kill it permanently). On sight the -community is sealed read-only: `CommunityState.dissolved` records it, subscriptions halt, nothing new is -honored, existing history stays readable, and a member's delete of their own message is still honored. - -### 8.8 Pins and disappearing messages (`pins.rs`, `chat.rs`, `store.rs`) — implemented in M8 - -**Pins (CORD-04 §7).** One Pin List per Channel: `vsk 11` at `pins_locator(community_id, channel)`, -derived from the `community_id`, so it survives every refounding and a fresh joiner derives the same -coordinate. A pin does not quote a message, it *proves* one — the entry carries the original kind-20013 -seal verbatim plus that message's 76-byte NIP-44 key disclosure, so a reader holding no history and no -old keys still verifies author, words, Channel and signed time. - -```rust -pub const PIN_MAX_ENTRIES: usize = 25; -pub const PIN_MAX_CONTENT_BYTES: usize = 32_768; -pub const MESSAGE_KEYS_BYTES: usize = 76; - -pub struct MessageKeys { /* chacha_key[32] ‖ chacha_nonce[12] ‖ hmac_key[32] */ } -pub struct PinEditBundle { seal: Event, keys: String } -pub struct PinEntry { seal: Event, keys: String, wrap: Option, edit: Option, extra } -pub struct VerifiedPin { rumor_id, author, kind, content, tags, epoch, at_ms, created_at, wrap, edited, entry } -pub struct ReadPinList { entries: Vec, sealed: bool } -pub enum PinError { NotEncryptedSeal, BadPayload, Unverifiable, Unreadable, TooManyEntries, Oversize, Seal, Encode } - -pub fn build_entry(opened: &OpenedStream, group: &GroupKey, channel: &ChannelId) -> Result; -pub fn build_edit_bundle(edit: &OpenedStream, group, original: &VerifiedPin, channel) -> Result; -pub fn with_proven_edit(entry: &PinEntry, edit: &OpenedStream, group, channel) -> PinEntry; -pub fn verify_entry(entry: &PinEntry, channel: &ChannelId) -> Option; -pub fn read_list(content: &str, unseal: impl Fn(Epoch) -> Option) -> ReadPinList; -pub fn publishable(read: &ReadPinList, private: bool, group: &GroupKey, epoch: Epoch) -> Result; -pub fn killed_by(pin: &VerifiedPin, delete: &ChatRumor) -> bool; -``` - -- **The disclosure is a reproduction, not a re-use.** `nostr`'s `nip44::v2::get_message_keys` is a private -`fn` and both public entry points take the whole conversation key, so the expansion is rebuilt from the -same audited primitives: `hkdf::Hkdf::from_prk(conversation_key).expand(nonce, 76)`, split, then -HMAC-SHA256 over `nonce ‖ ciphertext` — compared in constant time, so the verify path is never a MAC -oracle — ChaCha20, and NIP-44's padding check. `MessageKeys::to_hex`/`from_hex` are the wire form -(lowercase-canonical), and one test round-trips the whole thing against nostr's own -`encrypt_to_bytes_with_nonce`, which is what keeps the copy honest. -- **The whole verification**, in order: the seal is kind 20013 and verifies → the entry's disclosure -decodes → MAC → decrypt → unpad → `UnsignedEvent::from_json` → the rumor's `pubkey` equals the seal's -(NIP-59's impersonation check) → the kind is `9` or `1111` → `channel` strict-equal to this list's -Channel → a canonical `epoch` → `verify_id()` and a *recomputed* identity. An Edit bundle failing any of -its own steps is dropped alone; the pin survives it. -- **Two deliberate divergences from Vector**, both stricter: the `epoch` tag is required (CORD-03 §3 makes -it mandatory and `chat::open` already enforces it) where Vector ignores it, and an out-of-range `ms` is -malformed rather than read as `0`, matching this crate's own reader. A duplicated `channel` tag resolves -to the first, as Vector does, because the rumor is author-signed and the lenient reading is not -exploitable. -- **Two self-describing content forms**, never signalled by the fold. Public is `{ "entries": [...] }`; -private is `{ "epoch": "", "sealed": seal_bytes(...) }`, whose sealed plaintext is that same -`{ "entries": [...] }`. A reader accepts either form regardless of its metadata fold; a writer must use -the form matching the Channel's folded type. -- **The caps are content-level, never chain-level.** `read_list` yields an EMPTY list for oversize -content, an over-cap count, a malformed envelope or a failed decrypt, and `sealed: true` — darkness, not -violation — when the named epoch's key is missing. The edition itself always folds, so every client walks -the same chain. -- **`publishable` is the only write path**, and it refuses a dark list outright: an unreadable entry and -an absent one are indistinguishable, so re-forming would silently drop every entry the writer cannot see. -That is the write half of §12's "never publish from a list you could not read". -- The Edit bundle is the same five steps with kind `3302` substituted plus the fold's own two rules — the -proven author equals the original's, and the `e` tag names the original's recomputed rumor id — so a -keyless reader reaches the verdict a keyed reader reaches by folding. At most one, ever: a later Edit -*replaces* it, since edits target the original and never each other. -- Deferred to §10: §7's "a private→public conversion MUST NOT mechanically re-form the list" has no -encoding — "mechanically" and "deliberately" are the same bytes — so it is a duty of the conversion flow -(§14.13); the re-heal and deletion-omission writes (`killed_by` exists, the re-fold-and-republish loop -does not); and the automatic Edit refresh. - -**Disappearing messages (CORD-08).** - -- `CommunityMetadata.message_expiration: Option` is the timer in seconds, with a lenient -deserializer: absent, `0`, a string, a float or any other garbage all read as `None`, and garbage never -poisons the rest of the entity. The write side normalizes `Some(0)` away. -- The timer is community state, so the tag rides the **signed rumor**: `build_message`, `build_comment`, -`build_reaction` and `build_edit` take `timer: Option` and attach -`["expiration", created_at + timer]` from their own `at_ms`, which is what makes a change -non-retroactive. `seal_rumor` had mirrored the tag onto the wrap since M4, so a NIP-40 relay deletes the -ciphertext itself. -- **Two kinds are exempt**, and a rumor carrying the tag is refused with `ChatError::ExemptExpiration`: -a delete (its target may outlive it) and the timer notice (the policy must not be erased by the policy). -- **The timer notice** is kind `1740`: `build_timer_notice` writes it, `ChatAction::TimerNotice { seconds }` -reads it with a canonical `timer` tag, and the fold emits it as a row of its own, like any message. -Whether its author may be believed about policy is `MANAGE_METADATA` in the roster — the registry's call, -not the fold's, and `roles.is_authorized(&author, &owner, Permissions::MANAGE_METADATA)` already answers -it. -- **Enforcement lives in two places, both keyed on the rumor's own signed tag.** `chat::fold(rumors, now, -can_delete)` drops an expired rumor, so it is never displayed whatever the ingest path. `store::cache_rumor` -refuses to store one that has already expired (returning whether it kept it) and `backfill` both skips the -cache and drops it from the page. `store::purge_expired` is the sweep: it re-reads each cached row's -rumor, collects the expired ids and physically deletes them, because hiding is not disappearing and the -local store is the artifact a seized device surrenders. A malformed tag expires nothing. - -## 9. Storage (`store.rs`) — local layer implemented in M1, state document in M2, fold bridge in M3, banlist in M5 - -Three layers, no new storage engine: - -1. **Raw wraps** (kind 1059) are persisted automatically by the SDK's relay pool when a subscription or fetch matches a filter. Nothing to write. -2. **Opened rumors** are cached locally as NIP-78 `Kind::ApplicationSpecificData` events signed by a session-local keypair, exactly like `chat::set_rumor`. Tags: `["d", rumor_id]` (replace key), `["c", channel_hex]`, `["p", author]`, `["k", kind]`, `["e", wrap_id]`, `["t", "concord"]`. Contents are the rumor JSON. - - The `c`/`t` keys deliberately differ from chat's `r` key so the two message namespaces can never collide in one database. - - `created_at` is the **message's own second** (from `at_ms`), not the wall clock. Otherwise `until` and the ordering would page on cache time rather than message time. - - The read path dedupes by rumor id and keeps the newest `created_at`, because the local signing key changes per session and each session leaves its own copy. The query therefore carries no filter `limit` — every copy has to be in hand before they can be collapsed — and the cap is applied to the deduplicated result instead. - - The layer takes `&dyn NostrDatabase`, not `&Client`: it is local-only, which keeps it testable without a relay or a GPUI context. - -```rust -pub async fn cache_rumor(database: &dyn NostrDatabase, channel: &ChannelId, opened: &OpenedStream) -> Result; -pub async fn query_rumors(database: &dyn NostrDatabase, channel: &ChannelId, until: Option, limit: usize) -> Result>; -pub async fn purge_expired(database: &dyn NostrDatabase, channel: &ChannelId, now: Timestamp) -> Result; -pub async fn backfill(client: &Client, database: &dyn NostrDatabase, channel: &ChannelId, - held: &[(Epoch, [u8; 32])], until: Option, limit: usize) - -> Result>; -``` - -`cache_rumor` returned `()` until M8, when it gained the CORD-08 §3 ingest rule and with it a reason to -report: `false` means an already-expired rumor was refused and nothing was written. `cache_rumor` and -`purge_expired` are the two halves of the timer's storage policy; §8.8 has the rest. - -`query_rumors` returns `UnsignedEvent`, not `Event`: the cached payload *is* a rumor, which is also what `OpenedStream` carries, so the caller never has to re-parse. - -**Landed in M4:** `backfill` — newest-first relay paging across every held epoch. It derives -every held epoch's plane key once, fetches `kinds [1059, 21059]` by all of those addresses in -one filter with an inclusive `until`, opens each wrap against the plane whose address it -carries, caches it, and pages until the page is short of the limit, adds nothing new, or the -cursor cannot advance. That last case is real: `until` has second granularity, so a page that -begins and ends inside one boundary second has nowhere left to step and its remainder stays -unreachable until a relay serves it. Capped at `MAX_PAGES` so a relay that only ever repeats -itself cannot loop a client forever. - -3. **Community state** — one local document per community, `Kind::ApplicationSpecificData` with `["d", "concord/"]`: - -```rust -pub struct CommunityState { - pub id: CommunityId, - pub owner: PublicKey, - pub owner_salt: [u8; 32], - pub community_root: [u8; 32], - pub root_epoch: Epoch, - pub control_root: Option<[u8; 32]>, // present iff the holder is staff - pub control_pks: BTreeMap, // epoch → the plane's signer address - pub channels: Vec, // id, name, private, epoch - pub relays: Vec, - pub heads: Vec, // entity, version, self_hash, inner id - pub banned: BTreeSet, // the held banlist, fed back into the next fold - pub added_at_ms: u64, -} -``` - -Landed in M2 with exactly the fields genesis can populate: `save_state`/`load_state` and `CommunityState::from_genesis`. `dissolved` landed in M7, set from a verified tombstone. Two fields the plan sketched are still absent: `epoch_keys`, because `ChannelKeyRef` carries no key for it to hold (§14.11), and `observed`/`guestbook`, which need the guestbook ingest of §10. `control_pks` keyed by `u64` rather than `Epoch` and `heads` as a `Vec` rather than a `BTreeMap<[u8; 32], _>`, because serde_json cannot use a byte-array map key. `banned` is a `BTreeSet` — serde has no such problem with a `Vec`-like sequence. - -M3 added the two bridges between this document and the fold: - -```rust -impl CommunityState { - pub fn floors(&self) -> Floors; // the fold's input - pub fn apply_fold(&mut self, fold: &ControlFold); // the fold's output -} -``` - -`apply_fold` merges channels rather than replacing them, so a locally-held key survives a metadata edit, and **assigns `banned` wholesale** (M5): the fold already retains a withheld list, so its output is the authority and a caller must not merge it by hand. `floors()` and `banned` are the two inputs the next `fold_control` call needs, which makes the state document a fold cache rather than a second source of truth. - -Writes are debounced (a fold head changes on every edition); reads load once at init. - -**Decision, stated for the record:** this document stores raw community keys unencrypted in a plaintext local database. That matches the existing posture — `chat` already caches decrypted message rumors in the same LMDB. If that posture ever changes, the state document is the one to wrap with NIP-44-to-self, since it is the only local artifact holding keys. - -## 10. Sync engine and GPUI conventions - -`ConcordRegistry` mirrors `ChatRegistry`'s shape exactly: a foreground GPUI entity holding `Entity` handles, a `flume` signal bus, one background notification listener, one foreground consumer, and task slots that are cleared when the signer changes. - -**Subscription.** Community relays come from the folded metadata. `init`/`join` add them to the client (`client.add_relay(url).and_connect()`), then: - -```rust -let filter = Filter::new() - .kinds([Kind::GiftWrap, Kind::Custom(KIND_WRAP_EPHEMERAL)]) - .authors(plane_authors) // guestbook, control signer, all held channel planes, - // rekey addresses for epoch + 1, dissolved address - .since(Timestamp::from_secs(now - FRESH_WINDOW)) - .limit(0); // live tail only; history comes from backfill -client.subscribe(target).with_id(SubscriptionId::new(CONCORD_SUB)).await?; -``` - -Targeted subscribe against the community relays, with a pool-wide subscribe as the fallback path. Rebuild idempotently whenever a plane's address changes (join, channel added, rekey folded). - -**Routing.** `dispatch` matches on the `subscription_id` carried in `RelayMessage::Event`, dedupes by wrap id (both subscriptions and several relays deliver the same wrap), then recognises the plane by **wrap author** against the derived addresses it holds — never by trial decryption. Recognition order: held channel planes → guestbook → control signer → rekey addresses → dissolved. - -**Ingest pipeline.** Unwrap, verify and fold all happen inside `cx.background_spawn`, never on the foreground thread: secp256k1 verification per edition and per seal is far too expensive for the UI thread. - -```rust -enum Signal { - Chat { community: CommunityId, channel: ChannelId, message: Box }, - Control { community: CommunityId, heads: Vec, roster: Box }, - Guestbook { community: CommunityId, members: BTreeSet }, - Rekey { community: CommunityId, scope: RekeyScope, epoch: Epoch }, - Dissolved(CommunityId), - Eose(SubscriptionId), - Error(ConcordError), -} -``` - -The consumer is `cx.spawn(async move |this, cx| { while let Ok(signal) = rx.recv_async().await { this.update(cx, |this, cx| this.apply(signal, cx))?; } })`, which updates entities and calls `cx.notify()`. - -Rules taken from the project guidelines: - -- Crypto, folding, database queries and network I/O only in `cx.background_spawn`. -- Foreground tasks are `cx.spawn` with `this.update(cx, ..)`; any entity update happens there, and the inner `cx` is always used. -- Tasks are stored in fields (`tasks`, `listener`, `consumer`) so they are cancelled on signer change and dropped with the registry. `detach()` only for genuinely fire-and-forget work such as the local state save. -- Long-running paging is bounded by explicit page and step caps, not by unbounded loops. -- Every fallible path returns `Result` and surfaces through `ConcordEvent::Error`; nothing is silently swallowed. - -**Registry API.** - -```rust -pub fn init(window: &mut Window, cx: &mut App); -pub struct ConcordRegistry { /* … */ } -impl ConcordRegistry { - pub fn global(cx: &App) -> Entity; - pub fn loading(&self) -> bool; - pub fn communities(&self) -> Vec>; - pub fn community(&self, id: &CommunityId, cx: &App) -> Option>; - pub fn find(&self, query: &str, cx: &App) -> Vec>; - - pub fn create(&mut self, params: CommunityParams, cx: &mut Context) -> Task>; - pub fn join(&mut self, link: &str, cx: &mut Context) -> Task>; - pub fn accept_direct_invite(&mut self, rumor: &UnsignedEvent, cx: &mut Context) -> Task>; - pub fn leave(&mut self, id: &CommunityId, cx: &mut Context); - pub fn discard_invite(&mut self, id: &CommunityId, cx: &mut Context); - pub fn refresh(&mut self, id: &CommunityId, cx: &mut Context); - pub fn shutdown(&mut self, cx: &mut Context); // halt subscriptions, keep our own state -} -``` - -**Community API** (`Entity`, `EventEmitter`): - -```rust -pub fn id(&self) -> CommunityId; -pub fn owner(&self) -> PublicKey; -pub fn name(&self) -> SharedString; pub fn description(&self) -> Option; -pub fn icon(&self) -> Option; -pub fn relays(&self) -> Vec; -pub fn epoch(&self) -> Epoch; -pub fn dissolved(&self) -> bool; -pub fn channels(&self) -> Vec>; -pub fn channel(&self, id: &ChannelId, cx: &App) -> Option>; -pub fn members(&self) -> BTreeSet; -pub fn banned(&self) -> BTreeSet; -pub fn roles(&self) -> &CommunityRoles; -pub fn permissions(&self, member: &PublicKey) -> u64; -pub fn is_staff(&self, member: &PublicKey) -> bool; -pub fn message_expiration(&self) -> Option; - -// authority actions — each returns a publish task and nothing optimistic -pub fn set_metadata(&mut self, meta: CommunityMetadata, cx: &mut Context) -> Task>; -pub fn create_channel(&mut self, name: &str, private: bool, cx: &mut Context) -> Task>; -pub fn edit_channel(&mut self, id: &ChannelId, meta: ChannelMetadata, cx: &mut Context) -> Task>; -pub fn create_role(&mut self, role: Role, cx: &mut Context) -> Task>; -pub fn assign_roles(&mut self, member: &PublicKey, roles: &[[u8; 32]], cx: &mut Context) -> Task>; -pub fn ban(&mut self, members: &[PublicKey], cx: &mut Context) -> Task>; -pub fn unban(&mut self, members: &[PublicKey], cx: &mut Context) -> Task>; -pub fn kick(&mut self, member: &PublicKey, cx: &mut Context) -> Task>; -pub fn rekey_channel(&mut self, id: &ChannelId, removed: &[PublicKey], cx: &mut Context) -> Task>; -pub fn refound(&mut self, removed: &[PublicKey], cx: &mut Context) -> Task>; -pub fn dissolve(&mut self, cx: &mut Context) -> Task>; -pub fn create_invite(&mut self, params: InviteParams, cx: &mut Context) -> Task>; -pub fn revoke_invite(&mut self, token: &[u8; 16], cx: &mut Context) -> Task>; -pub fn direct_invite(&mut self, receiver: &PublicKey, cx: &mut Context) -> Task>; -pub fn save_community_list(&mut self, cx: &mut Context) -> Task>; // kind 13302, multi-device sync -``` - -**Channel API** (`Entity`): `id`, `name`, `private`, `epoch`, `deleted`, plus - -```rust -pub fn messages(&self, until: Option, limit: usize, cx: &App) -> Task, Error>>; -pub fn send(&self, content: &str, reply_to: Option, cx: &App) -> Task, Error>>; -pub fn send_file(&self, file: FileAttachment, reply_to: Option, cx: &App) -> Task, Error>>; -pub fn edit(&self, id: EventId, content: &str, cx: &App) -> Task, Error>>; -pub fn delete(&self, id: EventId, cx: &App) -> Task, Error>>; -pub fn react(&self, id: EventId, emoji: &str, cx: &App) -> Task, Error>>; -pub fn typing(&self, cx: &App) -> Task>; // kind 23311, ephemeral -pub fn pin(&self, id: EventId, cx: &App) -> Task>; // vsk 11, PIN_MESSAGES -``` - -`CommunityEvent` and `ChannelEvent` mirror `ChatEvent`: one variant per thing the UI has to react to (`Updated`, `Members`, `Added`, `Removed`, `Dissolved`, `Error`, plus channel-level `Incoming`, `Reload`). - -Every send funnels through one function so the rules cannot drift: it seals and wraps the -rumor, mirrors any NIP-40 `expiration` onto the wrap, publishes via `send_event(..).to(relays)`, -retains the ephemeral wrap key for a later NIP-09 scrub, and echoes its own wrap through the -same ingest path so send-then-read never waits on a relay round-trip. - -## 11. Integration with existing crates - -1. **`crates/chat/src/lib.rs` — required fix, moved from M2 to the milestone that first subscribes.** `handle_notifications` currently treats *every* kind 1059 event as a NIP-59 gift wrap for the current user and pushes an unwrap failure into the trash. Concord wraps are kind 1059 with an ephemeral `p` tag, so they would flood the trash and leak error toasts. Route by `subscription_id` from `RelayMessage::Event` against `sub_id1`/`sub_id2`, and drop the `if rumor.tags.is_empty()` recipient heuristic. **M2 and M3 did not apply it**: the crate has no subscription and no `ConcordRegistry` yet, so no concord wrap can reach that handler and the change would be untestable. It lands with the sync engine (§10), as does the `concord::init` wiring in `desktop` and `web`. -2. **`desktop/src/main.rs` and `web/src/lib.rs`** — add `concord::init(window, cx)` after `chat::init(window, cx)`. -3. **`Cargo.toml`** — add `hkdf = "0.12"` to `[workspace.dependencies]`; add the crate to `desktop` and `web` dependencies. No other workspace changes. -4. **No changes** to `state`, `person`, `device`, `settings`, `common`, or `ui`. - -## 12. Security invariants to test, not to assume - -Each of these has burned a real implementation, or is a documented cross-client trap: - -- Recompute every rumor id and reject a claimed mismatch; never trust an embedded `id`. -- Require `rumor.pubkey == seal.pubkey`. -- Require the plaintext seal form on Control and the encrypted form on Chat/Guestbook/Rekey — a strict reader must drop a mis-sealed edition rather than fold a chain a later compaction would fork. -- Check `channel` **and** `epoch` against the plane whose key opened the wrap; reject duplicates of either tag. -- Reject duplicate `vsk`/`eid`/`ev`/`ep`/`vac` tags; require decimal-with-no-leading-zeros on every numeric tag. The one exception is `ms`, which takes its first value rather than erroring — see §7 for why rejecting it reached membership. -- Refuse a tombstone whose `eid` is not this community's id. -- Adopt a `control_root` from a Grant only if it derives to the `control_pk` held for that epoch; adopt a rekey blob only if its bound plaintext matches the scope and epoch and its `prevcommit` matches the key currently held. -- Never conclude removal from a partial rekey chunk set. -- Drop guestbook entries more than an hour in the future; treat an out-of-range `ms` as malformed, not as an interpretation opportunity. -- Never honour a Snapshot from anyone but the refounder of that epoch. -- Refuse to write a Pin List from a list the writer could not read. -- Enforce the NIP-44 65 535-byte cap at every layer before publishing, and the 5-relay / 50-membership / 100-role / 64-role-per-member / 500-banlist / 25-pin caps at their ingest and write points. -- Lowercase hex only; x-only pubkeys only; no version tag anywhere. -- **Enforced in M3:** a Role's `role_id` is its own coordinate and never 0; a Grant's `member` hashes to its coordinate; a `vsk 4` sits at this community's banlist locator; a banned npub's editions are dropped and a grant naming them carries no rank; a revocation carries a citation; the 100-role cap keeps the lowest ids *after* authorization; a below-floor edition is never a candidate. -- **Enforced in M4:** the chat plane's encrypted-seal requirement, at both publish and open; `channel` **and** `epoch` strict-equal to the plane whose key opened the wrap; a retired or unregistered rumor kind rejected on both sides; a target bearing tag that appears twice rejected outright; and a delete from anybody other than the message's own author refused by the caller's gate. -- **Enforced in M5:** a Kick counts only where the roster admits its actor under `KICK` with a strictly higher rank and a resolvable citation; a Snapshot counts only from the epoch's refounder, with no owner fallback; a guestbook entry more than an hour ahead is dropped, and an out-of-range `ms` or a non-verb `3306` entry is malformed, not interpreted; a duplicated `vac` is rejected outright; and a delete is honored from its target's author unconditionally, or from another actor only where `can_delete` admits them. -- **Enforced in M6:** an invite fragment whose version is not this one is refused in either direction, as is one with a bad count or trailing bytes, and its encoding caps bootstrap relays at three; a bundle past the channel cap, past the epoch ceiling, or carrying a secret that is not 32 bytes of hex is refused before it is used, and one whose `owner` + `owner_salt` does not reproduce its `community_id` is refused outright; a bundle event off its coordinate, off its author, or unsigned is refused, and a tombstone at the coordinate reads as revoked; a Direct Invite's wrap, its seal signature and its rumor/seal author bind are all verified before the bundle is even parsed; and a Community List refuses to build past its membership cap or the NIP-44 plaintext cap. -- **Enforced in M7:** a blob's bound scope and epoch are checked inside the ciphertext, so a channel blob cannot be opened under the base scope or under another epoch, and a 136-byte base blob whose secret does not derive to the pk beside it is refused whole rather than adopting a split control plane; the locator is never gated, because it derives from public keys and proves nothing; a removal is never concluded from a partial chunk set, and two chunks claiming one index union their blobs rather than letting the loser's recipients read as removed; a rotation needs its permission and must strictly outrank every target, so a rotator holding the prior root or a demoted staffer holding the `control_root` is dropped; a plaintext-sealed rekey is refused; and a tombstone is refused unless its signed `eid` is this community's own id — the all-zero placeholder and a sibling community of the same owner included. -- **Enforced in M8:** a plaintext seal carries no payload to disclose, so it cannot be pinned; a pin whose disclosure does not open its own seal is refused before it costs list budget, and a built entry is run through the same verification every reader applies; a `channel` or `epoch` bound to another Channel, a claimed rumor id that is not its own, another author's Edit, and a delete from anybody but the pin's own author are each refused; a list past 25 entries or 32 768 content bytes is refused on the write side and reads as EMPTY on every reader's; a sealed list that cannot be opened reads as darkness and never as an empty public one, and a writer can never re-form a list it could not read; a delete and a timer notice may not carry an `expiration` and a malformed `timer` is refused; an already-expired rumor is refused at ingest, dropped from every fold, and purged by the sweep. -- **The byte caps, closed out:** the NIP-44 plaintext cap is now checked inside `seal_bytes` as well as `seal_content`, so every raw envelope — the invite bundle, the sealed pin form, the List's to-self document — is inside it before it is published. 25 pins / 32 768 bytes; 100 roles and 64 roles per member, the member's cap also refusing the write; 500 banlist entries refused on the write side as well as capped in the fold; the 64-byte name cap refused on every community, channel and role write; 5 relays truncated on read as well as on write; 50 memberships in `list::fits`. The 256 in the old wording was the *invite bundle's* channel cap (M6), not a community-wide one — the spec states no community channel count, so none was invented. -- **Still owed:** the `vac`-carrying pin write and the 100-role write gate both need the roster, so both are the registry's (§10), as is the private→public re-form refusal (§14.13). - -## 13. Milestones - -| # | Deliverable | Done when | -| --- | --- | --- | -| M0 | Crate skeleton, `derive.rs`, golden vectors, workspace wiring | ✅ `cargo test -p concord` pins every derivation; all labels match Appendix A.6 | -| M1 | `stream.rs` + `store.rs` | ✅ seal/wrap/open round-trips for both seal forms; hostile wraps rejected in the documented order; the local cache reads back with the group key gone | -| M2 | `edition.rs` + `control.rs` genesis + `store.rs` state document | ✅ `cargo test -p concord` (7 tests): `edition_hash` reproduces the cross-client vector `2daf42e6…`, and a community minted by one holder has both genesis wraps open for a second holder holding only the invite keys, folding to version 1 | -| M3 | Control fold + roster + metadata/channels | ✅ `cargo test -p concord` (14 tests): the chain fold, its gaps, fork tiebreak, downgrade refusal and compaction dangle are pinned; the delegation fixpoint resolves outward from the owner and refuses escalation, an unauthorized higher version, rank inversion by republish and an uncited revoke; a community minted by one holder has its metadata and channel edits fold for a second holder from the invite keys alone | -| M4 | Chat plane | ✅ `cargo test -p concord` (19 tests): a second holder folds a message's reactions, its author's edit and its author's delete, and ignores an edit or a delete from anybody else; a comment's root and parent survive the wire; a foreign channel, a replayed epoch, a plaintext seal, a retired kind and a duplicated target are each rejected; and history pages backwards across a rekey in order | -| M5 | Guestbook + member list + moderation | ✅ `cargo test -p concord` (23 tests): a second holder folds joins, leaves, a cited kick and a chunked snapshot into one memberlist in either arrival order, with a ban and a Grant deciding the edges; an uncited, unranked or owner-directed kick and a foreign snapshot are dropped; a future-dated entry, a malformed `ms`, a non-verb `3306`, a duplicated `vac` and a bad snapshot chunk are each refused; and a moderator delete lands only under a citation the roster admits, while a self-delete never asks | -| M6 | Invites + Community List | ✅ `cargo test -p concord` (32 tests): the fragment's byte layout is pinned by golden base64url for the stock set, a dictionary mix and a verbatim literal, with a wrong version in either direction, trailing bytes and an over-cap count each fatal; a link round-trips as a full URL and as a bare naddr, and refuses a non-invite; a bundle round-trips while a revocation tombstone reads as revoked, and a wrong token, a squatter's author, a foreign `d`, a forged owner, a malformed secret and an over-cap channel count are each refused; a Direct Invite round-trips to its verified inviter and refuses a stranger's keys and a non-invite rumor; and the Community List keeps the earlier seed and the later current in either merge order, refuses to resurrect a tombstoned id until a newer join outruns it, and rebuilds on a second device with unknown fields intact while refusing an over-cap or oversized list | -| M7 | Rekeys + refounding + dissolution | ✅ `cargo test -p concord` (40 tests): a blob's bound scope and epoch are checked inside the ciphertext, so a channel blob cannot be opened under the base scope or under another epoch, and a staff secret that does not derive to the pk beside it refuses the whole blob; a removal is concluded only from a complete chunk set, and two chunks claiming one index union rather than drop a recipient's blob; continuity extends, gaps and forks, and the fork winner is the lowest key adopted only when it strictly lowers one already held; a rotation needs its permission and must strictly outrank every target, so holding a key is never authority; a full 80-blob chunk fits a 64 KB relay event and one more splits; compaction carries a settled head across a refounding with the original author's signature intact; and an owner's tombstone seals the community while an impostor's, the spec's all-zero `eid` and one re-wrapped from another community of the same owner are each refused | -| M8 | Pins + disappearing messages + hardening | ✅ `cargo test -p concord` (49 tests): a disclosed 76-byte expansion opens exactly the message it was derived from and nothing else, pinned by a round-trip against nostr's own encrypt; a built entry proves its author and its words while tampered keys, a re-signed seal, a claimed id that is not its own and a Channel it does not belong to are all refused; a proven Edit replaces the words and a stranger's never attaches; both list forms round-trip with a sealed one dark without its key and lit with it, 26 entries refusing to build and reading back as empty, and a writer never re-forms a list it could not read; a Pin List folds under a derived coordinate for a second client while a neighbouring Channel reads none; a timer tag rides a durable rumor and its wrap while a delete and a notice carrying it are refused; the metadata timer is set, off and garbage without poisoning the entity; an expired rumor is refused at ingest and purged by the sweep while an untimed one never is; and the banlist, grant, channel-name and relay caps hold on the way out | - -Ordering is deliberately dependency-first: each milestone is usable on its own, and nothing in M2+ depends on a later milestone. The sync engine and its GPUI wiring (§10, §11) have no row of their own because they are cross-cutting: every plane they consume has to exist first, so they follow M8, and they are the milestone that applies §11's `chat::handle_notifications` routing fix. - -M1 closed with `cargo test -p concord` (5 tests), `cargo clippy -p concord --all-targets`, and `cargo fmt -p concord --check` all clean. `rand` was added to the workspace pinned to the same `0.10.2` instance `nostr` already builds, so `Cargo.lock` gained no package. - -M2 closed the same way at 7 tests, with `serde` added to the crate's dependencies (`serde_json` promoted from dev to main for the metadata content types) — `Cargo.lock` gained no package again, only the `concord → serde` edge. - -M3 closed at 14 tests with no dependency change at all, and `Cargo.lock` untouched. New: `src/roles.rs` (permissions, Role/Grant/banlist content, `CommunityRoles`, the delegation fixpoint) and, in `src/control.rs`, `ControlFold` / `fold_control`, the metadata-and-channel fold, `ControlWriter` and its `Edition` input. `EntityHead` and `Floors` moved from `store.rs` into `edition.rs`, where `fold_head` now composes `fold` and `bootstrap_head` for the floor-aware case. - -M4 closed at 19 tests, again with no dependency change and `Cargo.lock` untouched — relay paging is `Client::fetch_events` over the same `NostrDatabase` the cache already used, so nothing new was needed. New: `src/chat.rs` (the whole channel plane) and, in `src/store.rs`, `backfill` plus the pure `advance` page step it is built from, which is what the paging test drives instead of a socket. `edition::canonical_decimal` became `pub(crate)` so the chat tag grammar shares one decimal check. - -What M3 still defers, and to what: the **sync engine's paging** driven by `ControlFold.gapped` and the **`chat::handle_notifications` routing fix** (both §10, together with the `concord::init` wiring — no concord wrap can reach that handler until the subscription exists); and the **NIP-46 remote signer**, since every writer takes `&Keys` rather than a `NostrSigner`. Its persisted banlist and its role/grant/banlist write wrappers both landed in M5. - -What M4 still defers, and to what: **`media`/`mentions`** on `ChatMessage` and **`send_file`** (the registry/UI milestone — the first needs a gpui type and the second needs the blob-upload path); and **the timer's policy** under the `expiration` tag that `seal_rumor` already mirrors (M8). Its moderator delete landed in M5. - -M5 closed at 23 tests, again with no dependency change and `Cargo.lock` untouched. New: `src/guestbook.rs` (the whole membership plane — three rumor codecs, the coalesce, the memberlist), a `vac` citation on a kind-5 delete plus the fold's `can_delete` gate, `ControlWriter::set_role`/`set_grant`/`set_banlist`, and `CommunityState.banned`. Two shared helpers moved into `edition.rs` — `citation_tag` and `citation_from` — so the control, chat and guestbook grammars parse one `vac`. Every metadata/role/grant/banlist wrapper now takes its citation, which is a fix rather than an addition: M3's wrappers hardcoded `None` and so were owner-only, which the delegated-metadata test had been working around with `publish`. - -What M5 still defers, and to what: the **guestbook's fetch and ingest path** — `coalesce` is fed wraps by its caller, so the plane has no `backfill` twin of `chat::backfill` until §10's sync engine needs one, and no `CommunityState.observed` to persist what it would learn; the **banlist head's timestamp** that `complete_memberlist`'s `banned_at` wants (the fold does not surface it, so an empty map means "every ban is terminal" until the registry plumbs it); and the **`Refound` seed** argument to `complete_memberlist`, which waits for M7 to mint one. - -M6 closed at 32 tests, again with no dependency change and `Cargo.lock` untouched — the fragment codec is `data_encoding::BASE64URL_NOPAD` and the Direct Invite is nostr's own `nip59` builder and unwrapper, so nothing new was needed. New: `src/invite.rs` (bundle, link, fragment, Direct Invite) and `src/list.rs` (the Community List). In `stream.rs`, the seal and open primitives were factored out of `seal_content`/`decode_content` as `seal_bytes`/`open_bytes`, so the bundle's raw-token key and the List's to-self envelope share the crate's one NIP-44 shape instead of re-implementing it twice; `edition::TAG_SUBKIND` became public so the invite sub-kind tags do not restate the string. - -What M6 still defers, and to what: the **Invite List (13303) and the Registry (`vsk 8`)** — both landed in M7, as planned; the **join gate that refuses a 51st membership** (the registry's, since `fits` protects the write rather than the add); and the byte-level cap audit, which is M8's. - -M7 closed at 40 tests, again with no dependency change and `Cargo.lock` untouched. New: `src/rekey.rs` (the blob atom, the 3303 chunk set and its collection, continuity and the fork winner, the authority gate, refounding planning and compaction, and dissolution). The Invite List landed in `src/invite.rs` beside the bundle it bookkeeps, and the Registry became a Control Plane entity: `ControlFold.registries` keyed by creator, folded under `CREATE_INVITE`, its aggregate exposed as `ControlFold::is_public`, with `ControlWriter::set_registry` as the write side. `invite_links_locator` and M6's revocation machinery needed no change. In `stream.rs`, `rewrap_seal` gained a separate `signer` group, because a split epoch reads under the rolled root but wraps as the new control signer. In `store.rs`, `CommunityState.dissolved` records the seal; `list.rs`'s `canonical`/`union` became `pub(crate)` so the Invite List's merge shares them rather than restating the same total order. - -M8 closed at 49 tests. New: `src/pins.rs` (the disclosure primitive, the entry codec and its full verification, the Edit bundle, both content forms and their caps) and, in `control.rs`, `ControlFold.pins` keyed by locator with `pin_content`/`set_pin_list`, plus the metadata timer and the write-side caps. `chat.rs` gained kind 1740, `ChatAction::TimerNotice`, the `timer` argument on the four durable builders, the exemption refusal, `expired`, and a `now` parameter on `fold`; `store.rs` gained the ingest refusal and `purge_expired`; `stream.rs` gained the plaintext-cap check inside `seal_bytes`; `roles.rs` gained the 64-role write gate; and `lib.rs`'s `decode_hex_32` was generalized into `decode_hex_lower::` so the 76-byte disclosure shares the one canonical-hex check rather than restating it. This is the first milestone that touched `Cargo.lock`: two direct-dependency edges (`chacha20`, `hmac`), no new package (§3). Evidence for one line of §12 that was wrong: the 256-channel cap does not exist in the spec — the 256 was `invite::MAX_BUNDLE_CHANNELS` all along. - -What M7 still defers, and to what: **epoch key retention** — the blob atom delivers every plane key a refounding mints, but nothing can persist one, because `ChannelKeyRef` is `{id, name, private, epoch}` with no key field (§14.11); the **rekey subscription**, precomputed from the next epoch's address for every held private channel plus the base, which is a `Filter` and belongs with the sync engine (§10); the **Server-Severing flow** that would set `severed`, which awaits the invite-link wiring; and the **`Refound` seed** to `complete_memberlist`, which M7 can now mint but whose consumer is still the guestbook ingest of §10. - -What M8 still defers, and to what: the **`vac`-carrying pin write, the 100-role write gate and the timer notice's roster gate** — all three need the roster, so they are the registry's (§10); the **private→public re-form refusal**, which cannot be encoded at the list level (§14.13); the **re-heal and deletion-omission pin writes** and the automatic Edit refresh, which are pins §7 behaviours a curator flow drives; and everything §10 already owed — the sync engine, the `chat::handle_notifications` routing fix, and the §14.11 key-retention schema change. - -**M2's "created and published" is verified offline**: "published" is the two wraps existing and being openable by the invite keys, not a relay round-trip. There is no registry to publish through until §10, and a relay test would be testing the SDK, not the protocol. - -## 14. Open questions and risks - -1. **Community List kind.** CORD-02 §8 specifies `13302`, replaceable. Vector has retired it in favour of fragmented `33302`, because a replaceable kind holds one event per pubkey and so cannot shard past the NIP-44 size cap. **M6 implements `13302` per spec**, with the 50-membership cap and the pre-publish size check as the write gate (`list::CommunityList::fits`); `33302` remains an interop follow-up, so a coop member's list is invisible to a Vector device until it lands. Confirm the sharded form with Armada before writing it. -2. **NIP-42 for stream-authored REQs.** Relays that gate kind 1059 by author (for example `ditto-relay`'s `AUTH_KINDS`) need an AUTH event signed by that plane's derived key. `nostr-sdk`'s `Authenticator` is per-client and returns one identity, while a Concord client holds many plane keys — so this cannot be solved by swapping the authenticator. Vector's answer is a dedicated stream-auth responder installed on the client (`community/v2/streamauth`), primed before any relay interaction, which retains the relay's challenge so plane keys registering later can still answer it: a gating relay challenges once per connection and will not re-challenge an authed one, so a responder attached later never gets the chance. Read that module before deciding; the alternative options remain a per-REQ auth hook upstream or documenting the limitation. -3. **`invite_bundle_key` — resolved in M0.** Appendix A.6 was read in full: the raw HKDF output *is* the NIP-44 conversation key, and the derivation is now pinned by a vector. -4. **`pins_locator` has no upstream vector.** Resolved in M0 by minting one from our own implementation and flagging it self-referential in the test. -5. **Relay set.** Up to 5 recommended, and both reads and writes fan out across them. Coop's client is a gossip client with `no_background_refresh`, so community relays must be added explicitly and re-added on metadata change. -6. **Local plaintext state.** §9 records the decision. Revisit only if the local database stops being treated as trusted. -7. **Was a `community_id` ever hashed into a tag?** No — it must never appear on the wire. A lint-style test should assert it appears in no published event other than inside an invite bundle and a direct invite. -8. **The Pin List's message-key disclosure had no public API — resolved in M8.** `nostr`'s `nip44::v2::get_message_keys` is a private `fn`, so the expansion is reproduced in `pins.rs` over the same audited primitives — `hkdf`'s `from_prk`/`expand`, `hmac`'s constant-time `verify_slice`, `chacha20` — as two direct-dependency lines on packages already in the graph (§3), pinned by a round-trip against nostr's own `encrypt_to_bytes_with_nonce`. That test is the whole contract: if the reproduction ever drifts, it fails loudly rather than silently unverifying every pin. A `pub` accessor upstream remains strictly better than a copy we must keep in sync and is still a viable PR, but it is now an improvement rather than a blocker. -9. **A remote signer is not plumbed.** `ControlWriter::publish`, `stream`'s seal builders and the invite builders all take `&Keys`. NIP-46 is a stated Vector feature (§10's task slots are already cleared on signer change), but making the writers async over a `NostrSigner` is a change to every builder, so it should be one deliberate pass rather than a patch per milestone. Nothing in M4–M8 depends on it except the UX of using a remote signer at all. -10. **The fold is not incremental.** `fold_control` re-parses and re-folds the whole control edition window on every call, and each fold is up to `2 × entities + 8` passes. That is fine at the caps the spec sets (100 roles, 400-odd grants) and it is the simplest thing that is correct, but if the sync engine ends up calling it per event rather than per batch, the candidate maps and their parse belong in a cache keyed by edition id. Measure before optimizing. -11. **No plane key can be persisted (found in M7).** The rekey blob atom hands a receiver every key a rotation mints — the next `community_root`, the `control_root`, and a private channel's fresh key — and `CommunityState` has nowhere to put any of them: `ChannelKeyRef` is `{id, name, private, epoch}`, and the state's only root fields are the *current* `community_root`/`root_epoch`. So a client can verify a rotation and still lose it on restart, and it cannot read history written under a prior root or a prior channel epoch. M7 therefore left `epoch_keys` out rather than add a field with no key to hold. The fix is a schema change — `ChannelKeyRef` gains the key and its retired `priors`, and the state gains a root-per-epoch map — and it belongs with the sync engine that reads them back, since it changes `apply_fold` and the `13302` join material together. Armada already carries `priors` for exactly this reason. -12. **`message_expiration` normalization is lossy, deliberately.** A `0`, a float or a string all fold to `None`, and a republish of that fold writes the field away rather than carrying the garbage through. CORD-08 §1 says malformed means off and a reader must not guess, so the value is uninterpretable by construction — but every other content struct in this crate round-trips bytes it does not understand via `extra`, and this field does not. Nothing depends on the distinction yet; giving an uninterpretable timer an `extra` slot is not worth it until something does. -13. **The private→public Pin List rule has no encoding.** §7 says a list MUST NOT be *mechanically* re-formed across a private→public conversion, because the pre-switch entries are private-era and a re-form republishes them community-wide — but "mechanically" and "deliberately" are the same bytes, so no reader-side check can tell them apart and `pins::publishable` does not pretend to. It refuses only what it can prove: a list it could not read. The refusal is therefore a duty of the conversion flow (§10), which must not auto-republish on a `private: false` metadata change. - -## 15. Test strategy - -- **Unit, pure:** derivations against golden vectors, edition hash, fold, coalesce, memberlist, blob codec, the NIP-44 disclosure, caps and rejection paths. These need no GPUI context and should be exhaustive — they are where cross-client divergence is caught. -- **Integration, GPUI:** `TestAppContext` with two registries sharing an in-memory database, driving wraps through the ingest path; timeouts and delays use `cx.background_executor().timer(..)` per the project guidelines, never `smol::Timer`. -- **Round-trip:** every builder paired with its parser, asserting the parse produces the identical structure, including unknown-field round-tripping on all content types. -- **Negative:** every bullet in §12 gets a test that constructs the hostile input and asserts the drop. diff --git a/crates/concord/src/chat.rs b/crates/concord/src/chat.rs index b6106287..4480dc7a 100644 --- a/crates/concord/src/chat.rs +++ b/crates/concord/src/chat.rs @@ -44,8 +44,7 @@ pub enum ChatError { MissingTag(&'static str), DuplicateTag(&'static str), BadTag(&'static str), - /// A delete is a tombstone and a timer notice documents the policy, so - /// neither may be erased by the policy it carries. + /// Neither a delete nor a timer notice may be erased by the policy it carries. ExemptExpiration, } @@ -73,16 +72,14 @@ impl From for ChatError { } } -/// A chat event another chat event refers to: a quote, a comment's parent, a -/// reaction's target. The author slot is a SHOULD on the wire, so it is optional. +/// A chat event another chat event refers to: a quote, a comment's parent, a reaction's target. #[derive(Debug, Clone, Copy, PartialEq, Eq)] pub struct ReplyRef { pub id: EventId, pub author: Option, } -/// A reference that also names the referenced event's kind, which a comment -/// (`K`/`k`) and a reaction (`k`) must commit on the wire. +/// A reference that also names the referenced event's kind, which `K`/`k` must commit on the wire. #[derive(Debug, Clone, Copy, PartialEq, Eq)] pub struct Target { pub reply: ReplyRef, @@ -166,8 +163,7 @@ pub fn build_message( build_rumor_ms(KIND_MESSAGE, author, content, tags, at_ms) } -/// A NIP-22 comment. `parent` is the immediate parent and `root` the thread's -/// immutable root; `None` means the parent is itself the root. +/// `parent` is the immediate parent; a `None` root means the parent is the thread's root. #[allow(clippy::too_many_arguments)] pub fn build_comment( author: PublicKey, @@ -238,8 +234,7 @@ pub fn build_edit( build_rumor_ms(KIND_EDIT, author, content, tags, at_ms) } -/// CORD-08 §4: an informational row in the timeline, gated by the roster rather -/// than by the fold, so it is built like any other chat rumor. +/// CORD-08 §4: informational, gated by the roster rather than by the fold. pub fn build_timer_notice( author: PublicKey, channel: &ChannelId, @@ -253,8 +248,7 @@ pub fn build_timer_notice( build_rumor_ms(KIND_TIMER_NOTICE, author, "", tags, at_ms) } -/// The tag is derived from the rumor's own signed `created_at`, so a later -/// metadata edit can never reach back into history. +/// Derived from the signed `created_at`, so a later metadata edit never reaches back. fn expiration_tag(at_ms: u64, timer: Option) -> Option { timer.map(|timer| Tag::custom(TAG_EXPIRATION, [(at_ms / 1000 + timer).to_string()])) } @@ -297,8 +291,7 @@ pub fn build_typing( ) } -/// Seals a chat rumor and wraps it at the channel's address. `ephemeral` picks -/// the 21059 wrap, which relays must not store. +/// `ephemeral` picks the 21059 wrap, which relays must not store. pub fn seal_rumor( rumor: &UnsignedEvent, group: &GroupKey, @@ -318,8 +311,7 @@ pub fn seal_rumor( KIND_WRAP }; - // CORD-08 §2: a NIP-40 expiration rides the wrap as well, so relays drop the - // stored event on schedule; the inner copy is what drives a local purge. + // The wrap's copy is for relays; the inner one drives the local purge. let expiration: Vec = rumor .tags .iter() @@ -336,9 +328,7 @@ pub fn seal_rumor( )?) } -/// Opens a wrap against the plane whose key is tried. The channel and epoch the -/// rumor claims must both be the ones that opened it, so a keyholder of two -/// planes cannot re-seal a rumor elsewhere or replay it across an epoch. +/// The claimed channel and epoch must both be the ones that opened the wrap. pub fn open( wrap: &Event, group: &GroupKey, @@ -358,9 +348,7 @@ pub fn open( Ok((opened, chat)) } -/// Every epoch's group key for one channel. `secret` is whatever feeds the -/// channel at that epoch: the `community_root` for a public one, its own key -/// for a private one. +/// `secret` is the `community_root` for a public channel, its own key for a private one. pub fn plane_keys( held: &[(Epoch, [u8; 32])], channel: &ChannelId, diff --git a/crates/concord/src/control.rs b/crates/concord/src/control.rs index 323c8e64..2f8785bb 100644 --- a/crates/concord/src/control.rs +++ b/crates/concord/src/control.rs @@ -63,8 +63,7 @@ pub struct CommunityMetadata { pub extra: Extra, } -/// CORD-08 §1: absent, `0` and malformed all mean off, and a reader must not -/// guess a default from garbage. +/// CORD-08 §1: absent, `0` and malformed all mean off. fn timer_seconds<'de, D>(deserializer: D) -> Result, D::Error> where D: serde::Deserializer<'de>, @@ -193,7 +192,6 @@ pub fn open_edition( Ok(parse_edition(&opened.rumor)?) } -/// Appends editions to entity chains. pub struct ControlWriter { pub author: PublicKey, pub read: GroupKey, @@ -204,9 +202,7 @@ pub struct Edition<'a> { pub subkind: &'a str, pub entity: [u8; 32], pub content: &'a str, - /// The head this edition supersedes. - /// - /// `None` starts the chain. + /// The head this edition supersedes; `None` starts the chain. pub head: Option<&'a EntityHead>, pub citation: Option, } @@ -400,8 +396,7 @@ impl ControlWriter { ) } - /// `content` is the whole Pin List, in whichever of CORD-04 §7's two - /// self-describing forms the Channel's folded type calls for. + /// The whole Pin List, in whichever of CORD-04 §7's two forms the Channel calls for. #[allow(clippy::too_many_arguments)] pub fn set_pin_list( &self, @@ -455,8 +450,7 @@ pub struct ControlFold { pub channels: BTreeMap, /// Each creator's live link-signer set. pub registries: BTreeMap>, - /// Head content per `pins_locator`: a Pin List is addressed by a one-way - /// coordinate, so a fold cannot name the Channel it belongs to. + /// Head content per `pins_locator`; the coordinate is one-way, so a fold cannot name its Channel. pub pins: BTreeMap<[u8; 32], String>, pub floors: Floors, pub gapped: bool, @@ -533,8 +527,7 @@ fn fold_metadata( for edition in editions { match edition.subkind.as_str() { - // A channel addressed at the community's own coordinate would share, and - // corrupt, the metadata chain's floor. + // A channel at the community's own coordinate would corrupt the metadata chain's floor. vsk::COMMUNITY_METADATA if edition.entity == community_entity => { community.push(edition) } @@ -557,8 +550,7 @@ fn fold_metadata( fold.community = serde_json::from_str::(&head.content) .ok() .map(|mut metadata| { - // Up to 5 relays is a recommendation, so a longer set is - // truncated rather than refused, on read as well as on write. + // Up to 5 relays is a recommendation, so a longer set is truncated, not refused. metadata.relays.truncate(MAX_RELAYS); metadata }); @@ -590,10 +582,7 @@ fn fold_metadata( fold } -/// A Pin List's coordinate derives one-way, so unlike the banlist, a grant or a -/// registry there is nothing to check the `eid` against: an edition at an -/// unknown coordinate is simply never read. Its content is stored verbatim, -/// because a violating list still folds but reads as empty (CORD-04 §7). +/// A one-way coordinate leaves the `eid` unchecked; violating content reads as empty. fn fold_pins( judge: &Judge<'_>, editions: &[ParsedEdition], diff --git a/crates/concord/src/derive.rs b/crates/concord/src/derive.rs index 374bb342..8ee78ad5 100644 --- a/crates/concord/src/derive.rs +++ b/crates/concord/src/derive.rs @@ -1,6 +1,3 @@ -use std::collections::HashMap; -use std::sync::{LazyLock, Mutex, PoisonError}; - use anyhow::{Result, bail}; use hkdf::Hkdf; use nostr::nips::nip44::v2::ConversationKey; @@ -77,27 +74,11 @@ pub struct GroupKey { impl GroupKey { fn derive(label: &str, secret: &[u8], id32: &[u8; 32], epoch: Option) -> Result { - let key = memo_key(label, secret, id32, epoch); - - if let Some(hit) = lock_memo().get(&key) { - return Ok(hit.clone()); - } - - let info = build_info(label, id32, epoch); - let secret_key = hkdf_to_secret_key(secret, &info)?; + let secret_key = hkdf_to_secret_key(secret, &build_info(label, id32, epoch))?; let keys = Keys::new(secret_key); let conversation = ConversationKey::derive(keys.secret_key(), &keys.public_key())?; - let group_key = Self { keys, conversation }; - let mut memo = lock_memo(); - - if memo.len() >= 1024 { - memo.clear(); - } - - memo.insert(key, group_key.clone()); - - Ok(group_key) + Ok(Self { keys, conversation }) } pub fn pk(&self) -> PublicKey { @@ -125,27 +106,6 @@ impl std::fmt::Debug for GroupKey { } } -static MEMO: LazyLock>> = LazyLock::new(Default::default); - -fn lock_memo() -> std::sync::MutexGuard<'static, HashMap<[u8; 32], GroupKey>> { - MEMO.lock().unwrap_or_else(PoisonError::into_inner) -} - -pub fn clear_memo() { - lock_memo().clear() -} - -fn memo_key(label: &str, secret: &[u8], id32: &[u8; 32], epoch: Option) -> [u8; 32] { - let mut hasher = Sha256::new(); - hasher.update(label.as_bytes()); - hasher.update([0x00]); - hasher.update(secret); - hasher.update(id32); - hasher.update(epoch.unwrap_or(u64::MAX).to_be_bytes()); - hasher.update([epoch.is_some() as u8]); - hasher.finalize().into() -} - /// `secret` is the `community_root` for a public channel. pub fn channel_group_key(secret: &[u8; 32], channel: &ChannelId, epoch: Epoch) -> Result { GroupKey::derive(LABEL_CHANNEL, secret, channel.as_bytes(), Some(epoch.0)) @@ -165,8 +125,7 @@ pub fn control_group_key( ) } -/// The plane's address and wrap signer, held only by staff. -/// Wraps still encrypt under [`control_group_key`]. +/// The plane's address and wrap signer, held only by staff; wraps still read under [`control_group_key`]. pub fn control_signer_group_key( control_root: &[u8; 32], community_id: &CommunityId, @@ -180,9 +139,7 @@ pub fn control_signer_group_key( ) } -/// Member-writable, unlike the Control Plane: -/// -/// - A join or a leave is each member's own word. +/// Member-writable, unlike the Control Plane: a join or a leave is each member's own word. pub fn guestbook_group_key( community_root: &[u8; 32], community_id: &CommunityId, @@ -196,8 +153,7 @@ pub fn guestbook_group_key( ) } -/// Keyed by the prior `community_root` rather than the channel key, -/// so any retained member recovers any epoch's rekey without a ratchet. +/// Keyed by the prior `community_root`, so any retained member recovers any epoch's rekey. pub fn channel_rekey_group_key( prior_root: &[u8; 32], channel: &ChannelId, diff --git a/crates/concord/src/edition.rs b/crates/concord/src/edition.rs index c5e070f4..c18c6cee 100644 --- a/crates/concord/src/edition.rs +++ b/crates/concord/src/edition.rs @@ -1,3 +1,4 @@ +use std::cmp::Reverse; use std::collections::BTreeMap; use std::fmt; @@ -55,8 +56,7 @@ impl fmt::Display for EditionError { impl std::error::Error for EditionError {} -/// A `vac` citation: the Grant edition an actor claims rank under, pinned by -/// coordinate, version and hash. It is a sync floor, not the verdict. +/// A `vac`: the Grant edition an actor claims rank under, pinned by coordinate, version and hash. #[derive(Debug, Clone, Copy, PartialEq, Eq)] pub struct AuthorityCitation { pub entity: [u8; 32], @@ -312,16 +312,7 @@ pub fn bootstrap_head(editions: &[EditionMeta]) -> Option { editions .iter() .enumerate() - .reduce(|(best_index, best), (index, candidate)| { - let supersedes = candidate.version > best.version - || (candidate.version == best.version && candidate.tiebreak_id < best.tiebreak_id); - - if supersedes { - (index, candidate) - } else { - (best_index, best) - } - }) + .min_by_key(|(_, edition)| (Reverse(edition.version), edition.tiebreak_id)) .map(|(index, _)| index) } @@ -554,18 +545,8 @@ mod tests { "2daf42e65a6bc259a4c99fac6df754a5d3d92310607cf13e2a1e8c94d42f6303" ); - // The golden vector only exercises the absent-prev encoding; pin the - // present-prev branch structurally so a swapped flag stays visible. + // The golden vector only exercises the absent-prev encoding, so pin the flag. let bytes = signing_bytes(&entity, 1, Some(&entity), b"hello"); - assert_eq!( - bytes.len(), - 8 + EDITION_LABEL.len() + 32 + 8 + 1 + 32 + 8 + 5 - ); - assert_eq!(&bytes[8..8 + EDITION_LABEL.len()], EDITION_LABEL); - assert_eq!( - bytes[8 + EDITION_LABEL.len() + 32..][..8], - 1u64.to_be_bytes() - ); assert_eq!(bytes[8 + EDITION_LABEL.len() + 32 + 8], 1); } } diff --git a/crates/concord/src/invite.rs b/crates/concord/src/invite.rs index 022cd0a9..5191b8f7 100644 --- a/crates/concord/src/invite.rs +++ b/crates/concord/src/invite.rs @@ -6,7 +6,6 @@ use data_encoding::BASE64URL_NOPAD; use nostr::nips::nip01::Coordinate; use nostr::nips::nip19::{Nip19, Nip19Coordinate}; use nostr::nips::nip44::v2::ConversationKey; -use nostr::nips::nip44::{self, Version}; use nostr::nips::nip59::{GiftWrapBuilder, UnwrappedGift}; use nostr_sdk::prelude::*; use serde::{Deserialize, Serialize}; @@ -595,13 +594,7 @@ pub fn build_invite_list(keys: &Keys, list: &InviteList) -> Result Result for Epoch { - fn from(value: u64) -> Self { - Self(value) - } -} - -impl From for u64 { - fn from(value: Epoch) -> Self { - value.0 - } -} - impl fmt::Display for Epoch { fn fmt(&self, f: &mut fmt::Formatter<'_>) -> fmt::Result { write!(f, "{}", self.0) diff --git a/crates/concord/src/list.rs b/crates/concord/src/list.rs index bb8eb86d..0979359c 100644 --- a/crates/concord/src/list.rs +++ b/crates/concord/src/list.rs @@ -2,12 +2,11 @@ use std::collections::BTreeMap; use std::collections::btree_map::Entry; use std::fmt; -use nostr::nips::nip44::{self, Version}; use nostr_sdk::prelude::*; use serde::{Deserialize, Serialize}; use crate::invite::{ChannelGrant, CommunityInvite}; -use crate::stream::NIP44_MAX_PLAINTEXT; +use crate::stream::{self, NIP44_MAX_PLAINTEXT}; use crate::{CommunityId, Epoch, Extra}; pub const KIND_COMMUNITY_LIST: u16 = 13302; @@ -43,6 +42,12 @@ impl fmt::Display for ListError { impl std::error::Error for ListError {} +impl From for ListError { + fn from(error: stream::StreamError) -> Self { + ListError::Crypto(error.to_string()) + } +} + #[derive(Debug, Clone, PartialEq, Serialize, Deserialize)] pub struct JoinMaterial { pub community_id: CommunityId, @@ -182,13 +187,7 @@ pub fn build_list_event(keys: &Keys, list: &CommunityList) -> Result Result) -> fmt::Result { - f.write_str("MessageKeys()") - } -} - impl MessageKeys { pub fn to_hex(&self) -> String { let mut packed = [0u8; MESSAGE_KEYS_BYTES]; @@ -340,16 +334,14 @@ pub fn verify_entry(entry: &PinEntry, channel: &ChannelId) -> Option Option { let seal = &bundle.seal; - // Nobody else may revise another member's words, and this is checkable - // before any crypto. + // Checkable before any crypto: nobody else may revise another member's words. if seal.kind.as_u16() != stream::KIND_SEAL_ENCRYPTED || seal.pubkey != *original_author { return None; } @@ -632,11 +623,12 @@ mod tests { .expect("encrypts"); assert!(open_payload(&BASE64.encode(&other), &disclosure).is_none()); + let hex = disclosure.to_hex(); assert_eq!( - MessageKeys::from_hex(&disclosure.to_hex()), - Some(disclosure) + MessageKeys::from_hex(&hex).map(|keys| keys.to_hex()), + Some(hex.clone()) ); - assert!(MessageKeys::from_hex(&disclosure.to_hex().to_uppercase()).is_none()); + assert!(MessageKeys::from_hex(&hex.to_uppercase()).is_none()); } #[test] diff --git a/crates/concord/src/rekey.rs b/crates/concord/src/rekey.rs index fd955b3b..8709f63d 100644 --- a/crates/concord/src/rekey.rs +++ b/crates/concord/src/rekey.rs @@ -224,7 +224,9 @@ pub fn parse_blob_plaintext( }); } - if width != MEMBER_BASE_BLOB_LEN && width != STAFF_BASE_BLOB_LEN && width < STAFF_BASE_BLOB_LEN + // Between the frozen forms is malformed; wider is a future form, kept below. + if (CHANNEL_BLOB_LEN + 1..MEMBER_BASE_BLOB_LEN).contains(&width) + || (MEMBER_BASE_BLOB_LEN + 1..STAFF_BASE_BLOB_LEN).contains(&width) { return Err(RekeyError::BadBaseBlobWidth(width)); } diff --git a/crates/concord/src/store.rs b/crates/concord/src/store.rs index 97bd81b9..55052065 100644 --- a/crates/concord/src/store.rs +++ b/crates/concord/src/store.rs @@ -25,8 +25,7 @@ const WRAP_TAG: &str = "e"; const KIND_TAG: &str = "k"; const STATE_PREFIX: &str = "concord/"; -/// CORD-08 §3: an already-expired rumor is refused at ingest, never stored. -/// Returns whether the rumor was kept. +/// An already-expired rumor is refused at ingest. Returns whether it was kept. pub async fn cache_rumor( database: &dyn NostrDatabase, channel: &ChannelId, diff --git a/crates/concord/src/stream.rs b/crates/concord/src/stream.rs index 931c8cde..8da49232 100644 --- a/crates/concord/src/stream.rs +++ b/crates/concord/src/stream.rs @@ -107,8 +107,7 @@ pub fn split_ms(at_ms: u64) -> (u64, u16) { (at_ms / 1000, (at_ms % 1000) as u16) } -/// Build a rumor carrying a full epoch-ms time: `created_at` -/// holds the seconds and an `["ms", 0..=999]` tag the remainder. +/// Build a rumor carrying a full epoch-ms time: seconds in `created_at`, the remainder as `["ms", 0..=999]`. pub fn build_rumor_ms( kind: u16, author: PublicKey, @@ -198,6 +197,23 @@ pub fn open_bytes(conversation: &ConversationKey, content: &str) -> Result Result { + seal_bytes( + &ConversationKey::derive(keys.secret_key(), &keys.public_key()) + .map_err(|error| StreamError::Encrypt(error.to_string()))?, + plaintext, + ) +} + +pub fn open_to_self(keys: &Keys, content: &str) -> Result, StreamError> { + open_bytes( + &ConversationKey::derive(keys.secret_key(), &keys.public_key()) + .map_err(|error| StreamError::Decrypt(error.to_string()))?, + content, + ) +} + pub fn build_seal( rumor: &UnsignedEvent, form: SealForm, diff --git a/docs/concord-usage.md b/docs/concord-usage.md new file mode 100644 index 00000000..03599d02 --- /dev/null +++ b/docs/concord-usage.md @@ -0,0 +1,438 @@ +# Using the Concord backend + +`crates/concord` is a protocol crate: derivations, envelopes, folds and the local +state document. It has no GPUI dependency and owns no strings a user reads — the +UI layer decides every rendering. This document is the map from a UI action to +the calls it makes. + +A community is addressed by a `community_id` (never on the wire) plus three +secrets: `community_root` (read access — holding it *is* membership), +`control_root` (write access to the Control Plane, held by staff), and per-Channel +keys for private channels. Authority is a roster of owner-rooted signed grants, +folded independently by every client. + +## Modules + +| Module | Owns | +| --- | --- | +| `derive` | Every frozen HKDF derivation and coordinate | +| `stream` | The CORD-01 envelope: seal, wrap, open, and the NIP-44 helpers | +| `edition` | Chained, versioned editions: parse, hash, fold, floors | +| `roles` | Permissions, roles, grants, the banlist, the authority fixpoint | +| `control` | The Control Plane: genesis, the fold, the writer, metadata | +| `chat` | The Chat Plane: message/reaction/edit/delete builders and the fold | +| `guestbook` | Joins, leaves, kicks, snapshots, the member list | +| `invite` | Invite bundles, links, the Direct Invite, the Invite List | +| `list` | The Community List (a member's own memberships, across devices) | +| `rekey` | Key rotations, refounding, compaction, dissolution | +| `pins` | Pin Lists, and the key disclosure a keyless reader verifies | +| `store` | Local rumor cache, the community state document, relay paging | + +Read `CommunityId` as "this community", `ChannelId` as "this channel", `Epoch` as +"which key generation". Nothing else in the API needs internal state. + +## Creating a community + +```rust +use concord::control::{self, CommunityMetadata}; +use concord::store::{self, CommunityState, save_state}; + +let metadata = CommunityMetadata { name: "Room".into(), ..Default::default() }; +let minted = control::genesis(&owner_keys, &metadata, now_secs)?; + +// minted.identity — community_id, owner, owner_salt (verify() recomputes it) +// minted.wraps — the two owner-signed genesis editions, already sealed +// minted.channel_id — the #general channel +for wrap in &minted.wraps { + client.send_event(wrap).to(&relays).await?; +} +``` + +The owner then needs the folded state, which is also what every member does on +join: + +```rust +use concord::derive::{control_group_key, control_signer_group_key}; +use concord::edition::ParsedEdition; + +let read = control_group_key(&minted.community_root, &minted.identity.community_id, Epoch(0))?; +let signer = control_signer_group_key(&minted.control_root, &minted.identity.community_id, Epoch(0))?; +let editions: Vec = minted + .wraps + .iter() + .map(|wrap| control::open_edition(wrap, &read, &signer.pk(), true)) + .collect::>()?; + +let mut state = CommunityState::from_genesis(&minted, &editions, added_at_ms)?; +save_state(database, &state).await?; +``` + +Put the community's relay list into `state.relays` and add those relays to the +client explicitly — coop's client is a gossip client with no background refresh. + +## Joining + +An invite link resolves to a bundle: + +```rust +use concord::invite::{self, BundleState, invite_bundle_key}; + +let link = invite::parse_link(url)?; // link_signer, token, bootstrap_relays, naddr + +// The crate does no I/O: fetch the naddr from the fragment's relays, then: +let invite = match invite::parse_bundle_event(&event, &link.link_signer, &invite_bundle_key(&link.token))? { + BundleState::Live(invite) => invite, // validate() already ran + BundleState::Revoked => return Ok(None), // a tombstone at the coordinate +}; +``` + +A Direct Invite arrives as a NIP-59 gift wrap addressed to the member: + +```rust +let (inviter, invite) = invite::unwrap_direct_invite(&wrap, &my_keys)?; +``` + +Either way the invite carries `community_id`, `owner`, `owner_salt`, +`community_root`, `root_epoch`, `control_pk`, the granted `channels` +(`ChannelGrant { id, key, epoch, name }`) and the relay set. `invite.expired(now_ms)` +is a preview rule: a past expiry still renders, but the join is refused. + +Then publish a join so the member list sees the member before any backfill: + +```rust +use concord::derive::guestbook_group_key; +use concord::guestbook; + +let guestbook = guestbook_group_key(&invite.community_root, &invite.community_id, invite.root_epoch)?; +let rumor = guestbook::build_join(my_pk, Some((creator_npub, label)), now_ms); +let (wrap, _) = guestbook::seal_rumor(&rumor, &guestbook, &my_keys)?; +client.send_event(&wrap).to(&relays).await?; +``` + +## Reading the Control Plane + +```rust +use concord::control::{self, ControlFold}; + +let editions: Vec = wraps + .iter() + .filter_map(|wrap| control::open_edition(wrap, &read, &control_pk, true).ok()) + .collect(); + +let control: ControlFold = + control::fold_control(&owner, &community_id, &editions, &state.floors(), &state.banned); +state.apply_fold(&control); +``` + +`ControlFold` is everything the community UI needs: + +| Field | Use | +| --- | --- | +| `community` | name, description, icon, banner, `message_expiration` | +| `channels` | the channel list; `deleted: true` means drop it | +| `roles` | `role()`, `roles_of()`, `effective_permissions()`, `is_authorized()`, `is_staff()` | +| `banned` | the banlist | +| `registries` / `is_public()` | each invite creator's live link signers | +| `pins` / `pin_content(id, channel)` | Pin List content per channel | +| `floors` | the committed heads the next fold is judged against | +| `gapped` | a chain hole: refetch the Control Plane before trusting what is missing | + +`None` on `community` or a channel means "this client saw no authorized edition", +never "the value is gone" — keep what the state already holds rather than walking +the community backwards. Feed `state.floors()` and `state.banned` into the next +fold; they are its memory. + +## Sending a message + +```rust +use concord::chat::{self, build_message}; +use concord::derive::channel_group_key; + +let plane = channel_group_key(&community_root, &channel, epoch)?; // public channel +let rumor = build_message(my_pk, &channel, epoch, text, None, at_ms, timer); +let (wrap, wrap_key) = chat::seal_rumor(&rumor, &plane, &my_keys, false)?; +client.send_event(&wrap).to(&relays).await?; +``` + +- `epoch` is the channel's current epoch (`state.channels` carries it). A private + channel derives from its own key instead of `community_root`. +- `timer` is `control.community.message_expiration`; pass `None` when it is off. + The builder attaches the NIP-40 tag and `seal_rumor` mirrors it onto the wrap, + so relays drop the ciphertext too. +- `ephemeral: true` picks kind `21059` for typing indicators. Keep the returned + wrap key if the message may be deleted later — a kind-5 delete needs it. +- That `send_event` is the whole publish path; there is no optimistic echo. Feed + the wrap through the same ingest path the subscription uses so send-then-read + never waits on a relay round trip. + +`build_edit`, `build_reaction` and `build_delete` are the same shape, each a rumor +about an existing `EventId` rather than a mutation. + +## Reading a channel + +```rust +use concord::chat::{self, fold, plane_keys}; + +let planes = plane_keys(&held, &channel)?; // &[(Epoch, secret)] +let mut rumors = Vec::new(); + +for wrap in &wraps { + let Some((epoch, group)) = planes.iter().find(|(_, group)| group.pk() == wrap.pubkey) else { + continue; + }; + let Ok((opened, rumor)) = chat::open(wrap, group, &channel, *epoch) else { + continue; + }; + store::cache_rumor(database, &channel, &opened).await?; + rumors.push(rumor); +} + +let messages = fold(&rumors, Timestamp::now(), |actor, citation, author| { + citation_ok(&owner, &community_id, actor, citation, &control.roles.floors) + && control.roles.can_act_on_member(actor, &owner, author, Permissions::MANAGE_MESSAGES) +}); +``` + +`ChatMessage` carries `content`, `at_ms`, `edited_at`, `deleted`, `reactions`, +`reply_to` and `thread_root` already resolved. The fold drops expired rumors; a +`deleted` row is still returned so the timeline keeps its shape. + +Relay history pages through the local cache: + +```rust +let page = store::backfill(client, database, &channel, &held, until, 50).await?; +let cached = store::query_rumors(database, &channel, None, 50).await?; +``` + +`backfill` walks newest-first across every held epoch, caches what it opens, and +stops on a short page. `query_rumors` is the read path when the group keys are +gone. Run `store::purge_expired(database, &channel, now)` on the same cadence as +any other local sweep — the timer is cooperative, so the local store is the +artifact that has to forget. + +`ChatAction::TimerNotice { seconds }` is a policy notice, not a message: render it +as an inline row only when its author passes +`control.roles.is_authorized(&author, &owner, Permissions::MANAGE_METADATA)`. + +## Membership + +```rust +let states = guestbook::coalesce(&rumors, now_ms, Some(&refounder_pk), |actor, target, citation| { + citation_ok(&owner, &community_id, actor, citation, &control.roles.floors) + && control.roles.can_act_on_member(actor, &owner, target, Permissions::KICK) +}); +let members = guestbook::complete_memberlist(&states, &observed, &granted, &control.banned, &BTreeMap::new()); +``` + +- `observed` is npub → ms for every author this client has seen publish anything + usable, which is what makes a member visible before their Join arrives. Only + count it forward. +- `granted` is every npub the roster ranks; they are members with no Guestbook + entry at all. +- `banned_at` is empty today, so a ban is terminal in the fold. Fill it when the + banlist head's timestamp is plumbed through. +- Removal is three separate actions, composed by the caller: strip the grant + (immediate and cheap), then the kick directive, then — for a ban — the rotation + that actually enforces it. + +## Moderation writes + +Every Control Plane write goes through one writer and one edition shape: + +```rust +let writer = ControlWriter { author: my_pk, read: read.clone(), signer: signer.clone() }; +let head = control.floors.get(entity).cloned(); + +let (wrap, new_head) = writer.set_community_metadata( + &my_keys, &community_id, &metadata, head.as_ref(), citation, now_secs)?; +``` + +`citation` is the `vac` the actor acts under — `None` only for the owner. Build it +from the folded Grant that ranks them (`AuthorityCitation { entity, version, hash }`) +and pass the head from the current fold, so the chain cannot silently fork. + +Wrappers: `set_community_metadata`, `set_channel_metadata`, `set_role`, +`set_grant`, `set_banlist`, `set_registry`, `set_pin_list`, plus raw `publish`. +A ban is a `set_banlist` followed by a base rekey; a kick is a `set_grant` with an +empty `role_ids` followed by `guestbook::build_kick`. + +## Pins + +```rust +use concord::pins; + +let entry = pins::build_entry(&opened_message, &plane, &channel)?; +let head_content = control.pin_content(&community_id, &channel).unwrap_or(""); +let read = pins::read_list(head_content, |epoch| channel_group_key(&root, &channel, epoch).ok()); +let content = pins::publishable(&read, channel_is_private, &plane, epoch)?; +let (wrap, _) = writer.set_pin_list( + &my_keys, &community_id, &channel, &content, head, citation, now_secs)?; +``` + +Reading is verification: `read_list` decodes either content form (public, or +sealed under the channel key at the named epoch), and +`pins::verify_entry(entry, &channel)` returns a `VerifiedPin` with the proven +author, words and time — no history and no old keys needed. `read.sealed` means +the list is sealed under an epoch this client never held: show it as unavailable, +and never write from it (`publishable` refuses). `pins::killed_by(&pin, &delete)` +answers whether a folded kind-5 erases an entry. + +## Invites + +```rust +use concord::derive::{invite_bundle_key, TOKEN_LEN}; +use concord::invite::{self, InviteEntry, InviteTombstone}; + +let token: [u8; TOKEN_LEN] = /* 16 bytes from any CSPRNG */; +let bundle_key = invite_bundle_key(&token); +let link_signer = Keys::generate(); +let bundle = invite::build_bundle_event(&link_signer, &invite, &bundle_key)?; +let url = invite::build_invite_url(BASE, &link_signer.public_key(), &token, &relays)?; +``` + +A link is a coordinate plus a fragment: the naddr fetches the bundle, the token +unlocks it, and the fragment names the relays to fetch from. +`invite::stock_relays()` is what a fragment with no relays of its own means. + +The `link_signer` secret is what lets the creator refresh or retire the link, so +keep it against the token in the member's own Invite List — a local document +encrypted to self, exactly like the Community List: + +```rust +let mut list = invite::parse_invite_list(&my_keys, &event)?; +list.entries.push(InviteEntry { + token: HEXLOWER.encode(&token), + signer_sk: link_signer.secret_key().to_secret_hex(), + community_id: invite.community_id, + url, + label: None, + created_at: now_ms, + expires_at: None, + extra: Default::default(), +}); +let event = invite::build_invite_list(&my_keys, &list)?; // kind 13303 + +// Retiring is a tombstone, never a deletion: it beats a stale copy terminally. +list.tombstones.push(InviteTombstone { + token: HEXLOWER.encode(&token), + community_id: invite.community_id, + extra: Default::default(), +}); +``` + +`merge_invite_lists` merges two devices' copies, `is_live(&token_hex)` answers +whether a link still stands, and `fits()` is the write gate. + +## Rekeys, refounding and dissolution + +A rotation is authority plus delivery: `rekey_authorized(&control.roles, &owner, &me, permission, &removed)` +gates it, `plan_refounding(epoch)` mints the new pair, and `build_rekey_chunks` +seals one blob per remaining member: + +```rust +use concord::derive::epoch_key_commitment; +use concord::rekey::{self, RekeyScope}; + +let scope = RekeyScope::Channel(channel_id); // or RekeyScope::Base +let plan = rekey::plan_refounding(Epoch(epoch + 1))?; + +// A base rotation delivers the new control-plane keys beside the root; a channel +// rotation delivers only that channel's fresh key. +let new_key = plan.new_root; +let (control_pk, control_root) = match scope { + RekeyScope::Base => { + let pk = plan.signer(&community_id)?.pk().to_bytes(); + (Some(pk), is_staff.then_some(&plan.new_control_root)) + } + RekeyScope::Channel(_) => (None, None), +}; + +let blobs = members + .iter() + .map(|member| { + rekey::build_blob(&my_keys, member, scope, plan.epoch, &new_key, control_pk.as_ref(), control_root) + }) + .collect::, _>>()?; + +let rekey_group = rekey::rekey_group(scope, &community_root, &community_id, plan.epoch)?; +let wraps = rekey::build_rekey_chunks( + &my_keys, + &rekey_group, + scope, + plan.epoch, + Epoch(epoch), + &epoch_key_commitment(Epoch(epoch), &community_root), + &blobs, + citation, + false, + now_secs, +)?; +``` + +On the receiving side, `rekey::parse_rekey_chunk(&opened)` per wrap, then +`collect_rotations(&chunks)`, then `am_i_removed(&rotation, &me)` — which is +`None` until every chunk is held, because an incomplete set is never a removal. A +member finds their delivery with `find_my_blobs` / `open_blob`, and adopts the key +only if the plaintext binds to the scope and epoch they expect and its `prevcommit` +matches the key they already hold. Two concurrent rotations settle on `fork_winner`. + +Dissolution is owner-only and terminal: + +```rust +let rumor = rekey::dissolved_tombstone_rumor(owner_pk, &community_id, now_secs); +let wrap = rekey::seal_dissolved(&rumor, &community_id, &my_keys, now_secs)?; + +// A receiver seals the community read-only on sight. +if rekey::verify_dissolved(&wrap, &identity) { + state.dissolved = true; +} +``` + +## The Community List + +A member's own memberships, synced across their devices: + +```rust +use concord::list; + +let material = list::join_material(&invite, staff.then_some(&control_root)); +let mut mine = list::parse_list_event(&my_keys, &event)?; +mine = list::merge(mine, list::CommunityList { + entries: vec![list::CommunityListEntry { community_id, seed: material.clone(), current: material, added_at: now_ms, extra: Default::default() }], + ..Default::default() +}); +let event = list::build_list_event(&my_keys, &mine)?; // kind 13302, NIP-44 to self +``` + +`is_live(&id)` answers joined-versus-left: a tombstone is terminal until a +strictly newer join outruns it. `fits()` is the write gate — 50 memberships and +the NIP-44 size cap, both protocol constants. + +## GPUI conventions + +- Wrap, decrypt, verify, fold and every database or relay call go in + `cx.background_spawn`. A secp256k1 verification per edition is far too + expensive for the foreground thread. +- Hold entities foreground: `cx.spawn` with `this.update(cx, |this, cx| …)` and + the inner `cx`, keeping the returned `Task` in a field so it is cancelled with + the view. +- In tests, use `cx.background_executor().timer(..)` for delays, never + `smol::Timer`, or `run_until_parked()` will find nothing left to run. + +## Not wired up yet + +- **No registry and no sync engine.** `crates/concord` has no subscriptions, no + `init`, and no `Entity`; the UI owns subscribing, routing a wrap to + the plane whose address it carries, and rebuilding a subscription when a plane's + address changes (join, channel added, rekey folded). +- **Every writer takes `&Keys`, not a `NostrSigner`.** NIP-46 is one deliberate + pass over the builders, not a per-call patch. +- **`crates/chat/src/lib.rs::handle_notifications` treats every kind 1059 event as + a NIP-59 gift wrap for the current user.** Concord wraps are kind 1059 too, so + that handler must route by subscription id before any concord subscription goes + live, or every stream wrap lands in the DM trash and raises a toast. +- **No plane key can be persisted yet.** `CommunityState` has nowhere to keep a + key a rotation delivered and `ChannelKeyRef` carries no key of its own, so a + client can verify a rotation and still lose it on restart — history under a + prior root or a prior channel epoch is unreadable until that schema change + lands. -- 2.54.0 From 28c987cfe73d814afa128d42baf46936ff97e732 Mon Sep 17 00:00:00 2001 From: Ren Amamiya Date: Thu, 17 Sep 2026 11:28:28 +0700 Subject: [PATCH 12/12] update doc --- docs/concord-usage.md | 118 ++++++++++++++++++++++++++++++++++++++---- 1 file changed, 108 insertions(+), 10 deletions(-) diff --git a/docs/concord-usage.md b/docs/concord-usage.md index 03599d02..eb68470e 100644 --- a/docs/concord-usage.md +++ b/docs/concord-usage.md @@ -408,23 +408,121 @@ let event = list::build_list_event(&my_keys, &mine)?; // kind 13302, NIP-44 strictly newer join outruns it. `fits()` is the write gate — 50 memberships and the NIP-44 size cap, both protocol constants. -## GPUI conventions +## GPUI integration -- Wrap, decrypt, verify, fold and every database or relay call go in - `cx.background_spawn`. A secp256k1 verification per edition is far too - expensive for the foreground thread. -- Hold entities foreground: `cx.spawn` with `this.update(cx, |this, cx| …)` and - the inner `cx`, keeping the returned `Task` in a field so it is cancelled with - the view. -- In tests, use `cx.background_executor().timer(..)` for delays, never - `smol::Timer`, or `run_until_parked()` will find nothing left to run. +`crates/concord` stays GPUI-free. The UI layer adds a registry global and one +entity per community, and moves every decrypt, verification, fold and I/O off +the foreground thread. + +### Entities + +Same shape as `ChatRegistry`: + +```rust +pub fn init(window: &mut Window, cx: &mut App) { + ConcordRegistry::set_global(cx.new(|cx| ConcordRegistry::new(window, cx)), cx); +} + +impl ConcordRegistry { + pub fn global(cx: &App) -> Entity { + cx.global::().0.clone() + } +} +``` + +Call it after `chat::init` in `desktop/src/main.rs` and `web/src/lib.rs`, and +subscribe to `NostrRegistry` for `SignerChanged` so the communities reset with +the account. + +- `ConcordRegistry` holds `communities: Vec>`, an index by + `CommunityId`, and `tasks: SmallVec<[Task>; 2]>`. +- `Community` owns one `CommunityState`, the last `ControlFold`, the member list + and the channel list. Views render `Entity`; no protocol state + lives in a view. +- `CommunityState::apply_fold` is one assignment: run it in the task that + produced the fold and send only the result to the foreground. +- Emit an event on every fold so dependents re-read. + +### Foreground and background + +A background task never touches an entity. It sends results through a bounded +`flume` channel that a foreground `cx.spawn` drains with `this.update(...)`. + +```rust +let (signal_tx, signal_rx) = flume::bounded::(256); +let database = client.database().clone(); + +// Background: open, verify, fold — no entities. +self.ingress = Some(cx.background_spawn(async move { + for wrap in &wraps { + let Some(plane) = planes.iter().find(|plane| plane.group.pk() == wrap.pubkey) else { + continue; + }; + let (opened, rumor) = chat::open(wrap, &plane.group, &plane.channel, plane.epoch)?; + store::cache_rumor(database.as_ref(), &plane.channel, &opened).await?; + signal_tx.send_async(Signal::Chat { channel: plane.channel, rumor }).await?; + } + Ok(()) +})); + +// Foreground: the only place entities change. +self.consumer = Some(cx.spawn(async move |this, cx| { + while let Ok(signal) = signal_rx.recv_async().await { + this.update(cx, |this, cx| this.apply(signal, cx))?; + } + Ok(()) +})); +``` + +- `client.database()` is a `&Arc` and `store::save_state` + wants `&dyn NostrDatabase`, so clone the `Arc` and pass `database.as_ref()`. +- Keep long-lived tasks in fields — dropping a `Task` cancels it. Assign `None` + to an `Option>` before respawning it; a signer change replaces both + the listener and the consumer. +- `cx.spawn` when the work updates an entity after awaiting, and + `cx.background_spawn` when it only produces a value. A query the foreground awaits can be returned straight out: `fn messages(&self, cx: &App) -> Task, Error>>`. +- Do the first load in `cx.defer_in(window, ...)` so `init` returns before the + first relay request. +- NIP-46 signing is async: call `signer.get_public_key_async()` / + `sign_event_async` inside the background task. The builders still take + `&Keys`, so run them where device keys are available. + +### Subscriptions + +A wrap is addressed to a plane, so the plane's public key is the routing key and +one `Filter` per held plane is enough: + +```rust +let filter = Filter::new() + .kinds([Kind::from(KIND_WRAP), Kind::from(KIND_WRAP_EPHEMERAL)]) + .pubkey(plane.group.pk()) + .since(joined_at); +client.subscribe(filter).with_id(sub_id).await?; +``` + +- `pubkeys([...])` carries every plane of a community on one subscription. Call + `subscribe` again with the new address whenever a join, a channel add or a + rekey fold changes it. +- Route inbound events by `subscription_id` from `RelayMessage::Event`, never by + kind. +- Watch one epoch ahead: while holding `root_N`, subscribe to + `base_rekey_group_key(&root_N, &community_id, Epoch(N + 1))` and to + `channel_rekey_group_key(&root_N, &channel, Epoch(N + 1))` for each private + channel. A second epoch ahead is not derivable until the new root arrives. + +### Tests + +`cx.background_executor().timer(..)` for delays, never `smol::Timer`, or +`run_until_parked()` finds nothing left to run. Push a wrap into the channel and +`run_until_parked()` to drive the foreground consumer. ## Not wired up yet - **No registry and no sync engine.** `crates/concord` has no subscriptions, no `init`, and no `Entity`; the UI owns subscribing, routing a wrap to the plane whose address it carries, and rebuilding a subscription when a plane's - address changes (join, channel added, rekey folded). + address changes (join, channel added, rekey folded). GPUI integration above is + the shape to build, not code that exists. - **Every writer takes `&Keys`, not a `NostrSigner`.** NIP-46 is one deliberate pass over the builders, not a per-call patch. - **`crates/chat/src/lib.rs::handle_notifications` treats every kind 1059 event as -- 2.54.0