# NIP-29 groups backend Coop has two relay-backed backends today: `crates/chat` (NIP-17 gift-wrap DMs, rendered by `crates/chat_ui`) and `crates/community` (Concord). This document is a plan, not a description of shipped code; nothing below is wired up yet. It covers the backend only — reading, writing, joining and leaving relay-based groups as defined by [NIP-29](https://github.com/nostr-protocol/nips/blob/master/29.md) — and ends with the hooks a UI will use later. ## The spec in backend terms A group is a `(relay, id)` pair. The relay enforces membership and signs the group's state; there is no cross-relay identity, no encryption, and no portable "create". The same `id` on two relays is two groups (a fork), which the plan models as two entities. | Concern | Wire | Backend rule | | --- | --- | --- | | Group id | random string, relay-chosen | `h` tag on everything a user sends; `d` tag on relay state events | | Timeline | any event kind + `h` | v1 renders kinds `1`, `9`, `11`, `1111` and `7`; every other kind is cached for the `previous` window but not rendered, except the membership kinds below | | Timeline refs | `previous` tag, first 8 hex chars of event ids | built from the last 50 events seen on that relay (any kind — NIP-29 timeline references), excluding our own; send up to 3 | | Relay state | `39000` metadata, `39001` admins, `39002` members, `39003` roles | relay-signed, addressable by `d`; read, never written by the client | | Membership | `9021` join, `9022` leave (user-signed); relay answers with `9000` put-user / `9001` remove-user | membership = latest `9000`/`9001` naming our pubkey; no event means not a member, and `Pending`/`Refused` are tracked locally for private groups | | Own group list | `10009` (NIP-51 simple groups) | `["group", id, relay, name?]` plus `["r", relay]`; replaceable, published to NIP-65 relays | | Reference | `naddr1…` of kind `39000` with the relay self pubkey and a relay hint, optionally `?invite=` | the self pubkey is the author of the fetched `39000`; no `39000` means no canonical naddr | Two behaviours that shape the code: - **Late publication is rejected.** Relays drop messages with old timestamps, so there is no offline send queue in v1: a send either goes now or fails visibly. - **`previous` must reference events the relay already has.** Relays may enforce at least 3 references; a group we have little history for may have its sends rejected until the backlog loads. The relay's `OK false` message is surfaced as-is. ### Not in v1 Moderation writes (`9000`–`9010`), group creation (`9007`, relay-dependent), pinned events (`39005`/`9010`), subgroups (`parent`/`child` — parsed but inert), LiveKit audio/video (`livekit` tag, `39004`, the token endpoint), fork/migration detection through admins' `10009`, and relay browsing (`{"kinds":[39000]}` to list a relay's public groups). All are additive; none change the model above. ## What the local rust-nostr checkout gives us Cross-checked against the pinned checkout (`~/.cargo/git/checkouts/nostr-619b808bb247a9ed/01f1950`): - **Kinds exist** for every NIP-29 event in v1: `Kind::GroupPutUser` (9000), `GroupRemoveUser` (9001), `GroupEditMetadata` (9002), `GroupDeleteEvent` (9005), `GroupCreateGroup` (9007), `GroupDeleteGroup` (9008), `GroupCreateInvite` (9009), `GroupJoinRequest` (9021), `GroupLeaveRequest` (9022), `GroupMetadata` (39000), `GroupAdmins` (39001), `GroupMembers` (39002), `GroupRoles` (39003), `GroupLivekitParticipants` (39004), and `Kind::SimpleGroups` (10009). `Kind::ChatMessage` is kind 9. - **`9010` and `39005` are missing** from this revision (`Kind::Custom(9010)`, `Kind::Custom(39005)` if they are ever needed — they are not, in v1). - **There is no NIP-29 module.** `nostr/src/nips/` has no `nip29.rs`, and no `GroupId`, metadata type, parser or builder exists. Everything in `protocol.rs` below is ours; it is thin — tags and filters, no crypto. - **The APIs to lean on** are all present and already used in the app: - `Tag::custom`, `Tag::event`, `Tag::identifier`, `Tag::public_key`, `Tag::content`, `Tag::as_slice`. - `Filter::custom_tag(SingleLetterTag::LOWERCASE_H, id)`, `.identifier(id)`, `.author`, `.kinds`, `.limit`, `.since`. - `ReqTarget::single(url, filters)` for a group-scoped REQ, `client.subscribe(..).with_id(..)`, `client.fetch_events(..)`, `client.send_event(&event).to([url])`. - `EventBuilder::new(kind, content).tags(tags).finalize_async(&signer)` — the app's `UniversalSigner` works here (NIP-46 included), as it does in `crates/chat/src/room.rs`. - `client.database().query/save_event/delete`. - `Nip19Coordinate { coordinate, relays }` with `ToBech32`/`FromBech32` for `naddr`. - **NIP-11 is not exposed.** `Relay` in this SDK has no document accessor, so the relay self pubkey is taken from the author of the group's `39000` event; a group whose relay publishes no `39000` therefore has no canonical naddr and cannot be shared or joined in v1. ## What the app already does (and we copy) The NIP-17 backend is the template. `crates/chat` has a global `ChatRegistry` entity, one `Room` entity per conversation, a background task over `client.notifications()` that sends signals through a `flume` channel, and a foreground `cx.spawn` consumer that is the only place entities change. Local messages are cached as `ApplicationSpecificData` (30078) events with a `d` tag for the target id and an `r` tag for the room key (`set_rumor`/`get_rumor`, `crates/chat/src/lib.rs`). `crates/chat_ui` renders `chat::Message` into a `ListState`, resolving names through `PersonRegistry`. `crates/community` shows the same pattern for relay-scoped state: routing notifications by subscription id (`route_of`), bringing relays into the pool before use (`connect_relays`), and treating a `CLOSED` message as a refusal. Replacing a list is done in `crates/workspace/src/panels/contact_list.rs`: sign the builder, `client.send_event(&event).to_nip65().await` — the client saves the event locally even when the publish fails. NIP-29 reuses all of this and drops what only NIP-17 needs: no encryption, no per-member fan-out, no gift wraps, no `SendReport` per receiver — one relay, one event, one `OK`. ## Where this lands A new crate `crates/nip29`, because NIP-29 is a different transport (plain relay events), a different state authority (relay-signed `39000`s) and a different UI later. Making `chat::Room` polymorphic over gift-wrapped DMs and relay groups would put two protocols in one entity. The alternative — modules inside `crates/chat` — saves a `Cargo.toml` but was set aside (see Decisions). | File | Owns | | --- | --- | | `crates/nip29/src/protocol.rs` | Pure NIP-29 types and their `impl` blocks: `GroupId`, `GroupKey`, `GroupReference`, `GroupMetadata`, admins/members/roles, `GroupList`, `Membership`, `TimelineWindow`. No I/O. | | `crates/nip29/src/group.rs` | `Group` entity: state, timeline, membership, `connect`, `ingest`, `send_message`, `send_reaction`, `join`, `leave`, `GroupEvent`. | | `crates/nip29/src/lib.rs` | `GroupsRegistry` global, `init`, standing subscriptions and the signal pump, the envelope cache, `load`/`publish` of the account's `10009`. | | `desktop/src/main.rs`, `web/src/lib.rs` | `nip29::init(cx)` after `chat::init(cx)`. | | UI later | `crates/chat_ui` (reuse `chat::Message` rendering) or a new `crates/nip29_ui`; sidebar section; a join dialog. | `protocol` is re-exported at the crate root (`pub use protocol::*`) so callers write `nip29::GroupId`, `nip29::GroupMetadata`. The crate's dependencies are the ones `chat` already carries: `state` (client and signer), `chat` (`Message`), `gpui`, `nostr-sdk`, `anyhow`, `flume`, `futures`, `smallvec`, `log`; dev-dependencies `nostr-memory` and `nostr-sdk` with the `local-relay` feature for the tests below. ## Data flow ``` own 10009 (NIP-65 relays) group relay (one per group) │ load / subscribe │ REQ #h= (timeline + 9000/9001) ▼ │ REQ #d= (39000..39003) GroupsRegistry ──ensure relay──► Group entity ◄──ingest── background listener │ index: GroupKey │ (saves envelope first) │ │ └── flume Signal ◄──────────────────────┘ ▲ UI later: GroupPanel (Vec), JoinDialog (naddr) ``` ## Phase 0 — `crates/nip29/src/protocol.rs` (pure, no I/O) The protocol is expressed as types with `impl` blocks: a parsed thing is a struct and every operation on it is a method, so call sites read `id.message(...)`, `metadata.naddr(...)`, `admins.contains(&pk)` instead of handing the pieces to free builder functions. ```rust /// The relay-chosen group id. pub struct GroupId(String); impl GroupId { pub fn new(id: impl Into) -> Result; // trimmed, non-empty pub fn as_str(&self) -> &str; pub fn h_tag(&self) -> Tag; // ["h", id] pub fn timeline_filter(&self, limit: usize) -> Filter; // #h = id pub fn state_filter(&self) -> Filter; // #d = id, kinds 39000..39003 pub fn membership_filter(&self, me: PublicKey) -> Filter; // #h = id, #p = me, kinds 9000/9001 pub fn message(&self, content: String, replies: &[EventId], previous: Option) -> EventBuilder; // kind 9 pub fn reaction(&self, target: EventId, emoji: String, previous: Option) -> EventBuilder; // kind 7 + e pub fn join_request(&self, code: Option<&str>, reason: &str) -> EventBuilder; // 9021 pub fn leave_request(&self, reason: &str) -> EventBuilder; // 9022 } /// Entity identity: the same id on another relay is another group (a fork). pub struct GroupKey { relay: RelayUrl, id: GroupId } impl GroupKey { pub fn new(relay: RelayUrl, id: GroupId) -> Self; pub fn relay(&self) -> &RelayUrl; pub fn id(&self) -> &GroupId; pub fn cache_tag(&self) -> String; // "|" pub fn uniq_id(&self) -> u64; // hashed, for panel and element ids } /// What a user pastes to join: naddr1…(+"?invite="). pub struct GroupReference { id: GroupId, relay: RelayUrl, invite: Option } impl GroupReference { pub fn parse(input: &str) -> Result; pub fn id(&self) -> &GroupId; pub fn relay(&self) -> &RelayUrl; pub fn invite(&self) -> Option<&str>; pub fn key(&self) -> GroupKey; } impl FromStr for GroupReference { .. } // so a pasted naddr can use `.parse()` too /// kind:39000, relay-signed. pub struct GroupMetadata { pub name: Option, pub picture: Option, pub banner: Option, pub about: Option, pub private: bool, pub restricted: bool, pub hidden: bool, pub closed: bool, pub livekit: bool, pub parent: Option, pub supported_kinds: Option>, // None = unspecified (all); Some(vec![]) = none } impl GroupMetadata { pub fn parse(event: &Event) -> Result; pub fn name(&self) -> Option<&str>; pub fn supports(&self, kind: Kind) -> bool; pub fn naddr(&self, relay: &RelayUrl, invite: Option<&str>) -> Result; // Coordinate { kind: 39000, public_key: , identifier: } } /// kind:39001 — each pubkey with its role labels. pub struct GroupAdmins(Vec<(PublicKey, Vec)>); impl GroupAdmins { pub fn parse(event: &Event) -> Result; pub fn entries(&self) -> &[(PublicKey, Vec)]; pub fn contains(&self, public_key: &PublicKey) -> bool; } /// kind:39002 — may be absent or a subset; absence means "not published". pub struct GroupMembers(Vec); impl GroupMembers { pub fn parse(event: &Event) -> Result; pub fn contains(&self, public_key: &PublicKey) -> bool; pub fn iter(&self) -> impl Iterator; } /// kind:39003 — role names are relay-defined. pub struct GroupRoles(Vec); pub struct Role { pub name: String, pub description: Option } impl GroupRoles { pub fn parse(event: &Event) -> Result; pub fn roles(&self) -> &[Role]; } /// kind:10009 — the account's own list. pub struct GroupList(Vec); pub struct GroupListEntry { pub id: GroupId, pub relay: RelayUrl, pub name: Option } impl GroupList { pub fn parse(event: &Event) -> Result; pub fn entries(&self) -> &[GroupListEntry]; pub fn upsert(&mut self, entry: GroupListEntry); pub fn remove(&mut self, key: &GroupKey); pub fn to_builder(&self) -> EventBuilder; // kind 10009, group + r tags } pub enum Membership { Unknown, Pending { since: Timestamp }, // our 9021 went out; no answer yet Member, Refused { reason: Option }, // the relay's OK false / CLOSED message Removed, } impl Membership { /// Folds 9000/9001 only; `Pending` and `Refused` are set by the join path. pub fn from_events(events: &[Event], me: PublicKey) -> Self; } /// The newest-first window (≤50) that `previous` references are drawn from. pub struct TimelineWindow(Vec); impl TimelineWindow { pub fn new() -> Self; pub fn push(&mut self, event: Event); // dedups, caps at 50 pub fn previous_tag(&self, me: PublicKey) -> Option; // excludes `me`, ≤3 distinct 8-hex } ``` Details that must be right: - Every user event carries `["h", id]` (from `GroupId::h_tag`); `previous` is added only when we have at least one eligible event, and its values are the first 8 characters of the hex ids. - `GroupReference::parse` splits `?invite=` before bech32 decoding; a missing relay hint is an error (there is nowhere to send the join request). - `GroupList::parse` accepts `group` tags with 1–3 values (id, relay, name). - `Membership::from_events` takes the newest `9000`/`9001` whose `p` tags name `me`; on an exact tie it prefers `9001` (removal). `Pending` and `Refused` are not things the relay folds into an event: the join path sets them from the `9021` send and from the relay's refusal message. - `GroupMetadata::supports` distinguishes "tag absent" from "tag present with no values": the first means all kinds, the second means no text messages. - The builders return `EventBuilder`s so the caller decides when to sign; nothing in this file does I/O. ## Phase 1 — read path (`group.rs` + `lib.rs`) `GroupsRegistry` (global entity, mirroring `CommunityRegistry`): ```rust pub struct GroupsRegistry { groups: Vec>, index: HashMap>, /// The account's 10009, loaded at start and mutated by join/leave list: GroupList, /// subscription id → group, rebuilt whenever a group resubscribes routes: Arc>>, signal_tx: flume::Sender, signal_rx: flume::Receiver, notification_listener: Option>>, // cancelled on signer change signal_consumer: Option>>, tasks: SmallVec<[Task>; 2]>, _subscriptions: SmallVec<[Subscription; 2]>, } impl GroupsRegistry { pub fn global(cx: &App) -> Entity; pub fn groups(&self) -> &[Entity]; pub fn group(&self, key: &GroupKey) -> Option>; pub fn join(&mut self, reference: GroupReference, cx: &mut Context); pub fn leave(&mut self, key: &GroupKey, cx: &mut Context); fn load(&mut self, cx); // own 10009 from the database into `self.list` fn publish_list(&mut self, cx); // sign `self.list.to_builder()`, to_nip65, best-effort } ``` - **Subscriptions.** One standing REQ per group, two filters in one id: `key.id().timeline_filter(100)` and `key.id().state_filter()`. The id is `grp-<8 hex of the group key hash>` (NIP-01 caps ids at 64 chars; the real key lives in `routes`). The relay is added and connected first (`add_relay(url).and_connect()`), like `community::sync::connect_relays`. On open, also issue a one-shot `key.id().membership_filter(me)` fetch so a restart learns `9000`/`9001` immediately; the state filter is fetched once (limit 1 per kind) so the sidebar has names before the live REQ settles. A dead relay costs only its own group. - **Pump.** The listener matches `ClientNotification::Message` and routes `RelayMessage::Event`/`EndOfStoredEvents`/`Closed` by subscription id through `routes`. For `h`-tagged events it saves the envelope first, then sends `Signal::Event(key, event)`; state events send `Signal::State(key, event)`; `Closed` sends `Signal::Refused(key, message)`. The consumer calls `Group::ingest`, which is the only place entity state changes. - **Envelope cache.** Same shape as `set_rumor`/`get_rumor` in `crates/chat`: `ApplicationSpecificData` (30078, addressable) with `d` = event id (dedup on replay), `r` = `"|"` (per-fork history), `k` = kind, content = the raw event JSON, signed by a crate-local `LOCAL_KEYS`. Every `h`-tagged event the standing subscription delivers is enveloped — any kind, because the `previous` window is literally "the last 50 events seen" (NIP-29 timeline references) — while the `d`-tagged state events (`39000`–`39003`) are not. Reading is `Filter::new().custom_tag(LOWERCASE_R, key)`, parsed and sorted by `created_at`, then partitioned: the render set (`1`, `7`, `9`, `11`, `1111`) becomes messages, `9000`/`9001` fold membership, the rest only feed the window. This is what makes the timeline fork-safe and restart-safe without teaching the database about relays. - **`Group` state.** `GroupKey`, `metadata: Option`, `metadata_event: Option` (what `GroupMetadata::naddr` is built from), `members`, `admins`, `roles`, `membership: Membership` (including the local `Pending`/`Refused` access states), `messages: Vec` for the render set (reusing `chat::Message`, which already parses any event into display fields), an id index, a `TimelineWindow` for `previous`, `loading`, `refused`. It emits `GroupEvent::{Incoming(Event), Updated, Refused(String), Eose}` — the panel subscribes to these the way `ChatPanel` subscribes to `RoomEvent`. - **`load()`.** Subscribe to our own `10009` (`ReqTarget::auto`, limit 1), then read the newest one from the database into `self.list`, and add one group per entry. A newer `10009` arriving from another device is merged (`GroupList::upsert`) and the new groups are subscribed; removals by another client cannot be detected in a replaceable list and are not guessed at. ## Phase 2 — write path All writes sign with the app signer (`finalize_async(&signer)`), send to the group's relay only, and treat the relay's `OK` as the truth. - **Send a message.** Build with `key.id().message(content, &replies, window.previous_tag(me))`, sign, then `client.send_event(&event).to([relay])`. Kind 9 is the default message kind; when `metadata.supports(Kind::ChatMessage)` is false the composer is disabled rather than sending a kind the relay does not take. On acceptance: save the envelope, insert into `messages`, emit `GroupEvent::Incoming`. On `OK false`: return the relay's message (it already carries `invalid:` / `restricted:` / `duplicate:` prefixes) and let the panel show it. Reactions go through the same path with `key.id().reaction(..)`. - **Join.** `GroupReference::parse` → error if no relay hint → add/connect the relay → fetch `39000` (`limit 1`) to confirm the group and learn the self pubkey; `GroupMetadata::parse` it; a hidden group may refuse this — proceed without metadata rather than failing → create the entity, fetch state + membership, subscribe → send `key.id().join_request(code, ..)` — the `h` tag plus the `code` tag when an invite code was parsed → mark `Membership::Pending { since }` and `self.list.upsert(..)`, so the group stays listed while it waits. The relay either emits `9000` (member), refuses the `9021` with `OK false` (set `Refused { reason }` from its message — the NIP says the message explains whether the rejection is final, pending review or payment-required), or refuses the timeline REQ with `restricted:` / `auth-required:` in a `CLOSED` (keep the state, do not raise an error toast). All three are observed on the standing subscription or the send result. - **Leave.** Send `key.id().leave_request(..)`; on `OK`, forget the group's envelopes (`client.database().delete(filter)` by `r` tag), drop the entity and `self.list.remove(key)`, then publish. The relay also emits `9001`, which the (now gone) subscription would have seen. - **Publish `10009`.** Sign `self.list.to_builder()` and publish with `to_nip65()`. The registry's `GroupList` is the one loaded at start, join `upsert`s an entry and leave `remove`s one, so a rewrite can never drop a group we loaded. The client saves the event locally even when the publish fails, so the list survives offline. ## Phase 3 — later, in rough order 1. Moderation writes: `9000` put-user (roles), `9001` remove-user, `9002` edit-metadata, `9005` delete-event, `9009` create-invite; gate the UI on the caller appearing in `39001`, let the relay be the enforcer. 2. Group creation (`9007`) where the relay supports it, and relay browsing (`fetch_events(ReqTarget::single(relay, [Filter::new().kind(Kind::GroupMetadata).limit(..)]))`). 3. Pins: `9010` write, `39005` read (`Kind::Custom`); subgroups: parse `parent`/`child` and lay groups out as a tree. 4. Range backfill: page the timeline with `until` for "load more". 5. LiveKit AV (`livekit` tag, `39004`, the `/.well-known/nip29/livekit` token endpoint with a NIP-98 `27235` authorization event) and fork/migration detection through the admins' `10009`. ## UI later — the hooks this backend leaves | UI action | Call | | --- | --- | | Paste an naddr and join (or retry a refused join) | `GroupReference::parse` → `GroupsRegistry::join` | | Open a group | `GroupsRegistry::group(&key, cx)` → panel subscribes to `GroupEvent` | | Timeline | `Group::messages()` (already `chat::Message`s) | | Send / react | `Group::send_message` / `send_reaction` | | Members, admins, roles, flags | `Group::members/admins/roles/metadata` | | Membership badge (member, pending, refused, removed) | `Group::membership()` | | Copy an invite link | `GroupMetadata::naddr(relay, invite)` | | Leave | `GroupsRegistry::leave` | Where it lands: the sidebar gains a section reading `GroupsRegistry::groups()` (the same shape as its Communities section) with pending and refused groups listed alongside members; a new panel renders `Vec` with the existing `ListState` + `ui::message::MessageRow` and `PersonRegistry` for authors. The message *data* is already shared (`chat::Message`), so only `ChatPanel`'s row-rendering methods need lifting into `ui` (or copying) rather than the type. A join dialog accepts a pasted naddr (with or without `?invite=`). System notifications can hang off `GroupEvent::Incoming` when the window is inactive. Nothing new is needed from GPUI to support this backend. The local checkouts are `gpui-pre` 0.3.5 (aliased as `gpui`) and `gpui-base` 0.7.0, which `crates/ui` styles; the only GPUI APIs the backend uses are the ones `chat` and `community` already use — `Entity`/`WeakEntity`, `Context::spawn`, `background_spawn`, `EventEmitter`, `Subscription` — and the panel's components (`ListState`, `MessageRow`, `Avatar`, `Button`, `Scrollbar`) already exist in the app. ## Tests - `protocol.rs` unit tests: `GroupReference::parse` (naddr, `?invite=`, garbage, missing relay hint, an naddr whose kind is not `39000`); `GroupMetadata::parse` flags and `supports` with `supported_kinds` absent vs empty; admins with roles; members; roles; `GroupList` round trip (`parse` → `upsert` → `to_builder`); `TimelineWindow::previous_tag` (excludes self, caps at 3, 8-char prefixes, any-kind window, `None` when empty); `Membership::from_events` (`9000` vs `9001`, newest wins, tie prefers removal; `Pending`/`Refused` are exercised through the join path); `GroupId::message` carries `h` and the reply `e` tags; `GroupMetadata::naddr` round trip with and without an invite code. - Cache round trip with `nostr-memory` as a dev-dependency (the pattern `crates/community` already uses): save two envelopes (a kind 9 and a kind `9000`), read them back sorted, assert the partition — the kind 9 renders, the `9000` only folds membership — re-save one and confirm no duplicate, `delete` clears the group. - Routing: subscription id ↔ `GroupKey` mapping survives a resubscribe. - One integration test against the SDK's `LocalRelay` (`nostr-sdk` dev-dependency with the `local-relay` feature): seed a `39000` and a kind 9 with an `h` tag, run `Group::connect`, assert metadata and the timeline land. Timeouts in GPUI-driven tests use `cx.background_executor().timer(..)`, never `smol::Timer`, per `AGENTS.md`. ## Decisions 1. **Crate.** `crates/nip29` (the body above is written for it). 2. **Timeline kinds.** Render `1`, `9`, `11`, `1111` and `7`; every other kind is still cached and feeds the `previous` window, per the NIP's timeline references rule. 3. **Default message kind.** Kind 9; the composer is disabled when `supported_kinds` is present and lacks it. 4. **Pending joins.** Pending (and refused) groups are listed in the sidebar, with an access state on the row. 5. **`10009` merge.** Union-only; removals made by another client are not detected and no diffing is done. 6. **Leave and the cache.** Leaving wipes the group's envelopes. 7. **Restricted/private groups.** A visible access state is required: `Membership::{Unknown, Pending, Member, Refused, Removed}` with the relay's refusal reason shown and a retry, while the client's NIP-42 authenticator handles the reads. 8. **References.** `naddr` only (`?invite=` accepted); `id@relay` is not. A group whose relay publishes no `39000` cannot be referenced or joined in v1.