clean up
This commit is contained in:
@@ -1,780 +0,0 @@
|
||||
# Using the Concord backend
|
||||
|
||||
`crates/concord` is a protocol crate: derivations, envelopes, folds and the local
|
||||
state document. It has no GPUI dependency and owns no strings a user reads — the
|
||||
UI layer decides every rendering. This document is the map from a UI action to
|
||||
the calls it makes.
|
||||
|
||||
A community is addressed by a `community_id` (never on the wire) plus three
|
||||
secrets: `community_root` (read access — holding it *is* membership),
|
||||
`control_root` (write access to the Control Plane, held by staff), and per-Channel
|
||||
keys for private channels. Authority is a roster of owner-rooted signed grants,
|
||||
folded independently by every client.
|
||||
|
||||
## Modules
|
||||
|
||||
Files follow the CORD documents. The frozen derivations of Appendix A and the id
|
||||
vocabulary are shared substrate — every document calls them — so they live outside
|
||||
`cords` in `utils::derive` (re-exported as `derive`) and `types`.
|
||||
|
||||
| Module | Owns |
|
||||
| --- | --- |
|
||||
| `cord01` | Private Streams: the seal/wrap envelope and the NIP-44 helpers |
|
||||
| `cord02` | Communities: identity, epochs, metadata, the Control Plane fold and writer |
|
||||
| `cord02::guestbook` | Joins, leaves, kicks, snapshots, the member list |
|
||||
| `cord02::list` | The Community List (a member's own memberships, across devices) |
|
||||
| `cord03` | Channels: Channel metadata and the Chat Plane |
|
||||
| `cord04` | Roles: chained editions, parse/hash/fold, the roster, permissions, the banlist |
|
||||
| `cord04::pins` | Pin Lists, and the key disclosure a keyless reader verifies |
|
||||
| `cord05` | Invite bundles, links, the Direct Invite, the Invite List |
|
||||
| `cord06` | Key rotations, refounding, compaction, dissolution |
|
||||
| `derive` | Every frozen HKDF derivation and coordinate |
|
||||
| `state` | The community state document and its pure folds — no I/O |
|
||||
|
||||
`CommunityId`, `ChannelId`, `RoleId`, `Epoch` and `Extra` (crate-internal) come from
|
||||
the private `types` module and are re-exported at the crate root.
|
||||
|
||||
Nothing in `concord` touches a relay or a database. The local rumor cache, the
|
||||
state documents and the relay paging walk live one crate up, in
|
||||
`community::cache` and `community::history`, and are what a client actually calls
|
||||
(read on below).
|
||||
|
||||
CORD-07 (audio/video) is unimplemented. CORD-08's timer has no file of its own: it
|
||||
lives in the metadata it reads (`cord02`) and the fold it filters (`cord03`).
|
||||
|
||||
Read `CommunityId` as "this community", `ChannelId` as "this channel", `Epoch` as
|
||||
"which key generation". Nothing else in the API needs internal state.
|
||||
|
||||
## Creating a community
|
||||
|
||||
```rust
|
||||
use concord::cord02::{self, CommunityMetadata};
|
||||
use concord::state::CommunityState;
|
||||
use community::cache::save_state;
|
||||
|
||||
let metadata = CommunityMetadata { name: "Room".into(), ..Default::default() };
|
||||
let minted = cord02::genesis(&owner_keys, &metadata, now_secs).await?;
|
||||
|
||||
// minted.identity — community_id, owner, owner_salt (verify() recomputes it)
|
||||
// minted.wraps — the two owner-signed genesis editions, already sealed
|
||||
// minted.channel_id — the #general channel
|
||||
for wrap in &minted.wraps {
|
||||
client.send_event(wrap).to(&relays).await?;
|
||||
}
|
||||
```
|
||||
|
||||
The owner then needs the folded state, which is also what every member does on
|
||||
join:
|
||||
|
||||
```rust
|
||||
use concord::derive::{control_group_key, control_signer_group_key};
|
||||
use concord::cord04::ParsedEdition;
|
||||
|
||||
let read = control_group_key(&minted.community_root, &minted.identity.community_id, Epoch(0))?;
|
||||
let signer = control_signer_group_key(&minted.control_root, &minted.identity.community_id, Epoch(0))?;
|
||||
let editions: Vec<ParsedEdition> = minted
|
||||
.wraps
|
||||
.iter()
|
||||
.map(|wrap| cord02::open_edition(wrap, &read, &signer.pk(), true))
|
||||
.collect::<Result<_, _>>()?;
|
||||
|
||||
let mut state = CommunityState::from_genesis(&minted, &editions, added_at_ms)?;
|
||||
save_state(&client, &state).await?;
|
||||
```
|
||||
|
||||
Put the community's relay list into `state.relays` and add those relays to the
|
||||
client explicitly — coop's client is a gossip client with no background refresh.
|
||||
|
||||
Creating is not finished until the membership is announced. The two writes are
|
||||
independent and both best-effort: the genesis wraps go to the community's
|
||||
relays, and the membership goes to the account's own Community List (below), so a
|
||||
new device — or another client — can find the community without an invite.
|
||||
`crates/community`'s `sync::create` performs both.
|
||||
|
||||
## Joining
|
||||
|
||||
An invite link resolves to a bundle:
|
||||
|
||||
```rust
|
||||
use concord::cord05::{self, BundleState, invite_bundle_key};
|
||||
|
||||
let link = cord05::parse_link(url)?; // link_signer, token, bootstrap_relays, naddr
|
||||
|
||||
// The crate does no I/O: fetch the naddr from the fragment's relays, then:
|
||||
let invite = match cord05::parse_bundle_event(&event, &link.link_signer, &invite_bundle_key(&link.token))? {
|
||||
BundleState::Live(invite) => invite, // validate() already ran
|
||||
BundleState::Revoked => return Ok(None), // a tombstone at the coordinate
|
||||
};
|
||||
```
|
||||
|
||||
A Direct Invite arrives as a NIP-59 gift wrap addressed to the member:
|
||||
|
||||
```rust
|
||||
let (inviter, invite) = cord05::unwrap_direct_invite(&wrap, &my_keys).await?;
|
||||
```
|
||||
|
||||
Either way the invite carries `community_id`, `owner`, `owner_salt`,
|
||||
`community_root`, `root_epoch`, `control_pk`, the granted `channels`
|
||||
(`ChannelGrant { id, key, epoch, name }`) and the relay set. `invite.expired(now_ms)`
|
||||
is a preview rule: a past expiry still renders, but the join is refused.
|
||||
|
||||
Then publish a join so the member list sees the member before any backfill:
|
||||
|
||||
```rust
|
||||
use concord::derive::guestbook_group_key;
|
||||
use concord::cord02::guestbook;
|
||||
|
||||
let guestbook = guestbook_group_key(&invite.community_root, &invite.community_id, invite.root_epoch)?;
|
||||
let rumor = cord02::guestbook::build_join(my_pk, Some((creator_npub, label)), now_ms);
|
||||
let (wrap, _) = cord02::guestbook::seal_rumor(&rumor, &guestbook, &my_keys).await?;
|
||||
client.send_event(&wrap).to(&relays).await?;
|
||||
```
|
||||
|
||||
## Reading the Control Plane
|
||||
|
||||
```rust
|
||||
use concord::cord02::{self, ControlFold};
|
||||
|
||||
let editions: Vec<ParsedEdition> = wraps
|
||||
.iter()
|
||||
.filter_map(|wrap| cord02::open_edition(wrap, &read, &control_pk, true).ok())
|
||||
.collect();
|
||||
|
||||
let control: ControlFold =
|
||||
cord02::fold_control(&owner, &community_id, &editions, &state.floors(), &state.banned);
|
||||
state.apply_fold(&control);
|
||||
```
|
||||
|
||||
`ControlFold` is everything the community UI needs:
|
||||
|
||||
| Field | Use |
|
||||
| --- | --- |
|
||||
| `community` | name, description, icon, banner, `message_expiration` |
|
||||
| `channels` | the channel list; `deleted: true` means drop it |
|
||||
| `roles` | `role()`, `roles_of()`, `effective_permissions()`, `is_authorized()`, `is_staff()` |
|
||||
| `banned` | the banlist |
|
||||
| `registries` / `is_public()` | each invite creator's live link signers |
|
||||
| `pins` / `pin_content(id, channel)` | Pin List content per channel |
|
||||
| `floors` | the committed heads the next fold is judged against |
|
||||
| `gapped` | a chain hole: refetch the Control Plane before trusting what is missing |
|
||||
|
||||
`None` on `community` or a channel means "this client saw no authorized edition",
|
||||
never "the value is gone" — keep what the state already holds rather than walking
|
||||
the community backwards. Feed `state.floors()` and `state.banned` into the next
|
||||
fold; they are its memory.
|
||||
|
||||
## Sending a message
|
||||
|
||||
```rust
|
||||
use concord::cord03::{self, build_message};
|
||||
use concord::derive::channel_group_key;
|
||||
|
||||
let plane = channel_group_key(&community_root, &channel, epoch)?; // public channel
|
||||
let rumor = build_message(my_pk, &channel, epoch, text, None, at_ms, timer);
|
||||
let (wrap, wrap_key) = cord03::seal_rumor(&rumor, &plane, &my_keys, false).await?;
|
||||
client.send_event(&wrap).to(&relays).await?;
|
||||
```
|
||||
|
||||
- `epoch` is the **root** epoch for a public channel: its plane is
|
||||
`channel_group_key(community_root, channel, root_epoch)`, so a Refounding moves
|
||||
it along with the root. A private channel passes its own current channel epoch
|
||||
(`state.channels` carries it) and derives from its own key instead of
|
||||
`community_root`.
|
||||
- `timer` is `control.community.message_expiration`; pass `None` when it is off.
|
||||
The builder attaches the NIP-40 tag and `seal_rumor` mirrors it onto the wrap,
|
||||
so relays drop the ciphertext too.
|
||||
- `ephemeral: true` picks kind `21059` for typing indicators. Keep the returned
|
||||
wrap key if the message may be deleted later — a kind-5 delete needs it.
|
||||
- That `send_event` is the whole publish path; there is no optimistic echo. Feed
|
||||
the wrap through the same ingest path the subscription uses so send-then-read
|
||||
never waits on a relay round trip.
|
||||
|
||||
`build_edit`, `build_reaction` and `build_delete` are the same shape, each a rumor
|
||||
about an existing `EventId` rather than a mutation.
|
||||
|
||||
## Reading a channel
|
||||
|
||||
```rust
|
||||
use concord::cord03::{self, fold, plane_keys};
|
||||
use community::cache;
|
||||
|
||||
let planes = plane_keys(&held, &channel)?; // &[(Epoch, secret)]
|
||||
let mut rumors = Vec::new();
|
||||
|
||||
for wrap in &wraps {
|
||||
let Some((epoch, group)) = planes.iter().find(|(_, group)| group.pk() == wrap.pubkey) else {
|
||||
continue;
|
||||
};
|
||||
let Ok((opened, rumor)) = cord03::open(wrap, group, &channel, *epoch) else {
|
||||
continue;
|
||||
};
|
||||
cache::cache_rumor(&client, &channel, &opened).await?;
|
||||
rumors.push(rumor);
|
||||
}
|
||||
|
||||
let messages = fold(&rumors, Timestamp::now(), |actor, citation, author| {
|
||||
citation_ok(&owner, &community_id, actor, citation, &control.roles.floors)
|
||||
&& control.roles.can_act_on_member(actor, &owner, author, Permissions::MANAGE_MESSAGES)
|
||||
});
|
||||
```
|
||||
|
||||
`ChatMessage` carries `content`, `at_ms`, `edited_at`, `deleted`, `reactions`,
|
||||
`reply_to` and `thread_root` already resolved. The fold drops expired rumors; a
|
||||
`deleted` row is still returned so the timeline keeps its shape.
|
||||
|
||||
Relay history pages through the local cache:
|
||||
|
||||
```rust
|
||||
use community::history::{self, PageRegistry, Window};
|
||||
|
||||
// Every key the client still holds for the channel, newest epoch first: the
|
||||
// state's current key (`ChannelKeyRef::key` at `epoch`) plus every `priors`
|
||||
// entry a rotation stepped off. A public channel derives one per held root.
|
||||
let held: Vec<concord::state::HeldKey> = state.held_keys(&channel);
|
||||
|
||||
// The same registry `CommunityRegistry` shares with its notification pump: it is
|
||||
// what carries each page's EOSE and CLOSED back to the round waiting on it.
|
||||
let page = history::page(
|
||||
client,
|
||||
&pages,
|
||||
&channel,
|
||||
&held,
|
||||
&relays,
|
||||
Window::opening(cursor),
|
||||
20,
|
||||
50,
|
||||
)
|
||||
.await?;
|
||||
let cached = cache::query_rumors(&client, &channel, None, 50, Some(&cord03::ROW_KINDS)).await?;
|
||||
```
|
||||
|
||||
A key's `retired_at` (a `Timestamp`, set when a rotation supersedes it) is a read
|
||||
cutoff: a wrap at that epoch with a later `created_at` is refused, so a retired
|
||||
epoch is history and never a live plane an ejected holder can keep writing into.
|
||||
|
||||
A relay is only ever *asked*: `history::page` installs **one REQ per page** over
|
||||
`ReqTarget::manual` for the community's own relays (which is what makes the client
|
||||
verify a wrap, deduplicate it and persist it), waits for every relay to settle,
|
||||
and then reads the page back out of the local database — no wrap is ever consumed
|
||||
straight off the wire, and `fetch_events` is not used anywhere. The wrap lands in
|
||||
the client's shared event store through that ordinary ingest path; `cache_rumor`
|
||||
then puts the decrypted rumor beside it, and only the rumor is ever served to a
|
||||
reader.
|
||||
|
||||
EOSE and CLOSED are **not** watched inside the page. The registry's notification
|
||||
pump is the only consumer of `client.notifications()`; it delivers one
|
||||
`PageReport` per settling relay into the `PageRegistry` under the page's
|
||||
subscription id, and the page waits on that channel with `PAGE_TIMEOUT`. The id
|
||||
is opaque (`concord-history-<n>`) because a NIP-01 id is capped at 64 characters,
|
||||
and the registry is what maps it back to the waiting round.
|
||||
|
||||
A page ends on EOSE. A CLOSED ends it as unanswered, *except* NIP-42's
|
||||
`auth-required`: the SDK re-issues that REQ under the same subscription id once
|
||||
the handshake completes, so the page waits for the resubscribed answer rather
|
||||
than writing off a relay that only wanted to authenticate.
|
||||
|
||||
`history::page` walks newest-first across every held epoch, caches what it opens, and
|
||||
reports what it saw: `oldest`/`newest` feed the caller's `ChannelCursor`,
|
||||
`exhausted` is earned only by a short page *after* history was seen, and an
|
||||
all-empty answer sets `failed` so a later round re-asks instead of sealing the
|
||||
channel at "no more history". `unreadable` counts the wraps the page reached that
|
||||
no held key could open — sealed past the cutoff their key's rotation set, or bound
|
||||
to another channel — because those are history the reader is missing, not history
|
||||
that is not there. Page down with `Window::older_than(seen.oldest)`,
|
||||
open a channel with `Window::opening(cursor)` (wide cold, `newest - 60s` warm),
|
||||
and read the region between two cursors with `Window::between(..)`. A `Window`
|
||||
carries `Timestamp`s, so its bounds go straight into a NIP-01 filter with no unit
|
||||
conversion; only a reader's millisecond `at_ms` narrows to a second at the query.
|
||||
`query_rumors` is the read path when the group keys are gone; pass `kinds` to
|
||||
budget rows apart from the events that only decorate them. `cache::wrapper_index`
|
||||
reads the cached rows back keyed by the wrap they came from, which is what lets
|
||||
`sync::fold` observe author and message times without re-opening a wrap it already
|
||||
cached, and `cache::purge_expired(client, &channel, now)` runs at the top of every
|
||||
round — the timer is cooperative, so the local store is the artifact that has to
|
||||
forget.
|
||||
|
||||
`sync::fold` counts the same thing over the whole store, per channel, in
|
||||
`Snapshot.unreadable`. `sync_round` sums a round's pages into
|
||||
`Progress.unreadable`, and `Community` keeps both per channel: `progress(channel)`
|
||||
is the last completed round, `unreadable(channel)` is the count, and
|
||||
`missing_key(channel)`, `channel_removed_at(channel)`, `removed_at()` and
|
||||
`stranded()` are the rest of the honest read surface a panel needs to tell an
|
||||
empty room from one it cannot read. `Community::due(channel)` is the other half —
|
||||
whether a round is worth asking for yet (a round for `Older` and an explicit retry
|
||||
never ask it), and `Community::tick`, driven by `CommunityRegistry` once per
|
||||
`community::MIN_ROUND_INTERVAL`, re-folds and re-rounds the active channel of a
|
||||
community whose last round is older than `STALE_AFTER`.
|
||||
|
||||
`ChatAction::TimerNotice { seconds }` is a policy notice, not a message: render it
|
||||
as an inline row only when its author passes
|
||||
`control.roles.is_authorized(&author, &owner, Permissions::MANAGE_METADATA)`.
|
||||
|
||||
## Membership
|
||||
|
||||
```rust
|
||||
let states = cord02::guestbook::coalesce(&rumors, now_ms, &refounders, |actor, target, citation| {
|
||||
citation_ok(&owner, &community_id, actor, citation, &control.roles.floors)
|
||||
&& control.roles.can_act_on_member(actor, &owner, target, Permissions::KICK)
|
||||
});
|
||||
let members = cord02::guestbook::complete_memberlist(&states, &observed, &granted, &control.banned, &BTreeMap::new());
|
||||
```
|
||||
|
||||
- `refounders` is the set of npubs whose rotations minted an epoch this client
|
||||
verified (`CommunityState.refounders`) — a snapshot chunk is honored only from
|
||||
one of them, and an empty set honors none. A rotation that delivered this
|
||||
client no key still names its minter, so the set is filled by any base rotation
|
||||
whose continuity verifies against a root held (`rekey::walk`); a List entry's
|
||||
`refounder`, read by `list::JoinMaterial::refounder`, is the other source.
|
||||
- `observed` is npub → ms for every author this client has seen publish anything
|
||||
usable, which is what makes a member visible before their Join arrives. Only
|
||||
count it forward.
|
||||
- `granted` is every npub the roster ranks; they are members with no Guestbook
|
||||
entry at all.
|
||||
- `banned_at` is empty today, so a ban is terminal in the fold. Fill it when the
|
||||
banlist head's timestamp is plumbed through.
|
||||
- Removal is three separate actions, composed by the caller: strip the grant
|
||||
(immediate and cheap), then the kick directive, then — for a ban — the rotation
|
||||
that actually enforces it.
|
||||
|
||||
## Moderation writes
|
||||
|
||||
Every Control Plane write goes through one writer and one edition shape:
|
||||
|
||||
```rust
|
||||
let writer = ControlWriter { author: my_pk, read: read.clone(), signer: signer.clone() };
|
||||
let head = control.floors.get(entity).cloned();
|
||||
|
||||
let (wrap, new_head) = writer.set_community_metadata(
|
||||
&my_keys, &community_id, &metadata, head.as_ref(), citation, now_secs).await?;
|
||||
```
|
||||
|
||||
`citation` is the `vac` the actor acts under — `None` only for the owner. Build it
|
||||
from the folded Grant that ranks them (`AuthorityCitation { entity, version, hash }`)
|
||||
and pass the head from the current fold, so the chain cannot silently fork.
|
||||
|
||||
Wrappers: `set_community_metadata`, `set_channel_metadata`, `set_role`,
|
||||
`set_grant`, `set_banlist`, `set_registry`, `set_pin_list`, plus raw `publish`.
|
||||
A ban is a `set_banlist` followed by a base rekey; a kick is a `set_grant` with an
|
||||
empty `role_ids` followed by `cord02::guestbook::build_kick`.
|
||||
|
||||
## Pins
|
||||
|
||||
```rust
|
||||
use concord::cord04::pins;
|
||||
|
||||
let entry = cord04::pins::build_entry(&opened_message, &plane, &channel)?;
|
||||
let head_content = control.pin_content(&community_id, &channel).unwrap_or("");
|
||||
let read = cord04::pins::read_list(head_content, |epoch| channel_group_key(&root, &channel, epoch).ok());
|
||||
let content = cord04::pins::publishable(&read, channel_is_private, &plane, epoch)?;
|
||||
let (wrap, _) = writer.set_pin_list(
|
||||
&my_keys, &community_id, &channel, &content, head, citation, now_secs).await?;
|
||||
```
|
||||
|
||||
Reading is verification: `read_list` decodes either content form (public, or
|
||||
sealed under the channel key at the named epoch), and
|
||||
`cord04::pins::verify_entry(entry, &channel)` returns a `VerifiedPin` with the proven
|
||||
author, words and time — no history and no old keys needed. `read.sealed` means
|
||||
the list is sealed under an epoch this client never held: show it as unavailable,
|
||||
and never write from it (`publishable` refuses). `cord04::pins::killed_by(&pin, &delete)`
|
||||
answers whether a folded kind-5 erases an entry.
|
||||
|
||||
## Invites
|
||||
|
||||
```rust
|
||||
use concord::derive::{invite_bundle_key, TOKEN_LEN};
|
||||
use concord::cord05::{self, InviteEntry, InviteTombstone};
|
||||
|
||||
let token: [u8; TOKEN_LEN] = /* 16 bytes from any CSPRNG */;
|
||||
let bundle_key = invite_bundle_key(&token);
|
||||
let link_signer = Keys::generate();
|
||||
let bundle = cord05::build_bundle_event(&link_signer, &invite, &bundle_key)?;
|
||||
let url = cord05::build_invite_url(BASE, &link_signer.public_key(), &token, &relays)?;
|
||||
```
|
||||
|
||||
A link is a coordinate plus a fragment: the naddr fetches the bundle, the token
|
||||
unlocks it, and the fragment names the relays to fetch from.
|
||||
`cord05::stock_relays()` is what a fragment with no relays of its own means.
|
||||
|
||||
The `link_signer` secret is what lets the creator refresh or retire the link, so
|
||||
keep it against the token in the member's own Invite List — a local document
|
||||
encrypted to self, exactly like the Community List:
|
||||
|
||||
```rust
|
||||
let mut list = cord05::parse_invite_list(&my_keys, &event).await?;
|
||||
list.entries.push(InviteEntry {
|
||||
token: HEXLOWER.encode(&token),
|
||||
signer_sk: link_signer.secret_key().to_secret_hex(),
|
||||
community_id: invite.community_id,
|
||||
url,
|
||||
label: None,
|
||||
created_at: now_ms,
|
||||
expires_at: None,
|
||||
extra: Default::default(),
|
||||
});
|
||||
let event = cord05::build_invite_list(&my_keys, &list).await?; // kind 13303
|
||||
|
||||
// Retiring is a tombstone, never a deletion: it beats a stale copy terminally.
|
||||
list.tombstones.push(InviteTombstone {
|
||||
token: HEXLOWER.encode(&token),
|
||||
community_id: invite.community_id,
|
||||
extra: Default::default(),
|
||||
});
|
||||
```
|
||||
|
||||
`merge_invite_lists` merges two devices' copies, `is_live(&token_hex)` answers
|
||||
whether a link still stands, and `fits()` is the write gate.
|
||||
|
||||
## Rekeys, refounding and dissolution
|
||||
|
||||
A rotation is authority plus delivery: `rekey_authorized(&control.roles, &owner, &me, permission, &removed)`
|
||||
gates it, `plan_rotation(scope, epoch)` mints what it delivers, and
|
||||
`build_rekey_chunks` seals one blob per remaining member:
|
||||
|
||||
```rust
|
||||
use concord::derive::epoch_key_commitment;
|
||||
use concord::cord06::{self, RekeyScope, RotationPlan};
|
||||
|
||||
let scope = RekeyScope::Channel(channel_id); // or RekeyScope::Base
|
||||
let plan = cord06::plan_rotation(scope, Epoch(epoch + 1))?;
|
||||
let new_key = plan.new_key();
|
||||
|
||||
// A base rotation delivers the new control-plane keys beside the root; a channel
|
||||
// rotation delivers only that channel's fresh key.
|
||||
let (control_pk, control_root) = match &plan {
|
||||
RotationPlan::Base(refounding) => {
|
||||
let pk = refounding.signer(&community_id)?.pk().to_bytes();
|
||||
(Some(pk), is_staff.then_some(&refounding.new_control_root))
|
||||
}
|
||||
RotationPlan::Channel { .. } => (None, None),
|
||||
};
|
||||
|
||||
let mut blobs = Vec::with_capacity(members.len());
|
||||
|
||||
for member in &members {
|
||||
blobs.push(
|
||||
cord06::build_blob(
|
||||
&my_keys,
|
||||
member,
|
||||
scope,
|
||||
plan.epoch(),
|
||||
&new_key,
|
||||
control_pk.as_ref(),
|
||||
control_root,
|
||||
)
|
||||
.await?,
|
||||
);
|
||||
}
|
||||
|
||||
let rekey_group = cord06::rekey_group(scope, &community_root, &community_id, plan.epoch())?;
|
||||
let wraps = cord06::build_rekey_chunks(
|
||||
&my_keys,
|
||||
&rekey_group,
|
||||
scope,
|
||||
plan.epoch(),
|
||||
Epoch(epoch),
|
||||
&epoch_key_commitment(Epoch(epoch), &community_root),
|
||||
&blobs,
|
||||
citation,
|
||||
false,
|
||||
now_secs,
|
||||
)
|
||||
.await?;
|
||||
```
|
||||
|
||||
On the receiving side, `cord06::parse_rekey_chunk(&opened)` per wrap, then
|
||||
`collect_rotations(&chunks)`, then `am_i_removed(&rotation, &me)` — which is
|
||||
`None` until every chunk is held, because an incomplete set is never a removal. A
|
||||
member finds their delivery with `find_my_blobs` / `open_blob`, and adopts the key
|
||||
only if the plaintext binds to the scope and epoch they expect and its `prevcommit`
|
||||
matches the key they already hold. Two concurrent rotations settle on `fork_winner`.
|
||||
|
||||
`community::rekey::adopt` is that receiver, run against the local database: it
|
||||
walks a scope forward one epoch at a time off the key actually held (never
|
||||
waiving a gap, never adopting a fork), keeps each stepped-off key as a prior
|
||||
with the rotation's publish time as its read cutoff, and reports a removal or a
|
||||
strand when a complete rotation carries no blob for the member.
|
||||
|
||||
`community::rekey::rotate` is the sender, run against the held state: it resolves
|
||||
the epoch and key the scope is stepping off, refuses a rotation that would skip or
|
||||
cut off its own rotator, delivers one blob per recipient (`Rewrite.recipients`,
|
||||
which the caller names because a private channel's audience is not in the local
|
||||
state), marks the rotation severed when it excludes somebody, and — for a base
|
||||
scope — carries the settled Control Plane heads onto the new epoch's groups with
|
||||
`cord06::compact`. `Community::rotate` checks `Rewrite::authorized` first, the same
|
||||
`rekey_authorized` under the same permissions `adopt` applies, publishes to the
|
||||
community's relays, and then adopts what it published through `rekey::adopt`
|
||||
rather than by construction: a rotation this client wrote and one it received take
|
||||
one path.
|
||||
|
||||
The blob plaintext is a fixed-width binary record, but a signer's NIP-44 is
|
||||
text-only, so `build_blob` carries it base64-encoded inside the envelope.
|
||||
`open_blob` mirrors that, so the record layout and the `locator` are unchanged.
|
||||
|
||||
Dissolution is owner-only and terminal:
|
||||
|
||||
```rust
|
||||
let rumor = cord06::dissolved_tombstone_rumor(owner_pk, &community_id, now_secs);
|
||||
let wrap = cord06::seal_dissolved(&rumor, &community_id, &my_keys, now_secs).await?;
|
||||
|
||||
// A receiver seals the community read-only on sight.
|
||||
if cord06::verify_dissolved(&wrap, &identity) {
|
||||
state.dissolved = true;
|
||||
}
|
||||
```
|
||||
|
||||
## The Community List
|
||||
|
||||
A member's own memberships, synced across their devices:
|
||||
|
||||
```rust
|
||||
use concord::cord02::list;
|
||||
|
||||
let entry = concord::state::list_entry(&state, &metadata.name); // state → §8 material
|
||||
let held = list::parse_list_event(&my_keys, &event).await?; // validates the d tag
|
||||
let mine = held.joined(entry); // community_id-keyed union
|
||||
let event = list::build_list_event(&my_keys, &mine, 0, now_secs).await?; // kind 33302, d = 0
|
||||
client.send_event(&event).to_nip65().await?; // account's own write relays
|
||||
```
|
||||
|
||||
The publish needs no separate database write: `send_event` persists the event
|
||||
locally *before* it resolves targets, so the fragment is available to the
|
||||
`concord/list` read path even if every relay is unreachable.
|
||||
|
||||
`join_material(&invite, control_root)` makes the §8 material from a CORD-05
|
||||
invite; `concord::state::list_entry(state, name)` makes it from a `CommunityState`, and is
|
||||
what a write path uses after a create, join or rename. `joined` and `tombstoned`
|
||||
are the two mutations: both are `community_id`-keyed unions, so neither an append
|
||||
nor a leave can lose a membership the other writer has.
|
||||
|
||||
The entry is signed by the member's real key and sealed to that same key
|
||||
(`seal_to_self`), so only the member's devices read it — a stranger's
|
||||
`parse_list_event` fails rather than returning a partial list. Publishing goes to
|
||||
the member's **NIP-65 write relays**, the same set the `concord/list`
|
||||
subscription resolves for its `author` filter.
|
||||
|
||||
Kind `33302` is **addressable and fragmented**: one event per fragment, its `d`
|
||||
tag the fragment index in decimal. `frags` in the payload declares how many the
|
||||
List has, and `is_complete(held_indices)` answers whether the client has a
|
||||
fragment at every index below it. `merge` resolves a `frags` disagreement to the
|
||||
larger value. (`13302`, the single-event List, is retired by the spec — a
|
||||
replaceable kind cannot fragment.)
|
||||
|
||||
A join material may also carry two hex extensions this crate does not write but
|
||||
does read, because they are the only place a device that never held a rotation can
|
||||
learn who minted an epoch: `refounder` names the npub whose Refounding minted the
|
||||
entry's `root_epoch`, and `held_roots` is the retained prior epochs
|
||||
(`[{epoch, key, refounder?, control_pk?, retired_at?}]`). `JoinMaterial::refounder()`
|
||||
and `JoinMaterial::held_roots()` decode both; unknown fields are round-tripped
|
||||
regardless, so a document that carries them keeps them.
|
||||
|
||||
The payload's 32-byte values are **unpadded base64url at every depth**, which is
|
||||
section-scoped to §8: CORD-05 invites stay hex. The writer re-encodes them on
|
||||
every serialization, so its output is always the canonical 43-character spelling;
|
||||
the reader also accepts non-zero trailing bits, because the spec's own worked
|
||||
example contains them and no reader can tell a mis-encoded named field from a
|
||||
correct one. The codec is `utils::base64url` and the wire structs behind the
|
||||
List's `Serialize`/`Deserialize` are the only callers, so no other encoding path
|
||||
is touched.
|
||||
|
||||
Three write-time rules are folded into serialization, so an in-memory document
|
||||
and its wire form differ:
|
||||
|
||||
- an embedded snapshot omits `community_id` and inherits the entry's;
|
||||
- `seed` is omitted when it equals `current`, and its cosmetic fields (`name`,
|
||||
`relays`, each channel's `name`) are overwritten from `current` first, so a
|
||||
rename collapses the snapshots instead of forking them;
|
||||
- an entry whose `added_at` does not outrun its tombstone is omitted — the
|
||||
tombstone alone carries the state.
|
||||
|
||||
`is_live(&id)` answers joined-versus-left: a tombstone is terminal until a
|
||||
strictly newer join outruns it. `fits()` is the write gate: 50 memberships and
|
||||
the NIP-44 plaintext cap. The 50 is a stopgap inherited from the retired
|
||||
single-event design — §8 has **no membership limit**, its only bound is the
|
||||
65,536-byte encoded event, and the real fix is to start a new fragment on write.
|
||||
Until that lands, an append onto a List that already spans more than one fragment
|
||||
is refused rather than performed against a partial read, because placing a new
|
||||
membership needs a repack. The community stays local (`load` keeps a membership
|
||||
the List never mentions) and the write is deferred with a warning.
|
||||
|
||||
Discovery is a **subscription, not a fetch**: subscribe with
|
||||
`Filter::new().kind(Kind::Custom(KIND_COMMUNITY_LIST)).author(my_pk)` and read the
|
||||
fragments back out of `client.database()`. The client persists a relay's event
|
||||
before it notifies, so a subscription plus a database read loses nothing and
|
||||
needs no explicit save. Parse each event with `parse_list_event`, keep the newest
|
||||
per `fragment_index`, and `merge` them — reading an incomplete List is safe, since
|
||||
a missing fragment is only news not yet heard.
|
||||
|
||||
## GPUI integration
|
||||
|
||||
`crates/concord` stays GPUI-free; the registry and sync engine live in
|
||||
`crates/community`. That layer adds a registry global and one
|
||||
entity per community, and moves every decrypt, verification, fold and I/O off
|
||||
the foreground thread.
|
||||
|
||||
### Entities
|
||||
|
||||
Same shape as `ChatRegistry`:
|
||||
|
||||
```rust
|
||||
pub fn init(cx: &mut App) {
|
||||
CommunityRegistry::set_global(cx.new(CommunityRegistry::new), cx);
|
||||
}
|
||||
|
||||
impl CommunityRegistry {
|
||||
pub fn global(cx: &App) -> Entity<Self> {
|
||||
cx.global::<GlobalCommunityRegistry>().0.clone()
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
Call it after `cord03::init` in `desktop/src/main.rs` and `web/src/lib.rs`, and
|
||||
subscribe to `NostrRegistry` for `SignerChanged` so the communities reset with
|
||||
the account.
|
||||
|
||||
- `CommunityRegistry` holds `communities: Vec<Entity<Community>>`, an index by
|
||||
`CommunityId`, and `tasks: SmallVec<[Task<Result<()>>; 2]>`.
|
||||
- `Community` owns one `CommunityState`, the last `ControlFold`, the member list
|
||||
and the channel list. Views render `Entity<Community>`; no protocol state
|
||||
lives in a view.
|
||||
- `CommunityState::apply_fold` is one assignment: run it in the task that
|
||||
produced the fold and send only the result to the foreground.
|
||||
- Emit an event on every fold so dependents re-read.
|
||||
|
||||
### Foreground and background
|
||||
|
||||
A background task never touches an entity. It sends results through a bounded
|
||||
`flume` channel that a foreground `cx.spawn` drains with `this.update(...)`.
|
||||
|
||||
```rust
|
||||
let (signal_tx, signal_rx) = flume::bounded::<Signal>(256);
|
||||
let client = client.clone();
|
||||
|
||||
// Background: open, verify, fold — no entities.
|
||||
self.ingress = Some(cx.background_spawn(async move {
|
||||
for wrap in &wraps {
|
||||
let Some(plane) = planes.iter().find(|plane| plane.group.pk() == wrap.pubkey) else {
|
||||
continue;
|
||||
};
|
||||
let (opened, rumor) = cord03::open(wrap, &plane.group, &plane.channel, plane.epoch)?;
|
||||
cache::cache_rumor(&client, &plane.channel, &opened).await?;
|
||||
signal_tx.send_async(Signal::Chat { channel: plane.channel, rumor }).await?;
|
||||
}
|
||||
Ok(())
|
||||
}));
|
||||
|
||||
// Foreground: the only place entities change.
|
||||
self.consumer = Some(cx.spawn(async move |this, cx| {
|
||||
while let Ok(signal) = signal_rx.recv_async().await {
|
||||
this.update(cx, |this, cx| this.apply(signal, cx))?;
|
||||
}
|
||||
Ok(())
|
||||
}));
|
||||
```
|
||||
|
||||
- Every cache function takes the `&Client` and reaches the database through
|
||||
`client.database()`, so clone the `Client` into the background task.
|
||||
- Keep long-lived tasks in fields — dropping a `Task` cancels it. Assign `None`
|
||||
to an `Option<Task<_>>` before respawning it; a signer change replaces both
|
||||
the listener and the consumer.
|
||||
- `cx.spawn` when the work updates an entity after awaiting, and
|
||||
`cx.background_spawn` when it only produces a value. A query the foreground awaits can be returned straight out: `fn messages(&self, cx: &App) -> Task<Result<Vec<ChatMessage>, Error>>`.
|
||||
- Do the first load in `cx.defer_in(window, ...)` so `init` returns before the
|
||||
first relay request.
|
||||
- NIP-46 signing is async: call `signer.get_public_key_async()` /
|
||||
`sign_event_async` inside the background task. Every account-key writer takes
|
||||
any signer (`Keys` or the app's `UniversalSigner`) and is `async`, so `await`
|
||||
it there rather than requiring device keys.
|
||||
|
||||
### Subscriptions
|
||||
|
||||
A wrap is addressed to a plane, so the plane's public key is the routing key and
|
||||
one `Filter` per held plane is enough:
|
||||
|
||||
```rust
|
||||
let filter = Filter::new()
|
||||
.kinds([Kind::from(KIND_WRAP), Kind::from(KIND_WRAP_EPHEMERAL)])
|
||||
.pubkey(plane.group.pk())
|
||||
.since(joined_at);
|
||||
client.subscribe(filter).with_id(sub_id).await?;
|
||||
```
|
||||
|
||||
- `pubkeys([...])` carries every plane of a community on one subscription. Call
|
||||
`subscribe` again with the new address whenever a join, a channel add or a
|
||||
rekey fold changes it.
|
||||
- Route inbound events by `subscription_id` from `RelayMessage::Event`, never by
|
||||
kind.
|
||||
- Watch one epoch ahead for the base, and a **window** of channel epochs under
|
||||
every held root for private channels: `base_rekey_group_key(&root_N, &id,
|
||||
Epoch(N + 1))`, plus `channel_rekey_group_key(&root, &channel, Epoch(channel_epoch
|
||||
+ ahead))` for `ahead in 1..=8` and each root in `state.roots()`. A Refounding
|
||||
seals its channel rekeys under the root current when it was minted, so a member
|
||||
who adopted the base rotation first must still ask under the prior root; the
|
||||
window is what lets a member who missed a rotation catch up, or learn they were
|
||||
cut. This is `community::rekey::watches`, installed as a second standing REQ
|
||||
whose id resolves back to its community through `rekey::WatchRegistry` (the id
|
||||
cannot carry a 64-hex community id within the NIP-01 length cap).
|
||||
- `community::rekey::adopt` then reads those wraps **from the database** and walks
|
||||
each scope forward one epoch at a time: a complete, authorized rotation whose
|
||||
`prevcommit` extends the key actually held hands over the next key (raced
|
||||
rotations settle on the lowest); a gap is fetched, never waived; a fork is never
|
||||
adopted. An addressed-but-unverifiable rotation is neither adopted nor read as a
|
||||
removal. Only a complete rotation at/after the join, from a rotator who outranks
|
||||
the member, with no blob for them, is a removal; one that predates the join is a
|
||||
strand — a stale invite landed the member on a superseded epoch.
|
||||
|
||||
### Tests
|
||||
|
||||
`cx.background_executor().timer(..)` for delays, never `smol::Timer`, or
|
||||
`run_until_parked()` finds nothing left to run. Push a wrap into the channel and
|
||||
`run_until_parked()` to drive the foreground consumer.
|
||||
|
||||
## Not wired up yet
|
||||
|
||||
- **`crates/concord` stays protocol-only; the registry lives in
|
||||
`crates/community`.** `concord` has no subscriptions, no `init`, and no
|
||||
`Entity<Community>`; `community::CommunityRegistry` owns one `Entity<Community>`
|
||||
per state document, subscribes when a community's plane set changes, and
|
||||
re-folds on an inbound wrap. The sidebar subscribes to the registry, surfaces
|
||||
`CommunityEvent::Error` as a window notification, and its "New community" row
|
||||
in the Communities tab dispatches `Command::NewCommunity`, whose name prompt
|
||||
calls `CommunityRegistry::create`. `create` persists the
|
||||
genesis locally, publishes the wraps to the community's relays, and records the
|
||||
membership in the account's Community List — all best-effort, so a relay that is
|
||||
down warns without losing the community. Discovery
|
||||
subscribes to the account's CORD-02 Community List (`33302`) under the
|
||||
`concord/list` subscription id and reads the fragments back out of
|
||||
`client.database()` — the SDK persists a relay's event before notifying, so the
|
||||
read is always current. A `concord/list` notification re-runs `load`, which
|
||||
materializes a community from each live List entry (`from_join_material`) and
|
||||
keeps any state document the List does not mention, so a fresh install — or one
|
||||
signing in as an account that joined elsewhere — finds its communities. See
|
||||
`docs/concord-community-discovery-plan.md` (including its note on the retired
|
||||
`13302` the current reference client still writes).
|
||||
- **Account-key writers take any signer, not `&Keys`.** `genesis`,
|
||||
`ControlWriter`, the guestbook and chat `seal_rumor`s, the `list` builders, and
|
||||
the `cord05` invite writers (`build_direct_invite` / `unwrap_direct_invite`,
|
||||
`build_invite_list` / `parse_invite_list`) and `cord06` blob writers
|
||||
(`build_blob` / `open_blob`) are `async` and generic over the SDK's
|
||||
`AsyncGetPublicKey` / `AsyncSignEvent` / `AsyncNip44` traits, so a `Keys` and an
|
||||
app `UniversalSigner` both work. The NIP-59 paths (`build_direct_invite`,
|
||||
`unwrap_direct_invite`) stay `Sized` because the SDK's gift-wrap helpers are.
|
||||
Group-key and locally-held-secret writers (`cord01` wrap functions,
|
||||
`cord05::build_bundle_event`, `community::cache`) still take the raw key material they
|
||||
genuinely need.
|
||||
- **`crates/chat/src/lib.rs::handle_notifications` treats every kind 1059 event as
|
||||
a NIP-59 gift wrap for the current user.** Concord wraps are kind 1059 too, so
|
||||
that handler must route by subscription id before any concord subscription goes
|
||||
live, or every stream wrap lands in the DM trash and raises a toast.
|
||||
- **Rotation-delivered plane keys are persisted, and history spans them.**
|
||||
`CommunityState.held_roots` keeps every root a rotation stepped off (with the
|
||||
retired root's Control signer and the publish time that retires it), and each
|
||||
`ChannelKeyRef.priors` keeps every channel key a rotation stepped off. The read
|
||||
side derives planes from all of them, and a retired key's `retired_at` is a hard
|
||||
read cutoff at both `history::page` and `sync::fold`. The state
|
||||
`Community::removed_at`/`Community::stranded`/`channel_removed_at` carries is
|
||||
rendered as a notice **and** enforced at write time: `Community::send` refuses
|
||||
when `channel_secret` is `None` (a removal, a strand, a channel cut, or a key we
|
||||
never held), because a wrap sealed under a superseded root would reach nobody who
|
||||
rotated. A rotation this client publishes itself (`Community::rotate`) is adopted
|
||||
through the same `rekey::adopt` an arriving one goes through, so the held state
|
||||
and the wire cannot diverge.
|
||||
@@ -1,650 +0,0 @@
|
||||
# NIP-29 groups backend
|
||||
|
||||
Coop has two relay-backed backends today: `crates/chat` (NIP-17 gift-wrap DMs,
|
||||
rendered by `crates/chat_ui`) and `crates/community` (Concord). This document is a
|
||||
plan, not a description of shipped code; nothing below is wired up yet. It covers
|
||||
the backend only — reading, writing, joining and leaving relay-based groups as
|
||||
defined by [NIP-29](https://github.com/nostr-protocol/nips/blob/master/29.md) —
|
||||
and ends with the hooks a UI will use later.
|
||||
|
||||
## The spec in backend terms
|
||||
|
||||
A group is a `(relay, id)` pair. The relay enforces membership and signs the
|
||||
group's state; there is no cross-relay identity, no encryption, and no portable
|
||||
"create". The same `id` on two relays is two groups (a fork), which the plan
|
||||
models as two entities.
|
||||
|
||||
| Concern | Wire | Backend rule |
|
||||
| --- | --- | --- |
|
||||
| Group id | random string, relay-chosen | `h` tag on everything a user sends; `d` tag on relay state events |
|
||||
| Timeline | any event kind + `h` | v1 renders kinds `1`, `9`, `11`, `1111` and `7`; every other kind is cached for the `previous` window but not rendered, except the membership kinds below |
|
||||
| Timeline refs | `previous` tag, first 8 hex chars of event ids | built from the last 50 events seen on that relay (any kind — NIP-29 timeline references), excluding our own; send up to 3 |
|
||||
| Relay state | `39000` metadata, `39001` admins, `39002` members, `39003` roles, `39004` LiveKit participants | relay-signed, addressable by `d`; read, never written by the client |
|
||||
| Membership | `9021` join, `9022` leave (user-signed); relay answers with `9000` put-user / `9001` remove-user | membership = latest `9000`/`9001` naming our pubkey; no event means not a member, and `Pending`/`Refused` are tracked locally for private groups |
|
||||
| Own group list | `10009` (NIP-51 simple groups) | `["group", id, relay, name?]` plus `["r", relay]`; replaceable, published to NIP-65 relays |
|
||||
| Reference | `naddr1…` of kind `39000` with the relay self pubkey and a relay hint, optionally `?invite=<code>` | the self pubkey is the author of the fetched `39000`; no `39000` means no canonical naddr |
|
||||
|
||||
Two behaviours that shape the code:
|
||||
|
||||
- **Late publication is rejected.** Relays drop messages with old timestamps, so
|
||||
there is no offline send queue in v1: a send either goes now or fails visibly.
|
||||
- **`previous` must reference events the relay already has.** Relays may enforce
|
||||
at least 3 references; a group we have little history for may have its sends
|
||||
rejected until the backlog loads. The relay's `OK false` message is surfaced
|
||||
as-is.
|
||||
|
||||
### Not in v1
|
||||
|
||||
Moderation writes (`9000`–`9010`), group creation (`9007`, relay-dependent),
|
||||
pinned events (`39005`/`9010`), subgroups (`parent`/`child` — parsed but inert),
|
||||
LiveKit audio/video (`livekit` tag, `39004`, the token endpoint), fork/migration
|
||||
detection through admins' `10009`, and relay browsing (`{"kinds":[39000]}` to
|
||||
list a relay's public groups). All are additive; none change the model above.
|
||||
|
||||
## What the local rust-nostr checkout gives us
|
||||
|
||||
Cross-checked against the pinned checkout
|
||||
(`~/.cargo/git/checkouts/nostr-619b808bb247a9ed/01f1950`):
|
||||
|
||||
- **Kinds exist** for every NIP-29 event in v1: `Kind::GroupPutUser` (9000),
|
||||
`GroupRemoveUser` (9001), `GroupEditMetadata` (9002), `GroupDeleteEvent`
|
||||
(9005), `GroupCreateGroup` (9007), `GroupDeleteGroup` (9008),
|
||||
`GroupCreateInvite` (9009), `GroupJoinRequest` (9021), `GroupLeaveRequest`
|
||||
(9022), `GroupMetadata` (39000), `GroupAdmins` (39001), `GroupMembers`
|
||||
(39002), `GroupRoles` (39003), `GroupLivekitParticipants` (39004), and
|
||||
`Kind::SimpleGroups` (10009). `Kind::ChatMessage` is kind 9.
|
||||
- **`9010` and `39005` are missing** from this revision (`Kind::Custom(9010)`,
|
||||
`Kind::Custom(39005)` if they are ever needed — they are not, in v1).
|
||||
- **There is no NIP-29 module.** `nostr/src/nips/` has no `nip29.rs`, and no
|
||||
`GroupId`, metadata type, parser or builder exists. Everything in `protocol.rs`
|
||||
below is ours; it is thin — tags and filters, no crypto.
|
||||
- **The APIs to lean on** are all present and already used in the app:
|
||||
- `Tag::custom`, `Tag::event`, `Tag::identifier`, `Tag::public_key`,
|
||||
`Tag::content`, `Tag::as_slice`.
|
||||
- `Filter::custom_tag(SingleLetterTag::LOWERCASE_H, id)`,
|
||||
`.identifier(id)`, `.author`, `.kinds`, `.limit`, `.since`.
|
||||
- `ReqTarget::single(url, filters)` for a group-scoped REQ,
|
||||
`client.subscribe(..).with_id(..)`, `client.fetch_events(..)`,
|
||||
`client.send_event(&event).to([url])`.
|
||||
- `EventBuilder::new(kind, content).tags(tags).finalize_async(&signer)` — the
|
||||
app's `UniversalSigner` works here (NIP-46 included), as it does in
|
||||
`crates/chat/src/room.rs`.
|
||||
- `client.database().query/save_event/delete`.
|
||||
- `Nip19Coordinate { coordinate, relays }` with `ToBech32`/`FromBech32` for
|
||||
`naddr`.
|
||||
- **NIP-11 is not exposed.** `Relay` in this SDK has no document accessor, so the
|
||||
relay self pubkey is taken from the author of the group's `39000` event; a
|
||||
group whose relay publishes no `39000` therefore has no canonical naddr and
|
||||
cannot be shared or joined in v1.
|
||||
|
||||
## What the app already does (and we copy)
|
||||
|
||||
The NIP-17 backend is the template. `crates/chat` has a global `ChatRegistry`
|
||||
entity, one `Room` entity per conversation, a background task over
|
||||
`client.notifications()` that sends signals through a `flume` channel, and a
|
||||
foreground `cx.spawn` consumer that is the only place entities change. Local
|
||||
messages are cached as `ApplicationSpecificData` (30078) events with a `d` tag
|
||||
for the target id and an `r` tag for the room key (`set_rumor`/`get_rumor`,
|
||||
`crates/chat/src/lib.rs`). `crates/chat_ui` renders `chat::Message` into a
|
||||
`ListState`, resolving names through `PersonRegistry`.
|
||||
|
||||
`crates/community` shows the same pattern for relay-scoped state: routing
|
||||
notifications by subscription id (`route_of`), bringing relays into the pool
|
||||
before use (`connect_relays`), and treating a `CLOSED` message as a refusal.
|
||||
Replacing a list is done in `crates/workspace/src/panels/contact_list.rs`:
|
||||
sign the builder, `client.send_event(&event).to_nip65().await` — the client saves
|
||||
the event locally even when the publish fails.
|
||||
|
||||
NIP-29 reuses all of this and drops what only NIP-17 needs: no encryption, no
|
||||
per-member fan-out, no gift wraps, no `SendReport` per receiver — one relay, one
|
||||
event, one `OK`.
|
||||
|
||||
## Where this lands
|
||||
|
||||
A new crate `crates/nip29`, because NIP-29 is a different transport (plain relay
|
||||
events), a different state authority (relay-signed `39000`s) and a different UI
|
||||
later. Making `chat::Room` polymorphic over gift-wrapped DMs and relay groups
|
||||
would put two protocols in one entity. The alternative — modules inside
|
||||
`crates/chat` — saves a `Cargo.toml` but was set aside (see Decisions).
|
||||
|
||||
| File | Owns |
|
||||
| --- | --- |
|
||||
| `crates/nip29/src/protocol.rs` | Pure NIP-29 types and their `impl` blocks: `GroupId`, `GroupKey`, `GroupReference`, `GroupMetadata`, admins/members/roles, `GroupList`, `Membership`, `TimelineWindow`. No I/O. |
|
||||
| `crates/nip29/src/group.rs` | `Group` entity: state, timeline, membership, `connect`, `ingest`, `send_message`, `send_reaction`, `join`, `leave`, `GroupEvent`. |
|
||||
| `crates/nip29/src/lib.rs` | `GroupsRegistry` global, `init`, standing subscriptions and the signal pump, the envelope cache, `load`/`publish` of the account's `10009`. |
|
||||
| `desktop/src/main.rs`, `web/src/lib.rs` | `nip29::init(cx)` after `chat::init(cx)`. |
|
||||
| UI later | `crates/chat_ui` (reuse `chat::Message` rendering) or a new `crates/nip29_ui`; sidebar section; a join dialog. |
|
||||
|
||||
`protocol` is re-exported at the crate root (`pub use protocol::*`) so callers
|
||||
write `nip29::GroupId`, `nip29::GroupMetadata`. The crate's dependencies are the
|
||||
ones `chat` already carries: `state` (client and signer), `chat` (`Message`),
|
||||
`gpui`, `nostr-sdk`, `anyhow`, `flume`, `futures`, `smallvec`, `log`;
|
||||
dev-dependencies `nostr-memory` and `nostr-sdk` with the `local-relay` feature
|
||||
for the tests below.
|
||||
|
||||
## Data flow
|
||||
|
||||
```
|
||||
own 10009 (NIP-65 relays) group relay (one per group)
|
||||
│ load / subscribe │ REQ #h=<id> (timeline + 9000/9001)
|
||||
▼ │ REQ #d=<id> (39000..39003)
|
||||
GroupsRegistry ──ensure relay──► Group entity ◄──ingest── background listener
|
||||
│ index: GroupKey │ (saves envelope first)
|
||||
│ │
|
||||
└── flume Signal ◄──────────────────────┘
|
||||
▲
|
||||
UI later: GroupPanel (Vec<Message>), JoinDialog (naddr)
|
||||
```
|
||||
|
||||
A group's subscriptions are the standing `grp-<hash>` REQ (timeline + state +
|
||||
membership, the timeline replayed up to 100 events) and, on demand, a
|
||||
`pg-<hash>` REQ that re-asks the same timeline filter with `until` set to the
|
||||
oldest event hydrated so far. Both are recognised by the same pump and land in
|
||||
the same envelope cache, so paging needs no separate read path.
|
||||
|
||||
## Phase 0 — `crates/nip29/src/protocol.rs` (pure, no I/O)
|
||||
|
||||
The protocol is expressed as types with `impl` blocks: a parsed thing is a struct
|
||||
and every operation on it is a method, so call sites read `id.message(...)`,
|
||||
`metadata.naddr(...)`, `admins.contains(&pk)` instead of handing the pieces to
|
||||
free builder functions.
|
||||
|
||||
```rust
|
||||
/// The relay-chosen group id.
|
||||
pub struct GroupId(String);
|
||||
|
||||
impl GroupId {
|
||||
pub fn new(id: impl Into<String>) -> Result<Self, Error>; // trimmed, non-empty
|
||||
pub fn as_str(&self) -> &str;
|
||||
pub fn h_tag(&self) -> Tag; // ["h", id]
|
||||
|
||||
pub fn timeline_filter(&self, limit: usize) -> Filter; // #h = id
|
||||
pub fn state_filter(&self) -> Filter; // #d = id, kinds 39000..39003
|
||||
pub fn membership_filter(&self, me: PublicKey) -> Filter; // #h = id, #p = me, kinds 9000/9001
|
||||
|
||||
pub fn message(&self, content: String, replies: &[EventId], previous: Option<Tag>) -> EventBuilder; // kind 9
|
||||
pub fn reaction(&self, target: EventId, emoji: String, previous: Option<Tag>) -> EventBuilder; // kind 7 + e
|
||||
pub fn join_request(&self, code: Option<&str>, reason: &str) -> EventBuilder; // 9021
|
||||
pub fn leave_request(&self, reason: &str) -> EventBuilder; // 9022
|
||||
}
|
||||
|
||||
/// Entity identity: the same id on another relay is another group (a fork).
|
||||
pub struct GroupKey { relay: RelayUrl, id: GroupId }
|
||||
|
||||
impl GroupKey {
|
||||
pub fn new(relay: RelayUrl, id: GroupId) -> Self;
|
||||
pub fn relay(&self) -> &RelayUrl;
|
||||
pub fn id(&self) -> &GroupId;
|
||||
pub fn cache_tag(&self) -> String; // "<relay>|<id>"
|
||||
pub fn uniq_id(&self) -> u64; // hashed, for panel and element ids
|
||||
}
|
||||
|
||||
/// What a user pastes to join: naddr1…(+"?invite=<code>").
|
||||
pub struct GroupReference { id: GroupId, relay: RelayUrl, invite: Option<String> }
|
||||
|
||||
impl GroupReference {
|
||||
pub fn parse(input: &str) -> Result<Self, Error>;
|
||||
pub fn id(&self) -> &GroupId;
|
||||
pub fn relay(&self) -> &RelayUrl;
|
||||
pub fn invite(&self) -> Option<&str>;
|
||||
pub fn key(&self) -> GroupKey;
|
||||
}
|
||||
|
||||
impl FromStr for GroupReference { .. } // so a pasted naddr can use `.parse()` too
|
||||
|
||||
/// kind:39000, relay-signed.
|
||||
pub struct GroupMetadata {
|
||||
pub name: Option<String>, pub picture: Option<String>, pub banner: Option<String>,
|
||||
pub about: Option<String>, pub private: bool, pub restricted: bool, pub hidden: bool,
|
||||
pub closed: bool, pub livekit: bool, pub parent: Option<GroupId>,
|
||||
pub supported_kinds: Option<Vec<Kind>>, // None = unspecified (all); Some(vec![]) = none
|
||||
}
|
||||
|
||||
impl GroupMetadata {
|
||||
pub fn parse(event: &Event) -> Result<Self, Error>;
|
||||
pub fn name(&self) -> Option<&str>;
|
||||
pub fn supports(&self, kind: Kind) -> bool;
|
||||
pub fn naddr(&self, relay: &RelayUrl, invite: Option<&str>) -> Result<String, Error>;
|
||||
// Coordinate { kind: 39000, public_key: <the event author>, identifier: <the d tag> }
|
||||
}
|
||||
|
||||
/// kind:39001 — each pubkey with its role labels.
|
||||
pub struct GroupAdmins(Vec<(PublicKey, Vec<String>)>);
|
||||
|
||||
impl GroupAdmins {
|
||||
pub fn parse(event: &Event) -> Result<Self, Error>;
|
||||
pub fn entries(&self) -> &[(PublicKey, Vec<String>)];
|
||||
pub fn contains(&self, public_key: &PublicKey) -> bool;
|
||||
}
|
||||
|
||||
/// kind:39002 — may be absent or a subset; absence means "not published".
|
||||
pub struct GroupMembers(Vec<PublicKey>);
|
||||
|
||||
impl GroupMembers {
|
||||
pub fn parse(event: &Event) -> Result<Self, Error>;
|
||||
pub fn contains(&self, public_key: &PublicKey) -> bool;
|
||||
pub fn iter(&self) -> impl Iterator<Item = &PublicKey>;
|
||||
}
|
||||
|
||||
/// kind:39003 — role names are relay-defined.
|
||||
pub struct GroupRoles(Vec<Role>);
|
||||
pub struct Role { pub name: String, pub description: Option<String> }
|
||||
|
||||
impl GroupRoles {
|
||||
pub fn parse(event: &Event) -> Result<Self, Error>;
|
||||
pub fn roles(&self) -> &[Role];
|
||||
}
|
||||
|
||||
/// kind:39004 — who is live in the group's AV room; `participant` tags, not `p`.
|
||||
/// Absence means nothing is known, which is not the same as an empty room.
|
||||
pub struct GroupParticipants(Vec<PublicKey>);
|
||||
|
||||
impl GroupParticipants {
|
||||
pub fn parse(event: &Event) -> Result<Self, Error>;
|
||||
pub fn contains(&self, public_key: &PublicKey) -> bool;
|
||||
pub fn iter(&self) -> impl Iterator<Item = &PublicKey>;
|
||||
}
|
||||
|
||||
/// kind:10009 — the account's own list.
|
||||
pub struct GroupList(Vec<GroupListEntry>);
|
||||
pub struct GroupListEntry { pub id: GroupId, pub relay: RelayUrl, pub name: Option<String> }
|
||||
|
||||
impl GroupList {
|
||||
pub fn parse(event: &Event) -> Result<Self, Error>;
|
||||
pub fn entries(&self) -> &[GroupListEntry];
|
||||
pub fn upsert(&mut self, entry: GroupListEntry);
|
||||
pub fn remove(&mut self, key: &GroupKey);
|
||||
pub fn to_builder(&self) -> EventBuilder; // kind 10009, group + r tags
|
||||
}
|
||||
|
||||
pub enum Membership {
|
||||
Unknown,
|
||||
Pending { since: Timestamp }, // our 9021 went out; no answer yet
|
||||
Member,
|
||||
Refused { reason: Option<String> }, // the relay's OK false / CLOSED message
|
||||
Removed,
|
||||
}
|
||||
|
||||
impl Membership {
|
||||
/// Folds 9000/9001 only; `Pending` and `Refused` are set by the join path.
|
||||
pub fn from_events(events: &[Event], me: PublicKey) -> Self;
|
||||
}
|
||||
|
||||
/// The newest-first window (≤50) that `previous` references are drawn from.
|
||||
pub struct TimelineWindow(Vec<Event>);
|
||||
|
||||
impl TimelineWindow {
|
||||
pub fn new() -> Self;
|
||||
pub fn push(&mut self, event: Event); // dedups, caps at 50
|
||||
pub fn previous_tag(&self, me: PublicKey) -> Option<Tag>; // excludes `me`, ≤3 distinct 8-hex
|
||||
}
|
||||
```
|
||||
|
||||
Details that must be right:
|
||||
|
||||
- Every user event carries `["h", id]` (from `GroupId::h_tag`); `previous` is
|
||||
added only when we have at least one eligible event, and its values are the
|
||||
first 8 characters of the hex ids.
|
||||
- `GroupReference::parse` splits `?invite=` before bech32 decoding; a missing
|
||||
relay hint is an error (there is nowhere to send the join request).
|
||||
- `GroupList::parse` accepts `group` tags with 1–3 values (id, relay, name).
|
||||
- `Membership::from_events` takes the newest `9000`/`9001` whose `p` tags name
|
||||
`me`; on an exact tie it prefers `9001` (removal). `Pending` and `Refused` are
|
||||
not things the relay folds into an event: the join path sets them from the
|
||||
`9021` send and from the relay's refusal message.
|
||||
- `GroupMetadata::supports` distinguishes "tag absent" from "tag present with no
|
||||
values": the first means all kinds, the second means no text messages.
|
||||
- The builders return `EventBuilder`s so the caller decides when to sign; nothing
|
||||
in this file does I/O.
|
||||
|
||||
## Phase 1 — read path (`group.rs` + `lib.rs`)
|
||||
|
||||
`GroupsRegistry` (global entity, mirroring `CommunityRegistry`):
|
||||
|
||||
```rust
|
||||
pub struct GroupsRegistry {
|
||||
groups: Vec<Entity<Group>>,
|
||||
index: HashMap<GroupKey, Entity<Group>>,
|
||||
/// The account's 10009, loaded at start and mutated by join/leave
|
||||
list: GroupList,
|
||||
/// subscription id → group, rebuilt whenever a group resubscribes
|
||||
routes: Arc<RwLock<HashMap<SubscriptionId, GroupKey>>>,
|
||||
signal_tx: flume::Sender<Signal>, signal_rx: flume::Receiver<Signal>,
|
||||
notification_listener: Option<Task<Result<()>>>, // cancelled on signer change
|
||||
signal_consumer: Option<Task<Result<()>>>,
|
||||
tasks: SmallVec<[Task<Result<()>>; 2]>,
|
||||
_subscriptions: SmallVec<[Subscription; 2]>,
|
||||
}
|
||||
|
||||
impl GroupsRegistry {
|
||||
pub fn global(cx: &App) -> Entity<Self>;
|
||||
pub fn groups(&self) -> &[Entity<Group>];
|
||||
pub fn group(&self, key: &GroupKey) -> Option<Entity<Group>>;
|
||||
pub fn join(&mut self, reference: GroupReference, cx: &mut Context<Self>);
|
||||
pub fn leave(&mut self, key: &GroupKey, cx: &mut Context<Self>);
|
||||
fn load(&mut self, cx); // own 10009 from the database into `self.list`
|
||||
fn publish_list(&mut self, cx); // sign `self.list.to_builder()`, to_nip65, best-effort
|
||||
}
|
||||
```
|
||||
|
||||
- **Subscriptions.** One standing REQ per group, two filters in one id:
|
||||
`key.id().timeline_filter(100)` and `key.id().state_filter()`. The id is
|
||||
`grp-<8 hex of the group key hash>` (NIP-01 caps ids at 64 chars; the real key
|
||||
lives in `routes`). The relay is added and connected first
|
||||
(`add_relay(url).and_connect()`), like `community::sync::connect_relays`. On
|
||||
open, also issue a one-shot `key.id().membership_filter(me)` fetch so a restart
|
||||
learns `9000`/`9001` immediately; the state filter is fetched once (limit 1 per
|
||||
kind) so the sidebar has names before the live REQ settles. A dead relay costs
|
||||
only its own group.
|
||||
- **Pump.** The listener matches `ClientNotification::Message` and routes
|
||||
`RelayMessage::Event`/`EndOfStoredEvents`/`Closed` by subscription id through
|
||||
`routes`. For `h`-tagged events it saves the envelope first, then sends
|
||||
`Signal::Event(key, event)`; state events send `Signal::State(key, event)`;
|
||||
`Closed` sends `Signal::Refused(key, message)`. The consumer calls
|
||||
`Group::ingest`, which is the only place entity state changes.
|
||||
- **Envelope cache.** Same shape as `set_rumor`/`get_rumor` in `crates/chat`:
|
||||
`ApplicationSpecificData` (30078, addressable) with `d` = event id (dedup on
|
||||
replay), `r` = `"<relay>|<id>"` (per-fork history), `k` = kind, content = the
|
||||
raw event JSON, signed by a crate-local `LOCAL_KEYS`. Every `h`-tagged event
|
||||
the standing subscription delivers is enveloped — any kind, because the
|
||||
`previous` window is literally "the last 50 events seen" (NIP-29 timeline
|
||||
references) — while the `d`-tagged state events (`39000`–`39003`) are not.
|
||||
Reading is `Filter::new().custom_tag(LOWERCASE_R, key)`, parsed and sorted by
|
||||
`created_at`, then partitioned: the render set (`1`, `7`, `9`, `11`, `1111`)
|
||||
becomes messages, `9000`/`9001` fold membership, the rest only feed the
|
||||
window. This is what makes the timeline fork-safe and restart-safe without
|
||||
teaching the database about relays.
|
||||
- **`Group` state.** `GroupKey`, `metadata: Option<GroupMetadata>`,
|
||||
`metadata_event: Option<Event>` (what `GroupMetadata::naddr` is built from),
|
||||
`members`, `admins`, `roles`, `membership: Membership` (including the local
|
||||
`Pending`/`Refused` access states), `messages: Vec<Message>` for the render set
|
||||
(reusing `chat::Message`, which already parses any event into display fields),
|
||||
an id index, a `TimelineWindow` for `previous`, `loading`, `refused`. It emits
|
||||
`GroupEvent::{Incoming(Event), Updated, Refused(String), Eose}` — the panel
|
||||
subscribes to these the way `ChatPanel` subscribes to `RoomEvent`.
|
||||
- **`load()`.** Subscribe to our own `10009` (`ReqTarget::auto`, limit 1), then
|
||||
read the newest one from the database into `self.list`, and add one group per
|
||||
entry. A newer `10009` arriving from another device is merged
|
||||
(`GroupList::upsert`) and the new groups are subscribed; removals by another
|
||||
client cannot be detected in a replaceable list and are not guessed at.
|
||||
|
||||
## Phase 2 — write path
|
||||
|
||||
All writes sign with the app signer (`finalize_async(&signer)`), send to the
|
||||
group's relay only, and treat the relay's `OK` as the truth.
|
||||
|
||||
- **Send a message.** Build with
|
||||
`key.id().message(content, &replies, window.previous_tag(me))`, sign, then
|
||||
`client.send_event(&event).to([relay])`. Kind 9 is the default message kind;
|
||||
when `metadata.supports(Kind::ChatMessage)` is false the composer is disabled
|
||||
rather than sending a kind the relay does not take. On acceptance: save the
|
||||
envelope, insert into `messages`, emit `GroupEvent::Incoming`. On `OK false`:
|
||||
return the relay's message (it already carries `invalid:` / `restricted:` /
|
||||
`duplicate:` prefixes) and let the panel show it. Reactions go through the same
|
||||
path with `key.id().reaction(..)`.
|
||||
- **Join.** `GroupReference::parse` → error if no relay hint → add/connect the
|
||||
relay → fetch `39000` (`limit 1`) to confirm the group and learn the self
|
||||
pubkey; `GroupMetadata::parse` it; a hidden group may refuse this — proceed
|
||||
without metadata rather than failing → create the entity, fetch state +
|
||||
membership, subscribe → send `key.id().join_request(code, ..)` — the `h` tag
|
||||
plus the `code` tag when an invite code was parsed → mark
|
||||
`Membership::Pending { since }` and `self.list.upsert(..)`, so the group stays
|
||||
listed while it waits. The relay either emits `9000` (member), refuses the
|
||||
`9021` with `OK false` (set `Refused { reason }` from its message — the NIP
|
||||
says the message explains whether the rejection is final, pending review or
|
||||
payment-required), or refuses the timeline REQ with `restricted:` /
|
||||
`auth-required:` in a `CLOSED` (keep the state, do not raise an error toast).
|
||||
All three are observed on the standing subscription or the send result.
|
||||
- **Leave.** Send `key.id().leave_request(..)`; on `OK`, forget the group's
|
||||
envelopes (`client.database().delete(filter)` by `r` tag), drop the entity and
|
||||
`self.list.remove(key)`, then publish. The relay also emits `9001`, which the
|
||||
(now gone) subscription would have seen.
|
||||
- **Publish `10009`.** Sign `self.list.to_builder()` and publish with
|
||||
`to_nip65()`. The registry's `GroupList` is the one loaded at start, join
|
||||
`upsert`s an entry and leave `remove`s one, so a rewrite can never drop a group
|
||||
we loaded. The client saves the event locally even when the publish fails, so
|
||||
the list survives offline.
|
||||
|
||||
## Phase 3 — later, in rough order
|
||||
|
||||
1. Moderation writes: `9000` put-user (roles), `9001` remove-user, `9002`
|
||||
edit-metadata, `9005` delete-event, `9009` create-invite; gate the UI on the
|
||||
caller appearing in `39001`, let the relay be the enforcer. **Done** as pure
|
||||
writes: builders on `GroupId`, senders on `Group` sharing `dispatch`.
|
||||
`edit_metadata` takes a `GroupMetadata`, which is now `Default`-able (its
|
||||
private `id`/`author` origin are `Option`) so the UI can start an edit from
|
||||
scratch or from `Group::metadata()`. **`9005` is write-only for now**: the
|
||||
relay stops serving the event but nothing is removed locally, so the deleted
|
||||
message lingers in the cache and timeline. The UI phase must drop the target
|
||||
envelope (filter on its `d` = target id and the group's `r` cache tag) and
|
||||
evict it from `messages`/`index`, otherwise only a reload clears it.
|
||||
2. Group creation (`9007`) where the relay supports it, and relay browsing.
|
||||
**Done.** `GroupId::create_group` reuses the `9002` metadata tags, so the
|
||||
proposed id (`h`) and the initial name/flags go in one event; the relay
|
||||
republishes `39000`, which the standing subscription picks up.
|
||||
`GroupsRegistry::create` mirrors `join`: `ensure` → optimistic `10009` entry
|
||||
+ `publish_list` → publish, and on `OK false` it `forget`s the group and emits
|
||||
`GroupsEvent::Error(reason)` (that variant was previously unused). There is
|
||||
no NIP-11 support check available, so "where the relay supports it" is
|
||||
enforced by the relay refusing the `9007`.
|
||||
`GroupsRegistry::browse(relay, cx)` follows the same pipeline as every
|
||||
other read: it registers a `brw-<hash>` subscription (an opaque id, like
|
||||
`grp-<hash>`; the relay is keyed in state, not encoded in the id, because the
|
||||
pump needs a prefix to recognise its own ids and a bare
|
||||
`SubscriptionId::generate()` would be indistinguishable from the other
|
||||
crates' ids on the shared client). `add_relay` +
|
||||
`subscribe` (auto-close on EOSE, 10s cap), and returns immediately. The
|
||||
**global pump** observes the events, and `handle_event`/`handle_eose`/
|
||||
`handle_closed` fold each relay-signed `39000` into `self.browse`, keyed by
|
||||
the subscription id, so no DB scoping is needed. `GroupsRegistry::browsed(relay)`
|
||||
returns `Browse { relay, loading, groups: Vec<GroupCandidate> }` for the UI, and
|
||||
`GroupsEvent::Updated` says when to re-read it. No `fetch_events` anywhere:
|
||||
the DB cannot be scoped to a relay (a stored event carries no origin), so the
|
||||
subscription's own events are the source. `join` now delegates to a new
|
||||
`join_key(key, invite)`, which is what a browse result needs (a candidate has
|
||||
a key, not a `GroupReference`).
|
||||
3. Pins and subgroups. **Pins done:** `Pin::{Event, Address}` and
|
||||
`GroupPins` in `protocol.rs`, with `PIN_LIST` (39005) and `UPDATE_PIN_LIST`
|
||||
(9010) as `u16` consts used through `Kind::Custom`. `39005` joins
|
||||
`STATE_KINDS`, so the standing subscription fetches it, `absorb_state` folds
|
||||
it into `Group::pins()`, and `Group::update_pins(pins)` writes `9010` through
|
||||
the shared `dispatch`. **Subgroups parsed:** `GroupMetadata` gained
|
||||
`pub children: Vec<GroupId>` (order preserved, parsed from `child` tags) and
|
||||
`metadata_tags` re-emits them — `9002` requires it, an edit that drops a
|
||||
child is rejected. `parent`/`children` are per-relay, so assembling the tree
|
||||
from `Group::metadata()` is UI work; there is no backend tree helper.
|
||||
4. Range backfill: page the timeline with `until` for "load more". **Done.**
|
||||
`GroupId::timeline_page(limit, until)` is `timeline_filter` plus `.until(..)`,
|
||||
and `Group::load_page` sends it as a `pg-<hash>` subscription (auto-close on
|
||||
EOSE, 10s cap) alongside the standing `grp-<hash>` one, which keeps running so
|
||||
paging cannot replace the live tail. The read path is the usual one: `pg-`
|
||||
joins `is_own_subscription`, the id is registered in `routes` against the same
|
||||
key, and `handle_event` therefore caches each page event and `reload`s the
|
||||
group, so it arrives through the envelope cache and the timeline like any
|
||||
other event. `GroupsRegistry::load_more(key, cx)` is the entry point: it does
|
||||
nothing while a page is in flight (`Group::loading_more`) or before anything is
|
||||
cached, and registers the route before `Group::load_page` sets the flag.
|
||||
`handle_eose` and a page `CLOSED` remove the route and call
|
||||
`Group::page_finished`, which clears the flag and emits `GroupEvent::Updated`
|
||||
so a panel's spinner can settle. The cursor is `Group::oldest()`, the oldest
|
||||
`h`-tagged event ever hydrated, updated by `track` next to `window.push` —
|
||||
*not* read from `TimelineWindow`, which keeps only the newest 50 events, so a
|
||||
cursor taken from it would never move and would page the same window forever.
|
||||
`until` is inclusive, so a page repeats the event it starts from; `hydrate`
|
||||
dedups by id, so that costs one redundant fetch, never a gap. `forget` also
|
||||
drops and unsubscribes the `pg-` route (`reset` already sweeps `routes`). Two
|
||||
gaps left for the UI phase: a relay that accepts the REQ but never EOSEs leaves
|
||||
`loading_more` true (the auto-close timeout closes the subscription locally and
|
||||
emits no notification — the same stickiness as `Browse::loading`), and a group
|
||||
whose next page shares the `until` second cannot advance the cursor, so it can
|
||||
re-fetch the same page. Neither loops on its own; both need a "load older
|
||||
messages" button bound to `Group::loading_more()` to be observable at all.
|
||||
5. LiveKit AV and fork/migration detection, split in three:
|
||||
1. **`39004` participants. Done.** `GroupParticipants` joins `protocol.rs`
|
||||
with the same shape as `GroupMembers` but reads `participant` tags (the NIP
|
||||
does not use `p` here); one malformed pubkey fails the whole event, as in
|
||||
`GroupAdmins`. The kind joins `STATE_KINDS`, so the standing subscription
|
||||
already carries it, and `absorb_state` folds it into `Group::participants()`.
|
||||
Nothing is written by us — the relay publishes it and updates it as people
|
||||
come and go, so it rides the live tail like any other state event.
|
||||
"Absent" and "empty" are deliberately the same value here: unlike
|
||||
`supported_kinds`, the NIP gives the empty list no special meaning, so the
|
||||
UI reads "no one is live" either way.
|
||||
2. The `/.well-known/nip29/livekit/<group-id>` token endpoint and the `204`
|
||||
probe at `/.well-known/nip29/livekit`. **Done.** The HTTP client is the one
|
||||
GPUI already owns: `cx.http_client()` returning `Arc<dyn HttpClient>`,
|
||||
already wired to `reqwest_client::ReqwestClient` on desktop and to `fetch`
|
||||
on wasm — the same path `state::nip05` uses, which is why no HTTP
|
||||
dependency lands in this crate. Its `get()` convenience cannot set headers,
|
||||
so the request goes through `HttpClient::send` with a `Builder`, an
|
||||
explicit `RedirectPolicy::NoFollow` (the `Authorization` header is signed
|
||||
for exactly one URL, so a redirect could only ever fail verification) and a
|
||||
10s `timeout` — both are opt-in extensions, and a request without them has
|
||||
none. The header itself is NIP-98 from the SDK: `nostr/nip98` is now
|
||||
enabled on the workspace `nostr` entry and nip29 declares `nostr` itself,
|
||||
so `HttpData::new(url, HttpMethod::GET).to_authorization(&signer)` builds
|
||||
`Nostr <base64>` — the `u` + `method` tags that the crate's own
|
||||
`verify_auth_header` demands, though the NIP's prose mentions only `u`.
|
||||
`GroupKey::livekit_token_url` / `livekit_probe_url` derive the endpoint
|
||||
purely: the relay URL is rewritten to http(s) (keeping host and port),
|
||||
reset to the origin, and the path built with `pop_if_empty().push(..)` so
|
||||
each segment is escaped — a relay with its own path prefix loses it, since
|
||||
the NIP puts the endpoint at the absolute path. `Group::livekit_session`
|
||||
and `Group::livekit_supported` are the two calls, both `&App` +
|
||||
`background_spawn` like `publish`. The session is returned, not stored:
|
||||
tokens are per-join artifacts, not group state, so the caller owns it.
|
||||
Only `livekit_supported`'s unauthenticated GET ignores status beyond
|
||||
`204`; the token endpoint's response shape is **not specified by the NIP**
|
||||
(see Decisions) and no LiveKit client library is involved — the backend
|
||||
stops at the JWT and server URL.
|
||||
3. Fork/migration detection. **Done.** One standing `nip29-forks`
|
||||
subscription over `{authors: <every admin of every group>, kinds: [10009]}`
|
||||
on `ReqTarget::auto` — the admins' own group lists live on their outbox
|
||||
relays, so this works with the group's relay down, and the admin pubkeys
|
||||
come from `39001`, which is hydrated from the envelope cache, so it works
|
||||
after a restart too. Each event is folded into `forks` (author → newest
|
||||
`GroupList`, guarded on `created_at` because `10009` is replaceable), then
|
||||
`evaluate_forks` recomputes every group. `Group::elsewhere()` is `Some`
|
||||
when an admin of that group lists the same group id at a different relay;
|
||||
the data cannot distinguish a move from a fork, hence the neutral name.
|
||||
Evaluation is incremental, on each event, rather than batched on EOSE:
|
||||
`ReqTarget::auto` fans out to several relays, so the first EOSE would
|
||||
otherwise evaluate — and clear — against one relay's partial view.
|
||||
Nothing is auto-joined, since the NIP asks the client to *offer*; the
|
||||
offer is `GroupsRegistry::join_key(elsewhere, None, cx)`, which is exactly
|
||||
"fetch from the new relay and update our own `10009`". Two triggers,
|
||||
because the NIP's "periodically" and its "MUST when the relay is offline"
|
||||
are different things. Reactively, the group observer calls
|
||||
`sync_forks(false)` on `GroupEvent::Updated`, subscribing as soon as the
|
||||
first `39001` lands and re-subscribing only when the admin set actually
|
||||
changes — the guard matters, since `Updated` also fires for metadata,
|
||||
pins, membership and a finished page. Periodically, a
|
||||
`background_executor().timer` loop started in `load` pushes
|
||||
`Signal::Forks` every 15 minutes and `sync_forks(true)` re-queries
|
||||
regardless: that is the recovery path for a subscription that died or a
|
||||
client that was offline, not the detection path. `reset` drops the fork
|
||||
state and unsubscribes the REQ; a `CLOSED` on it is only logged, because
|
||||
the tick retries and there is nothing to repair.
|
||||
|
||||
## UI later — the hooks this backend leaves
|
||||
|
||||
| UI action | Call |
|
||||
| --- | --- |
|
||||
| Paste an naddr and join (or retry a refused join) | `GroupReference::parse` → `GroupsRegistry::join` |
|
||||
| Open a group | `GroupsRegistry::group(&key, cx)` → panel subscribes to `GroupEvent` |
|
||||
| Timeline | `Group::messages()` (already `chat::Message`s) |
|
||||
| Load older messages | `GroupsRegistry::load_more(&key, cx)`; spinner from `Group::loading_more()` |
|
||||
| Send / react | `Group::send_message` / `send_reaction` |
|
||||
| Members, admins, roles, flags | `Group::members/admins/roles/metadata` |
|
||||
| Who is in the AV room | `Group::participants()` |
|
||||
| Join AV (get a LiveKit session) | `Group::livekit_session(cx)` → `await` → `LivekitSession { token, server_url }` |
|
||||
| Is AV available here | `Group::livekit_supported(cx)` after seeing `metadata().livekit` |
|
||||
| The group may have moved or been forked | `Group::elsewhere()` → offer `GroupsRegistry::join_key(elsewhere, None, cx)` |
|
||||
| Membership badge (member, pending, refused, removed) | `Group::membership()` |
|
||||
| Copy an invite link | `GroupMetadata::naddr(relay, invite)` |
|
||||
| Leave | `GroupsRegistry::leave` |
|
||||
| Create a group | `GroupMetadata { ..Default::default() }` → `GroupsRegistry::create(key, metadata, cx)`; failure surfaces as `GroupsEvent::Error` |
|
||||
| Browse a relay's groups | `GroupsRegistry::browse(relay, cx)` → re-read `GroupsRegistry::browsed(relay)` on `GroupsEvent::Updated`; join a result with `GroupsRegistry::join_key(candidate.key, None, cx)` |
|
||||
| Moderate (put/remove user, edit metadata, delete event, invite) | `Group::put_user/remove_user/edit_metadata/delete_event/create_invite` |
|
||||
| Pins | `Group::pins()` to read, `Group::update_pins(pins)` to replace the list |
|
||||
| Subgroup tree | `Group::metadata()` → `parent`/`children`, assembled per relay |
|
||||
|
||||
Where it lands: the sidebar gains a section reading `GroupsRegistry::groups()`
|
||||
(the same shape as its Communities section) with pending and refused groups
|
||||
listed alongside members; a new panel renders `Vec<Message>` with the existing
|
||||
`ListState` + `ui::message::MessageRow` and `PersonRegistry` for authors. The
|
||||
message *data* is already shared (`chat::Message`), so only `ChatPanel`'s
|
||||
row-rendering methods need lifting into `ui` (or copying) rather than the type. A
|
||||
join dialog accepts a pasted naddr (with or without `?invite=`). System
|
||||
notifications can hang off `GroupEvent::Incoming` when the window is inactive.
|
||||
|
||||
Nothing new is needed from GPUI to support this backend. The local checkouts
|
||||
are `gpui-pre` 0.3.7 (aliased as `gpui`) and `gpui-base` 0.7.0, which `crates/ui`
|
||||
styles; the GPUI APIs the backend uses are the ones `chat` and `community`
|
||||
already use — `Entity`/`WeakEntity`, `Context::spawn`, `background_spawn`,
|
||||
`EventEmitter`, `Subscription` — plus two that `state` already uses:
|
||||
`App::http_client()` for the AV token fetch and
|
||||
`App::background_executor().timer(..)` for the fork tick. The panel's components
|
||||
(`ListState`, `MessageRow`, `Avatar`, `Button`, `Scrollbar`) already exist in the
|
||||
app.
|
||||
|
||||
## Tests
|
||||
|
||||
- `protocol.rs` unit tests: `GroupReference::parse` (naddr, `?invite=`, garbage,
|
||||
missing relay hint, an naddr whose kind is not `39000`); `GroupMetadata::parse`
|
||||
flags and `supports` with `supported_kinds` absent vs empty; admins with roles;
|
||||
members; roles; `GroupList` round trip (`parse` → `upsert` → `to_builder`);
|
||||
`TimelineWindow::previous_tag` (excludes self, caps at 3, 8-char prefixes,
|
||||
any-kind window, `None` when empty); `Membership::from_events` (`9000` vs
|
||||
`9001`, newest wins, tie prefers removal; `Pending`/`Refused` are exercised
|
||||
through the join path); `GroupId::message` carries `h` and the reply `e` tags;
|
||||
`GroupMetadata::naddr` round trip with and without an invite code.
|
||||
- Cache round trip with `nostr-memory` as a dev-dependency (the pattern
|
||||
`crates/community` already uses): save two envelopes (a kind 9 and a kind
|
||||
`9000`), read them back sorted, assert the partition — the kind 9 renders, the
|
||||
`9000` only folds membership — re-save one and confirm no duplicate, `delete`
|
||||
clears the group.
|
||||
- Routing: subscription id ↔ `GroupKey` mapping survives a resubscribe.
|
||||
- One integration test against the SDK's `LocalRelay`
|
||||
(`nostr-sdk` dev-dependency with the `local-relay` feature): seed a `39000`
|
||||
and a kind 9 with an `h` tag, run `Group::connect`, assert metadata and the
|
||||
timeline land. Timeouts in GPUI-driven tests use
|
||||
`cx.background_executor().timer(..)`, never `smol::Timer`, per `AGENTS.md`.
|
||||
|
||||
## Decisions
|
||||
|
||||
1. **Crate.** `crates/nip29` (the body above is written for it).
|
||||
2. **Timeline kinds.** Render `1`, `9`, `11`, `1111` and `7`; every other kind
|
||||
is still cached and feeds the `previous` window, per the NIP's timeline
|
||||
references rule.
|
||||
3. **Default message kind.** Kind 9; the composer is disabled when
|
||||
`supported_kinds` is present and lacks it.
|
||||
4. **Pending joins.** Pending (and refused) groups are listed in the sidebar,
|
||||
with an access state on the row.
|
||||
5. **`10009` merge.** Union-only; removals made by another client are not
|
||||
detected and no diffing is done.
|
||||
6. **Leave and the cache.** Leaving wipes the group's envelopes.
|
||||
7. **Restricted/private groups.** A visible access state is required:
|
||||
`Membership::{Unknown, Pending, Member, Refused, Removed}` with the relay's
|
||||
refusal reason shown and a retry, while the client's NIP-42 authenticator
|
||||
handles the reads.
|
||||
8. **References.** `naddr` only (`?invite=` accepted); `id@relay` is not. A group
|
||||
whose relay publishes no `39000` cannot be referenced or joined in v1.
|
||||
9. **LiveKit response shape.** The NIP says the token endpoint returns "the
|
||||
LiveKit JWT and the URL of the LiveKit server" but names no fields, so
|
||||
`LivekitSession::parse` reads `token` (or `jwt`) and `url` (or `server_url`)
|
||||
and fails loudly on anything else. This is a guess, not a spec, and it is the
|
||||
one part of the AV path that a real relay can contradict; it is worth
|
||||
re-checking against whichever relay is targeted before the UI lands.
|
||||
Likewise `livekit_supported` treats "not `204`" as "no AV" rather than an
|
||||
error, because it only feeds a button.
|
||||
10. **`Monitor`.** Not adopted. It is only reachable through
|
||||
`ClientBuilder::monitor(Monitor::new(n))`, so nothing can read per-relay
|
||||
status until `crates/state` opts in, and its `StatusChanged` stream does not
|
||||
distinguish a permanent outage from a reconnect — so it would need filtering
|
||||
before it could mean anything. The only thing it buys here is running the
|
||||
admins' `10009` check the instant a group's relay drops, rather than at the
|
||||
next 15-minute tick. The NIP frames that as MUST, but the substance is
|
||||
already covered: the admins come from the cached `39001`, so the check runs
|
||||
with the relay gone, just later. If that latency ever matters, the hook is
|
||||
`sync_forks(true, cx)` driven by a `StatusChanged` consumer — one place.
|
||||
Reference in New Issue
Block a user