reya/coop
1
0
Fork 0
Code Issues Pull Requests 1 Releases 3 Activity

update

Browse Source
This commit is contained in:
reya
2026-09-22 15:30:12 +07:00
parent 04fb70e657
commit 28f4c1596d
9 changed files with 1431 additions and 80 deletions
Download Patch File Download Diff File Expand all files Collapse all files
docs/community-history-sync-plan.md
+95 -18
View File
@@ -36,7 +36,8 @@ subscriptions too, which is what a page REQ is (`nostr-sdk/src/relay/inner.rs`
wait, and it is reported rather than rendered as an empty channel.
Read path today (phase 1 landed the walk, phase 2a moved each layer, phase 2b
swapped the transport and put the pump in charge of settling pages):
swapped the transport and put the pump in charge of settling pages, phase 3 added
the rekey watch and held epochs):
```
CommunityPanel::load community_ui/src/lib.rs:192
@@ -60,10 +61,17 @@ live wire community/src/lib.rs:309 (sync_subscriptions)
private channels,
Window::opening,
limit LIVE_REPLAY)
-> rekey::watches community/src/rekey.rs (base next epoch +
a channel window per
held root)
-> pump community/src/lib.rs:433 (routes by subscription
id, batches within
PUMP_WINDOW, settles
pages by EOSE/CLOSED)
-> rekey::adopt community/src/rekey.rs (reads the rekey wraps
back from the database,
adopts one epoch at a
time)
```
## What is wrong today
@@ -76,7 +84,7 @@ live wire community/src/lib.rs:309 (sync_subscriptions)
| 4 | Cursors existed but nothing used them; no "has more" signal. | fixed in phase 1 | `community/src/community.rs:337` |
| 5 | **Private channels are never subscribed and never folded**: `planes()` skips `channel.private`, so a private channel gets no live REQ and no `cache_rumor` from the subscription. | fixed in phase 2b (`planes` derives a private channel's plane from the held key; it is subscribed, paged and folded like a public one) | `community/src/sync.rs` |
| 6 | The standing REQ asks for **kind 1059 only**, and the pump drops anything that is not 1059, so 21059 (ephemeral) wraps can never be routed even though the read path asks for both kinds. | fixed in phase 2b (`live_filter`/`plane_filter` ask for both kinds; the pump routes by subscription id and never inspects the kind) | `community/src/sync.rs`, `community/src/lib.rs` |
| 7 | A **rekey strands history**: `ChannelKeyRef` holds one epoch/key, `channel_secret` returns one plane, `sync::refresh` overwrites a held key in place, and the rekey pseudonyms are never watched. | open (phase 3) | `community/src/sync.rs:331-368`, `community/src/community.rs:199-216` |
| 7 | A **rekey strands history**: `ChannelKeyRef` holds one epoch/key, `channel_secret` returns one plane, `sync::refresh` overwrites a held key in place, and the rekey pseudonyms are never watched. | fixed in phase 3 (`priors`/`held_roots` + `retired_at`, the rekey watch, and strict one-epoch-at-a-time adoption) | `community/src/rekey.rs`, `concord/src/state.rs` |
| 8 | **"No messages yet" is three different states**: unreadable wraps are dropped silently, a failed round is logged, and the panel renders all of them as an empty room. | open (phase 4; counts already exist in `Progress`) | `community/src/community.rs:38-44`, `community_ui/src/lib.rs:594-602` |
| 9 | A new message **replaced the whole timeline and forced `scroll_to_end()`**. | fixed in phase 1 (`FollowMode::Tail`, in-place merge) | `community_ui/src/lib.rs:121`, `:364-411` |
| 10 | Backfill fetched through `client.fetch_events(..)` with `ReqTarget::auto`, i.e. every relay in the pool. | fixed in phase 1 (relay-scoped, no `fetch_events` anywhere) | `community/src/community.rs:262` |
@@ -177,7 +185,7 @@ Notes:
| Community List | `concord/list` (existing) | registry | signer lifetime |
| Live planes | `<community hex>` (existing `sync::subscription_id`) | community | until the signer or the plane set changes; **kept alive** |
| History page | `concord-history-<n>` (opaque, unique) | the round | one page; auto-closes on EOSE |
| Rekey watch (phase 3) | `concord-rekey/<community hex>/<n>` | community | kept alive while the community is tracked |
| Rekey watch | `rekey-<community hex, first 32>` (opaque) | community | kept alive while the community is tracked |
`route_of(&SubscriptionId) -> Option<Route>` parses the two **standing** ids into
`Route::{List, Community(CommunityId)}`, so the pump routes an event by
@@ -192,7 +200,10 @@ instead — `PageRegistry`, a `HashMap<SubscriptionId, flume::Sender<PageReport>
shared between the registry and the rounds. `PageReport { relay, outcome }`
carries facts; the walk decides what they mean. A report for an id nobody is
waiting on is dropped at `debug`, which is the ordinary case when a round is
cancelled. Phase 3 can add a `Rekey` route the same way, by id.
cancelled. Phase 3 added the `Rekey` route the same way, through a parallel
`rekey::WatchRegistry` that maps an opaque `rekey-<32 hex>` id back to its
community: the whole id is sent to the relay, so a page's rule applies — an id
cannot carry a community hex and stay within the cap.
#### The pump
@@ -412,8 +423,9 @@ pub fn timeline(&self, channel: &ChannelId, before_ms: Option<u64>, limit: usize
### 5. Live completeness
- **Private channels get a plane.** `sync::planes` derives it from the held key
(`channel_secret` already knows the rule); a private channel is subscribed and
- **Private channels get a plane.** `sync::planes` derives one per held epoch
(the current key plus every `priors` entry), and a public channel one per held
root, so a rotation never blanks a plane; a private channel is subscribed and
folded exactly like a public one (finding 5).
- **Both wrap kinds.** The filter asks for `KIND_WRAP` and `KIND_WRAP_EPHEMERAL`
in the live REQ and in every page REQ, and the pump routes both (finding 6).
@@ -427,7 +439,7 @@ pub fn timeline(&self, channel: &ChannelId, before_ms: Option<u64>, limit: usize
twice then has one addressable coordinate, so `save_event` replaces its
predecessor instead of adding a copy per app run.
### 6. Held epochs and rekey adoption (phase 3)
### 6. Held epochs and rekey adoption (phase 3) — **landed**
The client has to be able to read the epochs it is supposed to read, and learn
the next ones:
@@ -465,6 +477,49 @@ the next ones:
epoch that predates my join and carries no blob for me, which says the invite
link is out of date).
**What landed.** `HeldKey { epoch, key, retired_at }` and
`HeldRoot { epoch, key, control_pk, retired_at }` live in `concord::state`;
`ChannelKeyRef` gained `priors` and `CommunityState` gained `held_roots`,
`channel_cuts`, `removed_at` and `stranded`. `CommunityState::roots()` and
`held_keys(channel)` are the read surface: a private channel returns its current
key plus every prior, a public channel one entry per held root. `sync::planes`
derives a plane for each, `history::page` takes `&[HeldKey]` and refuses a wrap
sealed under a retired key after its cutoff, and `sync::fold` applies the same
cutoff. `sync::refresh` no longer overwrites a held key in place — it pushes the
superseded one onto `priors` — and a recorded `channel_cuts` entry keeps a stale
grant from merging a removed channel back.
The wire side is `community/src/rekey.rs`: `watches(state)` builds the base
next-epoch address plus a `1..=8` channel window under every held root,
`watch_filter` is the one standing REQ over the community's relays, and `adopt`
reads the delivered wraps back out of the database and walks each scope forward.
`CommunityRegistry` installs the watch beside the live REQ when a community's
plane set changes, routes its events to `Signal::Rekey` through the pump, and
`Community::merge_adoptions` folds the result in, persists it, clears the moved
cursors' `exhausted` verdicts, and runs a `CatchUp` for the active channel.
Three deviations from the sketch above, all of them smaller than planned:
- **The rekey watch is one subscription per community, not per scope.** The base
and every private channel ride the same `authors` filter; the wraps are
addressed by pseudonym, so one REQ covers them all and one `watch_filter`
rebuild covers every adoption. The id is opaque (`rekey-<32 hex>`) and resolves
through `rekey::WatchRegistry` for the same 64-character reason a page id does.
- **A removal is judged without continuity, a strand with it.** Adoption requires
`Extends`; the removal/strand decision considers every complete authorized
rotation at the target epoch **except** one whose `prevcommit` forks (which is
neither, and acting on it is how a member ends up on a fork). A member who
missed a link is still removable — the reference's own channel watcher treats
"past my epoch" as the removal test.
- **Adoption chains within one pass.** `walk` loops `held + 1` while each step is
adoptable, so a member who missed several rotations catches up in one database
read instead of one pass per poll. The lookahead window is what feeds it.
Still deferred, and named here so it is not mistaken for landed: the panel does
not yet render `Community::removed_at()` / `Community::stranded()` (phase 4), and
a base removal is not enforced at send time — `channel_secret` still hands the
composer the retired root.
### 7. Honest states (phase 4)
- `Progress` already carries `fetched`, `opened`, `exhausted`, `failed`, `errors`;
@@ -578,23 +633,36 @@ not account for:
Also landed as the cheap win §3 promised: the older pass is skipped entirely when
`saved.exhausted` is already set.
### Phase 3 — epochs and rekeys
### Phase 3 — epochs and rekeys — **landed**
§6: `HeldKey`/`HeldRoot` + `priors`, `retired_at` as a read cutoff, the rekey
watch over every held root, strict one-epoch-at-a-time adoption, re-subscribe +
`CatchUp` after a delivery, removed versus stranded states.
1. The document (§6): `HeldKey`/`HeldRoot`, `ChannelKeyRef.priors`,
`CommunityState.held_roots`/`channel_cuts`/`removed_at`/`stranded`, and the
`roots()`/`held_keys()` read surface.
2. The read cutoff: `history::page` takes `&[HeldKey]` and refuses a wrap sealed
under a retired key after the rotation's publish time; `sync::fold` applies the
same rule from the channel's `priors`.
3. `sync::planes` derives planes from every held channel epoch and every held
root; `sync::refresh` preserves priors and honors recorded channel cuts.
4. The rekey watch and adoption (§6), `community/src/rekey.rs`, wired through the
pump's `Signal::Rekey` and a `WatchRegistry`.
5. Gate, all green: `cargo test -p concord -p community` (50 + 29),
`cargo clippy -p concord -p community -p community_ui --all-targets`,
`cargo +nightly fmt -p concord -p community -p community_ui --check`,
`cargo check -p workspace --all-targets`.
### Phase 4 — honest states and polish
§7 (empty/unreadable/failed in the panel, using the counts that already exist),
round progress in the UI, the `MIN_ROUND_INTERVAL = 30s` / `STALE_AFTER = 5min`
scheduler, and optional NIP-77 catch-up (`client.sync(filter)` where a relay
scheduler, the `removed`/`stranded` rendering phase 3 persisted but left
unpainted, and optional NIP-77 catch-up (`client.sync(filter)` where a relay
supports negentropy; "negentropy unsupported" means "fall back to the paged
walk", never "exhausted").
Each phase leaves the client consistent on its own. Phase 2a was invisible; phase
2b is what makes "open a community" a subscription and a database read. Phase 3
(§6) is next: it is what keeps a rekey from stranding history.
2b is what makes "open a community" a subscription and a database read; phase 3
is what keeps a rekey from stranding history. Phase 4 (§7) is next: it is what
makes an empty or unreadable room tell the truth.
## Phase 1 status (landed)
@@ -617,10 +685,10 @@ What phase 1 delivered, and what phase 2 replaces:
... the next round retries"). The SDK re-issues the REQ under the same id after
AUTH — for auto-closing subscriptions too — so the honest statement is: the
relay stays in the walk and the page waits for its resubscribed answer.
- Held epoch handling and rekeys are still open in §6, and honest empty states in
§7. Private planes, the second wrap kind, the expired-row sweep, the stable
cache key and the fold cost were open before phase 1 and are now landed (§5,
§9).
- Held epoch handling and rekeys landed in §6 (phase 3); honest empty states
remain open in §7. Private planes, the second wrap kind, the expired-row sweep,
the stable cache key and the fold cost were open before phase 1 and are now
landed (§5, §9).
## Tests
@@ -650,6 +718,15 @@ outstanding are the ones that need a GPUI harness or two live accounts.
carries a `since`, a cold open's does not) still needs a dev relay.
- The page registry — **landed in 2b**: a page that has already unregistered
receives nothing.
- The rekey watch and adoption — **landed in phase 3** (`community/src/rekey.rs`
tests): a complete base rotation whose `prevcommit` extends the held root is
adopted with the prior root retired at the rotation's publish time; a rotation
off a key we do not hold is never adopted; a complete blob-less rotation from a
rotator who outranks us, published after we joined, reads as a removal; a
channel rotation replaces the key and keeps the prior. The pump's half
(`community/src/lib.rs`) is that a rekey watch's event wakes its community.
- The page registry — **landed in 2b**: a page that has already unregistered
receives nothing.
- The read path: the side-event budget folds an edit/delete/reaction older than
the row window onto its message.
- The fold: a new live wrap costs one decrypt, and a fold over a community with
docs/concord-usage.md
+42 -10
View File
@@ -224,6 +224,11 @@ Relay history pages through the local cache:
```rust
use community::history::{self, PageRegistry, Window};
// Every key the client still holds for the channel, newest epoch first: the
// state's current key (`ChannelKeyRef::key` at `epoch`) plus every `priors`
// entry a rotation stepped off. A public channel derives one per held root.
let held: Vec<concord::state::HeldKey> = state.held_keys(&channel);
// The same registry `CommunityRegistry` shares with its notification pump: it is
// what carries each page's EOSE and CLOSED back to the round waiting on it.
let page = history::page(
@@ -240,6 +245,10 @@ let page = history::page(
let cached = cache::query_rumors(&client, &channel, None, 50, Some(&cord03::ROW_KINDS)).await?;
```
A key's `retired_at` (epoch seconds, set when a rotation supersedes it) is a read
cutoff: a wrap at that epoch with a later `created_at` is refused, so a retired
epoch is history and never a live plane an ejected holder can keep writing into.
A relay is only ever *asked*: `history::page` installs **one REQ per page** over
`ReqTarget::manual` for the community's own relays (which is what makes the client
verify a wrap, deduplicate it and persist it), waits for every relay to settle,
@@ -446,6 +455,12 @@ member finds their delivery with `find_my_blobs` / `open_blob`, and adopts the k
only if the plaintext binds to the scope and epoch they expect and its `prevcommit`
matches the key they already hold. Two concurrent rotations settle on `fork_winner`.
`community::rekey::adopt` is that receiver, run against the local database: it
walks a scope forward one epoch at a time off the key actually held (never
waiving a gap, never adopting a fork), keeps each stepped-off key as a prior
with the rotation's publish time as its read cutoff, and reports a removal or a
strand when a complete rotation carries no blob for the member.
The blob plaintext is a fixed-width binary record, but a signer's NIP-44 is
text-only, so `build_blob` carries it base64-encoded inside the envelope.
`open_blob` mirrors that, so the record layout and the `locator` are unchanged.
@@ -635,10 +650,24 @@ client.subscribe(filter).with_id(sub_id).await?;
rekey fold changes it.
- Route inbound events by `subscription_id` from `RelayMessage::Event`, never by
kind.
- Watch one epoch ahead: while holding `root_N`, subscribe to
`base_rekey_group_key(&root_N, &community_id, Epoch(N + 1))` and to
`channel_rekey_group_key(&root_N, &channel, Epoch(N + 1))` for each private
channel. A second epoch ahead is not derivable until the new root arrives.
- Watch one epoch ahead for the base, and a **window** of channel epochs under
every held root for private channels: `base_rekey_group_key(&root_N, &id,
Epoch(N + 1))`, plus `channel_rekey_group_key(&root, &channel, Epoch(channel_epoch
+ ahead))` for `ahead in 1..=8` and each root in `state.roots()`. A Refounding
seals its channel rekeys under the root current when it was minted, so a member
who adopted the base rotation first must still ask under the prior root; the
window is what lets a member who missed a rotation catch up, or learn they were
cut. This is `community::rekey::watches`, installed as a second standing REQ
whose id resolves back to its community through `rekey::WatchRegistry` (the id
cannot carry a 64-hex community id within the NIP-01 length cap).
- `community::rekey::adopt` then reads those wraps **from the database** and walks
each scope forward one epoch at a time: a complete, authorized rotation whose
`prevcommit` extends the key actually held hands over the next key (raced
rotations settle on the lowest); a gap is fetched, never waived; a fork is never
adopted. An addressed-but-unverifiable rotation is neither adopted nor read as a
removal. Only a complete rotation at/after the join, from a rotator who outranks
the member, with no blob for them, is a removal; one that predates the join is a
strand — a stale invite landed the member on a superseded epoch.
### Tests
@@ -683,9 +712,12 @@ client.subscribe(filter).with_id(sub_id).await?;
a NIP-59 gift wrap for the current user.** Concord wraps are kind 1059 too, so
that handler must route by subscription id before any concord subscription goes
live, or every stream wrap lands in the DM trash and raises a toast.
- **Rotation-delivered plane keys cannot be persisted yet.** `CommunityState` has
nowhere to keep a key a rotation delivered, so a client can verify a rotation
and still lose it on restart — history under a prior root or a prior channel
epoch is unreadable until that schema change lands. (A granted private-channel
key does now have a home: `ChannelKeyRef.key`, filled by
`CommunityState::from_join_material`.)
- **Rotation-delivered plane keys are persisted, and history spans them.**
`CommunityState.held_roots` keeps every root a rotation stepped off (with the
retired root's Control signer and the publish time that retires it), and each
`ChannelKeyRef.priors` keeps every channel key a rotation stepped off. The read
side derives planes from all of them, and a retired key's `retired_at` is a hard
read cutoff at both `history::page` and `sync::fold`. The residual gaps: the
panel does not yet render the `removed`/`stranded` state `Community::removed_at`
and `Community::stranded` carry (phase 4), and a base removal is not yet enforced
at send time — the composer still has the old root to write under.