reya/coop
1
0
Fork 0
Code Issues Pull Requests 1 Releases 3 Activity

.

Browse Source
This commit is contained in:
reya
2026-09-22 20:15:15 +07:00
parent 1c5ef58049
commit 706cc72c1f
14 changed files with 1021 additions and 104 deletions
Download Patch File Download Diff File Expand all files Collapse all files
docs/community-history-sync-plan.md
+308 -7
View File
@@ -45,10 +45,36 @@ wait, and it is reported rather than rendered as an empty channel.
divided by a thousand on the way out. §10 types the time fields (`Timestamp`,
`Epoch`) and leaves a millisecond only where a millisecond is real.
**Revision 4.** The six phases landed and the symptom was still there: a
community showed its full history, its full roster, and never a new message. The
audit that followed found the read path asking for addresses nobody writes to,
plus two smaller reasons the same shape comes back (§11):
5. **A public channel's plane follows the ROOT epoch, not the epoch its material
names.** `held_keys` derived every held root's plane at the *channel's*
recorded epoch, and `channel_secret` sealed at it, so after a Refounding the
current root's plane was asked for and written to at the retired epoch — an
address no other client reads. The channel kept everything written before the
rotation and received nothing after it, while the Control and Guestbook planes
(which derive from the roots directly) stayed current: the roster looked
complete while the room looked dead.
6. **The walk never reported its floor.** `Walk::accept` computed the page's
oldest wrap into a local and never into `Walk::oldest`, so every page reported
`oldest: None`. The older pass had no point to resume below (scroll-up could
only re-read the local cache), the bridge never ran (a burst larger than one
page left a hole the cursor then sealed as covered), and `exhausted` could
never be earned.
7. **Two ways for the live path to go quiet are closed.** A cursor is clamped to
the local clock, so a peer's future stamp cannot bound every later filter
below a region that has not happened; and a community whose relays go silent
re-issues its standing REQ, because a subscription that died without saying so
is indistinguishable from a quiet one.
Read path today (phase 1 landed the walk, phase 2a moved each layer, phase 2b
swapped the transport and put the pump in charge of settling pages, phase 3 added
the rekey watch and held epochs, phase 4 made an empty or unreadable room say so,
phase 5 typed the times they all compare):
phase 5 typed the times they all compare, revision 4 fixed the plane a public
channel is read at once its root has rotated):
```
CommunityPanel::load community_ui/src/lib.rs:192
@@ -109,11 +135,14 @@ scheduler community/src/lib.rs (one tick per
| 12 | `cache::purge_expired` is never called, so expired rows only drop at fold time, never from disk. | fixed in phase 2b (`sync_round` sweeps the channel before it starts) | `community/src/community.rs` |
| 13 | **Every relay and database operation lived in `concord`**: `fetch_page` installed its own subscriptions and read the client database, `cache_rumor`/`query_rumors`/`save_state`/`load_states` wrote and read it. Consequences: the pump could not see a page's REQ (so the panel's live view and the history path were two unrelated worlds), each page hand-rolled a per-relay notification loop, and the protocol crate could not be built or tested without a `Client`. | fixed in phase 2a (split into `concord/src/state.rs`, `community/src/cache.rs` and `community/src/history.rs`; the page REQ still waits on its own loop — §2 moves it into the pump) | `concord/src/state.rs`, `community/src/{cache,history}.rs` |
| 14 | **The fold is O(history)**: `sync::fold` re-reads every wrap in the community's planes and NIP-44-opens each one on every inbound wrap, purely to cache channel rumors and observe their authors for the member list. The live REQ replays the plane on every start, so this happens on every app run and every burst of messages. | fixed in phase 2b (channel wraps are opened once, on the way in; `fold` reads already-cached authors and times from `cache::wrapper_index` instead of opening them again) | `community/src/sync.rs`, `community/src/cache.rs` |
| 15 | **A public channel's plane was derived at the channel's recorded epoch for every held root**, so after a Refounding the current root's plane was subscribed, paged and sealed at the retired epoch: no live events, no new pages, and no cursor ever written for the channel — while the history of the prior epoch and the wholeControl and Guestbook planes still read. | fixed in revision 4 (§11) | `concord/src/state.rs` (`held_keys`), `community/src/community.rs` (`channel_secret`) |
| 16 | **The walk never reported its floor** (`Walk::oldest` was computed into a local), so the older pass never ran, the bridge never healed a gap, and `exhausted` was never earned. | fixed in revision 4 (§11) | `community/src/history.rs` (`Walk::accept`) |
Findings 1-4 and 9-10 are the user-visible symptom and were phase 1; 13 is where
that work landed in the wrong crate; 5-7 are why some channels look empty forever;
8 is why a failure looks like an empty room; 11 and 14 are why the store and the
fold get slower the fuller a community is.
fold get slower the fuller a community is; 15-16 are why a rotated community reads
as a complete archive that stopped receiving.
## What the reference client does
@@ -574,7 +603,12 @@ remaining audience, and rendering any of it is the moderation and
community-management surface §8 keeps out of scope, so `Community::rotate` is an
API with no caller in the app yet — and `Rewrite.recipients` is the caller's to
name, because a private channel's audience is in each member's own invite and not
in the local state.
in the local state. **And the new epoch's roster is not seeded:** CORD-02 §5 has
the refounder publish a snapshot of the present members into the Guestbook it
just minted, which is the only way a client that joins *after* the rotation ever
learns who was already there. `cord02::guestbook::build_snapshot_chunks` is
implemented and unobliged; a refounding published from here would leave every
later joiner with an inferred roster (§12c).
### 7. Honest states (phase 4) — **landed**
@@ -754,6 +788,231 @@ What landed:
No behaviour changes with it: the walk paged the same regions before and after,
because every millisecond it held was a second multiplied by a thousand.
### 11. A Refounding moves a public channel's plane (revision 4) — **landed**
The symptom that survived every phase: a channel showed its history and its room
members, and never a new message — not while the panel was open, and not after
reopening it, while another client was writing to the same channel. Three
findings, in order of how much they explain.
#### 11a. The plane a public channel is read and written at
A public channel holds no key of its own: CORD-03 §1 derives its Chat Plane from
the **community root**, at the **root epoch** — `channelGroupKey(root, channel,
rootEpoch)`. A Refounding moves the root to a new epoch, so every public
channel's plane moves with it. The reference therefore keeps one stream per held
**root** epoch and writes to the newest (`channelsView`'s `rootStreams` with
`current: rootStreams[0]`), which is also what makes history survive the
rotation.
This code paired every held root with the channel's *recorded* epoch — the root
epoch the channel's material was published at — and `channel_secret` sealed at
the same value. At genesis the two agree, so nothing looked wrong; after a
rotation they diverge, and every plane the client asked for was addressed at the
retired epoch:
| Plane | Derived from | After a Refounding |
| ----- | ------------ | ------------------ |
| Channel (`channel_group_key`) | `held_keys` | current root **at the old epoch** — an address nobody writes |
| Control (`control_group_key`) | `state.control_pks` epochs | current |
| Guestbook (`guestbook_group_key`) | `roots()` epochs | current |
That shape is the whole report: the roster, the channel list and the old history
were all right, the room received nothing, and because the newest pass came back
empty the round read as failed, so no cursor was ever written for that channel.
The community's own messages went to the same unreachable plane — readable by us,
invisible to everyone else.
**Fixed:** `held_keys` pairs each root with **its own** epoch
(`HeldKey { epoch: root.epoch, key: root.key, retired_at: root.retired_at }`), and
`channel_secret` seals a public channel at
`(state.root_epoch, state.community_root)` — the newest held root, the one the
reference calls `current`. The two halves move together on purpose: reading where
we write, and writing where the rest of the protocol reads, is the same statement.
#### 11b. The page's floor
`sync_round` uses `history::page`'s report three ways: the newest pass's `oldest`
is where the older pass resumes (`resume = saved.oldest.or(newest.oldest)`), the newest
pass's `oldest` is the top of the bridge (`Window::between(saved.newest, oldest)`),
and a page that reached the bottom earns `exhausted` so later rounds can stop
paging. `Walk::accept` computed that value into a local and never assigned
`Walk::oldest`, so every page reported `oldest: None` and all three consequences
followed at once: the older pass never ran (scrolling up could only ever re-read
the local cache), the bridge never ran (a burst larger than one page left a hole
that the next cursor advance then sealed as covered), and the bottom was never
reached. The persisted state said so plainly — no cursor carried an `oldest`, and
every `exhausted` was false.
**Fixed:** `accept` folds the page's floor into `Walk::oldest`.
#### 11c. Two ways for the live path to go quiet, closed
- **A cursor is clamped to the local clock.** A wrap stamped in the future — a
peer's skewed clock, or a hostile stamp — used to become
`ChannelCursor.newest`, and the cursor is persisted, so every later REQ for that
channel would open with `since` ahead of the present. Relays apply `since` to
live events as well as to the stored replay, so the channel goes deaf for as
long as the stamp leads and no restart heals it. The reference clamps at exactly
this point (`WireSync`'s `writeCursor`: an event stamped in the future "must not
drag the cursor past `now`"). `clamped` bounds `newest` both on the way into the
document and on the way into a filter, so a cursor already stored past `now`
heals on the next round instead of staying deaf.
- **The standing REQ is re-issued when a community goes quiet.** A relay can end
a subscription without a reason the SDK acts on, and nothing here re-installed
one whose plane set had not moved, so a community whose subscription died looked
exactly like a quiet community until the next launch. The reference rotates
every relay's REQ after 90s of silence for this reason ("never trust one
subscription for long"); `CommunityRegistry::tick` now does the same, because an
accepted REQ that yields nothing and a REQ that was never accepted look
identical from here. The re-issue resumes from the current window, so the seam
is replayed rather than lost.
What is deliberately still not here: a future-dated message is folded and shown
rather than held out of the timeline until its time comes (the reference's
`FUTURE_HOLD_MS`). The clamp already keeps such a stamp from bounding a filter;
hiding it is a display decision this client has not made.
And one gap this revision leaves open, because it is a different symptom (depth,
not freshness) and it cannot be closed honestly from what the code has at hand:
`sync::refresh` overwrites `community_root`/`root_epoch` from the community
list's material without retiring the root it replaces, and `held_roots` is
populated only by `rekey::adopt`. A client that learns of a Refounding from a
re-materialized list entry rather than from a rekey blob the watch delivered
therefore loses the prior root, which is exactly the history the reference keeps
in `heldRoots` ("every held epoch stays in the decode set"). The messages are
still on screen — they are in the local cache — but paging *below* them against
a relay needs that plane. Retiring a root correctly needs the rotation's publish
time; a list entry carries only its own `added_at`, so the honest fix is for the
re-materialization to carry the prior root the way an adoption does, not to guess
a cutoff here. **Closed in revision 5 (§12c)**, on the same terms: a root the
List moved past is retained as a key we were given, and the cutoff stays absent
rather than guessed.
### 12. Two reports from one client (revision 5) — **landed**
The plane fix worked — history, the roster and new messages all arrived — and the
same client then reported two things it could see: *"when the message list
updates, the order isn't by timestamp anymore"*, and *"the member list still
isn't showing full members like the other client"*. Two independent bugs, both
confirmed against the client's own database before anything was changed.
#### 12a. Rows are placed where the fold puts them, not appended
The panel treated a read as newer than the rows on screen: new ids were appended
to the end (`apply`) and the older page was spliced onto the front (`prepend`).
Both hold only while every read is *newer than everything shown* or *older than
everything shown* — and reads are neither. A catch-up round heals a gap by
caching history sealed long ago, the bridge pass fills the region *above* the
cursor's newest, and a message delivered late lands with its own (older) time. The
client's own database shows the ground for it: the room held 1,176 cached rows, so
every window read is a *selection* of its history, and a round finishing while the
panel is open hands `apply` rows older than the newest row already on screen. The
same read also reaches `apply` twice — once through `Updated` and once through
`load_older`'s own re-read — so whether a page of history landed at the top or the
bottom depended on which task ran first.
**Fixed:** one `merge` for both paths. Rows already shown keep their position (an
edit replaces its row in place); a row that is new is inserted where the fold
would have put it — `partition_point` over `(at_ms, id)` ascending, the exact
order `Community::timeline` returns — and the list state is spliced at that index
instead of at the end. The order the panel shows is now a function of the data,
not of the order reads happened to land in.
One other lie about time went with it: `send` stamped `at_ms` as
`Timestamp::now().as_secs() * 1000`, and a rumor's `ms` tag is the *remainder*
within the second (`split_ms`), so every message this client sent carried `0` —
placed at the start of its second, up to 999 ms before it was written, and before
any message from another client in the same second. The reference stamps
`Date.now()`; `now_ms()` does the same, with the clock error propagated rather
than flattened.
#### 12b. A Refounding snapshot could never be honored
`sync::fold` called `guestbook::coalesce` with `None` as the snapshot authority,
and `coalesce` drops every snapshot chunk whose refounder is not that authority —
so **every** Refounding snapshot this client ever received was ignored, and the
roster fell back to the members it could infer: authors seen publishing, granted
npubs, and joins it had itself received.
The client's database shows exactly what that costs. Vector Community is at root
epoch 9 with no retained roots; its epoch-9 Guestbook holds 13 wraps — 11 joins,
1 leave, and **one snapshot naming 240 members**, authored by `d133ecb0…`. Folded
with no authority the roster comes out at **41 members**; folded with that one
snapshot honored it comes out at **247**. Nothing about the wraps is missing or
unreadable — only the authority to believe them was.
**Fixed:** the refounder is recorded where the protocol states it and honored
where CORD-02 §5 asks for it.
- `CommunityState.refounders` — the npubs whose rotation minted an epoch this
client verified. `guestbook::coalesce` now takes that set (`&BTreeSet`) instead
of an `Option`, which is also what the reference passes: the wire never says
which epoch's stream carried a chunk, so the authorities are unioned and the
residual (one held epoch's refounder accepted on another's snapshot) is
documented there and here. An empty set honors no snapshot, and genesis — where
no rotation minted anything — has no authority at all.
- `rekey::walk` records the rotator of every base step it adopts
(`Delivery.rotator` → `Adopted.refounders` → `BaseAdoption.refounders`), and
`merge_adoptions` folds them into the state. A rotation is the only place a
refounder is ever named, and it is verified the way an adoption already was:
continuity from the key held, a blob for us, and the rotator's rank.
Which raised the question the client's own state answered: it holds epoch 9 but
never adopted the 8 → 9 rotation (its `held_roots` is empty), because it learned
of the refounding from a re-materialized List entry. A rotation can only be read
from the root it stepped off, so the refounder of an epoch the client was away
for was unreachable. **Fixed in §12c**, and the two fixes are one story: the
refounder of the current epoch is recoverable precisely because the root before
it is retained.
#### 12c. A superseded root is history, not a secret
Three points, one rule — a root the community has rotated past stays held:
- `sync::refresh` no longer overwrites `community_root`/`root_epoch` from the
List's material without retiring the root it replaces, which is what lost the
prior root in the first place (the gap §11 closed on paper).
- `sync::load` likewise keeps the root the List's **seed** material names: the
List carries the community as it was when *we* joined, so that root is a key
this client was given. `merge_entry` already preserves the lowest-epoch
material as the seed, so a client that joined three rotations ago still has
three rotations' worth of read keys recoverable.
- `rekey::watches` now watches the epoch after **every** held root, not only the
current one, and `rekey::adopt` walks from every held root before it, keeping
the furthest adoption (the epoch the community is actually on). That is what
turns a retained root into a refounder: the 7 → 8 and 8 → 9 rotations are
still verifiable from the roots they stepped off.
The retained roots pay for themselves a second time: `held_keys` derives channel
planes from them, so public-channel history *below* the local cache becomes
pageable again for a rotated community — the depth gap §11 named, not just the
refounder.
What is honestly still missing, in the order it matters:
- **Recovery needs the old rotation chunks to still be on a relay.** A refounder
is only learnable from the rotation that named it; if the relays have dropped
the chunk for the epoch a client is on, the seeded members stay unknown until
the next Refounding — the client cannot verify a snapshot it has no rotation
for, and accepting one from an unverified npub would let any member inject
arbitrary npubs into everyone's roster.
- **A Refounding this client publishes seeds nobody.** `build_snapshot_chunks`
exists and the reference publishes one at every refounding ("present members
only, chunked at 400"); `Community::rotate` does not yet. It has no caller, so
nothing regresses today, but the first client that refounds without it leaves
every later joiner with an inferred roster.
- **`complete_memberlist` is still given no ban times.** The reference passes the
Control Plane's authorized ban history so that activity *before* a ban does not
resurface as membership after an unban; `fold_control` exposes the Banlist as a
set and no times, so an unbanned member's older activity counts as present here.
Nobody is lost by it — the failure mode is a member shown who left — and it is
the fold in `concord` that would have to grow the times.
- **The panel's order is untested.** The merge is a few lines and the crate has
no harness for a panel; the invariant it restores is the one `Community::timeline`
already promises, and it is asserted here by reading it against the fold, not
by a test.
## Order of work
### Phase 2a — move the code (no behaviour change) — **landed**
@@ -945,8 +1204,45 @@ outstanding are the ones that need a GPUI harness or two live accounts.
`failed` and the next round re-asks the same region; `until` is exclusive and
the walk terminates; a channel that already holds its newest page still pages
older history; the bridge heals a hole; history pages across a rekey using
retained prior keys. (Phase 1 landed the `Walk` half; the page-REQ half needs a
relay.)
retained prior keys; and the walk reports the floor the older pass resumes from
— **landed in revision 4** (`a_walk_pages_back_across_a_rekey` asserts
`page.oldest`, the half `sync_round` resumes from). (The page-REQ half still
needs a relay.)
- A Refounding's plane — **landed in revision 4**
(`sync::tests::a_refounding_moves_a_public_channels_plane_to_the_new_root_epoch`):
with the root at epoch 1, the prior root held, and the channel's material still
naming epoch 0, the current root's channel plane is the one subscribed and read,
and a message sealed at it folds into the channel. Before the fix the plane was
derived at epoch 0 and the test failed on the missing plane.
- A delivered wrap — **landed in revision 4**
(`sync::tests::a_fold_caches_a_channel_wrap_a_relay_delivered`): a wrap saved
into the database (what the SDK does with what a subscription delivered) and
nothing else is opened and cached by the next fold, which is the step a live
message depends on. Its `unreadable` counterpart was already covered.
- The cursor's clock — **landed in revision 4**
(`a_future_stamp_cannot_push_a_cursor_past_now`): a round's future stamp is
clamped to `now` while its other findings land, and a cursor already stored past
`now` heals; `sync::live_window` starts at `now - CURSOR_OVERLAP` for such a
cursor instead of opening the REQ ahead of the present.
- The roster's authority — **landed in revision 5**
(`sync::tests::a_refounders_snapshot_seeds_the_members_it_names`): a snapshot
sealed to the current epoch's Guestbook seeds a member who has never been seen
publishing, and the same wraps seed nobody once the refounder is not in
`refounders` — the authority is the rotation, never the author alone.
- A rotation off a retained root — **landed in revision 5**
(`rekey::tests::a_rotation_past_a_retained_root_is_adopted_and_names_its_refounder`):
with the root before it retained, a rotation into epoch 2 is adopted from epoch
1 and its rotator comes back as the epoch's refounder. The unit-half of §12c's
recovery: what still needs a relay is whether the old chunk is *still there* to
be read at all.
- Retaining what was rotated past — **landed in revision 5**
(`sync::tests::a_list_material_at_a_newer_epoch_keeps_the_root_it_superseded`
and `a_list_that_moved_the_root_on_retains_the_root_of_our_join`): material at a
newer epoch replaces the current root and retires the one it moved past, and a
List whose seed names our join keeps that root held.
- The page a rotated community's history sits on — the same retention is what
`a_walk_pages_back_across_a_rekey` pages across, and revision 5 is what makes a
*List*-learned refounding leave those keys held, not only a blob-learned one.
- The pump — **landed in 2b** (`community/src/lib.rs` tests): an
`EndOfStoredEvents(id)` settles exactly the page that owns `id`; a non-auth
CLOSED settles its relay as refused; an `auth-required` CLOSED settles nothing;
@@ -997,7 +1293,10 @@ outstanding are the ones that need a GPUI harness or two live accounts.
- The scheduler — **landed in phase 4** structurally: `due()` is false inside
`MIN_ROUND_INTERVAL` after a round, `stale()` requires a recorded round older
than `STALE_AFTER`, and `tick` acts only on the active channel. Driving real
time needs the same harness.
time needs the same harness. Revision 4 added the rotation of a quiet
community's standing REQ to the same pass, which is structural in the same way:
what it needs to be observed is a relay that drops a subscription without
saying so.
- The fold: a new live wrap costs one decrypt, and a fold over a community with
5,000 cached rows does not re-open them. (The skip is in place and structurally
tested by `wrapper_index`; counting decrypts needs a harness.)
@@ -1006,7 +1305,9 @@ outstanding are the ones that need a GPUI harness or two live accounts.
- GPUI (`TestAppContext`): prepending older rows preserves the scroll anchor; a
live message does not scroll a reader who is scrolled up; `has_more == false`
disables the load-older row; the panel's notice precedence; `due()`/`stale()`
under a driven clock. (No GPUI test harness exists in the repo yet.)
under a driven clock; and — added by revision 5 — that a read carrying history
older than the last row lands *above* it instead of at the end. (No GPUI test
harness exists in the repo yet, so the merge is asserted by reading it.)
- Manual runs: two accounts, a channel with more than 200 messages, one account
offline long enough to miss a full page, one private channel, one rekey, and one
run with a relay stopped so the notice and its retry are visible. The acceptance