reya/coop
1
0
Fork 0
Code Issues Pull Requests 1 Releases 3 Activity

update

Browse Source
This commit is contained in:
reya
2026-09-22 16:03:03 +07:00
parent 28f4c1596d
commit 6fac58d494
7 changed files with 774 additions and 117 deletions
Download Patch File Download Diff File Expand all files Collapse all files
docs/community-history-sync-plan.md
+166 -30
View File
@@ -37,7 +37,7 @@ wait, and it is reported rather than rendered as an empty channel.
Read path today (phase 1 landed the walk, phase 2a moved each layer, phase 2b
swapped the transport and put the pump in charge of settling pages, phase 3 added
the rekey watch and held epochs):
the rekey watch and held epochs, phase 4 made an empty or unreadable room say so):
```
CommunityPanel::load community_ui/src/lib.rs:192
@@ -72,6 +72,12 @@ live wire community/src/lib.rs:309 (sync_subscriptions)
back from the database,
adopts one epoch at a
time)
scheduler community/src/lib.rs (one tick per
-> Community::tick community/src/community.rs MIN_ROUND_INTERVAL;
a channel that has
passed STALE_AFTER
re-folds and re-rounds)
```
## What is wrong today
@@ -85,7 +91,7 @@ live wire community/src/lib.rs:309 (sync_subscriptions)
| 5 | **Private channels are never subscribed and never folded**: `planes()` skips `channel.private`, so a private channel gets no live REQ and no `cache_rumor` from the subscription. | fixed in phase 2b (`planes` derives a private channel's plane from the held key; it is subscribed, paged and folded like a public one) | `community/src/sync.rs` |
| 6 | The standing REQ asks for **kind 1059 only**, and the pump drops anything that is not 1059, so 21059 (ephemeral) wraps can never be routed even though the read path asks for both kinds. | fixed in phase 2b (`live_filter`/`plane_filter` ask for both kinds; the pump routes by subscription id and never inspects the kind) | `community/src/sync.rs`, `community/src/lib.rs` |
| 7 | A **rekey strands history**: `ChannelKeyRef` holds one epoch/key, `channel_secret` returns one plane, `sync::refresh` overwrites a held key in place, and the rekey pseudonyms are never watched. | fixed in phase 3 (`priors`/`held_roots` + `retired_at`, the rekey watch, and strict one-epoch-at-a-time adoption) | `community/src/rekey.rs`, `concord/src/state.rs` |
| 8 | **"No messages yet" is three different states**: unreadable wraps are dropped silently, a failed round is logged, and the panel renders all of them as an empty room. | open (phase 4; counts already exist in `Progress`) | `community/src/community.rs:38-44`, `community_ui/src/lib.rs:594-602` |
| 8 | **"No messages yet" is three different states**: unreadable wraps are dropped silently, a failed round is logged, and the panel renders all of them as an empty room. | fixed in phase 4 (`WrapPage`/`Progress`/`Snapshot` carry `unreadable`, the community exposes `progress`/`unreadable`/`missing_key`/`channel_removed_at`/`removed_at`/`stranded`, and the panel renders a reason plus a retry instead of an empty room) | `community/src/community.rs:46-53`, `community_ui/src/lib.rs:235` |
| 9 | A new message **replaced the whole timeline and forced `scroll_to_end()`**. | fixed in phase 1 (`FollowMode::Tail`, in-place merge) | `community_ui/src/lib.rs:121`, `:364-411` |
| 10 | Backfill fetched through `client.fetch_events(..)` with `ReqTarget::auto`, i.e. every relay in the pool. | fixed in phase 1 (relay-scoped, no `fetch_events` anywhere) | `community/src/community.rs:262` |
| 11 | The local cache document is authored by a **process-random key**, so the same rumor cached in two runs is a different event id and the store keeps both copies; `fold` then re-reads all of it on every inbound wrap. | fixed in phase 2a (one fixed cache key) | `community/src/cache.rs` |
@@ -515,19 +521,96 @@ Three deviations from the sketch above, all of them smaller than planned:
adoptable, so a member who missed several rotations catches up in one database
read instead of one pass per poll. The lookahead window is what feeds it.
Still deferred, and named here so it is not mistaken for landed: the panel does
not yet render `Community::removed_at()` / `Community::stranded()` (phase 4), and
a base removal is not enforced at send time — `channel_secret` still hands the
composer the retired root.
Still deferred, and named here so it is not mistaken for landed: a rotation this
client published itself is not adopted locally (no rekey writer exists yet, so the
watch only ever adopts another member's).
### 7. Honest states (phase 4)
### 7. Honest states (phase 4) — **landed**
- `Progress` already carries `fetched`, `opened`, `exhausted`, `failed`, `errors`;
the fold reports what it could not open; `Snapshot` carries those counts.
- `CommunityEvent` gains "history exists that we cannot read" and "the last round
failed", and the panel renders "N messages here can't be read yet — the channel's
key for epoch 3 is missing" or "Couldn't reach the community's relays" with a
retry, instead of "No messages yet" (finding 8).
"No messages yet" is a claim, and finding 8 is that the client made it in three
situations it could not tell apart. The rule is that a room only says it is empty
when it *knows* it is empty; otherwise it says what is actually wrong.
What is carried out of the read path:
- `history::WrapPage.unreadable` counts the wraps a page reached under a held
plane that no held key could open — sealed past the cutoff a rotation set on the
key that reads them, or bound to another channel. `Walk`'s count is the page's;
`Progress.unreadable` sums the pages of a round.
- `sync::Snapshot.unreadable` is the same count over the whole store, per channel,
so a wrap is counted whether it arrived on a round or on the live wire. A wrap
already opened on the way in stays readable through its cached row and is never
counted.
- `Community` keeps the last completed round's `Progress` per channel and the
`unreadable` counts, merged monotonically: an unreadable wrap stays unreadable,
so a later quiet round cannot erase the count.
What the community answers:
- `progress(channel)`, `unreadable(channel)`, `missing_key(channel)` (a private
channel we know and hold no key for, with the epoch), `channel_removed_at(channel)`
(a channel rotation's cut), plus the phase-3 `removed_at()` and `stranded()`.
- `due(channel)`: whether an automatic catch-up is worth asking for yet.
What the panel renders, in precedence order, instead of an empty room:
| State | Rendered |
| ----- | -------- |
| `stranded()` | "This invite is stale — the community has rotated past the epoch it names" |
| `removed_at()` | "You were removed from this community at epoch N. Its history stays readable" |
| `channel_removed_at()` | "A rotation removed you from this channel at epoch N" |
| `missing_key()` | "Messages here can't be read yet — this channel's key for epoch N is missing" |
| `progress.failed && progress.errors > 0` | "Couldn't reach the community's relays" + **Retry** |
| `unreadable(channel) > 0` | "N messages here can't be read yet — no key we hold opens them" |
| otherwise, no rows | "No messages yet" |
The notice replaces the empty state, and sits as a one-line strip above the rows
when there are rows, so a stale room says it may be stale rather than looking
complete. Only *Retry* is actionable, and it runs the round directly — the
`MIN_ROUND_INTERVAL` gate below only paces automatic rounds.
One of those states is about writing, not reading. A room a rotation removed us
from, a channel it cut, a key we never held, or a stale invite all mean the same
thing at the wire: `channel_secret` is `None`, so a wrap sealed now would be
sealed under a root nobody who rotated reads. `Community::send` therefore refuses
in those cases (§6's deferral, closed), and the panel disables the send button
while the notice says why. A channel that is merely unreachable or partly
unreadable still writes: reading and publishing are separate paths.
Two `CommunityEvent`s carry the same news to any other view: `Failed(id)` (the
last round could not reach the relays) and `Unreadable(id)` (history here that no
held key opens). `Failed` replaces the `Error(..)` toast a failed round used to
raise, because the panel now says it in place.
### 7b. The round scheduler (phase 4) — **landed**
Two constants in `community/src/community.rs`:
- `MIN_ROUND_INTERVAL = 30s` — `Community::due(channel)` is false while a round
for that channel ran inside the window. The panel's automatic round on open (and
on a channel switch) is what consults it, so opening a channel twice in a breath
asks the relays once. A round for `Older`, a retry, and the catch-up a rekey
adoption triggers all bypass it — only the automatic open is paced.
- `STALE_AFTER = 300s` — `Community::tick` re-folds, and re-rounds the active
channel, once a channel that has been *synced before* has gone unsynced that
long. A channel with no recorded round is left alone, so a community nobody has
opened costs nothing, and a quiet one asks its relays at most once per five
minutes.
`CommunityRegistry` owns one task for this, armed beside the pump and the signal
consumer in `handle_notifications` and cleared in `reset`, so a signer change
stops it and re-arms it.
### 7c. Not done in phase 4
- **NIP-77 (`client.sync`) catch-up.** Still skipped deliberately. A negentropy
reconciliation is a second way to ask the same question, and getting its
"unsupported"/partial answers right — never reading one as `exhausted` — is its
own piece of work with its own failure modes. The paged walk is the fallback the
plan already specifies, so nothing here blocks on it.
- **The fold's control and guestbook planes** keep logging an unopenable wrap
rather than counting it: `Snapshot.unreadable` is per channel, and there is no
honest place yet to render "the community's own metadata is unreadable".
### 8. `community_ui`: reach the older rows
@@ -542,8 +625,10 @@ Two rules survive, both about GPUI rather than about history:
- **Never `set_follow_mode` or force a scroll position inside a scroll-handler
callback** — the list holds its state borrowed while it invokes the handler, so
touching `ListState` there panics. `load_older` is written to avoid it.
- The panel's `Intent::{CatchUp, Older}` calls do not change: the round they
trigger is now subscription-driven, which is invisible to the list.
- The panel's `Intent::{CatchUp, Older}` calls do not change the list: the round
they trigger is subscription-driven, which is invisible to it. Phase 4 added one
gate in front of the `CatchUp` one — `due()` — and it only paces how often the
relays are asked, not what the list does with an answer.
Still deferred: per-channel timeline state (switching back re-reads), and the
`MAX_TIMELINE_ROWS` trim (trimming the oldest rows fights `load_older`, which
@@ -650,19 +735,54 @@ Also landed as the cheap win §3 promised: the older pass is skipped entirely wh
`cargo +nightly fmt -p concord -p community -p community_ui --check`,
`cargo check -p workspace --all-targets`.
### Phase 4 — honest states and polish
### Phase 4 — honest states and polish — **landed**
§7 (empty/unreadable/failed in the panel, using the counts that already exist),
round progress in the UI, the `MIN_ROUND_INTERVAL = 30s` / `STALE_AFTER = 5min`
scheduler, the `removed`/`stranded` rendering phase 3 persisted but left
unpainted, and optional NIP-77 catch-up (`client.sync(filter)` where a relay
supports negentropy; "negentropy unsupported" means "fall back to the paged
walk", never "exhausted").
§7, §7b: the read path counts what it cannot open (`WrapPage`/`Progress`/
`Snapshot` `unreadable`), the community exposes the honest read surface
(`progress`, `unreadable`, `missing_key`, `channel_removed_at`, `due`, beside the
phase-3 `removed_at`/`stranded`), the panel renders a reason and a retry instead
of "No messages yet", the round scheduler paces automatic rounds
(`MIN_ROUND_INTERVAL`) and repairs a stale one (`STALE_AFTER`), and §6's
send-time gap is closed: a removed, cut, keyless or stranded room holds no write
key.
Gate, all green: `cargo test -p concord -p community` (50 + 32),
`cargo clippy -p concord -p community -p community_ui --all-targets`,
`cargo +nightly fmt -p concord -p community -p community_ui --check`,
`cargo check -p workspace --all-targets`.
Five recorded deviations:
- **A one-line strip above the rows, not only an empty state.** §7's finding is
about the empty room, but the stale case — rows on screen, relays unreachable or
history unreadable — is the same lie in a quieter form, and it is the one the
original report is actually about ("not full of messages and latest data"). A
failed round's strip is transient: the next round that succeeds replaces
`progress` and clears it. An `unreadable` count is monotone by design, because a
wrap no held key opens stays unopened.
- **`Failed` replaces the `Error(..)` toast** on a failed round rather than
accompanying it, so the panel is the single place that reports it.
- **`due()` gates the panel's automatic round, not `sync_channel`.** The plan
named the interval but not the seam; keeping the gate on the caller means every
explicit request (retry, load-older, a rekey's catch-up) stays exact.
- **§6's send-time deferral is closed in the same phase.** The plan framed it as a
wire-level gap to decide later, but it is the same lie in the other direction: a
member a rotation excluded could type, press enter, watch the message disappear
into a plane nobody reads, and be told nothing. `channel_secret` refusing is what
makes the notice actionable.
- **Spawned work is tracked, not detached.** `Community::tasks` and
`CommunityPanel::tasks` hold every fold, round bookkeeping step and publish, so
closing a panel or signing out cancels them instead of letting them finish
against a client nobody holds. Each push drops the tasks that already finished.
Phase 4 leaves the client honest about what it can see and what it can write.
What it does not do is make it see more: the rekey writer, and with it adopting a
rotation this client published itself, remains open (see §6).
Each phase leaves the client consistent on its own. Phase 2a was invisible; phase
2b is what makes "open a community" a subscription and a database read; phase 3
is what keeps a rekey from stranding history. Phase 4 (§7) is next: it is what
makes an empty or unreadable room tell the truth.
is what keeps a rekey from stranding history; phase 4 is what makes an empty or
unreadable room tell the truth.
## Phase 1 status (landed)
@@ -729,6 +849,20 @@ outstanding are the ones that need a GPUI harness or two live accounts.
receives nothing.
- The read path: the side-event budget folds an edit/delete/reaction older than
the row window onto its message.
- What cannot be read — **landed in phase 4**: a wrap sealed after the rotation
that retired the key reading it, and a wrap sealed to this plane but bound to
another channel, both read as unreadable rather than dropped
(`community/src/history.rs`); a fold counts a wrap addressed to a held channel
plane that will not open (`community/src/sync.rs`).
- The honest states — **landed in phase 4** structurally: `Snapshot.unreadable`
and `Progress.unreadable` are what the panel's notice and the `Unreadable`
event read, and the `Failed` event is what a failed round emits. The panel's
precedence (stranded → removed → missing key → unreachable → unreadable →
empty) itself needs a `TestAppContext`, which the repo still does not have.
- The scheduler — **landed in phase 4** structurally: `due()` is false inside
`MIN_ROUND_INTERVAL` after a round, `stale()` requires a recorded round older
than `STALE_AFTER`, and `tick` acts only on the active channel. Driving real
time needs the same harness.
- The fold: a new live wrap costs one decrypt, and a fold over a community with
5,000 cached rows does not re-open them. (The skip is in place and structurally
tested by `wrapper_index`; counting decrypts needs a harness.)
@@ -736,13 +870,15 @@ outstanding are the ones that need a GPUI harness or two live accounts.
regression that duplicates a community's history per app run). **Landed in 2a.**
- GPUI (`TestAppContext`): prepending older rows preserves the scroll anchor; a
live message does not scroll a reader who is scrolled up; `has_more == false`
disables the load-older row. (No GPUI test harness exists in the repo yet.)
disables the load-older row; the panel's notice precedence; `due()`/`stale()`
under a driven clock. (No GPUI test harness exists in the repo yet.)
- Manual runs: two accounts, a channel with more than 200 messages, one account
offline long enough to miss a full page, one private channel, one rekey. The
acceptance bar is the reference behaviour: open a channel cold and see history
arrive in pages without touching the scrollbar, reopen it and see one REQ with a
`since` instead of a replay, and see the other account's message appear without
a reload.
offline long enough to miss a full page, one private channel, one rekey, and one
run with a relay stopped so the notice and its retry are visible. The acceptance
bar is the reference behaviour: open a channel cold and see history arrive in
pages without touching the scrollbar, reopen it and see one REQ with a `since`
instead of a replay, see the other account's message appear without a reload,
and see a room we cannot read say so instead of "No messages yet".
## Out of scope