64 KiB
Community history and live sync
A community panel currently shows a short, stale slice of a channel: often only
the newest handful of wraps a relay happened to replay, no older history, no
messages for private channels, and nothing at all after a rekey. The reference
client (soapbox-pub/armada, src/concord/lib/channelSync.ts and friends) pages
history from relays into a local store, keeps a cursor per channel, and keeps the
timeline live. This document is the diagnosis and the plan to get there.
Revision 2. Three corrections to the first revision, all of them structural:
concordstays a thin protocol layer. Relay and database operations do not belong in it. Revision 1 put the whole paged fetch —fetch_page,Window,WrapPage,Walk, the per-page subscriptions — intoconcord/src/store.rs, next to a local cache layer that writes to the client database. That is now framed as the mistake it is:concordkeeps wire formats, crypto, and the plain community document;communitykeeps everyClient.- One notification pump, subscriptions that stay.
communityalready has a global relay notification handler (CommunityRegistry::handle_notifications). The history path must go through it: subscribe to a filter, read the page back from the local database, keep the live subscription alive for new data, and let the pump — not a hand-rolled per-relay wait loop inside the protocol crate — be the only consumer of relay notifications. - Cold asks for everything, warm asks for what is new. Opening a community
with nothing cached subscribes wide (
since = None) and pages down; opening it again asks only for the region since the cursor. Exhaustion still has to be earned (§3).
And one correction to a claim in revision 1: an auth-required CLOSE is not a
failure. The SDK marks that subscription for resubscription and re-issues the REQ
under the same id once the AUTH handshake completes — for auto-closing
subscriptions too, which is what a page REQ is (nostr-sdk/src/relay/inner.rs
handle_relay_message → MarkAsClosed, and the auto-closing handler's
RelayNotification::Authenticated arm). Only AuthenticationFailed ends a relay's
wait, and it is reported rather than rendered as an empty channel.
Revision 3. One more correction, from a review of what revision 2 landed:
- Time is a type, not a
u64with the unit in its name. The read path held three different units in the sameu64:HeldKey/HeldRoot.retired_atheld seconds (but were compared with aTimestampthrough.as_secs()),ChannelCursorheld milliseconds that were only ever seconds multiplied by a thousand, andCommunityState.channel_cutsheld epochs. Every filter boundary divided by a thousand on the way out. §10 types the time fields (Timestamp,Epoch) and leaves a millisecond only where a millisecond is real.
Read path today (phase 1 landed the walk, phase 2a moved each layer, phase 2b swapped the transport and put the pump in charge of settling pages, phase 3 added the rekey watch and held epochs, phase 4 made an empty or unreadable room say so, phase 5 typed the times they all compare):
CommunityPanel::load community_ui/src/lib.rs:192
-> Community::sync_channel community/src/community.rs:220
-> sync_round community/src/community.rs (purge_expired,
Window::opening)
-> history::page community/src/history.rs (one REQ per page
-> ask_page community/src/history.rs over the community's
relays, registered
in the page registry)
-> PageRegistry community/src/history.rs (EOSE / CLOSED arrive
through the pump)
-> client.database().query(filter) (the page)
-> cache::cache_rumor community/src/cache.rs
-> Community::timeline community/src/community.rs:370
-> cache::query_rumors community/src/cache.rs
-> cord03::fold concord/src/cords/cord03.rs
live wire community/src/lib.rs:309 (sync_subscriptions)
-> sync::live_filter community/src/sync.rs (both wrap kinds,
private channels,
Window::opening,
limit LIVE_REPLAY)
-> rekey::watches community/src/rekey.rs (base next epoch +
a channel window per
held root)
-> pump community/src/lib.rs:433 (routes by subscription
id, batches within
PUMP_WINDOW, settles
pages by EOSE/CLOSED)
-> rekey::adopt community/src/rekey.rs (reads the rekey wraps
back from the database,
adopts one epoch at a
time)
scheduler community/src/lib.rs (one tick per
-> Community::tick community/src/community.rs MIN_ROUND_INTERVAL;
a channel that has
passed STALE_AFTER
re-folds and re-rounds)
What is wrong today
| # | Finding | Status | Evidence |
|---|---|---|---|
| 1 | History was fetched at most once per channel, only when the cache was completely empty, so the relay round was almost always skipped. | fixed in phase 1 (sync_channel runs per open) |
community/src/community.rs:218 |
| 2 | That one round was one shallow page with no continuation. | fixed in phase 1 (paged walk + cursors) | community/src/history.rs |
| 3 | The timeline was a fixed 200-row window with no way to ask for older. | fixed in phase 1 (timeline + has_more + load-older) |
community_ui/src/lib.rs:263 |
| 4 | Cursors existed but nothing used them; no "has more" signal. | fixed in phase 1 | community/src/community.rs:337 |
| 5 | Private channels are never subscribed and never folded: planes() skips channel.private, so a private channel gets no live REQ and no cache_rumor from the subscription. |
fixed in phase 2b (planes derives a private channel's plane from the held key; it is subscribed, paged and folded like a public one) |
community/src/sync.rs |
| 6 | The standing REQ asks for kind 1059 only, and the pump drops anything that is not 1059, so 21059 (ephemeral) wraps can never be routed even though the read path asks for both kinds. | fixed in phase 2b (live_filter/plane_filter ask for both kinds; the pump routes by subscription id and never inspects the kind) |
community/src/sync.rs, community/src/lib.rs |
| 7 | A rekey strands history: ChannelKeyRef holds one epoch/key, channel_secret returns one plane, sync::refresh overwrites a held key in place, and the rekey pseudonyms are never watched. |
fixed in phase 3 (priors/held_roots + retired_at, the rekey watch, and strict one-epoch-at-a-time adoption) |
community/src/rekey.rs, concord/src/state.rs |
| 8 | "No messages yet" is three different states: unreadable wraps are dropped silently, a failed round is logged, and the panel renders all of them as an empty room. | fixed in phase 4 (WrapPage/Progress/Snapshot carry unreadable, the community exposes progress/unreadable/missing_key/channel_removed_at/removed_at/stranded, and the panel renders a reason plus a retry instead of an empty room) |
community/src/community.rs:46-53, community_ui/src/lib.rs:235 |
| 9 | A new message replaced the whole timeline and forced scroll_to_end(). |
fixed in phase 1 (FollowMode::Tail, in-place merge) |
community_ui/src/lib.rs:121, :364-411 |
| 10 | Backfill fetched through client.fetch_events(..) with ReqTarget::auto, i.e. every relay in the pool. |
fixed in phase 1 (relay-scoped, no fetch_events anywhere) |
community/src/community.rs:262 |
| 11 | The local cache document is authored by a process-random key, so the same rumor cached in two runs is a different event id and the store keeps both copies; fold then re-reads all of it on every inbound wrap. |
fixed in phase 2a (one fixed cache key) | community/src/cache.rs |
| 12 | cache::purge_expired is never called, so expired rows only drop at fold time, never from disk. |
fixed in phase 2b (sync_round sweeps the channel before it starts) |
community/src/community.rs |
| 13 | Every relay and database operation lived in concord: fetch_page installed its own subscriptions and read the client database, cache_rumor/query_rumors/save_state/load_states wrote and read it. Consequences: the pump could not see a page's REQ (so the panel's live view and the history path were two unrelated worlds), each page hand-rolled a per-relay notification loop, and the protocol crate could not be built or tested without a Client. |
fixed in phase 2a (split into concord/src/state.rs, community/src/cache.rs and community/src/history.rs; the page REQ still waits on its own loop — §2 moves it into the pump) |
concord/src/state.rs, community/src/{cache,history}.rs |
| 14 | The fold is O(history): sync::fold re-reads every wrap in the community's planes and NIP-44-opens each one on every inbound wrap, purely to cache channel rumors and observe their authors for the member list. The live REQ replays the plane on every start, so this happens on every app run and every burst of messages. |
fixed in phase 2b (channel wraps are opened once, on the way in; fold reads already-cached authors and times from cache::wrapper_index instead of opening them again) |
community/src/sync.rs, community/src/cache.rs |
Findings 1-4 and 9-10 are the user-visible symptom and were phase 1; 13 is where that work landed in the wrong crate; 5-7 are why some channels look empty forever; 8 is why a failure looks like an empty room; 11 and 14 are why the store and the fold get slower the fuller a community is.
What the reference client does
Observed in Armada (src/concord/lib/channelSync.ts, src/concord/hooks/useChannel.ts,
src/wire/spec.ts, src/concord/hooks/useRekey.ts):
| Mechanism | Reference detail |
|---|---|
| Local-first read | the query resolves on an IndexedDB read (WINDOW_SIZE = 100 rows) and network catch-up runs behind it, so the panel never waits on relays |
| Three-pass round | newest page → bridge (since: cursor.newest, until: newest.oldest - 1, healing an offline burst larger than one page) → older (until: cursor.oldest, resumed on later rounds) |
| Page size | BACKFILL_PAGE = 50 wraps per relay per page, BACKFILL_MAX_PAGES = 20 per round, LOAD_OLDER_MAX_PAGES = 6 per scroll-up |
| Cursors | { newest, oldest, exhausted } per channel, persisted locally, merged monotonically (newest forward only, oldest back only, exhausted sticky), advanced only when the region is verifiably complete |
| Failure semantics | a failed relay blocks exhaustion and cursor advancement; an all-empty round is a failure, never exhaustion (otherwise a notification-only room is sealed as "exhausted" and never pulls its history) |
| Page-by-page decode | each page is decrypted and committed as it lands; a deep round paints as it goes |
| Scroll-up paging | local store first (pure re-read), relays only when the store is exhausted |
| Side events | messages and their decorations (edits/deletes/reactions) get separate budgets (×4) so reaction floods cannot displace rows and edits outside the row window still fold |
| Live wire | one REQ per relay with the channel's current stream addresses; every held epoch stays in the decode set; retired epochs are asked once more, then frozen out of the REQ |
| Rekeys | a watch on baseRekeyGroupKey(root, id, rootEpoch + 1) plus CHANNEL_REKEY_LOOKAHEAD = 8 epochs of channel rekey pseudonyms; adoption clears exhausted and re-runs the channel's sync round |
| Polling | a scheduler re-runs a channel round when it is older than 5 minutes, at most once per 30 s |
The flow we are converging on: subscribe, then read the database
The Signed app (crates/signed_state/src) is the model, and it maps onto what
community already has:
Signed |
coop |
|---|---|
Backend owns the Client and runs one notification pump that batches relay events into BackendEvent::NostrUpdate(Vec<Update>) within PUMP_DEBOUNCE = 200ms |
state::NostrRegistry owns the Client; CommunityRegistry runs the pump |
Backend::subscribe_bootstrap(filters) / sync_bootstrap(filter) install the REQ; the pump delivers what relays send into the database |
CommunityRegistry::sync_subscriptions installs the live REQ; a round installs a page REQ |
Stores (RepoListStore, ProfileStore) subscribe to backend events and re-query only client.database(), never the network |
Community::refresh → community::cache reads |
RefreshGate (running/dirty, one coalesced follow-up) |
Community::{refresh_task, dirty} and Round{running, queued, waiters} |
| A store never awaits a relay: it reacts to a batched update and reads the database | a round awaits its own page's EOSE (through the pump) and then reads the database |
Two consequences for this codebase, stated as rules:
- A relay is only ever asked by a subscription. No
fetch_events, noReqTarget::auto, no per-relaynotifications()loop outside the pump. - Every read is a database read. Cursors, folds, timelines, member lists and pages are computed from what the database holds after a subscription put it there.
The fix
1. concord keeps the protocol, community keeps the relays and the database
concord gets one rule: no Client, no database, no subscription. Its
remaining job is wire formats, crypto and the community document. Everything
that talks to a relay or the local store moves one crate up:
| Today | Target |
|---|---|
concord/src/store.rs: CommunityState, ChannelKeyRef, ChannelCursor (+ merge), from_genesis, from_join_material, apply_fold, floors, identifier, list_entry |
concord/src/state.rs — the document and its pure folds, no I/O |
cache_rumor, purge_expired, query_rumors, save_state, load_state, load_states, state_identifier, LOCAL_KEYS, the c/t/k/e tags, STATE_PREFIX |
community/src/cache.rs — the local cache and the local documents |
Window, WrapPage, fetch_page (renamed history::page), wrap_filter, ingest_page, auth_required, history_subscription, within, Walk, Walker |
community/src/history.rs — the page REQ and the walk |
planes, subscription_filter, subscription_id, community_of, subscribe_list, load, fold |
stay community/src/sync.rs, extended |
Signal, handle_notifications, sync_subscriptions, track, reset |
stay community/src/lib.rs, extended into the pump (§2) |
The Walk/serve_page tests |
move to community/src/history.rs; the ChannelCursor::merge and from_join_material tests stay with the document; the load_states test moves to community (it needs a database) |
Notes:
- The document module is
concord::state; items are imported by full path (use concord::state::{ChannelCursor, CommunityState};) so the separatestatecrate keepsstate::…for itself. concord's dependency list shrinks:futuresandsmolare used only by the relay code that moves out (smolstays, as a dev-dependency, for the cord tests).ChannelCursorstays a plain data type insideCommunityState(it is persisted, and itsmergeis pure). What moves out is when to advance it — that iscommunity's judgement, not the protocol's.- Two deviations from the table, both to keep the layers honest.
STATE_PREFIX,state_identifierandCommunityState::identifierstay inconcord::state: they name the document's own local key, and the document cannot reach up intocommunityto borrow them. Andcommunity'scacheandhistoryarepub mod, so the surface that used to beconcord::storeis still reachable (a private module turnspurge_expiredandload_stateinto dead code). docs/concord-usage.mddocumentedstore::fetch_page; it was updated in the same change.
2. One notification pump, and subscriptions that stay
Subscription ids and their routes
| Subscription | Id | Owner | Lifetime |
|---|---|---|---|
| Community List | concord/list (existing) |
registry | signer lifetime |
| Live planes | <community hex> (existing sync::subscription_id) |
community | until the signer or the plane set changes; kept alive |
| History page | concord-history-<n> (opaque, unique) |
the round | one page; auto-closes on EOSE |
| Rekey watch | rekey-<community hex, first 32> (opaque) |
community | kept alive while the community is tracked |
route_of(&SubscriptionId) -> Option<Route> parses the two standing ids into
Route::{List, Community(CommunityId)}, so the pump routes an event by
subscription id and never has to guess from the event itself (which is what keeps
another crate's subscription out of a community's fold).
A page id cannot be parsed this way, and that is deliberate: a NIP-01 subscription
id is capped at 64 characters, which a community hex alone fills, so an id cannot
carry both a community and a channel. A page's id is therefore an opaque
concord-history-<n> and the pump resolves it through the page registry
instead — PageRegistry, a HashMap<SubscriptionId, flume::Sender<PageReport>>
shared between the registry and the rounds. PageReport { relay, outcome }
carries facts; the walk decides what they mean. A report for an id nobody is
waiting on is dropped at debug, which is the ordinary case when a round is
cancelled. Phase 3 added the Rekey route the same way, through a parallel
rekey::WatchRegistry that maps an opaque rekey-<32 hex> id back to its
community: the whole id is sent to the relay, so a page's rule applies — an id
cannot carry a community hex and stay within the cap.
The pump
CommunityRegistry::handle_notifications becomes the only consumer of
client.notifications(), and it handles the message variants it drops today
(route and the window loop live in community/src/lib.rs):
fn route(notification, pages, batch) -> Flow {
match notification {
ClientNotification::Event { subscription_id, .. } => match route_of(&subscription_id) {
Some(Route::List) => batch.list = true,
// Only the database is told; the fold reads it when the window closes.
Some(Route::Community(id)) => { batch.communities.insert(id); }
// A page's event needs no routing at all: the subscription already put
// it in the database, and the round reads it back from there.
None => {}
},
ClientNotification::Message { relay_url, message } => match *message {
RelayMessage::EndOfStoredEvents(id) => pages.deliver(&id, relay_url, Settled::Replayed),
RelayMessage::Closed { subscription_id, message }
if !auth_required(&message) =>
{
pages.deliver(&subscription_id, relay_url, Settled::Refused(message.into_owned()));
}
// auth-required: the SDK re-issues this REQ under the same id after
// AUTH, so the page keeps waiting for the resubscribed answer.
_ => {}
},
ClientNotification::Shutdown => return Flow::Stop,
}
Flow::Continue
}
- Kind 1059 and 21059 both route (finding 6); the previous revision's "drops every non-1059" check goes away entirely.
- The window is a fixed
PUMP_WINDOW(200 ms, the reference app's value) from the first notification of a burst, not a debounce that a steady stream can keep extending. What it produces is oneSignal::Event(id)per community that saw an event plus oneSignal::List— so a burst of fifty messages costs one fold, not fifty.Community::refreshkeeps its owndirtyfollow-up. - Page verdicts (
Message) are delivered immediately, inside the window, so an EOSE is never held up by a burst on another subscription.
The live REQ stays
CommunityRegistry::sync_subscriptions keeps installing one REQ per community
over state.relays (ReqTarget::manual, never auto), and it keeps that
subscription alive: a long-lived subscription is re-sent by the SDK after a
reconnect and after a successful AUTH, which is exactly the "resync on socket
reopen" behaviour the reference client implements by hand.
What changes is the filter: it now asks for both wrap kinds, includes private channels (§5), and carries a window and a bound:
/// What a community with no history in the database asks for: everything a relay
/// stores, bounded per relay so a cold start is not a whole plane.
const LIVE_REPLAY: usize = 500;
impl Window {
/// The window a channel is opened with.
///
/// Nothing held: ask wide, the round pages down from there.
/// Something held: ask only for what is new, plus an overlap for the seam.
pub fn opening(cursor: ChannelCursor) -> Self {
match cursor.newest {
Some(newest) => Window {
since: Some(newest - CURSOR_OVERLAP),
until: None,
},
None => Window::default(),
}
}
}
/// The window a community's standing subscription opens with.
///
/// One REQ covers every channel, so the floor is the **oldest** held cursor:
/// starting any newer would skip a channel's new region. A channel with no
/// cursor is left to its own round. `since = None` only when nothing is held.
pub fn live_window(state: &CommunityState) -> Window {
let floor = state
.channels
.iter()
.filter_map(|channel| state.cursors.get(&channel.id)?.newest)
.min();
match floor {
Some(floor) => Window {
since: Some(floor - CURSOR_OVERLAP),
until: None,
},
None => Window::default(),
}
}
The floor is computed when the REQ is installed (on open, on a plane change,
on a signer change) and is not recomputed as cursors advance: a live REQ that is
re-subscribed on every cursor merge would replay the seam on every message, and
the region it did not cover is the round's job, not the live REQ's. Note that a
since bounds only the stored replay; live events arrive whatever it is, so a
channel with no cursor still receives new messages and gets its history from its
own round.
3. A channel round over page subscriptions
Community::sync_channel(&self, channel, intent, cx) keeps its shape and its
coalescing (Round { running, queued, waiters }, one round per channel, a
request that arrives mid-round re-runs it once), and the panel keeps its two
intents:
pub enum Intent {
/// Opening a channel: newest data, the bridge, then a bounded older walk.
CatchUp,
/// Scrolling up: continue older history, at most `pages` pages.
Older { pages: usize },
}
What changes is the transport. A page is now:
- the round registers a
flume::Sender<PageReport>in the page registry under the page's id, before the REQ goes out; - it installs one REQ per page over the community's relays
(
client.subscribe(ReqTarget::manual(..)).with_id(page_id).close_on(ExitOnEOSE + PAGE_TIMEOUT)), after checking each relay is in the pool (the pool refuses a target it does not hold, and would fail the whole REQ); - the pump delivers one report per relay that settles (EOSE, or a non-auth
CLOSED); the round waits until every relay it asked has settled or
PAGE_TIMEOUTpasses, then unregisters; - it reads the page from the database with the same filter the REQ used.
A relay the pool does not hold is settled as refused up front rather than failing the page: its region was never read, so it blocks exhaustion like any other dead relay.
Rules carried over from phase 1, none of which are negotiable:
- A page boundary is exclusive.
until = oldest_seen - 1, so consecutive pages never share an event and the walk terminates. exhaustedis earned. Onlyraw > 0plus a bottomed-out page on every relay that answered may set it. An all-empty page setsfailed, so the next round re-asks the same region instead of sealing the channel at "no more history".- A relay that never answered (deadline) or refused (any CLOSED that is
not
auth-required) is out of the walk and blocksexhausted; it is counted inProgress.errors. auth-requiredis not a failure. The relay stays in the walk and the page waits for the re-issued REQ's EOSE. OnlyAuthenticationFailedsettles a relay as refused, and it is surfaced.- One cursor, one filter per page. The database is not per relay, so per-relay cursors do not exist; a relay is a source that fills the database.
newestadvances only on a complete round (!newest.failed && bridge.exhausted),oldestonly walks down on a complete page, andexhaustedis cleared by a rekey.
The cold/warm distinction is now the window rule rather than a special case:
| State of the channel | Window |
|---|---|
| no cursor, nothing cached (cold open) | Window::opening → since = None: subscribe for everything the relays have, then page down |
cursor with newest (warm open) |
Window::opening → since = newest - CURSOR_OVERLAP: new data only |
cursor with oldest |
Window::older_than(oldest): older data, on demand |
a hole between saved.newest and the newest seen |
Window::between(..): the bridge |
So an open does two things, and they follow the same rule: it makes sure the live REQ is installed — wide when nothing is held, which is where "subscribe to get all data" comes from — and it runs a round whose newest pass asks only for the region the live REQ did not already cover (a warm open therefore replays nothing: the REQ is already streaming).
And the passes fall out of that rule:
- Newest pass (
CatchUp): one page atWindow::opening(saved). Cold, this is "get all data"; warm, this is "get only new data". - Bridge: while the previous page was full and still above
saved.newest, keep walking down withuntil = oldest - 1, bounded byCATCH_UP_PAGES. A short page ends it, which is what letsnewestadvance. - Older pass: resume at
saved.oldest.or(oldest_seen)and walk down, bounded byCATCH_UP_PAGESon a catch-up andLOAD_OLDER_PAGESon a scroll-up.
Sizes stay the reference numbers: PAGE_WRAPS = 50, CATCH_UP_PAGES = 20,
LOAD_OLDER_PAGES = 6, CURSOR_OVERLAP = 60s.
Known limitation, unchanged and deliberately deferred: the bridge is bounded by
CATCH_UP_PAGES, so an offline burst larger than CATCH_UP_PAGES * PAGE_WRAPS
(1,000 wraps) is not repaired by one round and does not heal later (the bridge
restarts from the same point). Fixing it needs a fourth cursor field recording how
far a partial repair walked. One cheap improvement belongs in phase 2b: skip the
older pass entirely when saved.exhausted is already set.
4. The read path
Unchanged from phase 1, moved intact:
pub struct Timeline {
/// Oldest first, ready for a bottom-aligned list.
pub messages: Vec<ChatMessage>,
pub has_more: bool,
}
pub fn timeline(&self, channel: &ChannelId, before_ms: Option<u64>, limit: usize, cx: &App)
-> Task<Result<Timeline>>;
- Reads
limit + 1rows to computehas_more, plus the side-event budget over the same span (SIDE_EVENT_FACTOR = 4), folds withcord03::fold(unchanged — it already resolves edits, deletes and reactions), reverses, returns. TIMELINE_PAGE = 100is the first paint; older history stays on disk and comes back when the user scrolls up.- The panel's only read is
timeline.
5. Live completeness
- Private channels get a plane.
sync::planesderives one per held epoch (the current key plus everypriorsentry), and a public channel one per held root, so a rotation never blanks a plane; a private channel is subscribed and folded exactly like a public one (finding 5). - Both wrap kinds. The filter asks for
KIND_WRAPandKIND_WRAP_EPHEMERALin the live REQ and in every page REQ, and the pump routes both (finding 6). - Community relays only. The live REQ and every page REQ use
ReqTarget::manualoverstate.relays; a relay outside the community is never asked (finding 10). - Expired rows are swept on the open and round cadence with
cache::purge_expired(finding 12). - A stable cache key. One local signing key persisted in the config dir
replaces
LazyLock::new(Keys::generate)(finding 11); the same rumor cached twice then has one addressable coordinate, sosave_eventreplaces its predecessor instead of adding a copy per app run.
6. Held epochs and rekey adoption (phase 3) — landed
The client has to be able to read the epochs it is supposed to read, and learn the next ones:
ChannelKeyRefgains#[serde(default, skip_serializing_if = "Vec::is_empty")] priors: Vec<HeldKey>whereHeldKey { epoch: Epoch, key: [u8; 32], retired_at: Option<Timestamp> }, andCommunityStategains the same shape for roots (HeldRoot { epoch, key,control_pk, retired_at }).channel_secretreturns every held epoch,history::pagealready takes&[(Epoch, [u8; 32])], andsync::refreshstops overwriting a held key in place and pushes the old one ontopriors`.retired_atis a read cutoff. The reference retires the key a rotation steps off at the rotation's own publish time: nothing sealed under that key after it is read again. The read side needs the same floor: a wrap at epoch e whosecreated_atis pastpriors[e].retired_atis refused, at both the page and the fold.- The rekey watch is a subscription too: one long-lived REQ per community over
its relays for
authors(rekey pseudonyms),kinds([KIND_WRAP]), where the authors arecord06::rekey_group(RekeyScope::Base, held_root, id, epoch + 1)and — for every private channel and every held root (a Refounding seals channel rekeys under the prior root) —cord06::rekey_group(RekeyScope::Channel(channel), held_root, channel, epoch)forepoch in held_epoch + 1 ..= held_epoch + REKEY_LOOKAHEAD(8). - Adopt strictly, one epoch at a time, off the key actually held:
cord01::open_wrap→cord06::parse_rekey_chunk→cord06::collect_rotations→is_complete→rotation.continuity(held_epoch, held_key)must beExtends(Gapis fetched, never waived;Forkresolves withcord06::fork_winner's lowest-key rule) →cord06::find_my_blobs+open_blob→KeyDelivery. - Persist per delivery, then re-install the subscription (the plane set
changed), clear that channel's
exhausted, and run aCatchUpround for it. - Two terminal states, both persisted and both shown, never rendered as an empty
timeline: removed (
cord06::am_i_removedon a complete rotation at/after my join whose rotator outranks me —added_at_msandrekey_authorizedalready carry the comparison) and stranded (a complete rotation ahead of my epoch that predates my join and carries no blob for me, which says the invite link is out of date).
What landed. HeldKey { epoch, key, retired_at } and
HeldRoot { epoch, key, control_pk, retired_at } live in concord::state;
ChannelKeyRef gained priors and CommunityState gained held_roots,
channel_cuts, removed_at and stranded. CommunityState::roots() and
held_keys(channel) are the read surface: a private channel returns its current
key plus every prior, a public channel one entry per held root. sync::planes
derives a plane for each, history::page takes &[HeldKey] and refuses a wrap
sealed under a retired key after its cutoff, and sync::fold applies the same
cutoff. sync::refresh no longer overwrites a held key in place — it pushes the
superseded one onto priors — and a recorded channel_cuts entry keeps a stale
grant from merging a removed channel back.
The wire side is community/src/rekey.rs: watches(state) builds the base
next-epoch address plus a 1..=8 channel window under every held root,
watch_filter is the one standing REQ over the community's relays, and adopt
reads the delivered wraps back out of the database and walks each scope forward.
CommunityRegistry installs the watch beside the live REQ when a community's
plane set changes, routes its events to Signal::Rekey through the pump, and
Community::merge_adoptions folds the result in, persists it, clears the moved
cursors' exhausted verdicts, and runs a CatchUp for the active channel.
Three deviations from the sketch above, all of them smaller than planned:
- The rekey watch is one subscription per community, not per scope. The base
and every private channel ride the same
authorsfilter; the wraps are addressed by pseudonym, so one REQ covers them all and onewatch_filterrebuild covers every adoption. The id is opaque (rekey-<32 hex>) and resolves throughrekey::WatchRegistryfor the same 64-character reason a page id does. - A removal is judged without continuity, a strand with it. Adoption requires
Extends; the removal/strand decision considers every complete authorized rotation at the target epoch except one whoseprevcommitforks (which is neither, and acting on it is how a member ends up on a fork). A member who missed a link is still removable — the reference's own channel watcher treats "past my epoch" as the removal test. - Adoption chains within one pass.
walkloopsheld + 1while each step is adoptable, so a member who missed several rotations catches up in one database read instead of one pass per poll. The lookahead window is what feeds it.
The writer (phase 6) — landed
Phase 3 gave the client a receiver; phase 6 gives it a hand. rekey::rotate
builds the rotation from the key the client actually holds and publishes it, and
Community::rotate is the entry a moderation surface will call:
- One plan, drawn in the protocol crate.
cord06::plan_rotation(scope, epoch)mints what the rotation delivers — aRefounding(new root and control pair) for a base scope, a fresh key for a channel — besideplan_refounding, which is now the base arm of it. No caller ever holds a rotation secret before the rotation exists. - Authority first, and the same authority the receiver applies.
Rewrite::authorizedrunscord06::rekey_authorizedunderpermissions(scope)— the same listadoptnow walks with, shared instead of written twice — so the client cannot publish a rotation it would not itself adopt.Community::rotateadds the three states the role fold cannot see: a removal, a strand and a ban. - One blob per recipient, and the rotator is one of them. A rotation that delivered no blob to its own rotator would strand them, so the writer refuses one. Staff get the new Control Plane root beside the key, everyone else gets the key alone.
- A refounding carries its heads.
carry_headsreads the control editions back out of the store, picks the ones the settled floors name, andcompacts them onto the new epoch's groups. Without it the new Control Plane starts empty and the next reader that does not hold the old root folds no roles, metadata or banlist at all. (compactre-signs nothing, so this only works on the plaintext seals the Control Plane already uses.) - Then it adopts itself, through the ordinary receiver. The chunks are saved
into the local database as they are sent, so
Community::rotatecalls the samerekey::adoptthe watch calls instead of adopting by construction: one path for every rotation, whoever wrote it. Publish failures are logged per relay and do not change the held state, so a rotation that only reached some relays still leaves this client consistent — which is the same asymmetry the reference has. - The receiver now keeps the signing root it is handed.
BaseAdoptioncarries the deliveredcontrol_rootthrough tostate.control_root; before this, a refounding's root was dropped at the receiver and a staff member went on signing under the epoch they had left.
What is still not here: the trigger. Kicking a member, choosing a channel's
remaining audience, and rendering any of it is the moderation and
community-management surface §8 keeps out of scope, so Community::rotate is an
API with no caller in the app yet — and Rewrite.recipients is the caller's to
name, because a private channel's audience is in each member's own invite and not
in the local state.
7. Honest states (phase 4) — landed
"No messages yet" is a claim, and finding 8 is that the client made it in three situations it could not tell apart. The rule is that a room only says it is empty when it knows it is empty; otherwise it says what is actually wrong.
What is carried out of the read path:
history::WrapPage.unreadablecounts the wraps a page reached under a held plane that no held key could open — sealed past the cutoff a rotation set on the key that reads them, or bound to another channel.Walk's count is the page's;Progress.unreadablesums the pages of a round.sync::Snapshot.unreadableis the same count over the whole store, per channel, so a wrap is counted whether it arrived on a round or on the live wire. A wrap already opened on the way in stays readable through its cached row and is never counted.Communitykeeps the last completed round'sProgressper channel and theunreadablecounts, merged monotonically: an unreadable wrap stays unreadable, so a later quiet round cannot erase the count.
What the community answers:
progress(channel),unreadable(channel),missing_key(channel)(a private channel we know and hold no key for, with the epoch),channel_removed_at(channel)(a channel rotation's cut), plus the phase-3removed_at()andstranded().due(channel): whether an automatic catch-up is worth asking for yet.
What the panel renders, in precedence order, instead of an empty room:
| State | Rendered |
|---|---|
stranded() |
"This invite is stale — the community has rotated past the epoch it names" |
removed_at() |
"You were removed from this community at epoch N. Its history stays readable" |
channel_removed_at() |
"A rotation removed you from this channel at epoch N" |
missing_key() |
"Messages here can't be read yet — this channel's key for epoch N is missing" |
progress.failed && progress.errors > 0 |
"Couldn't reach the community's relays" + Retry |
unreadable(channel) > 0 |
"N messages here can't be read yet — no key we hold opens them" |
| otherwise, no rows | "No messages yet" |
The notice replaces the empty state, and sits as a one-line strip above the rows
when there are rows, so a stale room says it may be stale rather than looking
complete. Only Retry is actionable, and it runs the round directly — the
MIN_ROUND_INTERVAL gate below only paces automatic rounds.
One of those states is about writing, not reading. A room a rotation removed us
from, a channel it cut, a key we never held, or a stale invite all mean the same
thing at the wire: channel_secret is None, so a wrap sealed now would be
sealed under a root nobody who rotated reads. Community::send therefore refuses
in those cases (§6's deferral, closed), and the panel disables the send button
while the notice says why. A channel that is merely unreachable or partly
unreadable still writes: reading and publishing are separate paths.
Two CommunityEvents carry the same news to any other view: Failed(id) (the
last round could not reach the relays) and Unreadable(id) (history here that no
held key opens). Failed replaces the Error(..) toast a failed round used to
raise, because the panel now says it in place.
7b. The round scheduler (phase 4) — landed
Two constants in community/src/community.rs:
MIN_ROUND_INTERVAL = 30s—Community::due(channel)is false while a round for that channel ran inside the window. The panel's automatic round on open (and on a channel switch) is what consults it, so opening a channel twice in a breath asks the relays once. A round forOlder, a retry, and the catch-up a rekey adoption triggers all bypass it — only the automatic open is paced.STALE_AFTER = 300s—Community::tickre-folds, and re-rounds the active channel, once a channel that has been synced before has gone unsynced that long. A channel with no recorded round is left alone, so a community nobody has opened costs nothing, and a quiet one asks its relays at most once per five minutes.
CommunityRegistry owns one task for this, armed beside the pump and the signal
consumer in handle_notifications and cleared in reset, so a signer change
stops it and re-arms it.
7c. Not done in phase 4
- NIP-77 (
client.sync) catch-up. Still skipped deliberately. A negentropy reconciliation is a second way to ask the same question, and getting its "unsupported"/partial answers right — never reading one asexhausted— is its own piece of work with its own failure modes. The paged walk is the fallback the plan already specifies, so nothing here blocks on it. - The fold's control and guestbook planes keep logging an unopenable wrap
rather than counting it:
Snapshot.unreadableis per channel, and there is no honest place yet to render "the community's own metadata is unreadable".
8. community_ui: reach the older rows
Landed in phase 1, unchanged by the re-layering: per-channel rows, has_more,
loading, FollowMode::Tail instead of scroll_to_end(), load-older from the
scroll handler plus an always-index-0 "Load earlier messages" row, prepend via
splice(1..1, n), append via splice(at..at, n), and in-place merging unless the
newest window no longer reaches the rows on screen.
Two rules survive, both about GPUI rather than about history:
- Never
set_follow_modeor force a scroll position inside a scroll-handler callback — the list holds its state borrowed while it invokes the handler, so touchingListStatethere panics.load_olderis written to avoid it. - The panel's
Intent::{CatchUp, Older}calls do not change the list: the round they trigger is subscription-driven, which is invisible to it. Phase 4 added one gate in front of theCatchUpone —due()— and it only paces how often the relays are asked, not what the list does with an answer.
Still deferred: per-channel timeline state (switching back re-reads), and the
MAX_TIMELINE_ROWS trim (trimming the oldest rows fights load_older, which
prepends at the same end — the reader would oscillate).
9. The fold stops being O(history) (phase 2b) — landed
sync::fold used to open every wrap in the community's planes on every inbound
wrap: channel wraps were re-opened only to cache_rumor them again and to observe
their authors, which the database already holds as decoded rows (the cached row
carries the author in a p tag, the message time in created_at, and the wrap it
came from in an e tag). Finding 14.
What landed is smaller than the first sketch and needs no new queue:
cache::wrapper_index(client, channel)reads the cached rows back asEventId -> Observed { author, at_ms }, keyed by the wrap they were opened from.foldopens a channel wrap only when its id is not in that index; an already-cached wrap is observed from the row instead. A wrap is therefore opened once, on the way in — by a round for a page, by the first fold after it arrives for the live wire — and never again.- The control and guestbook planes are still read from wraps (they are small, and
cord02::fold_controlis a fold over all editions by construction).
The fold still reads every wrap from the database (plane_filter, which is now
its own unbounded filter separate from the live one); what it no longer does is
NIP-44-open them all. The plan's per-channel work queue in the pump turned out to
be unnecessary for that: the first fold after a live wrap already opens exactly
the uncached wraps.
10. Time is a type: Timestamp, not padded milliseconds (phase 5) — landed
The read path compared times through a bare u64 in three units, and each
boundary paid for the confusion:
| Value | Held | What it cost |
|---|---|---|
HeldKey.retired_at, HeldRoot.retired_at, Plane.retired_at |
seconds | compared with wrap.created_at.as_secs(), written as at_ms / 1000 |
ChannelCursor.newest_ms/oldest_ms, WrapPage, Window, the Walk bounds |
milliseconds, always secs * 1000 |
* 1000 on every accepted page, / 1000 on every filter |
CommunityState.channel_cuts |
epochs | Epoch unpacked to u64 and rewrapped at the boundary |
added_at_ms, a rumor's at_ms |
milliseconds, genuinely | — |
What landed:
HeldKey/HeldRoot.retired_at,ChannelCursor.{newest,oldest},WrapPage.{newest,oldest},Plane.retired_at,Window.{until,since}and theWalkbounds areTimestamps. A window bound and a wrap'screated_atare now the same type, soread_underandPlane::acceptscompare them directly,wrap_filter/live_filterhand them tountil/sinceunchanged, and the page's exclusive boundary is writtenoldest - 1instead ofoldest_ms - 1truncated back to seconds.CURSOR_OVERLAPis aDuration(60s), not60_000, sonewest - CURSOR_OVERLAPreads as what it is.channel_cutsisBTreeMap<ChannelId, Epoch>, which is whatrekey::Adoptions::cutsalready carried.- The two millisecond fields stay milliseconds. A rumor's time is genuinely
finer than a wrap's second-granular
created_at: CORD-01 carries the sub-second offset in anmstag andresolve_ms_strictputs it back, so a reader'sat_msandtimeline'sbefore_mskeep it. The cord02 list document'sadded_at/removed_atkeep it too, and not merely for fidelity:added_atis an ordering key judged against a tombstone'sremoved_atunder a strict>, so narrowing both to seconds could tie a rejoin with the leave that preceded it. - A stored cursor is retired, not reinterpreted. A document written when the
boundaries were milliseconds would deserialize into a
Timestampfar in the future, and itsexhaustedwould then wedge the older pass for good. The map's serde key changed (cursors→channel_cursors), so such a record is dropped and the next round rebuilds the cursor from the wraps it reads;a_cursor_stored_in_the_old_unit_is_dropped_rather_than_reinterpretedpins both halves of that.retired_atneeded no such treatment, because it was already written in seconds.
No behaviour changes with it: the walk paged the same regions before and after, because every millisecond it held was a second multiplied by a thousand.
Order of work
Phase 2a — move the code (no behaviour change) — landed
concord/src/store.rssplit intoconcord/src/state.rs,community/src/cache.rsandcommunity/src/history.rsper the table in §1; call sites updated (community/src/sync.rs,community/src/community.rs,community/src/lib.rs, and thecord02test'scrate::store::CommunityState).fetch_pageis nowhistory::page.- The local cache key is stable (
cache::LOCAL_KEYSis one fixed secret, not a per-processKeys::generate), so caching a rumor twice leaves one row.cache.rscarries the newcaching_the_same_rumor_twice_leaves_one_rowtest. docs/concord-usage.mdupdated;concorddroppedfuturesandnostr-memoryand demotedsmolto a dev-dependency;communitygainedfuturesandsmol.- Gate, all green with the phase-1 transport still in place:
cargo test -p concord -p community(50 + 14),cargo clippy -p concord -p community -p community_ui --all-targets,cargo +nightly fmt --all --check(no diff in the touched files),cargo check -p workspace --all-targets. No behaviour changed beyond item 2.
Phase 2b — subscribe, then read the database — landed
- The pump (§2): route by subscription id for events, EOSE and CLOSED; batch
within
PUMP_WINDOW; both wrap kinds;auth-requirednever a failure. - Page REQs (§3): the round registers a
PageReportchannel, installs one REQ per page over the community's relays, awaits the pump's reports withPAGE_TIMEOUT, reads the page from the database, and keeps the walk's rules (exhaustedearned, all-empty isfailed,newest_msonly on a complete round, skip the older pass whensaved.exhausted). - The live REQ (§2, §5): both kinds, private channels, the window rule with
LIVE_REPLAY = 500, kept alive across reconnects. cache::purge_expiredon the round cadence (every round is an open).- The fold issue (§9).
- Gate:
cargo test -p concord -p community(50 + 23),cargo clippy -p concord -p community -p community_ui --all-targets,cargo +nightly fmt --all --check,cargo check -p workspace --all-targets; plus the phase-1 manual bar and a dev-relay check that a warm open sends one REQ per relay carrying asince, and a cold open sends one with nosinceand pages down.
Three deviations from the revision-2 sketch, each forced by a constraint it did not account for:
- A page id is opaque and resolved through a registry, not parsed. A NIP-01 id
is capped at 64 characters and a community hex fills it, so
concord-history/<community>/<channel>/<n>could never be sent.route_oftherefore only parses the two standing ids, andPageRegistrymaps a page id back to the round waiting on it. - The live window is per community and aggregates cursors. There is one REQ
per community, so
Window::opening(cursor)applies to a channel's newest pass whilesync::live_window(state)derives the community floor as the oldest held cursor (a channel with no cursor forcessince = None). - The fold needs no pump work queue. Skipping already-cached wraps through
cache::wrapper_indexalready makes a wrap open once, on the way in, because the first fold after a live wrap is the one that opens it. See §9.
Also landed as the cheap win §3 promised: the older pass is skipped entirely when
saved.exhausted is already set.
Phase 3 — epochs and rekeys — landed
- The document (§6):
HeldKey/HeldRoot,ChannelKeyRef.priors,CommunityState.held_roots/channel_cuts/removed_at/stranded, and theroots()/held_keys()read surface. - The read cutoff:
history::pagetakes&[HeldKey]and refuses a wrap sealed under a retired key after the rotation's publish time;sync::foldapplies the same rule from the channel'spriors. sync::planesderives planes from every held channel epoch and every held root;sync::refreshpreserves priors and honors recorded channel cuts.- The rekey watch and adoption (§6),
community/src/rekey.rs, wired through the pump'sSignal::Rekeyand aWatchRegistry. - Gate, all green:
cargo test -p concord -p community(50 + 29),cargo clippy -p concord -p community -p community_ui --all-targets,cargo +nightly fmt -p concord -p community -p community_ui --check,cargo check -p workspace --all-targets.
Phase 4 — honest states and polish — landed
§7, §7b: the read path counts what it cannot open (WrapPage/Progress/
Snapshot unreadable), the community exposes the honest read surface
(progress, unreadable, missing_key, channel_removed_at, due, beside the
phase-3 removed_at/stranded), the panel renders a reason and a retry instead
of "No messages yet", the round scheduler paces automatic rounds
(MIN_ROUND_INTERVAL) and repairs a stale one (STALE_AFTER), and §6's
send-time gap is closed: a removed, cut, keyless or stranded room holds no write
key.
Gate, all green: cargo test -p concord -p community (50 + 32),
cargo clippy -p concord -p community -p community_ui --all-targets,
cargo +nightly fmt -p concord -p community -p community_ui --check,
cargo check -p workspace --all-targets.
Five recorded deviations:
- A one-line strip above the rows, not only an empty state. §7's finding is
about the empty room, but the stale case — rows on screen, relays unreachable or
history unreadable — is the same lie in a quieter form, and it is the one the
original report is actually about ("not full of messages and latest data"). A
failed round's strip is transient: the next round that succeeds replaces
progressand clears it. Anunreadablecount is monotone by design, because a wrap no held key opens stays unopened. Failedreplaces theError(..)toast on a failed round rather than accompanying it, so the panel is the single place that reports it.due()gates the panel's automatic round, notsync_channel. The plan named the interval but not the seam; keeping the gate on the caller means every explicit request (retry, load-older, a rekey's catch-up) stays exact.- §6's send-time deferral is closed in the same phase. The plan framed it as a
wire-level gap to decide later, but it is the same lie in the other direction: a
member a rotation excluded could type, press enter, watch the message disappear
into a plane nobody reads, and be told nothing.
channel_secretrefusing is what makes the notice actionable. - Spawned work is tracked, not detached.
Community::tasksandCommunityPanel::taskshold every fold, round bookkeeping step and publish, so closing a panel or signing out cancels them instead of letting them finish against a client nobody holds. Each push drops the tasks that already finished.
Phase 4 leaves the client honest about what it can see and what it can write. What it does not do is make it see more: the rekey writer, and with it adopting a rotation this client published itself, was still open there (§6, closed in phase 6).
Each phase leaves the client consistent on its own. Phase 2a was invisible; phase 2b is what makes "open a community" a subscription and a database read; phase 3 is what keeps a rekey from stranding history; phase 4 is what makes an empty or unreadable room tell the truth.
Phase 5 — typed time — landed
§10. HeldKey/HeldRoot.retired_at, ChannelCursor, WrapPage, Plane,
Window and the Walk bounds are Timestamps; channel_cuts is Epoch;
CURSOR_OVERLAP is a Duration; and the cursor map's serde key was retired so no
stored millisecond cursor is read as seconds. No behaviour change, no relay
contact, no new dependency — the units the read path already compared by hand are
now the types it compares.
Gate, all green: cargo test -p concord -p community (51 + 32),
cargo clippy -p concord -p community -p community_ui --all-targets,
cargo +nightly fmt -p concord -p community -p community_ui --check,
cargo check -p workspace --all-targets.
Phase 6 — the rekey writer — landed
§6's writer: cord06::plan_rotation, rekey::{Rewrite, rotate}, carry_heads,
and Community::rotate, which publishes a rotation and then adopts it through the
receiver phase 3 built. The receiver also keeps the signing root a refounding
hands it now, instead of dropping it.
Gate, all green: cargo test -p concord -p community (51 + 32),
cargo clippy -p concord -p community -p community_ui --all-targets,
cargo +nightly fmt -p concord -p community -p community_ui --check,
cargo check -p workspace --all-targets. No test was added for the writer itself:
the publish half needs a relay, and the same reason the reference's own rekey
paths are exercised by hand applies here — see Tests below.
Phase 1 status (landed)
What phase 1 delivered, and what phase 2 replaces:
Window,WrapPage,history::page,Walk/Walker, the paged walk with an exclusive page boundary,ChannelCursor+ monotonicmergepersisted inCommunityState.cursors,Community::sync_channel/timeline,cache_rumorcounting what opened, and the panel's load-older/follow-the-tail work. All of that survives; phase 2a moved it and phase 2b swaps its transport.fetch_eventsis gone. Phase 1 already replaced it with subscribe-then-read-the-database: each page subscribed every answering relay to one filter (relay.subscribe(..).close_on(ExitOnEOSE + PAGE_TIMEOUT)), waited for that relay'sEndOfStoredEvents(id), then read the page withclient.database().query(filter). Revision 2's correction is where that lives (community, notconcord) and who watches the notifications (the pump, not a loop inside a page).auth-requiredwas already excluded from the failure set in code, but revision 1's prose described it wrongly ("ends that relay's wait as a failure ... the next round retries"). The SDK re-issues the REQ under the same id after AUTH — for auto-closing subscriptions too — so the honest statement is: the relay stays in the walk and the page waits for its resubscribed answer.- Held epoch handling and rekeys landed in §6 (phase 3); honest empty states remain open in §7. Private planes, the second wrap kind, the expired-row sweep, the stable cache key and the fold cost were open before phase 1 and are now landed (§5, §9).
Tests
Following the existing MemoryDatabase and Walk/serve_page test style. The
2b pieces are covered by unit tests that need no relay; the rows still marked
outstanding are the ones that need a GPUI harness or two live accounts.
- The walk:
history::pagecontinues past a page whose wraps none of them open;exhaustedrequires a short page after history; an all-empty round isfailedand the next round re-asks the same region;untilis exclusive and the walk terminates; a channel that already holds its newest page still pages older history; the bridge heals a hole; history pages across a rekey using retained prior keys. (Phase 1 landed theWalkhalf; the page-REQ half needs a relay.) - The pump — landed in 2b (
community/src/lib.rstests): anEndOfStoredEvents(id)settles exactly the page that ownsid; a non-auth CLOSED settles its relay as refused; anauth-requiredCLOSED settles nothing; a page's event routes to nothing; a burst collapses to one signal per community; a shutdown stops the pump. - The windows — landed in 2b:
Window::opening(default)has nosince;Window::opening(saved)starts atnewest - CURSOR_OVERLAP;Window::older_thannever includes the boundary event;sync::live_windowis wide cold and resumes at the oldest held cursor warm. - The cursor's unit — landed in phase 5: a merge only moves forward and never
earns
exhausted; a document whose cursors were stored in the old millisecond encoding reads as no cursors at all, while a typed one round-trips (concord/src/state.rs). - The subscription plan — landed in 2b: a private channel's plane appears when
the key is held and is absent when it is not;
plane_filterasks for both wrap kinds and addresses every readable plane. The page-REQ half (a warm open's REQ carries asince, a cold open's does not) still needs a dev relay. - The page registry — landed in 2b: a page that has already unregistered receives nothing.
- The rekey watch and adoption — landed in phase 3 (
community/src/rekey.rstests): a complete base rotation whoseprevcommitextends the held root is adopted with the prior root retired at the rotation's publish time; a rotation off a key we do not hold is never adopted; a complete blob-less rotation from a rotator who outranks us, published after we joined, reads as a removal; a channel rotation replaces the key and keeps the prior. The pump's half (community/src/lib.rs) is that a rekey watch's event wakes its community. - The rekey writer — landed in phase 6, structurally only. No test covers it:
the half that could be tested without a relay (that
plan_rotationmints two unrelated keys, thatRewrite::authorizedagrees with the receiver) would not catch the failures that matter (a blob addressed to the wrong epoch, a chunk set the receiver cannot collect, a refounding that carries no head), and the half that would — publish, re-read, adopt — needs a real relay and a second account. What stands in for it is that the writer cannot take a path the receiver does not: samepermissions, samebuild_rekey_chunks, and adoption goes throughrekey::adoptrather than beside it. - The read path: the side-event budget folds an edit/delete/reaction older than the row window onto its message.
- What cannot be read — landed in phase 4: a wrap sealed after the rotation
that retired the key reading it, and a wrap sealed to this plane but bound to
another channel, both read as unreadable rather than dropped
(
community/src/history.rs); a fold counts a wrap addressed to a held channel plane that will not open (community/src/sync.rs). - The honest states — landed in phase 4 structurally:
Snapshot.unreadableandProgress.unreadableare what the panel's notice and theUnreadableevent read, and theFailedevent is what a failed round emits. The panel's precedence (stranded → removed → missing key → unreachable → unreadable → empty) itself needs aTestAppContext, which the repo still does not have. - The scheduler — landed in phase 4 structurally:
due()is false insideMIN_ROUND_INTERVALafter a round,stale()requires a recorded round older thanSTALE_AFTER, andtickacts only on the active channel. Driving real time needs the same harness. - The fold: a new live wrap costs one decrypt, and a fold over a community with
5,000 cached rows does not re-open them. (The skip is in place and structurally
tested by
wrapper_index; counting decrypts needs a harness.) - A stable cache key: caching the same rumor twice leaves one row (the phase-1 regression that duplicates a community's history per app run). Landed in 2a.
- GPUI (
TestAppContext): prepending older rows preserves the scroll anchor; a live message does not scroll a reader who is scrolled up;has_more == falsedisables the load-older row; the panel's notice precedence;due()/stale()under a driven clock. (No GPUI test harness exists in the repo yet.) - Manual runs: two accounts, a channel with more than 200 messages, one account
offline long enough to miss a full page, one private channel, one rekey, and one
run with a relay stopped so the notice and its retry are visible. The acceptance
bar is the reference behaviour: open a channel cold and see history arrive in
pages without touching the scrollbar, reopen it and see one REQ with a
sinceinstead of a replay, see the other account's message appear without a reload, and see a room we cannot read say so instead of "No messages yet".
Out of scope
Unread badges, notifications, message threads, pins, typing indicators and
presence (21059 wraps are wired for routing here, not for those features), file
and media rendering, moderation actions, and the community-management surfaces.
The rekey writer is the one exception, and only its mechanism: Community::rotate
exists to be called, but nothing in the app calls it. Kicking a member, choosing
the audience a rotation keeps, and showing any of it stay out.
crates/chat's DM path shares none of this code and is not touched; a later
change can lift the page walk/cursor into a shared module if DMs grow the same
paging.