reya/signed
1
0
Fork 0
Code Issues Pull Requests 1 Actions 7 Releases Activity

chore: clean up

Browse Source
This commit is contained in:
reya
2026-09-07 09:24:10 +07:00
parent cf1c9e2162
commit 02a0134164
5 changed files with 117 additions and 589 deletions
Download Patch File Download Diff File Expand all files Collapse all files
docs/.keep
View File
docs/GRASP_PUSH_FIX.md
-298
View File
@@ -1,298 +0,0 @@
# Plan: reliable pushes to multiple grasp servers
Status: implemented (2026-09-07), see the change summary below.
A push to a repo announced with several grasp servers sometimes reaches only
one of them. The grasp server on `relay.ngit.dev` rejected the git push with
`remote: ERR authorisation failed: No state events in purgatory`, while
`gitnostr.com` accepted it. The push still reported success because the app
counts an operation as pushed when at least one grasp server accepted;
individual failures only hit the log as `WARN grasp push failed: ...`. The
failed server silently stays behind until something pushes to it again.
## Background: how a grasp push is authorized
A push to a GRASP server is a two-stage transaction, not a plain git push:
1. The client publishes a NIP-34 kind-30618 *state event* listing the refs it
is about to push (`refs/heads/main -> <sha>` etc.).
2. The client git-pushes the objects to `https://<host>/<owner>/<repo>.git`.
The grasp server does not trust the git push on its own. State events it
accepts are held in an in-memory **purgatory** - accepted but not served,
"until git data arrives" - and the git push is authorized **only against
purgatory state events**, never against the database (the database is the
current state; purgatory holds the intended future state). When a push arrives
and purgatory holds no state event for that repo id, the server rejects with
`No state events in purgatory` (ngit-grasp `git/authorization.rs`), surfaced to
the client through the git protocol as the `remote: ERR authorisation failed:
...` line above. Parked events that are never matched by git data are discarded
after ~30 minutes.
The relay and the git server of a grasp entry share a host: `wss://gitnostr.com`
and `wss://relay.ngit.dev` are relay URLs, and the git URL is derived with
`grasp_base_url` (`https://<host>/<owner>/<repo>.git`). The grasp learns about
state events from its own relay. A relay accept for a state event parks it in
purgatory before the grasp sends its OK, and the nostr-sdk default ack policy
(`AckPolicy::all`) waits for that OK - so a confirmed stage means the state is
in place to authorize the push.
## Root causes in the app
| # | Problem | Consequence |
|---|---|---|
| 1 | State event is published to the whole relay pool, then git is pushed to every grasp server - no per-server staging, no confirmation that the target grasp's own relay accepted it | a push can reach a grasp whose purgatory is still empty |
| 2 | No retry for that denial class | a self-resolving race permanently leaves one server behind |
| 3 | `push_to_grasp_servers` returns `Ok` when >= 1 server accepts; failures only `log::warn!` | UI shows success; the failed grasp silently out of sync |
| 4 | When a grasp's relay is down, the client still fires a doomed git push | confusing git-level `ERR` instead of a clear "state not accepted by relay" |
Relevant code (`crates/signed_state/src/backend.rs`):
- `push_repo_from` (L752-834): broadcast the state event to the whole pool,
then `push_to_grasp_servers`.
- `push_to_grasp_servers` (L1536-1570): sequential git pushes, `Ok` when at
least one server accepted, `warn!` per failure.
- `broadcast_event` (L1336-1350): whole-pool `client.send_event`, errors only
when no relay accepted.
- `send` / `publish_task` (L1239-1292): sign, broadcast, store locally (the
client saves accepted events), emit `BackendEvent::Published`.
- `create_repository` (L554-575) and `publish_local_repo` (L692-708): direct
`push_to_grasp_servers` calls; retract announcement+state when zero servers
accept.
The fix mirrors ngit's own client (its `state_transaction.rs` stages the state
event on the grasp relays, gates which servers are pushed, and fans the state
out to other relays only after a git server accepted) and adds retries, because
the app must also cope with a denial after a relay ack.
## Target flow: staged push with retries
```mermaid
sequenceDiagram
participant App as signed app
participant R1 as gitnostr.com relay+grasp
participant R2 as relay.ngit.dev relay+grasp
participant O as other relays
Note over App: build state event S0 from local refs
App->>R1: stage S0 to grasp relay only (.to(url))
App->>R2: stage S0 to grasp relay only (.to(url))
R1-->>App: OK (parked in purgatory) -> eligible
R2-->>App: OK (parked in purgatory) -> eligible
App->>R1: git push
R1-->>App: authorized by S0 in purgatory
App->>R2: git push
R2-->>App: ERR ... No state events in purgatory (transient)
Note over App: retry: stage a fresh S1 to R2, wait ~1s, git push again
App->>R2: stage fresh S1 (new event id)
App->>R2: git push
R2-->>App: authorized
Note over App: >= 1 server ok -> fan out state to the pool
App->>O: broadcast state (only after a git server holds it)
```
Retries stage a **fresh** state event (new `created_at` -> new id) rather than
resending the same one: a grasp relay treats a same-id event as a duplicate and
will not re-run its policy, so a lost purgatory entry (e.g. a server restart
without its state file) cannot be re-parked by a resend. A fresh id re-runs
validation and re-parks. Superseded parked events expire server-side after 30
minutes, so the residue is bounded.
## Changes
Implemented:
- `crates/signed_state/src/backend.rs`: `push_to_grasp_servers` replaced by the
staged orchestration `push_staged_to_grasps` plus `stage_event_on_relay`,
`sign_state_event`, `is_transient_grasp_denial`, and the `GraspServerResult` /
`PushOutcome` result types. `create_repository`, `publish_local_repo` and
`push_repo_from` (repo push and checkout push) all push through it and fan the
state out to the relay pool only after at least one git server accepted.
- `crates/signed_state/src/repo.rs`: push outcomes surface partial failures as
`RepoStore::last_push_warning`.
- `crates/workspace/src/views/repo_detail/mod.rs`: a warning banner with a
Republish action shows when a push did not reach every grasp server.
- Unit tests for the denial classifier and the outcome reporting.
Decisions taken while implementing:
- State events are no longer broadcast before any git data exists. They are
staged per grasp relay, and only fanned out to the relay pool after a git
server accepted. A total push failure therefore leaves nothing served to
retract except the announcement (create/publish flows retract that, as
before). Staged-but-unmatched state events expire in the grasp's purgatory
after ~30 minutes.
- Servers whose relay rejects the state event are **skipped**, not pushed
(the eligibility gate): a doomed git push would only produce the same
denial.
- **Stale-ref races are retried and verified.** The grasp server runs its own
background sync that aligns repository refs to parked state events as soon
as the git objects are present - including objects an earlier denied attempt
of this same push already uploaded. `git receive-pack` then rejects the ref
update against its stale advertisement with `cannot lock ref ... is at ...
but expected ...` / `incorrect old value provided`. Such rejections are
retried against a fresh advertisement *and* probed for convergence: when
the sync already aligned the refs to this push's target (`git ls-remote`
matches, `signed_git::remote_has_refs`), the server is counted as accepted
even though the push's compare-and-swap never returned `Ok` - because the
data is already there. The sync can land a moment after the retry window, so
the probe is what makes these pushes succeed without a manual republish.
- Create/publish partial failures (some servers accepted, some not) stay
`Ok` and are logged, matching the pre-existing contract; the repo-push
paths additionally set `last_push_warning` so the failure is visible and
one-click republishable.
- The push warning lives in a dedicated `RepoStore::last_push_warning` so it
never collides with the PR-flow `last_warning` on the shared store.
### Staged orchestration (`crates/signed_state/src/backend.rs`)
1. **Per-relay publish helper** (`stage_event_on_relay`). Publishes only to
one relay and verifies its acceptance, using the pinned nostr-sdk 0.45 API
(`send_event_to` is deprecated at 0.45):
```rust
// client.send_event(event).to([relay.clone()]).await
// -> Ok only when the relay is in the output's success map
```
The relay is added and connected first (reusing the `add_relay` pattern);
a failed connect is a staged failure with a clear reason.
2. **Transient-denial classifier** (`is_transient_grasp_denial`, pure,
unit-tested). Two retry families:
- **Purgatory denials** (ngit-grasp `git/authorization.rs`):
- `No state events in purgatory` (the originally reported failure)
- `no matching state event found in purgatory`
- `in purgatory ... doesn't match push`
- `none from authorized publishers`
- `No repository announcement found` (new repos whose announcement is
still propagating)
- **Stale-ref races** (git receive-pack compare-and-swap against the
advertised value, when the grasp's background sync moved the ref):
- `cannot lock ref`
- `incorrect old value provided`
Stale-ref race rejections are additionally probed for convergence with
`git ls-remote`; a server that already advertises the pushed refs counts as
accepted (`remote_has_refs` in `signed_git`).
Everything else - HTTP auth rejection, not a maintainer, a genuine
non-fast-forward divergence, network failure - is permanent for that
attempt. Only the transient classes are retried.
3. **Orchestration** (`push_staged_to_grasps`, replaces `push_to_grasp_servers`):
```rust
pub struct GraspServerResult { relay: RelayUrl, git_url: String, reason: Option<String> }
pub struct PushOutcome { servers: Vec<GraspServerResult>, state_event: Option<Event> }
async fn push_staged_to_grasps(
client: &Client,
signer: &UniversalSigner,
repo_id: &str,
refs: &[(String, String)],
head: Option<&str>,
path: &Path,
owner: &str,
servers: &[RelayUrl],
push: fn(&Path, &str, &str, &str) -> Result<(), Error>,
) -> PushOutcome
```
Per server, in `servers` order:
- **Stage**: publish the state event to that relay (one retry for a connect
blip). Not accepted => record the reason and skip the git push (the
eligibility gate - no doomed push).
- **Push**: git push; on a transient denial, stage a **fresh** state event
(see below), back off ~1s, and retry. Budget: three attempts total.
- Permanent git error => record it and stop for that server.
Retries stage a fresh event (new `created_at` -> new id) rather than
resending the same one: a grasp relay treats a same-id event as a duplicate
and will not re-run its policy, so a lost purgatory entry (e.g. a server
restart without its state file) cannot be re-parked by a resend. When a
retry would fall in the same wall-clock second, `sign_state_event` nudges
`created_at` forward by one second so the event id differs. Superseded
parked events expire server-side after 30 minutes.
4. **Reorder `push_repo_from`**: replace the pre-push whole-pool `send(state)`
with staging + push through `push_staged_to_grasps`, then, after at least
one server accepted, fan the state out to the pool (`broadcast_event`) and
emit `BackendEvent::Published` once. Moving the fan-out after the first git
success also closes today's leak where state is broadcast before any server
holds the objects (ngit made the same change).
5. **Other callers updated** (`create_repository`, `publish_local_repo`): same
orchestration. Their announcement broadcast stays first - grasps park
brand-new announcements in announcement purgatory until the first git data,
so there is no leak - and "zero servers accepted => retract announcement" is
preserved. An empty repository (no refs) is announced without staging or
pushing anything.
### Outcome plumbing (`crates/signed_state/src/repo.rs`)
- `RepoStore::push_repository` / `push_checkout` (L1075-1150) gain
```rust
last_push_warning: Option<String>
```
populated from `PushOutcome::partial_warning()` (a one-line summary naming
the rejected servers and reasons). `last_error` (zero-success hard failure)
and the PR-flow `last_warning` are unchanged.
### UI (`crates/workspace/src/views/repo_detail/mod.rs`)
- When `store.last_push_warning` is set, `render_push_warning_banner` shows a
warning banner above the header:
> Pushed to 1 of 2 grasp servers: wss://relay.ngit.dev: remote: ERR
> authorisation failed: No state events in purgatory fatal: ... Republish to
> sync.
- The banner's Republish action reuses `push_repository` (an idempotent
re-push of every announced server - the simplest catch-up for any drifted
server); a dismiss control clears the warning.
- Zero-success failures continue through the existing error banner (`self.error`
/ `store.last_error`, rendered in `render`).
### Tests
- Unit (backend tests module): the classifier (both retry families - the
reported purgatory stderr line and the reported `cannot lock ref` race are
transient; HTTP 403, not-a-maintainer and a non-fast-forward divergence are
not), the outcome reporting (partial warnings are single-line and name the
failing servers, an all-ok outcome has no warning) and `flatten_whitespace`.
All `cargo test -p signed_state` tests pass.
- The staged orchestration itself is exercised against live grasps by manual
validation; the pure decision helpers are unit-tested.
### Manual validation
- Create and push a repo with both `gitnostr.com` and `relay.ngit.dev`; after
each push, `git ls-remote https://<host>/<npub>/<repo>.git` against **both**
hosts to confirm convergence; repeat over several commits to shake out the
race. Then repeat with `relay.ngit.dev` temporarily unreachable to exercise
the banner and the Retry action.
## Behavior decisions
- At least one server accepted => the operation succeeds (existing contract),
but partial failure is explicit and retryable (warning banner + Republish).
- Announcement publishing stays first; state fan-out moves after the first git
success (fixes the state-before-data leak).
- No wire-protocol changes; error text stays user-readable in the banner.
## Open questions (status)
1. **Auto-retry** after a partial success: not implemented - retries happen
inside the push (up to 3 attempts per server for transient denials), and a
residual failure stays visible with a manual Republish. A delayed
background re-push of failed servers could be added later.
2. **Post-push verification** (`git ls-remote` per grasp after a successful
push to confirm the refs are advertised): not implemented; could catch
ack-but-not-promoted cases at the cost of one extra round-trip per server.
3. **Scope**: the create, publish-local and repo/checkout push paths were all
converted in one change.
docs/PR_FLOW.md
-173
View File
@@ -1,173 +0,0 @@
# Pull request flow
How a pull request moves through Signed from creation to merge. A PR is a
kind-1618 root event whose content is the markdown description; its changes
live in a NIP-10-chained series of kind-1617 patch events (one per commit),
whose root the PR references via an `e` tag. Revisions publish new patch
events plus kind-1619 updates; statuses (kind 1630-1633) resolve the PR's
state.
## Whole lifecycle
```mermaid
graph TD
A["New pull request panel"] --> B{"Compare source"}
B -->|"Local checkout"| C["Pick folder (or auto-prefilled from remembered checkouts)"]
B -->|"Announced fork"| D["Pick fork repo + branch"]
D --> D1["Ensure base mirror (GitCache), fetch origin"]
D1 --> D2["Import fork heads as refs/fork/&lt;owner&gt;/&lt;id&gt;/*"]
C --> E["Defaults: target = announced HEAD, source = current branch / fork main"]
E --> F["merge-base + commits + diff of target..source (Files/Commits tabs)"]
F --> H["Submit: format-patch base..tip at publish time"]
H --> I["split_patch_series: one part per commit"]
I --> J{"Any part over 60 KB?"}
J -->|"Yes"| K["Refuse with message"]
J -->|"No"| L["tip = last part's From commit"]
L --> M["Publish kind-1617 patch series: first has t root, later parts e-reply chained"]
M --> N["Build kind-1618 PR event: c = tip, e = root patch, branch-name, merge-base, clone"]
N --> O["Sign early - learn the event id"]
O --> P["Push tip to refs/nostr/event-id: author /prs/ grasp servers first, then the announced servers"]
P -->|"All rejected"| Q["last_warning banner in PR list"]
P --> R["Publish kind-1618 PR event"]
Q --> R
R --> S{"Draft?"}
S -->|"Yes"| T["Publish kind-1633 draft status"]
S -->|"No"| U["PR open"]
T --> U
U --> V{"Author updates?"}
V -->|"Yes"| W["Publish revision patch series: first has t root-revision and e-replies to the original root"]
W --> X["Publish kind-1619 update: E/P NIP-22 tags, c = new tip"]
X --> U
V -->|"No"| Y{"Repository author merges?"}
Y -->|"Yes"| Z["Apply the series with git am on the mirror clone"]
Z --> AA["applied = rev-list previous-head..HEAD"]
AA --> AB["Publish kind-1631 applied status: applied-as-commits plus r per commit, q plus e-reply per patch event"]
AB --> AC["PR merged"]
Y -->|"Close instead"| AD["Publish kind-1632 closed status"]
AD --> AE["PR closed"]
```
Key points of the write side:
- **Compare sources** (NIP-34 / GRASP-06 native, no fork identity on the
wire):
- *Local checkout*: both branch selectors list a picked folder's
branches; all git ops run in that folder. Checkouts of the target repo
are remembered (folder pick + app clones) and matched implicitly
(origin URL or EUC against the announcement), so the panel prefills the
freshest one - no folder dialog for the common case.
- *Announced fork*: the fork's heads are fetched into the target repo's
GitCache mirror under `refs/fork/<owner-hex>/<id>/*` (private
namespace; the browser never sees them). "Merge Into" lists the
mirror's `refs/remotes/origin/*`, "Pull From" the imported fork
branches, and every git op - merge-base, range diff/commits,
format-patch, tip push - runs in the mirror, which holds both
histories. Fork candidates are announcements related to the target by
`u` tag or shared EUC, own forks first, without `clone` URLs excluded.
- **GRASP-06 hosting**: the tip is pushed under `refs/nostr/<event-id>`
(nak's convention) to the *author's* grasp servers first -
`https://<host>/prs/<author-npub>/<repo-id>.git`, resolved from the
author's kind-10317 grasp list, falling back to the settings defaults -
then to the base repository's announced grasp servers. The `clone` tag
lists those `/prs/` URLs first, then the announced clone URLs (fixed
before signing; dead URLs are inert, the patches stay the source of
truth). Contributing therefore never depends on the other project's
servers accepting a push.
- **Patch series**: each commit becomes its own kind-1617 event so no event
grows past NIP-34's 60 KB guidance; the PR's `c` tag carries the *last*
commit of the series (the tip), and each part carries its own
`commit`/`r` tags.
- **Push before publish**: failure is non-fatal - the patch events remain
the source of truth - and surfaces as a `last_warning` banner.
- **1619 updates are paste-only today** (no repo path holds the new tip's
objects), so updates are not pushed; hosting them is deferred until the
update dialog gains a local-checkout source.
## Creating a pull request - event ordering
```mermaid
sequenceDiagram
participant User
participant P as Base mirror (GitCache)
participant F as Fork grasp server
participant A as Author grasp (GRASP-06 /prs/)
participant B as Base repo grasps
participant R as Nostr relays
User->>P: ensure mirror (fork mode) / pick local checkout
P-->>F: fetch fork heads -> refs/fork/... (fork mode)
User->>P: merge-base, range commits, range diff
User->>P: submit: format-patch base..compare-ref
loop each patch of the series
User->>R: publish kind-1617 (first: t root, later: e reply)
end
User->>User: build and sign kind-1618 (clone = /prs/ URLs + announced)
User->>A: push tip to refs/nostr/event-id (author servers, first)
User->>B: push tip to refs/nostr/event-id (best-effort)
A-->>User: accepted or rejected (all rejected -> warning)
User->>R: publish kind-1618 PR event
opt draft
User->>R: publish kind-1633 draft status
end
```
## Ready to contribute (suggestions)
Local checkouts are matched to announced repositories (remembered records
freshest-first ∪ scanned matches by origin URL or EUC). While a repository's
detail panel is open, each associated checkout is checked off the main
thread: current branch vs its base (announced HEAD, else `main`, else the
first branch), commits ahead, dirty worktrees excluded. A banner in the
repository panel then offers a prefilled New PR panel for the first branch
that is ahead with **no open PR by you** proposing it (`branch-name` tag,
falling back to the `c` tip tag) - NIP-34-native dedupe, refreshed
periodically and whenever the checkouts/announcements change. The panel
never submits anything on its own; suggestions only navigate and prefill.
## Updating and merging
```mermaid
sequenceDiagram
participant Author
participant Relays as Nostr relays
participant Maintainer
participant Clone as Mirror clone
Note over Author,Relays: Update - PR author only (paste flow, no push yet)
Author->>Relays: publish revision patch series (t root-revision, e reply to original root)
Author->>Relays: publish kind-1619 update (E/P tags, c = new tip)
Note over Maintainer,Clone: Merge - repository author only (store-only today)
Maintainer->>Clone: git am the patch series
Clone-->>Maintainer: applied commits (rev-list previous-head..HEAD)
Maintainer->>Relays: publish kind-1631 applied status
Note over Relays: applied-as-commits and r per commit, q and e-reply per applied patch event
```
## Reading side
```mermaid
graph TD
A["PR root kind-1618"] --> B{"Newest status event by author or maintainer?"}
B -->|"1633"| C["Draft"]
B -->|"1631"| D["Applied / merged"]
B -->|"1632"| E["Closed"]
B -->|"1630 or none"| F["Open"]
A --> G{"Newest kind-1619 update by PR author?"}
G -->|"Yes"| H["tip = update's c tag"]
G -->|"No"| I["tip = root's c tag"]
A --> J{"Patch set present?"}
J -->|"Yes"| K["Root patch via e tag, follow reply chain (newest wins per revision)"]
J -->|"No"| L["Diff merge-base..tip from the git clone"]
```
Reader rules that keep the flow consistent:
- **Status**: only status events by the root author or a repository
maintainer count; the newest wins, `Open` is the default.
- **Tip**: only kind-1619 updates by the PR author move the tip - a
stranger's update is ignored.
- **Diff**: the patch set is preferred (NIP-34 `e`-linked chain); PRs from
other clients without patch events fall back to diffing
`merge-base..tip` in the local clone. Fetching tips from `clone` URLs
(ngit `pr checkout` analog) is not implemented yet.