This commit is contained in:
2026-09-15 19:03:20 +07:00
parent 584ab34df6
commit 68d3d77ae3
4 changed files with 510 additions and 0 deletions
Generated
+49
View File
@@ -139,6 +139,20 @@ dependencies = [
"zeroize",
]
[[package]]
name = "aes-gcm"
version = "0.10.3"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "831010a0f742e1209b3bcea8fab6a8e149051ba6099432c8cb2cc117dec3ead1"
dependencies = [
"aead",
"aes",
"cipher 0.4.4",
"ctr",
"ghash",
"subtle",
]
[[package]]
name = "ahash"
version = "0.8.12"
@@ -1686,6 +1700,7 @@ source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "78c8292055d1c1df0cce5d180393dc8cce0abec0a7102adb6c7b1eef6016d60a"
dependencies = [
"generic-array",
"rand_core 0.6.4",
"typenum",
]
@@ -1708,6 +1723,15 @@ dependencies = [
"linktime-proc-macro",
]
[[package]]
name = "ctr"
version = "0.9.2"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "0369ee1ad671834580515889b80f2ea915f23b8be8d0daa4bbaf2ac5c7590835"
dependencies = [
"cipher 0.4.4",
]
[[package]]
name = "ctutils"
version = "0.4.2"
@@ -2542,6 +2566,16 @@ dependencies = [
"wasm-bindgen",
]
[[package]]
name = "ghash"
version = "0.5.1"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "f0d8a4362ccb29cb0b265253fb0a2728f592895ee6854fd9bc13f2ffda266ff1"
dependencies = [
"opaque-debug",
"polyval",
]
[[package]]
name = "gif"
version = "0.14.2"
@@ -5191,6 +5225,18 @@ dependencies = [
"arrayvec",
]
[[package]]
name = "polyval"
version = "0.6.2"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "9d1fe60d06143b2430aa532c94cfe9e29783047f06c0d7fd359a9a51b729fa25"
dependencies = [
"cfg-if",
"cpufeatures 0.2.17",
"opaque-debug",
"universal-hash",
]
[[package]]
name = "portable-atomic"
version = "1.15.0"
@@ -6631,7 +6677,9 @@ checksum = "6ce2be8dc25455e1f91df71bfa12ad37d7af1092ae736f3a6cd0e37bc7810596"
name = "state"
version = "1.0.1"
dependencies = [
"aes-gcm",
"anyhow",
"base64 0.22.1",
"browser-signer-proxy",
"common",
"flume 0.11.1",
@@ -6650,6 +6698,7 @@ dependencies = [
"nostr-sdk",
"rustls",
"serde_json",
"sha2 0.10.9",
"smol",
"webbrowser",
]
+5
View File
@@ -27,6 +27,11 @@ nostr-connect = { git = "https://github.com/rust-nostr/nostr" }
nostr-sdk = { git = "https://github.com/rust-nostr/nostr" }
nostr = { git = "https://github.com/rust-nostr/nostr", features = [ "nip59", "nip49", "nip44" ] }
# Crypto (NIP-17 encrypted file messages)
aes-gcm = "0.10"
sha2 = "0.10"
base64 = "0.22"
# Others
anyhow = "1.0.44"
chrono = { version = "0.4.38", features = ["wasmbind"] }
+452
View File
@@ -0,0 +1,452 @@
# NIP-17 Encrypted File Messages (kind 15) — Backend Plan
Implementation plan for sending and receiving NIP-17 **file messages** in coop.
**Scope: backend only.** This document covers crypto, blob upload/download, rumor
construction, rumor parsing and local caching. No UI work (composer button, file
rendering, image cache, decryption-on-render state) — that is a separate follow-up
once these APIs exist.
---
## 1. What the protocol requires
NIP-17 file messages are **not** a new transport. They reuse everything coop already
has (NIP-44, NIP-59 seal + gift wrap, kind 10050 inbox relays) and only add:
1. A new inner **rumor kind: `15`** (`Kind::Custom(15)`), whose `.content` is the URL
of an **encrypted** blob and whose tags carry the MIME type and the decryption
material.
2. **AES-256-GCM** encryption of the file bytes before upload.
Kind 15 tags (per NIP-17):
| Tag | Required | Meaning |
|---|---|---|
| `p` | yes | receivers (as for kind 14) |
| `e` | if reply | parent message id |
| `subject` | optional | conversation title |
| `file-type` | yes | MIME type of the **plaintext** file |
| `encryption-algorithm` | yes | `aes-gcm` (only supported value) |
| `decryption-key` | yes | key for the recipient |
| `decryption-nonce` | yes | nonce for the recipient |
| `x` | yes | SHA-256 hex of the **encrypted** file |
| `ox` | expected | SHA-256 hex of the file **before** encryption |
| `size` | optional | size of the **encrypted** file in bytes |
| `dim` | optional | `<width>x<height>` in pixels |
| `thumbhash` / `blurhash` | optional | placeholder previews |
| `thumb` | optional | thumbnail URL (same key/nonce) |
| `fallback` | optional | extra file sources (same key/nonce) |
Key material travels **inside** the gift wrap, so the public blob URL is useless
without it. That property falls out of the existing seal/gift-wrap code for free.
`thumbhash`, `blurhash`, `thumb`, `fallback` are out of scope for v1 (all optional).
---
## 2. What already exists in coop (verified against the pinned deps)
| Requirement | Location | Status |
|---|---|---|
| NIP-59 seal + gift wrap | `crates/chat/src/room.rs::send_gift_wrap` (`nip59::GiftWrapBuilder`) | exists |
| Per-recipient publish + self backup | `crates/chat/src/room.rs::send` | exists |
| Inbox relays (kind 10050) | `crates/chat/src/lib.rs::handle_notifications` | exists |
| Rumor unwrap + local cache | `crates/chat/src/lib.rs::{extract_rumor,try_unwrap_with,set_rumor,get_rumor}` | exists |
| Kind 14 rumor construction | `crates/chat/src/room.rs::rumor` | exists |
| Blossom upload | `crates/state/src/blossom.rs::upload` (plaintext, random signing key) | partial |
| **Kind 15 rumor + parse** | — | **to add** |
| **AES-256-GCM encrypt/decrypt** | — | **to add** |
| **Encrypted blob upload/download** | — | **to add** |
Verified facts about the pinned SDK (`rust-nostr@b230cec`, `nostr 0.45.4`) and
`gpui@69af529`:
- rust-nostr has **no** kind-15 helper and **no** AES-GCM/AES-GCM-tag support anywhere
(`FileMetadata` in NIP-94 is kind 1063, unrelated). `nip17.rs` only covers kind 14
and 10050.
- `nip59::GiftWrapBuilder::new(receiver, rumor: UnsignedEvent)` accepts **any**
`UnsignedEvent`, so kind 15 flows through the existing wrap/send path unchanged.
- `Kind` has no named variant for 15; `Kind::Custom(15)` is required, and
`Kind: Display` prints `as_u16()`, so `rumor.kind.to_string()` yields `"15"`.
- `nostr::nips::nip94::Sha256Hash` is public (`from_byte_array`, `to_hex`, `Display`,
`from_hex`) even though it lives in the nip94 module — usable for `x`/`ox` hex
formatting without a new hashing crate.
- `gpui::App::http_client()` returns `Arc<dyn HttpClient>` and
`gpui_web/src/http_client.rs` implements it, so HTTP download is cross-platform.
`AsyncApp` exposes `update(|app| ...)`, which is how backend async code reaches it.
- `nostr-blossom` exposes `upload_blob`, `get_blob`, `has_blob`, `list_blobs`,
`delete_blob`.
- The `k` tag is currently hardcoded to `"14"` in two places
(`room.rs::send_gift_wrap`, `lib.rs::set_rumor`) and used as a room-list filter
(`lib.rs::get_rooms_task`, `custom_tag(LOWERCASE_K, "14")`).
---
## 3. Dependencies
Add to `[workspace.dependencies]` in `Cargo.toml`, then reference from the crates below.
```toml
aes-gcm = "0.10" # NEW - RustCrypto: Aes256Gcm, aead::{Aead, KeyInit, OsRng}
sha2 = "0.10" # NEW (already in Cargo.lock, cached) - SHA-256 for x / ox
base64 = "0.22" # NEW as a direct dep (already in the tree transitively)
```
- No new RNG dependency: `aes_gcm::aead::OsRng` (the wasm getrandom backends are
already configured in `web/Cargo.toml`).
- No new hashing/hex dependency: `sha2` output → `nostr::nips::nip94::Sha256Hash::from_byte_array(...).to_hex()`.
- `aes-gcm` is the only crate that needs a crates.io fetch (`aes 0.8` / `aead 0.5` are
already in the lock file), so it is a small addition to the build graph.
- Hand-rolling AES-GCM is explicitly **not** an option.
---
## 4. Layering and type ownership
`chat` depends on `state` (see `crates/chat/Cargo.toml`), never the reverse, so the
shared types and tag names must live in `state`.
- `crates/state/src/file.rs` (new) owns:
- `EncryptedFile`, `FileAttachment`, the tag-name constants, `ALGORITHM = "aes-gcm"`
- `encrypt` / `decrypt` / `sha256_hex`
- `FileAttachment::from_tags` / `FileAttachment::tags` (single source of truth for
tag names, so build and parse can't drift)
- `upload_encrypted`, `download_and_decrypt`
- re-exported from `crates/state/src/lib.rs`: `mod file; pub use file::*;`
- `crates/chat` consumes it: `message.rs` (parse into `Message`), `room.rs`
(build kind-15 rumor), `lib.rs` (cache tag + room list query).
- `crates/chat` should re-export the type for the future UI layer:
`pub use state::FileAttachment;` in `crates/chat/src/lib.rs`.
---
## 5. Data flow (backend)
```mermaid
flowchart TD
A[caller: path + blossom server] --> B[read bytes + guess mime]
B --> C[encrypt: random 32B key, 12B nonce, AES-256-GCM]
C --> D[sha256 ciphertext = x, sha256 plaintext = ox]
D --> E[Blossom upload ciphertext]
E --> F[FileAttachment]
F --> G[Room::file_rumor -> kind 15 rumor]
G --> H[Room::send -> existing NIP-59 wrap per member + self backup]
H --> I[existing extract_rumor on receiver]
I --> J[Message.file]
J --> K[download_and_decrypt: GET url, verify x, decrypt]
```
---
## 6. Implementation steps
### Step 1 — Dependencies
Add the three lines from section 3 and wire them into `crates/state/Cargo.toml`
(`aes-gcm`, `sha2`, `base64`). Run `cargo check -p state` to confirm the fetch works.
### Step 2 — `crates/state/src/file.rs` (new, ~180 LOC)
```rust
use aes_gcm::aead::{Aead, KeyInit, OsRng};
use aes_gcm::{AeadCore, Aes256Gcm, Key, Nonce};
use nostr::nips::nip94::Sha256Hash;
use sha2::{Digest, Sha256};
pub const ALGORITHM: &str = "aes-gcm";
/// Ciphertext hash tag (NIP-17).
const TAG_SHA256: &str = "x";
const TAG_ORIGINAL_SHA256: &str = "ox";
const TAG_FILE_TYPE: &str = "file-type";
const TAG_ALGORITHM: &str = "encryption-algorithm";
const TAG_KEY: &str = "decryption-key";
const TAG_NONCE: &str = "decryption-nonce";
const TAG_SIZE: &str = "size";
const TAG_DIM: &str = "dim";
const TAG_ALT: &str = "alt";
/// Maximum blob size accepted when downloading (bytes). See edge cases.
pub const MAX_FILE_SIZE: usize = 25 * 1024 * 1024;
/// Result of encrypting a file: ciphertext to upload plus NIP-17 key material.
#[derive(Debug, Clone, PartialEq, Eq)]
pub struct EncryptedFile {
pub data: Vec<u8>,
/// base64-encoded 32-byte key
pub key: String,
/// base64-encoded 12-byte nonce
pub nonce: String,
}
/// NIP-17 kind 15 attachment metadata (tags + `.content` URL).
#[derive(Debug, Clone, PartialEq, Eq, PartialOrd, Ord)]
pub struct FileAttachment {
pub url: Url,
pub mime: String,
pub key: String,
pub nonce: String,
pub sha256: Option<String>,
pub original_sha256: Option<String>,
pub size: Option<u64>,
pub dim: Option<(u32, u32)>,
/// Non-standard display name (see "Open decisions" #2).
pub name: Option<String>,
}
impl FileAttachment {
/// Build the NIP-17 tags for a kind 15 rumor.
pub fn tags(&self) -> Vec<Tag>;
/// Parse a kind 15 rumor's tags. Returns `None` if key material is missing
/// or `encryption-algorithm` is not `aes-gcm`.
pub fn from_tags(tags: &Tags) -> Option<Self>;
pub fn is_image(&self) -> bool;
pub fn display_name(&self) -> SharedString; // falls back to mime/size
}
/// AES-256-GCM encrypt with a fresh random key and nonce.
pub fn encrypt(data: &[u8]) -> Result<EncryptedFile>;
/// AES-256-GCM decrypt using the values from a kind 15 rumor.
pub fn decrypt(data: &[u8], key: &str, nonce: &str) -> Result<Vec<u8>>;
/// Lowercase hex SHA-256, matching the NIP-94 `x`/`ox` convention.
pub fn sha256_hex(data: &[u8]) -> String;
/// Read a file, encrypt it, upload the ciphertext to Blossom, return the attachment.
#[cfg(not(target_arch = "wasm32"))]
pub async fn upload_encrypted(server: Url, path: PathBuf, cx: &AsyncApp) -> Result<FileAttachment>;
/// Fetch the blob, verify its SHA-256 against `expected_sha256`, then decrypt.
pub async fn download_and_decrypt(
url: &Url,
key: &str,
nonce: &str,
expected_sha256: Option<&str>,
cx: &AsyncApp,
) -> Result<Vec<u8>>;
```
Implementation notes, in priority order:
1. **Compute `x` and `ox` locally with `sha256_hex`.** Do **not** derive `x` from
`BlobDescriptor::sha256` — that field is a `bitcoin_hashes::sha256::Hash` whose
`Display` byte order could not be confirmed from the vendored sources, and a
reversed digest would silently break integrity checks in other clients. Use the
Blossom response only for `blob.url`. (`sha2``Sha256Hash::from_byte_array(...).to_hex()`
gives the conventional digest order and is unambiguous.)
2. **Key/nonce decoding must be tolerant.** NIP-17 does not specify an encoding:
accept base64 standard (padded and unpadded), base64url, and hex on read; emit
base64 standard on write. See "Open decisions" #1.
3. Validate sizes after decoding: key must be exactly 32 bytes, nonce exactly 12.
Return a descriptive error otherwise.
4. `decrypt` must fail closed on a bad tag (GCM authentication failure) — never
return partial plaintext.
5. `upload_encrypted` mirrors the existing `state::blossom::upload` shape:
`smol::fs::read` + `mime_guess::from_path`, `BlossomClient::new(server)`,
`upload_blob(ciphertext, Some("application/octet-stream"), None, Some(&keys))`
wrapped in `Tokio::spawn(cx, ...)`. Note the content type describes the
**ciphertext**; the plaintext MIME goes in the `file-type` tag.
On `wasm32` return `Err(anyhow!("File upload not supported on web"))`, matching
the existing stub in `blossom.rs`.
6. `download_and_decrypt` uses gpui's HTTP client
(`cx.update(|app| app.http_client())`, then
`client.get(url.as_str(), AsyncBody::default(), true)`), reads the body with
`futures::AsyncReadExt`, caps the read at `MAX_FILE_SIZE + 1` and rejects anything
larger, verifies `sha256_hex(&ciphertext)` against `expected_sha256`
(case-insensitive) when present, then decrypts. This path works on desktop and web.
7. `size` in `FileAttachment` is the **ciphertext** size (per the NIP wording), i.e.
`encrypted.data.len()`. `dim` is optional; if wanted, decode dimensions with the
`image` crate (`image::ImageReader`) — mark as a nice-to-have, not a blocker.
8. `encrypt` should take `&[u8]` (not a path) so it stays pure and testable without I/O.
### Step 3 — `crates/chat/src/message.rs` (~80 LOC)
- Add the kind constant next to `Message`:
```rust
/// NIP-17 file message (rust-nostr has no named `Kind` variant for 15).
pub const KIND_FILE_MESSAGE: Kind = Kind::Custom(15);
```
- Add the field:
```rust
pub struct Message {
// ... existing fields ...
/// Encrypted file attachment (NIP-17 kind 15)
pub file: Option<FileAttachment>,
}
```
- In all three `From` impls (`&Event`, `&UnsignedEvent`, `&NewMessage`), parse from
tags when `kind == KIND_FILE_MESSAGE`, and when a file is present **do not** run
`extract_and_remove_media_urls` or `extract_mentions` on `.content` — for kind 15
`.content` is a ciphertext URL that must never be rendered as text or as media:
```rust
let file = (val.kind == KIND_FILE_MESSAGE)
.then(|| FileAttachment::from_tags(&val.tags))
.flatten();
let (media, content) = if file.is_some() {
(vec![], String::new())
} else {
extract_and_remove_media_urls(&val.content)
};
```
- Add a helper for non-UI consumers that display a message one-line
(reply previews, notifications, copy):
```rust
/// Single-line representation: text for kind 14, `[File] <name>` for kind 15.
pub fn preview(&self) -> SharedString;
```
- `Ord`/`Eq`/`Hash` are keyed on `id`/`created_at`, so adding the field does not
change `Message` ordering semantics.
### Step 4 — `crates/chat/src/room.rs` (~70 LOC)
- Extract the shared tag logic from `rumor()` into a private helper so kind 14 and
kind 15 cannot drift:
```rust
/// `subject` + reply `e` tags + receiver `p` tags (excluding `sender`).
fn conversation_tags(&self, replies: &[EventId], sender: PublicKey, cx: &App) -> Vec<Tag>;
```
`rumor()` then becomes `EventBuilder::new(kind, content).tags(self.conversation_tags(...))`,
and gains no behavior change.
- Add:
```rust
/// Build a NIP-17 kind 15 (file message) rumor.
///
/// WARNING: never sign and send this event to relays; it is only ever wrapped.
pub fn file_rumor<I>(&self, file: FileAttachment, replies: I, cx: &App) -> Option<UnsignedEvent>
where
I: IntoIterator<Item = EventId>,
{
// tags = conversation_tags(...) ++ file.tags()
// content = file.url.to_string()
// EventBuilder::new(KIND_FILE_MESSAGE, content).tags(tags).finalize_unsigned(sender)
// then event.ensure_id()
}
```
- Fix the gift wrap's `k` tag so relays see the real inner kind
(`send_gift_wrap`, currently hardcoded):
```rust
let k_tag = Tag::custom("k", [rumor.kind.to_string()]);
```
- `Room::send` needs **no change**: it already accepts any `UnsignedEvent`, and the
self-backup branch publishes the same rumor wrapped for the sender, so the sender's
other devices receive the file message with the same key material.
### Step 5 — `crates/chat/src/lib.rs` (~5 LOC)
- `set_rumor`: store the real kind in the local cache tag:
```rust
Tag::custom("k", [rumor.kind.to_string()]),
```
- `get_rooms_task` currently filters `custom_tag(SingleLetterTag::LOWERCASE_K, "14")`,
which would hide rooms whose only messages are file messages. Simplest fix: drop
the `k` filter and keep `.kind(Kind::ApplicationSpecificData)`; the existing
in-loop JSON parse plus `tags.public_keys().next().is_some()` check already
discards junk, and kind 30078 rows in the local DB exist only for this rumor cache.
Fallback if the extra rows are a concern: run two queries (`k = "14"` and
`k = "15"`) and merge the results.
- No changes to `handle_notifications`, `extract_rumor`, `try_unwrap_with`, `new_message`,
`Room::get_messages` — all are already kind-agnostic. (`Room::get_messages` filters
only on the `r` room tag, so file messages load with the rest of the history.)
### Step 6 — (out of scope) UI wiring
Composer entry point, decryption cache, image/file rendering live in
`crates/chat_ui`. Not part of this plan; the APIs above are what that layer needs:
`state::upload_encrypted`, `FileAttachment::tags/from_tags`, `Room::file_rumor`,
`Room::send`, `state::download_and_decrypt`.
---
## 7. Open decisions
1. **Key/nonce encoding (interop risk — decide/verify first).** NIP-17 does not specify
how `decryption-key` / `decryption-nonce` are encoded, and no reference
implementation could be reached from this environment to confirm. Plan: emit
**base64 standard**, accept base64 std/unpadded/url-safe/hex on read. Before
release, verify against another NIP-17 client (0xchat, Amethyst) and tighten if
needed. Everything else in the plan is encoding-agnostic.
2. **Display name.** The NIP defines no filename tag and Blossom URLs end in a content
hash, so a name has to come from a non-standard tag. Proposal: emit
`["alt", <file name>]` (NIP-94 tag; NIP-17 allows "rest of tags", and `alt` is a
benign a11y field), tolerate its absence, fall back to mime + size. Alternative:
ship strictly spec-shaped and show mime + size only.
3. **`thumbhash`/`blurhash`/`thumb`/`fallback`/`dim`**: all optional; recommend skipping
in v1 (and letting the UI show a placeholder until the blob is decrypted).
4. **Web (wasm)**: encrypted upload stays unsupported on web, matching today's
`state::blossom::upload` stub. Download/decrypt is cross-platform via gpui's client.
5. **Ciphertext caching**: recommend *not* persisting decrypted plaintext in the local
database in v1; decryption happens on demand from the blob URL.
---
## 8. Edge cases and failure handling
| Case | Behavior |
|---|---|
| `encryption-algorithm` != `aes-gcm` | do not decrypt; report "unsupported encryption" (forward compatibility) |
| missing `decryption-key` / `decryption-nonce` | `FileAttachment::from_tags` returns `None` → surfaced as a failed message, not a panic |
| key != 32 bytes or nonce != 12 bytes | descriptive error |
| GCM authentication failure | error; never emit partial plaintext |
| `sha256_hex(ciphertext) != x` | error (tampered or corrupted blob), checked **before** decrypting |
| blob larger than `MAX_FILE_SIZE` | rejected while reading the body |
| HTTP error / non-200 | error message including status |
| `content` not a valid URL | error; message still cached so it renders as a failed attachment |
| sender's own copy | works automatically via the existing self-backup gift wrap |
| multiple receivers | one blob, one key, key re-sent inside each recipient's gift wrap |
---
## 9. Verification
The feature cannot be exercised end-to-end without the UI step (out of scope), so
backend verification is:
1. `cargo check -p state -p chat` after each step.
2. `cargo check -p coop_web --target wasm32-unknown-unknown` to confirm the
`cfg(target_arch = "wasm32")` stubs and the new module compile for web.
3. Cheap unit checks (no network, ~15 LOC in `state/src/file.rs`), despite "no tests
required" for the feature, because these are pure functions and catch the two
riskiest bugs — encoding and tag ordering:
- `encrypt` → `decrypt` round trip returns the original bytes.
- flipping one ciphertext bit makes `decrypt` fail.
- `sha256_hex(b"abc") == "ba7816bf8f01cfea414140de5dae2223b00361a396177a9cb410ff61f20015ad"`
(confirms the digest order used for `x`/`ox`).
- `FileAttachment` tags → `from_tags` round trip preserves key/nonce/mime/url.
4. End-to-end (requires the UI step or a temporary call site): send to self with
`RoomConfig::backup` enabled and confirm the rumor unwraps, downloads, verifies and
decrypts; then a second account; then a cross-client check for decision #1.
---
## 10. Estimated size
| File | Change | ~LOC |
|---|---|---|
| `Cargo.toml`, `crates/state/Cargo.toml` | deps | 6 |
| `crates/state/src/file.rs` | new | 180 |
| `crates/state/src/lib.rs` | module + re-export | 2 |
| `crates/chat/src/message.rs` | parse + preview | 80 |
| `crates/chat/src/room.rs` | `file_rumor` + tag refactor + `k` tag | 70 |
| `crates/chat/src/lib.rs` | cache tag + room list query | 5 |
Order of work: 1 → 2 → 3 → 4 → 5, each step compiling on its own. Steps 24 are
independent of any UI change, so the branch stays green without touching `chat_ui`.
+4
View File
@@ -24,6 +24,10 @@ serde_json.workspace = true
mime_guess = "2.0.4"
aes-gcm.workspace = true
sha2.workspace = true
base64.workspace = true
[target.'cfg(target_arch = "wasm32")'.dependencies]
nostr-memory.workspace = true