11 Commits
Author SHA1 Message Date
Bendik Aagaard LynghaugandClaude Opus 5.5 6ca859c691 release v0.1.6
ci / quality (push) Failing after 5s
release / build (aarch64-unknown-linux-gnu) (push) Successful in 2m25s
release / build (x86_64-unknown-linux-gnu) (push) Successful in 2m0s
release / publish (push) Successful in 48s
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-03 13:30:44 +02:00
Bendik Aagaard LynghaugandClaude Opus 5.5 c2b9167e1e Pushes name the peer by ticket, and an allowance can confirm its push
PeerAllowPush and Push take a blob ticket the peer issued as well as a
NodeId; the ticket's addresses are remembered, so the push dials
directly. A peer allowed to push may observe too: the sender confirms a
push by observing it, and without that every allowance-only push
reported failure though it had landed. The allowance test now fails
instead of skipping on anything but an unreachable peer.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-03 13:29:03 +02:00
Bendik Aagaard LynghaugandClaude Opus 5.5 4d9985c9cb release v0.1.5
ci / quality (push) Failing after 5s
release / build (aarch64-unknown-linux-gnu) (push) Successful in 2m26s
release / build (x86_64-unknown-linux-gnu) (push) Successful in 2m18s
release / publish (push) Successful in 47s
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-03 13:24:51 +02:00
Bendik Aagaard LynghaugandClaude Opus 5.5 2063d11890 PeerAllowPush: consent to exchange pushes, for a while, and nothing else
PeerTrust syncs a peer's pins and serves it everything - too much to
grant a stranger. An app that wants two peers sharing something (two
phones at the same place) to hand each other new content allows pushes
for ttl_secs (at most a day): both sides may push, the receiver pins,
and the allowance lapses on its own. Asking again extends it;
PeerUntrust ends it. varde-ctl peer allow-push.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-03 13:21:50 +02:00
Bendik Aagaard LynghaugandClaude Opus 5.5 0006a188e3 release v0.1.4
ci / quality (push) Failing after 8s
release / build (aarch64-unknown-linux-gnu) (push) Successful in 2m26s
release / build (x86_64-unknown-linux-gnu) (push) Successful in 2m6s
release / publish (push) Successful in 48s
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-03 13:08:52 +02:00
Bendik Aagaard LynghaugandClaude Opus 5.5 687af8bd2a Put and Read: content on the socket, for clients that share no files
The system daemon runs as its own user behind ProtectHome, so it can
neither read a file a desktop app wrote for Add nor write one back for
Materialize - redoal on the system varde failed every get with EACCES.
Put imports base64 bytes carried in the request and Read returns a raw
blob's bytes in the reply, both up to MAX_INLINE_BYTES (16 MiB).

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-03 13:08:50 +02:00
Bendik Aagaard LynghaugandClaude Opus 5.5 c9dbc4a576 release v0.1.3
ci / quality (push) Successful in 2m40s
release / build (aarch64-unknown-linux-gnu) (push) Successful in 2m19s
release / build (x86_64-unknown-linux-gnu) (push) Successful in 1m58s
release / publish (push) Successful in 48s
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-03 12:45:45 +02:00
Bendik Aagaard LynghaugandClaude Opus 5.5 3382a42b99 varde in the uhhm Arch registry
dist/arch/PKGBUILD packages the release tarballs: system and per-user
units, the per-user socket on for every session (as PipeWire's is; the
daemon starts only when an app connects), config and man pages. The
release workflow publishes it for x86_64 and aarch64 from klokka.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-03 12:43:46 +02:00
Bendik Aagaard LynghaugandClaude Opus 5.5 b7752da694 The ctl roundtrip test sets relay_url
ci / quality (push) Successful in 2m36s
Config grew relay_url in 98db877; the test's literal never did, so CI
had not compiled since.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01MznPTC31w5mSUK4XMehcpi
2026-10-02 09:27:44 +02:00
Bendik Aagaard LynghaugandClaude Opus 4.8 4ff007e403 ci: v4 builtin:checkout in ci + release (no Node, no token-in-URL)
ci / quality (push) Failing after 53s
builtin:checkout is native Go, so the rust:1 container needs no Node and the
job token no longer rides in the fetch URL (was visible in process args).

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-09-28 13:39:06 +02:00
Bendik Aagaard LynghaugandClaude Fable 5 98db87703e Config: relay_url for a self-hosted iroh relay (RelayMode::Custom)
ci / quality (push) Failing after 56s
release / build (aarch64-unknown-linux-gnu) (push) Successful in 2m17s
release / build (x86_64-unknown-linux-gnu) (push) Successful in 1m58s
The sovereign hole-punch hook redoal's ADR-0021 milestone 3 needs: when
relay_url is set (VARDE_RELAY_URL or the config file), the endpoint
coordinates NAT traversal through the operator's own iroh relay instead
of n0's or none — takes precedence over wan_upload's n0 defaults. None
keeps the existing LAN-only / n0 behaviour, so nothing changes by default.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-09-09 23:15:28 +02:00
19 changed files with 474 additions and 45 deletions
+3 -10
View File
@@ -12,17 +12,10 @@ jobs:
runs-on: ubuntu-latest runs-on: ubuntu-latest
container: rust:1 container: rust:1
steps: steps:
# Not the checkout action: it runs on node, which the rust:1 job # v4 built-in checkout: native Go, needs no Node (rust:1 has none) and
# container doesn't have and this runner doesn't inject (the v5 # keeps the token out of a fetch URL. Uses the job token for the private repo.
# run died with 'exec: "node": not found'). Plain git needs
# neither, and the token URL works against the private repo.
- name: checkout - name: checkout
run: | uses: builtin:checkout
git init -q .
git fetch --depth 1 \
"https://gitea:${{ secrets.GITHUB_TOKEN }}@${GITHUB_SERVER_URL#https://}/${GITHUB_REPOSITORY}.git" \
"$GITHUB_SHA"
git checkout -q FETCH_HEAD
- name: rustfmt - name: rustfmt
run: | run: |
rustup component add rustfmt rustup component add rustfmt
+45 -10
View File
@@ -19,17 +19,10 @@ jobs:
matrix: matrix:
target: [x86_64-unknown-linux-gnu, aarch64-unknown-linux-gnu] target: [x86_64-unknown-linux-gnu, aarch64-unknown-linux-gnu]
steps: steps:
# Not the checkout action: it runs on node, which the rust:1 job # v4 built-in checkout: native Go, needs no Node (rust:1 has none) and
# container doesn't have and this runner doesn't inject (the v5 # keeps the token out of a fetch URL. Uses the job token for the private repo.
# run died with 'exec: "node": not found'). Plain git needs
# neither, and the token URL works against the private repo.
- name: checkout - name: checkout
run: | uses: builtin:checkout
git init -q .
git fetch --depth 1 \
"https://gitea:${{ secrets.GITHUB_TOKEN }}@${GITHUB_SERVER_URL#https://}/${GITHUB_REPOSITORY}.git" \
"$GITHUB_SHA"
git checkout -q FETCH_HEAD
- name: install tooling - name: install tooling
run: | run: |
@@ -96,3 +89,45 @@ jobs:
-H "Authorization: token $TOKEN" \ -H "Authorization: token $TOKEN" \
-F "attachment=@$f" -F "attachment=@$f"
done done
# The uhhm Arch registry (pacman.conf: .../api/packages/bl/arch/uhhm),
# from the tarballs just released. On klokka: an Arch host with
# makepkg. REGISTRY_TOKEN is the uhhm org's write:package token.
publish:
needs: build
runs-on: aarch64
steps:
- name: checkout
uses: builtin:checkout
- name: fetch tarballs and pin the PKGBUILD
run: |
set -eu
TAG="${GITHUB_REF_NAME}"
VERSION="${TAG#v}"
BASE="${GITHUB_SERVER_URL}/${GITHUB_REPOSITORY}/releases/download/${TAG}"
cd dist/arch
for t in x86_64 aarch64; do
f="varde-${VERSION}-${t}-unknown-linux-gnu.tar.gz"
# The repo is private: anonymous release downloads 404.
curl -fsSL -H "Authorization: token ${{ secrets.GITHUB_TOKEN }}" -o "$f" "$BASE/$f"
done
SUM_X86=$(sha256sum "varde-${VERSION}-x86_64-unknown-linux-gnu.tar.gz" | cut -d' ' -f1)
SUM_AARCH=$(sha256sum "varde-${VERSION}-aarch64-unknown-linux-gnu.tar.gz" | cut -d' ' -f1)
sed -i "s/^pkgver=.*/pkgver=${VERSION}/" PKGBUILD
sed -i "s/sha256sums_x86_64=('.*')/sha256sums_x86_64=('${SUM_X86}')/" PKGBUILD
sed -i "s/sha256sums_aarch64=('.*')/sha256sums_aarch64=('${SUM_AARCH}')/" PKGBUILD
- name: publish to the uhhm Arch registry
run: |
set -eu
if [ -z "${{ secrets.REGISTRY_TOKEN }}" ]; then
echo "::error::REGISTRY_TOKEN not set - add it to publish"
exit 1
fi
cd dist/arch
for carch in aarch64 x86_64; do
pkgfile=$(CARCH="$carch" makepkg --packagelist | tail -1)
CARCH="$carch" makepkg -f --nodeps --noconfirm
curl --fail-with-body --user "bl:${{ secrets.REGISTRY_TOKEN }}" \
--upload-file "$pkgfile" \
"${GITHUB_SERVER_URL}/api/packages/bl/arch/uhhm"
done
+1
View File
@@ -1,2 +1,3 @@
target/ target/
.claude/ .claude/
.DS_Store
Generated
+4 -3
View File
@@ -4491,7 +4491,7 @@ checksum = "ba73ea9cf16a25df0c8caa16c51acb937d5712a8429db78a3ee29d5dcacd3a65"
[[package]] [[package]]
name = "varde-ctl" name = "varde-ctl"
version = "0.1.1" version = "0.1.6"
dependencies = [ dependencies = [
"anyhow", "anyhow",
"clap", "clap",
@@ -4504,9 +4504,10 @@ dependencies = [
[[package]] [[package]]
name = "varde-daemon" name = "varde-daemon"
version = "0.1.1" version = "0.1.6"
dependencies = [ dependencies = [
"anyhow", "anyhow",
"base64",
"iroh", "iroh",
"iroh-blobs", "iroh-blobs",
"iroh-mdns-address-lookup", "iroh-mdns-address-lookup",
@@ -4529,7 +4530,7 @@ dependencies = [
[[package]] [[package]]
name = "varde-proto" name = "varde-proto"
version = "0.1.1" version = "0.1.6"
dependencies = [ dependencies = [
"serde", "serde",
"serde_json", "serde_json",
+1 -1
View File
@@ -3,7 +3,7 @@ resolver = "2"
members = ["varde-proto", "varde-daemon", "varde-ctl"] members = ["varde-proto", "varde-daemon", "varde-ctl"]
[workspace.package] [workspace.package]
version = "0.1.1" version = "0.1.6"
edition = "2021" edition = "2021"
license = "MIT OR Apache-2.0" license = "MIT OR Apache-2.0"
repository = "https://project.uhhm.no/uhhm/varde" repository = "https://project.uhhm.no/uhhm/varde"
+41
View File
@@ -0,0 +1,41 @@
# The uhhm Arch registry package, built by .gitea/workflows/release.yml
# from the release tarballs (the CI pins pkgver and the sums). For a
# from-source build (AUR style) see ../PKGBUILD.
pkgname=varde
pkgver=0.1.3
pkgrel=1
pkgdesc="Content-addressed blob mirror daemon on iroh"
arch=('x86_64' 'aarch64')
url="https://project.uhhm.no/uhhm/varde"
license=('MIT OR Apache-2.0')
depends=('gcc-libs' 'glibc')
optdepends=('networkmanager: metered-connection detection')
backup=('etc/varde/config.toml')
install=varde.install
options=('!strip' '!debug')
source_x86_64=("varde-${pkgver}-x86_64-unknown-linux-gnu.tar.gz")
source_aarch64=("varde-${pkgver}-aarch64-unknown-linux-gnu.tar.gz")
sha256sums_x86_64=('SKIP')
sha256sums_aarch64=('SKIP')
package() {
cd "varde-${pkgver}-${CARCH}-unknown-linux-gnu"
install -Dm755 bin/varde-daemon "$pkgdir/usr/bin/varde-daemon"
install -Dm755 bin/varde-ctl "$pkgdir/usr/bin/varde-ctl"
install -Dm644 systemd/varde.service "$pkgdir/usr/lib/systemd/system/varde.service"
install -Dm644 systemd/varde.socket "$pkgdir/usr/lib/systemd/system/varde.socket"
install -Dm644 systemd/user/varde.service "$pkgdir/usr/lib/systemd/user/varde.service"
install -Dm644 systemd/user/varde.socket "$pkgdir/usr/lib/systemd/user/varde.socket"
# The per-user socket is on for every session, the way PipeWire's
# is: it costs nothing until an app connects, and then the daemon
# starts by socket activation. The system daemon stays opt-in.
install -dm755 "$pkgdir/usr/lib/systemd/user/sockets.target.wants"
ln -s ../varde.socket "$pkgdir/usr/lib/systemd/user/sockets.target.wants/varde.socket"
install -Dm644 config.toml.example "$pkgdir/etc/varde/config.toml"
install -Dm644 man/varde.8 "$pkgdir/usr/share/man/man8/varde.8"
install -Dm644 man/varde-ctl.1 "$pkgdir/usr/share/man/man1/varde-ctl.1"
install -Dm644 LICENSE-MIT "$pkgdir/usr/share/licenses/$pkgname/LICENSE-MIT"
install -Dm644 LICENSE-APACHE "$pkgdir/usr/share/licenses/$pkgname/LICENSE-APACHE"
}
+10
View File
@@ -0,0 +1,10 @@
post_install() {
echo ":: The per-user varde socket starts with your next login."
echo " To use it now: systemctl --user daemon-reload && systemctl --user start varde.socket"
echo " The system daemon is opt-in: systemctl enable --now varde.socket"
}
post_upgrade() {
echo ":: Restart a running daemon to pick up the new version:"
echo " systemctl --user try-restart varde.service (and/or: systemctl try-restart varde.service)"
}
+14 -1
View File
@@ -73,7 +73,8 @@ enum Command {
/// Peer trust operations /// Peer trust operations
#[command(subcommand)] #[command(subcommand)]
Peer(PeerCommand), Peer(PeerCommand),
/// Push fully-present content to a trusted peer /// Push fully-present content to a trusted peer, or one allowed to
/// exchange pushes
Push { Push {
/// BLAKE3 hash (hex) /// BLAKE3 hash (hex)
hash: String, hash: String,
@@ -116,6 +117,14 @@ enum PeerCommand {
/// iroh NodeId (z-base-32) /// iroh NodeId (z-base-32)
node_id: String, node_id: String,
}, },
/// Allow pushes with a peer for a while, without trusting it
AllowPush {
/// iroh NodeId (z-base-32)
node_id: String,
/// How long, in seconds (at most a day)
#[arg(long, default_value_t = 1800)]
ttl: u64,
},
/// List trusted peers /// List trusted peers
List, List,
} }
@@ -187,6 +196,9 @@ fn to_request(command: Command) -> Result<Request> {
}, },
Command::Peer(PeerCommand::Trust { node_id }) => Request::PeerTrust { node_id }, Command::Peer(PeerCommand::Trust { node_id }) => Request::PeerTrust { node_id },
Command::Peer(PeerCommand::Untrust { node_id }) => Request::PeerUntrust { node_id }, Command::Peer(PeerCommand::Untrust { node_id }) => Request::PeerUntrust { node_id },
Command::Peer(PeerCommand::AllowPush { node_id, ttl }) => {
Request::PeerAllowPush { node_id, ttl_secs: ttl }
}
Command::Peer(PeerCommand::List) => Request::PeerList {}, Command::Peer(PeerCommand::List) => Request::PeerList {},
Command::Push { hash, node_id } => Request::Push { hash, node_id }, Command::Push { hash, node_id } => Request::Push { hash, node_id },
Command::Gc => Request::Gc {}, Command::Gc => Request::Gc {},
@@ -246,6 +258,7 @@ fn print_human(data: Option<&ResponseData>) {
match data { match data {
ResponseData::Added { hash, bytes } => println!("added {hash} ({bytes} bytes)"), ResponseData::Added { hash, bytes } => println!("added {hash} ({bytes} bytes)"),
ResponseData::Pinned { hash } => println!("pinned {hash}"), ResponseData::Pinned { hash } => println!("pinned {hash}"),
ResponseData::Bytes { data } => println!("{data}"),
ResponseData::Done {} => println!("ok"), ResponseData::Done {} => println!("ok"),
ResponseData::Materialized { bytes, reflinked } => { ResponseData::Materialized { bytes, reflinked } => {
let how = if *reflinked { "reflinked" } else { "copied" }; let how = if *reflinked { "reflinked" } else { "copied" };
+1
View File
@@ -18,6 +18,7 @@ fn ctl_status_round_trips() {
max_download_bytes_per_sec: 0, max_download_bytes_per_sec: 0,
discovery: false, discovery: false,
wan_upload: false, wan_upload: false,
relay_url: None,
gc_interval_secs: 300, gc_interval_secs: 300,
}; };
+2
View File
@@ -16,6 +16,8 @@ required-features = ["socket-api"]
varde-proto = { workspace = true } varde-proto = { workspace = true }
iroh = { workspace = true } iroh = { workspace = true }
iroh-blobs = { workspace = true } iroh-blobs = { workspace = true }
# Request::Put / Read carry content inline, base64.
base64 = "0.22"
# mDNS/local-network address lookup left iroh core in 1.0. # mDNS/local-network address lookup left iroh core in 1.0.
iroh-mdns-address-lookup = "0.4" iroh-mdns-address-lookup = "0.4"
# Interop with iroh-blobs streams and channels; both are already in the # Interop with iroh-blobs streams and channels; both are already in the
+11
View File
@@ -24,6 +24,12 @@ pub struct Config {
/// Whether any WAN (non-link-local) upload is permitted. Default off: /// Whether any WAN (non-link-local) upload is permitted. Default off:
/// LAN-only posture with zero WAN upload. /// LAN-only posture with zero WAN upload.
pub wan_upload: bool, pub wan_upload: bool,
/// A self-hosted iroh relay URL for NAT traversal. When set, the
/// endpoint uses this relay (`RelayMode::Custom`) instead of n0's
/// or none — the sovereign hole-punch coordinator an embedder runs
/// on its own infrastructure. Takes precedence over `wan_upload`'s
/// n0 defaults. `None` keeps the LAN-only / n0 behaviour.
pub relay_url: Option<String>,
/// Interval of the store's built-in garbage collector in seconds. /// Interval of the store's built-in garbage collector in seconds.
/// Since iroh-blobs 0.103 there is no on-demand gc; unpinned blobs /// Since iroh-blobs 0.103 there is no on-demand gc; unpinned blobs
/// are swept by this loop. /// are swept by this loop.
@@ -41,6 +47,7 @@ struct FileConfig {
max_download_bytes_per_sec: Option<u64>, max_download_bytes_per_sec: Option<u64>,
discovery: Option<bool>, discovery: Option<bool>,
wan_upload: Option<bool>, wan_upload: Option<bool>,
relay_url: Option<String>,
gc_interval_secs: Option<u64>, gc_interval_secs: Option<u64>,
} }
@@ -176,6 +183,10 @@ impl Config {
wan_upload: env_bool("VARDE_WAN_UPLOAD")? wan_upload: env_bool("VARDE_WAN_UPLOAD")?
.or(file.wan_upload) .or(file.wan_upload)
.unwrap_or(false), .unwrap_or(false),
relay_url: std::env::var("VARDE_RELAY_URL")
.ok()
.filter(|s| !s.is_empty())
.or(file.relay_url),
gc_interval_secs: env_u64("VARDE_GC_INTERVAL")? gc_interval_secs: env_u64("VARDE_GC_INTERVAL")?
.or(file.gc_interval_secs) .or(file.gc_interval_secs)
.unwrap_or(300), .unwrap_or(300),
+70 -8
View File
@@ -335,6 +335,26 @@ impl Daemon {
} }
} }
/// The peer a request names: a NodeId, or a blob ticket from it (whose
/// addresses are remembered, so dialing it needs no discovery). An app
/// that knows peers by the tickets they hand out - redoal does - can
/// allow and push without unpacking them.
fn peer_of(&self, peer: &str) -> Result<iroh::EndpointId, OpError> {
if let Ok(id) = peer.parse::<iroh::EndpointId>() {
return Ok(id);
}
match crate::transfer::parse_ticket(peer) {
Ok((_, _, addr)) => {
let id = addr.id;
self.transfer.remember(addr);
Ok(id)
}
Err(_) => Err(OpError::InvalidArgument(format!(
"{peer:?} is neither a node id nor a ticket"
))),
}
}
/// Trusted peers seen on the LAN recently. /// Trusted peers seen on the LAN recently.
fn present_trusted_peers(&self) -> Vec<PeerInfo> { fn present_trusted_peers(&self) -> Vec<PeerInfo> {
let peers = self.lan_peers.lock().expect("lan peers lock"); let peers = self.lan_peers.lock().expect("lan peers lock");
@@ -371,6 +391,39 @@ impl Daemon {
bytes, bytes,
}) })
} }
Request::Put { data } => {
use base64::Engine as _;
let bytes = base64::engine::general_purpose::STANDARD
.decode(data.as_bytes())
.map_err(|e| OpError::InvalidArgument(format!("put: data is not base64: {e}")))?;
if bytes.len() as u64 > varde_proto::MAX_INLINE_BYTES {
return Err(OpError::InvalidArgument(format!(
"put: {} bytes, more than {} inline; add it by path",
bytes.len(),
varde_proto::MAX_INLINE_BYTES
)));
}
let (hash, bytes) = self.store.add_bytes(bytes).await?;
self.meta.record_format(&hash.to_hex(), StoredFormat::Raw)?;
Ok(ResponseData::Added {
hash: hash.to_hex().to_string(),
bytes,
})
}
Request::Read { hash } => {
use base64::Engine as _;
let parsed = parse_hash(&hash)?;
if self.meta.format_of(&parsed.to_hex()) == StoredFormat::HashSeq {
return Err(OpError::InvalidArgument(format!(
"read: {} is a collection; materialize it",
parsed.to_hex()
)));
}
let bytes = self.store.read_bytes(parsed, varde_proto::MAX_INLINE_BYTES).await?;
Ok(ResponseData::Bytes {
data: base64::engine::general_purpose::STANDARD.encode(bytes),
})
}
Request::Pin { hash, policy } => { Request::Pin { hash, policy } => {
let parsed = parse_hash(&hash)?; let parsed = parse_hash(&hash)?;
self.meta.pin(&parsed.to_hex(), policy)?; self.meta.pin(&parsed.to_hex(), policy)?;
@@ -533,24 +586,33 @@ impl Daemon {
let parsed: iroh::EndpointId = node_id let parsed: iroh::EndpointId = node_id
.parse() .parse()
.map_err(|e| OpError::InvalidArgument(format!("invalid node id: {e}")))?; .map_err(|e| OpError::InvalidArgument(format!("invalid node id: {e}")))?;
if !self.meta.untrust_peer(&parsed.to_string())? { let was_trusted = self.meta.untrust_peer(&parsed.to_string())?;
let was_allowed = self.meta.disallow_push(&parsed.to_string())?;
if !was_trusted && !was_allowed {
return Err(OpError::NotFound(format!("{parsed} was not trusted"))); return Err(OpError::NotFound(format!("{parsed} was not trusted")));
} }
Ok(ResponseData::Done {}) Ok(ResponseData::Done {})
} }
Request::PeerAllowPush { node_id, ttl_secs } => {
let parsed = self.peer_of(&node_id)?;
let ttl = ttl_secs.min(varde_proto::MAX_PUSH_TTL_SECS);
self.meta
.allow_push(&parsed.to_string(), crate::meta::unix_now() + ttl)?;
debug!(peer = %parsed, ttl, "pushes allowed");
Ok(ResponseData::Done {})
}
Request::PeerList {} => Ok(ResponseData::Peers { Request::PeerList {} => Ok(ResponseData::Peers {
peers: self.present_trusted_peers(), peers: self.present_trusted_peers(),
}), }),
Request::Push { hash, node_id } => { Request::Push { hash, node_id } => {
let parsed = parse_hash(&hash)?; let parsed = parse_hash(&hash)?;
let peer: iroh::EndpointId = node_id let peer = self.peer_of(&node_id)?;
.parse() // Consent, sending side: we only push to peers we trust or
.map_err(|e| OpError::InvalidArgument(format!("invalid node id: {e}")))?; // hold a push allowance for. (The receiver independently
// Consent, sending side: we only push to peers we trust. // requires the same of us.)
// (The receiver independently requires that it trusts us.) if !self.meta.may_push(&peer.to_string()) {
if !self.meta.is_trusted(&peer.to_string()) {
return Err(OpError::InvalidArgument(format!( return Err(OpError::InvalidArgument(format!(
"{peer} is not a trusted peer; trust it before pushing" "{peer} is neither trusted nor allowed to exchange pushes; trust it or allow pushes first"
))); )));
} }
let (_, _, complete) = self.store.presence(parsed).await?; let (_, _, complete) = self.store.presence(parsed).await?;
+60
View File
@@ -40,6 +40,10 @@ struct MetaState {
/// Trusted peer NodeIds (z-base-32). /// Trusted peer NodeIds (z-base-32).
#[serde(default)] #[serde(default)]
trusted_peers: BTreeSet<String>, trusted_peers: BTreeSet<String>,
/// Peers allowed to push to us and be pushed to, until a unix time
/// (`PeerAllowPush`): consent for that one thing, lapsing on its own.
#[serde(default)]
push_peers: BTreeMap<String, u64>,
/// Hashes (hex) the user shared via ticket export: standing consent /// Hashes (hex) the user shared via ticket export: standing consent
/// to serve them to anyone, like an `open_lan` pin. /// to serve them to anyone, like an `open_lan` pin.
#[serde(default)] #[serde(default)]
@@ -164,6 +168,31 @@ impl Meta {
.clone() .clone()
} }
/// Allow pushes with `node_id` until `until` (unix seconds). Extends,
/// never shortens, an allowance already held; expired ones are
/// dropped on the way.
pub fn allow_push(&self, node_id: &str, until: u64) -> Result<()> {
let now = unix_now();
self.mutate(|s| {
s.push_peers.retain(|_, u| *u > now);
let entry = s.push_peers.entry(node_id.to_string()).or_insert(0);
*entry = (*entry).max(until);
})
}
/// Whether pushes with `node_id` are allowed now: a trusted peer, or
/// one holding an unexpired push allowance.
pub fn may_push(&self, node_id: &str) -> bool {
let state = self.state.lock().expect("meta lock poisoned");
state.trusted_peers.contains(node_id)
|| state.push_peers.get(node_id).is_some_and(|u| *u > unix_now())
}
/// End a push allowance. Returns whether there was one.
pub fn disallow_push(&self, node_id: &str) -> Result<bool> {
self.mutate(|s| s.push_peers.remove(node_id).is_some())
}
pub fn is_trusted(&self, node_id: &str) -> bool { pub fn is_trusted(&self, node_id: &str) -> bool {
self.state self.state
.lock() .lock()
@@ -173,6 +202,14 @@ impl Meta {
} }
} }
/// Seconds since the unix epoch.
pub fn unix_now() -> u64 {
std::time::SystemTime::now()
.duration_since(std::time::UNIX_EPOCH)
.map(|d| d.as_secs())
.unwrap_or(0)
}
#[cfg(test)] #[cfg(test)]
mod tests { mod tests {
use super::*; use super::*;
@@ -201,4 +238,27 @@ mod tests {
assert!(meta.unpin("h").unwrap()); assert!(meta.unpin("h").unwrap());
assert!(!meta.is_pinned("h")); assert!(!meta.is_pinned("h"));
} }
#[test]
fn a_push_allowance_lapses_and_never_shortens() {
let dir = tempfile::tempdir().unwrap();
let meta = Meta::load(dir.path()).unwrap();
let now = unix_now();
assert!(!meta.may_push("p"));
meta.allow_push("p", now + 600).unwrap();
assert!(meta.may_push("p"));
assert!(!meta.is_trusted("p"), "a push allowance is not trust");
meta.allow_push("p", now + 10).unwrap();
assert!(meta.may_push("p"), "a shorter ask does not cut it short");
meta.allow_push("q", now.saturating_sub(1)).unwrap();
assert!(!meta.may_push("q"), "expired");
// Survives a reload.
let again = Meta::load(dir.path()).unwrap();
assert!(again.may_push("p"));
assert!(again.disallow_push("p").unwrap());
assert!(!again.may_push("p"));
// Full trust allows pushes too.
again.trust_peer("t").unwrap();
assert!(again.may_push("t"));
}
} }
+18 -8
View File
@@ -111,6 +111,8 @@ pub struct Gate {
#[derive(Debug, Clone)] #[derive(Debug, Clone)]
struct ConnInfo { struct ConnInfo {
trusted: bool, trusted: bool,
/// Trusted, or holding a push allowance (`PeerAllowPush`).
may_push: bool,
endpoint_id: Option<String>, endpoint_id: Option<String>,
} }
@@ -142,6 +144,11 @@ pub fn provider_events(gate: Gate) -> EventSender {
.endpoint_id .endpoint_id
.map(|id| gate.meta.is_trusted(&id.to_string())) .map(|id| gate.meta.is_trusted(&id.to_string()))
.unwrap_or(false); .unwrap_or(false);
let may_push = m
.inner
.endpoint_id
.map(|id| gate.meta.may_push(&id.to_string()))
.unwrap_or(false);
let ClientConnected { let ClientConnected {
connection_id, connection_id,
endpoint_id, endpoint_id,
@@ -151,6 +158,7 @@ pub fn provider_events(gate: Gate) -> EventSender {
connection_id, connection_id,
ConnInfo { ConnInfo {
trusted, trusted,
may_push,
endpoint_id: endpoint_id.map(|id| id.to_string()), endpoint_id: endpoint_id.map(|id| id.to_string()),
}, },
); );
@@ -192,11 +200,12 @@ pub fn provider_events(gate: Gate) -> EventSender {
ProviderMessage::ObserveRequestReceived(m) => { ProviderMessage::ObserveRequestReceived(m) => {
// Presence is information too: gate it like data. // Presence is information too: gate it like data.
// Bitfields are metadata, not payload, so trusted // Bitfields are metadata, not payload, so trusted
// peers may observe even while serving is off or the // peers - and peers allowed to push, whose push is
// connection is metered (push confirmation needs it). // confirmed by observing it - may observe even while
// serving is off or the connection is metered.
let trusted = conns let trusted = conns
.get(&m.inner.connection_id) .get(&m.inner.connection_id)
.map(|i| i.trusted) .map(|i| i.trusted || i.may_push)
.unwrap_or(false); .unwrap_or(false);
let allowed = trusted let allowed = trusted
|| allow(&gate, &conns, m.inner.connection_id, [m.inner.request.hash]); || allow(&gate, &conns, m.inner.connection_id, [m.inner.request.hash]);
@@ -208,12 +217,13 @@ pub fn provider_events(gate: Gate) -> EventSender {
m.tx.send(verdict).await.ok(); m.tx.send(verdict).await.ok();
} }
ProviderMessage::PushRequestReceived(m) => { ProviderMessage::PushRequestReceived(m) => {
// Inbound writes: trusted peers only, deferred while // Inbound writes: trusted peers and push allowances
// metered (RateLimited invites a retry later). // only, deferred while metered (RateLimited invites a
// retry later).
let info = conns.get(&m.inner.connection_id); let info = conns.get(&m.inner.connection_id);
let trusted = info.map(|i| i.trusted).unwrap_or(false); let may_push = info.map(|i| i.may_push).unwrap_or(false);
debug!(hash = %m.inner.request.hash.to_hex(), trusted, "push request received"); debug!(hash = %m.inner.request.hash.to_hex(), may_push, "push request received");
let verdict = if !trusted { let verdict = if !may_push {
Err(AbortReason::Permission) Err(AbortReason::Permission)
} else if gate.metered.is_metered() { } else if gate.metered.is_metered() {
Err(AbortReason::RateLimited) Err(AbortReason::RateLimited)
+24
View File
@@ -228,6 +228,30 @@ impl BlobStore {
/// Export `hash` to `dest`. Returns bytes written and whether at least /// Export `hash` to `dest`. Returns bytes written and whether at least
/// one file was reflinked. /// one file was reflinked.
/// Import `data` as one raw blob (`Request::Put`).
pub async fn add_bytes(&self, data: Vec<u8>) -> Result<(Hash, u64), OpError> {
let size = data.len() as u64;
let tag = self.store.add_bytes(data).temp_tag().await.map_err(internal)?;
info!(hash = %tag.hash().to_hex(), size, "imported bytes");
Ok((tag.hash(), size))
}
/// The whole content of a complete raw blob (`Request::Read`), when it
/// is no larger than `limit`.
pub async fn read_bytes(&self, hash: Hash, limit: u64) -> Result<Vec<u8>, OpError> {
match self.store.blobs().status(hash).await.map_err(internal)? {
BlobStatus::Complete { size } if size > limit => Err(OpError::InvalidArgument(format!(
"{} is {size} bytes, more than {limit} can be read inline; materialize it",
hash.to_hex()
))),
BlobStatus::Complete { .. } => {
let bytes = self.store.blobs().get_bytes(hash).await.map_err(internal)?;
Ok(bytes.to_vec())
}
_ => Err(OpError::NotFound(format!("{} is not complete in the store", hash.to_hex()))),
}
}
pub async fn materialize( pub async fn materialize(
&self, &self,
hash: Hash, hash: Hash,
+22 -4
View File
@@ -62,10 +62,19 @@ impl Transfer {
metered: MeteredState, metered: MeteredState,
) -> Result<Transfer> { ) -> Result<Transfer> {
let secret = load_or_create_secret(&config.store_dir.join("secret.key"))?; let secret = load_or_create_secret(&config.store_dir.join("secret.key"))?;
// With wan_upload the n0 defaults apply (their relays and DNS // A self-hosted relay (relay_url) is the sovereign hole-punch
// lookup, matching the pre-1.0 default relay mode); otherwise the // path and takes precedence: the endpoint coordinates through
// endpoint gets no external services at all. // the operator's own iroh relay, never n0's. Failing that,
let builder = if config.wan_upload { // wan_upload opts into n0's relays + DNS; otherwise the endpoint
// gets no external services at all (LAN-only).
let builder = if let Some(url) = &config.relay_url {
let relay_url: iroh::RelayUrl = url
.parse()
.with_context(|| format!("parsing relay_url {url:?}"))?;
let relay_map = iroh::RelayMap::from(relay_url);
Endpoint::builder(iroh::endpoint::presets::Minimal)
.relay_mode(RelayMode::Custom(relay_map))
} else if config.wan_upload {
Endpoint::builder(iroh::endpoint::presets::N0) Endpoint::builder(iroh::endpoint::presets::N0)
} else { } else {
Endpoint::builder(iroh::endpoint::presets::Minimal).relay_mode(RelayMode::Disabled) Endpoint::builder(iroh::endpoint::presets::Minimal).relay_mode(RelayMode::Disabled)
@@ -289,6 +298,15 @@ impl Transfer {
Ok(ticket.to_string()) Ok(ticket.to_string())
} }
/// Teach the endpoint where a peer can be reached (the addresses a
/// ticket carries), so a push dials it directly instead of waiting on
/// discovery.
pub fn remember(&self, addr: EndpointAddr) {
if !addr.addrs.is_empty() {
self.known_addrs.add_endpoint_info(addr);
}
}
/// Queue a background fetch of `content` from `providers`. Emits /// Queue a background fetch of `content` from `providers`. Emits
/// progress events and [`Event::PinComplete`] when fully verified. /// progress events and [`Event::PinComplete`] when fully verified.
pub fn spawn_fetch(&self, content: HashAndFormat, providers: Vec<EndpointAddr>) { pub fn spawn_fetch(&self, content: HashAndFormat, providers: Vec<EndpointAddr>) {
+58
View File
@@ -422,3 +422,61 @@ fn push_rejected_without_receiver_trust() {
// And the receiver holds none of it. // And the receiver holds none of it.
assert!(!is_complete(&b.socket, &hash)); assert!(!is_complete(&b.socket, &hash));
} }
/// A push allowance is enough on both sides: two daemons that only
/// allowed pushes with each other (strangers sharing a place, in redoal)
/// exchange content without trusting each other - and the allowance
/// grants nothing else, so neither lists the other as a trusted peer.
#[test]
fn push_allowance_is_enough_and_is_not_trust() {
let _serial = MDNS_SERIAL.lock().unwrap_or_else(|e| e.into_inner());
let dir_a = tempfile::tempdir().unwrap();
let dir_b = tempfile::tempdir().unwrap();
let a = support::spawn_daemon_with_env(dir_a.path(), &[("VARDE_DISCOVERY", "true")]);
let b = support::spawn_daemon_with_env(dir_b.path(), &[("VARDE_DISCOVERY", "true")]);
let mut client_a = support::Client::connect(&a.socket);
let mut client_b = support::Client::connect(&b.socket);
let id_a = node_id(&mut client_a);
let dir = tempfile::tempdir().unwrap();
let src = dir.path().join("voice.bin");
let payload = vec![7u8; 120_000];
std::fs::write(&src, &payload).unwrap();
let hash = add_file(&mut client_a, &src);
pin(&mut client_a, &hash, false);
// A knows B by a ticket B handed out (as redoal peers do), not by
// its bare node id.
let b_src = dir.path().join("b-own.bin");
std::fs::write(&b_src, vec![1u8; 1_000]).unwrap();
let b_hash = add_file(&mut client_b, &b_src);
let b_ticket = match client_b.request(&Request::TicketExport { hash: b_hash }).data {
Some(ResponseData::Ticket { ticket }) => ticket,
other => panic!("unexpected ticket reply: {other:?}"),
};
assert!(client_a.request(&Request::PeerAllowPush { node_id: b_ticket.clone(), ttl_secs: 600 }).ok);
assert!(client_b.request(&Request::PeerAllowPush { node_id: id_a, ttl_secs: 600 }).ok);
match client_a.request(&Request::PeerList {}).data {
Some(ResponseData::Peers { peers }) => assert!(peers.is_empty(), "an allowance is not trust: {peers:?}"),
other => panic!("unexpected peer list: {other:?}"),
}
// The peer's address arrives by mdns; retry until it does.
let deadline = Instant::now() + Duration::from_secs(30);
let reply = loop {
let reply = client_a.request(&Request::Push { hash: hash.clone(), node_id: b_ticket.clone() });
if reply.ok || Instant::now() > deadline {
break reply;
}
std::thread::sleep(Duration::from_millis(500));
};
if !reply.ok {
let message = reply.error.map(|e| e.message).unwrap_or_default();
// Only an unreachable peer is the environment's fault; any other
// refusal is a real failure.
assert!(message.contains("connect"), "push failed: {message}");
eprintln!("SKIP: push never reached the peer (multicast unavailable?): {message}");
return;
}
assert!(wait_complete(&b.socket, &hash, Duration::from_secs(60)), "pushed content not complete on receiver");
}
+42
View File
@@ -276,3 +276,45 @@ fn reflink_when_supported() {
drop(daemon); drop(daemon);
let _ = std::fs::remove_dir_all(&base); let _ = std::fs::remove_dir_all(&base);
} }
/// Put and Read carry the bytes on the socket: a client that shares no
/// filesystem with the daemon (a desktop app and the system daemon, which
/// runs as its own user behind ProtectHome) stores and reads back without
/// a path either side must reach. A Put blob reads back by Materialize too.
#[test]
fn bytes_round_trip_inline() {
use base64::Engine as _;
let b64 = base64::engine::general_purpose::STANDARD;
let daemon = support::spawn_daemon();
let mut client = support::Client::connect(&daemon.socket);
let payload: Vec<u8> = (0..50_000u32).flat_map(|i| i.to_le_bytes()).collect();
let reply = client.request(&Request::Put { data: b64.encode(&payload) });
assert!(reply.ok, "put failed: {:?}", reply.error);
let hash = match reply.data {
Some(ResponseData::Added { hash, bytes }) => {
assert_eq!(bytes, payload.len() as u64);
hash
}
other => panic!("unexpected put reply: {other:?}"),
};
let reply = client.request(&Request::Read { hash: hash.clone() });
assert!(reply.ok, "read failed: {:?}", reply.error);
match reply.data {
Some(ResponseData::Bytes { data }) => assert_eq!(b64.decode(data).unwrap(), payload),
other => panic!("unexpected read reply: {other:?}"),
}
let dir = tempfile::tempdir().unwrap();
let dest = dir.path().join("out.bin");
materialize(&mut client, &hash, &dest, MaterializeMode::Copy).unwrap();
assert_eq!(std::fs::read(&dest).unwrap(), payload);
// Not base64: refused, not stored.
let reply = client.request(&Request::Put { data: "not base64!".into() });
assert_eq!(reply.error.expect("error body").code, ErrorCode::InvalidArgument);
// Unknown hash: not found.
let reply = client.request(&Request::Read { hash: "0".repeat(64) });
assert!(!reply.ok);
}
+47
View File
@@ -20,6 +20,13 @@ use serde::{Deserialize, Serialize};
/// Bumped on incompatible changes; reported in [`GlobalStatus`]. /// Bumped on incompatible changes; reported in [`GlobalStatus`].
pub const PROTOCOL_VERSION: u32 = 2; pub const PROTOCOL_VERSION: u32 = 2;
/// The longest a [`Request::PeerAllowPush`] allowance may last: a day.
pub const MAX_PUSH_TTL_SECS: u64 = 24 * 60 * 60;
/// The most bytes a [`Request::Put`] or a [`Request::Read`] reply carries
/// inline. Larger content goes through `Add`/`Materialize` and a path.
pub const MAX_INLINE_BYTES: u64 = 16 * 1024 * 1024;
/// A request from a client to the daemon. One JSON object per line. /// A request from a client to the daemon. One JSON object per line.
#[derive(Debug, Clone, PartialEq, Eq, Serialize, Deserialize)] #[derive(Debug, Clone, PartialEq, Eq, Serialize, Deserialize)]
#[serde(tag = "type", rename_all = "snake_case")] #[serde(tag = "type", rename_all = "snake_case")]
@@ -61,6 +68,24 @@ pub enum Request {
#[serde(default)] #[serde(default)]
mode: MaterializeMode, mode: MaterializeMode,
}, },
/// Import bytes carried in the request, base64 (standard alphabet),
/// at most [`MAX_INLINE_BYTES`]. Replies [`ResponseData::Added`].
///
/// For a client that shares no filesystem with the daemon: the system
/// daemon runs as its own user with `ProtectHome`, so it can neither
/// read a file a desktop app wrote for `Add` nor write one back for
/// `Materialize`. Added in varde 0.1.4.
Put {
/// The content, base64.
data: String,
},
/// Read a complete raw blob back in the reply, base64, at most
/// [`MAX_INLINE_BYTES`]. Replies [`ResponseData::Bytes`]. The
/// counterpart of [`Request::Put`]; added in varde 0.1.4.
Read {
/// Hash (hex) of the blob.
hash: String,
},
/// Query daemon status, globally or for one hash. /// Query daemon status, globally or for one hash.
Status { Status {
/// If set, report on this hash; otherwise report global status. /// If set, report on this hash; otherwise report global status.
@@ -94,6 +119,20 @@ pub enum Request {
/// The peer's iroh NodeId (z-base-32). /// The peer's iroh NodeId (z-base-32).
node_id: String, node_id: String,
}, },
/// Allow pushes between this daemon and a peer for `ttl_secs` (at most
/// [`MAX_PUSH_TTL_SECS`]): we may push to it, and accept what it pushes
/// to us. Nothing else - unlike [`Request::PeerTrust`] it neither
/// syncs the peer's pins nor serves it our content - so an app can
/// grant it to strangers it shares something with for a while (two
/// phones at the same place) and let it lapse. Asking again extends
/// it; [`Request::PeerUntrust`] ends it early. Added in varde 0.1.5.
PeerAllowPush {
/// The peer's iroh NodeId (z-base-32), or a blob ticket it issued
/// (varde 0.1.6+), whose addresses are then used to reach it.
node_id: String,
/// How long the allowance lasts, in seconds.
ttl_secs: u64,
},
/// List trusted peers and their current connectivity. /// List trusted peers and their current connectivity.
PeerList {}, PeerList {},
/// Push fully-present content to a trusted peer, unprompted. /// Push fully-present content to a trusted peer, unprompted.
@@ -104,6 +143,9 @@ pub enum Request {
/// policy), so the content survives gc and shows up in `List`. /// policy), so the content survives gc and shows up in `List`.
Push { Push {
/// Hash (hex) of the fully-present blob or HashSeq root to push. /// Hash (hex) of the fully-present blob or HashSeq root to push.
///
/// The receiver below may be a NodeId or, from varde 0.1.6, a blob
/// ticket it issued.
hash: String, hash: String,
/// The receiving peer's iroh NodeId (z-base-32). /// The receiving peer's iroh NodeId (z-base-32).
node_id: String, node_id: String,
@@ -228,6 +270,11 @@ pub enum ResponseData {
/// Whether at least one file was reflinked rather than copied. /// Whether at least one file was reflinked rather than copied.
reflinked: bool, reflinked: bool,
}, },
/// A blob's content, base64. Reply to [`Request::Read`].
Bytes {
/// The content, base64 (standard alphabet).
data: String,
},
/// Global daemon status. Reply to [`Request::Status`] without a hash. /// Global daemon status. Reply to [`Request::Status`] without a hash.
Status(GlobalStatus), Status(GlobalStatus),
/// Per-hash status. Reply to [`Request::Status`] with a hash. /// Per-hash status. Reply to [`Request::Status`] with a hash.