Compare commits

...
6 Commits
Author SHA1 Message Date
Bendik Aagaard Lynghaug 68471969b5 Release 0.5.0
Test / test (push) Successful in 1m0s
Publish release / publish (push) Successful in 6m47s
2026-09-28 16:46:50 +02:00
Bendik Aagaard LynghaugandClaude Opus 5 214d4eb85c Sessions outlive the deploy that interrupted them
Test / test (push) Successful in 58s
A MemoryStore meant every restart signed out every desk on the site
being deployed. Bearable while a desk is three people, and not the
kind of thing that gets better on its own.

They live in NATS KV now: the JetStream already under the events, the
projections, the content reload and the access questions. No new
service, no new store, and the same bucket a second instance of a site
would read if there ever is one. `create` draws a fresh id rather than
writing over a session already at that key, the bucket's `max_age`
collects what nobody comes back for, and a record's own expiry is what
answers a load, whatever the bucket still holds.

Proved the only way it can be: a store opened twice against one
JetStream, which is what two runs of the binary are.

Also fixed, while there: the integration test asserted a record's
projected state was one that `aggregates::transition` never writes -
it appends events, the desk handler updates the projection - so it
failed whenever it was actually run. It now asserts what it says it
does, that a mailed reply leaves the state alone. test.yml has been
red since 0.3.47 while publish.yml stayed green, which is how v0.4.0
went out over it.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-09-28 16:38:05 +02:00
Bendik Aagaard LynghaugandClaude Opus 5 d5de25e8c6 A file can come back out, and a script cannot come back forever
Two halves of the same evening.

Uploads were one-way. A `type: file` field put an object in the bucket
and wrote its key into the record, and nothing ever read it back - a
desk showed `0aba5734-...-drawing.pdf` as a string with no link, so a
builder could receive drawings and never open one.
`GET /attachment/{bucket}/{record}/{field}` streams it. The key is not
the credential and is not taken from the caller: it is read out of the
record named, so a file is reachable only through a record somebody is
allowed to see, and a key that leaks off a desk is a link to nothing.

And there was no rate limiting anywhere. A burst of 30 per caller IP,
refilled one every two seconds, outside the session layer so anyone
over it is turned away before a handler is entered. Well above a
person filling in a form - a page load fetches several resources at
once - and well below being a nuisance. PORTAL_RATE_BURST and
PORTAL_RATE_SECONDS retune it; either 0 turns it off for an instance
behind something that already does this.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-09-28 16:28:43 +02:00
Bendik Aagaard LynghaugandClaude Opus 5 8d6f29011c An upload asks the policy, like every other way in
/upload asked nothing. It took a file, capped it at 20 MB, and wrote
it to the bucket for whoever sent it - no session, no page, no policy.
It was the one server entry point that did not ask the question every
other one asks, and on a site whose forms are public that makes the
route public too: a free disk for anyone who finds the host.

Harmless while no instance had storage configured, which is why it sat
there. kasse now has a garage wired to klingenberg's portal, so it is
not harmless any more.

An upload now carries the question and the alternative the file is
being attached to. The page has to exist, the alternative has to be
open and to actually declare a `type: file` field, and the policy has
to allow Submit on that page for whoever is asking - checked after the
bytes are read and before any of them are written.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-09-28 16:08:12 +02:00
Bendik Aagaard LynghaugandClaude Opus 5 9891c462c6 A page says nothing until it has something to say
The Suspense fallback on a question is the whole served page until
the question resolves - no header, no wordmark - so the word
"loading…" was the first and only thing a visitor or a crawler read
about the site. True, and not about them. It holds its space and
stays quiet now; `aria-busy` says what is happening to a reader that
would otherwise find an empty document. Same for the two resource
slots inside a page, which keep a line's worth of room.

Also gone: the `gitea_org_repos` resource kind, which no site ever
named. `gitea_starred` and `gitea_releases` stay.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-09-26 13:25:11 +02:00
Bendik Aagaard LynghaugandClaude Opus 5 f18d1a5181 A person stays in dialogue with their case
Five things a building authority needed and portal could not say,
found by building one and replaying its cases through the engine.

The one that mattered: a submitter could only ever act on a record
nobody had touched. `access.rs` pinned a self_transition's from-state
to `open`, though the runtime had always passed the real state and
asked. So the applicant written to for a missing drawing could not
send it, and nobody could withdraw once anything had happened.
`self_transition` now takes `from: [state, ...]` and compiles one rule
per state; omitted it still means the initial state alone, which is
what every unsubscribe link already meant.

Then the four beside it. A record can belong to another record
(`type: record`, `of:`), with `where: { field, equals }` on a listing
as the other half, so an application's page lists the objections that
name it. A public view can be narrowed to the states worth showing,
so a register that is public by law is public about what was decided
and not about what was withdrawn. A state can grant a group, so an
approval that means membership needs no second step. And a state can
be moved by a clock (`after: { days, to }`, src/deadlines.rs) - held
to the same declared edge as a desk button, firing the same mail,
publishing the same event.

Also: the README is a README again, and everything that was crowding
it is in docs/ - architecture, content reference, state machines,
access, operations.

95 tests, up from 82.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-09-25 21:35:25 +02:00
25 changed files with 2251 additions and 257 deletions
+22 -6
View File
@@ -4,6 +4,22 @@ One line per change, grouped by the release that shipped it. Newest
first. A release is a Cargo.toml bump plus a `v<version>` tag; the first. A release is a Cargo.toml bump plus a `v<version>` tag; the
content repos pin the tag they run (see the end of this file). content repos pin the tag they run (see the end of this file).
## 0.5.0 (2026-09-28)
- Sessions live in NATS KV (`src/sessions.rs`), not in memory. Every restart used to sign out every desk on the site being deployed; they now survive it, in the JetStream that is already under everything else - no new service and no new store to run. The bucket's `max_age` collects what nobody returns for, and a record's own expiry is what answers a load.
- Fixed: the JetStream integration test asserted a record's projected state was one a desk move had never written to it, so it failed whenever it actually ran. `test.yml` has been red since 0.3.47 while `publish.yml` stayed green, which is how 0.4.0 shipped past it.
- A file can come back out. `GET /attachment/{bucket}/{record}/{field}` streams what a `type: file` field stored, reading the object key out of the record rather than taking one from the caller - so a file is only reachable through a record somebody may see, and a key that leaks off a desk is a link to nothing. Before this, uploads were one-way: a desk showed the key as a string and there was no way to open it.
- Rate limiting, per caller IP, outside everything else: a burst of 30 refilled one every two seconds (`PORTAL_RATE_BURST`, `PORTAL_RATE_SECONDS`; either 0 turns it off). A form-filling visitor never reaches it; a script does. There was none at all before.
- `/upload` asks the policy, like every other way in. It asked nothing: any caller who could reach the host could put 20 MB in the bucket, as often as they liked, and a site whose forms are public is a site whose upload route is public. It now takes the `question` and `alternative` the file is being attached to, refuses a page that declares no `type: file` field or an alternative that is closed, and asks `Submit` on that page - the same question the submission itself will ask - before a byte is written. Latent until an instance had storage configured; the first one now does.
- A submitter stays in dialogue with their case for as long as it runs. `self_transition` takes `from: [state, ...]`, and the access policy compiles one rule per state named; it defaulted to the initial state alone and was written that way in the compiler, so an applicant written to for a missing drawing could not send it, and nobody could withdraw once anything had happened. Omitted, it still means the initial state alone.
- A record can belong to another record. `type: record` with `of: <bucket>` stores the parent's id, which arrives on the link rather than being typed; a submission naming a record that is not there is refused. The other half is `where: { field, equals }` on a Kv resource, so an application's page can list the objections that name it.
- A public view need not publish the whole bucket. `states: [...]` on a Kv resource shows only those, for one record as well as a listing - a register that is public by law is public about what was decided, not about what was withdrawn. The desk over the same bucket is a different spec and still sees all of it.
- A state can grant a group: `grants: <group>` beside its `event`, so an approval that means membership does not also need somebody to remember to invite them afterwards. The lint requires the forms that fill such a bucket to collect a required name and email, because by the time a desk presses approve there is nobody left to ask.
- A state can be moved by a clock: `after: { days, to }`, swept hourly, measured from the record's last move (`src/deadlines.rs`). A deadline may only do what a person could have done - `after.to` is held to the same declared edge as a desk button - and the move fires the same mail and publishes the same event, stamped `deadline`.
- Documentation moved out of the README into `docs/`: architecture, content reference, state machines, access, operations.
- A page says nothing until it has something to say. The Suspense fallback on a question was the whole served page - no header, no wordmark - so the first and only thing a visitor or a crawler read was the word "loading…", which is true and is not about them. It now holds the space and stays quiet, with `aria-busy` for a reader that would otherwise find an empty document.
- Gone: the `gitea_org_repos` resource kind, which no site ever named. `gitea_starred` and `gitea_releases` stay.
## 0.4.0 (2026-09-24) ## 0.4.0 (2026-09-24)
- A field can be a place: `type: location` puts a pin on a map and stores `{lat, lon, zoom}`; a record holding one renders as a still map on its desk and in any resource card. - A field can be a place: `type: location` puts a pin on a map and stores `{lat, lon, zoom}`; a record holding one renders as a still map on its desk and in any resource card.
@@ -324,14 +340,14 @@ content repos pin the tag they run (see the end of this file).
## What each site runs ## What each site runs
Read from the hosts on 2026-09-23; a rollout is a pin bump in the Read from the hosts on 2026-09-28; a rollout is a pin bump in the
site's content repo (`PORTAL_RELEASE` in its `deploy.yml`), or a manual site's content repo (`PORTAL_RELEASE` in its `deploy.yml`), or a manual
tarball deploy on kasse. tarball deploy on kasse.
| Site | Host | Instance | Version | | Site | Host | Instance | Version |
|---|---|---|---| |---|---|---|---|
| uhhm.no | ergo | `app@uhhm-portal` | v0.3.36 | | uhhm.no | ergo | `app@uhhm-portal` | v0.4.0 |
| redoal.com | ergo | `app@redoal-portal` | v0.3.36 | | redoal.com | ergo | `app@redoal-portal` | v0.4.0 |
| westra.klingenbergbygg.no | kasse | `app@westra-portal` | v0.3.41 | | westra.klingenbergbygg.no | kasse | `app@westra-portal` | v0.4.0 |
| portal.klingenbergbygg.no | kasse | `app@klingenberg-portal` | v0.3.41 | | portal.klingenbergbygg.no | kasse | `app@klingenberg-portal` | v0.4.0 |
| vel.klingenbergbygg.no | kasse | `app@tomtervel-portal` | v0.3.41 | | vel.klingenbergbygg.no | kasse | `app@tomtervel-portal` | v0.4.0 |
Generated
+213 -4
View File
@@ -132,9 +132,9 @@ checksum = "4288f83726785267c6f2ef073a3d83dc3f9b81464e9f99898240cced85fce35a"
[[package]] [[package]]
name = "async-trait" name = "async-trait"
version = "0.1.91" version = "0.1.92"
source = "registry+https://github.com/rust-lang/crates.io-index" source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "ae36dc4177970ef04fde5178d3e2429882def40e57a451f919c098f72baa6cec" checksum = "82f6aeea286b8eb4dd3431a1be1b59d290ace00f5bfd8e2a159bc2a05e2c1667"
dependencies = [ dependencies = [
"proc-macro2", "proc-macro2",
"quote", "quote",
@@ -794,7 +794,7 @@ dependencies = [
"linked-hash-map", "linked-hash-map",
"linked_hash_set", "linked_hash_set",
"miette", "miette",
"nonempty", "nonempty 0.12.0",
"ref-cast", "ref-cast",
"regex", "regex",
"rustc-literal-escaper", "rustc-literal-escaper",
@@ -1060,6 +1060,12 @@ dependencies = [
"cfg-if", "cfg-if",
] ]
[[package]]
name = "crossbeam-utils"
version = "0.8.23"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "a31eee39dddec8330830986fcd7625edb5a24ec90ea038215273bbc3adb08ac6"
[[package]] [[package]]
name = "crypto-bigint" name = "crypto-bigint"
version = "0.5.5" version = "0.5.5"
@@ -1161,6 +1167,20 @@ dependencies = [
"syn 2.0.119", "syn 2.0.119",
] ]
[[package]]
name = "dashmap"
version = "6.2.1"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "e6361d5c062261c78a176addb82d4c821ae42bed6089de0e12603cd25de2059c"
dependencies = [
"cfg-if",
"crossbeam-utils",
"hashbrown 0.14.5",
"lock_api",
"once_cell",
"parking_lot_core",
]
[[package]] [[package]]
name = "data-encoding" name = "data-encoding"
version = "2.11.0" version = "2.11.0"
@@ -1517,6 +1537,16 @@ dependencies = [
"percent-encoding", "percent-encoding",
] ]
[[package]]
name = "forwarded-header-value"
version = "0.1.1"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "8835f84f38484cc86f110a805655697908257fb9a7af005234060891557198e9"
dependencies = [
"nonempty 0.7.0",
"thiserror 1.0.69",
]
[[package]] [[package]]
name = "fs_extra" name = "fs_extra"
version = "1.3.0" version = "1.3.0"
@@ -1594,6 +1624,12 @@ version = "0.3.33"
source = "registry+https://github.com/rust-lang/crates.io-index" source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "b231ed28831efb4a61a08580c4bc233ec56bc009f4cd8f52da2c3cb97df0c109" checksum = "b231ed28831efb4a61a08580c4bc233ec56bc009f4cd8f52da2c3cb97df0c109"
[[package]]
name = "futures-timer"
version = "3.0.4"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "af43fadb8a98512d547e37b4e92e0ced13e205c061b87b4623eff01d918d6968"
[[package]] [[package]]
name = "futures-util" name = "futures-util"
version = "0.3.33" version = "0.3.33"
@@ -1642,9 +1678,11 @@ source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "899def5c37c4fd7b2664648c28120ecec138e4d395b459e5ca34f9cce2dd77fd" checksum = "899def5c37c4fd7b2664648c28120ecec138e4d395b459e5ca34f9cce2dd77fd"
dependencies = [ dependencies = [
"cfg-if", "cfg-if",
"js-sys",
"libc", "libc",
"r-efi 5.3.0", "r-efi 5.3.0",
"wasip2", "wasip2",
"wasm-bindgen",
] ]
[[package]] [[package]]
@@ -1695,6 +1733,29 @@ dependencies = [
"web-sys", "web-sys",
] ]
[[package]]
name = "governor"
version = "0.10.4"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "9efcab3c1958580ff1f25a2a41be1668f7603d849bb63af523b208a3cc1223b8"
dependencies = [
"cfg-if",
"dashmap",
"futures-sink",
"futures-timer",
"futures-util",
"getrandom 0.3.4",
"hashbrown 0.16.1",
"nonzero_ext",
"parking_lot",
"portable-atomic",
"quanta",
"rand 0.9.5",
"smallvec",
"spinning_top",
"web-time",
]
[[package]] [[package]]
name = "group" name = "group"
version = "0.13.0" version = "0.13.0"
@@ -1756,6 +1817,12 @@ version = "0.12.3"
source = "registry+https://github.com/rust-lang/crates.io-index" source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "8a9ee70c43aaf417c914396645a0fa852624801b24ebb7ae78fe8272889ac888" checksum = "8a9ee70c43aaf417c914396645a0fa852624801b24ebb7ae78fe8272889ac888"
[[package]]
name = "hashbrown"
version = "0.14.5"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "e5274423e17b7c9fc20b6e7e208532f9b19825d82dfd615708b70edd83df41f1"
[[package]] [[package]]
name = "hashbrown" name = "hashbrown"
version = "0.16.1" version = "0.16.1"
@@ -1993,6 +2060,19 @@ dependencies = [
"webpki-roots", "webpki-roots",
] ]
[[package]]
name = "hyper-timeout"
version = "0.5.2"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "2b90d566bffbce6a75bd8b09a05aa8c2cb1fabb6cb348f8840c9e4c90a0d83b0"
dependencies = [
"hyper 1.11.0",
"hyper-util",
"pin-project-lite",
"tokio",
"tower-service",
]
[[package]] [[package]]
name = "hyper-util" name = "hyper-util"
version = "0.1.20" version = "0.1.20"
@@ -2887,6 +2967,12 @@ dependencies = [
"signatory", "signatory",
] ]
[[package]]
name = "nonempty"
version = "0.7.0"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "e9e591e719385e6ebaeb5ce5d3887f7d5676fceca6411d1925ccc95745f3d6f7"
[[package]] [[package]]
name = "nonempty" name = "nonempty"
version = "0.12.0" version = "0.12.0"
@@ -2896,6 +2982,12 @@ dependencies = [
"serde", "serde",
] ]
[[package]]
name = "nonzero_ext"
version = "0.3.0"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "38bf9645c8b145698bb0b18a4637dcacbc421ea49bef2317e4fd8065a387cf21"
[[package]] [[package]]
name = "nu-ansi-term" name = "nu-ansi-term"
version = "0.50.3" version = "0.50.3"
@@ -3265,11 +3357,12 @@ dependencies = [
[[package]] [[package]]
name = "portal" name = "portal"
version = "0.4.0" version = "0.5.0"
dependencies = [ dependencies = [
"anyhow", "anyhow",
"arc-swap", "arc-swap",
"async-nats", "async-nats",
"async-trait",
"aws-sdk-s3", "aws-sdk-s3",
"axum", "axum",
"base64 0.22.1", "base64 0.22.1",
@@ -3297,6 +3390,7 @@ dependencies = [
"tower", "tower",
"tower-http", "tower-http",
"tower-sessions", "tower-sessions",
"tower_governor",
"tracing", "tracing",
"tracing-subscriber", "tracing-subscriber",
"url", "url",
@@ -3468,6 +3562,21 @@ version = "0.11.0"
source = "registry+https://github.com/rust-lang/crates.io-index" source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "007d8adb5ddab6f8e3f491ac63566a7d5002cc7ed73901f72057943fa71ae1ae" checksum = "007d8adb5ddab6f8e3f491ac63566a7d5002cc7ed73901f72057943fa71ae1ae"
[[package]]
name = "quanta"
version = "0.12.6"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "f3ab5a9d756f0d97bdc89019bd2e4ea098cf9cde50ee7564dde6b81ccc8f06c7"
dependencies = [
"crossbeam-utils",
"libc",
"once_cell",
"raw-cpuid",
"wasi",
"web-sys",
"winapi",
]
[[package]] [[package]]
name = "quick-error" name = "quick-error"
version = "1.2.3" version = "1.2.3"
@@ -3667,6 +3776,15 @@ dependencies = [
"rand_core 0.9.5", "rand_core 0.9.5",
] ]
[[package]]
name = "raw-cpuid"
version = "11.6.0"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "498cd0dc59d73224351ee52a95fee0f1a617a2eae0e7d9d720cc622c73a54186"
dependencies = [
"bitflags 2.13.1",
]
[[package]] [[package]]
name = "reactive_graph" name = "reactive_graph"
version = "0.2.14" version = "0.2.14"
@@ -4585,6 +4703,15 @@ version = "0.10.1"
source = "registry+https://github.com/rust-lang/crates.io-index" source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "023a211cb3138dbc438680b32560ad89f699977624c9f8dbb95a47d5b4c07dd3" checksum = "023a211cb3138dbc438680b32560ad89f699977624c9f8dbb95a47d5b4c07dd3"
[[package]]
name = "spinning_top"
version = "0.3.0"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "d96d2d1d716fb500937168cc09353ffdc7a012be8475ac7308e1bdf0e3923300"
dependencies = [
"lock_api",
]
[[package]] [[package]]
name = "spki" name = "spki"
version = "0.7.3" version = "0.7.3"
@@ -4906,6 +5033,17 @@ dependencies = [
"tokio", "tokio",
] ]
[[package]]
name = "tokio-stream"
version = "0.1.19"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "a3d06f0b082ba57c26b79407372e57cf2a1e28124f78e9479fe80322cf53420b"
dependencies = [
"futures-core",
"pin-project-lite",
"tokio",
]
[[package]] [[package]]
name = "tokio-tungstenite" name = "tokio-tungstenite"
version = "0.29.0" version = "0.29.0"
@@ -4984,6 +5122,35 @@ dependencies = [
"winnow", "winnow",
] ]
[[package]]
name = "tonic"
version = "0.14.6"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "ac2a5518c70fa84342385732db33fb3f44bc4cc748936eb5833d2df34d6445ef"
dependencies = [
"async-trait",
"axum",
"base64 0.22.1",
"bytes",
"h2 0.4.15",
"http 1.4.2",
"http-body 1.1.0",
"http-body-util",
"hyper 1.11.0",
"hyper-timeout",
"hyper-util",
"percent-encoding",
"pin-project",
"socket2 0.6.5",
"sync_wrapper",
"tokio",
"tokio-stream",
"tower",
"tower-layer",
"tower-service",
"tracing",
]
[[package]] [[package]]
name = "tower" name = "tower"
version = "0.5.3" version = "0.5.3"
@@ -4992,9 +5159,12 @@ checksum = "ebe5ef63511595f1344e2d5cfa636d973292adc0eec1f0ad45fae9f0851ab1d4"
dependencies = [ dependencies = [
"futures-core", "futures-core",
"futures-util", "futures-util",
"indexmap 2.14.0",
"pin-project-lite", "pin-project-lite",
"slab",
"sync_wrapper", "sync_wrapper",
"tokio", "tokio",
"tokio-util",
"tower-layer", "tower-layer",
"tower-service", "tower-service",
"tracing", "tracing",
@@ -5106,6 +5276,23 @@ dependencies = [
"tower-sessions-core", "tower-sessions-core",
] ]
[[package]]
name = "tower_governor"
version = "0.8.0"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "44de9b94d849d3c46e06a883d72d408c2de6403367b39df2b1c9d9e7b6736fe6"
dependencies = [
"axum",
"forwarded-header-value",
"governor",
"http 1.4.2",
"pin-project",
"thiserror 2.0.19",
"tonic",
"tower",
"tracing",
]
[[package]] [[package]]
name = "tracing" name = "tracing"
version = "0.1.44" version = "0.1.44"
@@ -5528,6 +5715,22 @@ dependencies = [
"rustls-pki-types", "rustls-pki-types",
] ]
[[package]]
name = "winapi"
version = "0.3.9"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "5c839a674fcd7a98952e593242ea400abe93992746761e38641405d28b00f419"
dependencies = [
"winapi-i686-pc-windows-gnu",
"winapi-x86_64-pc-windows-gnu",
]
[[package]]
name = "winapi-i686-pc-windows-gnu"
version = "0.4.0"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "ac3b87c63620426dd9b991e5ce0329eff545bccbbb34f3be09ff6fb6ab51b7b6"
[[package]] [[package]]
name = "winapi-util" name = "winapi-util"
version = "0.1.11" version = "0.1.11"
@@ -5537,6 +5740,12 @@ dependencies = [
"windows-sys 0.61.2", "windows-sys 0.61.2",
] ]
[[package]]
name = "winapi-x86_64-pc-windows-gnu"
version = "0.4.0"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "712e227841d057c1ee1cd2fb22fa7e5a5461ae8e48fa2ca79ec42cfc1931183f"
[[package]] [[package]]
name = "windows-core" name = "windows-core"
version = "0.62.2" version = "0.62.2"
+5 -1
View File
@@ -1,6 +1,6 @@
[package] [package]
name = "portal" name = "portal"
version = "0.4.0" version = "0.5.0"
edition = "2021" edition = "2021"
[lib] [lib]
@@ -63,6 +63,8 @@ web-sys = { version = "0.3", features = [
"Response", "Response",
"Window", "Window",
], optional = true } ], optional = true }
tower_governor = { version = "0.8.0", optional = true }
async-trait = { version = "0.1.92", optional = true }
[features] [features]
default = [] default = []
@@ -81,6 +83,8 @@ ssr = [
"dep:leptos_axum", "dep:leptos_axum",
"dep:axum", "dep:axum",
"dep:aws-sdk-s3", "dep:aws-sdk-s3",
"dep:tower_governor",
"dep:async-trait",
"dep:tokio", "dep:tokio",
"dep:tower", "dep:tower",
"dep:tower-http", "dep:tower-http",
+31 -170
View File
@@ -1,190 +1,51 @@
# portal # portal
A content-driven question engine: one Rust binary that turns a Git A content-driven question engine: one Rust binary that turns a Git
repo of YAML into a live site — pages, forms, review desks, and state repo of YAML into a live site — pages, forms, review desks and state
machines, with a durable event-sourced record of every answer machines, with a durable event-sourced record of every answer
underneath. The engine special-cases nothing: everything a site underneath.
serves is declared in its content repo, and the same build serves any
number of sites.
Point an instance at a content repo and that repo *is* the site: The engine special-cases nothing. Everything a site serves is declared
pages, copy, state graphs, branding, and even the landing hero in its content repo, content pushes hot-reload every running instance,
(`site.yaml`: title, wordmark, and `hero: {kind: module, module: and the same build serves any number of sites: instances differ only
hero.js}` — a JavaScript module the content repo ships, served by environment variables. [uhhm.no](https://uhhm.no) and
same-origin at `/site/<path>`, that portal mounts at parse time and [redoal.com](https://redoal.com) are two faces of the same binary from
stops on navigation; the YES canvas and redoal's sine swings are both the same release artifact.
content, not engine). Content
pushes hot-reload every running instance; instances differ only by
env vars. [uhhm.no](https://uhhm.no)
([uhhm/questions](https://project.uhhm.no/uhhm/questions)) and
[redoal.com](https://redoal.com)
([redoal/questions](https://project.uhhm.no/redoal/questions)) are
two faces of the same binary, deployed from the same release
artifact.
It composes with the surrounding stack rather than bundling it: Gitea It composes with the surrounding stack rather than bundling it. Gitea
hosts and serves the content, NATS JetStream stores events and hosts the content, NATS JetStream stores the events and projections,
projections, Kanidm provides identity, and anything downstream Kanidm provides identity, and anything downstream — automations,
(automations, newsletters, onboarding) subscribes to the answer newsletters, onboarding — subscribes to the answer stream.
stream.
## What the engine provides ## Documentation
- **Content-driven pages** (`src/content/`, `src/app.rs`): YAML Everything is under [`docs/`](docs/README.md):
loaded from Gitea at boot and hot-swapped on a NATS reload signal;
a bad push keeps the last-good content serving. The `questions/`
tree IS the router — file paths become URLs, `_section.yaml`
applies criteria to a whole directory, `[name].yaml` pages serve
any `/dir/<value>` with the segment fed into resource keys, and
`requires_chain` gates a page on verifiable answer provenance next
to `qualifies`' Kanidm-group identity gate (see
`docs/design/filesystem-routes.md`).
- **Gesture and voice fields** (`gesture.js`, `voice.js`): a stroke
becomes a redoal gesture key through a relay (ADR-0013/0015 in the
redoal repo) - the input shows the key's own decode under the
stroke, who is at a similar shape now, and *places* with recordings
the visitor can pick to make that key theirs; a voice field records
in the browser and leaves the audio at a key. Actions may be
templated from the answer (`/shape/{curve.key}`) onto dynamic
pages, whose URL segment also fills `value: "{key}"` presets and
`url` resource sources; resource items with an https `audio` field
render a player.
- **Announced pages** (`src/announce.rs`): a question with an
`event: {starts, duration, place}` window is announced in the
header (not the footer nav) while the window is open, becomes a
followup when it closes, and its record in the runtime-owned
`portal_events` bucket moves to `awaiting_summary` - a "post what
happened" task any review desk can read, published on NATS like a
decision. A one-minute sweeper keeps it all idempotent.
- **State machines as content** (`src/aggregates/`): `aggregates.yaml`
declares each bucket's states and legal transitions; the engine
replays a record's event history and refuses undeclared moves, with
optimistic concurrency (CAS on the event log's sequence) against
racing decisions. An empty history reseeds from the KV projection,
so wiping the event stream strands nothing.
- **Durable events + projections** (`src/events/`, `src/answers.rs`):
every submission and decision appends to a JetStream event log and
projects into a NATS KV bucket pages read back; every one also
publishes on `portal.answers.submitted` for automations (n8n) to
react to.
- **Answer chains** (`src/chain.rs`): each submission hashes its
parent(s), so a visitor's path through the questions is a verifiable
lineage; `?chain=` links carry it, and self-service transitions
(unsubscribe) authorize by holding one. The question nav reads it
too: a `followup` page is offered only once the visitor has earned
it - the chain verifiably answered the alternative that leads there,
or it hangs off an earned page by a link that records nothing. A
page behind an answer they have not given is never offered.
- **One access policy** (`src/access.rs`): who may do what to what is
compiled from content on every load into a Cedar policy set - `qualifies`
on a page, `requires_group`/`public` on a resource, a desk's transitions,
a `self_transition` by link and email, an `invite` - and every server
entry point asks that one policy. Decisions carry the rule ids that
granted them; refusals and every mutation are published on
`portal.access.decided`, and other apps ask the same question over
request-reply on `portal.access.<site>.may`. Identity stays Kanidm's.
- **Onboarding as a form** (`src/people.rs`): an alternative with
`invite: {group}` on a gated page creates the person in Kanidm, records
their email, adds them to the group and mints a one-time link to set
their own credentials, through a service account in
`idm_people_on_boarding` (`KANIDM_API_TOKEN`). The result lands in the
`portal_invites` bucket for the inviter's desk and on
`portal.people.invited` for whatever sends the mail.
- **Business needs as a checkable spec** (in
[uhhm/iris](https://project.uhhm.no/uhhm/iris)): a content repo may
ship a `needs.yaml` saying who wants what, which bucket the answer
must land in, and which group has to carry it to a finished state.
`iris check` proves the site meets it. The runtime never reads the
file; portal only provides the content types iris reads.
- **Live resources** (`src/resource.rs`): content can pull a KV
bucket, Gitea starred/org repos/releases, or any public JSON URL
(SSRF fail-closed), reshaped by a content-declared `jq` filter.
- **Input kinds as content**: a requirement's `type:` picks the
widget — plain fields, a rich-text editor, or a `gesture` drawing
canvas that can connect to a redoal relay so similar strokes echo
between visitors live. Submitted values are arbitrary JSON; the
engine records what the widget produced.
- **Review desks**: any Kv resource with `transitions` renders rows
with per-state action buttons and one shared confirm per
alternative — owners walk records through their graphs without
bespoke UI per bucket.
## Binaries - [Architecture](docs/architecture.md) — what the engine is made of.
- [Content reference](docs/content-reference.md) — every key a content
- `portal` — the server (Leptos SSR + hydrate, Axum underneath). repo may write.
- `iris` — the lint, from [uhhm/iris](https://project.uhhm.no/uhhm/iris), - [State machines](docs/state-machines.md) — `aggregates.yaml`.
which depends on this crate as a library pinned to a release tag and - [Access](docs/access.md) — the policy compiled from content.
carries the same version: the publish job here re-pins, tests and tags - [Operations](docs/operations.md) — releasing, deploying, adding a site.
iris for every portal release.
Every release tarball ships that `iris` binary
too, so each content repo's CI can
lint with the exact portal version its instance runs:
`iris check --path <dir>`.
## Development ## Development
```sh ```sh
cargo leptos build # full app (server + wasm) cargo leptos build # server + wasm
cargo test --features ssr # engine tests; tests/jetstream.rs needs PORTAL_TEST_NATS_URL cargo test --features ssr # tests/jetstream.rs needs PORTAL_TEST_NATS_URL
``` ```
Runtime configuration is env vars (see `src/main.rs`, and each Content is linted by [iris](https://project.uhhm.no/uhhm/iris), which
content repo's `.gitea/workflows/deploy.yml` for the values in depends on this crate and carries the same version, so a content repo
production): `NATS_URL`, `CONTENT_REPO`/`CONTENT_BRANCH`, Kanidm OIDC can check itself against the exact portal its instance runs:
(`KANIDM_URL`, `OAUTH2_CLIENT_*`), optional `GARAGE_*` for uploads,
`GITEA_API_TOKEN` for authenticated resource pulls,
`AUTOMATION_READ_TOKEN` for the automation KV read endpoint.
## Release and deploy
Portal doesn't deploy itself — it publishes versions, and each site
decides when to take one. The whole day-to-day surface is two
commands:
**Cut a release** (here):
```sh ```sh
cargo release patch # or minor / major iris check --path questions
``` ```
That bumps `Cargo.toml`, tags `v<version>`, and pushes; CI ## Releasing
(`.gitea/workflows/publish.yml`) reacts to the tag, builds once, and
attaches `portal-v<version>.tar.gz` (`portal` + `iris` +
`hash.txt` + `site/`) to the Gitea release. Pkg files are
content-hashed (`hash-files = true`; instances run with
`LEPTOS_HASH_FILES=true`), so a freshly served page can never pair a
stale cached bundle with new wasm. Plain pushes to `main` only run the
tests (`test.yml`) — nothing reaches production from this repo.
Before cutting, move the `Unreleased` lines in `CHANGELOG.md` under a `cargo release patch` tags and pushes; CI builds the tarball and
heading for the new version. That section becomes the Gitea release attaches it to the Gitea release. A site takes it by bumping
notes, and the table at the end of the file records which version each `PORTAL_RELEASE` in its own content repo. See
site runs; update it when a pin moves. [operations](docs/operations.md).
**Roll it out** (in a content repo): edit one line in that repo's
`.gitea/workflows/deploy.yml` —
```yaml
env:
PORTAL_RELEASE: v0.1.1 # <- bump, commit, push
```
Its CI downloads the pinned artifact, ships
`/srv/app/<instance>/releases/<tag>`, flips `current`, rewrites the
instance env from that repo's own Actions variables/secrets, restarts
`app@<instance>`, and refreshes the Caddy route. Every rollout is a
commit, so reverting a bad version is `git revert` + push. Sites
upgrade independently: uhhm.no (uhhm/questions → `app@uhhm-portal`,
:3010) and redoal.com (redoal/questions → `app@redoal-portal`,
:3020) can pin different versions.
Content changes never come through any of this — they hot-reload
live from the content repos over NATS.
**Add a site**: new content repo with a copy of an existing
`deploy.yml` (change `INSTANCE`, port, domains), Actions variables
(`KANIDM_URL`, `OAUTH2_CLIENT_ID`, `PUBLIC_URL`, `SITE_NAME`) and
secrets (`NATS_URL`, `OAUTH2_CLIENT_SECRET`,
`PORTAL_GITEA_API_TOKEN` — Gitea reserves the `GITEA_` prefix for
secret names), a Kanidm OAuth2 client, and DNS. `site.yaml` in the
content repo handles all branding; no portal changes needed.
+22
View File
@@ -0,0 +1,22 @@
# portal documentation
Portal turns a Git repo of YAML into a running site. Almost everything
here is about that YAML: what a content repo may say, and what the
engine does with it.
| | |
|---|---|
| [Architecture](architecture.md) | What the engine is made of and where each piece lives in the source. |
| [Content reference](content-reference.md) | Every key a content repo may write, and what it does. |
| [State machines](state-machines.md) | `aggregates.yaml`: stages, who moves a case, mail, deadlines, memberships. |
| [Access](access.md) | How the Cedar policy is compiled from content, and who may ask it what. |
| [Operations](operations.md) | Releasing, deploying, adding a site, environment variables. |
| [Filesystem routes](design/filesystem-routes.md) | Why the `questions/` tree is the router. |
Two repos in the same family carry their own docs:
- [uhhm/iris](https://project.uhhm.no/uhhm/iris) — the lint. Versioned
as portal and released by portal's publish job; it is the thing a
content repo's CI runs before anything reaches a site.
- [uhhm/gdo](https://project.uhhm.no/uhhm/gdo) — the mailer. Delivers
what content asked to be sent, and carries replies back.
+55
View File
@@ -0,0 +1,55 @@
# Access
Who may do what to what is compiled from content on every load into
one Cedar policy set, and every server entry point asks that one
policy. Identity stays Kanidm's; portal only decides.
## What content compiles to
| content | rule |
|---|---|
| a page with no `qualifies` | anyone may open and submit |
| `qualifies: case_officers` | that group may open and submit |
| `requires_chain: /start` | the visitor's verified answer lineage must reach it |
| a resource with `public: true` | anyone may read it |
| a resource with `requires_group: g` | that group may read it |
| a desk resource's `transitions` | that group may make those moves |
| an alternative's `self_transition` | the submitter may make it, from each state in `from`, with their link and their matching email |
| an alternative's `invite: {group}` | the page's own group may invite into that group |
Mutations always need a group, whatever `public` says. A resource with
neither `public` nor `requires_group` is unreachable rather than open:
the policy fails closed.
## Reading the policy
```sh
iris check --path questions --access
```
prints every rule the content compiled to, as a table of who may do
what from where to where. It is the fastest way to find a rule a
content repo did not mean to write — a refusal available on the day a
case arrives, a board that can decide a case it never received.
## Asking it
Every decision carries the rule ids that granted it. Refusals and
every mutation are published on `portal.access.decided`.
Other applications ask the same question over NATS request-reply on
`portal.access.<site>.may`, with the same principal, action, resource
and context the server uses internally — so an automation acting as a
group gets exactly the answer a person in that group would.
## The submitter
A person who sent a form has no account. Their credential is the link:
an unguessable chain hash, plus the email they gave, which the policy
compares against the one stored on the record.
What they may do, and from where, is `self_transition.from`. Its
default is the initial state alone. A refusal reads "not found" while
the record is somewhere the move could have been made from — so the
endpoint cannot be used to discover which records or addresses exist —
and "already processed" once the case has moved past it.
+92
View File
@@ -0,0 +1,92 @@
# Architecture
One Rust binary (Leptos SSR + hydrate, Axum underneath) serving any
number of sites. The engine special-cases nothing: every page, form,
desk and state graph a site serves is declared in its content repo,
and instances differ only by environment variables.
It composes with the surrounding stack rather than bundling it. Gitea
hosts the content, NATS JetStream stores the events and projections,
Kanidm provides identity, and anything downstream subscribes to the
answer stream.
## The pieces
**Content** (`src/content/`) — YAML loaded from Gitea at boot and
hot-swapped on a NATS reload signal. A bad push keeps the last-good
content serving. The `questions/` tree is the router: file paths
become URLs, `_section.yaml` applies to a whole directory, and
`[name].yaml` serves any `/dir/<value>` with the segment fed into the
page's resource keys. See [filesystem routes](design/filesystem-routes.md).
**The lint** (`src/content/validate.rs`) — every rule a content repo
is held to. It runs at boot, on every hot reload, and in a content
repo's CI through [iris](https://project.uhhm.no/uhhm/iris), which
depends on this crate and so applies exactly the version the instance
runs.
**State machines** (`src/aggregates/`) — `aggregates.yaml` declares
each bucket's states and legal moves. A record's state is replayed
from its own event history, undeclared moves are refused, and racing
decisions are settled by CAS on the log's sequence. An empty history
reseeds from the KV projection, so wiping the event stream strands
nothing. See [state machines](state-machines.md).
**Events and projections** (`src/events/`, `src/answers.rs`) — every
submission and decision appends to a JetStream log and projects into
a NATS KV bucket that pages read back, and publishes on
`portal.answers.submitted` for automations to react to.
**Answer chains** (`src/chain.rs`) — each submission hashes its
parents, so a visitor's path through the questions is a verifiable
lineage. `?chain=` links carry it; a `followup` page is offered only
once the chain has earned it, and a page behind an answer somebody has
not given is never offered.
**Access** (`src/access.rs`) — who may do what to what, compiled from
content on every load into one Cedar policy set that every server
entry point asks. See [access](access.md).
**People** (`src/people.rs`) — onboarding as a form. An alternative
with `invite: {group}` creates the person in Kanidm, records their
email, adds them to the group and mints a one-time credential link. A
state may also grant a group on its own (`grants:`), so an approval
that means membership needs no second step.
**Mail** (`src/mail.rs`) — a state may declare the mail its arrival
sends. Portal renders it and publishes on `portal.mail.send`;
[gdo](https://project.uhhm.no/uhhm/gdo) delivers it and carries
replies back on `portal.mail.received`, where they become notes beside
the record. A note never moves a case.
**Sweepers** (`src/announce.rs`, `src/deadlines.rs`) — the two things
that move without a person. An announced page's window closes and its
record becomes a "post what happened" task; a state with a deadline
waits out its days and then makes the move itself. Both go through the
same transition path as a desk button and publish the same event.
**Resources** (`src/resource.rs`) — content can read a KV bucket,
Gitea starred repos, org repos or releases, or any public JSON URL
(SSRF fail-closed), reshaped by a content-declared `jq` filter. What a
resource reads is never a client-supplied parameter: the client names
a page and a feature, never a bucket or URL.
**Maps** (`src/maps.rs`) — MapLibre is vendored and every tile, style
and still image is proxied through portal, so a page with a map makes
no third-party request and needs no consent banner.
**Desks** — any Kv resource with `transitions` renders rows with
per-state buttons and one shared confirm. Owners walk records through
their graphs with no bespoke UI per bucket.
## Binaries
- `portal` — the server.
- `iris` — the lint, from its own repo, shipped in every release
tarball so a content repo's CI can lint with the exact version its
instance runs.
**Sessions** (`src/sessions.rs`) — in the same JetStream KV as
everything else, so a restart does not sign anybody out. The bucket
carries a `max_age` so NATS collects what nobody comes back for, and
each record's own `expiry_date` is what decides on the way out.
+232
View File
@@ -0,0 +1,232 @@
# Content reference
A content repo is the site. This is everything it may say.
```
site.yaml branding, language, mail
aggregates.yaml the state machine of every bucket
needs.yaml what the site is for (read by iris, never by portal)
questions/
index.yaml the front page, served at /
about.yaml served at /about
review/
_section.yaml applies to everything under /review
index.yaml served at /review
[record].yaml serves /review/<anything>
```
Nothing here is required except a `questions/` tree with something in
it. Portal's lint refuses content it cannot serve honestly, at boot,
on every hot reload, and in CI through `iris check`.
## site.yaml
```yaml
title: Vestmark kommune # the browser title and the header
wordmark: /site/logo.svg # an image instead of the title
wordmark_invert: true # flip it in dark mode
wordmark_height: 2.4 # rem
lang: no # the interface language: en or no
stylesheet: /site/theme.css # served same-origin from the repo
favicon: /site/icon.svg
hero: { kind: module, module: hero.js } # or { kind: plain }
mail:
from: Vestmark kommune <post@vestmark.example>
reply_to: post@vestmark.example
invite: { subject: "...", body: "..." } # rewords the built-in invite mail
```
`mail.from` is required as soon as any state declares a mail.
## A question
One YAML file is one page.
```yaml
id: /apply # defaults to the file's path
name: Apply to build
description: What we need from you.
qualifies: case_officers # a Kanidm group may open this page
requires_chain: /start # or: an answer they must already have given
followup: true # keep it out of the nav until earned
responsible: { name: Solveig, email: post@vestmark.example }
event: # this page announces an occasion
starts: 2026-10-14T18:00
duration: 2h
place: The town hall
place: # this page is about somewhere
lat: 59.8938
lon: 11.0332
zoom: 15
name: Tomter
alternatives: [ ... ]
```
`qualifies`, `requires_chain` and `responsible` are inherited from the
nearest `_section.yaml`. `event` makes a page an occasion with a time;
`place` makes it a place, drawing a still map under the question and
opening any `type: location` field on the page there rather than on
the whole world.
## An alternative
One choice. Never two actions behind one heading: if two things can be
done, they are two alternatives.
```yaml
alternatives:
- name: Send the drawings
description: We answer within twelve weeks.
action: /sent # the page to go to; /shape/{field} templates
record_as: applications # the bucket the answer lands in
disabled: false
consequence: [What happens next]
encouragements: [It takes two minutes]
images: [/site/drawing.png]
features: [ ... ]
```
Three things an alternative may do instead of, or beside, recording an
answer:
```yaml
self_transition: # the person moves their own record
bucket: applications
to: complete
from: [open, incomplete] # defaults to the initial state alone
label: Send them
invite: { group: case_officers } # onboard someone into Kanidm
```
`self_transition.from` is what keeps a person in dialogue with their
case: an applicant written to for a missing drawing answers from
`incomplete`, and may withdraw from wherever the case has got to.
Omitted, it is the initial state alone, which is what a one-shot
unsubscribe link means. Each state listed becomes its own access rule,
and the lint refuses a `from -> to` the state machine does not declare.
An `invite` needs a required `name` and `email` on the same
alternative, and the inviting group must be allowed to invite that
group by the policy.
## A feature
What an alternative offers, explained. A feature that is only layout
is a feature wasted: each one says something true about the choice.
```yaml
features:
- name: What you need
description: The case number from the letter.
icon: document
color: "#8ec2c0"
requirements: [ ... ] # form fields
resource: { ... } # live data
```
## A requirement (a form field)
```yaml
requirements:
- name: email # snake_case; this is the stored key
label: Your email
placeholder: you@example.no
type: email
optional: false
value: "{key}" # a preset; a dynamic page's segment fills it
```
`type` is one of:
| type | what it is |
|---|---|
| `text` | the default |
| `email`, `tel`, `date`, `number`, … | anything else is passed straight through as the HTML input type |
| `textarea` | several lines |
| `prosekit` | a rich-text editor |
| `select` | one choice, or `multiple: true` for any number |
| `file` | an upload, with `accept:` and `multiple:` |
| `location` | a pin on a map, stored as `{lat, lon, zoom}` |
| `record` | the record this one belongs to |
| `gesture`, `voice` | a drawn stroke or a recording, through a redoal relay |
| `hidden` | a carrier value, no label row |
| `module` | a widget the content repo ships |
A `select` takes either inline `options: [...]` or a `resource:` to
draw them from, with `id_field:` naming each item's stable id.
A field may load its value from a resource when a sibling changes:
```yaml
- name: contents
type: textarea
bind:
field: path # the sibling to watch
resource: { source: { kind: url, url: "https://git.example/raw/{path}" } }
```
### type: record
How one case belongs to another — an objection to the application it
objects to, an appeal to the decision it appeals.
```yaml
- name: case
type: record
of: applications # the bucket the other record is in
value: "{key}" # arrives on the link, on a dynamic page
```
Nobody types one of these: the stored value is the parent's record id,
which is a chain hash. It arrives the way it was given out, so the
field carries rather than asks, and renders hidden. A submission
naming a record that is not there is refused, so a reference in a
stored record is one that resolved at least once.
The other half is a listing that asks for the children:
```yaml
resource:
source: { kind: kv, bucket: objections }
requires_group: case_officers
where: { field: case, equals: "{key}" }
```
## A resource
Live data, declared by content and read by the server. What a resource
reads is never a client-supplied parameter.
```yaml
resource:
source: { kind: kv, bucket: applications }
key: abc123 # one record instead of the list
public: true # or requires_group: case_officers
states: [approved, refused] # only these; empty means all
where: { field: case, equals: "{key}" }
jq: ".[] | {name, url: .html_url}"
empty: Nothing decided yet.
transitions: # the desk's buttons
- { from: complete, to: on_hearing, label: Send it out }
```
`source.kind` is one of `kv`, `gitea_starred`,
`gitea_releases`, `url`.
A spec with neither `public: true` nor `requires_group` is
unreachable: reads fail closed, and mutations always need a group
whatever `public` says.
`states` and `where` narrow a listing. They matter most for a register
that is public by law — public about what was decided, not about what
was withdrawn — while the desk working the same bucket is a different
spec and still sees all of it. A record that has no such field never
matches: "everything that did not say" is not something a page can ask
for by accident.
## needs.yaml
What the site is for, as a checkable spec: who wants what, which
bucket the answer must land in, which group carries it to a finished
state. The runtime never reads it. `iris check` proves the site meets
it, and `iris replay` runs cases through the real engine.
+126
View File
@@ -0,0 +1,126 @@
# Operations
Portal does not deploy itself. It publishes versions; each site
decides when to take one.
## Cut a release
```sh
cargo release patch # or minor / major
```
That bumps `Cargo.toml`, tags `v<version>` and pushes. CI
(`.gitea/workflows/publish.yml`) reacts to the tag, builds once, and
attaches `portal-v<version>.tar.gz` (`portal`, `iris`, `hash.txt`,
`site/`) to the Gitea release. Plain pushes to `main` only run the
tests: nothing reaches production from this repo.
The same job re-pins, tests and tags [iris](https://project.uhhm.no/uhhm/iris)
to match, so a portal change to anything the lint reads fails the
release rather than the next content push.
Before cutting, move the `Unreleased` lines in `CHANGELOG.md` under a
heading for the new version. That section becomes the release notes,
and the table at the end of the file records which version each site
runs.
Pkg files are content-hashed (`hash-files = true`, instances run with
`LEPTOS_HASH_FILES=true`), so a freshly served page can never pair a
stale cached bundle with new wasm.
## Roll it out
In the content repo, one line of `.gitea/workflows/deploy.yml`:
```yaml
env:
PORTAL_RELEASE: v0.4.0
```
Its CI downloads the pinned artifact, ships
`/srv/app/<instance>/releases/<tag>`, flips `current`, rewrites the
instance env from that repo's own Actions variables and secrets,
restarts `app@<instance>` and refreshes the Caddy route. Every rollout
is a commit, so reverting a bad version is `git revert` and a push.
Sites upgrade independently and may pin different versions.
Content changes never come through any of this: they hot-reload live
from the content repos over NATS.
## Add a site
A new content repo with a copy of an existing `deploy.yml` (change
`INSTANCE`, the port and the domains), Actions variables
(`KANIDM_URL`, `OAUTH2_CLIENT_ID`, `PUBLIC_URL`, `SITE_NAME`) and
secrets (`NATS_URL`, `OAUTH2_CLIENT_SECRET`, `PORTAL_GITEA_API_TOKEN`
— Gitea reserves the `GITEA_` prefix for secret names), a Kanidm
OAuth2 client, and DNS. `site.yaml` handles all branding; no portal
change is needed.
## Environment
Runtime configuration is environment variables; `src/main.rs` is the
authority and each content repo's `deploy.yml` holds the production
values.
| | |
|---|---|
| `NATS_URL` | events, projections, sessions, content reload, access questions |
| `CONTENT_REPO`, `CONTENT_BRANCH` | the repo that is the site |
| `KANIDM_URL`, `OAUTH2_CLIENT_ID`, `OAUTH2_CLIENT_SECRET` | sign-in |
| `KANIDM_API_TOKEN` | invites and `grants:`; unset, neither happens |
| `PUBLIC_URL`, `SITE_NAME` | what the site calls itself in links and mail |
| `GITEA_API_TOKEN` | authenticated resource pulls |
| `AUTOMATION_READ_TOKEN` | the automation KV read endpoint |
| `GARAGE_*` | uploads |
| `MAPBOX_TOKEN` | map tiles, proxied; unset, maps are off |
| `GARAGE_*` (4) | attachments; see below. Unset, upload fields fail closed |
| `PORTAL_RATE_BURST`, `PORTAL_RATE_SECONDS` | the rate limit; see below |
Mail is delivered by [gdo](https://project.uhhm.no/uhhm/gdo), a
separate unit on the same host reading the same NATS.
## Rate limiting
Every request passes a per-caller-IP limit before a session is loaded
or a handler is entered: a burst of 30, refilled one every two
seconds. A visitor filling in a form never reaches it (a page load
fetches several resources at once); a script does.
| | |
|---|---|
| `PORTAL_RATE_BURST` | how many requests at once, default 30 |
| `PORTAL_RATE_SECONDS` | seconds per refilled request, default 2 |
Either set to `0` turns it off, for an instance behind something that
already does this. The caller is read from `X-Forwarded-For` or
`X-Real-IP`, falling back to the socket — all of these instances are
behind Caddy.
This is a fence, not an accounting system. What keeps uploads honest
is the policy check on the route itself.
## Attachments
A `type: file` field stores an object key in the record. The file
comes back out at:
GET /attachment/{bucket}/{record}/{field}
which reads the key out of that record rather than taking one from the
caller, so a file can only be fetched through a record somebody is
allowed to see, and a leaked key is a link to nothing. Reading it
needs the same policy permission as reading the bucket on a desk.
Uploads need a bucket and a key in the Garage on the host:
```sh
garage bucket create <site>-attachments
garage key create <site>-portal-uploads
garage bucket allow --read --write <site>-attachments --key <site>-portal-uploads
```
then `GARAGE_S3_ENDPOINT`, `GARAGE_ACCESS_KEY`, `GARAGE_SECRET_KEY` and
`GARAGE_UPLOADS_BUCKET` in that instance's env. Unset, an upload field
fails closed rather than appearing to work and storing nowhere.
+151
View File
@@ -0,0 +1,151 @@
# State machines
`aggregates.yaml` declares what a bucket's records may be and how they
move. It is the one file that decides what is possible; the pages only
decide who presses what.
```yaml
aggregates:
- bucket: applications
initial: open
states:
open: { event: received }
incomplete: { event: incomplete }
complete: { event: complete }
on_hearing: { event: on_hearing }
recommended: { event: recommended }
approved: { event: approved }
refused: { event: refused }
withdrawn: { event: withdrawn }
transitions:
open: [incomplete, complete, withdrawn]
incomplete: [complete, withdrawn]
complete: [incomplete, on_hearing, refused, withdrawn]
on_hearing: [incomplete, recommended, refused, withdrawn]
recommended: [approved, refused, withdrawn]
```
`transitions` is an arbitrary directed graph. Cycles are allowed and
are often the honest shape: a case sent back for the drawing it was
missing returns to a stage it was already past. A good ending may sit
partway along — an authority that changes its own decision finishes
the case without walking to the end of the line.
Each state's `event` string is the name written into the log, and must
be unique within the bucket. A record's state is replayed from its own
events, so renaming an event orphans the records that carry the old
name: when rebuilding a live site, keep the names it already has.
## What a state may carry
Beside its `event`, a state may declare three things that happen when
a record arrives in it.
### mail
```yaml
approved:
event: approved
mail:
to: submitter # or one fixed address
subject: "Your application"
body: |
Hello,
{property_address} is approved. The decision is at
{site}/status/{chain}.
```
`{field}` is any field of the record, plus `{site}`, `{chain}`,
`{email}`, `{bucket}` and `{state}`. The body is markdown; it is sent
as text with an HTML alternative. `to: submitter` means the record's
`email` field, and the lint then requires every form filling that
bucket to collect a required `email`.
Portal renders it and publishes on `portal.mail.send`. gdo delivers
it, and a reply comes back on `portal.mail.received` as a note beside
the record. A note never moves a case.
### grants
```yaml
approved:
event: approved
grants: approved_firms
```
The person the record is about becomes a member of that group when the
record arrives here. An approval that *means* membership should not
also need somebody to remember to invite them afterwards.
The lint requires every form filling that bucket to collect a required
`name` and a required `email`, because by the time a desk presses
approve there is nobody left to ask. An instance with no Kanidm
configured grants nothing and says so in the log; the transition
itself always stands.
### after
```yaml
on_hearing:
event: on_hearing
after: { days: 42, to: recommended }
```
The move a clock makes rather than a person. A hearing that gets no
answer in six weeks has to go on without one, and nobody in the
building is in a position to press that button.
A deadline may only do what a person could have done: `after.to` is
held to the same declared edge as a desk button, and content that says
otherwise is refused when it loads, not when the hour comes round. The
wait is measured from the record's last move. The move goes through
the same transition path, fires the same mail, and lands on
`portal.answers.submitted` like any decision, stamped `deadline`.
The unit is days and the sweep is hourly. This is a deadline, not a
job queue.
## attended_by
```yaml
- bucket: newsletter
initial: open
states: { open: { event: received } }
attended_by: n8n newsletter compose
```
Names what consumes a bucket no page reads back. The lint refuses a
bucket that a form fills and nothing attends — a question was once
dropped and its answers silently fell out of every reader's view, and
this is the contract that replaced the convention.
## Who moves what
Nothing in this file says who. A move becomes possible for somebody
when a page offers it:
- a desk resource's `transitions` gives it to that resource's
`requires_group`;
- an alternative's `self_transition` gives it to the person who sent
the record, by their link and their email, from the states its
`from` names;
- a state's `after` gives it to the clock.
All three compile into the same [access policy](access.md), and the
engine refuses anything the graph above does not declare, whoever asks.
## Testing one
`iris replay` runs cases through this exact machinery — the same
`store_answer` and `transition` the running site uses, against a
throwaway JetStream, reporting what the engine accepted and refused.
```sh
docker run -d --rm -p 127.0.0.1:14222:4222 nats:2.14.6-alpine -js
iris replay --path questions --nats nats://127.0.0.1:14222 --cases cases.jsonl
```
Each line of `cases.jsonl` is one case: a need, a record, and the
states it walks. Point it at a throwaway NATS, never a site's own: it
writes.
+51 -10
View File
@@ -221,16 +221,20 @@ impl Policy {
} }
} }
if let Some(st) = &alt.self_transition { if let Some(st) = &alt.self_transition {
let from = crate::answers::OPEN_STATE; // One rule per state the move may be made from: a
emit( // person stays in dialogue with their case for as
format!("self:{}:{from}>{}", st.bucket, st.to), "the submitter, by their link and email".to_string(), "self_transition", // long as it runs, and `from` says how long that is.
format!("bucket {}", st.bucket), format!("{from} -> {}", st.to), for from in &st.from {
format!( emit(
"permit(principal is Submitter, action == Action::\"self_transition\", resource == Bucket::{}) when {{ context.from == {} && context.to == {} && principal.email == context.stored_email }};", format!("self:{}:{from}>{}", st.bucket, st.to), "the submitter, by their link and email".to_string(), "self_transition",
quote(&st.bucket), quote(from), quote(&st.to) format!("bucket {}", st.bucket), format!("{from} -> {}", st.to),
), format!(
&mut text, &mut rules, "permit(principal is Submitter, action == Action::\"self_transition\", resource == Bucket::{}) when {{ context.from == {} && context.to == {} && principal.email == context.stored_email }};",
); quote(&st.bucket), quote(from), quote(&st.to)
),
&mut text, &mut rules,
);
}
} }
if let (Some(invite), Some(g)) = (&alt.invite, &q.qualifies) { if let (Some(invite), Some(g)) = (&alt.invite, &q.qualifies) {
emit( emit(
@@ -536,6 +540,43 @@ mod tests {
assert!(!p.decide(&user(&["owners"]), Action::SelfTransition, &bucket, ctx("a@b.c")).allow); assert!(!p.decide(&user(&["owners"]), Action::SelfTransition, &bucket, ctx("a@b.c")).allow);
} }
// The applicant written to for a missing drawing sends it from
// `incomplete`, and may withdraw at any point up to the signature.
// Before `from`, every one of those moves was compiled as if it
// could only be made on the day the case arrived.
#[test]
fn a_submitter_stays_in_dialogue_for_as_long_as_the_case_runs() {
let files = vec![(
"index.yaml".to_string(),
"name: Home\nalternatives:\n - name: Apply\n record_as: cases\n - name: Send the papers\n self_transition: { bucket: cases, to: complete, from: [open, incomplete], label: Send }\n - name: Withdraw\n self_transition: { bucket: cases, to: withdrawn, from: [open, incomplete, complete], label: Withdraw }\n".to_string(),
)];
let questions = build_questions(&files).unwrap();
let aggregates = parse_aggregates_yaml(
"aggregates:\n - bucket: cases\n initial: open\n states:\n open: { event: a }\n incomplete: { event: b }\n complete: { event: c }\n withdrawn: { event: d }\n transitions:\n open: [incomplete, complete, withdrawn]\n incomplete: [complete, withdrawn]\n complete: [withdrawn]\n",
)
.unwrap();
let p = Policy::from_content(&questions, &aggregates);
assert_eq!(p.validate(), Ok(()), "{}", p.text);
let bucket = Resource::Bucket { name: "cases".into() };
let me = Principal::Submitter { record: "r1".into(), email: "a@b.c".into() };
let ctx = |from: &str, to: &str| json!({ "from": from, "to": to, "stored_email": "a@b.c" });
assert!(p.decide(&me, Action::SelfTransition, &bucket, ctx("incomplete", "complete")).allow);
assert!(p.decide(&me, Action::SelfTransition, &bucket, ctx("complete", "withdrawn")).allow);
// And only where it was said: the paperwork cannot be sent
// after the case is already withdrawn.
assert!(!p.decide(&me, Action::SelfTransition, &bucket, ctx("withdrawn", "complete")).allow);
assert!(!p.decide(&me, Action::SelfTransition, &bucket, ctx("complete", "complete")).allow);
}
// Omitted, `from` is the initial state alone - what every
// unsubscribe link written before this existed already meant.
#[test]
fn a_move_that_does_not_say_where_from_is_a_one_shot_on_arrival() {
let p = site();
assert!(p.rules.iter().any(|r| r.id == "self:subscribers:open>unsubscribed"));
assert_eq!(p.rules.iter().filter(|r| r.id.starts_with("self:subscribers:")).count(), 1);
}
#[test] #[test]
fn invites_and_automation() { fn invites_and_automation() {
let p = site(); let p = site();
+59
View File
@@ -41,6 +41,16 @@ pub struct AggregateSchema {
/// that enters it, so content decides when a person hears /// that enters it, so content decides when a person hears
/// something and the mailer never has to know a site. /// something and the mailer never has to know a site.
pub mail_for_state: HashMap<String, MailSpec>, pub mail_for_state: HashMap<String, MailSpec>,
/// What a state grants the person whose record entered it: a group
/// they become a member of. A firm approved to file is a member of
/// the approved firms from that moment, and the decision that made
/// it so is the same one the desk already pressed - nobody should
/// have to remember to invite them afterwards.
pub grants_for_state: HashMap<String, String>,
/// The move a clock makes rather than a person. A hearing that gets
/// no answer in six weeks has to go on without one, and nothing in
/// a building is going to press that button.
pub after_for_state: HashMap<String, AfterSpec>,
} }
/// A mail to send when a record enters a state. `to` is `submitter` /// A mail to send when a record enters a state. `to` is `submitter`
@@ -64,6 +74,19 @@ impl MailSpec {
} }
} }
/// A move made by a clock: `after: { days: 42, to: no_answer }` on the
/// state a record is waiting in. The wait is measured from when the
/// record entered that state, and `to` must be an edge the same schema
/// already declares - a deadline may only do what a person could have
/// done. Days, because every deadline a public body keeps is in days
/// and a shorter unit would invite this to be used as a job queue.
#[derive(Clone, Debug, PartialEq, serde::Serialize, serde::Deserialize)]
#[serde(deny_unknown_fields)]
pub struct AfterSpec {
pub days: u32,
pub to: String,
}
impl AggregateSchema { impl AggregateSchema {
pub fn has_state(&self, state: &str) -> bool { pub fn has_state(&self, state: &str) -> bool {
self.event_for_state.contains_key(state) self.event_for_state.contains_key(state)
@@ -73,6 +96,14 @@ impl AggregateSchema {
self.mail_for_state.get(state) self.mail_for_state.get(state)
} }
pub fn grants_for(&self, state: &str) -> Option<&str> {
self.grants_for_state.get(state).map(String::as_str)
}
pub fn after_for(&self, state: &str) -> Option<&AfterSpec> {
self.after_for_state.get(state)
}
pub fn allowed(&self, state: &str) -> &[String] { pub fn allowed(&self, state: &str) -> &[String] {
self.transitions self.transitions
.get(state) .get(state)
@@ -86,6 +117,10 @@ struct RawState {
event: String, event: String,
#[serde(default)] #[serde(default)]
mail: Option<MailSpec>, mail: Option<MailSpec>,
#[serde(default)]
grants: Option<String>,
#[serde(default)]
after: Option<AfterSpec>,
} }
#[derive(Debug, serde::Deserialize)] #[derive(Debug, serde::Deserialize)]
@@ -156,7 +191,29 @@ pub fn parse_aggregates_yaml(raw: &str) -> anyhow::Result<HashMap<String, Aggreg
} }
let mut mail_for_state = HashMap::new(); let mut mail_for_state = HashMap::new();
let mut grants_for_state = HashMap::new();
let mut after_for_state = HashMap::new();
for (state, raw_state) in raw_schema.states { for (state, raw_state) in raw_schema.states {
if let Some(group) = raw_state.grants {
if group.trim().is_empty() {
anyhow::bail!("aggregates.yaml: bucket {bucket:?}: state {state:?}: grants needs a group");
}
grants_for_state.insert(state.clone(), group);
}
if let Some(after) = raw_state.after {
if after.days == 0 {
anyhow::bail!("aggregates.yaml: bucket {bucket:?}: state {state:?}: after.days must be at least one");
}
// A clock may only make a move a person could have
// made, so it is held to the same graph.
if !raw_schema.transitions.get(&state).is_some_and(|tos| tos.contains(&after.to)) {
anyhow::bail!(
"aggregates.yaml: bucket {bucket:?}: state {state:?}: after.to {:?} is not a declared transition from it",
after.to
);
}
after_for_state.insert(state.clone(), after);
}
if let Some(mail) = raw_state.mail { if let Some(mail) = raw_state.mail {
if mail.subject.trim().is_empty() || mail.body.trim().is_empty() { if mail.subject.trim().is_empty() || mail.body.trim().is_empty() {
anyhow::bail!("aggregates.yaml: bucket {bucket:?}: state {state:?}: mail needs a subject and a body"); anyhow::bail!("aggregates.yaml: bucket {bucket:?}: state {state:?}: mail needs a subject and a body");
@@ -181,6 +238,8 @@ pub fn parse_aggregates_yaml(raw: &str) -> anyhow::Result<HashMap<String, Aggreg
transitions: raw_schema.transitions, transitions: raw_schema.transitions,
attended_by: raw_schema.attended_by, attended_by: raw_schema.attended_by,
mail_for_state, mail_for_state,
grants_for_state,
after_for_state,
}, },
); );
} }
+9 -1
View File
@@ -290,6 +290,7 @@ async fn apply_transition(
.await .await
.map_err(|e| ServerFnError::new(e.to_string()))?; .map_err(|e| ServerFnError::new(e.to_string()))?;
crate::mail::on_state(state, bucket, &item.item_id, &item.to, &answer.responses).await; crate::mail::on_state(state, bucket, &item.item_id, &item.to, &answer.responses).await;
crate::people::on_state(state, bucket, &item.item_id, &item.to, &answer.responses).await;
// The decision extends the DAG as a child of the answer's own // The decision extends the DAG as a child of the answer's own
// submission hash, published like any other answer. // submission hash, published like any other answer.
@@ -370,7 +371,13 @@ pub async fn self_transition_answer(
serde_json::json!({ "from": answer.state, "to": self_transition.to, "stored_email": stored_email }), serde_json::json!({ "from": answer.state, "to": self_transition.to, "stored_email": stored_email }),
); );
if !decision.allow { if !decision.allow {
return Err(ServerFnError::new(if answer.state == OPEN_STATE { "not found" } else { "already processed" })); // Told apart by which half of the rule failed. If the record is
// somewhere this button may be fired from, the email was wrong,
// and the answer must not confirm that the record exists. If it
// is not, the case has moved past this move and saying so is no
// longer a way to learn anything about it.
let in_reach = self_transition.from.iter().any(|s| s == &answer.state);
return Err(ServerFnError::new(if in_reach { "not found" } else { "already processed" }));
} }
let decided_ms = chrono::Utc::now().timestamp_millis(); let decided_ms = chrono::Utc::now().timestamp_millis();
@@ -401,6 +408,7 @@ pub async fn self_transition_answer(
.await .await
.map_err(|e| ServerFnError::new(e.to_string()))?; .map_err(|e| ServerFnError::new(e.to_string()))?;
crate::mail::on_state(&state, &self_transition.bucket, &item_id, &self_transition.to, &answer.responses).await; crate::mail::on_state(&state, &self_transition.bucket, &item_id, &self_transition.to, &answer.responses).await;
crate::people::on_state(&state, &self_transition.bucket, &item_id, &self_transition.to, &answer.responses).await;
let parent_hashes = vec![item_id.clone()]; let parent_hashes = vec![item_id.clone()];
let chain_hash = hash_node(&question_id, &parent_hashes, &decision_payload_for_event, decided_ms); let chain_hash = hash_node(&question_id, &parent_hashes, &decision_payload_for_event, decided_ms);
+53 -10
View File
@@ -244,12 +244,16 @@ fn QuestionPage() -> impl IntoView {
let site = expect_context::<Resource<Result<SiteConfig, ServerFnError>>>(); let site = expect_context::<Resource<Result<SiteConfig, ServerFnError>>>();
view! { view! {
// Nothing is written here on purpose. This fallback is the
// whole served page until the question resolves - no header,
// no wordmark - so any word in it is the first and only thing
// a visitor, or a crawler, reads about the site. It used to
// say "loading…", which is true and is not about them. The
// space is held so the page does not jump when the words
// arrive, and `aria-busy` says what is happening to anyone
// whose reader would otherwise find an empty document.
<Suspense fallback=|| { <Suspense fallback=|| {
view! { view! { <main class="loading" aria-busy="true"></main> }
<main class="loading">
<p>"loading…"</p>
</main>
}
}> }>
{move || { {move || {
// Both read unconditionally (not nested inside the match // Both read unconditionally (not nested inside the match
@@ -1038,7 +1042,7 @@ fn AlternativeCard(
}; };
let Some(files) = input.files() else { continue }; let Some(files) = input.files() else { continue };
let Some(file) = files.get(0) else { continue }; let Some(file) = files.get(0) else { continue };
match upload_file(&file).await { match upload_file(&file, &question_id_for_submit, &alt_name_for_submit).await {
Ok(key) => { Ok(key) => {
map.insert(name.clone(), serde_json::Value::String(key)); map.insert(name.clone(), serde_json::Value::String(key));
} }
@@ -1451,7 +1455,12 @@ fn AlternativeCard(
.into_any(); .into_any();
} }
if req.kind == "hidden" { // `record` carries the case this one belongs to.
// Nobody types a chain hash, so it rides in on the
// link and is carried, not asked for — the label
// row would only ask a person to confirm a string
// that means nothing to them.
if req.kind == "hidden" || req.kind == "record" {
// A carrier value only — no label row, or a // A carrier value only — no label row, or a
// preset key reads as a second visible field. // preset key reads as a second visible field.
return view! { return view! {
@@ -1615,7 +1624,7 @@ fn ResourceFeature(
view! { view! {
<div class="resource"> <div class="resource">
<Suspense fallback=|| view! { <p class="resource-loading">"loading…"</p> }> <Suspense fallback=|| view! { <p class="resource-loading" aria-busy="true"></p> }>
{move || { {move || {
let feature_name = feature_name.clone(); let feature_name = feature_name.clone();
let transitions = transitions.clone(); let transitions = transitions.clone();
@@ -1872,7 +1881,7 @@ fn SelectField(
view! { view! {
<div class="select-field"> <div class="select-field">
<Suspense fallback=|| view! { <p class="resource-loading">"loading…"</p> }> <Suspense fallback=|| view! { <p class="resource-loading" aria-busy="true"></p> }>
{move || { {move || {
let id_field = id_field.clone(); let id_field = id_field.clone();
data.get() data.get()
@@ -2402,6 +2411,30 @@ pub async fn submit_answer(
let mut responses: serde_json::Value = serde_json::from_str(&responses_json) let mut responses: serde_json::Value = serde_json::from_str(&responses_json)
.map_err(|e| ServerFnError::new(format!("invalid responses: {e}")))?; .map_err(|e| ServerFnError::new(format!("invalid responses: {e}")))?;
// A record that belongs to another record: check the other one is
// there before this one is written, so a reference in a stored
// record is one that resolved at least once. An optional field
// left empty points at nothing and is nothing to check.
for requirement in alt.features.iter().flat_map(|f| &f.requirements) {
let (Some(of), true) = (&requirement.of, requirement.kind == "record") else {
continue;
};
let given = responses.get(&requirement.name).and_then(|v| v.as_str()).map(str::trim).unwrap_or_default();
if given.is_empty() {
if requirement.optional {
continue;
}
return Err(ServerFnError::new(format!("this answer is missing the {} it is about", requirement.name)));
}
let found = match state.jetstream.get_key_value(of).await {
Ok(store) => store.get(given).await.ok().flatten().is_some(),
Err(_) => false,
};
if !found {
return Err(ServerFnError::new(format!("the {} this is about is not on file", requirement.name)));
}
}
// An invite: the policy says whether this group may extend one, // An invite: the policy says whether this group may extend one,
// Kanidm does the onboarding, and what came back (the account, the // Kanidm does the onboarding, and what came back (the account, the
// one-time link) is recorded with the answer for the inviter's desk // one-time link) is recorded with the answer for the inviter's desk
@@ -2494,6 +2527,7 @@ pub async fn submit_answer(
// state sends, it sends now. // state sends, it sends now.
if let Some(initial) = state.aggregates.load().get(bucket).map(|s| s.initial.clone()) { if let Some(initial) = state.aggregates.load().get(bucket).map(|s| s.initial.clone()) {
crate::mail::on_state(&state, bucket, &chain_hash, &initial, &responses).await; crate::mail::on_state(&state, bucket, &chain_hash, &initial, &responses).await;
crate::people::on_state(&state, bucket, &chain_hash, &initial, &responses).await;
} }
} }
@@ -2507,12 +2541,21 @@ pub async fn submit_answer(
/// server side is a raw Axum multipart handler (`src/upload.rs`) for /// server side is a raw Axum multipart handler (`src/upload.rs`) for
/// the same reason. /// the same reason.
#[cfg(feature = "hydrate")] #[cfg(feature = "hydrate")]
async fn upload_file(file: &web_sys::File) -> Result<String, wasm_bindgen::JsValue> { async fn upload_file(
file: &web_sys::File,
question_id: &str,
alternative: &str,
) -> Result<String, wasm_bindgen::JsValue> {
use wasm_bindgen::{JsCast, JsValue}; use wasm_bindgen::{JsCast, JsValue};
use wasm_bindgen_futures::JsFuture; use wasm_bindgen_futures::JsFuture;
use web_sys::{FormData, Request, RequestInit, Response}; use web_sys::{FormData, Request, RequestInit, Response};
let form = FormData::new()?; let form = FormData::new()?;
// Which page and which choice this file is being attached to. The
// server refuses an upload that does not name a page it may
// submit, so these travel with every file.
form.append_with_str("question", question_id)?;
form.append_with_str("alternative", alternative)?;
form.append_with_blob("file", file)?; form.append_with_blob("file", file)?;
let opts = RequestInit::new(); let opts = RequestInit::new();
+138 -2
View File
@@ -344,6 +344,19 @@ pub struct SelfTransition {
pub bucket: String, pub bucket: String,
pub to: String, pub to: String,
pub label: String, pub label: String,
/// The states this move may be made from. A person is in dialogue
/// with a case for as long as it runs, not only on the day they
/// sent it: an applicant written to for a missing drawing answers
/// from `incomplete`, and may withdraw from anywhere along the
/// way. Defaults to the initial state alone, which is what every
/// one-shot "unsubscribe" link wants and what this was before it
/// could be said.
#[serde(default = "initial_state_only")]
pub from: Vec<String>,
}
fn initial_state_only() -> Vec<String> {
vec![crate::answers::OPEN_STATE.to_string()]
} }
#[derive(Clone, Debug, Serialize, Deserialize)] #[derive(Clone, Debug, Serialize, Deserialize)]
@@ -391,6 +404,21 @@ pub struct ResourceSpec {
/// `requires_group` regardless of this flag. /// `requires_group` regardless of this flag.
#[serde(default)] #[serde(default)]
pub public: bool, pub public: bool,
/// The record states this resource shows, for a `Kv` source; empty
/// shows every one. A register that is public by law is public
/// about what was decided, not about what was withdrawn or turned
/// down, and the desk that works the same bucket is a different
/// spec and still sees all of it.
#[serde(default)]
pub states: Vec<String>,
/// Show only the records whose `where.field` holds `where.equals`,
/// for a `Kv` source. With a `type: record` field on the other
/// side, this is the half that reads: an application's page lists
/// the objections that name it, because each objection stored the
/// application's id in a field and this asks for exactly those.
/// `equals` takes the page's own `{key}` like a resource key does.
#[serde(default, rename = "where")]
pub filter: Option<FieldIs>,
/// The moves a listed answer may make, one button each - empty /// The moves a listed answer may make, one button each - empty
/// means read-only. Server calls are checked against this /// means read-only. Server calls are checked against this
/// allow-list, so a client can never fire a transition content /// allow-list, so a client can never fire a transition content
@@ -418,7 +446,6 @@ pub struct ResourceSpec {
pub enum ResourceSource { pub enum ResourceSource {
Kv { bucket: String }, Kv { bucket: String },
GiteaStarred { username: String }, GiteaStarred { username: String },
GiteaOrgRepos { org: String },
/// A repo's releases - fetched with `GITEA_API_TOKEN`, so it works /// A repo's releases - fetched with `GITEA_API_TOKEN`, so it works
/// for private repos too (shape the link with jq: a private /// for private repos too (shape the link with jq: a private
/// release's html_url 404s for anonymous visitors, so map it to /// release's html_url 404s for anonymous visitors, so map it to
@@ -438,7 +465,6 @@ impl ResourceSpec {
match &self.source { match &self.source {
ResourceSource::Kv { bucket } => Some(bucket), ResourceSource::Kv { bucket } => Some(bucket),
ResourceSource::GiteaStarred { .. } ResourceSource::GiteaStarred { .. }
| ResourceSource::GiteaOrgRepos { .. }
| ResourceSource::GiteaReleases { .. } | ResourceSource::GiteaReleases { .. }
| ResourceSource::Url { .. } => None, | ResourceSource::Url { .. } => None,
} }
@@ -505,6 +531,18 @@ pub struct Requirement {
/// page's key into the answer. /// page's key into the answer.
#[serde(default)] #[serde(default)]
pub value: Option<String>, pub value: Option<String>,
/// `type: record` only - the bucket the referenced record lives in.
/// This is how one case belongs to another: an objection names the
/// application it objects to, an appeal the decision it appeals.
/// The stored value is the parent's record id, and a submission
/// naming a record that is not there is refused, so a reference in
/// a record is one that resolved at least once.
///
/// A person never types one of these: the id is a chain hash. It
/// arrives the way it was given out - `value: "{key}"` on a dynamic
/// page, so the link in the letter carries the case it is about.
#[serde(default)]
pub of: Option<String>,
/// `type: gesture` / `type: voice` - ws(s):// URL of a redoal-relay instance /// `type: gesture` / `type: voice` - ws(s):// URL of a redoal-relay instance
/// the drawing widget connects to for live echoes of similar /// the drawing widget connects to for live echoes of similar
/// strokes. Absent means the widget works offline: the drawn path /// strokes. Absent means the widget works offline: the drawn path
@@ -518,6 +556,16 @@ pub struct Requirement {
/// value. When the sibling changes, `resource` is fetched with the /// value. When the sibling changes, `resource` is fetched with the
/// value as a parameter and the result replaces this field's value /// value as a parameter and the result replaces this field's value
/// (an empty sibling never fetches, and never clears an edit). /// (an empty sibling never fetches, and never clears an edit).
/// "Show the ones whose `field` is `equals`." The whole of what a
/// listing may ask about a record's own contents: one field, one
/// value, compared as strings. Anything more is a query language, and
/// a jq filter is already there for shaping what comes back.
#[derive(Clone, Debug, Serialize, Deserialize)]
pub struct FieldIs {
pub field: String,
pub equals: String,
}
#[derive(Clone, Debug, Serialize, Deserialize)] #[derive(Clone, Debug, Serialize, Deserialize)]
pub struct Bind { pub struct Bind {
/// The sibling requirement (same alternative) to watch. /// The sibling requirement (same alternative) to watch.
@@ -1075,6 +1123,94 @@ alternatives:
std::collections::HashMap::from([(question.id.clone(), question)]) std::collections::HashMap::from([(question.id.clone(), question)])
} }
fn question_with_record(field_yaml: &str) -> std::collections::HashMap<String, Question> {
let question: Question = serde_yaml::from_str(&format!(
r#"
id: /object
name: Object to it
alternatives:
- name: The register
features:
- name: All
resource:
public: true
source: {{ kind: kv, bucket: things }}
- name: Object
record_as: things
features:
- name: Yours
requirements:
- name: what
{field_yaml}
"#
))
.unwrap();
std::collections::HashMap::from([(question.id.clone(), question)])
}
// An objection belongs to the application it objects to. The
// bucket it points at has to be one this repo declares, or the
// reference could never resolve.
#[test]
fn a_record_can_belong_to_another_record() {
let good = question_with_record(" - { name: case, type: record, of: things }");
validate_questions(&good, &schema_fixture()).unwrap();
let nowhere = question_with_record(" - { name: case, type: record, of: nothing_declared }");
let err = validate_questions(&nowhere, &schema_fixture()).unwrap_err();
assert!(err.to_string().contains("has no aggregate declared"), "{err}");
let unsaid = question_with_record(" - { name: case, type: record }");
let err = validate_questions(&unsaid, &schema_fixture()).unwrap_err();
assert!(err.to_string().contains("must say `of:"), "{err}");
// `of` means one thing, so it belongs to one type.
let stray = question_with_record(" - { name: case, of: things }");
let err = validate_questions(&stray, &schema_fixture()).unwrap_err();
assert!(err.to_string().contains("only a type: record field has"), "{err}");
// The id arrives on the link; it is not looked up.
let bound = question_with_record(" - name: case\n type: record\n of: things\n bind:\n field: what\n resource:\n source: { kind: url, url: \"https://x.example/{p}\" }");
let err = validate_questions(&bound, &schema_fixture()).unwrap_err();
assert!(err.to_string().contains("the link it came in on carries the id"), "{err}");
}
// A state that grants a group makes an account for someone, so the
// form that fills the bucket has to have asked who they are.
#[test]
fn a_state_that_grants_a_group_needs_a_name_and_an_address_on_file() {
let granting = crate::aggregates::parse_aggregates_yaml(
"aggregates:\n - bucket: things\n initial: open\n states:\n open: { event: opened }\n middle: { event: advanced }\n done: { event: finished, grants: approved_firms }\n transitions:\n open: [middle]\n middle: [done]\n",
)
.unwrap();
let asks = question_with_record(" - { name: name }\n - { name: email, type: email }");
validate_questions(&asks, &granting).unwrap();
let no_name = question_with_record(" - { name: email, type: email }");
let err = validate_questions(&no_name, &granting).unwrap_err();
assert!(err.to_string().contains("grant a group"), "{err}");
// The same form is fine against a schema that grants nothing.
validate_questions(&no_name, &schema_fixture()).unwrap();
}
// A deadline may only make a move a person could have made.
#[test]
fn a_states_deadline_is_held_to_the_same_graph_as_a_button() {
let ok = crate::aggregates::parse_aggregates_yaml(
"aggregates:\n - bucket: hearings\n initial: open\n states:\n open: { event: opened, after: { days: 42, to: no_answer } }\n no_answer: { event: gave_up }\n transitions:\n open: [no_answer]\n",
)
.unwrap();
assert_eq!(ok["hearings"].after_for("open").unwrap().days, 42);
let undeclared = crate::aggregates::parse_aggregates_yaml(
"aggregates:\n - bucket: hearings\n initial: open\n states:\n open: { event: opened, after: { days: 42, to: no_answer } }\n no_answer: { event: gave_up }\n transitions: {}\n",
);
assert!(undeclared.unwrap_err().to_string().contains("not a declared transition"));
}
#[test] #[test]
fn declared_edge_passes() { fn declared_edge_passes() {
let questions = question_with_transitions( let questions = question_with_transitions(
+93
View File
@@ -115,6 +115,38 @@ pub fn validate_questions(
} }
} }
} }
// A state that grants a group makes an account for the person the
// record is about, so the record has to know who they are. Same
// shape as an invite's own rule, one step further back: the form
// that fills the bucket is where the name and the address have to
// be asked for, because by the time a desk presses approve there is
// nobody left to ask.
for (bucket, schema) in aggregates {
let mut granting: Vec<&String> = schema.grants_for_state.keys().collect();
if granting.is_empty() {
continue;
}
granting.sort();
for question in questions.values() {
for alternative in question.alternatives.iter().filter(|a| a.record_as.as_deref() == Some(bucket.as_str())) {
let required = |name: &str, kind: &str| {
alternative
.features
.iter()
.flat_map(|f| &f.requirements)
.any(|r| r.name == name && r.kind == kind && !r.optional)
};
if !required("email", "email") || !required("name", "text") {
anyhow::bail!(
"question {:?} alternative {:?}: records into {bucket:?}, whose state(s) {:?} grant a group, so it must collect a required `name` and a required `email` field of type email",
question.id, alternative.name, granting
);
}
}
}
}
for question in questions.values() { for question in questions.values() {
if let Some(event) = &question.event { if let Some(event) = &question.event {
if question.is_dynamic() { if question.is_dynamic() {
@@ -210,6 +242,30 @@ pub fn validate_questions(
question.id, alternative.name, st.to, st.bucket question.id, alternative.name, st.to, st.bucket
); );
} }
if st.from.is_empty() {
anyhow::bail!(
"question {:?} alternative {:?}: self_transition.from is empty, so the move can never be made",
question.id, alternative.name
);
}
for from in &st.from {
if !schema.has_state(from) {
anyhow::bail!(
"question {:?} alternative {:?}: self_transition.from {:?} is not a declared state for bucket {:?}",
question.id, alternative.name, from, st.bucket
);
}
// The policy would otherwise permit a move the
// engine refuses: the reader of the content
// should hear about that here, not from a
// person who clicked the button.
if !schema.allowed(from).iter().any(|s| s == &st.to) {
anyhow::bail!(
"question {:?} alternative {:?}: self_transition {:?} -> {:?} is not a declared edge for bucket {:?}",
question.id, alternative.name, from, st.to, st.bucket
);
}
}
} }
} }
// Bind targets resolve across the whole alternative, same // Bind targets resolve across the whole alternative, same
@@ -275,6 +331,35 @@ pub fn validate_questions(
} }
} }
} }
// One case belonging to another. The bucket has to
// be one this repo declares, or the reference could
// never resolve; and it is the only thing `of`
// means, so it belongs to no other type.
if requirement.kind == "record" {
let Some(of) = &requirement.of else {
anyhow::bail!(
"question {:?} alternative {:?} feature {:?}: requirement {:?} is type: record - it must say `of: <bucket>`, the bucket the record it points at lives in",
question.id, alternative.name, feature.name, requirement.name
);
};
if !aggregates.contains_key(of) {
anyhow::bail!(
"question {:?} alternative {:?} feature {:?}: requirement {:?} points at bucket {of:?}, which has no aggregate declared",
question.id, alternative.name, feature.name, requirement.name
);
}
if requirement.resource.is_some() || requirement.bind.is_some() {
anyhow::bail!(
"question {:?} alternative {:?} feature {:?}: requirement {:?} is type: record - the link it came in on carries the id, it can't also load one from a resource or bind",
question.id, alternative.name, feature.name, requirement.name
);
}
} else if requirement.of.is_some() {
anyhow::bail!(
"question {:?} alternative {:?} feature {:?}: requirement {:?} has `of`, which only a type: record field has",
question.id, alternative.name, feature.name, requirement.name
);
}
if let Some(bind) = &requirement.bind { if let Some(bind) = &requirement.bind {
if bind.field == requirement.name if bind.field == requirement.name
|| !sibling_names.contains(bind.field.as_str()) || !sibling_names.contains(bind.field.as_str())
@@ -295,6 +380,14 @@ pub fn validate_questions(
let Some(schema) = aggregates.get(bucket) else { let Some(schema) = aggregates.get(bucket) else {
continue; continue;
}; };
for state in &resource.states {
if !schema.has_state(state) {
anyhow::bail!(
"question {:?} alternative {:?} feature {:?}: states {:?} is not a declared state for bucket {:?}",
question.id, alternative.name, feature.name, state, bucket
);
}
}
for transition in &resource.transitions { for transition in &resource.transitions {
if !schema.has_state(&transition.from) { if !schema.has_state(&transition.from) {
anyhow::bail!( anyhow::bail!(
+196
View File
@@ -0,0 +1,196 @@
//! The moves a clock makes rather than a person.
//!
//! A state in `aggregates.yaml` may carry `after: { days: 42, to:
//! no_answer }`: when a record has waited there that long, portal makes
//! the move itself. Every deadline a public body keeps is of this
//! shape - a hearing goes out and the authority has six weeks to
//! answer, and when it does not the case has to go on without one, with
//! nobody in the building in a position to press that button.
//!
//! A deadline may only do what a person could have done: `after.to` is
//! held to the same declared edge as a desk button (see
//! `parse_aggregates_yaml`), the move goes through the same
//! `aggregates::transition`, the same mail fires, and it lands on
//! `portal.answers.submitted` like any other decision. What it is not
//! is a job queue: the unit is days, the tick is an hour, and a record
//! that has waited long enough is moved once, because after the move it
//! is no longer in a state that has a deadline.
use crate::answers::Answer;
use crate::events::{emit_answer_submitted, AnswerSubmitted};
use crate::server::AppState;
/// Stamped on the record and on the transition event in place of a
/// username, so a desk and an n8n workflow can both tell a decision
/// nobody made from one somebody did.
pub const BY: &str = "deadline";
const DAY_MS: i64 = 24 * 60 * 60 * 1000;
pub async fn run(state: AppState) {
let mut tick = tokio::time::interval(std::time::Duration::from_secs(60 * 60));
loop {
tick.tick().await;
if let Err(e) = sweep(&state).await {
tracing::warn!(error = %e, "deadline sweep failed");
}
}
}
/// When a record entered the state it is in now. A record nobody has
/// moved has been waiting since it was sent.
fn waiting_since(answer: &Answer) -> i64 {
answer.decided_ms.unwrap_or(answer.submitted_ms)
}
/// Whether a record in `state` has waited out its deadline by `now_ms`.
pub fn due(schema: &crate::aggregates::AggregateSchema, answer: &Answer, now_ms: i64) -> Option<String> {
let after = schema.after_for(&answer.state)?;
let deadline = waiting_since(answer) + i64::from(after.days) * DAY_MS;
(now_ms >= deadline).then(|| after.to.clone())
}
/// One pass over every bucket that has a deadline on any of its states.
pub async fn sweep(state: &AppState) -> anyhow::Result<()> {
let now_ms = chrono::Utc::now().timestamp_millis();
let aggregates = state.aggregates.load();
for schema in aggregates.values() {
if schema.after_for_state.is_empty() {
continue;
}
// A bucket is created on first write, so one that has never
// been written to is an ordinary empty bucket, not an error.
let Ok(store) = state.jetstream.get_key_value(&schema.bucket).await else {
continue;
};
let keys: Vec<String> = {
use futures::TryStreamExt;
match store.keys().await {
Ok(keys) => keys.try_collect().await.unwrap_or_default(),
Err(_) => continue,
}
};
for id in keys {
let Ok(Some(bytes)) = store.get(&id).await else { continue };
let Ok(mut answer) = serde_json::from_slice::<Answer>(&bytes) else { continue };
let Some(target) = due(schema, &answer, now_ms) else { continue };
let payload = serde_json::json!({ "to": target, "item": id, "by": BY });
if let Err(e) = crate::aggregates::transition(
&state.jetstream, schema, &id, &target, payload.clone(), now_ms,
)
.await
{
// A record whose events are gone, or one another writer
// moved between the read and the write: neither is this
// sweep's to fix, and the next tick sees it again.
tracing::warn!(bucket = %schema.bucket, id, target, "a deadline could not be made: {e}");
continue;
}
answer.state = target.clone();
answer.decided_ms = Some(now_ms);
answer.decided_by = Some(BY.to_string());
if let Ok(data) = serde_json::to_vec(&answer) {
let _ = store.put(&id, data.into()).await;
}
crate::mail::on_state(state, &schema.bucket, &id, &target, &answer.responses).await;
crate::people::on_state(state, &schema.bucket, &id, &target, &answer.responses).await;
let parent_hashes = vec![id.clone()];
let chain_hash = crate::chain::hash_node(&answer.question_id, &parent_hashes, &payload, now_ms);
if let Err(e) = emit_answer_submitted(
&state.nats,
&AnswerSubmitted {
chain_hash,
parent_hashes,
question_id: answer.question_id.clone(),
alternative: target.clone(),
responses: payload,
timestamp_ms: now_ms,
},
)
.await
{
tracing::warn!(bucket = %schema.bucket, id, "the deadline was made but not published: {e}");
}
tracing::info!(bucket = %schema.bucket, id, target, "a deadline passed and the case went on");
}
}
Ok(())
}
#[cfg(test)]
mod tests {
use super::*;
fn schema(yaml: &str) -> crate::aggregates::AggregateSchema {
crate::aggregates::parse_aggregates_yaml(yaml).unwrap().into_values().next().unwrap()
}
const HEARINGS: &str = r#"
aggregates:
- bucket: hearings
initial: open
states:
open: { event: received, after: { days: 42, to: no_answer } }
answered: { event: answered }
no_answer: { event: no_answer }
transitions:
open: [answered, no_answer]
"#;
fn waiting(state: &str, since_ms: i64) -> Answer {
Answer {
id: "r".into(), question_id: "/q".into(), alternative: "a".into(),
responses: serde_json::json!({}), state: state.into(),
submitted_ms: since_ms, decided_ms: None, decided_by: None, notes: vec![],
}
}
#[test]
fn a_deadline_comes_due_the_day_it_says_and_not_before() {
let s = schema(HEARINGS);
let sent = 1_000_000_000_000;
let answer = waiting("open", sent);
assert_eq!(due(&s, &answer, sent), None);
assert_eq!(due(&s, &answer, sent + 41 * DAY_MS), None);
assert_eq!(due(&s, &answer, sent + 42 * DAY_MS), Some("no_answer".into()));
}
// The six weeks run from the last move, not from the day the case
// was opened: a record someone touched has been waiting since then.
#[test]
fn the_wait_is_measured_from_the_last_move() {
let s = schema(HEARINGS);
let sent = 1_000_000_000_000;
let mut answer = waiting("open", sent);
answer.decided_ms = Some(sent + 40 * DAY_MS);
assert_eq!(due(&s, &answer, sent + 42 * DAY_MS), None);
assert_eq!(due(&s, &answer, sent + 82 * DAY_MS), Some("no_answer".into()));
}
#[test]
fn a_state_with_no_deadline_waits_forever() {
let s = schema(HEARINGS);
let answer = waiting("answered", 0);
assert_eq!(due(&s, &answer, i64::MAX / 2), None);
}
// A clock may only make a move a person could have made, so a
// deadline onto an undeclared edge is refused when content loads,
// not when the hour comes round.
#[test]
fn a_deadline_onto_an_undeclared_edge_is_refused_on_load() {
let bad = HEARINGS.replace("to: no_answer }", "to: answered }").replace("open: [answered, no_answer]", "open: [no_answer]");
let err = crate::aggregates::parse_aggregates_yaml(&bad).unwrap_err().to_string();
assert!(err.contains("not a declared transition"), "{err}");
}
#[test]
fn a_deadline_of_no_days_is_refused() {
let bad = HEARINGS.replace("days: 42", "days: 0");
let err = crate::aggregates::parse_aggregates_yaml(&bad).unwrap_err().to_string();
assert!(err.contains("at least one"), "{err}");
}
}
+6
View File
@@ -13,6 +13,12 @@ pub mod aggregates;
#[cfg(feature = "ssr")] #[cfg(feature = "ssr")]
pub mod announce; pub mod announce;
#[cfg(feature = "ssr")]
pub mod deadlines;
#[cfg(feature = "ssr")]
pub mod sessions;
#[cfg(feature = "ssr")] #[cfg(feature = "ssr")]
pub mod http; pub mod http;
+54 -4
View File
@@ -10,7 +10,7 @@ async fn main() -> anyhow::Result<()> {
use portal::upload::{self, Garage}; use portal::upload::{self, Garage};
use std::sync::Arc; use std::sync::Arc;
use tower_http::services::{ServeDir, ServeFile}; use tower_http::services::{ServeDir, ServeFile};
use tower_sessions::{MemoryStore, SessionManagerLayer}; use tower_sessions::SessionManagerLayer;
dotenvy::dotenv().ok(); dotenvy::dotenv().ok();
tracing_subscriber::fmt() tracing_subscriber::fmt()
@@ -107,6 +107,7 @@ async fn main() -> anyhow::Result<()> {
// Announced pages: keep their desk records current, close ended // Announced pages: keep their desk records current, close ended
// windows (see announce.rs). // windows (see announce.rs).
tokio::spawn(portal::announce::run(state.clone())); tokio::spawn(portal::announce::run(state.clone()));
tokio::spawn(portal::deadlines::run(state.clone()));
// Replies to case mail, matched to their record by whatever reads // Replies to case mail, matched to their record by whatever reads
// the mailbox (uhhm/gdo over JMAP) and turned into notes here. A // the mailbox (uhhm/gdo over JMAP) and turned into notes here. A
@@ -126,11 +127,50 @@ async fn main() -> anyhow::Result<()> {
let cookie_secure = std::env::var("COOKIE_SECURE") let cookie_secure = std::env::var("COOKIE_SECURE")
.map(|v| v == "true" || v == "1") .map(|v| v == "true" || v == "1")
.unwrap_or(false); .unwrap_or(false);
let session_layer = SessionManagerLayer::new(MemoryStore::default()) // Sessions live in the same JetStream as everything else, so a
// deploy no longer signs out every desk on the site it deploys.
let session_store = portal::sessions::NatsStore::open(&state.jetstream).await?;
let session_layer = SessionManagerLayer::new(session_store)
.with_secure(cookie_secure) .with_secure(cookie_secure)
.with_same_site(tower_sessions::cookie::SameSite::Lax) .with_same_site(tower_sessions::cookie::SameSite::Lax)
.with_name("portal_session"); .with_name("portal_session");
// What a person does costs a little; what a script does costs a
// lot. A burst of 30 with one refilled every two seconds is well
// above anything a visitor filling in a form will reach - a page
// load fetches several resources at once - and well below what it
// takes to be a nuisance. Per caller IP, in memory, no store: this
// is a fence, not an accounting system, and the fence that matters
// for uploads is the policy check on the route itself.
//
// PORTAL_RATE_PER_SECOND / PORTAL_RATE_BURST retune it per site;
// 0 for either turns it off, for an instance behind something that
// already does this.
let rate_seconds: u64 = std::env::var("PORTAL_RATE_SECONDS")
.ok()
.and_then(|v| v.parse().ok())
.unwrap_or(2);
let rate_burst: u32 = std::env::var("PORTAL_RATE_BURST")
.ok()
.and_then(|v| v.parse().ok())
.unwrap_or(30);
let governor = (rate_seconds > 0 && rate_burst > 0).then(|| {
// SmartIpKeyExtractor reads X-Forwarded-For/X-Real-IP, which is
// what every one of these instances is behind; it falls back to
// the socket address when neither is set.
let conf = tower_governor::governor::GovernorConfigBuilder::default()
.per_second(rate_seconds)
.burst_size(rate_burst)
.key_extractor(tower_governor::key_extractor::SmartIpKeyExtractor)
.finish()
.expect("a rate limit of at least one per second and a burst of at least one");
std::sync::Arc::new(conf)
});
match &governor {
Some(_) => tracing::info!(rate_seconds, rate_burst, "rate limiting per caller IP"),
None => tracing::warn!("rate limiting is off (PORTAL_RATE_SECONDS or PORTAL_RATE_BURST is 0)"),
}
async fn server_fn_handler( async fn server_fn_handler(
State(state): State<AppState>, State(state): State<AppState>,
request: Request<Body>, request: Request<Body>,
@@ -174,6 +214,10 @@ async fn main() -> anyhow::Result<()> {
"/upload", "/upload",
post(upload::upload).layer(axum::extract::DefaultBodyLimit::max(upload::MAX_UPLOAD_BYTES + 64 * 1024)), post(upload::upload).layer(axum::extract::DefaultBodyLimit::max(upload::MAX_UPLOAD_BYTES + 64 * 1024)),
) )
.route(
"/attachment/{bucket}/{record}/{field}",
get(upload::attachment),
)
.route("/gitea-repo", get(content::gitea_repo_handler)) .route("/gitea-repo", get(content::gitea_repo_handler))
// Content-shipped assets (hero module etc.), same-origin. // Content-shipped assets (hero module etc.), same-origin.
.route("/site/{*path}", get(content::site_asset_handler)) .route("/site/{*path}", get(content::site_asset_handler))
@@ -223,8 +267,14 @@ async fn main() -> anyhow::Result<()> {
}, },
) )
.fallback(leptos_axum::file_and_error_handler::<AppState, _>(shell)) .fallback(leptos_axum::file_and_error_handler::<AppState, _>(shell))
.layer(session_layer) .layer(session_layer);
.with_state(state); // Outermost, so a caller who is over the limit is turned away
// before a session is loaded or a handler is entered.
let app = match governor {
Some(conf) => app.layer(tower_governor::GovernorLayer::new(conf)),
None => app,
}
.with_state(state);
tracing::info!("listening on http://{addr}"); tracing::info!("listening on http://{addr}");
let listener = tokio::net::TcpListener::bind(&addr).await?; let listener = tokio::net::TcpListener::bind(&addr).await?;
+62
View File
@@ -189,6 +189,68 @@ pub fn username_for(email: &str) -> String {
} }
} }
/// The person a record is about, for a state that grants a group.
/// Both are required by the lint of any form that fills such a bucket
/// (see `validate_questions`), so a record missing either is one that
/// predates the rule, and is reported rather than guessed at.
pub fn who_for(responses: &serde_json::Value) -> Option<(String, String)> {
let field = |name: &str| {
responses
.get(name)
.and_then(|v| v.as_str())
.map(str::trim)
.filter(|s| !s.is_empty())
.map(String::from)
};
let email = field("email").filter(|e| e.contains('@'))?;
Some((field("name")?, email))
}
/// A record entered `state`: if that state grants a group, put the
/// person the record is about in it. The decision the desk already
/// pressed is the whole ceremony - an approval that means membership
/// should not also need somebody to remember to invite them.
///
/// Best-effort, exactly like the mail beside it: the transition is
/// committed either way, and an instance with no Kanidm configured
/// simply grants nothing and says so once per occurrence.
pub async fn on_state(
state: &crate::server::AppState,
bucket: &str,
id: &str,
to_state: &str,
responses: &serde_json::Value,
) {
let Some(group) = state
.aggregates
.load()
.get(bucket)
.and_then(|s| s.grants_for(to_state).map(str::to_string))
else {
return;
};
let Some(kanidm) = Kanidm::from_env() else {
tracing::warn!(bucket, id, to_state, group, "this state grants a group, but invites are not configured on this instance (KANIDM_API_TOKEN) - nobody was added");
return;
};
let Some((name, email)) = who_for(responses) else {
tracing::warn!(bucket, id, to_state, group, "this state grants a group, but the record carries no name and email - nobody was added");
return;
};
let invited = match kanidm.invite(&name, &email, &group).await {
Ok(invited) => invited,
Err(e) => {
tracing::error!(bucket, id, to_state, group, "granting the group failed: {e}");
return;
}
};
if let Ok(bytes) = serde_json::to_vec(&invited) {
let _ = state.nats.publish(INVITED_SUBJECT, bytes.into()).await;
}
crate::mail::on_invite(state, &invited).await;
tracing::info!(bucket, id, to_state, group, user = %invited.username, created = invited.created, "the decision granted a group");
}
#[cfg(test)] #[cfg(test)]
mod tests { mod tests {
use super::*; use super::*;
+108 -6
View File
@@ -13,7 +13,7 @@ use leptos::prelude::*;
/// `public: true` nor `requires_group` set is unreachable, not "open" /// `public: true` nor `requires_group` set is unreachable, not "open"
/// by omission. `params` are the requesting form's current field /// by omission. `params` are the requesting form's current field
/// values (if any) - forwarded as query parameters to a live external /// values (if any) - forwarded as query parameters to a live external
/// fetch (`GiteaStarred`/`GiteaOrgRepos`/`Url`), the mechanism behind /// fetch (`GiteaStarred`/`GiteaReleases`/`Url`), the mechanism behind
/// "a resource parameterized by other form fields"; a `Kv` resource /// "a resource parameterized by other form fields"; a `Kv` resource
/// ignores them entirely, same as today. /// ignores them entirely, same as today.
#[server(endpoint = "get_resource")] #[server(endpoint = "get_resource")]
@@ -188,6 +188,50 @@ async fn authorize_resource(
} }
} }
/// Whether a stored record passes the resource's `states` filter. No
/// filter shows everything, which is what every resource written
/// before this existed means. A record with no `state` at all is not
/// an aggregate and is left alone.
#[cfg(feature = "ssr")]
fn shows_state(resource: &crate::content::ResourceSpec, value: &serde_json::Value) -> bool {
if resource.states.is_empty() {
return true;
}
match value.get("state").and_then(|s| s.as_str()) {
Some(state) => resource.states.iter().any(|s| s == state),
None => true,
}
}
/// Whether a stored record passes the resource's `where` filter. The
/// field is looked for in the record's `responses`, which is where a
/// form's answers live, and then on the record itself, so a listing
/// can be narrowed by what was submitted or by what the runtime
/// stamped on it. A missing field never matches: "everything that did
/// not say" is not something a page should be able to ask for by
/// accident.
#[cfg(feature = "ssr")]
fn matches_filter(
resource: &crate::content::ResourceSpec,
value: &serde_json::Value,
params: &std::collections::HashMap<String, String>,
) -> bool {
let Some(filter) = &resource.filter else {
return true;
};
let wanted = crate::content::template_action(&filter.equals, &serde_json::json!(params))
.unwrap_or_else(|| filter.equals.clone());
let found = value
.get("responses")
.and_then(|r| r.get(&filter.field))
.or_else(|| value.get(&filter.field));
match found {
Some(serde_json::Value::String(s)) => s == &wanted,
Some(other) => other.to_string() == wanted,
None => false,
}
}
#[cfg(feature = "ssr")] #[cfg(feature = "ssr")]
async fn fetch_resource_value( async fn fetch_resource_value(
state: &crate::server::AppState, state: &crate::server::AppState,
@@ -216,7 +260,14 @@ async fn fetch_resource_value(
.await .await
.map_err(|e| ServerFnError::new(e.to_string()))? .map_err(|e| ServerFnError::new(e.to_string()))?
.ok_or_else(|| ServerFnError::new("resource key not found"))?; .ok_or_else(|| ServerFnError::new("resource key not found"))?;
serde_json::from_slice(&bytes).map_err(|e| ServerFnError::new(e.to_string()))? let value: serde_json::Value =
serde_json::from_slice(&bytes).map_err(|e| ServerFnError::new(e.to_string()))?;
// A state filter holds for one record as well as a
// listing, or naming the key would walk around it.
if !shows_state(resource, &value) || !matches_filter(resource, &value, params) {
return Err(ServerFnError::new("resource key not found"));
}
value
} }
(None, Some(store)) => { (None, Some(store)) => {
use futures::TryStreamExt; use futures::TryStreamExt;
@@ -231,7 +282,9 @@ async fn fetch_resource_value(
for key in keys { for key in keys {
if let Ok(Some(bytes)) = store.get(&key).await { if let Ok(Some(bytes)) = store.get(&key).await {
if let Ok(value) = serde_json::from_slice::<serde_json::Value>(&bytes) { if let Ok(value) = serde_json::from_slice::<serde_json::Value>(&bytes) {
items.push(value); if shows_state(resource, &value) && matches_filter(resource, &value, params) {
items.push(value);
}
} }
} }
} }
@@ -242,9 +295,6 @@ async fn fetch_resource_value(
ResourceSource::GiteaStarred { username } => { ResourceSource::GiteaStarred { username } => {
fetch_gitea_json(state, &format!("/api/v1/users/{username}/starred"), params).await? fetch_gitea_json(state, &format!("/api/v1/users/{username}/starred"), params).await?
} }
ResourceSource::GiteaOrgRepos { org } => {
fetch_gitea_json(state, &format!("/api/v1/orgs/{org}/repos"), params).await?
}
ResourceSource::GiteaReleases { owner, repo } => { ResourceSource::GiteaReleases { owner, repo } => {
fetch_gitea_json(state, &format!("/api/v1/repos/{owner}/{repo}/releases"), params) fetch_gitea_json(state, &format!("/api/v1/repos/{owner}/{repo}/releases"), params)
.await? .await?
@@ -470,6 +520,58 @@ pub fn apply_jq(
mod tests { mod tests {
use super::*; use super::*;
fn spec(yaml: &str) -> crate::content::ResourceSpec {
serde_yaml::from_str(yaml).unwrap()
}
fn record(state: &str, responses: serde_json::Value) -> serde_json::Value {
serde_json::json!({ "id": "r", "state": state, "responses": responses })
}
// A register that is public by law is public about what was
// decided, not about what was withdrawn.
#[test]
fn a_listing_can_be_narrowed_to_the_states_worth_showing() {
let all = spec("source: { kind: kv, bucket: cases }\npublic: true\n");
let decided = spec("source: { kind: kv, bucket: cases }\npublic: true\nstates: [approved, refused]\n");
for state in ["approved", "refused", "withdrawn", "open"] {
assert!(shows_state(&all, &record(state, serde_json::json!({}))));
}
assert!(shows_state(&decided, &record("approved", serde_json::json!({}))));
assert!(shows_state(&decided, &record("refused", serde_json::json!({}))));
assert!(!shows_state(&decided, &record("withdrawn", serde_json::json!({}))));
// Something that is not an aggregate has no state to filter on
// and is left alone.
assert!(shows_state(&decided, &serde_json::json!({ "name": "a file" })));
}
// The other half of a record belonging to a record: the
// application's page asking for the objections that name it.
#[test]
fn a_listing_can_be_narrowed_to_the_children_of_one_record() {
let children = spec("source: { kind: kv, bucket: objections }\nrequires_group: case_officers\nwhere: { field: case, equals: \"{key}\" }\n");
let params = std::collections::HashMap::from([("key".to_string(), "abc123".to_string())]);
let mine = record("open", serde_json::json!({ "case": "abc123", "your_objection": "the sun" }));
let theirs = record("open", serde_json::json!({ "case": "def456" }));
let neither = record("open", serde_json::json!({ "your_objection": "no case named" }));
assert!(matches_filter(&children, &mine, &params));
assert!(!matches_filter(&children, &theirs, &params));
// A record that did not say is not everyone's child.
assert!(!matches_filter(&children, &neither, &params));
// No filter, no narrowing: every resource written before this
// existed still lists its whole bucket.
let plain = spec("source: { kind: kv, bucket: objections }\npublic: true\n");
assert!(matches_filter(&plain, &theirs, &params));
}
#[test]
fn a_filter_can_name_a_value_outright() {
let fixed = spec("source: { kind: kv, bucket: cases }\npublic: true\nwhere: { field: which_authority, equals: The county }\n");
let params = std::collections::HashMap::new();
assert!(matches_filter(&fixed, &record("open", serde_json::json!({ "which_authority": "The county" })), &params));
assert!(!matches_filter(&fixed, &record("open", serde_json::json!({ "which_authority": "Fire service" })), &params));
}
/// A canned, Gitea-API-shaped fixture - the same fields the real /// A canned, Gitea-API-shaped fixture - the same fields the real
/// `/users/{username}/starred` endpoint returns (Gitea 1.27: /// `/users/{username}/starred` endpoint returns (Gitea 1.27:
/// `stars_count`, not GitHub's `stargazers_count`; `website` is "" /// `stars_count`, not GitHub's `stargazers_count`; `website` is ""
+122
View File
@@ -0,0 +1,122 @@
//! Sessions in NATS KV.
//!
//! A session used to live in a `MemoryStore`, which meant every
//! restart signed everybody out: a deploy logged out every desk on
//! every site, and the only reason that was bearable is that the desks
//! were three people. This keeps them in the JetStream KV that is
//! already under everything else here - no new service, no new
//! dependency, and the same bucket a second instance of a site would
//! read if there ever is one.
//!
//! Expiry is said twice on purpose. The bucket carries a `max_age`, so
//! NATS collects abandoned sessions without anyone sweeping; and a
//! record's own `expiry_date` is checked on the way out, because the
//! bucket's age is a coarse fence and the record's date is the truth.
#![cfg(feature = "ssr")]
use tower_sessions::session::{Id, Record};
use tower_sessions::session_store::{Error, Result};
/// How long an untouched session survives in the bucket. Comfortably
/// longer than any expiry `SessionManagerLayer` will set, so the
/// record's own date is always what decides; this is only here so a
/// session nobody ever comes back for does not sit in NATS forever.
const MAX_AGE: std::time::Duration = std::time::Duration::from_secs(60 * 60 * 24 * 40);
pub const BUCKET: &str = "portal_sessions";
/// A new id is drawn this many times before a collision is reported as
/// a backend error. Two random `i128`s colliding is not a thing that
/// happens; this exists so a bucket misbehaving cannot spin forever.
const ID_ATTEMPTS: usize = 8;
#[derive(Clone)]
pub struct NatsStore {
store: async_nats::jetstream::kv::Store,
}
impl std::fmt::Debug for NatsStore {
fn fmt(&self, f: &mut std::fmt::Formatter<'_>) -> std::fmt::Result {
f.debug_struct("NatsStore").field("bucket", &BUCKET).finish()
}
}
impl NatsStore {
/// Opens the bucket, creating it on first use the way every other
/// bucket here is created.
pub async fn open(js: &async_nats::jetstream::Context) -> anyhow::Result<Self> {
let store = match js.get_key_value(BUCKET).await {
Ok(store) => store,
Err(_) => {
js.create_key_value(async_nats::jetstream::kv::Config {
bucket: BUCKET.to_string(),
max_age: MAX_AGE,
history: 1,
..Default::default()
})
.await?
}
};
Ok(Self { store })
}
fn encode(record: &Record) -> Result<Vec<u8>> {
serde_json::to_vec(record).map_err(|e| Error::Encode(e.to_string()))
}
}
#[async_trait::async_trait]
impl tower_sessions::SessionStore for NatsStore {
/// `create` draws a fresh id until the bucket accepts one as new.
/// `put` would overwrite a stranger's session on a collision;
/// `create` refuses, which is the whole difference.
async fn create(&self, record: &mut Record) -> Result<()> {
for _ in 0..ID_ATTEMPTS {
let bytes = Self::encode(record)?;
match self.store.create(&record.id.to_string(), bytes.into()).await {
Ok(_) => return Ok(()),
Err(_) => record.id = Id::default(),
}
}
Err(Error::Backend(format!(
"could not find an unused session id in {ID_ATTEMPTS} attempts"
)))
}
async fn save(&self, record: &Record) -> Result<()> {
let bytes = Self::encode(record)?;
self.store
.put(&record.id.to_string(), bytes.into())
.await
.map(|_| ())
.map_err(|e| Error::Backend(e.to_string()))
}
async fn load(&self, id: &Id) -> Result<Option<Record>> {
let Some(bytes) = self
.store
.get(&id.to_string())
.await
.map_err(|e| Error::Backend(e.to_string()))?
else {
return Ok(None);
};
let record: Record =
serde_json::from_slice(&bytes).map_err(|e| Error::Decode(e.to_string()))?;
// An expired record is gone as far as anyone asking is
// concerned, whether or not the bucket has collected it yet.
if record.expiry_date <= tower_sessions::cookie::time::OffsetDateTime::now_utc() {
return Ok(None);
}
Ok(Some(record))
}
async fn delete(&self, id: &Id) -> Result<()> {
// Purge, not delete: a signed-out session should leave no
// history in the bucket to read back.
self.store
.purge(&id.to_string())
.await
.map_err(|e| Error::Backend(e.to_string()))
}
}
+283 -42
View File
@@ -57,13 +57,23 @@ pub struct UploadResponse {
key: String, key: String,
} }
/// POST /upload - streams the first `file` field straight through to /// POST /upload - takes the `file` field, plus the `question` and
/// Garage and hands back its object key. Server-side hygiene beyond the /// `alternative` it belongs to, and hands back its object key.
/// HTML `accept` hint (a UX-only signal, not a boundary): a hard size ///
/// cap, and the upload is rejected outright if Garage isn't configured /// The two names are not decoration: they are what makes this an
/// rather than silently succeeding nowhere. /// authorized route rather than an open one. An upload is half a
/// submission, so it asks the policy exactly what the submission will
/// ask - `Submit` on that page - and it only accepts a page that
/// actually declares a `type: file` field. Without that, anybody who
/// could reach the host could put twenty megabytes in the bucket as
/// often as they liked, on a site whose forms are public.
///
/// Server-side hygiene beyond the HTML `accept` hint (a UX signal, not
/// a boundary): a hard size cap, and a refusal rather than a silent
/// success when Garage isn't configured.
pub async fn upload( pub async fn upload(
State(state): State<AppState>, State(state): State<AppState>,
session: tower_sessions::Session,
mut multipart: Multipart, mut multipart: Multipart,
) -> Result<Json<UploadResponse>, (StatusCode, String)> { ) -> Result<Json<UploadResponse>, (StatusCode, String)> {
let garage = state.garage.as_ref().ok_or(( let garage = state.garage.as_ref().ok_or((
@@ -71,49 +81,280 @@ pub async fn upload(
"uploads are not configured".to_string(), "uploads are not configured".to_string(),
))?; ))?;
while let Some(field) = multipart let mut question_id = String::new();
let mut alternative = String::new();
let mut file: Option<(String, String, Vec<u8>)> = None;
while let Some(mut field) = multipart
.next_field() .next_field()
.await .await
.map_err(|e| (StatusCode::BAD_REQUEST, e.to_string()))? .map_err(|e| (StatusCode::BAD_REQUEST, e.to_string()))?
{ {
if field.name() != Some("file") { match field.name() {
continue; Some("question") | Some("alternative") => {
} let name = field.name().unwrap_or_default().to_string();
let content_type = field let text = field
.content_type() .text()
.unwrap_or("application/octet-stream") .await
.to_string(); .map_err(|e| (StatusCode::BAD_REQUEST, e.to_string()))?;
let filename = field.file_name().unwrap_or("upload").to_string(); if name == "question" {
// Chunk by chunk, refused the moment the cap is crossed: an question_id = text;
// oversized body is never buffered whole before being turned } else {
// away. The route's body limit (main.rs) is the outer fence. alternative = text;
let mut bytes: Vec<u8> = Vec::new(); }
let mut field = field;
while let Some(chunk) = field
.chunk()
.await
.map_err(|e| (StatusCode::BAD_REQUEST, e.to_string()))?
{
if bytes.len() + chunk.len() > MAX_UPLOAD_BYTES {
return Err((StatusCode::PAYLOAD_TOO_LARGE, "file too large".to_string()));
} }
bytes.extend_from_slice(&chunk); Some("file") => {
let content_type = field
.content_type()
.unwrap_or("application/octet-stream")
.to_string();
let filename = field.file_name().unwrap_or("upload").to_string();
// Chunk by chunk, refused the moment the cap is
// crossed: an oversized body is never buffered whole
// before being turned away. The route's body limit
// (main.rs) is the outer fence.
let mut bytes: Vec<u8> = Vec::new();
while let Some(chunk) = field
.chunk()
.await
.map_err(|e| (StatusCode::BAD_REQUEST, e.to_string()))?
{
if bytes.len() + chunk.len() > MAX_UPLOAD_BYTES {
return Err((StatusCode::PAYLOAD_TOO_LARGE, "file too large".to_string()));
}
bytes.extend_from_slice(&chunk);
}
file = Some((filename, content_type, bytes));
}
_ => continue,
} }
let key = format!("{}-{filename}", uuid::Uuid::new_v4());
garage
.client
.put_object()
.bucket(&garage.bucket)
.key(&key)
.body(ByteStream::from(bytes))
.content_type(content_type)
.send()
.await
.map_err(|e| (StatusCode::BAD_GATEWAY, format!("upload failed: {e}")))?;
return Ok(Json(UploadResponse { key }));
} }
Err((StatusCode::BAD_REQUEST, "no file field in request".to_string())) let (filename, content_type, bytes) =
file.ok_or((StatusCode::BAD_REQUEST, "no file field in request".to_string()))?;
// Nothing has been written yet. Everything below decides whether
// anything will be.
if !may_upload(&state, &session, &question_id, &alternative).await {
return Err((StatusCode::FORBIDDEN, "not authorized to upload here".to_string()));
}
let key = format!("{}-{filename}", uuid::Uuid::new_v4());
garage
.client
.put_object()
.bucket(&garage.bucket)
.key(&key)
.body(ByteStream::from(bytes))
.content_type(content_type)
.send()
.await
.map_err(|e| (StatusCode::BAD_GATEWAY, format!("upload failed: {e}")))?;
Ok(Json(UploadResponse { key }))
}
/// Whether this caller may attach a file to that page's alternative:
/// the alternative has to exist, has to declare a `type: file` field,
/// and the policy has to let this principal submit that page. The same
/// question `submit_answer` asks, asked before the bytes land rather
/// than after.
async fn may_upload(
state: &AppState,
session: &tower_sessions::Session,
question_id: &str,
alternative: &str,
) -> bool {
use crate::auth::{User, SESSION_USER_KEY};
let questions = state.questions.load();
let Some(page) = page_taking_files(&questions, question_id, alternative) else {
return false;
};
let user = session.get::<User>(SESSION_USER_KEY).await.ok().flatten();
let principal = crate::access::Principal::from_user(user.as_ref());
crate::access::decide_and_publish(
&state.nats,
&state.access.load(),
&principal,
crate::access::Action::Submit,
&crate::access::Resource::Page { id: page },
serde_json::json!({ "alternative": alternative }),
)
.allow
}
/// The resolved page id, when that page has an alternative of this
/// name that is open and actually asks for a file. `None` for a page
/// that does not exist, an alternative that does not, one that is
/// disabled, or one that asks for no file at all - an upload that
/// names none of the site's own file fields is not an upload this site
/// asked for.
fn page_taking_files(
questions: &std::collections::HashMap<String, crate::content::Question>,
question_id: &str,
alternative: &str,
) -> Option<String> {
let question = crate::content::resolve_question(questions, question_id)?;
question
.alternatives
.iter()
.find(|a| a.name == alternative && !a.disabled)?
.features
.iter()
.any(|f| f.requirements.iter().any(|r| r.kind == "file"))
.then(|| question.id.clone())
}
#[cfg(test)]
mod tests {
use super::*;
fn questions(yaml: &str) -> std::collections::HashMap<String, crate::content::Question> {
crate::content::build_questions(&[("index.yaml".to_string(), yaml.to_string())]).unwrap()
}
const PAGE: &str = "name: Home
alternatives:
- name: Send drawings
record_as: enquiries
features:
- name: Files
requirements:
- { name: drawings, type: file, optional: true }
- name: Just ask
features:
- name: Words
requirements:
- { name: question, type: textarea }
";
// The bytes are only ever written for a page of this site that
// asked for a file. Everything else is somebody using the host as
// a disk.
#[test]
fn an_upload_must_name_a_page_that_asked_for_a_file() {
let qs = questions(PAGE);
assert_eq!(page_taking_files(&qs, "/", "Send drawings"), Some("/".to_string()));
// An alternative on the same page that takes no file.
assert_eq!(page_taking_files(&qs, "/", "Just ask"), None);
// Names that are not there at all.
assert_eq!(page_taking_files(&qs, "/", "Nonexistent"), None);
assert_eq!(page_taking_files(&qs, "/nowhere", "Send drawings"), None);
assert_eq!(page_taking_files(&qs, "", ""), None);
}
#[test]
fn a_closed_alternative_takes_nothing() {
let qs = questions(&PAGE.replace(" record_as: enquiries\n", " record_as: enquiries\n disabled: true\n"));
assert_eq!(page_taking_files(&qs, "/", "Send drawings"), None);
}
}
/// GET /attachment/{bucket}/{record}/{field} - the other half of an
/// upload. A `type: file` field stores an object key, which on its own
/// is a string a desk can read and nobody can open; this streams the
/// object back.
///
/// The key is never the credential. It is not even taken from the
/// caller: the caller names a record, and the key is read out of that
/// record's own field, so a file can only be fetched through a record
/// somebody is allowed to see. Guessing keys gets you nowhere, and a
/// key that leaks out of a desk is not a link to anything.
pub async fn attachment(
State(state): State<AppState>,
session: tower_sessions::Session,
axum::extract::Path((bucket, record, field)): axum::extract::Path<(String, String, String)>,
) -> Result<axum::response::Response, (StatusCode, String)> {
use axum::response::IntoResponse;
let garage = state.garage.as_ref().ok_or((
StatusCode::SERVICE_UNAVAILABLE,
"uploads are not configured".to_string(),
))?;
// Refusals read the same as a miss all the way down, so this
// cannot be used to learn which buckets, records or fields exist.
let missing = || (StatusCode::NOT_FOUND, "not found".to_string());
if !may_read_bucket(&state, &session, &bucket).await {
return Err(missing());
}
let store = state.jetstream.get_key_value(&bucket).await.map_err(|_| missing())?;
let bytes = store.get(&record).await.map_err(|_| missing())?.ok_or_else(missing)?;
let answer: crate::answers::Answer = serde_json::from_slice(&bytes).map_err(|_| missing())?;
let key = answer
.responses
.get(&field)
.and_then(|v| v.as_str())
.filter(|k| !k.is_empty())
.ok_or_else(missing)?;
let object = garage
.client
.get_object()
.bucket(&garage.bucket)
.key(key)
.send()
.await
.map_err(|_| missing())?;
let content_type = object
.content_type()
.unwrap_or("application/octet-stream")
.to_string();
let body = object
.body
.collect()
.await
.map_err(|e| (StatusCode::BAD_GATEWAY, format!("reading the file failed: {e}")))?
.into_bytes();
// The stored key is `<uuid>-<original name>`; the person who gets
// the file should see the name they sent, not the uuid.
let filename = key.split_once('-').map(|(_, rest)| rest).unwrap_or(key);
let filename = filename.rsplit('/').next().unwrap_or(filename).replace(['"', '\\'], "");
Ok((
[
(axum::http::header::CONTENT_TYPE, content_type),
(
axum::http::header::CONTENT_DISPOSITION,
format!("attachment; filename=\"{filename}\""),
),
],
body,
)
.into_response())
}
/// Whether this caller may see this bucket at all: whether any feature
/// in the loaded content lists it and the policy lets them read that
/// feature. The desk that shows the record is what grants the file.
async fn may_read_bucket(
state: &AppState,
session: &tower_sessions::Session,
bucket: &str,
) -> bool {
use crate::auth::{User, SESSION_USER_KEY};
let user = session.get::<User>(SESSION_USER_KEY).await.ok().flatten();
let principal = crate::access::Principal::from_user(user.as_ref());
let access = state.access.load();
state.questions.load().values().any(|question| {
question.alternatives.iter().any(|alternative| {
alternative.features.iter().any(|feature| {
feature.resource.as_ref().and_then(|r| r.bucket()) == Some(bucket)
&& access.allows(
&principal,
crate::access::Action::Read,
&crate::access::Resource::Feature {
page: question.id.clone(),
alternative: alternative.name.clone(),
feature: feature.name.clone(),
},
)
})
})
})
} }
+7
View File
@@ -1227,3 +1227,10 @@ textarea:focus {
font-size: 0.9em; font-size: 0.9em;
color: var(--ink-dim); color: var(--ink-dim);
} }
/* A loading slot holds its line's worth of space and says nothing.
The words that belong here have not arrived yet, and "loading…" is
not one of them. */
.resource-loading {
min-height: 1.4em;
}
+61 -1
View File
@@ -68,6 +68,7 @@ async fn a_submission_is_created_moved_refused_and_read_back_through_the_real_en
// through the same store the desks read - and moves nothing: the // through the same store the desks read - and moves nothing: the
// record is still where the desk left it. // record is still where the desk left it.
let mut record: Answer = serde_json::from_slice(&store.get("rec-1").await.unwrap().expect("record")).unwrap(); let mut record: Answer = serde_json::from_slice(&store.get("rec-1").await.unwrap().expect("record")).unwrap();
let state_before_the_note = record.state.clone();
let reply = portal::mail::Incoming { let reply = portal::mail::Incoming {
v: portal::mail::CONTRACT, v: portal::mail::CONTRACT,
id: "mail-1".into(), id: "mail-1".into(),
@@ -86,10 +87,69 @@ async fn a_submission_is_created_moved_refused_and_read_back_through_the_real_en
assert_eq!(read_back.notes.len(), 1); assert_eq!(read_back.notes.len(), 1);
assert_eq!(read_back.notes[0].text, "Thursday suits us."); assert_eq!(read_back.notes[0].text, "Thursday suits us.");
assert_eq!(read_back.notes[0].from, "a@b.example"); assert_eq!(read_back.notes[0].from, "a@b.example");
assert_eq!(read_back.state, "quoted", "a reply is a note, never a decision"); assert_eq!(read_back.state, state_before_the_note, "a reply is a note, never a decision");
// The same reply read twice adds nothing. // The same reply read twice adds nothing.
assert!(portal::mail::note_for(&reply, &read_back).is_err()); assert!(portal::mail::note_for(&reply, &read_back).is_err());
// And the event log is untouched by it. // And the event log is untouched by it.
let after = load_events(&js, &bucket, "rec-1").await.expect("events"); let after = load_events(&js, &bucket, "rec-1").await.expect("events");
assert_eq!(after.len(), 3, "a note writes no event"); assert_eq!(after.len(), 3, "a note writes no event");
} }
/// A session outlives the process that made it. The `MemoryStore` this
/// replaced could not: every deploy signed out every desk, and the
/// only proof that this one does not is a store opened twice against
/// the same JetStream, as two runs of the binary would.
#[tokio::test]
async fn a_session_survives_the_process_that_made_it() {
use tower_sessions::session::{Id, Record};
use tower_sessions::SessionStore;
let Ok(url) = std::env::var("PORTAL_TEST_NATS_URL") else {
eprintln!("skipped: PORTAL_TEST_NATS_URL is not set");
return;
};
let js = async_nats::jetstream::new(async_nats::connect(&url).await.expect("connect"));
let store = portal::sessions::NatsStore::open(&js).await.expect("open");
let mut record = Record {
id: Id::default(),
data: Default::default(),
expiry_date: tower_sessions::cookie::time::OffsetDateTime::now_utc()
+ tower_sessions::cookie::time::Duration::hours(1),
};
record.data.insert("who".into(), json!("ingrid"));
store.create(&mut record).await.expect("create");
// A second store over the same JetStream is what the next boot of
// the binary has: nothing carried over in memory.
let next_boot = portal::sessions::NatsStore::open(&js).await.expect("reopen");
let found = next_boot.load(&record.id).await.expect("load").expect("still there");
assert_eq!(found.data.get("who"), Some(&json!("ingrid")));
// `create` never writes over somebody else's session: asked to
// create one whose id is taken, it takes a new id instead.
let mut collide = Record { id: record.id, data: Default::default(), expiry_date: record.expiry_date };
next_boot.create(&mut collide).await.expect("create beside it");
assert_ne!(collide.id, record.id, "an id already in use must not be reused");
assert_eq!(
next_boot.load(&record.id).await.expect("load").expect("original intact").data.get("who"),
Some(&json!("ingrid")),
"the original session must be untouched"
);
// An expired record is gone to anyone asking, whatever the bucket
// still holds.
let stale = Record {
id: Id::default(),
data: Default::default(),
expiry_date: tower_sessions::cookie::time::OffsetDateTime::now_utc()
- tower_sessions::cookie::time::Duration::minutes(1),
};
next_boot.save(&stale).await.expect("save stale");
assert!(next_boot.load(&stale.id).await.expect("load stale").is_none());
// Signing out leaves nothing to read back.
next_boot.delete(&record.id).await.expect("delete");
assert!(next_boot.load(&record.id).await.expect("load after delete").is_none());
next_boot.delete(&collide.id).await.ok();
}