Compare commits

..
24 Commits
Author SHA1 Message Date
Bendik LynghaugandClaude Opus 5 c9faaeb4a6 Serve a content repo's .well-known/ at the site root
docs / deploy (push) Successful in 2s
Test / test (push) Successful in 2m3s
Publish release / publish (push) Successful in 1m51s
Everything a site has to publish for another program to find it lives
under /.well-known/, and portal had no way to put a file there: the
catch-all answered those paths 200 with the app shell, so a client
probing for security.txt or matrix/server got HTML where it expected a
document, and could not tell a missing file from a present one.

The prefix is the one place this is safe to overlay. RFC 8615 reserves
it, and a question's id comes from a file name under questions/, which
cannot begin with a dot - so no file a repo ships can shadow a page. The
path still goes through is_safe_site_path, so it cannot climb out of the
repo either, and a miss is a real 404.

Types: an extensionless file under the prefix is JSON, which is what
matrix/server and nodeinfo are; .mobileconfig is the Apple profile type,
without which iOS downloads a profile instead of offering to install it.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_015GeH3y8fwTaTudaD88P57o
2026-10-07 10:30:13 +02:00
Bendik Aagaard LynghaugandClaude Fable 5.1 1a2c064d3a Release 0.6.20
docs / deploy (push) Successful in 2s
Test / test (push) Successful in 2m11s
Publish release / publish (push) Successful in 1m55s
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
2026-10-05 12:56:30 +02:00
Bendik Aagaard LynghaugandClaude Fable 5.1 fe7a868b18 A platform's file may name its architecture
A Mac is offered the macos-arm64 file rather than sent past it to
TestFlight.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
2026-10-05 12:56:19 +02:00
Bendik Aagaard LynghaugandClaude Fable 5.1 aee5048b01 Release 0.6.19
docs / deploy (push) Successful in 3s
Publish release / publish (push) Successful in 2m50s
Test / test (push) Successful in 2m10s
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
2026-10-05 12:48:42 +02:00
Bendik Aagaard LynghaugandClaude Fable 5.1 268d545843 A releases manifest renders as a download card, the device's file first
A kind: url resource whose JSON is a releases manifest - latest,
versions newest first with per-platform assets, channels such as a
package repo or TestFlight - is one card: the newest version, the file
for the visitor's own platform (read from the browser on hydration;
the server renders every file), the Linux package repo lines beside a
Linux file, TestFlight for iPhones and Macs, and the rest folded under
"All downloads". Decided by shape, as every resource is; written for
redoal's public releases at dl.redoal.com.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
2026-10-05 12:48:31 +02:00
Bendik Aagaard LynghaugandClaude Opus 5 22d086f9e1 A resource can show only its newest few
docs / deploy (push) Successful in 2s
Test / test (push) Successful in 2m0s
`limit: n` on a resource spec, applied after `jq`.

The releases card is what asked for it: a page that grows by one card
per release gets longer forever, and the cut a visitor should run is the
newest one — the rest are history, not an invitation. redoal's card had
every release on it.

After `jq` rather than before, for two reasons: it caps what the page
will actually render whatever the filter did, and content that reorders
with the filter can then take the top of its own ordering rather than
the top of the source's. A source yielding a single record has no length
and passes through untouched — turning one record into an empty list
would be worse than ignoring a limit that cannot apply. `limit: 0` on a
list does mean nothing, and the `empty` line speaks instead, which is a
coherent thing for content to ask for.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01ByshXuDEzH8hXh5qp4VgGk
2026-10-04 22:51:12 +02:00
Bendik Aagaard LynghaugandClaude Opus 5.5 48ae5e07c2 Release 0.6.18
docs / deploy (push) Successful in 2s
Test / test (push) Successful in 2m0s
Publish release / publish (push) Successful in 1m50s
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-03 10:48:22 +02:00
Bendik Aagaard LynghaugandClaude Opus 5.5 b619926a11 generate follows its findings through, and stops when it stalls
Each structure round is logged as fixed, still open, new and reopened;
a fix round sees the open findings with their fixes and only the files
they name; a change that reopens a closed finding is undone; a round
that closes nothing ends the phase with what is left in run.log. A
hesitant visitor alone no longer triggers a wording rewrite.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-03 10:48:09 +02:00
Bendik Aagaard LynghaugandClaude Opus 5.5 706c8108df iris check --findings: every finding as JSON with a stable id
The id hashes scope, need, code and subject, nothing of the wording, so
the same gap keeps its id across runs. Every message that knows its fix
says it after "fix:". Codes listed in docs/lint.md.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-03 10:48:09 +02:00
Bendik Aagaard LynghaugandClaude Opus 5.5 2feeea0810 Release 0.6.17
docs / deploy (push) Successful in 2s
Test / test (push) Successful in 2m12s
Publish release / publish (push) Successful in 1m55s
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01MznPTC31w5mSUK4XMehcpi
2026-10-02 11:26:13 +02:00
Bendik Aagaard LynghaugandClaude Opus 5.5 6b91258ebe One set of mail variables for every mail, and {link} in state mails
{site} and {site_name} everywhere; a state mail adds {self_link} (the
submitter's own page, when their record's state allows a move there),
{desk_link} and {link}, which picks by who the mail is to. gullberg's
{link} mails went out blank until now. Documented in docs/mail.md.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01MznPTC31w5mSUK4XMehcpi
2026-10-02 11:25:49 +02:00
Bendik Aagaard LynghaugandClaude Opus 5.5 ae119af987 A desk shows the three newest finished records, and older on request
Every record still moving stays; of those in a final state only the
three most recently decided, plus one so the page knows to offer
"Show older", three more at a time.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01MznPTC31w5mSUK4XMehcpi
2026-10-02 11:25:49 +02:00
Bendik Aagaard LynghaugandClaude Opus 5.5 92dd9bfc79 The lint walks every state of every bucket a need lands in
Clocks count as moves and a visitor's link only from its own states.
Per bucket: a state nothing reaches fails, a state a record can reach
and never leave fails (after done too), a declared move nobody offers
warns, a desk gated twice warns, and a mail from an unreachable state
or naming a field no form collects is caught. Working stages must
follow each other. Every finding ends in its fix, and a schema error no
longer hides the needs findings.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01MznPTC31w5mSUK4XMehcpi
2026-10-02 11:25:49 +02:00
Bendik Aagaard LynghaugandClaude Opus 5.5 2cc74e3354 CI builds and lints every plan with this portal
Test / test (push) Successful in 2m35s
The check portal-trainer's CI ran with the standalone trainer, which
kept failing on a builder bug fixed here long ago. Now it runs where the
trainer lives, with the same binary.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01MznPTC31w5mSUK4XMehcpi
2026-10-02 09:27:44 +02:00
Bendik Aagaard Lynghaug 7b54c4c6d7 Release 0.6.16
Test / test (push) Successful in 1m8s
Publish release / publish (push) Successful in 1m56s
2026-10-01 21:46:45 +02:00
bl 951fac9aec Merge pull request 'Test fixtures get the action the lint now requires' (#28) from fixture-action into main
Test / test (push) Successful in 1m10s
2026-10-01 19:46:29 +00:00
Bendik Aagaard LynghaugandClaude Fable 5.1 738dbac7fe Test fixtures: every form has the action the lint now requires
Test / test (pull_request) Successful in 21s
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01GT37Z1Xtfd9pUuQtMTg6Yt
2026-10-01 21:46:24 +02:00
bl 98f93f5ad5 Merge pull request 'A form with no action is refused; portal iris' (#27) from form-needs-action into main
docs / deploy (push) Successful in 3s
Test / test (push) Failing after 1m7s
2026-10-01 19:42:03 +00:00
Bendik Aagaard LynghaugandClaude Opus 5.5 4049809aa4 The lint refuses a form with no action, which would have no button; portal iris names the lint
Test / test (pull_request) Failing after 1m11s
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01GT37Z1Xtfd9pUuQtMTg6Yt
2026-10-01 21:41:56 +02:00
Bendik Aagaard Lynghaug b43ff57eab Release 0.6.15
Test / test (push) Successful in 1m14s
Publish release / publish (push) Successful in 2m5s
2026-10-01 18:06:16 +02:00
bl ed235cd134 Merge pull request 'An invite may carry its own mail' (#26) from invite-mail into main
docs / deploy (push) Successful in 3s
Test / test (push) Successful in 1m52s
2026-10-01 16:06:14 +00:00
Bendik Aagaard LynghaugandClaude Opus 5.5 cc9f1c5d87 An invite may carry its own mail, with the form's fields and a spoken date and time
Test / test (pull_request) Successful in 1m9s
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01GT37Z1Xtfd9pUuQtMTg6Yt
2026-10-01 18:06:12 +02:00
bl 01abf79005 Merge pull request 'Klingenberg Bygg plan: Fredrik's caption corrections' (#25) from kb-fredrik-captions into main
Test / test (push) Successful in 1m9s
2026-10-01 15:57:10 +00:00
Bendik Aagaard LynghaugandClaude Opus 5.5 506bb712fe Klingenberg Bygg plan: Fredrik's two caption corrections from the live site
Test / test (pull_request) Successful in 1m10s
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01GT37Z1Xtfd9pUuQtMTg6Yt
2026-10-01 17:57:08 +02:00
29 changed files with 1847 additions and 119 deletions
+19
View File
@@ -38,6 +38,25 @@ jobs:
- name: Test - name: Test
run: cargo test --features ssr run: cargo test --features ssr
# Every plan under plans/ built deterministically and held to the
# lint, both by this very portal: a builder change that renders a
# site the lint rejects fails here, not at the next training run.
# (portal-trainer's CI did this until the trainer moved in.)
- name: Build and lint every plan
run: |
set -euo pipefail
cargo build --features ssr --bin portal
bin="$CARGO_TARGET_DIR/debug/portal"
tmp=$(mktemp -d)
lint="$tmp/portal-lint"
printf '#!/bin/sh\nexec %s lint "$@"\n' "$bin" > "$lint" && chmod +x "$lint"
for plan in plans/*.yaml; do
name=$(basename "$plan" .yaml)
echo "== $plan"
rm -rf "$tmp/plan-$name"
"$bin" build --plan "$plan" --out "$tmp/plan-$name" --lint "$lint"
done
- name: Stop the throwaway JetStream - name: Stop the throwaway JetStream
if: always() if: always()
run: docker rm -f portal-test-nats >/dev/null 2>&1 || true run: docker rm -f portal-test-nats >/dev/null 2>&1 || true
+32
View File
@@ -4,6 +4,38 @@ One line per change, grouped by the release that shipped it. Newest
first. A release is a Cargo.toml bump plus a `v<version>` tag; the first. A release is a Cargo.toml bump plus a `v<version>` tag; the
content repos pin the tag they run (see the end of this file). content repos pin the tag they run (see the end of this file).
## 0.6.21 (2026-10-07)
- A content repo's `.well-known/` is served at the site root (`/.well-known/security.txt`, `/.well-known/assetlinks.json`, `/.well-known/matrix/server`), and a miss there is a 404 instead of the app shell answering 200 with an HTML page. RFC 8615 reserves the prefix and a question's id cannot begin with a dot, so nothing a repo ships can shadow a page. An extensionless file under it is typed as JSON, by the convention those files follow; `.txt`, `.xml`, `.webmanifest` and `.mobileconfig` are typed too, the last so an Apple configuration profile offers to install rather than download.
## 0.6.20 (2026-10-05)
- A releases card lets a platform's file carry an architecture: a Mac visitor is offered the `macos-arm64` file, not sent to TestFlight past it.
## 0.6.19 (2026-10-05)
- A `kind: url` resource whose JSON is a releases manifest (`latest`, `versions` with per-platform `assets`, `channels` such as a package repo or TestFlight) renders as a download card: the file for the visitor's own platform first, decided in the browser, the rest and the channels folded. Written for redoal's public releases at dl.redoal.com (docs: content-reference, "A releases manifest").
## 0.6.18 (2026-10-03)
- `iris check --findings <out.jsonl>` writes every finding as JSON: a stable `id` (scope, need, code and subject, nothing of the wording), `level`, `code`, `scope`, `need`, `subject`, `message` and `fix`. Every message that knows its fix now says it after "fix:". The codes are listed in docs/lint.md.
- `portal generate` follows its findings through: each structure round is logged as fixed, still open, new and reopened; a fix round is shown the open findings with their fixes and only the files they name; a change that reopens a closed finding is undone; and the phase stops when a round closes nothing, writing what is left to run.log. A hesitant visitor no longer triggers a wording rewrite by itself, only a misroute does. A lint without `--findings` is still driven the old way.
## 0.6.17 (2026-10-02)
- A desk shows every record still moving and only the three most recently finished (a final state: no move out of it), with "Show older" for three more at a time. A desk that has run for a year is still about the work in hand.
- Mail variables are one set for every kind of mail (docs: Mail). Every mail has `{site}` and `{site_name}`; a state mail also has `{link}`, `{self_link}` and `{desk_link}`: the reader's own page for the record (a self-service page with `?chain=&email=`, when the record's state allows a move there), the desk that moves it, and `{link}` choosing between them by who the mail is to. `{link}` in a state mail used to render as nothing.
- The lint follows every state of every bucket a need lands in: a state no desk, link or clock reaches fails; a state a record can reach and never leave fails, after "done" too; a declared move nobody offers warns; a desk gated to one group on the page and another on its list warns; working stages must follow each other in the business's order; a mail from an unreachable state warns, and one naming a field no form collects fails. Clocks (`after:`) count as moves, and a visitor's own link only from the states it names. Each finding ends in the fix: the `{ from, to, label }` and the desk page it goes on, or the group that already has the button. A schema error no longer hides the needs findings.
## 0.6.16 (2026-10-01)
- The lint refuses an alternative that has a form to send (fields to fill in, an invite, or a bucket to record into) and no `action`. The button is only drawn for an alternative with an `action`, so such a form could be filled in and never sent, which is what happened to uhhm.no's first invite form. An alternative that moves its own record (`self_transition`) draws its own button and is left alone.
- `portal iris` is the lint's name, after the iris that keeps bad content out of the portal. `portal lint` is the same command and keeps working.
## 0.6.15 (2026-10-01)
- An invite may carry its own mail: `invite: { group, mail: { subject, body } }`. Every field of the form is a placeholder besides `{name}`, `{link}`, `{expires}` and the rest, and a date and time (`type: datetime-local`) is written out in words ("tirsdag 6. oktober kl. 15:00"). The words and the form are kept with the invite, so sending it again sends the same mail. This is for a desk that invites someone to a conversation at an hour, not into a group for good.
## 0.6.14 (2026-10-01) ## 0.6.14 (2026-10-01)
- Less room above the wordmark: 1.75rem, 1.25rem on a narrow screen (it was 4rem). The space was kept for who is here, which now sits on the wordmark's own line, and on a tall, narrow screen it read as dead space. - Less room above the wordmark: 1.75rem, 1.25rem on a narrow screen (it was 4rem). The space was kept for who is here, which now sits on the wordmark's own line, and on a tall, narrow screen it read as dead space.
Generated
+1 -1
View File
@@ -3357,7 +3357,7 @@ dependencies = [
[[package]] [[package]]
name = "portal" name = "portal"
version = "0.6.14" version = "0.6.21"
dependencies = [ dependencies = [
"anyhow", "anyhow",
"arc-swap", "arc-swap",
+2 -1
View File
@@ -1,6 +1,6 @@
[package] [package]
name = "portal" name = "portal"
version = "0.6.14" version = "0.6.21"
edition = "2021" edition = "2021"
[lib] [lib]
@@ -60,6 +60,7 @@ web-sys = { version = "0.3", features = [
"FileList", "FileList",
"FormData", "FormData",
"HtmlInputElement", "HtmlInputElement",
"Navigator",
"Request", "Request",
"RequestInit", "RequestInit",
"Response", "Response",
+1
View File
@@ -7,6 +7,7 @@
- [Getting started](getting-started.md) - [Getting started](getting-started.md)
- [Content reference](content-reference.md) - [Content reference](content-reference.md)
- [State machines](state-machines.md) - [State machines](state-machines.md)
- [Mail](mail.md)
- [Access](access.md) - [Access](access.md)
- [Storylines](storylines.md) - [Storylines](storylines.md)
- [Memberships](memberships.md) - [Memberships](memberships.md)
+33
View File
@@ -125,6 +125,10 @@ answer:
from: [open, incomplete] # defaults to the initial state alone from: [open, incomplete] # defaults to the initial state alone
label: Send them label: Send them
invite: { group: case_officers } # onboard someone into Kanidm invite: { group: case_officers } # onboard someone into Kanidm
# or with words of its own, using any field of the form:
# invite:
# group: guests
# mail: { subject: "...", body: "... {tidspunkt} ... {link} ..." }
``` ```
`self_transition.from` is what keeps a person in dialogue with their `self_transition.from` is what keeps a person in dialogue with their
@@ -251,6 +255,7 @@ reads is never a client-supplied parameter.
states: [approved, refused] # only these; empty means all states: [approved, refused] # only these; empty means all
where: { field: case, equals: "{key}" } where: { field: case, equals: "{key}" }
jq: ".[] | {name, url: .html_url}" jq: ".[] | {name, url: .html_url}"
limit: 1 # at most this many, after jq
empty: Nothing decided yet. empty: Nothing decided yet.
link: { to: "/notices/{id}", label: Answer this } # a page per record link: { to: "/notices/{id}", label: Answer this } # a page per record
transitions: # the desk's buttons transitions: # the desk's buttons
@@ -260,6 +265,14 @@ reads is never a client-supplied parameter.
`source.kind` is one of `kv`, `gitea_starred`, `source.kind` is one of `kv`, `gitea_starred`,
`gitea_releases`, `url`. `gitea_releases`, `url`.
`limit: 1` shows only the first item — for a feed where only the newest
matters. A releases card wants the cut someone should run, not every
cut there has ever been, and a page that grows by one card per release
gets longer forever. It applies *after* `jq`, so a filter can reorder
and the limit takes the top of that ordering. A source that yields a
single record has no length and ignores it; `limit: 0` on a list shows
nothing, and the `empty` line speaks instead.
A spec with neither `public: true` nor `requires_group` is A spec with neither `public: true` nor `requires_group` is
unreachable: reads fail closed, and mutations always need a group unreachable: reads fail closed, and mutations always need a group
whatever `public` says. whatever `public` says.
@@ -281,6 +294,26 @@ markdown on stricter rules than the site's own: paragraphs, line
breaks as typed, lists, emphasis and links to https, mailto or tel breaks as typed, lists, emphasis and links to https, mailto or tel
(marked `nofollow ugc`); no images, no raw HTML. (marked `nofollow ugc`); no images, no raw HTML.
A desk (a list with `transitions`) shows every record still moving,
and of those in a final state - one the state machine has no move out
of - only the three most recently decided, newest first. "Show older"
under the list brings three more each time it is pressed. The desk
stays about the work in hand however long the site has run; nothing
is hidden from a read-only list, and `leaves_after` still takes a
record out of every list altogether.
A **releases manifest** - a `kind: url` resource whose JSON is
`{ name, latest, versions: [{ version, date, notes, assets: [{ platform,
url, sha256, size }] }], channels: { arch: { repo, package, note },
ios: { testflight, note } } }`, newest version first, as a release
pipeline writes it - renders as one card: the newest version, the file
for the visitor's own platform first (read from the browser: `ios`,
`macos`, `linux-x86_64`, `linux-aarch64`, `windows`, `android`), the
other files and the channels folded under "All downloads". Linux gets
the package repo lines beside its file; iPhones and Macs get
TestFlight. Platforms a manifest lacks fall through to the folded
list.
`states` and `where` narrow a listing. They matter most for a register `states` and `where` narrow a listing. They matter most for a register
that is public by law — public about what was decided, not about what that is public by law — public about what was decided, not about what
was withdrawn — while the desk working the same bucket is a different was withdrawn — while the desk working the same bucket is a different
+42 -2
View File
@@ -4,11 +4,15 @@ The iris over a site. A content push is an incoming wormhole; nothing
comes through until it checks out. comes through until it checks out.
``` ```
portal lint --path questions # a local checkout portal iris --path questions # a local checkout
portal lint --repo https://project.uhhm.no/uhhm/questions portal lint --repo https://project.uhhm.no/uhhm/questions
portal lint replay --path questions --nats nats://127.0.0.1:14222 portal lint replay --path questions --nats nats://127.0.0.1:14222
``` ```
`portal iris` is the lint's name, after the iris that keeps bad content
out of the portal; `portal lint` runs the same thing, so workflows that
already call it keep working.
It was its own binary, iris, until portal 0.6: the same code, released It was its own binary, iris, until portal 0.6: the same code, released
as portal's twin. Now it is the `portal` binary's `lint` command, so a as portal's twin. Now it is the `portal` binary's `lint` command, so a
content repo's CI downloads the release and runs it: content repo's CI downloads the release and runs it:
@@ -30,7 +34,43 @@ in `needs.yaml`, when it has one:
- the handling group holds a desk over that bucket, and every - the handling group holds a desk over that bucket, and every
finished state is reachable through the buttons it (and any finished state is reachable through the buttons it (and any
`also_moved_by` group) offers; no record strands on the way; `also_moved_by` group) offers; no record strands on the way;
- every stage the business named is a state of the machine. - every stage the business named is a state of the machine, and each
working stage can follow the one before it;
- across each bucket, whichever need brought a record in: every state
is somewhere a record can get to, no state a record gets to holds it
with no way on (before done or after), every declared move is a
button, a visitor's own link or a clock (a move nobody offers warns),
no desk is gated to one group on the page and another on its list,
and every mail goes out from a state records reach and names only
fields the forms collect.
Every finding says what to change and where: the missing
`{ from, to, label }` and the desk page it goes on, the group that
already has the button, or the edge aggregates.yaml lacks. A schema
error no longer hides the needs findings - both are printed, so one
round can fix them together.
`--findings <out.jsonl>` writes the same findings one JSON object per
line: `id`, `level` (`fail`/`warn`), `code`, `scope` (`need`,
`bucket`, or `repo` for a schema error), `need` (the need's id, or the
bucket's name), `subject` (the state, the move as `from>to`, or the
page), `message` and `fix`. The file is written empty when the content
is clean. `id` hashes scope, need, code and subject and nothing of the
wording, so the same gap keeps its id from one run to the next: a
trainer can tell what a round closed, what is still open and what it
broke. The codes:
| code | |
|---|---|
| `schema` | content the site would refuse to load |
| `need-duplicate`, `persona-unknown-need` | needs.yaml itself |
| `entry-missing`, `entry-gated`, `entry-no-responsible` | the page a visitor arrives on |
| `path-missing`, `path-closed`, `path-over-budget`, `path-no-action` | getting the answer recorded |
| `machine-missing`, `desk-missing`, `desk-gated-twice` | who works the bucket |
| `done-unknown`, `done-unreachable`, `success-not-done` | finishing a case |
| `stage-unknown`, `stage-unreachable`, `stage-order` | the business's own stages |
| `state-unreached`, `strand`, `edge-unoffered` | every state and move of the bucket |
| `mail-unreachable`, `mail-unknown-field` | what a state mails |
A need marked `planned` warns instead of failing. Exit status is the A need marked `planned` warns instead of failing. Exit status is the
verdict, so a content repo's CI is one line. verdict, so a content repo's CI is one line.
+59
View File
@@ -0,0 +1,59 @@
# Mail
Portal writes every mail itself and hands it to gdo to deliver
(`portal.mail.send`); a reply comes back as a note beside the record
(`portal.mail.received`). A mail is a subject and a markdown body with
`{placeholders}`, sent as text with an HTML alternative. The kinds of
mail are the ones content declares on a state (see
[State machines](state-machines.md#mail)), the invite and its
rewording (`invite: { group, mail }`, `mail.invite` in site.yaml), the
note to a person named responsible for a question, and the answers to
a suggested change.
## Variables
A placeholder is `{` letters, digits, `_`, `-` or `.` `}`. One that
names nothing renders as nothing; a brace that opens no name stays a
brace. The variables come in layers, so a template written for one
kind of mail reads the same in another.
**On every mail**
| | |
|---|---|
| `{site}` | the instance's public URL, no trailing slash |
| `{site_name}` | site.yaml's `title` |
**On a mail about a record** (every state mail)
| | |
|---|---|
| `{field}` | any field of the record, as the form sent it |
| `{chain}` | the record's id |
| `{email}` | the record's `email` field, trimmed |
| `{bucket}`, `{state}` | where it is and the state it just entered |
| `{notice.field}` | a field of the record this one belongs to, through its `type: record` field `notice` |
| `{self_link}` | the record's own page for the person it belongs to: a page offering a self-service move (`self_transition`) on the bucket from the state the record is in, with the `?chain=&email=` that lets them act without signing in. Empty when there is no such page or no address |
| `{desk_link}` | the desk that moves the record: the page whose list over the bucket has `transitions`, else one that reads it. Empty when no page reads the bucket |
| `{link}` | where this reader acts: `{self_link}` in a mail `to: submitter`, `{desk_link}` in one to anyone else, and `{site}` when that link is empty |
`{link}` is the one to reach for. "Se meldingen og avtal befaring:
{link}" sent to the caretaker opens the desk the report is on; "Feilen
du meldte om er utbedret. {link}" sent to the tenant opens their own
case where they can still act on it, or the site when there is
nothing left for them to do.
**On an invite and the note to someone responsible**, beside every
field of the invite's form: `{name}`, `{username}`, `{email}`,
`{group}` (invite), `{questions}` (responsible), `{expires}`, and
`{link}`, which here is the one-time sign-in link.
## What the lint holds a mail to
A state mail that sends from a state no record reaches warns; one that
names a field no form recording into the bucket collects fails, since
it would go out with a blank where the field should be. The lint
knows every variable above, so `{link}` and the rest are never
flagged. `to: submitter` requires a required `email` on every form
filling the bucket, and `to: "{notice.email}"` requires the `notice`
field.
+3 -1
View File
@@ -57,7 +57,9 @@ a record arrives in it.
``` ```
`{field}` is any field of the record, plus `{site}`, `{chain}`, `{field}` is any field of the record, plus `{site}`, `{chain}`,
`{email}`, `{bucket}` and `{state}`. The body is markdown; it is sent `{email}`, `{bucket}`, `{state}` and `{link}` - the page where the
reader acts on the record, their own or the desk's (every variable is
in [Mail](mail.md#variables)). The body is markdown; it is sent
as text with an HTML alternative. `to: submitter` means the record's as text with an HTML alternative. `to: submitter` means the record's
`email` field, and the lint then requires every form filling that `email` field, and the lint then requires every form filling that
bucket to collect a required `email`. bucket to collect a required `email`.
+16 -5
View File
@@ -243,13 +243,24 @@ against the lint. It is kept for the record of what it measured.
visitor will tolerate. visitor will tolerate.
2. **Structure.** Pages, state machines and desks are generated, then 2. **Structure.** Pages, state machines and desks are generated, then
revised against the lint, `iris check`, until every need is met: revised against the lint, `iris check`, until every need is met:
a path within budget, a desk held by the right group, every a path within budget, a desk held by the right group, every state
finished state reachable, no record stranding. The page-writing of every bucket reachable and none stranding a record. The
model sees the needs and never the personas. page-writing model sees the needs and never the personas. Each
round the lint writes its findings with stable ids (`--findings`,
kept in `<out>/findings/round-N.jsonl`), and the round is logged as
what it fixed, what is still open, what is new and what it
reopened. A fix round is shown only the open findings, each with
the change that closes it, and the files they name (the whole repo
when a finding is about getting around the site). A change that
reopens a closed finding is undone and the model told what it
broke. A round that closes nothing, or leaves as many open as it
found, ends the phase: what is left is written to `run.log` rather
than polished at again.
3. **Wording.** A separate model plays each persona, blind, first with 3. **Wording.** A separate model plays each persona, blind, first with
headings only and then with descriptions. A misroute is a wording headings only and then with descriptions. A misroute is a wording
defect and goes back to the generator. A rewrite that breaks defect and goes back to the generator; a visitor who chose right
structure is reverted. but hesitated is reported and rewrites nothing by itself. A rewrite
that breaks structure is reverted.
Model output is only ever written to `aggregates.yaml`, `site.yaml` Model output is only ever written to `aggregates.yaml`, `site.yaml`
and `questions/**.yaml` inside the output directory. and `questions/**.yaml` inside the output directory.
+2 -2
View File
@@ -289,7 +289,7 @@ slideshow:
- image: images/bildeserie/20.webp - image: images/bildeserie/20.webp
width: 1867 width: 1867
height: 1400 height: 1400
caption: Her heises Moelvens modulskole i Vestby på plass. Den blir stående i 10 år. caption: Her heises Moelvens modulskole i Vestby på plass. Den blir stående i minst 10 år.
- image: images/bildeserie/21.webp - image: images/bildeserie/21.webp
width: 1867 width: 1867
height: 1400 height: 1400
@@ -309,4 +309,4 @@ slideshow:
- image: images/bildeserie/25.webp - image: images/bildeserie/25.webp
width: 1487 width: 1487
height: 1400 height: 1400
caption: pH-test av borprøver i mur fra forskjellige dybder caption: pH-test av borprøver i betong fra forskjellige dybder
+8
View File
@@ -38,6 +38,11 @@ pub struct Answer {
/// not see before. /// not see before.
#[serde(default, skip_serializing_if = "Vec::is_empty")] #[serde(default, skip_serializing_if = "Vec::is_empty")]
pub notes: Vec<Note>, pub notes: Vec<Note>,
/// Set by a desk listing, never stored: the record is in a final
/// state (one with no move out). A desk shows the newest few of
/// those and offers the rest on request (`resource::cap_finished`).
#[serde(default, skip_serializing_if = "std::ops::Not::not")]
pub finished: bool,
} }
/// One reply on a record. It lives here rather than with the mail /// One reply on a record. It lives here rather than with the mail
@@ -97,6 +102,7 @@ pub async fn store_answer(
storyline, storyline,
decided_ms: None, decided_ms: None,
notes: Vec::new(), notes: Vec::new(),
finished: false,
decided_by: None, decided_by: None,
}; };
let data = serde_json::to_vec(&answer)?; let data = serde_json::to_vec(&answer)?;
@@ -537,6 +543,7 @@ mod tests {
decided_ms: None, decided_ms: None,
decided_by: None, decided_by: None,
notes: Vec::new(), notes: Vec::new(),
finished: false,
}, },
Answer { Answer {
id: "def456".to_string(), id: "def456".to_string(),
@@ -549,6 +556,7 @@ mod tests {
decided_ms: Some(1_700_000_002_000), decided_ms: Some(1_700_000_002_000),
decided_by: Some("bl".to_string()), decided_by: Some("bl".to_string()),
notes: Vec::new(), notes: Vec::new(),
finished: false,
}, },
]; ];
let value = serde_json::Value::Array( let value = serde_json::Value::Array(
+213 -7
View File
@@ -1936,18 +1936,26 @@ fn ResourceFeature(
pending_transitions: RwSignal<std::collections::HashMap<(String, String), String>>, pending_transitions: RwSignal<std::collections::HashMap<(String, String), String>>,
transition_batch: ServerAction<TransitionAnswers>, transition_batch: ServerAction<TransitionAnswers>,
) -> impl IntoView { ) -> impl IntoView {
// A desk shows the newest few finished records; "show older" asks
// for more (see `resource::cap_finished`).
let finished_shown = RwSignal::new(crate::resource::FINISHED_SHOWN);
let data = Resource::new( let data = Resource::new(
{ {
let question_id = question_id.clone(); let question_id = question_id.clone();
let alternative = alternative.clone(); let alternative = alternative.clone();
let feature_name = feature_name.clone(); let feature_name = feature_name.clone();
move || (question_id.clone(), alternative.clone(), feature_name.clone()) move || (question_id.clone(), alternative.clone(), feature_name.clone(), finished_shown.get())
},
// Forwarding sibling requirement values as query params (for a
// resource parameterized by other form fields) is deferred, see
// the resource-backed multi-select requirement UI follow-up.
|(q, a, f, shown)| {
let mut params = std::collections::HashMap::new();
if shown != crate::resource::FINISHED_SHOWN {
params.insert(crate::resource::FINISHED_PARAM.to_string(), shown.to_string());
}
get_resource(q, a, f, params)
}, },
// Empty params for now - forwarding sibling requirement values
// as query params (for a resource parameterized by other form
// fields) is deferred, see the resource-backed multi-select
// requirement UI follow-up.
|(q, a, f)| get_resource(q, a, f, std::collections::HashMap::new()),
); );
// The shared batch button lives on `AlternativeCard`, above every // The shared batch button lives on `AlternativeCard`, above every
@@ -1982,6 +1990,7 @@ fn ResourceFeature(
value=value value=value
pending_transitions=pending_transitions pending_transitions=pending_transitions
transition_batch=transition_batch transition_batch=transition_batch
finished_shown=finished_shown
/> />
} }
.into_any() .into_any()
@@ -2011,6 +2020,7 @@ fn ResourceValue(
value: serde_json::Value, value: serde_json::Value,
pending_transitions: RwSignal<std::collections::HashMap<(String, String), String>>, pending_transitions: RwSignal<std::collections::HashMap<(String, String), String>>,
transition_batch: ServerAction<TransitionAnswers>, transition_batch: ServerAction<TransitionAnswers>,
finished_shown: RwSignal<usize>,
) -> impl IntoView { ) -> impl IntoView {
// A jq like `.[] | {...}` over an empty list yields no output at // A jq like `.[] | {...}` over an empty list yields no output at
// all - that arrives as null, and it's the same silence as []. // all - that arrives as null, and it's the same silence as [].
@@ -2034,6 +2044,32 @@ fn ResourceValue(
let when = if cards { a.submitted_ms.cmp(&b.submitted_ms) } else { b.submitted_ms.cmp(&a.submitted_ms) }; let when = if cards { a.submitted_ms.cmp(&b.submitted_ms) } else { b.submitted_ms.cmp(&a.submitted_ms) };
open(&a.state).cmp(&open(&b.state)).then(when) open(&a.state).cmp(&open(&b.state)).then(when)
}); });
// The desk sent one finished record more than it shows, when
// there are older ones: drop the oldest and offer the rest.
let shown = finished_shown.get_untracked();
let older = answers.iter().filter(|a| a.finished).count() > shown;
if older {
if let Some(oldest) = answers
.iter()
.enumerate()
.filter(|(_, a)| a.finished)
.min_by_key(|(_, a)| a.decided_ms.unwrap_or(a.submitted_ms))
.map(|(i, _)| i)
{
answers.remove(oldest);
}
}
let show_older = older.then(|| {
view! {
<button
type="button"
class="show-older"
on:click=move |_| finished_shown.update(|n| *n += crate::resource::FINISHED_SHOWN)
>
{move || t(&lang().get(), "show_older")}
</button>
}
});
return view! { return view! {
<div class="answer-list" class:answer-cards=cards> <div class="answer-list" class:answer-cards=cards>
<For <For
@@ -2060,6 +2096,7 @@ fn ResourceValue(
} }
} }
/> />
{show_older}
</div> </div>
} }
.into_any(); .into_any();
@@ -2087,9 +2124,148 @@ fn ResourceValue(
} }
} }
// A releases manifest (content-reference, "A releases manifest"):
// the newest version, its files per platform, and the standing
// channels, with the visitor's own platform picked in the browser.
if is_releases_manifest(&value) {
return view! { <ReleasesCard manifest=value/> }.into_any();
}
view! { <pre class="resource-raw">{value.to_string()}</pre> }.into_any() view! { <pre class="resource-raw">{value.to_string()}</pre> }.into_any()
} }
/// `{ name, latest, versions: [{ version, date, notes, assets: [{
/// platform, url, sha256, size }] }], channels: {...} }` - the shape a
/// release pipeline writes (redoal's does) for a site to offer downloads.
pub fn is_releases_manifest(value: &serde_json::Value) -> bool {
value.get("latest").and_then(|v| v.as_str()).is_some()
&& value.get("versions").and_then(|v| v.as_array()).is_some_and(|v| !v.is_empty())
}
/// The platform a browser says it is, in the manifest's words:
/// `ios`, `macos`, `android`, `windows`, `linux-x86_64`,
/// `linux-aarch64`, or `other`. Read, never trusted for anything but
/// which download to put first.
pub fn platform_of(user_agent: &str) -> &'static str {
let ua = user_agent.to_ascii_lowercase();
if ua.contains("iphone") || ua.contains("ipad") {
"ios"
} else if ua.contains("mac os x") || ua.contains("macintosh") {
"macos"
} else if ua.contains("android") {
"android"
} else if ua.contains("windows") {
"windows"
} else if ua.contains("linux") || ua.contains("x11") {
if ua.contains("aarch64") || ua.contains("arm64") {
"linux-aarch64"
} else {
"linux-x86_64"
}
} else {
"other"
}
}
/// The newest release as a card: the file for this device first (the
/// platform is read in the browser, so the server renders every file
/// and the pick lands on hydration), the rest after, and the channels
/// that are not files - a package repo, TestFlight.
#[component]
fn ReleasesCard(manifest: serde_json::Value) -> impl IntoView {
let name = manifest.get("name").and_then(|v| v.as_str()).unwrap_or("").to_string();
let latest = manifest.get("latest").and_then(|v| v.as_str()).unwrap_or("").to_string();
let newest = manifest["versions"][0].clone();
let date = newest.get("date").and_then(|v| v.as_str()).unwrap_or("").to_string();
let notes = newest.get("notes").and_then(|v| v.as_str()).filter(|u| u.starts_with("https://")).map(str::to_string);
let assets: Vec<(String, String, String)> = newest
.get("assets")
.and_then(|a| a.as_array())
.map(|a| {
a.iter()
.filter_map(|x| {
let platform = x.get("platform")?.as_str()?.to_string();
let url = x.get("url")?.as_str()?.to_string();
let sha = x.get("sha256").and_then(|s| s.as_str()).unwrap_or("").to_string();
Some((platform, url, sha))
})
.collect()
})
.unwrap_or_default();
let channels = manifest.get("channels").cloned().unwrap_or(serde_json::Value::Null);
let arch = channels.get("arch").cloned();
let ios = channels.get("ios").cloned();
let platform = RwSignal::new(None::<&'static str>);
Effect::new(move |_| {
#[cfg(feature = "hydrate")]
if let Some(ua) = web_sys::window().and_then(|w| w.navigator().user_agent().ok()) {
platform.set(Some(platform_of(&ua)));
}
});
let label = |p: &str| match p {
"linux-x86_64" => "Linux (x86_64)",
"linux-aarch64" => "Linux (aarch64)",
"macos" | "macos-arm64" => "macOS",
"windows" | "windows-x86_64" => "Windows",
other => other,
}.to_string();
let assets_for_list = assets.clone();
let arch_for_pick = arch.clone();
let ios_for_pick = ios.clone();
let assets_for_pick = assets.clone();
view! {
<article class="item-card releases">
<div class="item-card-title">
<strong>{name.clone()} " " {latest.clone()}</strong>
{(!date.is_empty()).then(|| view! { <span class="dim">" · " {date.clone()}</span> })}
{notes.clone().map(|u| view! { " · " <a href=u target="_blank" rel="noopener noreferrer">{move || t(&lang().get(), "release_notes")}</a> })}
</div>
// What this device should take, decided in the browser.
{move || {
let p = platform.get()?;
let arch = arch_for_pick.clone();
let ios = ios_for_pick.clone();
// "macos" is satisfied by a "macos-arm64" file, and so on.
let own = assets_for_pick.iter().find(|(pl, _, _)| pl == p || pl.starts_with(&format!("{p}-"))).cloned();
let pick: Option<AnyView> = match (p, own) {
(_, Some((pl, url, _))) => Some(view! {
<p><a class="pill" href=url download>{t(&lang().get(), "for_this_device")} ": " {label(&pl)}</a></p>
{(p.starts_with("linux") && arch.is_some()).then(|| {
let a = arch.clone().unwrap();
let repo = a.get("repo").and_then(|v| v.as_str()).unwrap_or("").to_string();
let pkg = a.get("package").and_then(|v| v.as_str()).unwrap_or("").to_string();
view! { <pre class="monospace dim">{repo} "\n\npacman -Syu " {pkg}</pre> }
})}
}.into_any()),
("ios" | "macos", None) => ios.as_ref().and_then(|i| i.get("testflight").and_then(|v| v.as_str()).map(str::to_string)).map(|tf| {
let note = ios.as_ref().and_then(|i| i.get("note").and_then(|v| v.as_str())).unwrap_or("").to_string();
view! { <p><a class="pill" href=tf target="_blank" rel="noopener noreferrer">{t(&lang().get(), "for_this_device")} ": TestFlight"</a> <span class="dim">" " {note}</span></p> }.into_any()
}),
_ => None,
};
pick
}}
<details class="releases-all">
<summary>{move || t(&lang().get(), "all_downloads")}</summary>
<ul>
{assets_for_list.iter().map(|(pl, url, sha)| view! {
<li><a href=url.clone() download>{label(pl)}</a> {(!sha.is_empty()).then(|| view! { <span class="dim monospace">" sha256 " {sha[..sha.len().min(12)].to_string()} "…"</span> })}</li>
}).collect::<Vec<_>>()}
{arch.as_ref().map(|a| {
let note = a.get("note").and_then(|v| v.as_str()).unwrap_or("").to_string();
view! { <li>{note}</li> }
})}
{ios.as_ref().and_then(|i| i.get("testflight").and_then(|v| v.as_str()).map(|tf| {
let note = i.get("note").and_then(|v| v.as_str()).unwrap_or("").to_string();
view! { <li><a href=tf.to_string() target="_blank" rel="noopener noreferrer">"TestFlight"</a> " " {note}</li> }
}))}
</ul>
</details>
</article>
}
}
/// Pulls the first present, non-null string field out of a JSON object /// Pulls the first present, non-null string field out of a JSON object
/// matching one of `keys`, tried in order - shared between `ItemCard` /// matching one of `keys`, tried in order - shared between `ItemCard`
/// and `SelectField`'s option rendering, since both need "guess a /// and `SelectField`'s option rendering, since both need "guess a
@@ -3082,10 +3258,16 @@ pub async fn submit_answer(
let Some((name, who)) = crate::people::who_for(&responses) else { let Some((name, who)) = crate::people::who_for(&responses) else {
return Err(ServerFnError::new("an invite needs a name, and a phone number or an email")); return Err(ServerFnError::new("an invite needs a name, and a phone number or an email"));
}; };
let invited = people let mut invited = people
.invite(&state, &name, &who, &invite.group, &principal_name(&principal)) .invite(&state, &name, &who, &invite.group, &principal_name(&principal))
.await .await
.map_err(|e| ServerFnError::new(format!("invite failed: {e}")))?; .map_err(|e| ServerFnError::new(format!("invite failed: {e}")))?;
// This invite's own words, and the form they draw on: kept with
// the invite, so sending it again says the same thing.
invited.mail = invite.mail.clone();
if invited.mail.is_some() {
invited.fields = responses.as_object().cloned().unwrap_or_default();
}
if let Some(map) = responses.as_object_mut() { if let Some(map) = responses.as_object_mut() {
map.insert("invited".to_string(), serde_json::to_value(&invited).unwrap_or_default()); map.insert("invited".to_string(), serde_json::to_value(&invited).unwrap_or_default());
} }
@@ -3322,3 +3504,27 @@ mod picture_tests {
assert_eq!(picture_of(&answer, None), None); assert_eq!(picture_of(&answer, None), None);
} }
} }
#[cfg(test)]
mod releases_tests {
use super::*;
#[test]
fn a_browser_names_its_platform_in_the_manifests_words() {
assert_eq!(platform_of("Mozilla/5.0 (iPhone; CPU iPhone OS 19_0 like Mac OS X)"), "ios");
assert_eq!(platform_of("Mozilla/5.0 (Macintosh; Intel Mac OS X 15_0)"), "macos");
assert_eq!(platform_of("Mozilla/5.0 (X11; Linux x86_64) Firefox"), "linux-x86_64");
assert_eq!(platform_of("Mozilla/5.0 (X11; Linux aarch64) Firefox"), "linux-aarch64");
assert_eq!(platform_of("Mozilla/5.0 (Windows NT 10.0; Win64; x64)"), "windows");
assert_eq!(platform_of("Mozilla/5.0 (Linux; Android 15)"), "android");
assert_eq!(platform_of("curl/8"), "other");
}
#[test]
fn only_a_manifest_shape_is_a_manifest() {
let m = serde_json::json!({"name":"redoal","latest":"0.1.17","versions":[{"version":"0.1.17","assets":[]}]});
assert!(is_releases_manifest(&m));
assert!(!is_releases_manifest(&serde_json::json!({"latest":"1","versions":[]})));
assert!(!is_releases_manifest(&serde_json::json!([{"name":"x"}])));
}
}
+115 -15
View File
@@ -49,15 +49,56 @@ pub struct GiteaRepoQuery {
pub async fn site_asset_handler( pub async fn site_asset_handler(
axum::extract::State(state): axum::extract::State<crate::server::AppState>, axum::extract::State(state): axum::extract::State<crate::server::AppState>,
axum::extract::Path(path): axum::extract::Path<String>, axum::extract::Path(path): axum::extract::Path<String>,
) -> axum::response::Response {
serve_repo_file(&state, &path, false).await
}
/// `GET /.well-known/{*path}` - the content repo overlaid on the one part
/// of the URL space that can never collide with a question: RFC 8615
/// reserves `/.well-known/`, and a question's id is derived from a file
/// name under `questions/`, which cannot begin with a dot. A repo ships
/// `.well-known/security.txt`, `.well-known/assetlinks.json`,
/// `.well-known/matrix/server` and so on at the path they are served at,
/// and a miss is a real 404 rather than the app shell answering 200 with
/// an HTML page - which is what a client probing for one of these would
/// otherwise have to parse.
///
/// Paths Caddy claims in front of portal (mail autoconfiguration, the
/// CalDAV and CardDAV redirects) never reach here.
#[cfg(feature = "ssr")]
pub async fn well_known_handler(
axum::extract::State(state): axum::extract::State<crate::server::AppState>,
axum::extract::Path(path): axum::extract::Path<String>,
) -> axum::response::Response {
serve_repo_file(&state, &format!(".well-known/{path}"), true).await
}
/// Proxies one repo-relative file from the content repo's default branch.
/// `well_known` only changes what an extensionless file is typed as: under
/// `/.well-known/` those are JSON by convention (`matrix/server`,
/// `nodeinfo`), while elsewhere a file with no extension is an unknown.
#[cfg(feature = "ssr")]
async fn serve_repo_file(
state: &crate::server::AppState,
path: &str,
well_known: bool,
) -> axum::response::Response { ) -> axum::response::Response {
use axum::http::{header, StatusCode}; use axum::http::{header, StatusCode};
use axum::response::IntoResponse; use axum::response::IntoResponse;
if !is_safe_site_path(&path) { if !is_safe_site_path(path) {
return (StatusCode::BAD_REQUEST, "invalid asset path").into_response(); return (StatusCode::BAD_REQUEST, "invalid asset path").into_response();
} }
let url = format!("{}/{path}?ref={}", state.content_raw_base, state.content_branch); let url = format!(
let resp = match crate::http::client().get(&url).send().await.and_then(|r| r.error_for_status()) { "{}/{path}?ref={}",
state.content_raw_base, state.content_branch
);
let resp = match crate::http::client()
.get(&url)
.send()
.await
.and_then(|r| r.error_for_status())
{
Ok(r) => r, Ok(r) => r,
Err(e) => { Err(e) => {
tracing::warn!("site asset {path}: {e}"); tracing::warn!("site asset {path}: {e}");
@@ -79,12 +120,26 @@ pub async fn site_asset_handler(
Some("avif") => "image/avif", Some("avif") => "image/avif",
Some("ico") => "image/x-icon", Some("ico") => "image/x-icon",
Some("woff2") => "font/woff2", Some("woff2") => "font/woff2",
Some("txt") => "text/plain; charset=utf-8",
Some("xml") => "application/xml; charset=utf-8",
Some("webmanifest") => "application/manifest+json",
// An Apple configuration profile only offers to install when it
// arrives under this type; anything else is just a download.
Some("mobileconfig") => "application/x-apple-aspen-config",
// A file with no extension at all: rsplit yields the whole name.
_ if well_known && !path.rsplit('/').next().unwrap_or_default().contains('.') => {
"application/json; charset=utf-8"
}
_ => "application/octet-stream", _ => "application/octet-stream",
}; };
// Photographs are large and do not change under a name, so a // Photographs are large and do not change under a name, so a
// browser may keep them for an hour; everything else revalidates, // browser may keep them for an hour; everything else revalidates,
// so a content push shows on reload. // so a content push shows on reload.
let cache = if matches!(mime, "image/webp" | "image/avif" | "image/jpeg") { "public, max-age=3600" } else { "no-cache" }; let cache = if matches!(mime, "image/webp" | "image/avif" | "image/jpeg") {
"public, max-age=3600"
} else {
"no-cache"
};
( (
[(header::CONTENT_TYPE, mime), (header::CACHE_CONTROL, cache)], [(header::CONTENT_TYPE, mime), (header::CACHE_CONTROL, cache)],
bytes, bytes,
@@ -114,14 +169,22 @@ pub async fn gitea_repo_handler(
.all(|c| c.is_ascii_alphanumeric() || c == '-' || c == '_' || c == '.') .all(|c| c.is_ascii_alphanumeric() || c == '-' || c == '_' || c == '.')
}; };
if !is_safe_segment(&query.owner) || !is_safe_segment(&query.repo) { if !is_safe_segment(&query.owner) || !is_safe_segment(&query.repo) {
return Err((axum::http::StatusCode::BAD_REQUEST, "invalid owner/repo".to_string())); return Err((
axum::http::StatusCode::BAD_REQUEST,
"invalid owner/repo".to_string(),
));
} }
let GiteaRepoQuery { owner, repo } = query; let GiteaRepoQuery { owner, repo } = query;
let api_url = format!("{}/api/v1/repos/{owner}/{repo}", state.gitea_base); let api_url = format!("{}/api/v1/repos/{owner}/{repo}", state.gitea_base);
let body = crate::http::get_text(&api_url, crate::http::GITEA_CAP) let body = crate::http::get_text(&api_url, crate::http::GITEA_CAP)
.await .await
.map_err(|e| (axum::http::StatusCode::BAD_GATEWAY, format!("fetching {api_url}: {e}")))?; .map_err(|e| {
(
axum::http::StatusCode::BAD_GATEWAY,
format!("fetching {api_url}: {e}"),
)
})?;
let json: serde_json::Value = serde_json::from_str(&body).map_err(|e| { let json: serde_json::Value = serde_json::from_str(&body).map_err(|e| {
( (
axum::http::StatusCode::BAD_GATEWAY, axum::http::StatusCode::BAD_GATEWAY,
@@ -168,27 +231,38 @@ pub async fn automation_kv_handler(
.and_then(|v| v.strip_prefix("Bearer ")) .and_then(|v| v.strip_prefix("Bearer "))
.unwrap_or(""); .unwrap_or("");
if expected.is_empty() || presented != expected { if expected.is_empty() || presented != expected {
return Err((axum::http::StatusCode::UNAUTHORIZED, "unauthorized".to_string())); return Err((
axum::http::StatusCode::UNAUTHORIZED,
"unauthorized".to_string(),
));
} }
// The token authenticates the automation service; what it may read // The token authenticates the automation service; what it may read
// is the policy's to say, like every other principal. // is the policy's to say, like every other principal.
let decision = crate::access::decide_and_publish( let decision = crate::access::decide_and_publish(
&state.nats, &state.nats,
&state.access.load(), &state.access.load(),
&crate::access::Principal::Service { name: "automation".to_string() }, &crate::access::Principal::Service {
name: "automation".to_string(),
},
crate::access::Action::AutomationRead, crate::access::Action::AutomationRead,
&crate::access::Resource::Bucket { name: bucket.clone() }, &crate::access::Resource::Bucket {
name: bucket.clone(),
},
serde_json::json!({}), serde_json::json!({}),
); );
if !decision.allow { if !decision.allow {
return Err((axum::http::StatusCode::FORBIDDEN, "not authorized".to_string())); return Err((
axum::http::StatusCode::FORBIDDEN,
"not authorized".to_string(),
));
} }
let store = state let store = state.jetstream.get_key_value(&bucket).await.map_err(|e| {
.jetstream (
.get_key_value(&bucket) axum::http::StatusCode::BAD_GATEWAY,
.await format!("bucket unavailable: {e}"),
.map_err(|e| (axum::http::StatusCode::BAD_GATEWAY, format!("bucket unavailable: {e}")))?; )
})?;
use futures::TryStreamExt; use futures::TryStreamExt;
let keys: Vec<String> = store let keys: Vec<String> = store
@@ -209,3 +283,29 @@ pub async fn automation_kv_handler(
Ok(axum::Json(serde_json::Value::Array(items))) Ok(axum::Json(serde_json::Value::Array(items)))
} }
#[cfg(test)]
mod tests {
use super::*;
/// The overlay's whole safety argument: `/.well-known/` is reserved by
/// RFC 8615 and a question's id comes from a file under `questions/`, so
/// a repo file served at the site root cannot shadow a page - and the
/// path it is fetched by still cannot climb out of the repo.
#[test]
fn well_known_paths_stay_inside_the_repo() {
for path in [
".well-known/security.txt",
".well-known/matrix/server",
".well-known/assetlinks.json",
] {
assert!(is_safe_site_path(path), "{path} should be servable");
}
for path in [
".well-known/../questions/index.yaml",
".well-known//escape",
"/.well-known/security.txt",
] {
assert!(!is_safe_site_path(path), "{path} should be refused");
}
}
}
+25 -3
View File
@@ -391,6 +391,13 @@ pub struct Alternative {
#[derive(Clone, Debug, Serialize, Deserialize)] #[derive(Clone, Debug, Serialize, Deserialize)]
pub struct Invite { pub struct Invite {
pub group: String, pub group: String,
/// This invite's own mail, in place of the site's (`mail.invite`):
/// an invitation to a conversation at an hour, say. Besides
/// `{name}`, `{link}`, `{expires}`, `{username}`, `{group}`,
/// `{site}` and `{site_name}`, every field of the form is a
/// placeholder; a date and time is written out in words.
#[serde(default, skip_serializing_if = "Option::is_none")]
pub mail: Option<MailTemplate>,
} }
#[derive(Clone, Debug, Serialize, Deserialize)] #[derive(Clone, Debug, Serialize, Deserialize)]
@@ -512,6 +519,18 @@ pub struct ResourceSpec {
/// it where the silence needs a voice ("Nothing released yet"). /// it where the silence needs a voice ("Nothing released yet").
#[serde(default)] #[serde(default)]
pub empty: Option<String>, pub empty: Option<String>,
/// Show at most this many items, applied after `jq`. For a feed
/// where only the newest matters: a releases card wants the cut you
/// should run, not every cut there has ever been, and a page that
/// grows by one card per release gets longer forever.
///
/// After `jq` rather than before, so it caps what the page will
/// actually render whatever the filter did — and so content can
/// reorder with jq and still take the top of its own ordering. A
/// source that yields a single record has no length and passes
/// through untouched. `None` shows everything.
#[serde(default)]
pub limit: Option<usize>,
} }
/// Where a resource's live data comes from. `Kv` (a NATS KV bucket /// Where a resource's live data comes from. `Kv` (a NATS KV bucket
@@ -1316,6 +1335,7 @@ alternatives:
public: true public: true
source: {{ kind: kv, bucket: things }} source: {{ kind: kv, bucket: things }}
- name: Object - name: Object
action: /object
record_as: things record_as: things
features: features:
- name: Yours - name: Yours
@@ -1413,6 +1433,7 @@ id: /t
name: Where is it? name: Where is it?
alternatives: alternatives:
- name: A - name: A
action: /t
features: features:
- name: Where - name: Where
requirements: requirements:
@@ -1516,6 +1537,7 @@ id: /b
name: B name: B
alternatives: alternatives:
- name: A - name: A
action: /b
features: features:
- name: "" - name: ""
requirements: requirements:
@@ -1557,7 +1579,7 @@ alternatives:
fn question_with_requirement(req_yaml: &str) -> std::collections::HashMap<String, Question> { fn question_with_requirement(req_yaml: &str) -> std::collections::HashMap<String, Question> {
let question: Question = serde_yaml::from_str(&format!( let question: Question = serde_yaml::from_str(&format!(
"id: /g\nname: G\nalternatives:\n - name: A\n features:\n - name: \"\"\n requirements:\n{req_yaml}\n" "id: /g\nname: G\nalternatives:\n - name: A\n action: /g\n features:\n - name: \"\"\n requirements:\n{req_yaml}\n"
)) ))
.unwrap(); .unwrap();
std::collections::HashMap::from([(question.id.clone(), question)]) std::collections::HashMap::from([(question.id.clone(), question)])
@@ -1828,7 +1850,7 @@ alternatives:
fn recorded_buckets_must_be_attended() { fn recorded_buckets_must_be_attended() {
let write_only = vec![( let write_only = vec![(
"index.yaml".to_string(), "index.yaml".to_string(),
"name: Home\nalternatives:\n - name: send\n record_as: black_hole\n".to_string(), "name: Home\nalternatives:\n - name: send\n action: /\n record_as: black_hole\n".to_string(),
)]; )];
let questions = build_questions(&write_only).unwrap(); let questions = build_questions(&write_only).unwrap();
let err = validate_questions(&questions, &Default::default()).unwrap_err(); let err = validate_questions(&questions, &Default::default()).unwrap_err();
@@ -1839,7 +1861,7 @@ alternatives:
let with_desk = vec![ let with_desk = vec![
( (
"index.yaml".to_string(), "index.yaml".to_string(),
"name: Home\nalternatives:\n - name: send\n record_as: inbox\n".to_string(), "name: Home\nalternatives:\n - name: send\n action: /\n record_as: inbox\n".to_string(),
), ),
( (
"review/index.yaml".to_string(), "review/index.yaml".to_string(),
+14
View File
@@ -512,6 +512,20 @@ pub fn validate_questions(
} }
for question in questions.values() { for question in questions.values() {
for alternative in &question.alternatives { for alternative in &question.alternatives {
// A form is sent with its button, and the button is only
// drawn for an alternative that says where to go afterwards:
// without `action` the person fills it in and cannot send it.
let asks = alternative.features.iter().any(|f| f.requirements.iter().any(|r| r.kind != "hidden"));
if (asks || alternative.invite.is_some() || alternative.record_as.is_some())
&& alternative.action.is_none()
&& alternative.self_transition.is_none()
{
anyhow::bail!(
"question {:?} alternative {:?}: it has a form to send but no `action`, so no button is drawn; give it `action:`, the page to go to once it is sent (its own page is fine)",
question.id,
alternative.name
);
}
if alternative.one_per_person && alternative.record_as.is_none() { if alternative.one_per_person && alternative.record_as.is_none() {
anyhow::bail!( anyhow::bail!(
"question {:?} alternative {:?}: one_per_person keeps each person's own record, and there is no record_as to keep it in", "question {:?} alternative {:?}: one_per_person keeps each person's own record, and there is no record_as to keep it in",
+1 -1
View File
@@ -194,7 +194,7 @@ aggregates:
Answer { Answer {
id: "r".into(), question_id: "/q".into(), alternative: "a".into(), id: "r".into(), question_id: "/q".into(), alternative: "a".into(),
responses: serde_json::json!({}), state: state.into(), responses: serde_json::json!({}), state: state.into(),
submitted_ms: since_ms, decided_ms: None, decided_by: None, notes: vec![], submitted_ms: since_ms, decided_ms: None, decided_by: None, notes: vec![], finished: false,
storyline: 0, storyline: 0,
} }
} }
+12
View File
@@ -24,6 +24,10 @@ fn lookup(lang: &str, key: &str) -> Option<&'static str> {
("en", "sign_in") => "Sign in", ("en", "sign_in") => "Sign in",
("en", "sign_out") => "Sign out", ("en", "sign_out") => "Sign out",
("en", "nothing_here_yet") => "Nothing here yet.", ("en", "nothing_here_yet") => "Nothing here yet.",
("en", "show_older") => "Show older",
("en", "for_this_device") => "For this device",
("en", "all_downloads") => "All downloads",
("en", "release_notes") => "release notes",
("en", "upload_failed") => "The file did not arrive, so nothing was sent. Try again.", ("en", "upload_failed") => "The file did not arrive, so nothing was sent. Try again.",
("en", "upload_too_large") => "The file is larger than 20 MB, so nothing was sent. Choose a smaller one, or a picture taken at a lower resolution.", ("en", "upload_too_large") => "The file is larger than 20 MB, so nothing was sent. Choose a smaller one, or a picture taken at a lower resolution.",
("en", "upload_unavailable") => "This site cannot take files at the moment, so nothing was sent. Let whoever runs the site know.", ("en", "upload_unavailable") => "This site cannot take files at the moment, so nothing was sent. Let whoever runs the site know.",
@@ -60,6 +64,10 @@ fn lookup(lang: &str, key: &str) -> Option<&'static str> {
("no", "sign_in") => "Logg inn", ("no", "sign_in") => "Logg inn",
("no", "sign_out") => "Logg ut", ("no", "sign_out") => "Logg ut",
("no", "nothing_here_yet") => "Ingenting her ennå.", ("no", "nothing_here_yet") => "Ingenting her ennå.",
("no", "show_older") => "Vis eldre",
("no", "for_this_device") => "Til denne enheten",
("no", "all_downloads") => "Alle nedlastinger",
("no", "release_notes") => "utgivelsesnotater",
("no", "upload_failed") => "Filen kom ikke fram, så ingenting ble sendt. Prøv igjen.", ("no", "upload_failed") => "Filen kom ikke fram, så ingenting ble sendt. Prøv igjen.",
("no", "upload_too_large") => "Filen er større enn 20 MB, så ingenting ble sendt. Velg en mindre fil, eller et bilde med lavere oppløsning.", ("no", "upload_too_large") => "Filen er større enn 20 MB, så ingenting ble sendt. Velg en mindre fil, eller et bilde med lavere oppløsning.",
("no", "upload_unavailable") => "Siden kan ikke ta imot filer akkurat nå, så ingenting ble sendt. Si fra til den som har siden.", ("no", "upload_unavailable") => "Siden kan ikke ta imot filer akkurat nå, så ingenting ble sendt. Si fra til den som har siden.",
@@ -114,6 +122,10 @@ mod tests {
"sign_in", "sign_in",
"sign_out", "sign_out",
"nothing_here_yet", "nothing_here_yet",
"show_older",
"for_this_device",
"all_downloads",
"release_notes",
"upload_failed", "upload_failed",
"upload_too_large", "upload_too_large",
"upload_unavailable", "upload_unavailable",
+61 -12
View File
@@ -24,6 +24,7 @@ pub async fn run(argv: Vec<String>) -> anyhow::Result<()> {
let mut skim = false; let mut skim = false;
let mut sim_out: Option<String> = None; let mut sim_out: Option<String> = None;
let mut show_access = false; let mut show_access = false;
let mut findings_out: Option<String> = None;
while let Some(arg) = args.next() { while let Some(arg) = args.next() {
match arg.as_str() { match arg.as_str() {
@@ -35,6 +36,7 @@ pub async fn run(argv: Vec<String>) -> anyhow::Result<()> {
"--skim" => skim = true, "--skim" => skim = true,
"--needs-sim" => sim_out = args.next(), "--needs-sim" => sim_out = args.next(),
"--access" => show_access = true, "--access" => show_access = true,
"--findings" => findings_out = args.next(),
"--needs-score" => answers_in = args.next(), "--needs-score" => answers_in = args.next(),
"--min-accuracy" => { "--min-accuracy" => {
min_accuracy = args.next().and_then(|v| v.parse().ok()).unwrap_or(min_accuracy) min_accuracy = args.next().and_then(|v| v.parse().ok()).unwrap_or(min_accuracy)
@@ -66,13 +68,14 @@ pub async fn run(argv: Vec<String>) -> anyhow::Result<()> {
(questions, aggregates_map, site, needs_raw) (questions, aggregates_map, site, needs_raw)
} else { } else {
eprintln!( eprintln!(
"usage: iris check (--repo <gitea-url> [--branch main] [--subdir questions] | --path <local-dir>) [--needs-tasks <out.jsonl> [--skim]] [--needs-sim <model.json>] [--access] [--needs-score <answers.jsonl> [--min-accuracy 0.8]]" "usage: iris check (--repo <gitea-url> [--branch main] [--subdir questions] | --path <local-dir>) [--findings <out.jsonl>] [--needs-tasks <out.jsonl> [--skim]] [--needs-sim <model.json>] [--access] [--needs-score <answers.jsonl> [--min-accuracy 0.8]]"
); );
std::process::exit(2); std::process::exit(2);
}; };
if let Err(e) = site.validate() { if let Err(e) = site.validate() {
eprintln!("FAIL: {e}"); eprintln!("FAIL: {e}");
write_findings(findings_out.as_deref(), &[schema_finding(&e.to_string())]);
std::process::exit(1); std::process::exit(1);
} }
let aggregates_map = content::with_builtin_aggregates(aggregates_map); let aggregates_map = content::with_builtin_aggregates(aggregates_map);
@@ -114,6 +117,7 @@ pub async fn run(argv: Vec<String>) -> anyhow::Result<()> {
println!(); println!();
} }
check_needs( check_needs(
findings_out.as_deref(),
needs_raw.as_deref(), needs_raw.as_deref(),
&questions, &questions,
&aggregates_map, &aggregates_map,
@@ -126,15 +130,67 @@ pub async fn run(argv: Vec<String>) -> anyhow::Result<()> {
} }
Err(e) => { Err(e) => {
eprintln!("FAIL: {e}"); eprintln!("FAIL: {e}");
// One schema error is no reason to hide the rest: the needs
// pass reads the same content and says what else is wrong,
// so a fix round can take them all at once.
let mut all = vec![schema_finding(&e.to_string())];
if let Some(file) = needs_raw.as_deref().and_then(|raw| needs::parse(raw).ok()) {
let findings = needs::check(&file, &questions, &aggregates_map);
print_findings(&findings);
all.extend(findings);
}
write_findings(findings_out.as_deref(), &all);
std::process::exit(1); std::process::exit(1);
} }
} }
} }
/// A content error the schema pass stopped at, as a finding: the repo
/// is its scope and the message its subject, since it has no other.
fn schema_finding(message: &str) -> needs::Finding {
needs::Finding {
need: String::new(),
level: needs::Level::Fail,
message: message.to_string(),
scope: "repo",
code: "schema",
subject: message.to_string(),
}
}
/// `--findings`: every finding as one JSON line (id, level, code,
/// scope, need, subject, message, fix), the same ones the text output
/// prints - an empty file when there are none, so a reader can tell
/// "clean" from "not run".
fn write_findings(out: Option<&str>, findings: &[needs::Finding]) {
let Some(out) = out else { return };
let lines: String = findings.iter().map(|f| format!("{}\n", f.to_json())).collect();
if let Err(e) = std::fs::write(out, lines) {
eprintln!("WARN: could not write the findings to {out}: {e}");
}
}
/// Each finding on its own line, FAIL or WARN first; the count of FAILs.
fn print_findings(findings: &[needs::Finding]) -> usize {
let mut failed = 0;
for finding in findings {
let level = match finding.level {
needs::Level::Fail => {
failed += 1;
"FAIL"
}
needs::Level::Warn => "WARN",
};
eprintln!("{level}: {} {:?}: {}", finding.scope, finding.need, finding.message);
}
failed
}
/// The business-needs pass (see `crate::lint::needs`): structure always, /// The business-needs pass (see `crate::lint::needs`): structure always,
/// then the wording tasks written out and/or an engine's answers /// then the wording tasks written out and/or an engine's answers
/// scored, when asked. A repo without `needs.yaml` skips all of it. /// scored, when asked. A repo without `needs.yaml` skips all of it.
fn check_needs( fn check_needs(
findings_out: Option<&str>,
raw: Option<&str>, raw: Option<&str>,
questions: &std::collections::HashMap<String, content::Question>, questions: &std::collections::HashMap<String, content::Question>,
aggregates_map: &std::collections::HashMap<String, aggregates::AggregateSchema>, aggregates_map: &std::collections::HashMap<String, aggregates::AggregateSchema>,
@@ -145,6 +201,7 @@ fn check_needs(
min_accuracy: f64, min_accuracy: f64,
) -> anyhow::Result<()> { ) -> anyhow::Result<()> {
let Some(raw) = raw else { let Some(raw) = raw else {
write_findings(findings_out, &[]);
if tasks_out.is_some() || answers_in.is_some() { if tasks_out.is_some() || answers_in.is_some() {
eprintln!("FAIL: --needs-tasks/--needs-score given, but the repo has no needs.yaml"); eprintln!("FAIL: --needs-tasks/--needs-score given, but the repo has no needs.yaml");
std::process::exit(1); std::process::exit(1);
@@ -155,21 +212,13 @@ fn check_needs(
Ok(file) => file, Ok(file) => file,
Err(e) => { Err(e) => {
eprintln!("FAIL: {e}"); eprintln!("FAIL: {e}");
write_findings(findings_out, &[schema_finding(&e.to_string())]);
std::process::exit(1); std::process::exit(1);
} }
}; };
let findings = needs::check(&file, questions, aggregates_map); let findings = needs::check(&file, questions, aggregates_map);
let mut failed = 0; let failed = print_findings(&findings);
for finding in &findings { write_findings(findings_out, &findings);
let level = match finding.level {
needs::Level::Fail => {
failed += 1;
"FAIL"
}
needs::Level::Warn => "WARN",
};
eprintln!("{level}: need {:?}: {}", finding.need, finding.message);
}
for need in &file.needs { for need in &file.needs {
if let Some(path) = needs::path_for(questions, need) { if let Some(path) = needs::path_for(questions, need) {
let steps: Vec<String> = path let steps: Vec<String> = path
+551 -30
View File
@@ -137,9 +137,55 @@ pub enum Level {
#[derive(Debug)] #[derive(Debug)]
pub struct Finding { pub struct Finding {
/// The need's id, or the bucket's name when `scope` is "bucket".
pub need: String, pub need: String,
pub level: Level, pub level: Level,
pub message: String, pub message: String,
/// "need" for a need's own path; "bucket" for what holds for every
/// record in a bucket whichever need brought it in.
pub scope: &'static str,
/// What kind of gap it is, stable across releases: `strand`,
/// `edge-unoffered`, `done-unreachable`, ... (docs/lint.md).
pub code: &'static str,
/// What in the content it is about: a state, a move (`from>to`), a
/// page. With the scope, need and code it makes the finding's id.
pub subject: String,
}
impl Finding {
/// The same gap keeps the same id however its message is worded,
/// so a fix round can tell what it closed, what is still open and
/// what it broke. FNV-1a over scope, need, code and subject.
pub fn id(&self) -> String {
let mut hash: u64 = 0xcbf2_9ce4_8422_2325;
for part in [self.scope, &self.need, self.code, &self.subject] {
for byte in part.bytes().chain(std::iter::once(0)) {
hash ^= u64::from(byte);
hash = hash.wrapping_mul(0x0100_0000_01b3);
}
}
format!("{:08x}", hash >> 32)
}
/// The change that closes it, when the lint knows one: what follows
/// "fix:" in the message.
pub fn fix(&self) -> Option<&str> {
self.message.split_once(" - fix: ").map(|(_, fix)| fix)
}
/// One line of `--findings` JSONL.
pub fn to_json(&self) -> serde_json::Value {
serde_json::json!({
"id": self.id(),
"level": match self.level { Level::Fail => "fail", Level::Warn => "warn" },
"code": self.code,
"scope": self.scope,
"need": self.need,
"subject": self.subject,
"message": self.message,
"fix": self.fix(),
})
}
} }
#[derive(Clone, Debug, PartialEq, Eq)] #[derive(Clone, Debug, PartialEq, Eq)]
@@ -255,6 +301,10 @@ pub fn path_for(questions: &Questions, need: &Need) -> Option<Vec<Step>> {
struct Desk { struct Desk {
page: String, page: String,
group: Option<String>, group: Option<String>,
/// The page's own `qualifies`, kept apart from the resource's
/// `requires_group`: when both are set and differ, only members of
/// both ever see the buttons.
page_group: Option<String>,
/// (from, to, button label) /// (from, to, button label)
moves: Vec<(String, String, String)>, moves: Vec<(String, String, String)>,
} }
@@ -272,6 +322,7 @@ fn desks_over(questions: &Questions, bucket: &str) -> Vec<Desk> {
desks.push(Desk { desks.push(Desk {
page: q.id.clone(), page: q.id.clone(),
group: resource.requires_group.clone().or_else(|| q.qualifies.clone()), group: resource.requires_group.clone().or_else(|| q.qualifies.clone()),
page_group: q.qualifies.clone(),
moves: resource moves: resource
.transitions .transitions
.iter() .iter()
@@ -284,15 +335,49 @@ fn desks_over(questions: &Questions, bucket: &str) -> Vec<Desk> {
desks desks
} }
/// Self-service moves into `bucket` - a visitor holding their own /// Self-service moves in `bucket` - a visitor holding their own
/// record's link, no group involved (unsubscribe). /// record's link, no group involved (unsubscribe, withdraw) - as
fn self_moves(questions: &Questions, bucket: &str) -> Vec<String> { /// (from, to), from the states each may be made from.
fn self_moves(questions: &Questions, bucket: &str, initial: &str) -> Vec<(String, String)> {
questions questions
.values() .values()
.flat_map(|q| &q.alternatives) .flat_map(|q| &q.alternatives)
.filter_map(|a| a.self_transition.as_ref()) .filter_map(|a| a.self_transition.as_ref())
.filter(|s| s.bucket == bucket) .filter(|s| s.bucket == bucket)
.map(|s| s.to.clone()) .flat_map(|s| s.from_states(initial).into_iter().map(|from| (from, s.to.clone())))
.collect()
}
/// The moves a clock makes in `schema` (`after:` on a state), as
/// (from, to). Nobody presses them, and they still move records.
fn clock_moves(schema: &AggregateSchema) -> Vec<(String, String)> {
let mut moves: Vec<(String, String)> =
schema.after_for_state.iter().map(|(from, after)| (from.clone(), after.to.clone())).collect();
moves.sort();
moves
}
/// States reachable from `start` along `moves`.
fn reachable(start: &str, moves: &[(String, String)]) -> HashSet<String> {
let mut reached = HashSet::from([start.to_string()]);
let mut queue = VecDeque::from([start.to_string()]);
while let Some(state) = queue.pop_front() {
for (_, to) in moves.iter().filter(|(from, _)| *from == state) {
if reached.insert(to.clone()) {
queue.push_back(to.clone());
}
}
}
reached
}
/// Who else offers `from -> to` in the bucket, as "page (group)", for
/// a finding that has to say where the missing button already is.
fn offered_elsewhere(desks: &[Desk], from: &str, to: &str) -> Vec<String> {
desks
.iter()
.filter(|d| d.moves.iter().any(|(f, t, _)| f == from && t == to))
.map(|d| format!("{} ({})", d.page, d.group.as_deref().unwrap_or("ungated")))
.collect() .collect()
} }
@@ -305,10 +390,18 @@ pub fn check(
) -> Vec<Finding> { ) -> Vec<Finding> {
let mut findings = Vec::new(); let mut findings = Vec::new();
let mut seen = HashSet::new(); let mut seen = HashSet::new();
// (bucket, state) a need already reported - stranding on its way
// to done, or out of its reach - so the bucket pass does not say
// it twice.
let mut strands_reported: HashSet<(String, String)> = HashSet::new();
let mut unreached_reported: HashSet<(String, String)> = HashSet::new();
for need in &file.needs { for need in &file.needs {
let mut report = |level: Level, message: String| { let mut report = |level: Level, code: &'static str, subject: String, message: String| {
findings.push(Finding { findings.push(Finding {
need: need.id.clone(), need: need.id.clone(),
scope: "need",
code,
subject,
level: if need.planned || need.paused { Level::Warn } else { level }, level: if need.planned || need.paused { Level::Warn } else { level },
message: match (need.paused, need.planned, level == Level::Fail) { message: match (need.paused, need.planned, level == Level::Fail) {
(true, _, true) => format!("paused, not open now: {message}"), (true, _, true) => format!("paused, not open now: {message}"),
@@ -318,15 +411,17 @@ pub fn check(
}) })
}; };
if !seen.insert(need.id.as_str()) { if !seen.insert(need.id.as_str()) {
report(Level::Fail, "declared more than once".to_string()); report(Level::Fail, "need-duplicate", String::new(), "declared more than once".to_string());
continue; continue;
} }
// The visitor's side: can they get their answer recorded? // The visitor's side: can they get their answer recorded?
match questions.get(&need.enters) { match questions.get(&need.enters) {
None => report(Level::Fail, format!("entry page {:?} does not exist", need.enters)), None => report(Level::Fail, "entry-missing", need.enters.clone(), format!("entry page {:?} does not exist", need.enters)),
Some(page) if !visible(page, need.enters_as.as_deref()) => report( Some(page) if !visible(page, need.enters_as.as_deref()) => report(
Level::Fail, Level::Fail,
"entry-gated",
need.enters.clone(),
format!( format!(
"entry page {:?} is gated on {:?}, which this visitor does not hold", "entry page {:?} is gated on {:?}, which this visitor does not hold",
need.enters, page.qualifies need.enters, page.qualifies
@@ -336,6 +431,8 @@ pub fn check(
if page.responsible.is_none() { if page.responsible.is_none() {
report( report(
Level::Warn, Level::Warn,
"entry-no-responsible",
need.enters.clone(),
format!("entry page {:?} names no one responsible to turn to", need.enters), format!("entry page {:?} names no one responsible to turn to", need.enters),
); );
} }
@@ -363,6 +460,8 @@ pub fn check(
.unwrap_or_default(); .unwrap_or_default();
report( report(
Level::Fail, Level::Fail,
"path-closed",
at.clone(),
format!( format!(
"the path exists but is closed: {at} is `disabled`, so the answer is advertised and refused" "the path exists but is closed: {at} is `disabled`, so the answer is advertised and refused"
), ),
@@ -370,6 +469,8 @@ pub fn check(
} }
None => report( None => report(
Level::Fail, Level::Fail,
"path-missing",
String::new(),
format!( format!(
"no path from {:?}{} records into {:?} within {} submission(s)", "no path from {:?}{} records into {:?} within {} submission(s)",
need.enters, need.enters,
@@ -386,6 +487,8 @@ pub fn check(
if best.fields > budget { if best.fields > budget {
report( report(
Level::Fail, Level::Fail,
"path-over-budget",
String::new(),
format!( format!(
"the path asks for {} required field(s), over the budget of {budget}", "the path asks for {} required field(s), over the budget of {budget}",
best.fields best.fields
@@ -396,6 +499,8 @@ pub fn check(
if best.records_with_action == Some(false) { if best.records_with_action == Some(false) {
report( report(
Level::Warn, Level::Warn,
"path-no-action",
String::new(),
"the recording alternative has no `action` - the visitor is never told what happens next".to_string(), "the recording alternative has no `action` - the visitor is never told what happens next".to_string(),
); );
} }
@@ -408,6 +513,8 @@ pub fn check(
let Some(schema) = aggregates.get(&need.lands_in) else { let Some(schema) = aggregates.get(&need.lands_in) else {
report( report(
Level::Fail, Level::Fail,
"machine-missing",
need.lands_in.clone(),
format!("aggregates.yaml declares no state machine for {:?}", need.lands_in), format!("aggregates.yaml declares no state machine for {:?}", need.lands_in),
); );
continue; continue;
@@ -426,6 +533,8 @@ pub fn check(
.collect(); .collect();
report( report(
Level::Fail, Level::Fail,
"desk-missing",
need.handled_by.clone(),
format!( format!(
"no desk gated on {:?} reads {:?}{}", "no desk gated on {:?} reads {:?}{}",
need.handled_by, need.handled_by,
@@ -439,23 +548,44 @@ pub fn check(
); );
continue; continue;
} }
let selfs = self_moves(questions, &need.lands_in); // Every move that carries this need's records: the buttons on
// the desks of handled_by/also_moved_by, the visitor's own
// links, and the clocks.
let mut moves: Vec<(String, String)> = own
.iter()
.flat_map(|d| &d.moves)
.filter(|(from, to, _)| schema.allowed(from).contains(to))
.map(|(from, to, _)| (from.clone(), to.clone()))
.collect();
moves.extend(self_moves(questions, &need.lands_in, &schema.initial));
moves.extend(clock_moves(schema));
let exits = |state: &str| -> Vec<String> { let exits = |state: &str| -> Vec<String> {
let allowed = schema.allowed(state); moves.iter().filter(|(from, _)| from == state).map(|(_, to)| to.clone()).collect()
let mut to: Vec<String> = own };
.iter() let movers = std::iter::once(need.handled_by.as_str())
.flat_map(|d| &d.moves) .chain(need.also_moved_by.iter().map(String::as_str))
.filter(|(from, to, _)| from == state && allowed.contains(to)) .map(|g| format!("{g:?}"))
.map(|(_, to, _)| to.clone()) .collect::<Vec<_>>()
.collect(); .join(" or ");
to.extend(selfs.iter().filter(|to| allowed.contains(to)).cloned()); // The fix for a missing move: the button on handled_by's desk,
to // or the group whose desk already has it added to also_moved_by.
let fix_for = |from: &str, to: &str| -> String {
let elsewhere = offered_elsewhere(&desks, from, to);
let desk = own.first().map(|d| d.page.as_str()).unwrap_or("their desk");
if elsewhere.is_empty() {
format!("add `{{ from: {from}, to: {to}, label: ... }}` to the transitions on {desk}")
} else {
format!(
"{} already offers it - add that group to also_moved_by, or add `{{ from: {from}, to: {to}, label: ... }}` on {desk}",
elsewhere.join(", ")
)
}
}; };
let done: HashSet<&str> = need.done_when.iter().map(String::as_str).collect(); let done: HashSet<&str> = need.done_when.iter().map(String::as_str).collect();
// Reachability walks the whole machine (a finished state may // Reachability walks the whole machine (a finished state may
// lead on to another: subscribed, then unsubscribed). Stranding // lead on to another: subscribed, then unsubscribed). Stranding
// only counts on the way to the first finished state - what // only counts on the way to the first finished state here; what
// happens to a handled record afterwards is not this need. // strands a record afterwards is the bucket's finding (below).
let mut reached: HashSet<String> = HashSet::from([schema.initial.clone()]); let mut reached: HashSet<String> = HashSet::from([schema.initial.clone()]);
let mut queue = VecDeque::from([(schema.initial.clone(), false)]); let mut queue = VecDeque::from([(schema.initial.clone(), false)]);
let mut stranded = Vec::new(); let mut stranded = Vec::new();
@@ -471,18 +601,44 @@ pub fn check(
} }
} }
} }
// The declared edges into `state` from somewhere a record does
// get to: the moves that would close the gap.
let ways_in = |state: &str| -> Vec<(String, String)> {
let mut ways: Vec<(String, String)> = reached
.iter()
.filter(|from| schema.allowed(from).iter().any(|to| to == state))
.map(|from| (from.clone(), state.to_string()))
.collect();
ways.sort();
ways
};
let missing = |state: &str| -> String {
match ways_in(state).first() {
Some((from, to)) => format!(" - fix: {}", fix_for(from, to)),
None => format!(
" - fix: no state a record reaches has a declared edge into it; add one in aggregates.yaml under {:?}",
need.lands_in
),
}
};
for state in &need.done_when { for state in &need.done_when {
if !schema.has_state(state) { if !schema.has_state(state) {
report( report(
Level::Fail, Level::Fail,
"done-unknown",
state.clone(),
format!("done_when names {state:?}, which {:?} has no such state", need.lands_in), format!("done_when names {state:?}, which {:?} has no such state", need.lands_in),
); );
} else if !reached.contains(state) { } else if !reached.contains(state) {
unreached_reported.insert((need.lands_in.clone(), state.clone()));
report( report(
Level::Fail, Level::Fail,
"done-unreachable",
state.clone(),
format!( format!(
"{:?} cannot move a record to {state:?} - no desk of theirs offers the moves", "{:?} cannot move a record to {state:?} - no desk of theirs offers the moves{}",
need.handled_by need.handled_by,
missing(state)
), ),
); );
} }
@@ -491,17 +647,48 @@ pub fn check(
if !schema.has_state(stage) { if !schema.has_state(stage) {
report( report(
Level::Fail, Level::Fail,
"stage-unknown",
stage.clone(),
format!( format!(
"the business describes a stage {stage:?} that {:?} has no state for", "the business describes a stage {stage:?} that {:?} has no state for",
need.lands_in need.lands_in
), ),
); );
} else if !reached.contains(stage) { } else if !reached.contains(stage) {
unreached_reported.insert((need.lands_in.clone(), stage.clone()));
report( report(
Level::Fail, Level::Fail,
"stage-unreachable",
stage.clone(),
format!( format!(
"stage {stage:?} exists but {:?} is offered no moves that reach it", "stage {stage:?} exists but {:?} is offered no moves that reach it{}",
need.handled_by need.handled_by,
missing(stage)
),
);
}
}
// In the business's order: each working stage reachable from
// the one before it, so a machine where "quoted" cannot follow
// "surveyed" does not pass for one that goes surveyed -> quoted.
// The finished states that close the list are alternatives
// (won, lost), not a sequence, so they are left out.
for pair in need.stages.windows(2) {
let (before, after) = (&pair[0], &pair[1]);
if done.contains(before.as_str()) || done.contains(after.as_str()) {
continue;
}
if reached.contains(before) && reached.contains(after) && !reachable(before, &moves).contains(after) {
report(
Level::Fail,
"stage-order",
format!("{before}>{after}"),
format!(
"stage {after:?} comes after {before:?} for the business, but no move leads on from {before:?} to it{}",
match schema.allowed(before).iter().find(|to| reachable(to, &moves).contains(after) || *to == after) {
Some(to) => format!(" - fix: {}", fix_for(before, to)),
None => format!(" - fix: add an edge from {before:?} toward {after:?} in aggregates.yaml under {:?}", need.lands_in),
}
), ),
); );
} }
@@ -510,27 +697,43 @@ pub fn check(
if !need.done_when.contains(state) { if !need.done_when.contains(state) {
report( report(
Level::Fail, Level::Fail,
"success-not-done",
state.clone(),
format!("success names {state:?}, which is not one of done_when"), format!("success names {state:?}, which is not one of done_when"),
); );
} }
} }
stranded.sort(); stranded.sort();
for state in &stranded {
strands_reported.insert((need.lands_in.clone(), state.clone()));
}
for state in stranded { for state in stranded {
let fix = match schema.allowed(&state).first() {
Some(to) => format!(" - fix: {}", fix_for(&state, to)),
None => format!(
" - fix: {:?} declares no way out of it; add {state:?} to done_when if it is finished, or an edge out in aggregates.yaml",
need.lands_in
),
};
report( report(
Level::Fail, Level::Fail,
format!( "strand",
"records strand in {state:?}: {:?} is offered no move out of it", state.clone(),
need.handled_by format!("records strand in {state:?}: {movers} is offered no move out of it{fix}"),
),
); );
} }
} }
findings.extend(operations(file, questions, aggregates, &strands_reported, &unreached_reported));
let need_ids: HashSet<&str> = file.needs.iter().map(|n| n.id.as_str()).collect(); let need_ids: HashSet<&str> = file.needs.iter().map(|n| n.id.as_str()).collect();
for persona in &file.personas { for persona in &file.personas {
if !need_ids.contains(persona.need.as_str()) { if !need_ids.contains(persona.need.as_str()) {
findings.push(Finding { findings.push(Finding {
need: persona.need.clone(), need: persona.need.clone(),
scope: "need",
code: "persona-unknown-need",
subject: persona.id.clone(),
level: Level::Fail, level: Level::Fail,
message: format!("persona {:?} points at a need that is not declared", persona.id), message: format!("persona {:?} points at a need that is not declared", persona.id),
}); });
@@ -539,6 +742,200 @@ pub fn check(
findings findings
} }
/// What must hold for every record in a bucket some need lands in,
/// whichever need brought it: every state is somewhere a record can
/// get to, no state a record gets to holds it with no way on (before
/// done or after), every declared move is a button someone has, every
/// desk can be seen by whoever its buttons are for, and every mail
/// goes out from a state records reach, saying only what the record
/// holds.
fn operations(
file: &NeedsFile,
questions: &Questions,
aggregates: &HashMap<String, AggregateSchema>,
strands_reported: &HashSet<(String, String)>,
unreached_reported: &HashSet<(String, String)>,
) -> Vec<Finding> {
let mut findings = Vec::new();
let mut buckets: Vec<&str> = file.needs.iter().map(|n| n.lands_in.as_str()).collect();
buckets.sort();
buckets.dedup();
for bucket in buckets {
let Some(schema) = aggregates.get(bucket) else { continue };
let needs: Vec<&Need> = file.needs.iter().filter(|n| n.lands_in == bucket).collect();
// Only open needs make a gap fail; a bucket that only planned
// or paused needs use reports it as a warning.
let open = needs.iter().any(|n| !n.planned && !n.paused);
let mut report = |level: Level, code: &'static str, subject: String, message: String| {
findings.push(Finding {
need: bucket.to_string(),
scope: "bucket",
code,
subject,
level: if open { level } else { Level::Warn },
message,
})
};
let desks = desks_over(questions, bucket);
let mut moves: Vec<(String, String)> = desks
.iter()
.flat_map(|d| &d.moves)
.filter(|(from, to, _)| schema.allowed(from).contains(to))
.map(|(from, to, _)| (from.clone(), to.clone()))
.collect();
moves.extend(self_moves(questions, bucket, &schema.initial));
moves.extend(clock_moves(schema));
let reached = reachable(&schema.initial, &moves);
let offered = |from: &str, to: &str| moves.iter().any(|(f, t)| f == from && t == to);
// Where a missing button goes: the desk of the group answerable
// for the bucket's first need.
let home = needs
.iter()
.find_map(|n| desks.iter().find(|d| d.group.as_deref() == Some(n.handled_by.as_str())))
.map(|d| d.page.clone())
.unwrap_or_else(|| "the handling group's desk".to_string());
let button = |from: &str, to: &str| format!("add `{{ from: {from}, to: {to}, label: ... }}` to the transitions on {home}");
let mut states: Vec<&String> = schema.event_for_state.keys().collect();
states.sort();
for state in &states {
if reached.contains(*state) || unreached_reported.contains(&(bucket.to_string(), (*state).clone())) {
continue;
}
let mut ways: Vec<&String> = reached.iter().filter(|from| schema.allowed(from).contains(state)).collect();
ways.sort();
report(
Level::Fail,
"state-unreached",
(*state).clone(),
format!(
"state {state:?} is never reached: no desk, link or clock moves a record there - fix: {}",
match ways.first() {
Some(from) => button(from, state),
None => "declare an edge into it from a state records reach in aggregates.yaml, or remove it".to_string(),
}
),
);
}
for state in states.iter().filter(|s| reached.contains(**s)) {
let out = schema.allowed(state);
if out.is_empty() {
continue; // a final state: nothing should move it
}
let offered_out: Vec<&String> = out.iter().filter(|to| offered(state, to)).collect();
if offered_out.is_empty() {
if !strands_reported.contains(&(bucket.to_string(), (*state).clone())) {
report(
Level::Fail,
"strand",
(*state).clone(),
format!(
"records strand in {state:?}: {bucket:?} declares moves on to {} but no desk, link or clock offers one - fix: {}",
out.join(", "),
button(state, &out[0])
),
);
}
continue;
}
for to in out.iter().filter(|to| !offered(state, to)) {
report(
Level::Warn,
"edge-unoffered",
format!("{state}>{to}"),
format!(
"the move {state:?} -> {to:?} is declared but no desk, link or clock offers it - fix: {}, or drop the edge from aggregates.yaml",
button(state, to)
),
);
}
}
for desk in &desks {
if let (Some(page_group), Some(group)) = (&desk.page_group, &desk.group) {
if page_group != group {
report(
Level::Warn,
"desk-gated-twice",
desk.page.clone(),
format!(
"the desk on {} is gated twice: the page qualifies {page_group:?}, its list requires {group:?}, so only members of both see the buttons - fix: make them the same group",
desk.page
),
);
}
}
}
// The fields a record of this bucket holds: everything asked on
// a form that records into it.
let fields: Vec<(&str, &str)> = questions
.values()
.flat_map(|q| &q.alternatives)
.filter(|a| a.record_as.as_deref() == Some(bucket))
.flat_map(|a| &a.features)
.flat_map(|f| &f.requirements)
.map(|r| (r.name.as_str(), r.kind.as_str()))
.collect();
let mut mail_states: Vec<(&String, &crate::aggregates::MailSpec)> = schema.mail_for_state.iter().collect();
mail_states.sort_by(|a, b| a.0.cmp(b.0));
for (state, mail) in mail_states {
if !reached.contains(state) {
report(Level::Warn, "mail-unreachable", state.clone(), format!("the mail on entering {state:?} never goes out: no record reaches {state:?} - fix: make {state:?} reachable, or drop its mail"));
}
let mut unknown: Vec<String> = Vec::new();
for key in placeholders(&mail.subject).into_iter().chain(placeholders(&mail.body)) {
let known = match key.split_once('.') {
Some((parent, _)) => fields.iter().any(|(name, kind)| *name == parent && *kind == "record"),
None => MAIL_BUILTINS.contains(&key.as_str()) || fields.iter().any(|(name, _)| *name == key),
};
if !known && !unknown.contains(&key) {
unknown.push(key);
}
}
if !unknown.is_empty() {
let mut names: Vec<&str> = fields.iter().map(|(n, _)| *n).collect();
names.sort();
names.dedup();
report(
Level::Fail,
"mail-unknown-field",
state.clone(),
format!(
"the mail on entering {state:?} uses {}, which no form recording into {bucket:?} collects, so it would go out blank there - fix: use one of the fields the forms collect ({}), or ask for it on the form",
unknown.iter().map(|k| format!("{{{k}}}")).collect::<Vec<_>>().join(", "),
if names.is_empty() { "nothing".to_string() } else { names.join(", ") }
),
);
}
}
}
findings
}
/// What the mailer fills in on every state mail, whatever the record
/// holds (`mail::vars_for`, `mail::site_vars`, `mail::record_links`).
const MAIL_BUILTINS: &[&str] = &["site", "site_name", "chain", "email", "bucket", "state", "link", "self_link", "desk_link"];
/// The `{key}`s in a mail template, read the way `mail::render` reads
/// them: letters, digits, `_`, `-` and `.`, closed by `}`.
fn placeholders(template: &str) -> Vec<String> {
let mut keys = Vec::new();
let mut rest = template;
while let Some(start) = rest.find('{') {
let after = &rest[start + 1..];
let len = after
.char_indices()
.take_while(|(_, c)| c.is_ascii_alphanumeric() || *c == '_' || *c == '-' || *c == '.')
.last()
.map(|(i, c)| i + c.len_utf8())
.unwrap_or(0);
if len > 0 && after[len..].starts_with('}') {
keys.push(after[..len].to_string());
}
rest = after;
}
keys
}
/// Everything a simulation of the business needs, resolved: each /// Everything a simulation of the business needs, resolved: each
/// need's path with the choice on every page and the form that gets /// need's path with the choice on every page and the form that gets
/// filled, each bucket's state machine with the buttons every desk /// filled, each bucket's state machine with the buttons every desk
@@ -611,6 +1008,11 @@ pub fn sim_model(
"steps": steps, "steps": steps,
})); }));
if let Some(schema) = aggregates.get(&need.lands_in) { if let Some(schema) = aggregates.get(&need.lands_in) {
// The simulator's shape: the states a visitor's own link
// moves to (it does not model where from).
let mut self_move_targets: Vec<String> =
self_moves(questions, &need.lands_in, &schema.initial).into_iter().map(|(_, to)| to).collect();
self_move_targets.dedup();
buckets.entry(need.lands_in.clone()).or_insert_with(|| { buckets.entry(need.lands_in.clone()).or_insert_with(|| {
json!({ json!({
"initial": schema.initial, "initial": schema.initial,
@@ -623,7 +1025,7 @@ pub fn sim_model(
"from": from, "to": to, "label": label, "from": from, "to": to, "label": label,
})).collect::<Vec<_>>(), })).collect::<Vec<_>>(),
})).collect::<Vec<_>>(), })).collect::<Vec<_>>(),
"self_moves": self_moves(questions, &need.lands_in), "self_moves": self_move_targets,
}) })
}); });
} }
@@ -895,8 +1297,8 @@ mod tests {
assert_eq!( assert_eq!(
fails(&findings), fails(&findings),
vec![ vec![
"\"owners\" cannot move a record to \"won\" - no desk of theirs offers the moves", "\"owners\" cannot move a record to \"won\" - no desk of theirs offers the moves - fix: add `{ from: quoted, to: won, label: ... }` to the transitions on /desk",
"records strand in \"quoted\": \"owners\" is offered no move out of it", "records strand in \"quoted\": \"owners\" is offered no move out of it - fix: add `{ from: quoted, to: won, label: ... }` to the transitions on /desk",
] ]
); );
} }
@@ -1037,4 +1439,123 @@ mod tests {
assert_eq!(score(&tasks, &[], 0.6).unanswered, vec!["founder@/"]); assert_eq!(score(&tasks, &[], 0.6).unanswered, vec!["founder@/"]);
} }
/// The fixture site with its aggregates replaced, for the checks
/// that are about the state machine rather than the pages.
fn site_with(desk_moves: &str, projects: &str) -> (Questions, HashMap<String, AggregateSchema>) {
let (questions, _) = site(desk_moves);
let aggregates = parse_aggregates_yaml(&format!(
"aggregates:\n - bucket: projects\n{projects} - bucket: subscribers\n attended_by: a mailer\n initial: open\n states:\n open: {{ event: subscribed }}\n transitions: {{}}\n"
))
.unwrap();
(questions, aggregates)
}
fn warns(findings: &[Finding]) -> Vec<&str> {
findings.iter().filter(|f| f.level == Level::Warn).map(|f| f.message.as_str()).collect()
}
#[test]
fn a_clock_is_a_move() {
// Nobody closes a stale quote; six weeks do. That is not a strand.
let (questions, aggregates) = site_with(
" - { to: quoted, label: Quote }\n - { from: quoted, to: won, label: Win }\n",
" initial: open\n states:\n open: { event: submitted }\n quoted: { event: quoted, after: { days: 42, to: lost } }\n won: { event: won }\n lost: { event: lost }\n transitions:\n open: [quoted]\n quoted: [won, lost]\n",
);
let findings = check(&needs(""), &questions, &aggregates);
assert!(fails(&findings).is_empty(), "{findings:?}");
}
#[test]
fn a_state_nobody_reaches_and_a_strand_after_done_are_the_buckets() {
// "archived" follows "won" but no desk offers it; "parked" has
// no way in at all.
let (questions, aggregates) = site_with(
FULL_DESK,
" initial: open\n states:\n open: { event: submitted }\n quoted: { event: quoted }\n won: { event: won }\n lost: { event: lost }\n archived: { event: archived }\n done: { event: done }\n parked: { event: parked }\n transitions:\n open: [quoted, lost]\n quoted: [won, lost]\n won: [archived]\n archived: [done]\n parked: [open]\n",
);
let findings = check(&needs(""), &questions, &aggregates);
let bucket: Vec<&Finding> = findings.iter().filter(|f| f.scope == "bucket").collect();
assert!(bucket.iter().all(|f| f.need == "projects"), "{bucket:?}");
assert_eq!(
fails(&findings),
vec![
"state \"archived\" is never reached: no desk, link or clock moves a record there - fix: add `{ from: won, to: archived, label: ... }` to the transitions on /desk",
"state \"done\" is never reached: no desk, link or clock moves a record there - fix: declare an edge into it from a state records reach in aggregates.yaml, or remove it",
"state \"parked\" is never reached: no desk, link or clock moves a record there - fix: declare an edge into it from a state records reach in aggregates.yaml, or remove it",
"records strand in \"won\": \"projects\" declares moves on to archived but no desk, link or clock offers one - fix: add `{ from: won, to: archived, label: ... }` to the transitions on /desk",
]
);
}
#[test]
fn an_edge_nobody_offers_is_a_warning_when_there_is_another_way_on() {
let (questions, aggregates) = site(" - { to: quoted, label: Quote }\n - { from: quoted, to: won, label: Win }\n - { from: quoted, to: lost, label: Lose }\n");
let findings = check(&needs(""), &questions, &aggregates);
assert!(fails(&findings).is_empty(), "{findings:?}");
assert_eq!(
warns(&findings),
vec!["the move \"open\" -> \"lost\" is declared but no desk, link or clock offers it - fix: add `{ from: open, to: lost, label: ... }` to the transitions on /desk, or drop the edge from aggregates.yaml"]
);
}
#[test]
fn working_stages_must_follow_each_other() {
// The business surveys, then quotes; this machine can only
// quote straight from open, and surveying leads nowhere.
let (questions, aggregates) = site_with(
" - { to: quoted, label: Quote }\n - { to: surveyed, label: Survey }\n - { from: surveyed, to: lost, label: Lose }\n - { from: quoted, to: won, label: Win }\n - { from: quoted, to: lost, label: Lose }\n",
" initial: open\n states:\n open: { event: submitted }\n surveyed: { event: surveyed }\n quoted: { event: quoted }\n won: { event: won }\n lost: { event: lost }\n transitions:\n open: [surveyed, quoted]\n surveyed: [lost]\n quoted: [won, lost]\n",
);
let findings = check(&needs(" stages: [open, surveyed, quoted, won, lost]\n"), &questions, &aggregates);
assert_eq!(
fails(&findings),
vec!["stage \"quoted\" comes after \"surveyed\" for the business, but no move leads on from \"surveyed\" to it - fix: add an edge from \"surveyed\" toward \"quoted\" in aggregates.yaml under \"projects\""]
);
}
#[test]
fn a_mail_may_only_say_what_the_record_holds() {
let (questions, aggregates) = site_with(
FULL_DESK,
" initial: open\n states:\n open: { event: submitted, mail: { to: submitter, subject: 'Got it, {email}', body: 'We read {brief} and will reply at {site}. {refnr} {link}' } }\n quoted: { event: quoted }\n won: { event: won }\n lost: { event: lost }\n transitions:\n open: [quoted, lost]\n quoted: [won, lost]\n",
);
let findings = check(&needs(""), &questions, &aggregates);
assert_eq!(
fails(&findings),
vec!["the mail on entering \"open\" uses {refnr}, which no form recording into \"projects\" collects, so it would go out blank there - fix: use one of the fields the forms collect (brief, budget, email), or ask for it on the form"]
);
}
#[test]
fn a_desk_gated_twice_is_seen_only_by_members_of_both() {
let (mut questions, aggregates) = site(FULL_DESK);
questions.get_mut("/desk").unwrap().qualifies = Some("staff".into());
let findings = check(&needs(""), &questions, &aggregates);
assert!(
warns(&findings).contains(&"the desk on /desk is gated twice: the page qualifies \"staff\", its list requires \"owners\", so only members of both see the buttons - fix: make them the same group"),
"{findings:?}"
);
}
#[test]
fn a_finding_keeps_its_id_when_reworded_and_says_its_fix() {
let (questions, aggregates) = site(" - { to: quoted, label: Quote }\n - { to: lost, label: Lose }\n");
let findings = check(&needs(""), &questions, &aggregates);
let strand = findings.iter().find(|f| f.code == "strand").unwrap();
assert_eq!(strand.subject, "quoted");
assert_eq!(strand.fix(), Some("add `{ from: quoted, to: won, label: ... }` to the transitions on /desk"));
let reworded = Finding { message: "something else entirely".into(), ..clone_of(strand) };
assert_eq!(reworded.id(), strand.id());
let elsewhere = Finding { subject: "open".into(), ..clone_of(strand) };
assert_ne!(elsewhere.id(), strand.id());
let json = strand.to_json();
assert_eq!(json["code"], "strand");
assert_eq!(json["level"], "fail");
assert_eq!(json["id"], strand.id());
}
fn clone_of(f: &Finding) -> Finding {
Finding { need: f.need.clone(), level: f.level, message: f.message.clone(), scope: f.scope, code: f.code, subject: f.subject.clone() }
}
} }
+144 -6
View File
@@ -251,6 +251,80 @@ pub fn vars_for(bucket: &str, id: &str, state: &str, responses: &serde_json::Val
vars vars
} }
/// What every mail portal sends may say, whatever made it - a state
/// mail, an invite, a suggestion's answer: `{site}` (the instance's
/// public URL) and `{site_name}` (site.yaml's title). One place, so a
/// template written for one kind of mail reads the same in another
/// (docs/mail.md, Variables).
pub fn site_vars(site: &crate::content::SiteConfig) -> serde_json::Map<String, serde_json::Value> {
let mut vars = serde_json::Map::new();
vars.insert("site".into(), public_url().into());
vars.insert("site_name".into(), site_name(site).into());
vars
}
/// Where a mail about a record can send its reader (docs/mail.md):
///
/// - `{self_link}`: the record's own page for the person it belongs
/// to - a page offering a self-service move on this bucket from the
/// state the record is in, with the `?chain=&email=` that
/// lets them act without signing in;
/// - `{desk_link}`: the desk that moves the record - the page whose
/// list over the bucket carries transitions, else one that reads it;
/// - `{link}`: `{self_link}` in a mail to the submitter, `{desk_link}`
/// in one to anyone else, and the site itself when there is neither.
///
/// Empty strings, never missing keys, for a link the content has no
/// page for: a template naming one renders blank there rather than
/// with a broken URL.
pub fn record_links(
questions: &std::collections::HashMap<String, crate::content::Question>,
initial: &str,
bucket: &str,
id: &str,
state: &str,
email: &str,
to_submitter: bool,
) -> serde_json::Map<String, serde_json::Value> {
let mut pages: Vec<&crate::content::Question> = questions.values().collect();
pages.sort_by(|a, b| a.id.cmp(&b.id));
// Only a move the engine would accept from where the record is:
// a link to a button that refuses is worse than none.
let self_page = pages.iter().find(|q| {
q.alternatives
.iter()
.filter_map(|a| a.self_transition.as_ref())
.any(|st| st.bucket == bucket && st.from_states(initial).iter().any(|f| f == state))
});
let reads = |q: &&crate::content::Question, moving: bool| {
q.alternatives
.iter()
.flat_map(|a| &a.features)
.filter_map(|f| f.resource.as_ref())
.any(|r| r.bucket() == Some(bucket) && (!moving || !r.transitions.is_empty()))
};
let desk_page = pages.iter().find(|q| reads(q, true)).or_else(|| pages.iter().find(|q| reads(q, false)));
let site = public_url();
let self_link = match self_page {
Some(q) if email.contains('@') => {
let email: String = url::form_urlencoded::byte_serialize(email.as_bytes()).collect();
format!("{site}{}?chain={id}&email={email}", q.id)
}
_ => String::new(),
};
let desk_link = desk_page.map(|q| format!("{site}{}", q.id)).unwrap_or_default();
let link = match (to_submitter, self_link.is_empty(), desk_link.is_empty()) {
(true, false, _) => self_link.clone(),
(false, _, false) => desk_link.clone(),
_ => site,
};
let mut vars = serde_json::Map::new();
vars.insert("self_link".into(), self_link.into());
vars.insert("desk_link".into(), desk_link.into());
vars.insert("link".into(), link.into());
vars
}
/// The record field a `to` reaches through, when it is neither /// The record field a `to` reaches through, when it is neither
/// `submitter` nor an address: `{notice.email}` gives `notice`. /// `submitter` nor an address: `{notice.email}` gives `notice`.
pub fn parent_of(to: &str) -> Option<&str> { pub fn parent_of(to: &str) -> Option<&str> {
@@ -325,6 +399,13 @@ pub async fn on_state(state: &crate::server::AppState, bucket: &str, id: &str, t
}; };
let mut vars = vars_for(bucket, id, to_state, responses); let mut vars = vars_for(bucket, id, to_state, responses);
parent_vars(state, bucket, responses, &mut vars).await; parent_vars(state, bucket, responses, &mut vars).await;
vars.extend(site_vars(site));
{
let email = vars.get("email").and_then(|v| v.as_str()).unwrap_or_default().to_string();
let questions = state.questions.load();
let initial = state.aggregates.load().at(storyline).get(bucket).map(|s| s.initial.clone()).unwrap_or_default();
vars.extend(record_links(questions.at(storyline), &initial, bucket, id, to_state, &email, spec.to_submitter()));
}
let to = if spec.to_submitter() { let to = if spec.to_submitter() {
match vars.get("email").and_then(|v| v.as_str()).filter(|e| e.contains('@')) { match vars.get("email").and_then(|v| v.as_str()).filter(|e| e.contains('@')) {
Some(e) => e.to_string(), Some(e) => e.to_string(),
@@ -416,12 +497,19 @@ pub async fn on_responsible(state: &crate::server::AppState, invited: &crate::pe
.map(|t| t.format("%Y-%m-%d %H:%M UTC").to_string()) .map(|t| t.format("%Y-%m-%d %H:%M UTC").to_string())
.unwrap_or_default(); .unwrap_or_default();
let mut vars = serde_json::Map::new(); let mut vars = serde_json::Map::new();
// The form's own fields first, so the invite's fixed placeholders
// win a clash; a date and time reads as one.
for (key, value) in &invited.fields {
let text = value.as_str().map(|v| spoken_datetime(v, norwegian).unwrap_or_else(|| v.to_string()));
if let Some(text) = text {
vars.insert(key.clone(), text.into());
}
}
vars.insert("name".into(), invited.display_name.clone().into()); vars.insert("name".into(), invited.display_name.clone().into());
vars.insert("username".into(), invited.username.clone().into()); vars.insert("username".into(), invited.username.clone().into());
vars.insert("questions".into(), list.into()); vars.insert("questions".into(), list.into());
vars.extend(site_vars(&site));
vars.insert("link".into(), invited.reset_link.clone().unwrap_or_default().into()); vars.insert("link".into(), invited.reset_link.clone().unwrap_or_default().into());
vars.insert("site".into(), public_url().into());
vars.insert("site_name".into(), site_name(&site).into());
vars.insert("expires".into(), expires.into()); vars.insert("expires".into(), expires.into());
let text = render(&body, &vars); let text = render(&body, &vars);
let mail = Outgoing { let mail = Outgoing {
@@ -493,7 +581,7 @@ async fn on_suggestion_state(state: &crate::server::AppState, id: &str, to_state
return; return;
} }
let quoted: String = field("text").lines().map(|l| format!("> {l}")).collect::<Vec<_>>().join("\n"); let quoted: String = field("text").lines().map(|l| format!("> {l}")).collect::<Vec<_>>().join("\n");
let mut vars = serde_json::Map::new(); let mut vars = site_vars(&site);
vars.insert("name".into(), field("name").into()); vars.insert("name".into(), field("name").into());
vars.insert("question".into(), field("question").into()); vars.insert("question".into(), field("question").into());
vars.insert("page".into(), format!("{}{}", public_url(), field("page")).into()); vars.insert("page".into(), format!("{}{}", public_url(), field("page")).into());
@@ -553,6 +641,25 @@ pub async fn on_edit_suggestion(state: &crate::server::AppState, question_id: &s
/// Built in, since portal knows everything about it; `mail.invite` in /// Built in, since portal knows everything about it; `mail.invite` in
/// site.yaml rewords it. `{name}`, `{email}`, `{group}`, `{link}`, /// site.yaml rewords it. `{name}`, `{email}`, `{group}`, `{link}`,
/// `{site}`, `{expires}`. /// `{site}`, `{expires}`.
/// A form's date and time (`2026-10-06T15:00`, from a `datetime-local`
/// field) as a person says it: "tirsdag 6. oktober kl. 15:00", "Tuesday
/// 6 October at 15:00". Anything else is left as it is.
pub fn spoken_datetime(value: &str, norwegian: bool) -> Option<String> {
use chrono::{Datelike, NaiveDateTime, Timelike};
let t = NaiveDateTime::parse_from_str(value, "%Y-%m-%dT%H:%M").ok()?;
let day = t.weekday().num_days_from_monday() as usize;
let month = t.month0() as usize;
Some(if norwegian {
const DAYS: [&str; 7] = ["mandag", "tirsdag", "onsdag", "torsdag", "fredag", "lørdag", "søndag"];
const MONTHS: [&str; 12] = ["januar", "februar", "mars", "april", "mai", "juni", "juli", "august", "september", "oktober", "november", "desember"];
format!("{} {}. {} kl. {:02}:{:02}", DAYS[day], t.day(), MONTHS[month], t.hour(), t.minute())
} else {
const DAYS: [&str; 7] = ["Monday", "Tuesday", "Wednesday", "Thursday", "Friday", "Saturday", "Sunday"];
const MONTHS: [&str; 12] = ["January", "February", "March", "April", "May", "June", "July", "August", "September", "October", "November", "December"];
format!("{} {} {} at {:02}:{:02}", DAYS[day], t.day(), MONTHS[month], t.hour(), t.minute())
})
}
pub async fn on_invite(state: &crate::server::AppState, invited: &crate::people::Invited) { pub async fn on_invite(state: &crate::server::AppState, invited: &crate::people::Invited) {
let site = state.site.load(); let site = state.site.load();
let Some(cfg) = &site.mail else { let Some(cfg) = &site.mail else {
@@ -560,7 +667,7 @@ pub async fn on_invite(state: &crate::server::AppState, invited: &crate::people:
return; return;
}; };
let norwegian = site.lang.as_deref().is_some_and(|l| l.starts_with("no") || l.starts_with("nb") || l.starts_with("nn")); let norwegian = site.lang.as_deref().is_some_and(|l| l.starts_with("no") || l.starts_with("nb") || l.starts_with("nn"));
let (subject, body) = match (&cfg.invite, invited.reset_link.is_some(), norwegian) { let (subject, body) = match (invited.mail.as_ref().or(cfg.invite.as_ref()), invited.reset_link.is_some(), norwegian) {
(Some(t), _, _) => (t.subject.clone(), t.body.clone()), (Some(t), _, _) => (t.subject.clone(), t.body.clone()),
(None, true, true) => ( (None, true, true) => (
"Du er invitert inn i {group} på {site_name}".to_string(), "Du er invitert inn i {group} på {site_name}".to_string(),
@@ -590,9 +697,8 @@ pub async fn on_invite(state: &crate::server::AppState, invited: &crate::people:
vars.insert("username".into(), invited.username.clone().into()); vars.insert("username".into(), invited.username.clone().into());
vars.insert("email".into(), invited.email.clone().into()); vars.insert("email".into(), invited.email.clone().into());
vars.insert("group".into(), invited.group.clone().into()); vars.insert("group".into(), invited.group.clone().into());
vars.extend(site_vars(&site));
vars.insert("link".into(), invited.reset_link.clone().unwrap_or_default().into()); vars.insert("link".into(), invited.reset_link.clone().unwrap_or_default().into());
vars.insert("site".into(), public_url().into());
vars.insert("site_name".into(), site_name(&site).into());
vars.insert("expires".into(), expires.into()); vars.insert("expires".into(), expires.into());
let text = render(&body, &vars); let text = render(&body, &vars);
let mail = Outgoing { let mail = Outgoing {
@@ -613,6 +719,30 @@ pub async fn on_invite(state: &crate::server::AppState, invited: &crate::people:
#[cfg(test)] #[cfg(test)]
mod tests { mod tests {
#[test]
fn a_record_mail_links_its_reader_to_where_they_act() {
let files = vec![
("index.yaml".to_string(), "name: Meld fra\nalternatives:\n - name: Feil\n action: /takk\n record_as: feil\n features:\n - name: Feil\n requirements:\n - { name: email, type: email }\n".to_string()),
("takk.yaml".to_string(), "name: Takk\nfollowup: true\n".to_string()),
("trekk.yaml".to_string(), "name: Trekk saken\nalternatives:\n - name: Trekk\n self_transition: { bucket: feil, to: trukket, label: Trekk, from: [open, befaring] }\n".to_string()),
("styret/index.yaml".to_string(), "name: Styret\nqualifies: styret\nalternatives:\n - name: Feil\n features:\n - name: \"\"\n resource:\n source: { kind: kv, bucket: feil }\n requires_group: styret\n transitions:\n - { to: befaring, label: Befaring }\n".to_string()),
];
let questions = crate::content::build_questions(&files).unwrap();
let site = public_url();
let to_tenant = record_links(&questions, "open", "feil", "abc", "befaring", "kari@x.no", true);
assert_eq!(to_tenant["self_link"], format!("{site}/trekk?chain=abc&email=kari%40x.no"));
assert_eq!(to_tenant["desk_link"], format!("{site}/styret"));
assert_eq!(to_tenant["link"], to_tenant["self_link"]);
let to_board = record_links(&questions, "open", "feil", "abc", "befaring", "kari@x.no", false);
assert_eq!(to_board["link"], to_board["desk_link"]);
// Past where the self-service move can be made: the person has
// no page of their own, so `{link}` is the site.
let after = record_links(&questions, "open", "feil", "abc", "utbedret", "kari@x.no", true);
assert_eq!(after["self_link"], "");
assert_eq!(after["link"], site);
}
use super::*; use super::*;
#[test] #[test]
@@ -662,6 +792,13 @@ mod tests {
assert_eq!(vars["state"], "fixed"); assert_eq!(vars["state"], "fixed");
} }
#[test]
fn a_date_and_time_from_a_form_reads_as_a_person_says_it() {
assert_eq!(spoken_datetime("2026-10-06T15:00", true).as_deref(), Some("tirsdag 6. oktober kl. 15:00"));
assert_eq!(spoken_datetime("2026-10-06T15:00", false).as_deref(), Some("Tuesday 6 October at 15:00"));
assert_eq!(spoken_datetime("Anders Kluge", true), None);
}
#[test] #[test]
fn the_html_alternative_keeps_markdown_and_drops_raw_html() { fn the_html_alternative_keeps_markdown_and_drops_raw_html() {
let html = to_html("Hello **there**\n\n<script>x()</script>\n\n[link](https://x.no)"); let html = to_html("Hello **there**\n\n<script>x()</script>\n\n[link](https://x.no)");
@@ -709,6 +846,7 @@ mod tests {
decided_ms: None, decided_ms: None,
decided_by: None, decided_by: None,
notes: Vec::new(), notes: Vec::new(),
finished: false,
} }
} }
+53 -22
View File
@@ -1,5 +1,6 @@
/// One binary, three jobs: `portal` (or `portal serve`) runs a site; /// One binary, three jobs: `portal` (or `portal serve`) runs a site;
/// `portal lint` checks a content repo the way the site reads it; /// `portal iris` checks a content repo the way the site reads it (the
/// iris keeps bad content out of the portal; `portal lint` is the same);
/// `portal plan`, `build`, `report` and the rest train one. The lint /// `portal plan`, `build`, `report` and the rest train one. The lint
/// and the trainer are the same code as the server, at the same /// and the trainer are the same code as the server, at the same
/// version, so they never disagree about what content means. /// version, so they never disagree about what content means.
@@ -8,7 +9,7 @@
async fn main() -> anyhow::Result<()> { async fn main() -> anyhow::Result<()> {
let argv: Vec<String> = std::env::args().skip(1).collect(); let argv: Vec<String> = std::env::args().skip(1).collect();
match argv.first().map(String::as_str) { match argv.first().map(String::as_str) {
Some("lint") => return portal::lint::run(argv[1..].to_vec()).await, Some("iris" | "lint") => return portal::lint::run(argv[1..].to_vec()).await,
Some("plan" | "build" | "answer" | "rate" | "prefer" | "score" | "simulate" | "compare" | "report" | "generate") => { Some("plan" | "build" | "answer" | "rate" | "prefer" | "score" | "simulate" | "compare" | "report" | "generate") => {
// The trainer runs models through shell commands and waits // The trainer runs models through shell commands and waits
// on them; off the runtime's threads, as a CLI should. // on them; off the runtime's threads, as a CLI should.
@@ -19,14 +20,21 @@ async fn main() -> anyhow::Result<()> {
return Ok(()); return Ok(());
} }
Some("serve") | None => {} Some("serve") | None => {}
Some(other) => anyhow::bail!("portal: unknown command {other:?}; one of serve, lint, plan, build, answer, rate, prefer, score, simulate, compare, report, generate"), Some(other) => anyhow::bail!("portal: unknown command {other:?}; one of serve, iris (or lint), plan, build, answer, rate, prefer, score, simulate, compare, report, generate"),
} }
serve().await serve().await
} }
#[cfg(feature = "ssr")] #[cfg(feature = "ssr")]
async fn serve() -> anyhow::Result<()> { async fn serve() -> anyhow::Result<()> {
use axum::{body::Body, extract::State, http::Request, response::IntoResponse, routing::{any, get, post}, Router}; use axum::{
body::Body,
extract::State,
http::Request,
response::IntoResponse,
routing::{any, get, post},
Router,
};
use leptos::prelude::*; use leptos::prelude::*;
use leptos_axum::{generate_route_list, LeptosRoutes}; use leptos_axum::{generate_route_list, LeptosRoutes};
use portal::app::{shell, App}; use portal::app::{shell, App};
@@ -56,12 +64,15 @@ async fn serve() -> anyhow::Result<()> {
let gitea_base = content::gitea_api_base(&content_repo)?; let gitea_base = content::gitea_api_base(&content_repo)?;
let content_raw_base = content::gitea_raw_base(&content_repo)?; let content_raw_base = content::gitea_raw_base(&content_repo)?;
let aggregates = content::load_aggregates_from_gitea(&content_repo, &content_branch).await?; let aggregates = content::load_aggregates_from_gitea(&content_repo, &content_branch).await?;
let questions = content::load_questions_from_gitea(&content_repo, &content_branch, "questions").await?; let questions =
content::load_questions_from_gitea(&content_repo, &content_branch, "questions").await?;
content::validate_questions(&questions, &aggregates)?; content::validate_questions(&questions, &aggregates)?;
let site = content::load_site_from_gitea(&content_repo, &content_branch).await?; let site = content::load_site_from_gitea(&content_repo, &content_branch).await?;
let storylines = content::same_arity(questions.arity(), aggregates.arity(), site.arity())?; let storylines = content::same_arity(questions.arity(), aggregates.arity(), site.arity())?;
tracing::info!(count = questions.len(), storylines, repo = %content_repo, branch = %content_branch, "loaded content"); tracing::info!(count = questions.len(), storylines, repo = %content_repo, branch = %content_branch, "loaded content");
let access = Arc::new(arc_swap::ArcSwap::from_pointee(portal::access::Policy::from_content(&questions, &aggregates))); let access = Arc::new(arc_swap::ArcSwap::from_pointee(
portal::access::Policy::from_content(&questions, &aggregates),
));
let questions = Arc::new(arc_swap::ArcSwap::from_pointee(questions)); let questions = Arc::new(arc_swap::ArcSwap::from_pointee(questions));
let aggregates = Arc::new(arc_swap::ArcSwap::from_pointee(aggregates)); let aggregates = Arc::new(arc_swap::ArcSwap::from_pointee(aggregates));
let site = Arc::new(arc_swap::ArcSwap::from_pointee(site)); let site = Arc::new(arc_swap::ArcSwap::from_pointee(site));
@@ -127,7 +138,12 @@ async fn serve() -> anyhow::Result<()> {
// The same decisions, answerable by other apps over NATS, for a // The same decisions, answerable by other apps over NATS, for a
// person named by phone number or address as well as by groups. // person named by phone number or address as well as by groups.
tokio::spawn(portal::access::serve(state.nats.clone(), site_name_for_subjects(), state.access.clone(), Some(state.clone()))); tokio::spawn(portal::access::serve(
state.nats.clone(),
site_name_for_subjects(),
state.access.clone(),
Some(state.clone()),
));
// Announced pages: keep their desk records current, close ended // Announced pages: keep their desk records current, close ended
// windows (see announce.rs). // windows (see announce.rs).
tokio::spawn(portal::announce::run(state.clone())); tokio::spawn(portal::announce::run(state.clone()));
@@ -201,18 +217,17 @@ async fn serve() -> anyhow::Result<()> {
}); });
match &governor { match &governor {
Some(_) => tracing::info!(rate_per_second, rate_burst, "rate limiting per caller IP"), Some(_) => tracing::info!(rate_per_second, rate_burst, "rate limiting per caller IP"),
None => tracing::warn!("rate limiting is off (PORTAL_RATE_PER_SECOND or PORTAL_RATE_BURST is 0)"), None => tracing::warn!(
"rate limiting is off (PORTAL_RATE_PER_SECOND or PORTAL_RATE_BURST is 0)"
),
} }
async fn server_fn_handler( async fn server_fn_handler(
State(state): State<AppState>, State(state): State<AppState>,
request: Request<Body>, request: Request<Body>,
) -> impl IntoResponse { ) -> impl IntoResponse {
leptos_axum::handle_server_fns_with_context( leptos_axum::handle_server_fns_with_context(move || provide_context(state.clone()), request)
move || provide_context(state.clone()), .await
request,
)
.await
} }
// The app's own Leptos route list includes a root-level wildcard // The app's own Leptos route list includes a root-level wildcard
@@ -263,10 +278,7 @@ async fn serve() -> anyhow::Result<()> {
.route_service("/favicon-light.svg", ServeFile::new(favicon_light_path)) .route_service("/favicon-light.svg", ServeFile::new(favicon_light_path))
.route_service("/favicon-dark.svg", ServeFile::new(favicon_dark_path)) .route_service("/favicon-dark.svg", ServeFile::new(favicon_dark_path))
.route_service("/wordmark.svg", ServeFile::new(wordmark_path)) .route_service("/wordmark.svg", ServeFile::new(wordmark_path))
.route_service( .route_service("/swiper-element-bundle.min.js", ServeFile::new(swiper_path))
"/swiper-element-bundle.min.js",
ServeFile::new(swiper_path),
)
// Same name every release, so revalidated like /pkg. // Same name every release, so revalidated like /pkg.
.route_service( .route_service(
"/page-motion.js", "/page-motion.js",
@@ -290,7 +302,9 @@ async fn serve() -> anyhow::Result<()> {
// with room for the multipart framing. // with room for the multipart framing.
.route( .route(
"/upload", "/upload",
post(upload::upload).layer(axum::extract::DefaultBodyLimit::max(upload::MAX_UPLOAD_BYTES + 64 * 1024)), post(upload::upload).layer(axum::extract::DefaultBodyLimit::max(
upload::MAX_UPLOAD_BYTES + 64 * 1024,
)),
) )
.route( .route(
"/attachment/{bucket}/{record}/{field}", "/attachment/{bucket}/{record}/{field}",
@@ -301,7 +315,11 @@ async fn serve() -> anyhow::Result<()> {
.route("/report", get(portal::report::handler)) .route("/report", get(portal::report::handler))
// Content-shipped assets (hero module etc.), same-origin. // Content-shipped assets (hero module etc.), same-origin.
.route("/site/{*path}", get(content::site_asset_handler)) .route("/site/{*path}", get(content::site_asset_handler))
.route("/automation/kv/{bucket}", get(content::automation_kv_handler)) .route("/.well-known/{*path}", get(content::well_known_handler))
.route(
"/automation/kv/{bucket}",
get(content::automation_kv_handler),
)
// Maps, proxied: the browser asks this origin for every tile // Maps, proxied: the browser asks this origin for every tile
// and every still, so nothing third-party loads in a page and // and every still, so nothing third-party loads in a page and
// the token stays here (see maps.rs). // the token stays here (see maps.rs).
@@ -323,7 +341,10 @@ async fn serve() -> anyhow::Result<()> {
.fallback(leptos_axum::file_and_error_handler::<AppState, _>(shell)) .fallback(leptos_axum::file_and_error_handler::<AppState, _>(shell))
.layer(session_layer) .layer(session_layer)
// Which storyline this visitor reads, before anything renders. // Which storyline this visitor reads, before anything renders.
.layer(axum::middleware::from_fn_with_state(state.clone(), portal::server::storyline::assign)); .layer(axum::middleware::from_fn_with_state(
state.clone(),
portal::server::storyline::assign,
));
// Outermost, so a caller who is over the limit is turned away // Outermost, so a caller who is over the limit is turned away
// before a session is loaded or a handler is entered. // before a session is loaded or a handler is entered.
let app = match governor { let app = match governor {
@@ -361,7 +382,17 @@ fn site_name_for_subjects() -> String {
let raw = std::env::var("SITE_NAME").unwrap_or_else(|_| "portal".to_string()); let raw = std::env::var("SITE_NAME").unwrap_or_else(|_| "portal".to_string());
let folded: String = raw let folded: String = raw
.chars() .chars()
.map(|c| if c.is_ascii_alphanumeric() || c == '_' || c == '-' { c.to_ascii_lowercase() } else { '_' }) .map(|c| {
if c.is_ascii_alphanumeric() || c == '_' || c == '-' {
c.to_ascii_lowercase()
} else {
'_'
}
})
.collect(); .collect();
if folded.is_empty() { "portal".to_string() } else { folded } if folded.is_empty() {
"portal".to_string()
} else {
folded
}
} }
+3
View File
@@ -254,6 +254,8 @@ pub async fn grant(state: &crate::server::AppState, group: &str, name: &str, who
reset_link: None, reset_link: None,
link_expires_ms: None, link_expires_ms: None,
created, created,
mail: None,
fields: Default::default(),
already_member: !created, already_member: !created,
}; };
match existing { match existing {
@@ -351,6 +353,7 @@ mod tests {
decided_ms: None, decided_ms: None,
decided_by: None, decided_by: None,
notes: Vec::new(), notes: Vec::new(),
finished: false,
} }
} }
+9
View File
@@ -127,6 +127,12 @@ pub struct Invited {
/// the membership is not portal's to take back. /// the membership is not portal's to take back.
#[serde(default)] #[serde(default)]
pub already_member: bool, pub already_member: bool,
/// The invite's own mail and the form it was sent from, kept with
/// the invite so that sending it again sends the same words.
#[serde(default, skip_serializing_if = "Option::is_none")]
pub mail: Option<crate::content::MailTemplate>,
#[serde(default, skip_serializing_if = "serde_json::Map::is_empty")]
pub fields: serde_json::Map<String, serde_json::Value>,
} }
/// When a link runs out, from Kanidm's `expiry_time`: seconds since /// When a link runs out, from Kanidm's `expiry_time`: seconds since
@@ -367,6 +373,8 @@ impl Kanidm {
reset_link, reset_link,
link_expires_ms, link_expires_ms,
created, created,
mail: None,
fields: Default::default(),
already_member, already_member,
}) })
} }
@@ -714,6 +722,7 @@ mod tests {
decided_ms: None, decided_ms: None,
decided_by: None, decided_by: None,
notes: Vec::new(), notes: Vec::new(),
finished: false,
} }
} }
+137 -3
View File
@@ -328,6 +328,11 @@ async fn fetch_resource_value(
} }
} }
} }
if !resource.transitions.is_empty() {
if let Some(schema) = state.aggregates.load().at(0).get(bucket) {
items = cap_finished(items, schema, finished_wanted(params));
}
}
serde_json::Value::Array(items) serde_json::Value::Array(items)
} }
} }
@@ -342,12 +347,67 @@ async fn fetch_resource_value(
ResourceSource::Url { url } => fetch_url_json(url, params).await?, ResourceSource::Url { url } => fetch_url_json(url, params).await?,
}; };
match &resource.jq { let shaped = match &resource.jq {
Some(filter) => apply_jq(filter, &value), Some(filter) => apply_jq(filter, &value)?,
None => Ok(value), None => value,
};
Ok(apply_limit(resource.limit, shaped))
}
/// Keep at most `n` items, after `jq` has had its say.
///
/// Anything that is not a list passes through: a single record has no
/// length to cap, and silently turning one into an empty list would be
/// worse than ignoring a limit that cannot apply. A limit of zero does
/// mean nothing — the resource's `empty` line speaks instead, which is
/// a coherent thing for content to ask for.
#[cfg(feature = "ssr")]
fn apply_limit(limit: Option<usize>, value: serde_json::Value) -> serde_json::Value {
let Some(n) = limit else { return value };
match value {
serde_json::Value::Array(mut items) => {
items.truncate(n);
serde_json::Value::Array(items)
}
other => other,
} }
} }
/// How many finished records a desk listing should show: three, or
/// what the page asked for with `__finished` after "show older".
pub const FINISHED_PARAM: &str = "__finished";
pub const FINISHED_SHOWN: usize = 3;
#[cfg(feature = "ssr")]
fn finished_wanted(params: &std::collections::HashMap<String, String>) -> usize {
params.get(FINISHED_PARAM).and_then(|n| n.parse().ok()).unwrap_or(FINISHED_SHOWN)
}
/// A desk is for the cases still moving. Every record still open stays;
/// of those in a final state (no move out), only the `shown` most
/// recently decided, plus one more so the page knows there are older
/// ones to offer. Each finished record is marked, so the page can tell
/// them apart without knowing the state machine.
#[cfg(feature = "ssr")]
pub(crate) fn cap_finished(items: Vec<serde_json::Value>, schema: &crate::aggregates::AggregateSchema, shown: usize) -> Vec<serde_json::Value> {
let is_final = |v: &serde_json::Value| {
v.get("state").and_then(|s| s.as_str()).is_some_and(|s| schema.has_state(s) && schema.allowed(s).is_empty())
};
let when = |v: &serde_json::Value| {
v.get("decided_ms").and_then(|d| d.as_i64()).or_else(|| v.get("submitted_ms").and_then(|d| d.as_i64())).unwrap_or(0)
};
let (mut finished, mut open): (Vec<_>, Vec<_>) = items.into_iter().partition(is_final);
finished.sort_by_key(|v| std::cmp::Reverse(when(v)));
finished.truncate(shown + 1);
for v in &mut finished {
if let Some(obj) = v.as_object_mut() {
obj.insert("finished".into(), true.into());
}
}
open.extend(finished);
open
}
/// Fetches JSON from a path on the same Gitea instance content is /// Fetches JSON from a path on the same Gitea instance content is
/// loaded from (`AppState.gitea_base`), attaching `GITEA_API_TOKEN` if /// loaded from (`AppState.gitea_base`), attaching `GITEA_API_TOKEN` if
/// set - confirmed live against `project.uhhm.no` that at least the /// set - confirmed live against `project.uhhm.no` that at least the
@@ -624,6 +684,51 @@ mod tests {
assert!(!matches_filter(&fixed, &record("open", serde_json::json!({ "which_authority": "Fire service" })), &params)); assert!(!matches_filter(&fixed, &record("open", serde_json::json!({ "which_authority": "Fire service" })), &params));
} }
/// `limit` caps what the page renders, and does it after `jq` so
/// content can order with the filter and take the top of its own
/// ordering. The releases card is the case: the newest cut is the
/// one you should run, and a page that grows a card per release
/// gets longer forever.
#[test]
fn limit_caps_a_list_after_the_filter() {
let releases = serde_json::json!([
{"name": "redoal 0.1.17", "tag_name": "v0.1.17", "body": "newest"},
{"name": "redoal 0.1.16", "tag_name": "v0.1.16", "body": "older"},
{"name": "redoal 0.1.15", "tag_name": "v0.1.15", "body": "oldest"}
]);
// The filter redoal/questions actually declares: bare objects,
// never a wrapped array — the frontend re-wraps.
let filter = ".[] | {name: .name, description: .body, tag: .tag_name, url: null}";
let shaped = apply_jq(filter, &releases).expect("filter runs");
let all = apply_limit(None, shaped.clone());
assert_eq!(all.as_array().map(Vec::len), Some(3), "no limit shows everything");
let latest = apply_limit(Some(1), shaped.clone());
let rows = latest.as_array().expect("still a list");
assert_eq!(rows.len(), 1);
assert_eq!(rows[0]["tag"], "v0.1.17", "and it is the newest, which Gitea returns first");
let two = apply_limit(Some(2), shaped.clone());
assert_eq!(two.as_array().map(Vec::len), Some(2));
// Asking for more than there is is not an error.
let plenty = apply_limit(Some(99), shaped);
assert_eq!(plenty.as_array().map(Vec::len), Some(3));
}
/// A limit that cannot apply is ignored rather than guessed at.
#[test]
fn limit_leaves_a_single_record_alone() {
let one = serde_json::json!({"name": "a record", "state": "open"});
assert_eq!(apply_limit(Some(1), one.clone()), one);
assert_eq!(apply_limit(Some(0), one.clone()), one, "no length to cap");
// Zero on a list does mean nothing — the `empty` line speaks.
let list = serde_json::json!([{"a": 1}, {"a": 2}]);
assert_eq!(apply_limit(Some(0), list).as_array().map(Vec::len), Some(0));
}
/// A canned, Gitea-API-shaped fixture - the same fields the real /// A canned, Gitea-API-shaped fixture - the same fields the real
/// `/users/{username}/starred` endpoint returns (Gitea 1.27: /// `/users/{username}/starred` endpoint returns (Gitea 1.27:
/// `stars_count`, not GitHub's `stargazers_count`; `website` is "" /// `stars_count`, not GitHub's `stargazers_count`; `website` is ""
@@ -758,3 +863,32 @@ mod tests {
assert_eq!(out, serde_json::json!(["id: /x\n"])); assert_eq!(out, serde_json::json!(["id: /x\n"]));
} }
} }
#[cfg(all(test, feature = "ssr"))]
mod finished_tests {
use super::*;
#[test]
fn a_desk_keeps_every_open_case_and_the_newest_finished_ones() {
let schema = crate::aggregates::parse_aggregates_yaml(
"aggregates:\n - bucket: leads\n initial: open\n states:\n open: { event: in }\n won: { event: won }\n lost: { event: lost }\n transitions:\n open: [won, lost]\n",
)
.unwrap()
.remove("leads")
.unwrap();
let rec = |id: &str, state: &str, decided: i64| serde_json::json!({ "id": id, "state": state, "submitted_ms": 0, "decided_ms": decided });
let mut items = vec![rec("o1", "open", 0), rec("o2", "open", 0)];
items.extend((1..=6).map(|n| rec(&format!("f{n}"), if n % 2 == 0 { "won" } else { "lost" }, n)));
let ids = |v: &[serde_json::Value]| v.iter().map(|x| x["id"].as_str().unwrap().to_string()).collect::<Vec<_>>();
let shown = cap_finished(items.clone(), &schema, 3);
// Both open cases, then the three newest finished and one more
// to tell the page there are older ones.
assert_eq!(ids(&shown), ["o1", "o2", "f6", "f5", "f4", "f3"]);
assert!(shown[2..].iter().all(|v| v["finished"] == true));
assert!(shown[..2].iter().all(|v| v.get("finished").is_none()));
// Asking for all of them: no extra, so no "show older".
assert_eq!(cap_finished(items, &schema, 6).len(), 8);
}
}
+2
View File
@@ -410,6 +410,8 @@ mod tests {
reset_link: None, reset_link: None,
link_expires_ms: None, link_expires_ms: None,
created: true, created: true,
mail: None,
fields: Default::default(),
already_member, already_member,
}, },
) )
+254 -7
View File
@@ -132,34 +132,91 @@ pub fn run(argv: Vec<String>) -> anyhow::Result<()> {
.to_string(); .to_string();
let mut green = false; let mut green = false;
let mut report = String::new(); let mut report = String::new();
// What the lint found, by id, round over round (`--findings`): what
// a round closed, what it left, what it broke. `None` while the
// lint is one that predates `--findings`; then the whole report
// goes back, as it always did.
let mut ledger = Ledger::default();
let mut tracked = true;
let mut undone = String::new();
std::fs::create_dir_all(opts.out.join("findings"))?;
for round in 1..=opts.rounds { for round in 1..=opts.rounds {
let prompt = if round == 1 { let prompt = if round == 1 {
format!( format!(
"{REFERENCE}\n\n# Task\n\nWrite the content repo that meets this frozen `needs.yaml`: `aggregates.yaml`, `site.yaml`, and every page under `questions/` - the front door, followup pages, role-gated pages, one desk per handling group. Do not return needs.yaml.\n\n## Intake\n\n{intake}\n\n## needs.yaml (frozen)\n\n```yaml\n{needs}\n```\n\n{FORMAT}" "{REFERENCE}\n\n# Task\n\nWrite the content repo that meets this frozen `needs.yaml`: `aggregates.yaml`, `site.yaml`, and every page under `questions/` - the front door, followup pages, role-gated pages, one desk per handling group. Do not return needs.yaml.\n\n## Intake\n\n{intake}\n\n## needs.yaml (frozen)\n\n```yaml\n{needs}\n```\n\n{FORMAT}"
) )
} else if tracked {
let (heading, files) = match named_files(&opts.out, ledger.open.values()) {
Some(files) => ("Files the findings name", dump_files(&opts.out, &files)?),
None => ("Current files", dump(&opts.out)?),
};
format!(
"{REFERENCE}\n\n# Task\n\nThe verifier rejected the repo. Below is every finding still open, each with the change that closes it. Make those changes and nothing else: findings already closed stay closed. Return only the files you change, each complete. To remove a file, return it with the single line DELETE. needs.yaml is frozen; do not return it.\n\n## Open findings\n\n{}\n{undone}## needs.yaml (frozen)\n\n```yaml\n{needs}\n```\n\n## {heading}\n\n{files}\n\n{FORMAT}",
ledger.open_list(round)
)
} else { } else {
format!( format!(
"{REFERENCE}\n\n# Task\n\nThe verifier rejected the repo below. Fix it. Return only the files you change, each complete. To remove a file, return it with the single line DELETE. needs.yaml is frozen; do not return it.\n\n## Verifier output\n\n```\n{report}\n```\n\n## needs.yaml (frozen)\n\n```yaml\n{needs}\n```\n\n## Current files\n\n{}\n\n{FORMAT}", "{REFERENCE}\n\n# Task\n\nThe verifier rejected the repo below. Fix it. Return only the files you change, each complete. To remove a file, return it with the single line DELETE. needs.yaml is frozen; do not return it.\n\n## Verifier output\n\n```\n{report}\n```\n\n## needs.yaml (frozen)\n\n```yaml\n{needs}\n```\n\n## Current files\n\n{}\n\n{FORMAT}",
dump(&opts.out)? dump(&opts.out)?
) )
}; };
let before = snapshot(&opts.out)?;
match parse_files(&llm(&opts.llm, &prompt)?) { match parse_files(&llm(&opts.llm, &prompt)?) {
Ok(files) => apply(&opts.out, files)?, Ok(files) => apply(&opts.out, files)?,
Err(_) => eprintln!("structure round {round}: the reply had no file blocks"), Err(_) => eprintln!("structure round {round}: the reply had no file blocks"),
} }
let (ok, r) = lint(&opts, &[])?; let (ok, r, found) = lint_findings(&opts, &opts.out.join(format!("findings/round-{round}.jsonl")))?;
report = r; report = r;
let fails = report.lines().filter(|l| l.starts_with("FAIL")).count(); let Some(found) = found else {
eprintln!("structure round {round}: {}", if ok { "green".to_string() } else { format!("{fails} FAIL line(s)") }); tracked = false;
log.push(format!("structure round {round}: ok={ok} fails={fails}")); let fails = report.lines().filter(|l| l.starts_with("FAIL")).count();
eprintln!("structure round {round}: {}", if ok { "green".to_string() } else { format!("{fails} FAIL line(s)") });
log.push(format!("structure round {round}: ok={ok} fails={fails}"));
if ok {
green = true;
break;
}
continue;
};
let previous = ledger.clone();
let progress = ledger.update(round, found.into_iter().filter(|f| f.level == "fail").collect());
let line = format!("structure round {round}: {}", progress.summary(ledger.open.len()));
eprintln!("{line}");
log.push(line);
undone.clear();
if ok { if ok {
green = true; green = true;
break; break;
} }
if round == 1 {
continue;
}
// A fix that reopens what an earlier round closed is undone,
// and the model is told what it broke, rather than chasing the
// same findings round after round.
if !progress.reopened.is_empty() {
restore(&opts.out, before)?;
ledger = previous;
undone = format!(
"## Your last change was undone\n\nIt reopened findings an earlier round had closed:\n\n{}\nClose the open findings without touching what made these pass.\n\n",
progress.reopened.iter().map(|f| format!("- [{}] {}\n", f.id, f.message)).collect::<String>()
);
eprintln!("structure round {round}: reopened {} closed finding(s), undone", progress.reopened.len());
log.push(format!("structure round {round}: undone (reopened {})", progress.reopened.len()));
continue;
}
// No progress, no more rounds: another pass at the same findings
// is how a site gets over-trained. What is left is reported.
if progress.fixed.is_empty() || ledger.open.len() >= previous.open.len() {
eprintln!("structure round {round}: no finding closed, stopping");
log.push(format!("structure round {round}: stalled"));
break;
}
} }
if !green { if !green {
std::fs::write(opts.out.join("run.log"), format!("{}\n\n{report}", log.join("\n")))?; let open = if tracked { format!("\n\nStill open:\n{}", ledger.open_list(opts.rounds + 1)) } else { String::new() };
bail!("structure never went green in {} round(s); last verifier output is in run.log", opts.rounds); std::fs::write(opts.out.join("run.log"), format!("{}{open}\n\n{report}", log.join("\n")))?;
bail!("structure did not go green; what is still open is in run.log");
} }
// Phase 3: wording. // Phase 3: wording.
@@ -191,7 +248,10 @@ pub fn run(argv: Vec<String>) -> anyhow::Result<()> {
} }
eprintln!("wording round {round}: {}", summary.join(" | ")); eprintln!("wording round {round}: {}", summary.join(" | "));
log.push(format!("wording round {round}: {}", summary.join(" | "))); log.push(format!("wording round {round}: {}", summary.join(" | ")));
if defects.is_empty() || round == opts.wording_rounds { // A visitor who chose right, hesitantly, is not a defect worth a
// rewrite on its own: only a misroute is. Hesitations ride along
// as context when there is one.
if !defects.contains("MISROUTE") || round == opts.wording_rounds {
break; break;
} }
let personas = needs_full.split("\npersonas:").nth(1).unwrap_or_default(); let personas = needs_full.split("\npersonas:").nth(1).unwrap_or_default();
@@ -311,6 +371,152 @@ fn lint(opts: &Opts, extra: &[String]) -> anyhow::Result<(bool, String)> {
Ok((output.status.success(), report)) Ok((output.status.success(), report))
} }
/// One finding as the lint's `--findings` writes it (docs/lint.md).
#[derive(Clone, Debug, serde::Deserialize)]
struct Finding {
id: String,
level: String,
code: String,
#[serde(default)]
need: String,
message: String,
#[serde(default)]
fix: Option<String>,
}
/// The lint with `--findings` into `out`, and what it found; `None` for
/// the findings when the lint predates the flag (then it is run again
/// the old way, and only its text comes back).
fn lint_findings(opts: &Opts, out: &Path) -> anyhow::Result<(bool, String, Option<Vec<Finding>>)> {
let (ok, report) = lint(opts, &["--findings".to_string(), out.display().to_string()])?;
if report.contains("unknown argument: --findings") {
let (ok, report) = lint(opts, &[])?;
return Ok((ok, report, None));
}
let raw = std::fs::read_to_string(out).unwrap_or_default();
let found = raw
.lines()
.filter(|l| !l.trim().is_empty())
.map(|l| serde_json::from_str::<Finding>(l).with_context(|| format!("reading {}", out.display())))
.collect::<anyhow::Result<Vec<_>>>()?;
Ok((ok, report, Some(found)))
}
/// Every failing finding by id, across rounds.
#[derive(Clone, Default)]
struct Ledger {
open: BTreeMap<String, Finding>,
/// The round each open finding was first seen in.
since: BTreeMap<String, usize>,
/// Ids a round has closed, so one that comes back is a regression,
/// not news.
closed: std::collections::BTreeSet<String>,
}
struct Progress {
fixed: Vec<Finding>,
new: Vec<Finding>,
still: usize,
reopened: Vec<Finding>,
}
impl Progress {
fn summary(&self, open: usize) -> String {
if open == 0 {
return format!("green (fixed {})", self.fixed.len());
}
format!(
"{open} open - fixed {}, still open {}, new {}, reopened {}",
self.fixed.len(),
self.still,
self.new.len(),
self.reopened.len()
)
}
}
impl Ledger {
fn update(&mut self, round: usize, fails: Vec<Finding>) -> Progress {
let now: BTreeMap<String, Finding> = fails.into_iter().map(|f| (f.id.clone(), f)).collect();
let fixed: Vec<Finding> = self.open.values().filter(|f| !now.contains_key(&f.id)).cloned().collect();
let mut new = Vec::new();
let mut reopened = Vec::new();
let mut still = 0;
for (id, f) in &now {
if self.open.contains_key(id) {
still += 1;
} else if self.closed.contains(id) {
reopened.push(f.clone());
} else {
new.push(f.clone());
}
self.since.entry(id.clone()).or_insert(round);
}
for f in &fixed {
self.closed.insert(f.id.clone());
self.since.remove(&f.id);
}
self.open = now;
Progress { fixed, new, still, reopened }
}
/// The open findings for a prompt or a log: id, how long open, the
/// message and its fix.
fn open_list(&self, round: usize) -> String {
self.open
.values()
.map(|f| {
let age = match self.since.get(&f.id) {
Some(&first) if first + 1 < round => format!(", open since round {first}"),
_ => String::new(),
};
let scope = if f.need.is_empty() { String::new() } else { format!(" {:?}", f.need) };
format!("- [{}] {}{scope}{age}: {}\n", f.id, f.code, f.message)
})
.collect()
}
}
/// The files open findings point at - every desk page their fixes name,
/// plus aggregates.yaml - when that is enough to act on. `None` when a
/// finding is about getting around the site (a path, an entry page, a
/// missing desk) or names no page: then the model needs to see it all.
fn named_files<'a>(out: &Path, findings: impl Iterator<Item = &'a Finding>) -> Option<Vec<PathBuf>> {
const WHOLE_SITE: &[&str] = &["schema", "entry-missing", "entry-gated", "path-missing", "path-closed", "path-over-budget", "desk-missing", "need-duplicate"];
let mut files = vec![out.join("aggregates.yaml")];
for f in findings {
if WHOLE_SITE.contains(&f.code.as_str()) {
return None;
}
let text = format!("{} {}", f.message, f.fix.as_deref().unwrap_or_default());
for page in text.split_whitespace().filter(|w| w.starts_with('/')) {
let page = page.trim_end_matches(|c: char| !(c.is_alphanumeric() || c == '_' || c == '-' || c == '/'));
let rel = page.trim_start_matches('/');
let candidates = if rel.is_empty() {
vec![out.join("questions/index.yaml")]
} else {
vec![out.join(format!("questions/{rel}.yaml")), out.join(format!("questions/{rel}/index.yaml"))]
};
if let Some(path) = candidates.into_iter().find(|p| p.exists()) {
if !files.contains(&path) {
files.push(path);
}
}
}
}
Some(files)
}
/// `dump`, for the named files only.
fn dump_files(out: &Path, files: &[PathBuf]) -> anyhow::Result<String> {
let mut text = String::new();
for path in files.iter().filter(|p| p.exists()) {
let rel = path.strip_prefix(out).unwrap_or(path).display().to_string();
text.push_str(&format!("<<<FILE {rel}>>>\n{}<<<END>>>\n\n", std::fs::read_to_string(path)?));
}
Ok(text)
}
/// `<<<FILE path>>> ... <<<END>>>` blocks. Anything outside a block is /// `<<<FILE path>>> ... <<<END>>>` blocks. Anything outside a block is
/// ignored, so a model that adds a sentence does no harm. /// ignored, so a model that adds a sentence does no harm.
fn parse_files(text: &str) -> anyhow::Result<BTreeMap<String, String>> { fn parse_files(text: &str) -> anyhow::Result<BTreeMap<String, String>> {
@@ -591,3 +797,44 @@ mod tests {
assert!(!allowed(".gitea/workflows/deploy.yaml")); assert!(!allowed(".gitea/workflows/deploy.yaml"));
} }
} }
#[cfg(test)]
mod ledger_tests {
use super::*;
fn f(id: &str, code: &str, message: &str) -> Finding {
Finding { id: id.into(), level: "fail".into(), code: code.into(), need: "hire".into(), message: message.into(), fix: None }
}
#[test]
fn a_round_is_told_what_it_closed_kept_added_and_broke() {
let mut ledger = Ledger::default();
let p = ledger.update(1, vec![f("a", "strand", "x"), f("b", "strand", "y")]);
assert_eq!((p.fixed.len(), p.new.len(), p.still, p.reopened.len()), (0, 2, 0, 0));
let p = ledger.update(2, vec![f("b", "strand", "y"), f("c", "edge-unoffered", "z")]);
assert_eq!((p.fixed.len(), p.new.len(), p.still, p.reopened.len()), (1, 1, 1, 0));
assert_eq!(p.summary(ledger.open.len()), "2 open - fixed 1, still open 1, new 1, reopened 0");
// "a" was closed in round 2; its return is a regression.
let p = ledger.update(3, vec![f("a", "strand", "x"), f("b", "strand", "y")]);
assert_eq!(p.reopened.iter().map(|f| f.id.as_str()).collect::<Vec<_>>(), ["a"]);
assert!(ledger.open_list(4).contains("[b] strand \"hire\", open since round 1: y"));
let p = ledger.update(4, vec![]);
assert_eq!(p.summary(ledger.open.len()), "green (fixed 2)");
}
#[test]
fn a_fix_round_sees_the_pages_its_findings_name() {
let dir = std::env::temp_dir().join(format!("ledger-test-{}", std::process::id()));
std::fs::create_dir_all(dir.join("questions/desk")).unwrap();
std::fs::write(dir.join("aggregates.yaml"), "x").unwrap();
std::fs::write(dir.join("questions/desk/index.yaml"), "x").unwrap();
std::fs::write(dir.join("questions/index.yaml"), "x").unwrap();
let mut strand = f("a", "strand", "records strand in \"quoted\"");
strand.fix = Some("add `{ from: quoted, to: won, label: ... }` to the transitions on /desk".into());
let files = named_files(&dir, [&strand].into_iter()).unwrap();
assert_eq!(files, vec![dir.join("aggregates.yaml"), dir.join("questions/desk/index.yaml")]);
// A finding about getting around the site needs all of it.
assert!(named_files(&dir, [&strand, &f("b", "path-missing", "no path")].into_iter()).is_none());
std::fs::remove_dir_all(&dir).ok();
}
}
+10 -1
View File
@@ -190,7 +190,16 @@ not done. Its messages are exact:
- `no desk gated on "g" reads "b"`: add the desk. - `no desk gated on "g" reads "b"`: add the desk.
- `"g" cannot move a record to "s"` / `records strand in "s"`: the desk - `"g" cannot move a record to "s"` / `records strand in "s"`: the desk
is missing a transition the state machine allows, or the state is missing a transition the state machine allows, or the state
machine is missing the edge. machine is missing the edge. The message ends in `fix:` with the
exact `{ from, to, label }` and the desk page to put it on - do that.
- `state "s" is never reached`: nothing moves a record into it. Offer
the move it names, or remove the state.
- `stage "b" comes after "a" ... but no move leads on`: the machine
cannot go the way the business works; add the edge it names.
- `the mail on entering "s" uses {x}`: the mail names a field no form
collects. Use one of the fields it lists, or collect `x`.
- WARN `the move "a" -> "b" is declared but no desk ... offers it`:
offer it, or drop the edge - every edge should be a button.
- `action ... does not match any declared question id`: the page does - `action ... does not match any declared question id`: the page does
not exist. not exist.
- `entry page ... names no one responsible`: add `responsible`. - `entry page ... names no one responsible`: add `responsible`.
+25
View File
@@ -1280,6 +1280,31 @@ textarea:focus {
gap: 0.8rem; gap: 0.8rem;
} }
/* After the newest finished cases: the older ones, on request. Quiet,
since the desk is for what is still moving. */
.show-older {
justify-self: start;
background: none;
border: 0.06rem solid var(--line);
border-radius: 0.4rem;
color: var(--ink-dim);
font: inherit;
font-size: 0.9rem;
padding: 0.35rem 0.8rem;
cursor: pointer;
}
/* A releases manifest: the pick for this device, the rest folded. */
.releases .pill { display: inline-block; }
.releases pre { white-space: pre-wrap; font-size: 0.85rem; margin: 0.4rem 0 0; }
.releases-all summary { color: var(--ink-dim); font-size: 0.9rem; cursor: pointer; }
.releases-all ul { margin: 0.4rem 0 0 1rem; font-size: 0.9rem; }
.show-older:hover {
color: inherit;
border-color: var(--ink-dim);
}
.answer-row { .answer-row {
border: 0.06rem solid var(--line); border: 0.06rem solid var(--line);
border-radius: 0.8rem; border-radius: 0.8rem;