Compare commits
@@ -38,6 +38,25 @@ jobs:
|
||||
- name: Test
|
||||
run: cargo test --features ssr
|
||||
|
||||
# Every plan under plans/ built deterministically and held to the
|
||||
# lint, both by this very portal: a builder change that renders a
|
||||
# site the lint rejects fails here, not at the next training run.
|
||||
# (portal-trainer's CI did this until the trainer moved in.)
|
||||
- name: Build and lint every plan
|
||||
run: |
|
||||
set -euo pipefail
|
||||
cargo build --features ssr --bin portal
|
||||
bin="$CARGO_TARGET_DIR/debug/portal"
|
||||
tmp=$(mktemp -d)
|
||||
lint="$tmp/portal-lint"
|
||||
printf '#!/bin/sh\nexec %s lint "$@"\n' "$bin" > "$lint" && chmod +x "$lint"
|
||||
for plan in plans/*.yaml; do
|
||||
name=$(basename "$plan" .yaml)
|
||||
echo "== $plan"
|
||||
rm -rf "$tmp/plan-$name"
|
||||
"$bin" build --plan "$plan" --out "$tmp/plan-$name" --lint "$lint"
|
||||
done
|
||||
|
||||
- name: Stop the throwaway JetStream
|
||||
if: always()
|
||||
run: docker rm -f portal-test-nats >/dev/null 2>&1 || true
|
||||
|
||||
@@ -4,6 +4,29 @@ One line per change, grouped by the release that shipped it. Newest
|
||||
first. A release is a Cargo.toml bump plus a `v<version>` tag; the
|
||||
content repos pin the tag they run (see the end of this file).
|
||||
|
||||
## 0.6.21 (2026-10-07)
|
||||
|
||||
- A content repo's `.well-known/` is served at the site root (`/.well-known/security.txt`, `/.well-known/assetlinks.json`, `/.well-known/matrix/server`), and a miss there is a 404 instead of the app shell answering 200 with an HTML page. RFC 8615 reserves the prefix and a question's id cannot begin with a dot, so nothing a repo ships can shadow a page. An extensionless file under it is typed as JSON, by the convention those files follow; `.txt`, `.xml`, `.webmanifest` and `.mobileconfig` are typed too, the last so an Apple configuration profile offers to install rather than download.
|
||||
|
||||
## 0.6.20 (2026-10-05)
|
||||
|
||||
- A releases card lets a platform's file carry an architecture: a Mac visitor is offered the `macos-arm64` file, not sent to TestFlight past it.
|
||||
|
||||
## 0.6.19 (2026-10-05)
|
||||
|
||||
- A `kind: url` resource whose JSON is a releases manifest (`latest`, `versions` with per-platform `assets`, `channels` such as a package repo or TestFlight) renders as a download card: the file for the visitor's own platform first, decided in the browser, the rest and the channels folded. Written for redoal's public releases at dl.redoal.com (docs: content-reference, "A releases manifest").
|
||||
|
||||
## 0.6.18 (2026-10-03)
|
||||
|
||||
- `iris check --findings <out.jsonl>` writes every finding as JSON: a stable `id` (scope, need, code and subject, nothing of the wording), `level`, `code`, `scope`, `need`, `subject`, `message` and `fix`. Every message that knows its fix now says it after "fix:". The codes are listed in docs/lint.md.
|
||||
- `portal generate` follows its findings through: each structure round is logged as fixed, still open, new and reopened; a fix round is shown the open findings with their fixes and only the files they name; a change that reopens a closed finding is undone; and the phase stops when a round closes nothing, writing what is left to run.log. A hesitant visitor no longer triggers a wording rewrite by itself, only a misroute does. A lint without `--findings` is still driven the old way.
|
||||
|
||||
## 0.6.17 (2026-10-02)
|
||||
|
||||
- A desk shows every record still moving and only the three most recently finished (a final state: no move out of it), with "Show older" for three more at a time. A desk that has run for a year is still about the work in hand.
|
||||
- Mail variables are one set for every kind of mail (docs: Mail). Every mail has `{site}` and `{site_name}`; a state mail also has `{link}`, `{self_link}` and `{desk_link}`: the reader's own page for the record (a self-service page with `?chain=&email=`, when the record's state allows a move there), the desk that moves it, and `{link}` choosing between them by who the mail is to. `{link}` in a state mail used to render as nothing.
|
||||
- The lint follows every state of every bucket a need lands in: a state no desk, link or clock reaches fails; a state a record can reach and never leave fails, after "done" too; a declared move nobody offers warns; a desk gated to one group on the page and another on its list warns; working stages must follow each other in the business's order; a mail from an unreachable state warns, and one naming a field no form collects fails. Clocks (`after:`) count as moves, and a visitor's own link only from the states it names. Each finding ends in the fix: the `{ from, to, label }` and the desk page it goes on, or the group that already has the button. A schema error no longer hides the needs findings.
|
||||
|
||||
## 0.6.16 (2026-10-01)
|
||||
|
||||
- The lint refuses an alternative that has a form to send (fields to fill in, an invite, or a bucket to record into) and no `action`. The button is only drawn for an alternative with an `action`, so such a form could be filled in and never sent, which is what happened to uhhm.no's first invite form. An alternative that moves its own record (`self_transition`) draws its own button and is left alone.
|
||||
|
||||
Generated
+1
-1
@@ -3357,7 +3357,7 @@ dependencies = [
|
||||
|
||||
[[package]]
|
||||
name = "portal"
|
||||
version = "0.6.16"
|
||||
version = "0.6.21"
|
||||
dependencies = [
|
||||
"anyhow",
|
||||
"arc-swap",
|
||||
|
||||
+2
-1
@@ -1,6 +1,6 @@
|
||||
[package]
|
||||
name = "portal"
|
||||
version = "0.6.16"
|
||||
version = "0.6.21"
|
||||
edition = "2021"
|
||||
|
||||
[lib]
|
||||
@@ -60,6 +60,7 @@ web-sys = { version = "0.3", features = [
|
||||
"FileList",
|
||||
"FormData",
|
||||
"HtmlInputElement",
|
||||
"Navigator",
|
||||
"Request",
|
||||
"RequestInit",
|
||||
"Response",
|
||||
|
||||
@@ -7,6 +7,7 @@
|
||||
- [Getting started](getting-started.md)
|
||||
- [Content reference](content-reference.md)
|
||||
- [State machines](state-machines.md)
|
||||
- [Mail](mail.md)
|
||||
- [Access](access.md)
|
||||
- [Storylines](storylines.md)
|
||||
- [Memberships](memberships.md)
|
||||
|
||||
@@ -255,6 +255,7 @@ reads is never a client-supplied parameter.
|
||||
states: [approved, refused] # only these; empty means all
|
||||
where: { field: case, equals: "{key}" }
|
||||
jq: ".[] | {name, url: .html_url}"
|
||||
limit: 1 # at most this many, after jq
|
||||
empty: Nothing decided yet.
|
||||
link: { to: "/notices/{id}", label: Answer this } # a page per record
|
||||
transitions: # the desk's buttons
|
||||
@@ -264,6 +265,14 @@ reads is never a client-supplied parameter.
|
||||
`source.kind` is one of `kv`, `gitea_starred`,
|
||||
`gitea_releases`, `url`.
|
||||
|
||||
`limit: 1` shows only the first item — for a feed where only the newest
|
||||
matters. A releases card wants the cut someone should run, not every
|
||||
cut there has ever been, and a page that grows by one card per release
|
||||
gets longer forever. It applies *after* `jq`, so a filter can reorder
|
||||
and the limit takes the top of that ordering. A source that yields a
|
||||
single record has no length and ignores it; `limit: 0` on a list shows
|
||||
nothing, and the `empty` line speaks instead.
|
||||
|
||||
A spec with neither `public: true` nor `requires_group` is
|
||||
unreachable: reads fail closed, and mutations always need a group
|
||||
whatever `public` says.
|
||||
@@ -285,6 +294,26 @@ markdown on stricter rules than the site's own: paragraphs, line
|
||||
breaks as typed, lists, emphasis and links to https, mailto or tel
|
||||
(marked `nofollow ugc`); no images, no raw HTML.
|
||||
|
||||
A desk (a list with `transitions`) shows every record still moving,
|
||||
and of those in a final state - one the state machine has no move out
|
||||
of - only the three most recently decided, newest first. "Show older"
|
||||
under the list brings three more each time it is pressed. The desk
|
||||
stays about the work in hand however long the site has run; nothing
|
||||
is hidden from a read-only list, and `leaves_after` still takes a
|
||||
record out of every list altogether.
|
||||
|
||||
A **releases manifest** - a `kind: url` resource whose JSON is
|
||||
`{ name, latest, versions: [{ version, date, notes, assets: [{ platform,
|
||||
url, sha256, size }] }], channels: { arch: { repo, package, note },
|
||||
ios: { testflight, note } } }`, newest version first, as a release
|
||||
pipeline writes it - renders as one card: the newest version, the file
|
||||
for the visitor's own platform first (read from the browser: `ios`,
|
||||
`macos`, `linux-x86_64`, `linux-aarch64`, `windows`, `android`), the
|
||||
other files and the channels folded under "All downloads". Linux gets
|
||||
the package repo lines beside its file; iPhones and Macs get
|
||||
TestFlight. Platforms a manifest lacks fall through to the folded
|
||||
list.
|
||||
|
||||
`states` and `where` narrow a listing. They matter most for a register
|
||||
that is public by law — public about what was decided, not about what
|
||||
was withdrawn — while the desk working the same bucket is a different
|
||||
|
||||
+37
-1
@@ -34,7 +34,43 @@ in `needs.yaml`, when it has one:
|
||||
- the handling group holds a desk over that bucket, and every
|
||||
finished state is reachable through the buttons it (and any
|
||||
`also_moved_by` group) offers; no record strands on the way;
|
||||
- every stage the business named is a state of the machine.
|
||||
- every stage the business named is a state of the machine, and each
|
||||
working stage can follow the one before it;
|
||||
- across each bucket, whichever need brought a record in: every state
|
||||
is somewhere a record can get to, no state a record gets to holds it
|
||||
with no way on (before done or after), every declared move is a
|
||||
button, a visitor's own link or a clock (a move nobody offers warns),
|
||||
no desk is gated to one group on the page and another on its list,
|
||||
and every mail goes out from a state records reach and names only
|
||||
fields the forms collect.
|
||||
|
||||
Every finding says what to change and where: the missing
|
||||
`{ from, to, label }` and the desk page it goes on, the group that
|
||||
already has the button, or the edge aggregates.yaml lacks. A schema
|
||||
error no longer hides the needs findings - both are printed, so one
|
||||
round can fix them together.
|
||||
|
||||
`--findings <out.jsonl>` writes the same findings one JSON object per
|
||||
line: `id`, `level` (`fail`/`warn`), `code`, `scope` (`need`,
|
||||
`bucket`, or `repo` for a schema error), `need` (the need's id, or the
|
||||
bucket's name), `subject` (the state, the move as `from>to`, or the
|
||||
page), `message` and `fix`. The file is written empty when the content
|
||||
is clean. `id` hashes scope, need, code and subject and nothing of the
|
||||
wording, so the same gap keeps its id from one run to the next: a
|
||||
trainer can tell what a round closed, what is still open and what it
|
||||
broke. The codes:
|
||||
|
||||
| code | |
|
||||
|---|---|
|
||||
| `schema` | content the site would refuse to load |
|
||||
| `need-duplicate`, `persona-unknown-need` | needs.yaml itself |
|
||||
| `entry-missing`, `entry-gated`, `entry-no-responsible` | the page a visitor arrives on |
|
||||
| `path-missing`, `path-closed`, `path-over-budget`, `path-no-action` | getting the answer recorded |
|
||||
| `machine-missing`, `desk-missing`, `desk-gated-twice` | who works the bucket |
|
||||
| `done-unknown`, `done-unreachable`, `success-not-done` | finishing a case |
|
||||
| `stage-unknown`, `stage-unreachable`, `stage-order` | the business's own stages |
|
||||
| `state-unreached`, `strand`, `edge-unoffered` | every state and move of the bucket |
|
||||
| `mail-unreachable`, `mail-unknown-field` | what a state mails |
|
||||
|
||||
A need marked `planned` warns instead of failing. Exit status is the
|
||||
verdict, so a content repo's CI is one line.
|
||||
|
||||
@@ -0,0 +1,59 @@
|
||||
# Mail
|
||||
|
||||
Portal writes every mail itself and hands it to gdo to deliver
|
||||
(`portal.mail.send`); a reply comes back as a note beside the record
|
||||
(`portal.mail.received`). A mail is a subject and a markdown body with
|
||||
`{placeholders}`, sent as text with an HTML alternative. The kinds of
|
||||
mail are the ones content declares on a state (see
|
||||
[State machines](state-machines.md#mail)), the invite and its
|
||||
rewording (`invite: { group, mail }`, `mail.invite` in site.yaml), the
|
||||
note to a person named responsible for a question, and the answers to
|
||||
a suggested change.
|
||||
|
||||
## Variables
|
||||
|
||||
A placeholder is `{` letters, digits, `_`, `-` or `.` `}`. One that
|
||||
names nothing renders as nothing; a brace that opens no name stays a
|
||||
brace. The variables come in layers, so a template written for one
|
||||
kind of mail reads the same in another.
|
||||
|
||||
**On every mail**
|
||||
|
||||
| | |
|
||||
|---|---|
|
||||
| `{site}` | the instance's public URL, no trailing slash |
|
||||
| `{site_name}` | site.yaml's `title` |
|
||||
|
||||
**On a mail about a record** (every state mail)
|
||||
|
||||
| | |
|
||||
|---|---|
|
||||
| `{field}` | any field of the record, as the form sent it |
|
||||
| `{chain}` | the record's id |
|
||||
| `{email}` | the record's `email` field, trimmed |
|
||||
| `{bucket}`, `{state}` | where it is and the state it just entered |
|
||||
| `{notice.field}` | a field of the record this one belongs to, through its `type: record` field `notice` |
|
||||
| `{self_link}` | the record's own page for the person it belongs to: a page offering a self-service move (`self_transition`) on the bucket from the state the record is in, with the `?chain=&email=` that lets them act without signing in. Empty when there is no such page or no address |
|
||||
| `{desk_link}` | the desk that moves the record: the page whose list over the bucket has `transitions`, else one that reads it. Empty when no page reads the bucket |
|
||||
| `{link}` | where this reader acts: `{self_link}` in a mail `to: submitter`, `{desk_link}` in one to anyone else, and `{site}` when that link is empty |
|
||||
|
||||
`{link}` is the one to reach for. "Se meldingen og avtal befaring:
|
||||
{link}" sent to the caretaker opens the desk the report is on; "Feilen
|
||||
du meldte om er utbedret. {link}" sent to the tenant opens their own
|
||||
case where they can still act on it, or the site when there is
|
||||
nothing left for them to do.
|
||||
|
||||
**On an invite and the note to someone responsible**, beside every
|
||||
field of the invite's form: `{name}`, `{username}`, `{email}`,
|
||||
`{group}` (invite), `{questions}` (responsible), `{expires}`, and
|
||||
`{link}`, which here is the one-time sign-in link.
|
||||
|
||||
## What the lint holds a mail to
|
||||
|
||||
A state mail that sends from a state no record reaches warns; one that
|
||||
names a field no form recording into the bucket collects fails, since
|
||||
it would go out with a blank where the field should be. The lint
|
||||
knows every variable above, so `{link}` and the rest are never
|
||||
flagged. `to: submitter` requires a required `email` on every form
|
||||
filling the bucket, and `to: "{notice.email}"` requires the `notice`
|
||||
field.
|
||||
@@ -57,7 +57,9 @@ a record arrives in it.
|
||||
```
|
||||
|
||||
`{field}` is any field of the record, plus `{site}`, `{chain}`,
|
||||
`{email}`, `{bucket}` and `{state}`. The body is markdown; it is sent
|
||||
`{email}`, `{bucket}`, `{state}` and `{link}` - the page where the
|
||||
reader acts on the record, their own or the desk's (every variable is
|
||||
in [Mail](mail.md#variables)). The body is markdown; it is sent
|
||||
as text with an HTML alternative. `to: submitter` means the record's
|
||||
`email` field, and the lint then requires every form filling that
|
||||
bucket to collect a required `email`.
|
||||
|
||||
+16
-5
@@ -243,13 +243,24 @@ against the lint. It is kept for the record of what it measured.
|
||||
visitor will tolerate.
|
||||
2. **Structure.** Pages, state machines and desks are generated, then
|
||||
revised against the lint, `iris check`, until every need is met:
|
||||
a path within budget, a desk held by the right group, every
|
||||
finished state reachable, no record stranding. The page-writing
|
||||
model sees the needs and never the personas.
|
||||
a path within budget, a desk held by the right group, every state
|
||||
of every bucket reachable and none stranding a record. The
|
||||
page-writing model sees the needs and never the personas. Each
|
||||
round the lint writes its findings with stable ids (`--findings`,
|
||||
kept in `<out>/findings/round-N.jsonl`), and the round is logged as
|
||||
what it fixed, what is still open, what is new and what it
|
||||
reopened. A fix round is shown only the open findings, each with
|
||||
the change that closes it, and the files they name (the whole repo
|
||||
when a finding is about getting around the site). A change that
|
||||
reopens a closed finding is undone and the model told what it
|
||||
broke. A round that closes nothing, or leaves as many open as it
|
||||
found, ends the phase: what is left is written to `run.log` rather
|
||||
than polished at again.
|
||||
3. **Wording.** A separate model plays each persona, blind, first with
|
||||
headings only and then with descriptions. A misroute is a wording
|
||||
defect and goes back to the generator. A rewrite that breaks
|
||||
structure is reverted.
|
||||
defect and goes back to the generator; a visitor who chose right
|
||||
but hesitated is reported and rewrites nothing by itself. A rewrite
|
||||
that breaks structure is reverted.
|
||||
|
||||
Model output is only ever written to `aggregates.yaml`, `site.yaml`
|
||||
and `questions/**.yaml` inside the output directory.
|
||||
|
||||
@@ -38,6 +38,11 @@ pub struct Answer {
|
||||
/// not see before.
|
||||
#[serde(default, skip_serializing_if = "Vec::is_empty")]
|
||||
pub notes: Vec<Note>,
|
||||
/// Set by a desk listing, never stored: the record is in a final
|
||||
/// state (one with no move out). A desk shows the newest few of
|
||||
/// those and offers the rest on request (`resource::cap_finished`).
|
||||
#[serde(default, skip_serializing_if = "std::ops::Not::not")]
|
||||
pub finished: bool,
|
||||
}
|
||||
|
||||
/// One reply on a record. It lives here rather than with the mail
|
||||
@@ -97,6 +102,7 @@ pub async fn store_answer(
|
||||
storyline,
|
||||
decided_ms: None,
|
||||
notes: Vec::new(),
|
||||
finished: false,
|
||||
decided_by: None,
|
||||
};
|
||||
let data = serde_json::to_vec(&answer)?;
|
||||
@@ -537,6 +543,7 @@ mod tests {
|
||||
decided_ms: None,
|
||||
decided_by: None,
|
||||
notes: Vec::new(),
|
||||
finished: false,
|
||||
},
|
||||
Answer {
|
||||
id: "def456".to_string(),
|
||||
@@ -549,6 +556,7 @@ mod tests {
|
||||
decided_ms: Some(1_700_000_002_000),
|
||||
decided_by: Some("bl".to_string()),
|
||||
notes: Vec::new(),
|
||||
finished: false,
|
||||
},
|
||||
];
|
||||
let value = serde_json::Value::Array(
|
||||
|
||||
+206
-6
@@ -1936,18 +1936,26 @@ fn ResourceFeature(
|
||||
pending_transitions: RwSignal<std::collections::HashMap<(String, String), String>>,
|
||||
transition_batch: ServerAction<TransitionAnswers>,
|
||||
) -> impl IntoView {
|
||||
// A desk shows the newest few finished records; "show older" asks
|
||||
// for more (see `resource::cap_finished`).
|
||||
let finished_shown = RwSignal::new(crate::resource::FINISHED_SHOWN);
|
||||
let data = Resource::new(
|
||||
{
|
||||
let question_id = question_id.clone();
|
||||
let alternative = alternative.clone();
|
||||
let feature_name = feature_name.clone();
|
||||
move || (question_id.clone(), alternative.clone(), feature_name.clone())
|
||||
move || (question_id.clone(), alternative.clone(), feature_name.clone(), finished_shown.get())
|
||||
},
|
||||
// Forwarding sibling requirement values as query params (for a
|
||||
// resource parameterized by other form fields) is deferred, see
|
||||
// the resource-backed multi-select requirement UI follow-up.
|
||||
|(q, a, f, shown)| {
|
||||
let mut params = std::collections::HashMap::new();
|
||||
if shown != crate::resource::FINISHED_SHOWN {
|
||||
params.insert(crate::resource::FINISHED_PARAM.to_string(), shown.to_string());
|
||||
}
|
||||
get_resource(q, a, f, params)
|
||||
},
|
||||
// Empty params for now - forwarding sibling requirement values
|
||||
// as query params (for a resource parameterized by other form
|
||||
// fields) is deferred, see the resource-backed multi-select
|
||||
// requirement UI follow-up.
|
||||
|(q, a, f)| get_resource(q, a, f, std::collections::HashMap::new()),
|
||||
);
|
||||
|
||||
// The shared batch button lives on `AlternativeCard`, above every
|
||||
@@ -1982,6 +1990,7 @@ fn ResourceFeature(
|
||||
value=value
|
||||
pending_transitions=pending_transitions
|
||||
transition_batch=transition_batch
|
||||
finished_shown=finished_shown
|
||||
/>
|
||||
}
|
||||
.into_any()
|
||||
@@ -2011,6 +2020,7 @@ fn ResourceValue(
|
||||
value: serde_json::Value,
|
||||
pending_transitions: RwSignal<std::collections::HashMap<(String, String), String>>,
|
||||
transition_batch: ServerAction<TransitionAnswers>,
|
||||
finished_shown: RwSignal<usize>,
|
||||
) -> impl IntoView {
|
||||
// A jq like `.[] | {...}` over an empty list yields no output at
|
||||
// all - that arrives as null, and it's the same silence as [].
|
||||
@@ -2034,6 +2044,32 @@ fn ResourceValue(
|
||||
let when = if cards { a.submitted_ms.cmp(&b.submitted_ms) } else { b.submitted_ms.cmp(&a.submitted_ms) };
|
||||
open(&a.state).cmp(&open(&b.state)).then(when)
|
||||
});
|
||||
// The desk sent one finished record more than it shows, when
|
||||
// there are older ones: drop the oldest and offer the rest.
|
||||
let shown = finished_shown.get_untracked();
|
||||
let older = answers.iter().filter(|a| a.finished).count() > shown;
|
||||
if older {
|
||||
if let Some(oldest) = answers
|
||||
.iter()
|
||||
.enumerate()
|
||||
.filter(|(_, a)| a.finished)
|
||||
.min_by_key(|(_, a)| a.decided_ms.unwrap_or(a.submitted_ms))
|
||||
.map(|(i, _)| i)
|
||||
{
|
||||
answers.remove(oldest);
|
||||
}
|
||||
}
|
||||
let show_older = older.then(|| {
|
||||
view! {
|
||||
<button
|
||||
type="button"
|
||||
class="show-older"
|
||||
on:click=move |_| finished_shown.update(|n| *n += crate::resource::FINISHED_SHOWN)
|
||||
>
|
||||
{move || t(&lang().get(), "show_older")}
|
||||
</button>
|
||||
}
|
||||
});
|
||||
return view! {
|
||||
<div class="answer-list" class:answer-cards=cards>
|
||||
<For
|
||||
@@ -2060,6 +2096,7 @@ fn ResourceValue(
|
||||
}
|
||||
}
|
||||
/>
|
||||
{show_older}
|
||||
</div>
|
||||
}
|
||||
.into_any();
|
||||
@@ -2087,9 +2124,148 @@ fn ResourceValue(
|
||||
}
|
||||
}
|
||||
|
||||
// A releases manifest (content-reference, "A releases manifest"):
|
||||
// the newest version, its files per platform, and the standing
|
||||
// channels, with the visitor's own platform picked in the browser.
|
||||
if is_releases_manifest(&value) {
|
||||
return view! { <ReleasesCard manifest=value/> }.into_any();
|
||||
}
|
||||
|
||||
view! { <pre class="resource-raw">{value.to_string()}</pre> }.into_any()
|
||||
}
|
||||
|
||||
/// `{ name, latest, versions: [{ version, date, notes, assets: [{
|
||||
/// platform, url, sha256, size }] }], channels: {...} }` - the shape a
|
||||
/// release pipeline writes (redoal's does) for a site to offer downloads.
|
||||
pub fn is_releases_manifest(value: &serde_json::Value) -> bool {
|
||||
value.get("latest").and_then(|v| v.as_str()).is_some()
|
||||
&& value.get("versions").and_then(|v| v.as_array()).is_some_and(|v| !v.is_empty())
|
||||
}
|
||||
|
||||
/// The platform a browser says it is, in the manifest's words:
|
||||
/// `ios`, `macos`, `android`, `windows`, `linux-x86_64`,
|
||||
/// `linux-aarch64`, or `other`. Read, never trusted for anything but
|
||||
/// which download to put first.
|
||||
pub fn platform_of(user_agent: &str) -> &'static str {
|
||||
let ua = user_agent.to_ascii_lowercase();
|
||||
if ua.contains("iphone") || ua.contains("ipad") {
|
||||
"ios"
|
||||
} else if ua.contains("mac os x") || ua.contains("macintosh") {
|
||||
"macos"
|
||||
} else if ua.contains("android") {
|
||||
"android"
|
||||
} else if ua.contains("windows") {
|
||||
"windows"
|
||||
} else if ua.contains("linux") || ua.contains("x11") {
|
||||
if ua.contains("aarch64") || ua.contains("arm64") {
|
||||
"linux-aarch64"
|
||||
} else {
|
||||
"linux-x86_64"
|
||||
}
|
||||
} else {
|
||||
"other"
|
||||
}
|
||||
}
|
||||
|
||||
/// The newest release as a card: the file for this device first (the
|
||||
/// platform is read in the browser, so the server renders every file
|
||||
/// and the pick lands on hydration), the rest after, and the channels
|
||||
/// that are not files - a package repo, TestFlight.
|
||||
#[component]
|
||||
fn ReleasesCard(manifest: serde_json::Value) -> impl IntoView {
|
||||
let name = manifest.get("name").and_then(|v| v.as_str()).unwrap_or("").to_string();
|
||||
let latest = manifest.get("latest").and_then(|v| v.as_str()).unwrap_or("").to_string();
|
||||
let newest = manifest["versions"][0].clone();
|
||||
let date = newest.get("date").and_then(|v| v.as_str()).unwrap_or("").to_string();
|
||||
let notes = newest.get("notes").and_then(|v| v.as_str()).filter(|u| u.starts_with("https://")).map(str::to_string);
|
||||
let assets: Vec<(String, String, String)> = newest
|
||||
.get("assets")
|
||||
.and_then(|a| a.as_array())
|
||||
.map(|a| {
|
||||
a.iter()
|
||||
.filter_map(|x| {
|
||||
let platform = x.get("platform")?.as_str()?.to_string();
|
||||
let url = x.get("url")?.as_str()?.to_string();
|
||||
let sha = x.get("sha256").and_then(|s| s.as_str()).unwrap_or("").to_string();
|
||||
Some((platform, url, sha))
|
||||
})
|
||||
.collect()
|
||||
})
|
||||
.unwrap_or_default();
|
||||
let channels = manifest.get("channels").cloned().unwrap_or(serde_json::Value::Null);
|
||||
let arch = channels.get("arch").cloned();
|
||||
let ios = channels.get("ios").cloned();
|
||||
|
||||
let platform = RwSignal::new(None::<&'static str>);
|
||||
Effect::new(move |_| {
|
||||
#[cfg(feature = "hydrate")]
|
||||
if let Some(ua) = web_sys::window().and_then(|w| w.navigator().user_agent().ok()) {
|
||||
platform.set(Some(platform_of(&ua)));
|
||||
}
|
||||
});
|
||||
let label = |p: &str| match p {
|
||||
"linux-x86_64" => "Linux (x86_64)",
|
||||
"linux-aarch64" => "Linux (aarch64)",
|
||||
"macos" | "macos-arm64" => "macOS",
|
||||
"windows" | "windows-x86_64" => "Windows",
|
||||
other => other,
|
||||
}.to_string();
|
||||
let assets_for_list = assets.clone();
|
||||
let arch_for_pick = arch.clone();
|
||||
let ios_for_pick = ios.clone();
|
||||
let assets_for_pick = assets.clone();
|
||||
view! {
|
||||
<article class="item-card releases">
|
||||
<div class="item-card-title">
|
||||
<strong>{name.clone()} " " {latest.clone()}</strong>
|
||||
{(!date.is_empty()).then(|| view! { <span class="dim">" · " {date.clone()}</span> })}
|
||||
{notes.clone().map(|u| view! { " · " <a href=u target="_blank" rel="noopener noreferrer">{move || t(&lang().get(), "release_notes")}</a> })}
|
||||
</div>
|
||||
// What this device should take, decided in the browser.
|
||||
{move || {
|
||||
let p = platform.get()?;
|
||||
let arch = arch_for_pick.clone();
|
||||
let ios = ios_for_pick.clone();
|
||||
// "macos" is satisfied by a "macos-arm64" file, and so on.
|
||||
let own = assets_for_pick.iter().find(|(pl, _, _)| pl == p || pl.starts_with(&format!("{p}-"))).cloned();
|
||||
let pick: Option<AnyView> = match (p, own) {
|
||||
(_, Some((pl, url, _))) => Some(view! {
|
||||
<p><a class="pill" href=url download>{t(&lang().get(), "for_this_device")} ": " {label(&pl)}</a></p>
|
||||
{(p.starts_with("linux") && arch.is_some()).then(|| {
|
||||
let a = arch.clone().unwrap();
|
||||
let repo = a.get("repo").and_then(|v| v.as_str()).unwrap_or("").to_string();
|
||||
let pkg = a.get("package").and_then(|v| v.as_str()).unwrap_or("").to_string();
|
||||
view! { <pre class="monospace dim">{repo} "\n\npacman -Syu " {pkg}</pre> }
|
||||
})}
|
||||
}.into_any()),
|
||||
("ios" | "macos", None) => ios.as_ref().and_then(|i| i.get("testflight").and_then(|v| v.as_str()).map(str::to_string)).map(|tf| {
|
||||
let note = ios.as_ref().and_then(|i| i.get("note").and_then(|v| v.as_str())).unwrap_or("").to_string();
|
||||
view! { <p><a class="pill" href=tf target="_blank" rel="noopener noreferrer">{t(&lang().get(), "for_this_device")} ": TestFlight"</a> <span class="dim">" " {note}</span></p> }.into_any()
|
||||
}),
|
||||
_ => None,
|
||||
};
|
||||
pick
|
||||
}}
|
||||
<details class="releases-all">
|
||||
<summary>{move || t(&lang().get(), "all_downloads")}</summary>
|
||||
<ul>
|
||||
{assets_for_list.iter().map(|(pl, url, sha)| view! {
|
||||
<li><a href=url.clone() download>{label(pl)}</a> {(!sha.is_empty()).then(|| view! { <span class="dim monospace">" sha256 " {sha[..sha.len().min(12)].to_string()} "…"</span> })}</li>
|
||||
}).collect::<Vec<_>>()}
|
||||
{arch.as_ref().map(|a| {
|
||||
let note = a.get("note").and_then(|v| v.as_str()).unwrap_or("").to_string();
|
||||
view! { <li>{note}</li> }
|
||||
})}
|
||||
{ios.as_ref().and_then(|i| i.get("testflight").and_then(|v| v.as_str()).map(|tf| {
|
||||
let note = i.get("note").and_then(|v| v.as_str()).unwrap_or("").to_string();
|
||||
view! { <li><a href=tf.to_string() target="_blank" rel="noopener noreferrer">"TestFlight"</a> " " {note}</li> }
|
||||
}))}
|
||||
</ul>
|
||||
</details>
|
||||
</article>
|
||||
}
|
||||
}
|
||||
|
||||
/// Pulls the first present, non-null string field out of a JSON object
|
||||
/// matching one of `keys`, tried in order - shared between `ItemCard`
|
||||
/// and `SelectField`'s option rendering, since both need "guess a
|
||||
@@ -3328,3 +3504,27 @@ mod picture_tests {
|
||||
assert_eq!(picture_of(&answer, None), None);
|
||||
}
|
||||
}
|
||||
|
||||
#[cfg(test)]
|
||||
mod releases_tests {
|
||||
use super::*;
|
||||
|
||||
#[test]
|
||||
fn a_browser_names_its_platform_in_the_manifests_words() {
|
||||
assert_eq!(platform_of("Mozilla/5.0 (iPhone; CPU iPhone OS 19_0 like Mac OS X)"), "ios");
|
||||
assert_eq!(platform_of("Mozilla/5.0 (Macintosh; Intel Mac OS X 15_0)"), "macos");
|
||||
assert_eq!(platform_of("Mozilla/5.0 (X11; Linux x86_64) Firefox"), "linux-x86_64");
|
||||
assert_eq!(platform_of("Mozilla/5.0 (X11; Linux aarch64) Firefox"), "linux-aarch64");
|
||||
assert_eq!(platform_of("Mozilla/5.0 (Windows NT 10.0; Win64; x64)"), "windows");
|
||||
assert_eq!(platform_of("Mozilla/5.0 (Linux; Android 15)"), "android");
|
||||
assert_eq!(platform_of("curl/8"), "other");
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn only_a_manifest_shape_is_a_manifest() {
|
||||
let m = serde_json::json!({"name":"redoal","latest":"0.1.17","versions":[{"version":"0.1.17","assets":[]}]});
|
||||
assert!(is_releases_manifest(&m));
|
||||
assert!(!is_releases_manifest(&serde_json::json!({"latest":"1","versions":[]})));
|
||||
assert!(!is_releases_manifest(&serde_json::json!([{"name":"x"}])));
|
||||
}
|
||||
}
|
||||
|
||||
+115
-15
@@ -49,15 +49,56 @@ pub struct GiteaRepoQuery {
|
||||
pub async fn site_asset_handler(
|
||||
axum::extract::State(state): axum::extract::State<crate::server::AppState>,
|
||||
axum::extract::Path(path): axum::extract::Path<String>,
|
||||
) -> axum::response::Response {
|
||||
serve_repo_file(&state, &path, false).await
|
||||
}
|
||||
|
||||
/// `GET /.well-known/{*path}` - the content repo overlaid on the one part
|
||||
/// of the URL space that can never collide with a question: RFC 8615
|
||||
/// reserves `/.well-known/`, and a question's id is derived from a file
|
||||
/// name under `questions/`, which cannot begin with a dot. A repo ships
|
||||
/// `.well-known/security.txt`, `.well-known/assetlinks.json`,
|
||||
/// `.well-known/matrix/server` and so on at the path they are served at,
|
||||
/// and a miss is a real 404 rather than the app shell answering 200 with
|
||||
/// an HTML page - which is what a client probing for one of these would
|
||||
/// otherwise have to parse.
|
||||
///
|
||||
/// Paths Caddy claims in front of portal (mail autoconfiguration, the
|
||||
/// CalDAV and CardDAV redirects) never reach here.
|
||||
#[cfg(feature = "ssr")]
|
||||
pub async fn well_known_handler(
|
||||
axum::extract::State(state): axum::extract::State<crate::server::AppState>,
|
||||
axum::extract::Path(path): axum::extract::Path<String>,
|
||||
) -> axum::response::Response {
|
||||
serve_repo_file(&state, &format!(".well-known/{path}"), true).await
|
||||
}
|
||||
|
||||
/// Proxies one repo-relative file from the content repo's default branch.
|
||||
/// `well_known` only changes what an extensionless file is typed as: under
|
||||
/// `/.well-known/` those are JSON by convention (`matrix/server`,
|
||||
/// `nodeinfo`), while elsewhere a file with no extension is an unknown.
|
||||
#[cfg(feature = "ssr")]
|
||||
async fn serve_repo_file(
|
||||
state: &crate::server::AppState,
|
||||
path: &str,
|
||||
well_known: bool,
|
||||
) -> axum::response::Response {
|
||||
use axum::http::{header, StatusCode};
|
||||
use axum::response::IntoResponse;
|
||||
|
||||
if !is_safe_site_path(&path) {
|
||||
if !is_safe_site_path(path) {
|
||||
return (StatusCode::BAD_REQUEST, "invalid asset path").into_response();
|
||||
}
|
||||
let url = format!("{}/{path}?ref={}", state.content_raw_base, state.content_branch);
|
||||
let resp = match crate::http::client().get(&url).send().await.and_then(|r| r.error_for_status()) {
|
||||
let url = format!(
|
||||
"{}/{path}?ref={}",
|
||||
state.content_raw_base, state.content_branch
|
||||
);
|
||||
let resp = match crate::http::client()
|
||||
.get(&url)
|
||||
.send()
|
||||
.await
|
||||
.and_then(|r| r.error_for_status())
|
||||
{
|
||||
Ok(r) => r,
|
||||
Err(e) => {
|
||||
tracing::warn!("site asset {path}: {e}");
|
||||
@@ -79,12 +120,26 @@ pub async fn site_asset_handler(
|
||||
Some("avif") => "image/avif",
|
||||
Some("ico") => "image/x-icon",
|
||||
Some("woff2") => "font/woff2",
|
||||
Some("txt") => "text/plain; charset=utf-8",
|
||||
Some("xml") => "application/xml; charset=utf-8",
|
||||
Some("webmanifest") => "application/manifest+json",
|
||||
// An Apple configuration profile only offers to install when it
|
||||
// arrives under this type; anything else is just a download.
|
||||
Some("mobileconfig") => "application/x-apple-aspen-config",
|
||||
// A file with no extension at all: rsplit yields the whole name.
|
||||
_ if well_known && !path.rsplit('/').next().unwrap_or_default().contains('.') => {
|
||||
"application/json; charset=utf-8"
|
||||
}
|
||||
_ => "application/octet-stream",
|
||||
};
|
||||
// Photographs are large and do not change under a name, so a
|
||||
// browser may keep them for an hour; everything else revalidates,
|
||||
// so a content push shows on reload.
|
||||
let cache = if matches!(mime, "image/webp" | "image/avif" | "image/jpeg") { "public, max-age=3600" } else { "no-cache" };
|
||||
let cache = if matches!(mime, "image/webp" | "image/avif" | "image/jpeg") {
|
||||
"public, max-age=3600"
|
||||
} else {
|
||||
"no-cache"
|
||||
};
|
||||
(
|
||||
[(header::CONTENT_TYPE, mime), (header::CACHE_CONTROL, cache)],
|
||||
bytes,
|
||||
@@ -114,14 +169,22 @@ pub async fn gitea_repo_handler(
|
||||
.all(|c| c.is_ascii_alphanumeric() || c == '-' || c == '_' || c == '.')
|
||||
};
|
||||
if !is_safe_segment(&query.owner) || !is_safe_segment(&query.repo) {
|
||||
return Err((axum::http::StatusCode::BAD_REQUEST, "invalid owner/repo".to_string()));
|
||||
return Err((
|
||||
axum::http::StatusCode::BAD_REQUEST,
|
||||
"invalid owner/repo".to_string(),
|
||||
));
|
||||
}
|
||||
let GiteaRepoQuery { owner, repo } = query;
|
||||
|
||||
let api_url = format!("{}/api/v1/repos/{owner}/{repo}", state.gitea_base);
|
||||
let body = crate::http::get_text(&api_url, crate::http::GITEA_CAP)
|
||||
.await
|
||||
.map_err(|e| (axum::http::StatusCode::BAD_GATEWAY, format!("fetching {api_url}: {e}")))?;
|
||||
.map_err(|e| {
|
||||
(
|
||||
axum::http::StatusCode::BAD_GATEWAY,
|
||||
format!("fetching {api_url}: {e}"),
|
||||
)
|
||||
})?;
|
||||
let json: serde_json::Value = serde_json::from_str(&body).map_err(|e| {
|
||||
(
|
||||
axum::http::StatusCode::BAD_GATEWAY,
|
||||
@@ -168,27 +231,38 @@ pub async fn automation_kv_handler(
|
||||
.and_then(|v| v.strip_prefix("Bearer "))
|
||||
.unwrap_or("");
|
||||
if expected.is_empty() || presented != expected {
|
||||
return Err((axum::http::StatusCode::UNAUTHORIZED, "unauthorized".to_string()));
|
||||
return Err((
|
||||
axum::http::StatusCode::UNAUTHORIZED,
|
||||
"unauthorized".to_string(),
|
||||
));
|
||||
}
|
||||
// The token authenticates the automation service; what it may read
|
||||
// is the policy's to say, like every other principal.
|
||||
let decision = crate::access::decide_and_publish(
|
||||
&state.nats,
|
||||
&state.access.load(),
|
||||
&crate::access::Principal::Service { name: "automation".to_string() },
|
||||
&crate::access::Principal::Service {
|
||||
name: "automation".to_string(),
|
||||
},
|
||||
crate::access::Action::AutomationRead,
|
||||
&crate::access::Resource::Bucket { name: bucket.clone() },
|
||||
&crate::access::Resource::Bucket {
|
||||
name: bucket.clone(),
|
||||
},
|
||||
serde_json::json!({}),
|
||||
);
|
||||
if !decision.allow {
|
||||
return Err((axum::http::StatusCode::FORBIDDEN, "not authorized".to_string()));
|
||||
return Err((
|
||||
axum::http::StatusCode::FORBIDDEN,
|
||||
"not authorized".to_string(),
|
||||
));
|
||||
}
|
||||
|
||||
let store = state
|
||||
.jetstream
|
||||
.get_key_value(&bucket)
|
||||
.await
|
||||
.map_err(|e| (axum::http::StatusCode::BAD_GATEWAY, format!("bucket unavailable: {e}")))?;
|
||||
let store = state.jetstream.get_key_value(&bucket).await.map_err(|e| {
|
||||
(
|
||||
axum::http::StatusCode::BAD_GATEWAY,
|
||||
format!("bucket unavailable: {e}"),
|
||||
)
|
||||
})?;
|
||||
|
||||
use futures::TryStreamExt;
|
||||
let keys: Vec<String> = store
|
||||
@@ -209,3 +283,29 @@ pub async fn automation_kv_handler(
|
||||
Ok(axum::Json(serde_json::Value::Array(items)))
|
||||
}
|
||||
|
||||
#[cfg(test)]
|
||||
mod tests {
|
||||
use super::*;
|
||||
|
||||
/// The overlay's whole safety argument: `/.well-known/` is reserved by
|
||||
/// RFC 8615 and a question's id comes from a file under `questions/`, so
|
||||
/// a repo file served at the site root cannot shadow a page - and the
|
||||
/// path it is fetched by still cannot climb out of the repo.
|
||||
#[test]
|
||||
fn well_known_paths_stay_inside_the_repo() {
|
||||
for path in [
|
||||
".well-known/security.txt",
|
||||
".well-known/matrix/server",
|
||||
".well-known/assetlinks.json",
|
||||
] {
|
||||
assert!(is_safe_site_path(path), "{path} should be servable");
|
||||
}
|
||||
for path in [
|
||||
".well-known/../questions/index.yaml",
|
||||
".well-known//escape",
|
||||
"/.well-known/security.txt",
|
||||
] {
|
||||
assert!(!is_safe_site_path(path), "{path} should be refused");
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
@@ -519,6 +519,18 @@ pub struct ResourceSpec {
|
||||
/// it where the silence needs a voice ("Nothing released yet").
|
||||
#[serde(default)]
|
||||
pub empty: Option<String>,
|
||||
/// Show at most this many items, applied after `jq`. For a feed
|
||||
/// where only the newest matters: a releases card wants the cut you
|
||||
/// should run, not every cut there has ever been, and a page that
|
||||
/// grows by one card per release gets longer forever.
|
||||
///
|
||||
/// After `jq` rather than before, so it caps what the page will
|
||||
/// actually render whatever the filter did — and so content can
|
||||
/// reorder with jq and still take the top of its own ordering. A
|
||||
/// source that yields a single record has no length and passes
|
||||
/// through untouched. `None` shows everything.
|
||||
#[serde(default)]
|
||||
pub limit: Option<usize>,
|
||||
}
|
||||
|
||||
/// Where a resource's live data comes from. `Kv` (a NATS KV bucket
|
||||
|
||||
+1
-1
@@ -194,7 +194,7 @@ aggregates:
|
||||
Answer {
|
||||
id: "r".into(), question_id: "/q".into(), alternative: "a".into(),
|
||||
responses: serde_json::json!({}), state: state.into(),
|
||||
submitted_ms: since_ms, decided_ms: None, decided_by: None, notes: vec![],
|
||||
submitted_ms: since_ms, decided_ms: None, decided_by: None, notes: vec![], finished: false,
|
||||
storyline: 0,
|
||||
}
|
||||
}
|
||||
|
||||
+12
@@ -24,6 +24,10 @@ fn lookup(lang: &str, key: &str) -> Option<&'static str> {
|
||||
("en", "sign_in") => "Sign in",
|
||||
("en", "sign_out") => "Sign out",
|
||||
("en", "nothing_here_yet") => "Nothing here yet.",
|
||||
("en", "show_older") => "Show older",
|
||||
("en", "for_this_device") => "For this device",
|
||||
("en", "all_downloads") => "All downloads",
|
||||
("en", "release_notes") => "release notes",
|
||||
("en", "upload_failed") => "The file did not arrive, so nothing was sent. Try again.",
|
||||
("en", "upload_too_large") => "The file is larger than 20 MB, so nothing was sent. Choose a smaller one, or a picture taken at a lower resolution.",
|
||||
("en", "upload_unavailable") => "This site cannot take files at the moment, so nothing was sent. Let whoever runs the site know.",
|
||||
@@ -60,6 +64,10 @@ fn lookup(lang: &str, key: &str) -> Option<&'static str> {
|
||||
("no", "sign_in") => "Logg inn",
|
||||
("no", "sign_out") => "Logg ut",
|
||||
("no", "nothing_here_yet") => "Ingenting her ennå.",
|
||||
("no", "show_older") => "Vis eldre",
|
||||
("no", "for_this_device") => "Til denne enheten",
|
||||
("no", "all_downloads") => "Alle nedlastinger",
|
||||
("no", "release_notes") => "utgivelsesnotater",
|
||||
("no", "upload_failed") => "Filen kom ikke fram, så ingenting ble sendt. Prøv igjen.",
|
||||
("no", "upload_too_large") => "Filen er større enn 20 MB, så ingenting ble sendt. Velg en mindre fil, eller et bilde med lavere oppløsning.",
|
||||
("no", "upload_unavailable") => "Siden kan ikke ta imot filer akkurat nå, så ingenting ble sendt. Si fra til den som har siden.",
|
||||
@@ -114,6 +122,10 @@ mod tests {
|
||||
"sign_in",
|
||||
"sign_out",
|
||||
"nothing_here_yet",
|
||||
"show_older",
|
||||
"for_this_device",
|
||||
"all_downloads",
|
||||
"release_notes",
|
||||
"upload_failed",
|
||||
"upload_too_large",
|
||||
"upload_unavailable",
|
||||
|
||||
+61
-12
@@ -24,6 +24,7 @@ pub async fn run(argv: Vec<String>) -> anyhow::Result<()> {
|
||||
let mut skim = false;
|
||||
let mut sim_out: Option<String> = None;
|
||||
let mut show_access = false;
|
||||
let mut findings_out: Option<String> = None;
|
||||
|
||||
while let Some(arg) = args.next() {
|
||||
match arg.as_str() {
|
||||
@@ -35,6 +36,7 @@ pub async fn run(argv: Vec<String>) -> anyhow::Result<()> {
|
||||
"--skim" => skim = true,
|
||||
"--needs-sim" => sim_out = args.next(),
|
||||
"--access" => show_access = true,
|
||||
"--findings" => findings_out = args.next(),
|
||||
"--needs-score" => answers_in = args.next(),
|
||||
"--min-accuracy" => {
|
||||
min_accuracy = args.next().and_then(|v| v.parse().ok()).unwrap_or(min_accuracy)
|
||||
@@ -66,13 +68,14 @@ pub async fn run(argv: Vec<String>) -> anyhow::Result<()> {
|
||||
(questions, aggregates_map, site, needs_raw)
|
||||
} else {
|
||||
eprintln!(
|
||||
"usage: iris check (--repo <gitea-url> [--branch main] [--subdir questions] | --path <local-dir>) [--needs-tasks <out.jsonl> [--skim]] [--needs-sim <model.json>] [--access] [--needs-score <answers.jsonl> [--min-accuracy 0.8]]"
|
||||
"usage: iris check (--repo <gitea-url> [--branch main] [--subdir questions] | --path <local-dir>) [--findings <out.jsonl>] [--needs-tasks <out.jsonl> [--skim]] [--needs-sim <model.json>] [--access] [--needs-score <answers.jsonl> [--min-accuracy 0.8]]"
|
||||
);
|
||||
std::process::exit(2);
|
||||
};
|
||||
|
||||
if let Err(e) = site.validate() {
|
||||
eprintln!("FAIL: {e}");
|
||||
write_findings(findings_out.as_deref(), &[schema_finding(&e.to_string())]);
|
||||
std::process::exit(1);
|
||||
}
|
||||
let aggregates_map = content::with_builtin_aggregates(aggregates_map);
|
||||
@@ -114,6 +117,7 @@ pub async fn run(argv: Vec<String>) -> anyhow::Result<()> {
|
||||
println!();
|
||||
}
|
||||
check_needs(
|
||||
findings_out.as_deref(),
|
||||
needs_raw.as_deref(),
|
||||
&questions,
|
||||
&aggregates_map,
|
||||
@@ -126,15 +130,67 @@ pub async fn run(argv: Vec<String>) -> anyhow::Result<()> {
|
||||
}
|
||||
Err(e) => {
|
||||
eprintln!("FAIL: {e}");
|
||||
// One schema error is no reason to hide the rest: the needs
|
||||
// pass reads the same content and says what else is wrong,
|
||||
// so a fix round can take them all at once.
|
||||
let mut all = vec![schema_finding(&e.to_string())];
|
||||
if let Some(file) = needs_raw.as_deref().and_then(|raw| needs::parse(raw).ok()) {
|
||||
let findings = needs::check(&file, &questions, &aggregates_map);
|
||||
print_findings(&findings);
|
||||
all.extend(findings);
|
||||
}
|
||||
write_findings(findings_out.as_deref(), &all);
|
||||
std::process::exit(1);
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
/// A content error the schema pass stopped at, as a finding: the repo
|
||||
/// is its scope and the message its subject, since it has no other.
|
||||
fn schema_finding(message: &str) -> needs::Finding {
|
||||
needs::Finding {
|
||||
need: String::new(),
|
||||
level: needs::Level::Fail,
|
||||
message: message.to_string(),
|
||||
scope: "repo",
|
||||
code: "schema",
|
||||
subject: message.to_string(),
|
||||
}
|
||||
}
|
||||
|
||||
/// `--findings`: every finding as one JSON line (id, level, code,
|
||||
/// scope, need, subject, message, fix), the same ones the text output
|
||||
/// prints - an empty file when there are none, so a reader can tell
|
||||
/// "clean" from "not run".
|
||||
fn write_findings(out: Option<&str>, findings: &[needs::Finding]) {
|
||||
let Some(out) = out else { return };
|
||||
let lines: String = findings.iter().map(|f| format!("{}\n", f.to_json())).collect();
|
||||
if let Err(e) = std::fs::write(out, lines) {
|
||||
eprintln!("WARN: could not write the findings to {out}: {e}");
|
||||
}
|
||||
}
|
||||
|
||||
/// Each finding on its own line, FAIL or WARN first; the count of FAILs.
|
||||
fn print_findings(findings: &[needs::Finding]) -> usize {
|
||||
let mut failed = 0;
|
||||
for finding in findings {
|
||||
let level = match finding.level {
|
||||
needs::Level::Fail => {
|
||||
failed += 1;
|
||||
"FAIL"
|
||||
}
|
||||
needs::Level::Warn => "WARN",
|
||||
};
|
||||
eprintln!("{level}: {} {:?}: {}", finding.scope, finding.need, finding.message);
|
||||
}
|
||||
failed
|
||||
}
|
||||
|
||||
/// The business-needs pass (see `crate::lint::needs`): structure always,
|
||||
/// then the wording tasks written out and/or an engine's answers
|
||||
/// scored, when asked. A repo without `needs.yaml` skips all of it.
|
||||
fn check_needs(
|
||||
findings_out: Option<&str>,
|
||||
raw: Option<&str>,
|
||||
questions: &std::collections::HashMap<String, content::Question>,
|
||||
aggregates_map: &std::collections::HashMap<String, aggregates::AggregateSchema>,
|
||||
@@ -145,6 +201,7 @@ fn check_needs(
|
||||
min_accuracy: f64,
|
||||
) -> anyhow::Result<()> {
|
||||
let Some(raw) = raw else {
|
||||
write_findings(findings_out, &[]);
|
||||
if tasks_out.is_some() || answers_in.is_some() {
|
||||
eprintln!("FAIL: --needs-tasks/--needs-score given, but the repo has no needs.yaml");
|
||||
std::process::exit(1);
|
||||
@@ -155,21 +212,13 @@ fn check_needs(
|
||||
Ok(file) => file,
|
||||
Err(e) => {
|
||||
eprintln!("FAIL: {e}");
|
||||
write_findings(findings_out, &[schema_finding(&e.to_string())]);
|
||||
std::process::exit(1);
|
||||
}
|
||||
};
|
||||
let findings = needs::check(&file, questions, aggregates_map);
|
||||
let mut failed = 0;
|
||||
for finding in &findings {
|
||||
let level = match finding.level {
|
||||
needs::Level::Fail => {
|
||||
failed += 1;
|
||||
"FAIL"
|
||||
}
|
||||
needs::Level::Warn => "WARN",
|
||||
};
|
||||
eprintln!("{level}: need {:?}: {}", finding.need, finding.message);
|
||||
}
|
||||
let failed = print_findings(&findings);
|
||||
write_findings(findings_out, &findings);
|
||||
for need in &file.needs {
|
||||
if let Some(path) = needs::path_for(questions, need) {
|
||||
let steps: Vec<String> = path
|
||||
|
||||
+551
-30
@@ -137,9 +137,55 @@ pub enum Level {
|
||||
|
||||
#[derive(Debug)]
|
||||
pub struct Finding {
|
||||
/// The need's id, or the bucket's name when `scope` is "bucket".
|
||||
pub need: String,
|
||||
pub level: Level,
|
||||
pub message: String,
|
||||
/// "need" for a need's own path; "bucket" for what holds for every
|
||||
/// record in a bucket whichever need brought it in.
|
||||
pub scope: &'static str,
|
||||
/// What kind of gap it is, stable across releases: `strand`,
|
||||
/// `edge-unoffered`, `done-unreachable`, ... (docs/lint.md).
|
||||
pub code: &'static str,
|
||||
/// What in the content it is about: a state, a move (`from>to`), a
|
||||
/// page. With the scope, need and code it makes the finding's id.
|
||||
pub subject: String,
|
||||
}
|
||||
|
||||
impl Finding {
|
||||
/// The same gap keeps the same id however its message is worded,
|
||||
/// so a fix round can tell what it closed, what is still open and
|
||||
/// what it broke. FNV-1a over scope, need, code and subject.
|
||||
pub fn id(&self) -> String {
|
||||
let mut hash: u64 = 0xcbf2_9ce4_8422_2325;
|
||||
for part in [self.scope, &self.need, self.code, &self.subject] {
|
||||
for byte in part.bytes().chain(std::iter::once(0)) {
|
||||
hash ^= u64::from(byte);
|
||||
hash = hash.wrapping_mul(0x0100_0000_01b3);
|
||||
}
|
||||
}
|
||||
format!("{:08x}", hash >> 32)
|
||||
}
|
||||
|
||||
/// The change that closes it, when the lint knows one: what follows
|
||||
/// "fix:" in the message.
|
||||
pub fn fix(&self) -> Option<&str> {
|
||||
self.message.split_once(" - fix: ").map(|(_, fix)| fix)
|
||||
}
|
||||
|
||||
/// One line of `--findings` JSONL.
|
||||
pub fn to_json(&self) -> serde_json::Value {
|
||||
serde_json::json!({
|
||||
"id": self.id(),
|
||||
"level": match self.level { Level::Fail => "fail", Level::Warn => "warn" },
|
||||
"code": self.code,
|
||||
"scope": self.scope,
|
||||
"need": self.need,
|
||||
"subject": self.subject,
|
||||
"message": self.message,
|
||||
"fix": self.fix(),
|
||||
})
|
||||
}
|
||||
}
|
||||
|
||||
#[derive(Clone, Debug, PartialEq, Eq)]
|
||||
@@ -255,6 +301,10 @@ pub fn path_for(questions: &Questions, need: &Need) -> Option<Vec<Step>> {
|
||||
struct Desk {
|
||||
page: String,
|
||||
group: Option<String>,
|
||||
/// The page's own `qualifies`, kept apart from the resource's
|
||||
/// `requires_group`: when both are set and differ, only members of
|
||||
/// both ever see the buttons.
|
||||
page_group: Option<String>,
|
||||
/// (from, to, button label)
|
||||
moves: Vec<(String, String, String)>,
|
||||
}
|
||||
@@ -272,6 +322,7 @@ fn desks_over(questions: &Questions, bucket: &str) -> Vec<Desk> {
|
||||
desks.push(Desk {
|
||||
page: q.id.clone(),
|
||||
group: resource.requires_group.clone().or_else(|| q.qualifies.clone()),
|
||||
page_group: q.qualifies.clone(),
|
||||
moves: resource
|
||||
.transitions
|
||||
.iter()
|
||||
@@ -284,15 +335,49 @@ fn desks_over(questions: &Questions, bucket: &str) -> Vec<Desk> {
|
||||
desks
|
||||
}
|
||||
|
||||
/// Self-service moves into `bucket` - a visitor holding their own
|
||||
/// record's link, no group involved (unsubscribe).
|
||||
fn self_moves(questions: &Questions, bucket: &str) -> Vec<String> {
|
||||
/// Self-service moves in `bucket` - a visitor holding their own
|
||||
/// record's link, no group involved (unsubscribe, withdraw) - as
|
||||
/// (from, to), from the states each may be made from.
|
||||
fn self_moves(questions: &Questions, bucket: &str, initial: &str) -> Vec<(String, String)> {
|
||||
questions
|
||||
.values()
|
||||
.flat_map(|q| &q.alternatives)
|
||||
.filter_map(|a| a.self_transition.as_ref())
|
||||
.filter(|s| s.bucket == bucket)
|
||||
.map(|s| s.to.clone())
|
||||
.flat_map(|s| s.from_states(initial).into_iter().map(|from| (from, s.to.clone())))
|
||||
.collect()
|
||||
}
|
||||
|
||||
/// The moves a clock makes in `schema` (`after:` on a state), as
|
||||
/// (from, to). Nobody presses them, and they still move records.
|
||||
fn clock_moves(schema: &AggregateSchema) -> Vec<(String, String)> {
|
||||
let mut moves: Vec<(String, String)> =
|
||||
schema.after_for_state.iter().map(|(from, after)| (from.clone(), after.to.clone())).collect();
|
||||
moves.sort();
|
||||
moves
|
||||
}
|
||||
|
||||
/// States reachable from `start` along `moves`.
|
||||
fn reachable(start: &str, moves: &[(String, String)]) -> HashSet<String> {
|
||||
let mut reached = HashSet::from([start.to_string()]);
|
||||
let mut queue = VecDeque::from([start.to_string()]);
|
||||
while let Some(state) = queue.pop_front() {
|
||||
for (_, to) in moves.iter().filter(|(from, _)| *from == state) {
|
||||
if reached.insert(to.clone()) {
|
||||
queue.push_back(to.clone());
|
||||
}
|
||||
}
|
||||
}
|
||||
reached
|
||||
}
|
||||
|
||||
/// Who else offers `from -> to` in the bucket, as "page (group)", for
|
||||
/// a finding that has to say where the missing button already is.
|
||||
fn offered_elsewhere(desks: &[Desk], from: &str, to: &str) -> Vec<String> {
|
||||
desks
|
||||
.iter()
|
||||
.filter(|d| d.moves.iter().any(|(f, t, _)| f == from && t == to))
|
||||
.map(|d| format!("{} ({})", d.page, d.group.as_deref().unwrap_or("ungated")))
|
||||
.collect()
|
||||
}
|
||||
|
||||
@@ -305,10 +390,18 @@ pub fn check(
|
||||
) -> Vec<Finding> {
|
||||
let mut findings = Vec::new();
|
||||
let mut seen = HashSet::new();
|
||||
// (bucket, state) a need already reported - stranding on its way
|
||||
// to done, or out of its reach - so the bucket pass does not say
|
||||
// it twice.
|
||||
let mut strands_reported: HashSet<(String, String)> = HashSet::new();
|
||||
let mut unreached_reported: HashSet<(String, String)> = HashSet::new();
|
||||
for need in &file.needs {
|
||||
let mut report = |level: Level, message: String| {
|
||||
let mut report = |level: Level, code: &'static str, subject: String, message: String| {
|
||||
findings.push(Finding {
|
||||
need: need.id.clone(),
|
||||
scope: "need",
|
||||
code,
|
||||
subject,
|
||||
level: if need.planned || need.paused { Level::Warn } else { level },
|
||||
message: match (need.paused, need.planned, level == Level::Fail) {
|
||||
(true, _, true) => format!("paused, not open now: {message}"),
|
||||
@@ -318,15 +411,17 @@ pub fn check(
|
||||
})
|
||||
};
|
||||
if !seen.insert(need.id.as_str()) {
|
||||
report(Level::Fail, "declared more than once".to_string());
|
||||
report(Level::Fail, "need-duplicate", String::new(), "declared more than once".to_string());
|
||||
continue;
|
||||
}
|
||||
|
||||
// The visitor's side: can they get their answer recorded?
|
||||
match questions.get(&need.enters) {
|
||||
None => report(Level::Fail, format!("entry page {:?} does not exist", need.enters)),
|
||||
None => report(Level::Fail, "entry-missing", need.enters.clone(), format!("entry page {:?} does not exist", need.enters)),
|
||||
Some(page) if !visible(page, need.enters_as.as_deref()) => report(
|
||||
Level::Fail,
|
||||
"entry-gated",
|
||||
need.enters.clone(),
|
||||
format!(
|
||||
"entry page {:?} is gated on {:?}, which this visitor does not hold",
|
||||
need.enters, page.qualifies
|
||||
@@ -336,6 +431,8 @@ pub fn check(
|
||||
if page.responsible.is_none() {
|
||||
report(
|
||||
Level::Warn,
|
||||
"entry-no-responsible",
|
||||
need.enters.clone(),
|
||||
format!("entry page {:?} names no one responsible to turn to", need.enters),
|
||||
);
|
||||
}
|
||||
@@ -363,6 +460,8 @@ pub fn check(
|
||||
.unwrap_or_default();
|
||||
report(
|
||||
Level::Fail,
|
||||
"path-closed",
|
||||
at.clone(),
|
||||
format!(
|
||||
"the path exists but is closed: {at} is `disabled`, so the answer is advertised and refused"
|
||||
),
|
||||
@@ -370,6 +469,8 @@ pub fn check(
|
||||
}
|
||||
None => report(
|
||||
Level::Fail,
|
||||
"path-missing",
|
||||
String::new(),
|
||||
format!(
|
||||
"no path from {:?}{} records into {:?} within {} submission(s)",
|
||||
need.enters,
|
||||
@@ -386,6 +487,8 @@ pub fn check(
|
||||
if best.fields > budget {
|
||||
report(
|
||||
Level::Fail,
|
||||
"path-over-budget",
|
||||
String::new(),
|
||||
format!(
|
||||
"the path asks for {} required field(s), over the budget of {budget}",
|
||||
best.fields
|
||||
@@ -396,6 +499,8 @@ pub fn check(
|
||||
if best.records_with_action == Some(false) {
|
||||
report(
|
||||
Level::Warn,
|
||||
"path-no-action",
|
||||
String::new(),
|
||||
"the recording alternative has no `action` - the visitor is never told what happens next".to_string(),
|
||||
);
|
||||
}
|
||||
@@ -408,6 +513,8 @@ pub fn check(
|
||||
let Some(schema) = aggregates.get(&need.lands_in) else {
|
||||
report(
|
||||
Level::Fail,
|
||||
"machine-missing",
|
||||
need.lands_in.clone(),
|
||||
format!("aggregates.yaml declares no state machine for {:?}", need.lands_in),
|
||||
);
|
||||
continue;
|
||||
@@ -426,6 +533,8 @@ pub fn check(
|
||||
.collect();
|
||||
report(
|
||||
Level::Fail,
|
||||
"desk-missing",
|
||||
need.handled_by.clone(),
|
||||
format!(
|
||||
"no desk gated on {:?} reads {:?}{}",
|
||||
need.handled_by,
|
||||
@@ -439,23 +548,44 @@ pub fn check(
|
||||
);
|
||||
continue;
|
||||
}
|
||||
let selfs = self_moves(questions, &need.lands_in);
|
||||
// Every move that carries this need's records: the buttons on
|
||||
// the desks of handled_by/also_moved_by, the visitor's own
|
||||
// links, and the clocks.
|
||||
let mut moves: Vec<(String, String)> = own
|
||||
.iter()
|
||||
.flat_map(|d| &d.moves)
|
||||
.filter(|(from, to, _)| schema.allowed(from).contains(to))
|
||||
.map(|(from, to, _)| (from.clone(), to.clone()))
|
||||
.collect();
|
||||
moves.extend(self_moves(questions, &need.lands_in, &schema.initial));
|
||||
moves.extend(clock_moves(schema));
|
||||
let exits = |state: &str| -> Vec<String> {
|
||||
let allowed = schema.allowed(state);
|
||||
let mut to: Vec<String> = own
|
||||
.iter()
|
||||
.flat_map(|d| &d.moves)
|
||||
.filter(|(from, to, _)| from == state && allowed.contains(to))
|
||||
.map(|(_, to, _)| to.clone())
|
||||
.collect();
|
||||
to.extend(selfs.iter().filter(|to| allowed.contains(to)).cloned());
|
||||
to
|
||||
moves.iter().filter(|(from, _)| from == state).map(|(_, to)| to.clone()).collect()
|
||||
};
|
||||
let movers = std::iter::once(need.handled_by.as_str())
|
||||
.chain(need.also_moved_by.iter().map(String::as_str))
|
||||
.map(|g| format!("{g:?}"))
|
||||
.collect::<Vec<_>>()
|
||||
.join(" or ");
|
||||
// The fix for a missing move: the button on handled_by's desk,
|
||||
// or the group whose desk already has it added to also_moved_by.
|
||||
let fix_for = |from: &str, to: &str| -> String {
|
||||
let elsewhere = offered_elsewhere(&desks, from, to);
|
||||
let desk = own.first().map(|d| d.page.as_str()).unwrap_or("their desk");
|
||||
if elsewhere.is_empty() {
|
||||
format!("add `{{ from: {from}, to: {to}, label: ... }}` to the transitions on {desk}")
|
||||
} else {
|
||||
format!(
|
||||
"{} already offers it - add that group to also_moved_by, or add `{{ from: {from}, to: {to}, label: ... }}` on {desk}",
|
||||
elsewhere.join(", ")
|
||||
)
|
||||
}
|
||||
};
|
||||
let done: HashSet<&str> = need.done_when.iter().map(String::as_str).collect();
|
||||
// Reachability walks the whole machine (a finished state may
|
||||
// lead on to another: subscribed, then unsubscribed). Stranding
|
||||
// only counts on the way to the first finished state - what
|
||||
// happens to a handled record afterwards is not this need.
|
||||
// only counts on the way to the first finished state here; what
|
||||
// strands a record afterwards is the bucket's finding (below).
|
||||
let mut reached: HashSet<String> = HashSet::from([schema.initial.clone()]);
|
||||
let mut queue = VecDeque::from([(schema.initial.clone(), false)]);
|
||||
let mut stranded = Vec::new();
|
||||
@@ -471,18 +601,44 @@ pub fn check(
|
||||
}
|
||||
}
|
||||
}
|
||||
// The declared edges into `state` from somewhere a record does
|
||||
// get to: the moves that would close the gap.
|
||||
let ways_in = |state: &str| -> Vec<(String, String)> {
|
||||
let mut ways: Vec<(String, String)> = reached
|
||||
.iter()
|
||||
.filter(|from| schema.allowed(from).iter().any(|to| to == state))
|
||||
.map(|from| (from.clone(), state.to_string()))
|
||||
.collect();
|
||||
ways.sort();
|
||||
ways
|
||||
};
|
||||
let missing = |state: &str| -> String {
|
||||
match ways_in(state).first() {
|
||||
Some((from, to)) => format!(" - fix: {}", fix_for(from, to)),
|
||||
None => format!(
|
||||
" - fix: no state a record reaches has a declared edge into it; add one in aggregates.yaml under {:?}",
|
||||
need.lands_in
|
||||
),
|
||||
}
|
||||
};
|
||||
for state in &need.done_when {
|
||||
if !schema.has_state(state) {
|
||||
report(
|
||||
Level::Fail,
|
||||
"done-unknown",
|
||||
state.clone(),
|
||||
format!("done_when names {state:?}, which {:?} has no such state", need.lands_in),
|
||||
);
|
||||
} else if !reached.contains(state) {
|
||||
unreached_reported.insert((need.lands_in.clone(), state.clone()));
|
||||
report(
|
||||
Level::Fail,
|
||||
"done-unreachable",
|
||||
state.clone(),
|
||||
format!(
|
||||
"{:?} cannot move a record to {state:?} - no desk of theirs offers the moves",
|
||||
need.handled_by
|
||||
"{:?} cannot move a record to {state:?} - no desk of theirs offers the moves{}",
|
||||
need.handled_by,
|
||||
missing(state)
|
||||
),
|
||||
);
|
||||
}
|
||||
@@ -491,17 +647,48 @@ pub fn check(
|
||||
if !schema.has_state(stage) {
|
||||
report(
|
||||
Level::Fail,
|
||||
"stage-unknown",
|
||||
stage.clone(),
|
||||
format!(
|
||||
"the business describes a stage {stage:?} that {:?} has no state for",
|
||||
need.lands_in
|
||||
),
|
||||
);
|
||||
} else if !reached.contains(stage) {
|
||||
unreached_reported.insert((need.lands_in.clone(), stage.clone()));
|
||||
report(
|
||||
Level::Fail,
|
||||
"stage-unreachable",
|
||||
stage.clone(),
|
||||
format!(
|
||||
"stage {stage:?} exists but {:?} is offered no moves that reach it",
|
||||
need.handled_by
|
||||
"stage {stage:?} exists but {:?} is offered no moves that reach it{}",
|
||||
need.handled_by,
|
||||
missing(stage)
|
||||
),
|
||||
);
|
||||
}
|
||||
}
|
||||
// In the business's order: each working stage reachable from
|
||||
// the one before it, so a machine where "quoted" cannot follow
|
||||
// "surveyed" does not pass for one that goes surveyed -> quoted.
|
||||
// The finished states that close the list are alternatives
|
||||
// (won, lost), not a sequence, so they are left out.
|
||||
for pair in need.stages.windows(2) {
|
||||
let (before, after) = (&pair[0], &pair[1]);
|
||||
if done.contains(before.as_str()) || done.contains(after.as_str()) {
|
||||
continue;
|
||||
}
|
||||
if reached.contains(before) && reached.contains(after) && !reachable(before, &moves).contains(after) {
|
||||
report(
|
||||
Level::Fail,
|
||||
"stage-order",
|
||||
format!("{before}>{after}"),
|
||||
format!(
|
||||
"stage {after:?} comes after {before:?} for the business, but no move leads on from {before:?} to it{}",
|
||||
match schema.allowed(before).iter().find(|to| reachable(to, &moves).contains(after) || *to == after) {
|
||||
Some(to) => format!(" - fix: {}", fix_for(before, to)),
|
||||
None => format!(" - fix: add an edge from {before:?} toward {after:?} in aggregates.yaml under {:?}", need.lands_in),
|
||||
}
|
||||
),
|
||||
);
|
||||
}
|
||||
@@ -510,27 +697,43 @@ pub fn check(
|
||||
if !need.done_when.contains(state) {
|
||||
report(
|
||||
Level::Fail,
|
||||
"success-not-done",
|
||||
state.clone(),
|
||||
format!("success names {state:?}, which is not one of done_when"),
|
||||
);
|
||||
}
|
||||
}
|
||||
stranded.sort();
|
||||
for state in &stranded {
|
||||
strands_reported.insert((need.lands_in.clone(), state.clone()));
|
||||
}
|
||||
for state in stranded {
|
||||
let fix = match schema.allowed(&state).first() {
|
||||
Some(to) => format!(" - fix: {}", fix_for(&state, to)),
|
||||
None => format!(
|
||||
" - fix: {:?} declares no way out of it; add {state:?} to done_when if it is finished, or an edge out in aggregates.yaml",
|
||||
need.lands_in
|
||||
),
|
||||
};
|
||||
report(
|
||||
Level::Fail,
|
||||
format!(
|
||||
"records strand in {state:?}: {:?} is offered no move out of it",
|
||||
need.handled_by
|
||||
),
|
||||
"strand",
|
||||
state.clone(),
|
||||
format!("records strand in {state:?}: {movers} is offered no move out of it{fix}"),
|
||||
);
|
||||
}
|
||||
}
|
||||
|
||||
findings.extend(operations(file, questions, aggregates, &strands_reported, &unreached_reported));
|
||||
|
||||
let need_ids: HashSet<&str> = file.needs.iter().map(|n| n.id.as_str()).collect();
|
||||
for persona in &file.personas {
|
||||
if !need_ids.contains(persona.need.as_str()) {
|
||||
findings.push(Finding {
|
||||
need: persona.need.clone(),
|
||||
scope: "need",
|
||||
code: "persona-unknown-need",
|
||||
subject: persona.id.clone(),
|
||||
level: Level::Fail,
|
||||
message: format!("persona {:?} points at a need that is not declared", persona.id),
|
||||
});
|
||||
@@ -539,6 +742,200 @@ pub fn check(
|
||||
findings
|
||||
}
|
||||
|
||||
/// What must hold for every record in a bucket some need lands in,
|
||||
/// whichever need brought it: every state is somewhere a record can
|
||||
/// get to, no state a record gets to holds it with no way on (before
|
||||
/// done or after), every declared move is a button someone has, every
|
||||
/// desk can be seen by whoever its buttons are for, and every mail
|
||||
/// goes out from a state records reach, saying only what the record
|
||||
/// holds.
|
||||
fn operations(
|
||||
file: &NeedsFile,
|
||||
questions: &Questions,
|
||||
aggregates: &HashMap<String, AggregateSchema>,
|
||||
strands_reported: &HashSet<(String, String)>,
|
||||
unreached_reported: &HashSet<(String, String)>,
|
||||
) -> Vec<Finding> {
|
||||
let mut findings = Vec::new();
|
||||
let mut buckets: Vec<&str> = file.needs.iter().map(|n| n.lands_in.as_str()).collect();
|
||||
buckets.sort();
|
||||
buckets.dedup();
|
||||
for bucket in buckets {
|
||||
let Some(schema) = aggregates.get(bucket) else { continue };
|
||||
let needs: Vec<&Need> = file.needs.iter().filter(|n| n.lands_in == bucket).collect();
|
||||
// Only open needs make a gap fail; a bucket that only planned
|
||||
// or paused needs use reports it as a warning.
|
||||
let open = needs.iter().any(|n| !n.planned && !n.paused);
|
||||
let mut report = |level: Level, code: &'static str, subject: String, message: String| {
|
||||
findings.push(Finding {
|
||||
need: bucket.to_string(),
|
||||
scope: "bucket",
|
||||
code,
|
||||
subject,
|
||||
level: if open { level } else { Level::Warn },
|
||||
message,
|
||||
})
|
||||
};
|
||||
let desks = desks_over(questions, bucket);
|
||||
let mut moves: Vec<(String, String)> = desks
|
||||
.iter()
|
||||
.flat_map(|d| &d.moves)
|
||||
.filter(|(from, to, _)| schema.allowed(from).contains(to))
|
||||
.map(|(from, to, _)| (from.clone(), to.clone()))
|
||||
.collect();
|
||||
moves.extend(self_moves(questions, bucket, &schema.initial));
|
||||
moves.extend(clock_moves(schema));
|
||||
let reached = reachable(&schema.initial, &moves);
|
||||
let offered = |from: &str, to: &str| moves.iter().any(|(f, t)| f == from && t == to);
|
||||
// Where a missing button goes: the desk of the group answerable
|
||||
// for the bucket's first need.
|
||||
let home = needs
|
||||
.iter()
|
||||
.find_map(|n| desks.iter().find(|d| d.group.as_deref() == Some(n.handled_by.as_str())))
|
||||
.map(|d| d.page.clone())
|
||||
.unwrap_or_else(|| "the handling group's desk".to_string());
|
||||
let button = |from: &str, to: &str| format!("add `{{ from: {from}, to: {to}, label: ... }}` to the transitions on {home}");
|
||||
|
||||
let mut states: Vec<&String> = schema.event_for_state.keys().collect();
|
||||
states.sort();
|
||||
for state in &states {
|
||||
if reached.contains(*state) || unreached_reported.contains(&(bucket.to_string(), (*state).clone())) {
|
||||
continue;
|
||||
}
|
||||
let mut ways: Vec<&String> = reached.iter().filter(|from| schema.allowed(from).contains(state)).collect();
|
||||
ways.sort();
|
||||
report(
|
||||
Level::Fail,
|
||||
"state-unreached",
|
||||
(*state).clone(),
|
||||
format!(
|
||||
"state {state:?} is never reached: no desk, link or clock moves a record there - fix: {}",
|
||||
match ways.first() {
|
||||
Some(from) => button(from, state),
|
||||
None => "declare an edge into it from a state records reach in aggregates.yaml, or remove it".to_string(),
|
||||
}
|
||||
),
|
||||
);
|
||||
}
|
||||
for state in states.iter().filter(|s| reached.contains(**s)) {
|
||||
let out = schema.allowed(state);
|
||||
if out.is_empty() {
|
||||
continue; // a final state: nothing should move it
|
||||
}
|
||||
let offered_out: Vec<&String> = out.iter().filter(|to| offered(state, to)).collect();
|
||||
if offered_out.is_empty() {
|
||||
if !strands_reported.contains(&(bucket.to_string(), (*state).clone())) {
|
||||
report(
|
||||
Level::Fail,
|
||||
"strand",
|
||||
(*state).clone(),
|
||||
format!(
|
||||
"records strand in {state:?}: {bucket:?} declares moves on to {} but no desk, link or clock offers one - fix: {}",
|
||||
out.join(", "),
|
||||
button(state, &out[0])
|
||||
),
|
||||
);
|
||||
}
|
||||
continue;
|
||||
}
|
||||
for to in out.iter().filter(|to| !offered(state, to)) {
|
||||
report(
|
||||
Level::Warn,
|
||||
"edge-unoffered",
|
||||
format!("{state}>{to}"),
|
||||
format!(
|
||||
"the move {state:?} -> {to:?} is declared but no desk, link or clock offers it - fix: {}, or drop the edge from aggregates.yaml",
|
||||
button(state, to)
|
||||
),
|
||||
);
|
||||
}
|
||||
}
|
||||
for desk in &desks {
|
||||
if let (Some(page_group), Some(group)) = (&desk.page_group, &desk.group) {
|
||||
if page_group != group {
|
||||
report(
|
||||
Level::Warn,
|
||||
"desk-gated-twice",
|
||||
desk.page.clone(),
|
||||
format!(
|
||||
"the desk on {} is gated twice: the page qualifies {page_group:?}, its list requires {group:?}, so only members of both see the buttons - fix: make them the same group",
|
||||
desk.page
|
||||
),
|
||||
);
|
||||
}
|
||||
}
|
||||
}
|
||||
// The fields a record of this bucket holds: everything asked on
|
||||
// a form that records into it.
|
||||
let fields: Vec<(&str, &str)> = questions
|
||||
.values()
|
||||
.flat_map(|q| &q.alternatives)
|
||||
.filter(|a| a.record_as.as_deref() == Some(bucket))
|
||||
.flat_map(|a| &a.features)
|
||||
.flat_map(|f| &f.requirements)
|
||||
.map(|r| (r.name.as_str(), r.kind.as_str()))
|
||||
.collect();
|
||||
let mut mail_states: Vec<(&String, &crate::aggregates::MailSpec)> = schema.mail_for_state.iter().collect();
|
||||
mail_states.sort_by(|a, b| a.0.cmp(b.0));
|
||||
for (state, mail) in mail_states {
|
||||
if !reached.contains(state) {
|
||||
report(Level::Warn, "mail-unreachable", state.clone(), format!("the mail on entering {state:?} never goes out: no record reaches {state:?} - fix: make {state:?} reachable, or drop its mail"));
|
||||
}
|
||||
let mut unknown: Vec<String> = Vec::new();
|
||||
for key in placeholders(&mail.subject).into_iter().chain(placeholders(&mail.body)) {
|
||||
let known = match key.split_once('.') {
|
||||
Some((parent, _)) => fields.iter().any(|(name, kind)| *name == parent && *kind == "record"),
|
||||
None => MAIL_BUILTINS.contains(&key.as_str()) || fields.iter().any(|(name, _)| *name == key),
|
||||
};
|
||||
if !known && !unknown.contains(&key) {
|
||||
unknown.push(key);
|
||||
}
|
||||
}
|
||||
if !unknown.is_empty() {
|
||||
let mut names: Vec<&str> = fields.iter().map(|(n, _)| *n).collect();
|
||||
names.sort();
|
||||
names.dedup();
|
||||
report(
|
||||
Level::Fail,
|
||||
"mail-unknown-field",
|
||||
state.clone(),
|
||||
format!(
|
||||
"the mail on entering {state:?} uses {}, which no form recording into {bucket:?} collects, so it would go out blank there - fix: use one of the fields the forms collect ({}), or ask for it on the form",
|
||||
unknown.iter().map(|k| format!("{{{k}}}")).collect::<Vec<_>>().join(", "),
|
||||
if names.is_empty() { "nothing".to_string() } else { names.join(", ") }
|
||||
),
|
||||
);
|
||||
}
|
||||
}
|
||||
}
|
||||
findings
|
||||
}
|
||||
|
||||
/// What the mailer fills in on every state mail, whatever the record
|
||||
/// holds (`mail::vars_for`, `mail::site_vars`, `mail::record_links`).
|
||||
const MAIL_BUILTINS: &[&str] = &["site", "site_name", "chain", "email", "bucket", "state", "link", "self_link", "desk_link"];
|
||||
|
||||
/// The `{key}`s in a mail template, read the way `mail::render` reads
|
||||
/// them: letters, digits, `_`, `-` and `.`, closed by `}`.
|
||||
fn placeholders(template: &str) -> Vec<String> {
|
||||
let mut keys = Vec::new();
|
||||
let mut rest = template;
|
||||
while let Some(start) = rest.find('{') {
|
||||
let after = &rest[start + 1..];
|
||||
let len = after
|
||||
.char_indices()
|
||||
.take_while(|(_, c)| c.is_ascii_alphanumeric() || *c == '_' || *c == '-' || *c == '.')
|
||||
.last()
|
||||
.map(|(i, c)| i + c.len_utf8())
|
||||
.unwrap_or(0);
|
||||
if len > 0 && after[len..].starts_with('}') {
|
||||
keys.push(after[..len].to_string());
|
||||
}
|
||||
rest = after;
|
||||
}
|
||||
keys
|
||||
}
|
||||
|
||||
/// Everything a simulation of the business needs, resolved: each
|
||||
/// need's path with the choice on every page and the form that gets
|
||||
/// filled, each bucket's state machine with the buttons every desk
|
||||
@@ -611,6 +1008,11 @@ pub fn sim_model(
|
||||
"steps": steps,
|
||||
}));
|
||||
if let Some(schema) = aggregates.get(&need.lands_in) {
|
||||
// The simulator's shape: the states a visitor's own link
|
||||
// moves to (it does not model where from).
|
||||
let mut self_move_targets: Vec<String> =
|
||||
self_moves(questions, &need.lands_in, &schema.initial).into_iter().map(|(_, to)| to).collect();
|
||||
self_move_targets.dedup();
|
||||
buckets.entry(need.lands_in.clone()).or_insert_with(|| {
|
||||
json!({
|
||||
"initial": schema.initial,
|
||||
@@ -623,7 +1025,7 @@ pub fn sim_model(
|
||||
"from": from, "to": to, "label": label,
|
||||
})).collect::<Vec<_>>(),
|
||||
})).collect::<Vec<_>>(),
|
||||
"self_moves": self_moves(questions, &need.lands_in),
|
||||
"self_moves": self_move_targets,
|
||||
})
|
||||
});
|
||||
}
|
||||
@@ -895,8 +1297,8 @@ mod tests {
|
||||
assert_eq!(
|
||||
fails(&findings),
|
||||
vec![
|
||||
"\"owners\" cannot move a record to \"won\" - no desk of theirs offers the moves",
|
||||
"records strand in \"quoted\": \"owners\" is offered no move out of it",
|
||||
"\"owners\" cannot move a record to \"won\" - no desk of theirs offers the moves - fix: add `{ from: quoted, to: won, label: ... }` to the transitions on /desk",
|
||||
"records strand in \"quoted\": \"owners\" is offered no move out of it - fix: add `{ from: quoted, to: won, label: ... }` to the transitions on /desk",
|
||||
]
|
||||
);
|
||||
}
|
||||
@@ -1037,4 +1439,123 @@ mod tests {
|
||||
|
||||
assert_eq!(score(&tasks, &[], 0.6).unanswered, vec!["founder@/"]);
|
||||
}
|
||||
|
||||
/// The fixture site with its aggregates replaced, for the checks
|
||||
/// that are about the state machine rather than the pages.
|
||||
fn site_with(desk_moves: &str, projects: &str) -> (Questions, HashMap<String, AggregateSchema>) {
|
||||
let (questions, _) = site(desk_moves);
|
||||
let aggregates = parse_aggregates_yaml(&format!(
|
||||
"aggregates:\n - bucket: projects\n{projects} - bucket: subscribers\n attended_by: a mailer\n initial: open\n states:\n open: {{ event: subscribed }}\n transitions: {{}}\n"
|
||||
))
|
||||
.unwrap();
|
||||
(questions, aggregates)
|
||||
}
|
||||
|
||||
fn warns(findings: &[Finding]) -> Vec<&str> {
|
||||
findings.iter().filter(|f| f.level == Level::Warn).map(|f| f.message.as_str()).collect()
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn a_clock_is_a_move() {
|
||||
// Nobody closes a stale quote; six weeks do. That is not a strand.
|
||||
let (questions, aggregates) = site_with(
|
||||
" - { to: quoted, label: Quote }\n - { from: quoted, to: won, label: Win }\n",
|
||||
" initial: open\n states:\n open: { event: submitted }\n quoted: { event: quoted, after: { days: 42, to: lost } }\n won: { event: won }\n lost: { event: lost }\n transitions:\n open: [quoted]\n quoted: [won, lost]\n",
|
||||
);
|
||||
let findings = check(&needs(""), &questions, &aggregates);
|
||||
assert!(fails(&findings).is_empty(), "{findings:?}");
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn a_state_nobody_reaches_and_a_strand_after_done_are_the_buckets() {
|
||||
// "archived" follows "won" but no desk offers it; "parked" has
|
||||
// no way in at all.
|
||||
let (questions, aggregates) = site_with(
|
||||
FULL_DESK,
|
||||
" initial: open\n states:\n open: { event: submitted }\n quoted: { event: quoted }\n won: { event: won }\n lost: { event: lost }\n archived: { event: archived }\n done: { event: done }\n parked: { event: parked }\n transitions:\n open: [quoted, lost]\n quoted: [won, lost]\n won: [archived]\n archived: [done]\n parked: [open]\n",
|
||||
);
|
||||
let findings = check(&needs(""), &questions, &aggregates);
|
||||
let bucket: Vec<&Finding> = findings.iter().filter(|f| f.scope == "bucket").collect();
|
||||
assert!(bucket.iter().all(|f| f.need == "projects"), "{bucket:?}");
|
||||
assert_eq!(
|
||||
fails(&findings),
|
||||
vec![
|
||||
"state \"archived\" is never reached: no desk, link or clock moves a record there - fix: add `{ from: won, to: archived, label: ... }` to the transitions on /desk",
|
||||
"state \"done\" is never reached: no desk, link or clock moves a record there - fix: declare an edge into it from a state records reach in aggregates.yaml, or remove it",
|
||||
"state \"parked\" is never reached: no desk, link or clock moves a record there - fix: declare an edge into it from a state records reach in aggregates.yaml, or remove it",
|
||||
"records strand in \"won\": \"projects\" declares moves on to archived but no desk, link or clock offers one - fix: add `{ from: won, to: archived, label: ... }` to the transitions on /desk",
|
||||
]
|
||||
);
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn an_edge_nobody_offers_is_a_warning_when_there_is_another_way_on() {
|
||||
let (questions, aggregates) = site(" - { to: quoted, label: Quote }\n - { from: quoted, to: won, label: Win }\n - { from: quoted, to: lost, label: Lose }\n");
|
||||
let findings = check(&needs(""), &questions, &aggregates);
|
||||
assert!(fails(&findings).is_empty(), "{findings:?}");
|
||||
assert_eq!(
|
||||
warns(&findings),
|
||||
vec!["the move \"open\" -> \"lost\" is declared but no desk, link or clock offers it - fix: add `{ from: open, to: lost, label: ... }` to the transitions on /desk, or drop the edge from aggregates.yaml"]
|
||||
);
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn working_stages_must_follow_each_other() {
|
||||
// The business surveys, then quotes; this machine can only
|
||||
// quote straight from open, and surveying leads nowhere.
|
||||
let (questions, aggregates) = site_with(
|
||||
" - { to: quoted, label: Quote }\n - { to: surveyed, label: Survey }\n - { from: surveyed, to: lost, label: Lose }\n - { from: quoted, to: won, label: Win }\n - { from: quoted, to: lost, label: Lose }\n",
|
||||
" initial: open\n states:\n open: { event: submitted }\n surveyed: { event: surveyed }\n quoted: { event: quoted }\n won: { event: won }\n lost: { event: lost }\n transitions:\n open: [surveyed, quoted]\n surveyed: [lost]\n quoted: [won, lost]\n",
|
||||
);
|
||||
let findings = check(&needs(" stages: [open, surveyed, quoted, won, lost]\n"), &questions, &aggregates);
|
||||
assert_eq!(
|
||||
fails(&findings),
|
||||
vec!["stage \"quoted\" comes after \"surveyed\" for the business, but no move leads on from \"surveyed\" to it - fix: add an edge from \"surveyed\" toward \"quoted\" in aggregates.yaml under \"projects\""]
|
||||
);
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn a_mail_may_only_say_what_the_record_holds() {
|
||||
let (questions, aggregates) = site_with(
|
||||
FULL_DESK,
|
||||
" initial: open\n states:\n open: { event: submitted, mail: { to: submitter, subject: 'Got it, {email}', body: 'We read {brief} and will reply at {site}. {refnr} {link}' } }\n quoted: { event: quoted }\n won: { event: won }\n lost: { event: lost }\n transitions:\n open: [quoted, lost]\n quoted: [won, lost]\n",
|
||||
);
|
||||
let findings = check(&needs(""), &questions, &aggregates);
|
||||
assert_eq!(
|
||||
fails(&findings),
|
||||
vec!["the mail on entering \"open\" uses {refnr}, which no form recording into \"projects\" collects, so it would go out blank there - fix: use one of the fields the forms collect (brief, budget, email), or ask for it on the form"]
|
||||
);
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn a_desk_gated_twice_is_seen_only_by_members_of_both() {
|
||||
let (mut questions, aggregates) = site(FULL_DESK);
|
||||
questions.get_mut("/desk").unwrap().qualifies = Some("staff".into());
|
||||
let findings = check(&needs(""), &questions, &aggregates);
|
||||
assert!(
|
||||
warns(&findings).contains(&"the desk on /desk is gated twice: the page qualifies \"staff\", its list requires \"owners\", so only members of both see the buttons - fix: make them the same group"),
|
||||
"{findings:?}"
|
||||
);
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn a_finding_keeps_its_id_when_reworded_and_says_its_fix() {
|
||||
let (questions, aggregates) = site(" - { to: quoted, label: Quote }\n - { to: lost, label: Lose }\n");
|
||||
let findings = check(&needs(""), &questions, &aggregates);
|
||||
let strand = findings.iter().find(|f| f.code == "strand").unwrap();
|
||||
assert_eq!(strand.subject, "quoted");
|
||||
assert_eq!(strand.fix(), Some("add `{ from: quoted, to: won, label: ... }` to the transitions on /desk"));
|
||||
let reworded = Finding { message: "something else entirely".into(), ..clone_of(strand) };
|
||||
assert_eq!(reworded.id(), strand.id());
|
||||
let elsewhere = Finding { subject: "open".into(), ..clone_of(strand) };
|
||||
assert_ne!(elsewhere.id(), strand.id());
|
||||
let json = strand.to_json();
|
||||
assert_eq!(json["code"], "strand");
|
||||
assert_eq!(json["level"], "fail");
|
||||
assert_eq!(json["id"], strand.id());
|
||||
}
|
||||
|
||||
fn clone_of(f: &Finding) -> Finding {
|
||||
Finding { need: f.need.clone(), level: f.level, message: f.message.clone(), scope: f.scope, code: f.code, subject: f.subject.clone() }
|
||||
}
|
||||
}
|
||||
|
||||
+109
-5
@@ -251,6 +251,80 @@ pub fn vars_for(bucket: &str, id: &str, state: &str, responses: &serde_json::Val
|
||||
vars
|
||||
}
|
||||
|
||||
/// What every mail portal sends may say, whatever made it - a state
|
||||
/// mail, an invite, a suggestion's answer: `{site}` (the instance's
|
||||
/// public URL) and `{site_name}` (site.yaml's title). One place, so a
|
||||
/// template written for one kind of mail reads the same in another
|
||||
/// (docs/mail.md, Variables).
|
||||
pub fn site_vars(site: &crate::content::SiteConfig) -> serde_json::Map<String, serde_json::Value> {
|
||||
let mut vars = serde_json::Map::new();
|
||||
vars.insert("site".into(), public_url().into());
|
||||
vars.insert("site_name".into(), site_name(site).into());
|
||||
vars
|
||||
}
|
||||
|
||||
/// Where a mail about a record can send its reader (docs/mail.md):
|
||||
///
|
||||
/// - `{self_link}`: the record's own page for the person it belongs
|
||||
/// to - a page offering a self-service move on this bucket from the
|
||||
/// state the record is in, with the `?chain=&email=` that
|
||||
/// lets them act without signing in;
|
||||
/// - `{desk_link}`: the desk that moves the record - the page whose
|
||||
/// list over the bucket carries transitions, else one that reads it;
|
||||
/// - `{link}`: `{self_link}` in a mail to the submitter, `{desk_link}`
|
||||
/// in one to anyone else, and the site itself when there is neither.
|
||||
///
|
||||
/// Empty strings, never missing keys, for a link the content has no
|
||||
/// page for: a template naming one renders blank there rather than
|
||||
/// with a broken URL.
|
||||
pub fn record_links(
|
||||
questions: &std::collections::HashMap<String, crate::content::Question>,
|
||||
initial: &str,
|
||||
bucket: &str,
|
||||
id: &str,
|
||||
state: &str,
|
||||
email: &str,
|
||||
to_submitter: bool,
|
||||
) -> serde_json::Map<String, serde_json::Value> {
|
||||
let mut pages: Vec<&crate::content::Question> = questions.values().collect();
|
||||
pages.sort_by(|a, b| a.id.cmp(&b.id));
|
||||
// Only a move the engine would accept from where the record is:
|
||||
// a link to a button that refuses is worse than none.
|
||||
let self_page = pages.iter().find(|q| {
|
||||
q.alternatives
|
||||
.iter()
|
||||
.filter_map(|a| a.self_transition.as_ref())
|
||||
.any(|st| st.bucket == bucket && st.from_states(initial).iter().any(|f| f == state))
|
||||
});
|
||||
let reads = |q: &&crate::content::Question, moving: bool| {
|
||||
q.alternatives
|
||||
.iter()
|
||||
.flat_map(|a| &a.features)
|
||||
.filter_map(|f| f.resource.as_ref())
|
||||
.any(|r| r.bucket() == Some(bucket) && (!moving || !r.transitions.is_empty()))
|
||||
};
|
||||
let desk_page = pages.iter().find(|q| reads(q, true)).or_else(|| pages.iter().find(|q| reads(q, false)));
|
||||
let site = public_url();
|
||||
let self_link = match self_page {
|
||||
Some(q) if email.contains('@') => {
|
||||
let email: String = url::form_urlencoded::byte_serialize(email.as_bytes()).collect();
|
||||
format!("{site}{}?chain={id}&email={email}", q.id)
|
||||
}
|
||||
_ => String::new(),
|
||||
};
|
||||
let desk_link = desk_page.map(|q| format!("{site}{}", q.id)).unwrap_or_default();
|
||||
let link = match (to_submitter, self_link.is_empty(), desk_link.is_empty()) {
|
||||
(true, false, _) => self_link.clone(),
|
||||
(false, _, false) => desk_link.clone(),
|
||||
_ => site,
|
||||
};
|
||||
let mut vars = serde_json::Map::new();
|
||||
vars.insert("self_link".into(), self_link.into());
|
||||
vars.insert("desk_link".into(), desk_link.into());
|
||||
vars.insert("link".into(), link.into());
|
||||
vars
|
||||
}
|
||||
|
||||
/// The record field a `to` reaches through, when it is neither
|
||||
/// `submitter` nor an address: `{notice.email}` gives `notice`.
|
||||
pub fn parent_of(to: &str) -> Option<&str> {
|
||||
@@ -325,6 +399,13 @@ pub async fn on_state(state: &crate::server::AppState, bucket: &str, id: &str, t
|
||||
};
|
||||
let mut vars = vars_for(bucket, id, to_state, responses);
|
||||
parent_vars(state, bucket, responses, &mut vars).await;
|
||||
vars.extend(site_vars(site));
|
||||
{
|
||||
let email = vars.get("email").and_then(|v| v.as_str()).unwrap_or_default().to_string();
|
||||
let questions = state.questions.load();
|
||||
let initial = state.aggregates.load().at(storyline).get(bucket).map(|s| s.initial.clone()).unwrap_or_default();
|
||||
vars.extend(record_links(questions.at(storyline), &initial, bucket, id, to_state, &email, spec.to_submitter()));
|
||||
}
|
||||
let to = if spec.to_submitter() {
|
||||
match vars.get("email").and_then(|v| v.as_str()).filter(|e| e.contains('@')) {
|
||||
Some(e) => e.to_string(),
|
||||
@@ -427,9 +508,8 @@ pub async fn on_responsible(state: &crate::server::AppState, invited: &crate::pe
|
||||
vars.insert("name".into(), invited.display_name.clone().into());
|
||||
vars.insert("username".into(), invited.username.clone().into());
|
||||
vars.insert("questions".into(), list.into());
|
||||
vars.extend(site_vars(&site));
|
||||
vars.insert("link".into(), invited.reset_link.clone().unwrap_or_default().into());
|
||||
vars.insert("site".into(), public_url().into());
|
||||
vars.insert("site_name".into(), site_name(&site).into());
|
||||
vars.insert("expires".into(), expires.into());
|
||||
let text = render(&body, &vars);
|
||||
let mail = Outgoing {
|
||||
@@ -501,7 +581,7 @@ async fn on_suggestion_state(state: &crate::server::AppState, id: &str, to_state
|
||||
return;
|
||||
}
|
||||
let quoted: String = field("text").lines().map(|l| format!("> {l}")).collect::<Vec<_>>().join("\n");
|
||||
let mut vars = serde_json::Map::new();
|
||||
let mut vars = site_vars(&site);
|
||||
vars.insert("name".into(), field("name").into());
|
||||
vars.insert("question".into(), field("question").into());
|
||||
vars.insert("page".into(), format!("{}{}", public_url(), field("page")).into());
|
||||
@@ -617,9 +697,8 @@ pub async fn on_invite(state: &crate::server::AppState, invited: &crate::people:
|
||||
vars.insert("username".into(), invited.username.clone().into());
|
||||
vars.insert("email".into(), invited.email.clone().into());
|
||||
vars.insert("group".into(), invited.group.clone().into());
|
||||
vars.extend(site_vars(&site));
|
||||
vars.insert("link".into(), invited.reset_link.clone().unwrap_or_default().into());
|
||||
vars.insert("site".into(), public_url().into());
|
||||
vars.insert("site_name".into(), site_name(&site).into());
|
||||
vars.insert("expires".into(), expires.into());
|
||||
let text = render(&body, &vars);
|
||||
let mail = Outgoing {
|
||||
@@ -640,6 +719,30 @@ pub async fn on_invite(state: &crate::server::AppState, invited: &crate::people:
|
||||
|
||||
#[cfg(test)]
|
||||
mod tests {
|
||||
|
||||
#[test]
|
||||
fn a_record_mail_links_its_reader_to_where_they_act() {
|
||||
let files = vec![
|
||||
("index.yaml".to_string(), "name: Meld fra\nalternatives:\n - name: Feil\n action: /takk\n record_as: feil\n features:\n - name: Feil\n requirements:\n - { name: email, type: email }\n".to_string()),
|
||||
("takk.yaml".to_string(), "name: Takk\nfollowup: true\n".to_string()),
|
||||
("trekk.yaml".to_string(), "name: Trekk saken\nalternatives:\n - name: Trekk\n self_transition: { bucket: feil, to: trukket, label: Trekk, from: [open, befaring] }\n".to_string()),
|
||||
("styret/index.yaml".to_string(), "name: Styret\nqualifies: styret\nalternatives:\n - name: Feil\n features:\n - name: \"\"\n resource:\n source: { kind: kv, bucket: feil }\n requires_group: styret\n transitions:\n - { to: befaring, label: Befaring }\n".to_string()),
|
||||
];
|
||||
let questions = crate::content::build_questions(&files).unwrap();
|
||||
let site = public_url();
|
||||
let to_tenant = record_links(&questions, "open", "feil", "abc", "befaring", "kari@x.no", true);
|
||||
assert_eq!(to_tenant["self_link"], format!("{site}/trekk?chain=abc&email=kari%40x.no"));
|
||||
assert_eq!(to_tenant["desk_link"], format!("{site}/styret"));
|
||||
assert_eq!(to_tenant["link"], to_tenant["self_link"]);
|
||||
let to_board = record_links(&questions, "open", "feil", "abc", "befaring", "kari@x.no", false);
|
||||
assert_eq!(to_board["link"], to_board["desk_link"]);
|
||||
// Past where the self-service move can be made: the person has
|
||||
// no page of their own, so `{link}` is the site.
|
||||
let after = record_links(&questions, "open", "feil", "abc", "utbedret", "kari@x.no", true);
|
||||
assert_eq!(after["self_link"], "");
|
||||
assert_eq!(after["link"], site);
|
||||
}
|
||||
|
||||
use super::*;
|
||||
|
||||
#[test]
|
||||
@@ -743,6 +846,7 @@ mod tests {
|
||||
decided_ms: None,
|
||||
decided_by: None,
|
||||
notes: Vec::new(),
|
||||
finished: false,
|
||||
}
|
||||
}
|
||||
|
||||
|
||||
+49
-19
@@ -27,7 +27,14 @@ async fn main() -> anyhow::Result<()> {
|
||||
|
||||
#[cfg(feature = "ssr")]
|
||||
async fn serve() -> anyhow::Result<()> {
|
||||
use axum::{body::Body, extract::State, http::Request, response::IntoResponse, routing::{any, get, post}, Router};
|
||||
use axum::{
|
||||
body::Body,
|
||||
extract::State,
|
||||
http::Request,
|
||||
response::IntoResponse,
|
||||
routing::{any, get, post},
|
||||
Router,
|
||||
};
|
||||
use leptos::prelude::*;
|
||||
use leptos_axum::{generate_route_list, LeptosRoutes};
|
||||
use portal::app::{shell, App};
|
||||
@@ -57,12 +64,15 @@ async fn serve() -> anyhow::Result<()> {
|
||||
let gitea_base = content::gitea_api_base(&content_repo)?;
|
||||
let content_raw_base = content::gitea_raw_base(&content_repo)?;
|
||||
let aggregates = content::load_aggregates_from_gitea(&content_repo, &content_branch).await?;
|
||||
let questions = content::load_questions_from_gitea(&content_repo, &content_branch, "questions").await?;
|
||||
let questions =
|
||||
content::load_questions_from_gitea(&content_repo, &content_branch, "questions").await?;
|
||||
content::validate_questions(&questions, &aggregates)?;
|
||||
let site = content::load_site_from_gitea(&content_repo, &content_branch).await?;
|
||||
let storylines = content::same_arity(questions.arity(), aggregates.arity(), site.arity())?;
|
||||
tracing::info!(count = questions.len(), storylines, repo = %content_repo, branch = %content_branch, "loaded content");
|
||||
let access = Arc::new(arc_swap::ArcSwap::from_pointee(portal::access::Policy::from_content(&questions, &aggregates)));
|
||||
let access = Arc::new(arc_swap::ArcSwap::from_pointee(
|
||||
portal::access::Policy::from_content(&questions, &aggregates),
|
||||
));
|
||||
let questions = Arc::new(arc_swap::ArcSwap::from_pointee(questions));
|
||||
let aggregates = Arc::new(arc_swap::ArcSwap::from_pointee(aggregates));
|
||||
let site = Arc::new(arc_swap::ArcSwap::from_pointee(site));
|
||||
@@ -128,7 +138,12 @@ async fn serve() -> anyhow::Result<()> {
|
||||
|
||||
// The same decisions, answerable by other apps over NATS, for a
|
||||
// person named by phone number or address as well as by groups.
|
||||
tokio::spawn(portal::access::serve(state.nats.clone(), site_name_for_subjects(), state.access.clone(), Some(state.clone())));
|
||||
tokio::spawn(portal::access::serve(
|
||||
state.nats.clone(),
|
||||
site_name_for_subjects(),
|
||||
state.access.clone(),
|
||||
Some(state.clone()),
|
||||
));
|
||||
// Announced pages: keep their desk records current, close ended
|
||||
// windows (see announce.rs).
|
||||
tokio::spawn(portal::announce::run(state.clone()));
|
||||
@@ -202,18 +217,17 @@ async fn serve() -> anyhow::Result<()> {
|
||||
});
|
||||
match &governor {
|
||||
Some(_) => tracing::info!(rate_per_second, rate_burst, "rate limiting per caller IP"),
|
||||
None => tracing::warn!("rate limiting is off (PORTAL_RATE_PER_SECOND or PORTAL_RATE_BURST is 0)"),
|
||||
None => tracing::warn!(
|
||||
"rate limiting is off (PORTAL_RATE_PER_SECOND or PORTAL_RATE_BURST is 0)"
|
||||
),
|
||||
}
|
||||
|
||||
async fn server_fn_handler(
|
||||
State(state): State<AppState>,
|
||||
request: Request<Body>,
|
||||
) -> impl IntoResponse {
|
||||
leptos_axum::handle_server_fns_with_context(
|
||||
move || provide_context(state.clone()),
|
||||
request,
|
||||
)
|
||||
.await
|
||||
leptos_axum::handle_server_fns_with_context(move || provide_context(state.clone()), request)
|
||||
.await
|
||||
}
|
||||
|
||||
// The app's own Leptos route list includes a root-level wildcard
|
||||
@@ -264,10 +278,7 @@ async fn serve() -> anyhow::Result<()> {
|
||||
.route_service("/favicon-light.svg", ServeFile::new(favicon_light_path))
|
||||
.route_service("/favicon-dark.svg", ServeFile::new(favicon_dark_path))
|
||||
.route_service("/wordmark.svg", ServeFile::new(wordmark_path))
|
||||
.route_service(
|
||||
"/swiper-element-bundle.min.js",
|
||||
ServeFile::new(swiper_path),
|
||||
)
|
||||
.route_service("/swiper-element-bundle.min.js", ServeFile::new(swiper_path))
|
||||
// Same name every release, so revalidated like /pkg.
|
||||
.route_service(
|
||||
"/page-motion.js",
|
||||
@@ -291,7 +302,9 @@ async fn serve() -> anyhow::Result<()> {
|
||||
// with room for the multipart framing.
|
||||
.route(
|
||||
"/upload",
|
||||
post(upload::upload).layer(axum::extract::DefaultBodyLimit::max(upload::MAX_UPLOAD_BYTES + 64 * 1024)),
|
||||
post(upload::upload).layer(axum::extract::DefaultBodyLimit::max(
|
||||
upload::MAX_UPLOAD_BYTES + 64 * 1024,
|
||||
)),
|
||||
)
|
||||
.route(
|
||||
"/attachment/{bucket}/{record}/{field}",
|
||||
@@ -302,7 +315,11 @@ async fn serve() -> anyhow::Result<()> {
|
||||
.route("/report", get(portal::report::handler))
|
||||
// Content-shipped assets (hero module etc.), same-origin.
|
||||
.route("/site/{*path}", get(content::site_asset_handler))
|
||||
.route("/automation/kv/{bucket}", get(content::automation_kv_handler))
|
||||
.route("/.well-known/{*path}", get(content::well_known_handler))
|
||||
.route(
|
||||
"/automation/kv/{bucket}",
|
||||
get(content::automation_kv_handler),
|
||||
)
|
||||
// Maps, proxied: the browser asks this origin for every tile
|
||||
// and every still, so nothing third-party loads in a page and
|
||||
// the token stays here (see maps.rs).
|
||||
@@ -324,7 +341,10 @@ async fn serve() -> anyhow::Result<()> {
|
||||
.fallback(leptos_axum::file_and_error_handler::<AppState, _>(shell))
|
||||
.layer(session_layer)
|
||||
// Which storyline this visitor reads, before anything renders.
|
||||
.layer(axum::middleware::from_fn_with_state(state.clone(), portal::server::storyline::assign));
|
||||
.layer(axum::middleware::from_fn_with_state(
|
||||
state.clone(),
|
||||
portal::server::storyline::assign,
|
||||
));
|
||||
// Outermost, so a caller who is over the limit is turned away
|
||||
// before a session is loaded or a handler is entered.
|
||||
let app = match governor {
|
||||
@@ -362,7 +382,17 @@ fn site_name_for_subjects() -> String {
|
||||
let raw = std::env::var("SITE_NAME").unwrap_or_else(|_| "portal".to_string());
|
||||
let folded: String = raw
|
||||
.chars()
|
||||
.map(|c| if c.is_ascii_alphanumeric() || c == '_' || c == '-' { c.to_ascii_lowercase() } else { '_' })
|
||||
.map(|c| {
|
||||
if c.is_ascii_alphanumeric() || c == '_' || c == '-' {
|
||||
c.to_ascii_lowercase()
|
||||
} else {
|
||||
'_'
|
||||
}
|
||||
})
|
||||
.collect();
|
||||
if folded.is_empty() { "portal".to_string() } else { folded }
|
||||
if folded.is_empty() {
|
||||
"portal".to_string()
|
||||
} else {
|
||||
folded
|
||||
}
|
||||
}
|
||||
|
||||
@@ -353,6 +353,7 @@ mod tests {
|
||||
decided_ms: None,
|
||||
decided_by: None,
|
||||
notes: Vec::new(),
|
||||
finished: false,
|
||||
}
|
||||
}
|
||||
|
||||
|
||||
@@ -722,6 +722,7 @@ mod tests {
|
||||
decided_ms: None,
|
||||
decided_by: None,
|
||||
notes: Vec::new(),
|
||||
finished: false,
|
||||
}
|
||||
}
|
||||
|
||||
|
||||
+137
-3
@@ -328,6 +328,11 @@ async fn fetch_resource_value(
|
||||
}
|
||||
}
|
||||
}
|
||||
if !resource.transitions.is_empty() {
|
||||
if let Some(schema) = state.aggregates.load().at(0).get(bucket) {
|
||||
items = cap_finished(items, schema, finished_wanted(params));
|
||||
}
|
||||
}
|
||||
serde_json::Value::Array(items)
|
||||
}
|
||||
}
|
||||
@@ -342,12 +347,67 @@ async fn fetch_resource_value(
|
||||
ResourceSource::Url { url } => fetch_url_json(url, params).await?,
|
||||
};
|
||||
|
||||
match &resource.jq {
|
||||
Some(filter) => apply_jq(filter, &value),
|
||||
None => Ok(value),
|
||||
let shaped = match &resource.jq {
|
||||
Some(filter) => apply_jq(filter, &value)?,
|
||||
None => value,
|
||||
};
|
||||
Ok(apply_limit(resource.limit, shaped))
|
||||
}
|
||||
|
||||
/// Keep at most `n` items, after `jq` has had its say.
|
||||
///
|
||||
/// Anything that is not a list passes through: a single record has no
|
||||
/// length to cap, and silently turning one into an empty list would be
|
||||
/// worse than ignoring a limit that cannot apply. A limit of zero does
|
||||
/// mean nothing — the resource's `empty` line speaks instead, which is
|
||||
/// a coherent thing for content to ask for.
|
||||
#[cfg(feature = "ssr")]
|
||||
fn apply_limit(limit: Option<usize>, value: serde_json::Value) -> serde_json::Value {
|
||||
let Some(n) = limit else { return value };
|
||||
match value {
|
||||
serde_json::Value::Array(mut items) => {
|
||||
items.truncate(n);
|
||||
serde_json::Value::Array(items)
|
||||
}
|
||||
other => other,
|
||||
}
|
||||
}
|
||||
|
||||
/// How many finished records a desk listing should show: three, or
|
||||
/// what the page asked for with `__finished` after "show older".
|
||||
pub const FINISHED_PARAM: &str = "__finished";
|
||||
pub const FINISHED_SHOWN: usize = 3;
|
||||
|
||||
#[cfg(feature = "ssr")]
|
||||
fn finished_wanted(params: &std::collections::HashMap<String, String>) -> usize {
|
||||
params.get(FINISHED_PARAM).and_then(|n| n.parse().ok()).unwrap_or(FINISHED_SHOWN)
|
||||
}
|
||||
|
||||
/// A desk is for the cases still moving. Every record still open stays;
|
||||
/// of those in a final state (no move out), only the `shown` most
|
||||
/// recently decided, plus one more so the page knows there are older
|
||||
/// ones to offer. Each finished record is marked, so the page can tell
|
||||
/// them apart without knowing the state machine.
|
||||
#[cfg(feature = "ssr")]
|
||||
pub(crate) fn cap_finished(items: Vec<serde_json::Value>, schema: &crate::aggregates::AggregateSchema, shown: usize) -> Vec<serde_json::Value> {
|
||||
let is_final = |v: &serde_json::Value| {
|
||||
v.get("state").and_then(|s| s.as_str()).is_some_and(|s| schema.has_state(s) && schema.allowed(s).is_empty())
|
||||
};
|
||||
let when = |v: &serde_json::Value| {
|
||||
v.get("decided_ms").and_then(|d| d.as_i64()).or_else(|| v.get("submitted_ms").and_then(|d| d.as_i64())).unwrap_or(0)
|
||||
};
|
||||
let (mut finished, mut open): (Vec<_>, Vec<_>) = items.into_iter().partition(is_final);
|
||||
finished.sort_by_key(|v| std::cmp::Reverse(when(v)));
|
||||
finished.truncate(shown + 1);
|
||||
for v in &mut finished {
|
||||
if let Some(obj) = v.as_object_mut() {
|
||||
obj.insert("finished".into(), true.into());
|
||||
}
|
||||
}
|
||||
open.extend(finished);
|
||||
open
|
||||
}
|
||||
|
||||
/// Fetches JSON from a path on the same Gitea instance content is
|
||||
/// loaded from (`AppState.gitea_base`), attaching `GITEA_API_TOKEN` if
|
||||
/// set - confirmed live against `project.uhhm.no` that at least the
|
||||
@@ -624,6 +684,51 @@ mod tests {
|
||||
assert!(!matches_filter(&fixed, &record("open", serde_json::json!({ "which_authority": "Fire service" })), ¶ms));
|
||||
}
|
||||
|
||||
/// `limit` caps what the page renders, and does it after `jq` so
|
||||
/// content can order with the filter and take the top of its own
|
||||
/// ordering. The releases card is the case: the newest cut is the
|
||||
/// one you should run, and a page that grows a card per release
|
||||
/// gets longer forever.
|
||||
#[test]
|
||||
fn limit_caps_a_list_after_the_filter() {
|
||||
let releases = serde_json::json!([
|
||||
{"name": "redoal 0.1.17", "tag_name": "v0.1.17", "body": "newest"},
|
||||
{"name": "redoal 0.1.16", "tag_name": "v0.1.16", "body": "older"},
|
||||
{"name": "redoal 0.1.15", "tag_name": "v0.1.15", "body": "oldest"}
|
||||
]);
|
||||
// The filter redoal/questions actually declares: bare objects,
|
||||
// never a wrapped array — the frontend re-wraps.
|
||||
let filter = ".[] | {name: .name, description: .body, tag: .tag_name, url: null}";
|
||||
let shaped = apply_jq(filter, &releases).expect("filter runs");
|
||||
|
||||
let all = apply_limit(None, shaped.clone());
|
||||
assert_eq!(all.as_array().map(Vec::len), Some(3), "no limit shows everything");
|
||||
|
||||
let latest = apply_limit(Some(1), shaped.clone());
|
||||
let rows = latest.as_array().expect("still a list");
|
||||
assert_eq!(rows.len(), 1);
|
||||
assert_eq!(rows[0]["tag"], "v0.1.17", "and it is the newest, which Gitea returns first");
|
||||
|
||||
let two = apply_limit(Some(2), shaped.clone());
|
||||
assert_eq!(two.as_array().map(Vec::len), Some(2));
|
||||
|
||||
// Asking for more than there is is not an error.
|
||||
let plenty = apply_limit(Some(99), shaped);
|
||||
assert_eq!(plenty.as_array().map(Vec::len), Some(3));
|
||||
}
|
||||
|
||||
/// A limit that cannot apply is ignored rather than guessed at.
|
||||
#[test]
|
||||
fn limit_leaves_a_single_record_alone() {
|
||||
let one = serde_json::json!({"name": "a record", "state": "open"});
|
||||
assert_eq!(apply_limit(Some(1), one.clone()), one);
|
||||
assert_eq!(apply_limit(Some(0), one.clone()), one, "no length to cap");
|
||||
|
||||
// Zero on a list does mean nothing — the `empty` line speaks.
|
||||
let list = serde_json::json!([{"a": 1}, {"a": 2}]);
|
||||
assert_eq!(apply_limit(Some(0), list).as_array().map(Vec::len), Some(0));
|
||||
}
|
||||
|
||||
/// A canned, Gitea-API-shaped fixture - the same fields the real
|
||||
/// `/users/{username}/starred` endpoint returns (Gitea 1.27:
|
||||
/// `stars_count`, not GitHub's `stargazers_count`; `website` is ""
|
||||
@@ -758,3 +863,32 @@ mod tests {
|
||||
assert_eq!(out, serde_json::json!(["id: /x\n"]));
|
||||
}
|
||||
}
|
||||
|
||||
#[cfg(all(test, feature = "ssr"))]
|
||||
mod finished_tests {
|
||||
use super::*;
|
||||
|
||||
#[test]
|
||||
fn a_desk_keeps_every_open_case_and_the_newest_finished_ones() {
|
||||
let schema = crate::aggregates::parse_aggregates_yaml(
|
||||
"aggregates:\n - bucket: leads\n initial: open\n states:\n open: { event: in }\n won: { event: won }\n lost: { event: lost }\n transitions:\n open: [won, lost]\n",
|
||||
)
|
||||
.unwrap()
|
||||
.remove("leads")
|
||||
.unwrap();
|
||||
let rec = |id: &str, state: &str, decided: i64| serde_json::json!({ "id": id, "state": state, "submitted_ms": 0, "decided_ms": decided });
|
||||
let mut items = vec![rec("o1", "open", 0), rec("o2", "open", 0)];
|
||||
items.extend((1..=6).map(|n| rec(&format!("f{n}"), if n % 2 == 0 { "won" } else { "lost" }, n)));
|
||||
let ids = |v: &[serde_json::Value]| v.iter().map(|x| x["id"].as_str().unwrap().to_string()).collect::<Vec<_>>();
|
||||
|
||||
let shown = cap_finished(items.clone(), &schema, 3);
|
||||
// Both open cases, then the three newest finished and one more
|
||||
// to tell the page there are older ones.
|
||||
assert_eq!(ids(&shown), ["o1", "o2", "f6", "f5", "f4", "f3"]);
|
||||
assert!(shown[2..].iter().all(|v| v["finished"] == true));
|
||||
assert!(shown[..2].iter().all(|v| v.get("finished").is_none()));
|
||||
|
||||
// Asking for all of them: no extra, so no "show older".
|
||||
assert_eq!(cap_finished(items, &schema, 6).len(), 8);
|
||||
}
|
||||
}
|
||||
|
||||
+254
-7
@@ -132,34 +132,91 @@ pub fn run(argv: Vec<String>) -> anyhow::Result<()> {
|
||||
.to_string();
|
||||
let mut green = false;
|
||||
let mut report = String::new();
|
||||
// What the lint found, by id, round over round (`--findings`): what
|
||||
// a round closed, what it left, what it broke. `None` while the
|
||||
// lint is one that predates `--findings`; then the whole report
|
||||
// goes back, as it always did.
|
||||
let mut ledger = Ledger::default();
|
||||
let mut tracked = true;
|
||||
let mut undone = String::new();
|
||||
std::fs::create_dir_all(opts.out.join("findings"))?;
|
||||
for round in 1..=opts.rounds {
|
||||
let prompt = if round == 1 {
|
||||
format!(
|
||||
"{REFERENCE}\n\n# Task\n\nWrite the content repo that meets this frozen `needs.yaml`: `aggregates.yaml`, `site.yaml`, and every page under `questions/` - the front door, followup pages, role-gated pages, one desk per handling group. Do not return needs.yaml.\n\n## Intake\n\n{intake}\n\n## needs.yaml (frozen)\n\n```yaml\n{needs}\n```\n\n{FORMAT}"
|
||||
)
|
||||
} else if tracked {
|
||||
let (heading, files) = match named_files(&opts.out, ledger.open.values()) {
|
||||
Some(files) => ("Files the findings name", dump_files(&opts.out, &files)?),
|
||||
None => ("Current files", dump(&opts.out)?),
|
||||
};
|
||||
format!(
|
||||
"{REFERENCE}\n\n# Task\n\nThe verifier rejected the repo. Below is every finding still open, each with the change that closes it. Make those changes and nothing else: findings already closed stay closed. Return only the files you change, each complete. To remove a file, return it with the single line DELETE. needs.yaml is frozen; do not return it.\n\n## Open findings\n\n{}\n{undone}## needs.yaml (frozen)\n\n```yaml\n{needs}\n```\n\n## {heading}\n\n{files}\n\n{FORMAT}",
|
||||
ledger.open_list(round)
|
||||
)
|
||||
} else {
|
||||
format!(
|
||||
"{REFERENCE}\n\n# Task\n\nThe verifier rejected the repo below. Fix it. Return only the files you change, each complete. To remove a file, return it with the single line DELETE. needs.yaml is frozen; do not return it.\n\n## Verifier output\n\n```\n{report}\n```\n\n## needs.yaml (frozen)\n\n```yaml\n{needs}\n```\n\n## Current files\n\n{}\n\n{FORMAT}",
|
||||
dump(&opts.out)?
|
||||
)
|
||||
};
|
||||
let before = snapshot(&opts.out)?;
|
||||
match parse_files(&llm(&opts.llm, &prompt)?) {
|
||||
Ok(files) => apply(&opts.out, files)?,
|
||||
Err(_) => eprintln!("structure round {round}: the reply had no file blocks"),
|
||||
}
|
||||
let (ok, r) = lint(&opts, &[])?;
|
||||
let (ok, r, found) = lint_findings(&opts, &opts.out.join(format!("findings/round-{round}.jsonl")))?;
|
||||
report = r;
|
||||
let fails = report.lines().filter(|l| l.starts_with("FAIL")).count();
|
||||
eprintln!("structure round {round}: {}", if ok { "green".to_string() } else { format!("{fails} FAIL line(s)") });
|
||||
log.push(format!("structure round {round}: ok={ok} fails={fails}"));
|
||||
let Some(found) = found else {
|
||||
tracked = false;
|
||||
let fails = report.lines().filter(|l| l.starts_with("FAIL")).count();
|
||||
eprintln!("structure round {round}: {}", if ok { "green".to_string() } else { format!("{fails} FAIL line(s)") });
|
||||
log.push(format!("structure round {round}: ok={ok} fails={fails}"));
|
||||
if ok {
|
||||
green = true;
|
||||
break;
|
||||
}
|
||||
continue;
|
||||
};
|
||||
let previous = ledger.clone();
|
||||
let progress = ledger.update(round, found.into_iter().filter(|f| f.level == "fail").collect());
|
||||
let line = format!("structure round {round}: {}", progress.summary(ledger.open.len()));
|
||||
eprintln!("{line}");
|
||||
log.push(line);
|
||||
undone.clear();
|
||||
if ok {
|
||||
green = true;
|
||||
break;
|
||||
}
|
||||
if round == 1 {
|
||||
continue;
|
||||
}
|
||||
// A fix that reopens what an earlier round closed is undone,
|
||||
// and the model is told what it broke, rather than chasing the
|
||||
// same findings round after round.
|
||||
if !progress.reopened.is_empty() {
|
||||
restore(&opts.out, before)?;
|
||||
ledger = previous;
|
||||
undone = format!(
|
||||
"## Your last change was undone\n\nIt reopened findings an earlier round had closed:\n\n{}\nClose the open findings without touching what made these pass.\n\n",
|
||||
progress.reopened.iter().map(|f| format!("- [{}] {}\n", f.id, f.message)).collect::<String>()
|
||||
);
|
||||
eprintln!("structure round {round}: reopened {} closed finding(s), undone", progress.reopened.len());
|
||||
log.push(format!("structure round {round}: undone (reopened {})", progress.reopened.len()));
|
||||
continue;
|
||||
}
|
||||
// No progress, no more rounds: another pass at the same findings
|
||||
// is how a site gets over-trained. What is left is reported.
|
||||
if progress.fixed.is_empty() || ledger.open.len() >= previous.open.len() {
|
||||
eprintln!("structure round {round}: no finding closed, stopping");
|
||||
log.push(format!("structure round {round}: stalled"));
|
||||
break;
|
||||
}
|
||||
}
|
||||
if !green {
|
||||
std::fs::write(opts.out.join("run.log"), format!("{}\n\n{report}", log.join("\n")))?;
|
||||
bail!("structure never went green in {} round(s); last verifier output is in run.log", opts.rounds);
|
||||
let open = if tracked { format!("\n\nStill open:\n{}", ledger.open_list(opts.rounds + 1)) } else { String::new() };
|
||||
std::fs::write(opts.out.join("run.log"), format!("{}{open}\n\n{report}", log.join("\n")))?;
|
||||
bail!("structure did not go green; what is still open is in run.log");
|
||||
}
|
||||
|
||||
// Phase 3: wording.
|
||||
@@ -191,7 +248,10 @@ pub fn run(argv: Vec<String>) -> anyhow::Result<()> {
|
||||
}
|
||||
eprintln!("wording round {round}: {}", summary.join(" | "));
|
||||
log.push(format!("wording round {round}: {}", summary.join(" | ")));
|
||||
if defects.is_empty() || round == opts.wording_rounds {
|
||||
// A visitor who chose right, hesitantly, is not a defect worth a
|
||||
// rewrite on its own: only a misroute is. Hesitations ride along
|
||||
// as context when there is one.
|
||||
if !defects.contains("MISROUTE") || round == opts.wording_rounds {
|
||||
break;
|
||||
}
|
||||
let personas = needs_full.split("\npersonas:").nth(1).unwrap_or_default();
|
||||
@@ -311,6 +371,152 @@ fn lint(opts: &Opts, extra: &[String]) -> anyhow::Result<(bool, String)> {
|
||||
Ok((output.status.success(), report))
|
||||
}
|
||||
|
||||
/// One finding as the lint's `--findings` writes it (docs/lint.md).
|
||||
#[derive(Clone, Debug, serde::Deserialize)]
|
||||
struct Finding {
|
||||
id: String,
|
||||
level: String,
|
||||
code: String,
|
||||
#[serde(default)]
|
||||
need: String,
|
||||
message: String,
|
||||
#[serde(default)]
|
||||
fix: Option<String>,
|
||||
}
|
||||
|
||||
/// The lint with `--findings` into `out`, and what it found; `None` for
|
||||
/// the findings when the lint predates the flag (then it is run again
|
||||
/// the old way, and only its text comes back).
|
||||
fn lint_findings(opts: &Opts, out: &Path) -> anyhow::Result<(bool, String, Option<Vec<Finding>>)> {
|
||||
let (ok, report) = lint(opts, &["--findings".to_string(), out.display().to_string()])?;
|
||||
if report.contains("unknown argument: --findings") {
|
||||
let (ok, report) = lint(opts, &[])?;
|
||||
return Ok((ok, report, None));
|
||||
}
|
||||
let raw = std::fs::read_to_string(out).unwrap_or_default();
|
||||
let found = raw
|
||||
.lines()
|
||||
.filter(|l| !l.trim().is_empty())
|
||||
.map(|l| serde_json::from_str::<Finding>(l).with_context(|| format!("reading {}", out.display())))
|
||||
.collect::<anyhow::Result<Vec<_>>>()?;
|
||||
Ok((ok, report, Some(found)))
|
||||
}
|
||||
|
||||
/// Every failing finding by id, across rounds.
|
||||
#[derive(Clone, Default)]
|
||||
struct Ledger {
|
||||
open: BTreeMap<String, Finding>,
|
||||
/// The round each open finding was first seen in.
|
||||
since: BTreeMap<String, usize>,
|
||||
/// Ids a round has closed, so one that comes back is a regression,
|
||||
/// not news.
|
||||
closed: std::collections::BTreeSet<String>,
|
||||
}
|
||||
|
||||
struct Progress {
|
||||
fixed: Vec<Finding>,
|
||||
new: Vec<Finding>,
|
||||
still: usize,
|
||||
reopened: Vec<Finding>,
|
||||
}
|
||||
|
||||
impl Progress {
|
||||
fn summary(&self, open: usize) -> String {
|
||||
if open == 0 {
|
||||
return format!("green (fixed {})", self.fixed.len());
|
||||
}
|
||||
format!(
|
||||
"{open} open - fixed {}, still open {}, new {}, reopened {}",
|
||||
self.fixed.len(),
|
||||
self.still,
|
||||
self.new.len(),
|
||||
self.reopened.len()
|
||||
)
|
||||
}
|
||||
}
|
||||
|
||||
impl Ledger {
|
||||
fn update(&mut self, round: usize, fails: Vec<Finding>) -> Progress {
|
||||
let now: BTreeMap<String, Finding> = fails.into_iter().map(|f| (f.id.clone(), f)).collect();
|
||||
let fixed: Vec<Finding> = self.open.values().filter(|f| !now.contains_key(&f.id)).cloned().collect();
|
||||
let mut new = Vec::new();
|
||||
let mut reopened = Vec::new();
|
||||
let mut still = 0;
|
||||
for (id, f) in &now {
|
||||
if self.open.contains_key(id) {
|
||||
still += 1;
|
||||
} else if self.closed.contains(id) {
|
||||
reopened.push(f.clone());
|
||||
} else {
|
||||
new.push(f.clone());
|
||||
}
|
||||
self.since.entry(id.clone()).or_insert(round);
|
||||
}
|
||||
for f in &fixed {
|
||||
self.closed.insert(f.id.clone());
|
||||
self.since.remove(&f.id);
|
||||
}
|
||||
self.open = now;
|
||||
Progress { fixed, new, still, reopened }
|
||||
}
|
||||
|
||||
/// The open findings for a prompt or a log: id, how long open, the
|
||||
/// message and its fix.
|
||||
fn open_list(&self, round: usize) -> String {
|
||||
self.open
|
||||
.values()
|
||||
.map(|f| {
|
||||
let age = match self.since.get(&f.id) {
|
||||
Some(&first) if first + 1 < round => format!(", open since round {first}"),
|
||||
_ => String::new(),
|
||||
};
|
||||
let scope = if f.need.is_empty() { String::new() } else { format!(" {:?}", f.need) };
|
||||
format!("- [{}] {}{scope}{age}: {}\n", f.id, f.code, f.message)
|
||||
})
|
||||
.collect()
|
||||
}
|
||||
}
|
||||
|
||||
/// The files open findings point at - every desk page their fixes name,
|
||||
/// plus aggregates.yaml - when that is enough to act on. `None` when a
|
||||
/// finding is about getting around the site (a path, an entry page, a
|
||||
/// missing desk) or names no page: then the model needs to see it all.
|
||||
fn named_files<'a>(out: &Path, findings: impl Iterator<Item = &'a Finding>) -> Option<Vec<PathBuf>> {
|
||||
const WHOLE_SITE: &[&str] = &["schema", "entry-missing", "entry-gated", "path-missing", "path-closed", "path-over-budget", "desk-missing", "need-duplicate"];
|
||||
let mut files = vec![out.join("aggregates.yaml")];
|
||||
for f in findings {
|
||||
if WHOLE_SITE.contains(&f.code.as_str()) {
|
||||
return None;
|
||||
}
|
||||
let text = format!("{} {}", f.message, f.fix.as_deref().unwrap_or_default());
|
||||
for page in text.split_whitespace().filter(|w| w.starts_with('/')) {
|
||||
let page = page.trim_end_matches(|c: char| !(c.is_alphanumeric() || c == '_' || c == '-' || c == '/'));
|
||||
let rel = page.trim_start_matches('/');
|
||||
let candidates = if rel.is_empty() {
|
||||
vec![out.join("questions/index.yaml")]
|
||||
} else {
|
||||
vec![out.join(format!("questions/{rel}.yaml")), out.join(format!("questions/{rel}/index.yaml"))]
|
||||
};
|
||||
if let Some(path) = candidates.into_iter().find(|p| p.exists()) {
|
||||
if !files.contains(&path) {
|
||||
files.push(path);
|
||||
}
|
||||
}
|
||||
}
|
||||
}
|
||||
Some(files)
|
||||
}
|
||||
|
||||
/// `dump`, for the named files only.
|
||||
fn dump_files(out: &Path, files: &[PathBuf]) -> anyhow::Result<String> {
|
||||
let mut text = String::new();
|
||||
for path in files.iter().filter(|p| p.exists()) {
|
||||
let rel = path.strip_prefix(out).unwrap_or(path).display().to_string();
|
||||
text.push_str(&format!("<<<FILE {rel}>>>\n{}<<<END>>>\n\n", std::fs::read_to_string(path)?));
|
||||
}
|
||||
Ok(text)
|
||||
}
|
||||
|
||||
/// `<<<FILE path>>> ... <<<END>>>` blocks. Anything outside a block is
|
||||
/// ignored, so a model that adds a sentence does no harm.
|
||||
fn parse_files(text: &str) -> anyhow::Result<BTreeMap<String, String>> {
|
||||
@@ -591,3 +797,44 @@ mod tests {
|
||||
assert!(!allowed(".gitea/workflows/deploy.yaml"));
|
||||
}
|
||||
}
|
||||
|
||||
#[cfg(test)]
|
||||
mod ledger_tests {
|
||||
use super::*;
|
||||
|
||||
fn f(id: &str, code: &str, message: &str) -> Finding {
|
||||
Finding { id: id.into(), level: "fail".into(), code: code.into(), need: "hire".into(), message: message.into(), fix: None }
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn a_round_is_told_what_it_closed_kept_added_and_broke() {
|
||||
let mut ledger = Ledger::default();
|
||||
let p = ledger.update(1, vec![f("a", "strand", "x"), f("b", "strand", "y")]);
|
||||
assert_eq!((p.fixed.len(), p.new.len(), p.still, p.reopened.len()), (0, 2, 0, 0));
|
||||
let p = ledger.update(2, vec![f("b", "strand", "y"), f("c", "edge-unoffered", "z")]);
|
||||
assert_eq!((p.fixed.len(), p.new.len(), p.still, p.reopened.len()), (1, 1, 1, 0));
|
||||
assert_eq!(p.summary(ledger.open.len()), "2 open - fixed 1, still open 1, new 1, reopened 0");
|
||||
// "a" was closed in round 2; its return is a regression.
|
||||
let p = ledger.update(3, vec![f("a", "strand", "x"), f("b", "strand", "y")]);
|
||||
assert_eq!(p.reopened.iter().map(|f| f.id.as_str()).collect::<Vec<_>>(), ["a"]);
|
||||
assert!(ledger.open_list(4).contains("[b] strand \"hire\", open since round 1: y"));
|
||||
let p = ledger.update(4, vec![]);
|
||||
assert_eq!(p.summary(ledger.open.len()), "green (fixed 2)");
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn a_fix_round_sees_the_pages_its_findings_name() {
|
||||
let dir = std::env::temp_dir().join(format!("ledger-test-{}", std::process::id()));
|
||||
std::fs::create_dir_all(dir.join("questions/desk")).unwrap();
|
||||
std::fs::write(dir.join("aggregates.yaml"), "x").unwrap();
|
||||
std::fs::write(dir.join("questions/desk/index.yaml"), "x").unwrap();
|
||||
std::fs::write(dir.join("questions/index.yaml"), "x").unwrap();
|
||||
let mut strand = f("a", "strand", "records strand in \"quoted\"");
|
||||
strand.fix = Some("add `{ from: quoted, to: won, label: ... }` to the transitions on /desk".into());
|
||||
let files = named_files(&dir, [&strand].into_iter()).unwrap();
|
||||
assert_eq!(files, vec![dir.join("aggregates.yaml"), dir.join("questions/desk/index.yaml")]);
|
||||
// A finding about getting around the site needs all of it.
|
||||
assert!(named_files(&dir, [&strand, &f("b", "path-missing", "no path")].into_iter()).is_none());
|
||||
std::fs::remove_dir_all(&dir).ok();
|
||||
}
|
||||
}
|
||||
|
||||
@@ -190,7 +190,16 @@ not done. Its messages are exact:
|
||||
- `no desk gated on "g" reads "b"`: add the desk.
|
||||
- `"g" cannot move a record to "s"` / `records strand in "s"`: the desk
|
||||
is missing a transition the state machine allows, or the state
|
||||
machine is missing the edge.
|
||||
machine is missing the edge. The message ends in `fix:` with the
|
||||
exact `{ from, to, label }` and the desk page to put it on - do that.
|
||||
- `state "s" is never reached`: nothing moves a record into it. Offer
|
||||
the move it names, or remove the state.
|
||||
- `stage "b" comes after "a" ... but no move leads on`: the machine
|
||||
cannot go the way the business works; add the edge it names.
|
||||
- `the mail on entering "s" uses {x}`: the mail names a field no form
|
||||
collects. Use one of the fields it lists, or collect `x`.
|
||||
- WARN `the move "a" -> "b" is declared but no desk ... offers it`:
|
||||
offer it, or drop the edge - every edge should be a button.
|
||||
- `action ... does not match any declared question id`: the page does
|
||||
not exist.
|
||||
- `entry page ... names no one responsible`: add `responsible`.
|
||||
|
||||
@@ -1280,6 +1280,31 @@ textarea:focus {
|
||||
gap: 0.8rem;
|
||||
}
|
||||
|
||||
/* After the newest finished cases: the older ones, on request. Quiet,
|
||||
since the desk is for what is still moving. */
|
||||
.show-older {
|
||||
justify-self: start;
|
||||
background: none;
|
||||
border: 0.06rem solid var(--line);
|
||||
border-radius: 0.4rem;
|
||||
color: var(--ink-dim);
|
||||
font: inherit;
|
||||
font-size: 0.9rem;
|
||||
padding: 0.35rem 0.8rem;
|
||||
cursor: pointer;
|
||||
}
|
||||
|
||||
/* A releases manifest: the pick for this device, the rest folded. */
|
||||
.releases .pill { display: inline-block; }
|
||||
.releases pre { white-space: pre-wrap; font-size: 0.85rem; margin: 0.4rem 0 0; }
|
||||
.releases-all summary { color: var(--ink-dim); font-size: 0.9rem; cursor: pointer; }
|
||||
.releases-all ul { margin: 0.4rem 0 0 1rem; font-size: 0.9rem; }
|
||||
|
||||
.show-older:hover {
|
||||
color: inherit;
|
||||
border-color: var(--ink-dim);
|
||||
}
|
||||
|
||||
.answer-row {
|
||||
border: 0.06rem solid var(--line);
|
||||
border-radius: 0.8rem;
|
||||
|
||||
Reference in New Issue
Block a user