Compare commits

...
6 Commits
Author SHA1 Message Date
Bendik Aagaard LynghaugandClaude Fable 5.1 aee5048b01 Release 0.6.19
docs / deploy (push) Successful in 3s
Publish release / publish (push) Successful in 2m50s
Test / test (push) Successful in 2m10s
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
2026-10-05 12:48:42 +02:00
Bendik Aagaard LynghaugandClaude Fable 5.1 268d545843 A releases manifest renders as a download card, the device's file first
A kind: url resource whose JSON is a releases manifest - latest,
versions newest first with per-platform assets, channels such as a
package repo or TestFlight - is one card: the newest version, the file
for the visitor's own platform (read from the browser on hydration;
the server renders every file), the Linux package repo lines beside a
Linux file, TestFlight for iPhones and Macs, and the rest folded under
"All downloads". Decided by shape, as every resource is; written for
redoal's public releases at dl.redoal.com.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
2026-10-05 12:48:31 +02:00
Bendik Aagaard LynghaugandClaude Opus 5 22d086f9e1 A resource can show only its newest few
docs / deploy (push) Successful in 2s
Test / test (push) Successful in 2m0s
`limit: n` on a resource spec, applied after `jq`.

The releases card is what asked for it: a page that grows by one card
per release gets longer forever, and the cut a visitor should run is the
newest one — the rest are history, not an invitation. redoal's card had
every release on it.

After `jq` rather than before, for two reasons: it caps what the page
will actually render whatever the filter did, and content that reorders
with the filter can then take the top of its own ordering rather than
the top of the source's. A source yielding a single record has no length
and passes through untouched — turning one record into an empty list
would be worse than ignoring a limit that cannot apply. `limit: 0` on a
list does mean nothing, and the `empty` line speaks instead, which is a
coherent thing for content to ask for.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01ByshXuDEzH8hXh5qp4VgGk
2026-10-04 22:51:12 +02:00
Bendik Aagaard LynghaugandClaude Opus 5.5 48ae5e07c2 Release 0.6.18
docs / deploy (push) Successful in 2s
Test / test (push) Successful in 2m0s
Publish release / publish (push) Successful in 1m50s
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-03 10:48:22 +02:00
Bendik Aagaard LynghaugandClaude Opus 5.5 b619926a11 generate follows its findings through, and stops when it stalls
Each structure round is logged as fixed, still open, new and reopened;
a fix round sees the open findings with their fixes and only the files
they name; a change that reopens a closed finding is undone; a round
that closes nothing ends the phase with what is left in run.log. A
hesitant visitor alone no longer triggers a wording rewrite.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-03 10:48:09 +02:00
Bendik Aagaard LynghaugandClaude Opus 5.5 706c8108df iris check --findings: every finding as JSON with a stable id
The id hashes scope, need, code and subject, nothing of the wording, so
the same gap keeps its id across runs. Every message that knows its fix
says it after "fix:". Codes listed in docs/lint.md.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-03 10:48:09 +02:00
14 changed files with 747 additions and 35 deletions
+9
View File
@@ -4,6 +4,15 @@ One line per change, grouped by the release that shipped it. Newest
first. A release is a Cargo.toml bump plus a `v<version>` tag; the first. A release is a Cargo.toml bump plus a `v<version>` tag; the
content repos pin the tag they run (see the end of this file). content repos pin the tag they run (see the end of this file).
## 0.6.19 (2026-10-05)
- A `kind: url` resource whose JSON is a releases manifest (`latest`, `versions` with per-platform `assets`, `channels` such as a package repo or TestFlight) renders as a download card: the file for the visitor's own platform first, decided in the browser, the rest and the channels folded. Written for redoal's public releases at dl.redoal.com (docs: content-reference, "A releases manifest").
## 0.6.18 (2026-10-03)
- `iris check --findings <out.jsonl>` writes every finding as JSON: a stable `id` (scope, need, code and subject, nothing of the wording), `level`, `code`, `scope`, `need`, `subject`, `message` and `fix`. Every message that knows its fix now says it after "fix:". The codes are listed in docs/lint.md.
- `portal generate` follows its findings through: each structure round is logged as fixed, still open, new and reopened; a fix round is shown the open findings with their fixes and only the files they name; a change that reopens a closed finding is undone; and the phase stops when a round closes nothing, writing what is left to run.log. A hesitant visitor no longer triggers a wording rewrite by itself, only a misroute does. A lint without `--findings` is still driven the old way.
## 0.6.17 (2026-10-02) ## 0.6.17 (2026-10-02)
- A desk shows every record still moving and only the three most recently finished (a final state: no move out of it), with "Show older" for three more at a time. A desk that has run for a year is still about the work in hand. - A desk shows every record still moving and only the three most recently finished (a final state: no move out of it), with "Show older" for three more at a time. A desk that has run for a year is still about the work in hand.
Generated
+1 -1
View File
@@ -3357,7 +3357,7 @@ dependencies = [
[[package]] [[package]]
name = "portal" name = "portal"
version = "0.6.17" version = "0.6.19"
dependencies = [ dependencies = [
"anyhow", "anyhow",
"arc-swap", "arc-swap",
+2 -1
View File
@@ -1,6 +1,6 @@
[package] [package]
name = "portal" name = "portal"
version = "0.6.17" version = "0.6.19"
edition = "2021" edition = "2021"
[lib] [lib]
@@ -60,6 +60,7 @@ web-sys = { version = "0.3", features = [
"FileList", "FileList",
"FormData", "FormData",
"HtmlInputElement", "HtmlInputElement",
"Navigator",
"Request", "Request",
"RequestInit", "RequestInit",
"Response", "Response",
+21
View File
@@ -255,6 +255,7 @@ reads is never a client-supplied parameter.
states: [approved, refused] # only these; empty means all states: [approved, refused] # only these; empty means all
where: { field: case, equals: "{key}" } where: { field: case, equals: "{key}" }
jq: ".[] | {name, url: .html_url}" jq: ".[] | {name, url: .html_url}"
limit: 1 # at most this many, after jq
empty: Nothing decided yet. empty: Nothing decided yet.
link: { to: "/notices/{id}", label: Answer this } # a page per record link: { to: "/notices/{id}", label: Answer this } # a page per record
transitions: # the desk's buttons transitions: # the desk's buttons
@@ -264,6 +265,14 @@ reads is never a client-supplied parameter.
`source.kind` is one of `kv`, `gitea_starred`, `source.kind` is one of `kv`, `gitea_starred`,
`gitea_releases`, `url`. `gitea_releases`, `url`.
`limit: 1` shows only the first item — for a feed where only the newest
matters. A releases card wants the cut someone should run, not every
cut there has ever been, and a page that grows by one card per release
gets longer forever. It applies *after* `jq`, so a filter can reorder
and the limit takes the top of that ordering. A source that yields a
single record has no length and ignores it; `limit: 0` on a list shows
nothing, and the `empty` line speaks instead.
A spec with neither `public: true` nor `requires_group` is A spec with neither `public: true` nor `requires_group` is
unreachable: reads fail closed, and mutations always need a group unreachable: reads fail closed, and mutations always need a group
whatever `public` says. whatever `public` says.
@@ -293,6 +302,18 @@ stays about the work in hand however long the site has run; nothing
is hidden from a read-only list, and `leaves_after` still takes a is hidden from a read-only list, and `leaves_after` still takes a
record out of every list altogether. record out of every list altogether.
A **releases manifest** - a `kind: url` resource whose JSON is
`{ name, latest, versions: [{ version, date, notes, assets: [{ platform,
url, sha256, size }] }], channels: { arch: { repo, package, note },
ios: { testflight, note } } }`, newest version first, as a release
pipeline writes it - renders as one card: the newest version, the file
for the visitor's own platform first (read from the browser: `ios`,
`macos`, `linux-x86_64`, `linux-aarch64`, `windows`, `android`), the
other files and the channels folded under "All downloads". Linux gets
the package repo lines beside its file; iPhones and Macs get
TestFlight. Platforms a manifest lacks fall through to the folded
list.
`states` and `where` narrow a listing. They matter most for a register `states` and `where` narrow a listing. They matter most for a register
that is public by law — public about what was decided, not about what that is public by law — public about what was decided, not about what
was withdrawn — while the desk working the same bucket is a different was withdrawn — while the desk working the same bucket is a different
+22
View File
@@ -50,6 +50,28 @@ already has the button, or the edge aggregates.yaml lacks. A schema
error no longer hides the needs findings - both are printed, so one error no longer hides the needs findings - both are printed, so one
round can fix them together. round can fix them together.
`--findings <out.jsonl>` writes the same findings one JSON object per
line: `id`, `level` (`fail`/`warn`), `code`, `scope` (`need`,
`bucket`, or `repo` for a schema error), `need` (the need's id, or the
bucket's name), `subject` (the state, the move as `from>to`, or the
page), `message` and `fix`. The file is written empty when the content
is clean. `id` hashes scope, need, code and subject and nothing of the
wording, so the same gap keeps its id from one run to the next: a
trainer can tell what a round closed, what is still open and what it
broke. The codes:
| code | |
|---|---|
| `schema` | content the site would refuse to load |
| `need-duplicate`, `persona-unknown-need` | needs.yaml itself |
| `entry-missing`, `entry-gated`, `entry-no-responsible` | the page a visitor arrives on |
| `path-missing`, `path-closed`, `path-over-budget`, `path-no-action` | getting the answer recorded |
| `machine-missing`, `desk-missing`, `desk-gated-twice` | who works the bucket |
| `done-unknown`, `done-unreachable`, `success-not-done` | finishing a case |
| `stage-unknown`, `stage-unreachable`, `stage-order` | the business's own stages |
| `state-unreached`, `strand`, `edge-unoffered` | every state and move of the bucket |
| `mail-unreachable`, `mail-unknown-field` | what a state mails |
A need marked `planned` warns instead of failing. Exit status is the A need marked `planned` warns instead of failing. Exit status is the
verdict, so a content repo's CI is one line. verdict, so a content repo's CI is one line.
+16 -5
View File
@@ -243,13 +243,24 @@ against the lint. It is kept for the record of what it measured.
visitor will tolerate. visitor will tolerate.
2. **Structure.** Pages, state machines and desks are generated, then 2. **Structure.** Pages, state machines and desks are generated, then
revised against the lint, `iris check`, until every need is met: revised against the lint, `iris check`, until every need is met:
a path within budget, a desk held by the right group, every a path within budget, a desk held by the right group, every state
finished state reachable, no record stranding. The page-writing of every bucket reachable and none stranding a record. The
model sees the needs and never the personas. page-writing model sees the needs and never the personas. Each
round the lint writes its findings with stable ids (`--findings`,
kept in `<out>/findings/round-N.jsonl`), and the round is logged as
what it fixed, what is still open, what is new and what it
reopened. A fix round is shown only the open findings, each with
the change that closes it, and the files they name (the whole repo
when a finding is about getting around the site). A change that
reopens a closed finding is undone and the model told what it
broke. A round that closes nothing, or leaves as many open as it
found, ends the phase: what is left is written to `run.log` rather
than polished at again.
3. **Wording.** A separate model plays each persona, blind, first with 3. **Wording.** A separate model plays each persona, blind, first with
headings only and then with descriptions. A misroute is a wording headings only and then with descriptions. A misroute is a wording
defect and goes back to the generator. A rewrite that breaks defect and goes back to the generator; a visitor who chose right
structure is reverted. but hesitated is reported and rewrites nothing by itself. A rewrite
that breaks structure is reverted.
Model output is only ever written to `aggregates.yaml`, `site.yaml` Model output is only ever written to `aggregates.yaml`, `site.yaml`
and `questions/**.yaml` inside the output directory. and `questions/**.yaml` inside the output directory.
+162
View File
@@ -2124,9 +2124,147 @@ fn ResourceValue(
} }
} }
// A releases manifest (content-reference, "A releases manifest"):
// the newest version, its files per platform, and the standing
// channels, with the visitor's own platform picked in the browser.
if is_releases_manifest(&value) {
return view! { <ReleasesCard manifest=value/> }.into_any();
}
view! { <pre class="resource-raw">{value.to_string()}</pre> }.into_any() view! { <pre class="resource-raw">{value.to_string()}</pre> }.into_any()
} }
/// `{ name, latest, versions: [{ version, date, notes, assets: [{
/// platform, url, sha256, size }] }], channels: {...} }` - the shape a
/// release pipeline writes (redoal's does) for a site to offer downloads.
pub fn is_releases_manifest(value: &serde_json::Value) -> bool {
value.get("latest").and_then(|v| v.as_str()).is_some()
&& value.get("versions").and_then(|v| v.as_array()).is_some_and(|v| !v.is_empty())
}
/// The platform a browser says it is, in the manifest's words:
/// `ios`, `macos`, `android`, `windows`, `linux-x86_64`,
/// `linux-aarch64`, or `other`. Read, never trusted for anything but
/// which download to put first.
pub fn platform_of(user_agent: &str) -> &'static str {
let ua = user_agent.to_ascii_lowercase();
if ua.contains("iphone") || ua.contains("ipad") {
"ios"
} else if ua.contains("mac os x") || ua.contains("macintosh") {
"macos"
} else if ua.contains("android") {
"android"
} else if ua.contains("windows") {
"windows"
} else if ua.contains("linux") || ua.contains("x11") {
if ua.contains("aarch64") || ua.contains("arm64") {
"linux-aarch64"
} else {
"linux-x86_64"
}
} else {
"other"
}
}
/// The newest release as a card: the file for this device first (the
/// platform is read in the browser, so the server renders every file
/// and the pick lands on hydration), the rest after, and the channels
/// that are not files - a package repo, TestFlight.
#[component]
fn ReleasesCard(manifest: serde_json::Value) -> impl IntoView {
let name = manifest.get("name").and_then(|v| v.as_str()).unwrap_or("").to_string();
let latest = manifest.get("latest").and_then(|v| v.as_str()).unwrap_or("").to_string();
let newest = manifest["versions"][0].clone();
let date = newest.get("date").and_then(|v| v.as_str()).unwrap_or("").to_string();
let notes = newest.get("notes").and_then(|v| v.as_str()).filter(|u| u.starts_with("https://")).map(str::to_string);
let assets: Vec<(String, String, String)> = newest
.get("assets")
.and_then(|a| a.as_array())
.map(|a| {
a.iter()
.filter_map(|x| {
let platform = x.get("platform")?.as_str()?.to_string();
let url = x.get("url")?.as_str()?.to_string();
let sha = x.get("sha256").and_then(|s| s.as_str()).unwrap_or("").to_string();
Some((platform, url, sha))
})
.collect()
})
.unwrap_or_default();
let channels = manifest.get("channels").cloned().unwrap_or(serde_json::Value::Null);
let arch = channels.get("arch").cloned();
let ios = channels.get("ios").cloned();
let platform = RwSignal::new(None::<&'static str>);
Effect::new(move |_| {
#[cfg(feature = "hydrate")]
if let Some(ua) = web_sys::window().and_then(|w| w.navigator().user_agent().ok()) {
platform.set(Some(platform_of(&ua)));
}
});
let label = |p: &str| match p {
"linux-x86_64" => "Linux (x86_64)",
"linux-aarch64" => "Linux (aarch64)",
"macos" | "macos-arm64" => "macOS",
"windows" | "windows-x86_64" => "Windows",
other => other,
}.to_string();
let assets_for_list = assets.clone();
let arch_for_pick = arch.clone();
let ios_for_pick = ios.clone();
let assets_for_pick = assets.clone();
view! {
<article class="item-card releases">
<div class="item-card-title">
<strong>{name.clone()} " " {latest.clone()}</strong>
{(!date.is_empty()).then(|| view! { <span class="dim">" · " {date.clone()}</span> })}
{notes.clone().map(|u| view! { " · " <a href=u target="_blank" rel="noopener noreferrer">{move || t(&lang().get(), "release_notes")}</a> })}
</div>
// What this device should take, decided in the browser.
{move || {
let p = platform.get()?;
let arch = arch_for_pick.clone();
let ios = ios_for_pick.clone();
let own = assets_for_pick.iter().find(|(pl, _, _)| pl == p).cloned();
let pick: Option<AnyView> = match (p, own) {
(_, Some((pl, url, _))) => Some(view! {
<p><a class="pill" href=url download>{t(&lang().get(), "for_this_device")} ": " {label(&pl)}</a></p>
{(p.starts_with("linux") && arch.is_some()).then(|| {
let a = arch.clone().unwrap();
let repo = a.get("repo").and_then(|v| v.as_str()).unwrap_or("").to_string();
let pkg = a.get("package").and_then(|v| v.as_str()).unwrap_or("").to_string();
view! { <pre class="monospace dim">{repo} "\n\npacman -Syu " {pkg}</pre> }
})}
}.into_any()),
("ios" | "macos", None) => ios.as_ref().and_then(|i| i.get("testflight").and_then(|v| v.as_str()).map(str::to_string)).map(|tf| {
let note = ios.as_ref().and_then(|i| i.get("note").and_then(|v| v.as_str())).unwrap_or("").to_string();
view! { <p><a class="pill" href=tf target="_blank" rel="noopener noreferrer">{t(&lang().get(), "for_this_device")} ": TestFlight"</a> <span class="dim">" " {note}</span></p> }.into_any()
}),
_ => None,
};
pick
}}
<details class="releases-all">
<summary>{move || t(&lang().get(), "all_downloads")}</summary>
<ul>
{assets_for_list.iter().map(|(pl, url, sha)| view! {
<li><a href=url.clone() download>{label(pl)}</a> {(!sha.is_empty()).then(|| view! { <span class="dim monospace">" sha256 " {sha[..sha.len().min(12)].to_string()} "…"</span> })}</li>
}).collect::<Vec<_>>()}
{arch.as_ref().map(|a| {
let note = a.get("note").and_then(|v| v.as_str()).unwrap_or("").to_string();
view! { <li>{note}</li> }
})}
{ios.as_ref().and_then(|i| i.get("testflight").and_then(|v| v.as_str()).map(|tf| {
let note = i.get("note").and_then(|v| v.as_str()).unwrap_or("").to_string();
view! { <li><a href=tf.to_string() target="_blank" rel="noopener noreferrer">"TestFlight"</a> " " {note}</li> }
}))}
</ul>
</details>
</article>
}
}
/// Pulls the first present, non-null string field out of a JSON object /// Pulls the first present, non-null string field out of a JSON object
/// matching one of `keys`, tried in order - shared between `ItemCard` /// matching one of `keys`, tried in order - shared between `ItemCard`
/// and `SelectField`'s option rendering, since both need "guess a /// and `SelectField`'s option rendering, since both need "guess a
@@ -3365,3 +3503,27 @@ mod picture_tests {
assert_eq!(picture_of(&answer, None), None); assert_eq!(picture_of(&answer, None), None);
} }
} }
#[cfg(test)]
mod releases_tests {
use super::*;
#[test]
fn a_browser_names_its_platform_in_the_manifests_words() {
assert_eq!(platform_of("Mozilla/5.0 (iPhone; CPU iPhone OS 19_0 like Mac OS X)"), "ios");
assert_eq!(platform_of("Mozilla/5.0 (Macintosh; Intel Mac OS X 15_0)"), "macos");
assert_eq!(platform_of("Mozilla/5.0 (X11; Linux x86_64) Firefox"), "linux-x86_64");
assert_eq!(platform_of("Mozilla/5.0 (X11; Linux aarch64) Firefox"), "linux-aarch64");
assert_eq!(platform_of("Mozilla/5.0 (Windows NT 10.0; Win64; x64)"), "windows");
assert_eq!(platform_of("Mozilla/5.0 (Linux; Android 15)"), "android");
assert_eq!(platform_of("curl/8"), "other");
}
#[test]
fn only_a_manifest_shape_is_a_manifest() {
let m = serde_json::json!({"name":"redoal","latest":"0.1.17","versions":[{"version":"0.1.17","assets":[]}]});
assert!(is_releases_manifest(&m));
assert!(!is_releases_manifest(&serde_json::json!({"latest":"1","versions":[]})));
assert!(!is_releases_manifest(&serde_json::json!([{"name":"x"}])));
}
}
+12
View File
@@ -519,6 +519,18 @@ pub struct ResourceSpec {
/// it where the silence needs a voice ("Nothing released yet"). /// it where the silence needs a voice ("Nothing released yet").
#[serde(default)] #[serde(default)]
pub empty: Option<String>, pub empty: Option<String>,
/// Show at most this many items, applied after `jq`. For a feed
/// where only the newest matters: a releases card wants the cut you
/// should run, not every cut there has ever been, and a page that
/// grows by one card per release gets longer forever.
///
/// After `jq` rather than before, so it caps what the page will
/// actually render whatever the filter did — and so content can
/// reorder with jq and still take the top of its own ordering. A
/// source that yields a single record has no length and passes
/// through untouched. `None` shows everything.
#[serde(default)]
pub limit: Option<usize>,
} }
/// Where a resource's live data comes from. `Kv` (a NATS KV bucket /// Where a resource's live data comes from. `Kv` (a NATS KV bucket
+9
View File
@@ -25,6 +25,9 @@ fn lookup(lang: &str, key: &str) -> Option<&'static str> {
("en", "sign_out") => "Sign out", ("en", "sign_out") => "Sign out",
("en", "nothing_here_yet") => "Nothing here yet.", ("en", "nothing_here_yet") => "Nothing here yet.",
("en", "show_older") => "Show older", ("en", "show_older") => "Show older",
("en", "for_this_device") => "For this device",
("en", "all_downloads") => "All downloads",
("en", "release_notes") => "release notes",
("en", "upload_failed") => "The file did not arrive, so nothing was sent. Try again.", ("en", "upload_failed") => "The file did not arrive, so nothing was sent. Try again.",
("en", "upload_too_large") => "The file is larger than 20 MB, so nothing was sent. Choose a smaller one, or a picture taken at a lower resolution.", ("en", "upload_too_large") => "The file is larger than 20 MB, so nothing was sent. Choose a smaller one, or a picture taken at a lower resolution.",
("en", "upload_unavailable") => "This site cannot take files at the moment, so nothing was sent. Let whoever runs the site know.", ("en", "upload_unavailable") => "This site cannot take files at the moment, so nothing was sent. Let whoever runs the site know.",
@@ -62,6 +65,9 @@ fn lookup(lang: &str, key: &str) -> Option<&'static str> {
("no", "sign_out") => "Logg ut", ("no", "sign_out") => "Logg ut",
("no", "nothing_here_yet") => "Ingenting her ennå.", ("no", "nothing_here_yet") => "Ingenting her ennå.",
("no", "show_older") => "Vis eldre", ("no", "show_older") => "Vis eldre",
("no", "for_this_device") => "Til denne enheten",
("no", "all_downloads") => "Alle nedlastinger",
("no", "release_notes") => "utgivelsesnotater",
("no", "upload_failed") => "Filen kom ikke fram, så ingenting ble sendt. Prøv igjen.", ("no", "upload_failed") => "Filen kom ikke fram, så ingenting ble sendt. Prøv igjen.",
("no", "upload_too_large") => "Filen er større enn 20 MB, så ingenting ble sendt. Velg en mindre fil, eller et bilde med lavere oppløsning.", ("no", "upload_too_large") => "Filen er større enn 20 MB, så ingenting ble sendt. Velg en mindre fil, eller et bilde med lavere oppløsning.",
("no", "upload_unavailable") => "Siden kan ikke ta imot filer akkurat nå, så ingenting ble sendt. Si fra til den som har siden.", ("no", "upload_unavailable") => "Siden kan ikke ta imot filer akkurat nå, så ingenting ble sendt. Si fra til den som har siden.",
@@ -117,6 +123,9 @@ mod tests {
"sign_out", "sign_out",
"nothing_here_yet", "nothing_here_yet",
"show_older", "show_older",
"for_this_device",
"all_downloads",
"release_notes",
"upload_failed", "upload_failed",
"upload_too_large", "upload_too_large",
"upload_unavailable", "upload_unavailable",
+41 -3
View File
@@ -24,6 +24,7 @@ pub async fn run(argv: Vec<String>) -> anyhow::Result<()> {
let mut skim = false; let mut skim = false;
let mut sim_out: Option<String> = None; let mut sim_out: Option<String> = None;
let mut show_access = false; let mut show_access = false;
let mut findings_out: Option<String> = None;
while let Some(arg) = args.next() { while let Some(arg) = args.next() {
match arg.as_str() { match arg.as_str() {
@@ -35,6 +36,7 @@ pub async fn run(argv: Vec<String>) -> anyhow::Result<()> {
"--skim" => skim = true, "--skim" => skim = true,
"--needs-sim" => sim_out = args.next(), "--needs-sim" => sim_out = args.next(),
"--access" => show_access = true, "--access" => show_access = true,
"--findings" => findings_out = args.next(),
"--needs-score" => answers_in = args.next(), "--needs-score" => answers_in = args.next(),
"--min-accuracy" => { "--min-accuracy" => {
min_accuracy = args.next().and_then(|v| v.parse().ok()).unwrap_or(min_accuracy) min_accuracy = args.next().and_then(|v| v.parse().ok()).unwrap_or(min_accuracy)
@@ -66,13 +68,14 @@ pub async fn run(argv: Vec<String>) -> anyhow::Result<()> {
(questions, aggregates_map, site, needs_raw) (questions, aggregates_map, site, needs_raw)
} else { } else {
eprintln!( eprintln!(
"usage: iris check (--repo <gitea-url> [--branch main] [--subdir questions] | --path <local-dir>) [--needs-tasks <out.jsonl> [--skim]] [--needs-sim <model.json>] [--access] [--needs-score <answers.jsonl> [--min-accuracy 0.8]]" "usage: iris check (--repo <gitea-url> [--branch main] [--subdir questions] | --path <local-dir>) [--findings <out.jsonl>] [--needs-tasks <out.jsonl> [--skim]] [--needs-sim <model.json>] [--access] [--needs-score <answers.jsonl> [--min-accuracy 0.8]]"
); );
std::process::exit(2); std::process::exit(2);
}; };
if let Err(e) = site.validate() { if let Err(e) = site.validate() {
eprintln!("FAIL: {e}"); eprintln!("FAIL: {e}");
write_findings(findings_out.as_deref(), &[schema_finding(&e.to_string())]);
std::process::exit(1); std::process::exit(1);
} }
let aggregates_map = content::with_builtin_aggregates(aggregates_map); let aggregates_map = content::with_builtin_aggregates(aggregates_map);
@@ -114,6 +117,7 @@ pub async fn run(argv: Vec<String>) -> anyhow::Result<()> {
println!(); println!();
} }
check_needs( check_needs(
findings_out.as_deref(),
needs_raw.as_deref(), needs_raw.as_deref(),
&questions, &questions,
&aggregates_map, &aggregates_map,
@@ -129,14 +133,43 @@ pub async fn run(argv: Vec<String>) -> anyhow::Result<()> {
// One schema error is no reason to hide the rest: the needs // One schema error is no reason to hide the rest: the needs
// pass reads the same content and says what else is wrong, // pass reads the same content and says what else is wrong,
// so a fix round can take them all at once. // so a fix round can take them all at once.
let mut all = vec![schema_finding(&e.to_string())];
if let Some(file) = needs_raw.as_deref().and_then(|raw| needs::parse(raw).ok()) { if let Some(file) = needs_raw.as_deref().and_then(|raw| needs::parse(raw).ok()) {
print_findings(&needs::check(&file, &questions, &aggregates_map)); let findings = needs::check(&file, &questions, &aggregates_map);
print_findings(&findings);
all.extend(findings);
} }
write_findings(findings_out.as_deref(), &all);
std::process::exit(1); std::process::exit(1);
} }
} }
} }
/// A content error the schema pass stopped at, as a finding: the repo
/// is its scope and the message its subject, since it has no other.
fn schema_finding(message: &str) -> needs::Finding {
needs::Finding {
need: String::new(),
level: needs::Level::Fail,
message: message.to_string(),
scope: "repo",
code: "schema",
subject: message.to_string(),
}
}
/// `--findings`: every finding as one JSON line (id, level, code,
/// scope, need, subject, message, fix), the same ones the text output
/// prints - an empty file when there are none, so a reader can tell
/// "clean" from "not run".
fn write_findings(out: Option<&str>, findings: &[needs::Finding]) {
let Some(out) = out else { return };
let lines: String = findings.iter().map(|f| format!("{}\n", f.to_json())).collect();
if let Err(e) = std::fs::write(out, lines) {
eprintln!("WARN: could not write the findings to {out}: {e}");
}
}
/// Each finding on its own line, FAIL or WARN first; the count of FAILs. /// Each finding on its own line, FAIL or WARN first; the count of FAILs.
fn print_findings(findings: &[needs::Finding]) -> usize { fn print_findings(findings: &[needs::Finding]) -> usize {
let mut failed = 0; let mut failed = 0;
@@ -157,6 +190,7 @@ fn print_findings(findings: &[needs::Finding]) -> usize {
/// then the wording tasks written out and/or an engine's answers /// then the wording tasks written out and/or an engine's answers
/// scored, when asked. A repo without `needs.yaml` skips all of it. /// scored, when asked. A repo without `needs.yaml` skips all of it.
fn check_needs( fn check_needs(
findings_out: Option<&str>,
raw: Option<&str>, raw: Option<&str>,
questions: &std::collections::HashMap<String, content::Question>, questions: &std::collections::HashMap<String, content::Question>,
aggregates_map: &std::collections::HashMap<String, aggregates::AggregateSchema>, aggregates_map: &std::collections::HashMap<String, aggregates::AggregateSchema>,
@@ -167,6 +201,7 @@ fn check_needs(
min_accuracy: f64, min_accuracy: f64,
) -> anyhow::Result<()> { ) -> anyhow::Result<()> {
let Some(raw) = raw else { let Some(raw) = raw else {
write_findings(findings_out, &[]);
if tasks_out.is_some() || answers_in.is_some() { if tasks_out.is_some() || answers_in.is_some() {
eprintln!("FAIL: --needs-tasks/--needs-score given, but the repo has no needs.yaml"); eprintln!("FAIL: --needs-tasks/--needs-score given, but the repo has no needs.yaml");
std::process::exit(1); std::process::exit(1);
@@ -177,10 +212,13 @@ fn check_needs(
Ok(file) => file, Ok(file) => file,
Err(e) => { Err(e) => {
eprintln!("FAIL: {e}"); eprintln!("FAIL: {e}");
write_findings(findings_out, &[schema_finding(&e.to_string())]);
std::process::exit(1); std::process::exit(1);
} }
}; };
let failed = print_findings(&needs::check(&file, questions, aggregates_map)); let findings = needs::check(&file, questions, aggregates_map);
let failed = print_findings(&findings);
write_findings(findings_out, &findings);
for need in &file.needs { for need in &file.needs {
if let Some(path) = needs::path_for(questions, need) { if let Some(path) = needs::path_for(questions, need) {
let steps: Vec<String> = path let steps: Vec<String> = path
+124 -15
View File
@@ -144,6 +144,48 @@ pub struct Finding {
/// "need" for a need's own path; "bucket" for what holds for every /// "need" for a need's own path; "bucket" for what holds for every
/// record in a bucket whichever need brought it in. /// record in a bucket whichever need brought it in.
pub scope: &'static str, pub scope: &'static str,
/// What kind of gap it is, stable across releases: `strand`,
/// `edge-unoffered`, `done-unreachable`, ... (docs/lint.md).
pub code: &'static str,
/// What in the content it is about: a state, a move (`from>to`), a
/// page. With the scope, need and code it makes the finding's id.
pub subject: String,
}
impl Finding {
/// The same gap keeps the same id however its message is worded,
/// so a fix round can tell what it closed, what is still open and
/// what it broke. FNV-1a over scope, need, code and subject.
pub fn id(&self) -> String {
let mut hash: u64 = 0xcbf2_9ce4_8422_2325;
for part in [self.scope, &self.need, self.code, &self.subject] {
for byte in part.bytes().chain(std::iter::once(0)) {
hash ^= u64::from(byte);
hash = hash.wrapping_mul(0x0100_0000_01b3);
}
}
format!("{:08x}", hash >> 32)
}
/// The change that closes it, when the lint knows one: what follows
/// "fix:" in the message.
pub fn fix(&self) -> Option<&str> {
self.message.split_once(" - fix: ").map(|(_, fix)| fix)
}
/// One line of `--findings` JSONL.
pub fn to_json(&self) -> serde_json::Value {
serde_json::json!({
"id": self.id(),
"level": match self.level { Level::Fail => "fail", Level::Warn => "warn" },
"code": self.code,
"scope": self.scope,
"need": self.need,
"subject": self.subject,
"message": self.message,
"fix": self.fix(),
})
}
} }
#[derive(Clone, Debug, PartialEq, Eq)] #[derive(Clone, Debug, PartialEq, Eq)]
@@ -354,10 +396,12 @@ pub fn check(
let mut strands_reported: HashSet<(String, String)> = HashSet::new(); let mut strands_reported: HashSet<(String, String)> = HashSet::new();
let mut unreached_reported: HashSet<(String, String)> = HashSet::new(); let mut unreached_reported: HashSet<(String, String)> = HashSet::new();
for need in &file.needs { for need in &file.needs {
let mut report = |level: Level, message: String| { let mut report = |level: Level, code: &'static str, subject: String, message: String| {
findings.push(Finding { findings.push(Finding {
need: need.id.clone(), need: need.id.clone(),
scope: "need", scope: "need",
code,
subject,
level: if need.planned || need.paused { Level::Warn } else { level }, level: if need.planned || need.paused { Level::Warn } else { level },
message: match (need.paused, need.planned, level == Level::Fail) { message: match (need.paused, need.planned, level == Level::Fail) {
(true, _, true) => format!("paused, not open now: {message}"), (true, _, true) => format!("paused, not open now: {message}"),
@@ -367,15 +411,17 @@ pub fn check(
}) })
}; };
if !seen.insert(need.id.as_str()) { if !seen.insert(need.id.as_str()) {
report(Level::Fail, "declared more than once".to_string()); report(Level::Fail, "need-duplicate", String::new(), "declared more than once".to_string());
continue; continue;
} }
// The visitor's side: can they get their answer recorded? // The visitor's side: can they get their answer recorded?
match questions.get(&need.enters) { match questions.get(&need.enters) {
None => report(Level::Fail, format!("entry page {:?} does not exist", need.enters)), None => report(Level::Fail, "entry-missing", need.enters.clone(), format!("entry page {:?} does not exist", need.enters)),
Some(page) if !visible(page, need.enters_as.as_deref()) => report( Some(page) if !visible(page, need.enters_as.as_deref()) => report(
Level::Fail, Level::Fail,
"entry-gated",
need.enters.clone(),
format!( format!(
"entry page {:?} is gated on {:?}, which this visitor does not hold", "entry page {:?} is gated on {:?}, which this visitor does not hold",
need.enters, page.qualifies need.enters, page.qualifies
@@ -385,6 +431,8 @@ pub fn check(
if page.responsible.is_none() { if page.responsible.is_none() {
report( report(
Level::Warn, Level::Warn,
"entry-no-responsible",
need.enters.clone(),
format!("entry page {:?} names no one responsible to turn to", need.enters), format!("entry page {:?} names no one responsible to turn to", need.enters),
); );
} }
@@ -412,6 +460,8 @@ pub fn check(
.unwrap_or_default(); .unwrap_or_default();
report( report(
Level::Fail, Level::Fail,
"path-closed",
at.clone(),
format!( format!(
"the path exists but is closed: {at} is `disabled`, so the answer is advertised and refused" "the path exists but is closed: {at} is `disabled`, so the answer is advertised and refused"
), ),
@@ -419,6 +469,8 @@ pub fn check(
} }
None => report( None => report(
Level::Fail, Level::Fail,
"path-missing",
String::new(),
format!( format!(
"no path from {:?}{} records into {:?} within {} submission(s)", "no path from {:?}{} records into {:?} within {} submission(s)",
need.enters, need.enters,
@@ -435,6 +487,8 @@ pub fn check(
if best.fields > budget { if best.fields > budget {
report( report(
Level::Fail, Level::Fail,
"path-over-budget",
String::new(),
format!( format!(
"the path asks for {} required field(s), over the budget of {budget}", "the path asks for {} required field(s), over the budget of {budget}",
best.fields best.fields
@@ -445,6 +499,8 @@ pub fn check(
if best.records_with_action == Some(false) { if best.records_with_action == Some(false) {
report( report(
Level::Warn, Level::Warn,
"path-no-action",
String::new(),
"the recording alternative has no `action` - the visitor is never told what happens next".to_string(), "the recording alternative has no `action` - the visitor is never told what happens next".to_string(),
); );
} }
@@ -457,6 +513,8 @@ pub fn check(
let Some(schema) = aggregates.get(&need.lands_in) else { let Some(schema) = aggregates.get(&need.lands_in) else {
report( report(
Level::Fail, Level::Fail,
"machine-missing",
need.lands_in.clone(),
format!("aggregates.yaml declares no state machine for {:?}", need.lands_in), format!("aggregates.yaml declares no state machine for {:?}", need.lands_in),
); );
continue; continue;
@@ -475,6 +533,8 @@ pub fn check(
.collect(); .collect();
report( report(
Level::Fail, Level::Fail,
"desk-missing",
need.handled_by.clone(),
format!( format!(
"no desk gated on {:?} reads {:?}{}", "no desk gated on {:?} reads {:?}{}",
need.handled_by, need.handled_by,
@@ -556,7 +616,7 @@ pub fn check(
match ways_in(state).first() { match ways_in(state).first() {
Some((from, to)) => format!(" - fix: {}", fix_for(from, to)), Some((from, to)) => format!(" - fix: {}", fix_for(from, to)),
None => format!( None => format!(
" - and no state a record reaches has a declared edge into it: add one in aggregates.yaml under {:?}", " - fix: no state a record reaches has a declared edge into it; add one in aggregates.yaml under {:?}",
need.lands_in need.lands_in
), ),
} }
@@ -565,12 +625,16 @@ pub fn check(
if !schema.has_state(state) { if !schema.has_state(state) {
report( report(
Level::Fail, Level::Fail,
"done-unknown",
state.clone(),
format!("done_when names {state:?}, which {:?} has no such state", need.lands_in), format!("done_when names {state:?}, which {:?} has no such state", need.lands_in),
); );
} else if !reached.contains(state) { } else if !reached.contains(state) {
unreached_reported.insert((need.lands_in.clone(), state.clone())); unreached_reported.insert((need.lands_in.clone(), state.clone()));
report( report(
Level::Fail, Level::Fail,
"done-unreachable",
state.clone(),
format!( format!(
"{:?} cannot move a record to {state:?} - no desk of theirs offers the moves{}", "{:?} cannot move a record to {state:?} - no desk of theirs offers the moves{}",
need.handled_by, need.handled_by,
@@ -583,6 +647,8 @@ pub fn check(
if !schema.has_state(stage) { if !schema.has_state(stage) {
report( report(
Level::Fail, Level::Fail,
"stage-unknown",
stage.clone(),
format!( format!(
"the business describes a stage {stage:?} that {:?} has no state for", "the business describes a stage {stage:?} that {:?} has no state for",
need.lands_in need.lands_in
@@ -592,6 +658,8 @@ pub fn check(
unreached_reported.insert((need.lands_in.clone(), stage.clone())); unreached_reported.insert((need.lands_in.clone(), stage.clone()));
report( report(
Level::Fail, Level::Fail,
"stage-unreachable",
stage.clone(),
format!( format!(
"stage {stage:?} exists but {:?} is offered no moves that reach it{}", "stage {stage:?} exists but {:?} is offered no moves that reach it{}",
need.handled_by, need.handled_by,
@@ -613,11 +681,13 @@ pub fn check(
if reached.contains(before) && reached.contains(after) && !reachable(before, &moves).contains(after) { if reached.contains(before) && reached.contains(after) && !reachable(before, &moves).contains(after) {
report( report(
Level::Fail, Level::Fail,
"stage-order",
format!("{before}>{after}"),
format!( format!(
"stage {after:?} comes after {before:?} for the business, but no move leads on from {before:?} to it{}", "stage {after:?} comes after {before:?} for the business, but no move leads on from {before:?} to it{}",
match schema.allowed(before).iter().find(|to| reachable(to, &moves).contains(after) || *to == after) { match schema.allowed(before).iter().find(|to| reachable(to, &moves).contains(after) || *to == after) {
Some(to) => format!(" - fix: {}", fix_for(before, to)), Some(to) => format!(" - fix: {}", fix_for(before, to)),
None => format!(" - add an edge from {before:?} toward {after:?} in aggregates.yaml under {:?}", need.lands_in), None => format!(" - fix: add an edge from {before:?} toward {after:?} in aggregates.yaml under {:?}", need.lands_in),
} }
), ),
); );
@@ -627,6 +697,8 @@ pub fn check(
if !need.done_when.contains(state) { if !need.done_when.contains(state) {
report( report(
Level::Fail, Level::Fail,
"success-not-done",
state.clone(),
format!("success names {state:?}, which is not one of done_when"), format!("success names {state:?}, which is not one of done_when"),
); );
} }
@@ -639,12 +711,14 @@ pub fn check(
let fix = match schema.allowed(&state).first() { let fix = match schema.allowed(&state).first() {
Some(to) => format!(" - fix: {}", fix_for(&state, to)), Some(to) => format!(" - fix: {}", fix_for(&state, to)),
None => format!( None => format!(
" - and {:?} declares no way out of it: add {state:?} to done_when if it is finished, or an edge out in aggregates.yaml", " - fix: {:?} declares no way out of it; add {state:?} to done_when if it is finished, or an edge out in aggregates.yaml",
need.lands_in need.lands_in
), ),
}; };
report( report(
Level::Fail, Level::Fail,
"strand",
state.clone(),
format!("records strand in {state:?}: {movers} is offered no move out of it{fix}"), format!("records strand in {state:?}: {movers} is offered no move out of it{fix}"),
); );
} }
@@ -658,6 +732,8 @@ pub fn check(
findings.push(Finding { findings.push(Finding {
need: persona.need.clone(), need: persona.need.clone(),
scope: "need", scope: "need",
code: "persona-unknown-need",
subject: persona.id.clone(),
level: Level::Fail, level: Level::Fail,
message: format!("persona {:?} points at a need that is not declared", persona.id), message: format!("persona {:?} points at a need that is not declared", persona.id),
}); });
@@ -690,10 +766,12 @@ fn operations(
// Only open needs make a gap fail; a bucket that only planned // Only open needs make a gap fail; a bucket that only planned
// or paused needs use reports it as a warning. // or paused needs use reports it as a warning.
let open = needs.iter().any(|n| !n.planned && !n.paused); let open = needs.iter().any(|n| !n.planned && !n.paused);
let mut report = |level: Level, message: String| { let mut report = |level: Level, code: &'static str, subject: String, message: String| {
findings.push(Finding { findings.push(Finding {
need: bucket.to_string(), need: bucket.to_string(),
scope: "bucket", scope: "bucket",
code,
subject,
level: if open { level } else { Level::Warn }, level: if open { level } else { Level::Warn },
message, message,
}) })
@@ -728,6 +806,8 @@ fn operations(
ways.sort(); ways.sort();
report( report(
Level::Fail, Level::Fail,
"state-unreached",
(*state).clone(),
format!( format!(
"state {state:?} is never reached: no desk, link or clock moves a record there - fix: {}", "state {state:?} is never reached: no desk, link or clock moves a record there - fix: {}",
match ways.first() { match ways.first() {
@@ -747,6 +827,8 @@ fn operations(
if !strands_reported.contains(&(bucket.to_string(), (*state).clone())) { if !strands_reported.contains(&(bucket.to_string(), (*state).clone())) {
report( report(
Level::Fail, Level::Fail,
"strand",
(*state).clone(),
format!( format!(
"records strand in {state:?}: {bucket:?} declares moves on to {} but no desk, link or clock offers one - fix: {}", "records strand in {state:?}: {bucket:?} declares moves on to {} but no desk, link or clock offers one - fix: {}",
out.join(", "), out.join(", "),
@@ -759,8 +841,10 @@ fn operations(
for to in out.iter().filter(|to| !offered(state, to)) { for to in out.iter().filter(|to| !offered(state, to)) {
report( report(
Level::Warn, Level::Warn,
"edge-unoffered",
format!("{state}>{to}"),
format!( format!(
"the move {state:?} -> {to:?} is declared but no desk, link or clock offers it - {}, or drop the edge from aggregates.yaml", "the move {state:?} -> {to:?} is declared but no desk, link or clock offers it - fix: {}, or drop the edge from aggregates.yaml",
button(state, to) button(state, to)
), ),
); );
@@ -771,8 +855,10 @@ fn operations(
if page_group != group { if page_group != group {
report( report(
Level::Warn, Level::Warn,
"desk-gated-twice",
desk.page.clone(),
format!( format!(
"the desk on {} is gated twice: the page qualifies {page_group:?}, its list requires {group:?} - only members of both see the buttons; make them the same group", "the desk on {} is gated twice: the page qualifies {page_group:?}, its list requires {group:?}, so only members of both see the buttons - fix: make them the same group",
desk.page desk.page
), ),
); );
@@ -793,7 +879,7 @@ fn operations(
mail_states.sort_by(|a, b| a.0.cmp(b.0)); mail_states.sort_by(|a, b| a.0.cmp(b.0));
for (state, mail) in mail_states { for (state, mail) in mail_states {
if !reached.contains(state) { if !reached.contains(state) {
report(Level::Warn, format!("the mail on entering {state:?} never goes out: no record reaches {state:?}")); report(Level::Warn, "mail-unreachable", state.clone(), format!("the mail on entering {state:?} never goes out: no record reaches {state:?} - fix: make {state:?} reachable, or drop its mail"));
} }
let mut unknown: Vec<String> = Vec::new(); let mut unknown: Vec<String> = Vec::new();
for key in placeholders(&mail.subject).into_iter().chain(placeholders(&mail.body)) { for key in placeholders(&mail.subject).into_iter().chain(placeholders(&mail.body)) {
@@ -811,8 +897,10 @@ fn operations(
names.dedup(); names.dedup();
report( report(
Level::Fail, Level::Fail,
"mail-unknown-field",
state.clone(),
format!( format!(
"the mail on entering {state:?} uses {}, which no form recording into {bucket:?} collects - it would go out blank there; the forms collect: {}", "the mail on entering {state:?} uses {}, which no form recording into {bucket:?} collects, so it would go out blank there - fix: use one of the fields the forms collect ({}), or ask for it on the form",
unknown.iter().map(|k| format!("{{{k}}}")).collect::<Vec<_>>().join(", "), unknown.iter().map(|k| format!("{{{k}}}")).collect::<Vec<_>>().join(", "),
if names.is_empty() { "nothing".to_string() } else { names.join(", ") } if names.is_empty() { "nothing".to_string() } else { names.join(", ") }
), ),
@@ -1407,7 +1495,7 @@ mod tests {
assert!(fails(&findings).is_empty(), "{findings:?}"); assert!(fails(&findings).is_empty(), "{findings:?}");
assert_eq!( assert_eq!(
warns(&findings), warns(&findings),
vec!["the move \"open\" -> \"lost\" is declared but no desk, link or clock offers it - add `{ from: open, to: lost, label: ... }` to the transitions on /desk, or drop the edge from aggregates.yaml"] vec!["the move \"open\" -> \"lost\" is declared but no desk, link or clock offers it - fix: add `{ from: open, to: lost, label: ... }` to the transitions on /desk, or drop the edge from aggregates.yaml"]
); );
} }
@@ -1422,7 +1510,7 @@ mod tests {
let findings = check(&needs(" stages: [open, surveyed, quoted, won, lost]\n"), &questions, &aggregates); let findings = check(&needs(" stages: [open, surveyed, quoted, won, lost]\n"), &questions, &aggregates);
assert_eq!( assert_eq!(
fails(&findings), fails(&findings),
vec!["stage \"quoted\" comes after \"surveyed\" for the business, but no move leads on from \"surveyed\" to it - add an edge from \"surveyed\" toward \"quoted\" in aggregates.yaml under \"projects\""] vec!["stage \"quoted\" comes after \"surveyed\" for the business, but no move leads on from \"surveyed\" to it - fix: add an edge from \"surveyed\" toward \"quoted\" in aggregates.yaml under \"projects\""]
); );
} }
@@ -1435,7 +1523,7 @@ mod tests {
let findings = check(&needs(""), &questions, &aggregates); let findings = check(&needs(""), &questions, &aggregates);
assert_eq!( assert_eq!(
fails(&findings), fails(&findings),
vec!["the mail on entering \"open\" uses {refnr}, which no form recording into \"projects\" collects - it would go out blank there; the forms collect: brief, budget, email"] vec!["the mail on entering \"open\" uses {refnr}, which no form recording into \"projects\" collects, so it would go out blank there - fix: use one of the fields the forms collect (brief, budget, email), or ask for it on the form"]
); );
} }
@@ -1445,8 +1533,29 @@ mod tests {
questions.get_mut("/desk").unwrap().qualifies = Some("staff".into()); questions.get_mut("/desk").unwrap().qualifies = Some("staff".into());
let findings = check(&needs(""), &questions, &aggregates); let findings = check(&needs(""), &questions, &aggregates);
assert!( assert!(
warns(&findings).contains(&"the desk on /desk is gated twice: the page qualifies \"staff\", its list requires \"owners\" - only members of both see the buttons; make them the same group"), warns(&findings).contains(&"the desk on /desk is gated twice: the page qualifies \"staff\", its list requires \"owners\", so only members of both see the buttons - fix: make them the same group"),
"{findings:?}" "{findings:?}"
); );
} }
#[test]
fn a_finding_keeps_its_id_when_reworded_and_says_its_fix() {
let (questions, aggregates) = site(" - { to: quoted, label: Quote }\n - { to: lost, label: Lose }\n");
let findings = check(&needs(""), &questions, &aggregates);
let strand = findings.iter().find(|f| f.code == "strand").unwrap();
assert_eq!(strand.subject, "quoted");
assert_eq!(strand.fix(), Some("add `{ from: quoted, to: won, label: ... }` to the transitions on /desk"));
let reworded = Finding { message: "something else entirely".into(), ..clone_of(strand) };
assert_eq!(reworded.id(), strand.id());
let elsewhere = Finding { subject: "open".into(), ..clone_of(strand) };
assert_ne!(elsewhere.id(), strand.id());
let json = strand.to_json();
assert_eq!(json["code"], "strand");
assert_eq!(json["level"], "fail");
assert_eq!(json["id"], strand.id());
}
fn clone_of(f: &Finding) -> Finding {
Finding { need: f.need.clone(), level: f.level, message: f.message.clone(), scope: f.scope, code: f.code, subject: f.subject.clone() }
}
} }
+68 -3
View File
@@ -347,9 +347,29 @@ async fn fetch_resource_value(
ResourceSource::Url { url } => fetch_url_json(url, params).await?, ResourceSource::Url { url } => fetch_url_json(url, params).await?,
}; };
match &resource.jq { let shaped = match &resource.jq {
Some(filter) => apply_jq(filter, &value), Some(filter) => apply_jq(filter, &value)?,
None => Ok(value), None => value,
};
Ok(apply_limit(resource.limit, shaped))
}
/// Keep at most `n` items, after `jq` has had its say.
///
/// Anything that is not a list passes through: a single record has no
/// length to cap, and silently turning one into an empty list would be
/// worse than ignoring a limit that cannot apply. A limit of zero does
/// mean nothing — the resource's `empty` line speaks instead, which is
/// a coherent thing for content to ask for.
#[cfg(feature = "ssr")]
fn apply_limit(limit: Option<usize>, value: serde_json::Value) -> serde_json::Value {
let Some(n) = limit else { return value };
match value {
serde_json::Value::Array(mut items) => {
items.truncate(n);
serde_json::Value::Array(items)
}
other => other,
} }
} }
@@ -664,6 +684,51 @@ mod tests {
assert!(!matches_filter(&fixed, &record("open", serde_json::json!({ "which_authority": "Fire service" })), &params)); assert!(!matches_filter(&fixed, &record("open", serde_json::json!({ "which_authority": "Fire service" })), &params));
} }
/// `limit` caps what the page renders, and does it after `jq` so
/// content can order with the filter and take the top of its own
/// ordering. The releases card is the case: the newest cut is the
/// one you should run, and a page that grows a card per release
/// gets longer forever.
#[test]
fn limit_caps_a_list_after_the_filter() {
let releases = serde_json::json!([
{"name": "redoal 0.1.17", "tag_name": "v0.1.17", "body": "newest"},
{"name": "redoal 0.1.16", "tag_name": "v0.1.16", "body": "older"},
{"name": "redoal 0.1.15", "tag_name": "v0.1.15", "body": "oldest"}
]);
// The filter redoal/questions actually declares: bare objects,
// never a wrapped array — the frontend re-wraps.
let filter = ".[] | {name: .name, description: .body, tag: .tag_name, url: null}";
let shaped = apply_jq(filter, &releases).expect("filter runs");
let all = apply_limit(None, shaped.clone());
assert_eq!(all.as_array().map(Vec::len), Some(3), "no limit shows everything");
let latest = apply_limit(Some(1), shaped.clone());
let rows = latest.as_array().expect("still a list");
assert_eq!(rows.len(), 1);
assert_eq!(rows[0]["tag"], "v0.1.17", "and it is the newest, which Gitea returns first");
let two = apply_limit(Some(2), shaped.clone());
assert_eq!(two.as_array().map(Vec::len), Some(2));
// Asking for more than there is is not an error.
let plenty = apply_limit(Some(99), shaped);
assert_eq!(plenty.as_array().map(Vec::len), Some(3));
}
/// A limit that cannot apply is ignored rather than guessed at.
#[test]
fn limit_leaves_a_single_record_alone() {
let one = serde_json::json!({"name": "a record", "state": "open"});
assert_eq!(apply_limit(Some(1), one.clone()), one);
assert_eq!(apply_limit(Some(0), one.clone()), one, "no length to cap");
// Zero on a list does mean nothing — the `empty` line speaks.
let list = serde_json::json!([{"a": 1}, {"a": 2}]);
assert_eq!(apply_limit(Some(0), list).as_array().map(Vec::len), Some(0));
}
/// A canned, Gitea-API-shaped fixture - the same fields the real /// A canned, Gitea-API-shaped fixture - the same fields the real
/// `/users/{username}/starred` endpoint returns (Gitea 1.27: /// `/users/{username}/starred` endpoint returns (Gitea 1.27:
/// `stars_count`, not GitHub's `stargazers_count`; `website` is "" /// `stars_count`, not GitHub's `stargazers_count`; `website` is ""
+251 -4
View File
@@ -132,23 +132,43 @@ pub fn run(argv: Vec<String>) -> anyhow::Result<()> {
.to_string(); .to_string();
let mut green = false; let mut green = false;
let mut report = String::new(); let mut report = String::new();
// What the lint found, by id, round over round (`--findings`): what
// a round closed, what it left, what it broke. `None` while the
// lint is one that predates `--findings`; then the whole report
// goes back, as it always did.
let mut ledger = Ledger::default();
let mut tracked = true;
let mut undone = String::new();
std::fs::create_dir_all(opts.out.join("findings"))?;
for round in 1..=opts.rounds { for round in 1..=opts.rounds {
let prompt = if round == 1 { let prompt = if round == 1 {
format!( format!(
"{REFERENCE}\n\n# Task\n\nWrite the content repo that meets this frozen `needs.yaml`: `aggregates.yaml`, `site.yaml`, and every page under `questions/` - the front door, followup pages, role-gated pages, one desk per handling group. Do not return needs.yaml.\n\n## Intake\n\n{intake}\n\n## needs.yaml (frozen)\n\n```yaml\n{needs}\n```\n\n{FORMAT}" "{REFERENCE}\n\n# Task\n\nWrite the content repo that meets this frozen `needs.yaml`: `aggregates.yaml`, `site.yaml`, and every page under `questions/` - the front door, followup pages, role-gated pages, one desk per handling group. Do not return needs.yaml.\n\n## Intake\n\n{intake}\n\n## needs.yaml (frozen)\n\n```yaml\n{needs}\n```\n\n{FORMAT}"
) )
} else if tracked {
let (heading, files) = match named_files(&opts.out, ledger.open.values()) {
Some(files) => ("Files the findings name", dump_files(&opts.out, &files)?),
None => ("Current files", dump(&opts.out)?),
};
format!(
"{REFERENCE}\n\n# Task\n\nThe verifier rejected the repo. Below is every finding still open, each with the change that closes it. Make those changes and nothing else: findings already closed stay closed. Return only the files you change, each complete. To remove a file, return it with the single line DELETE. needs.yaml is frozen; do not return it.\n\n## Open findings\n\n{}\n{undone}## needs.yaml (frozen)\n\n```yaml\n{needs}\n```\n\n## {heading}\n\n{files}\n\n{FORMAT}",
ledger.open_list(round)
)
} else { } else {
format!( format!(
"{REFERENCE}\n\n# Task\n\nThe verifier rejected the repo below. Fix it. Return only the files you change, each complete. To remove a file, return it with the single line DELETE. needs.yaml is frozen; do not return it.\n\n## Verifier output\n\n```\n{report}\n```\n\n## needs.yaml (frozen)\n\n```yaml\n{needs}\n```\n\n## Current files\n\n{}\n\n{FORMAT}", "{REFERENCE}\n\n# Task\n\nThe verifier rejected the repo below. Fix it. Return only the files you change, each complete. To remove a file, return it with the single line DELETE. needs.yaml is frozen; do not return it.\n\n## Verifier output\n\n```\n{report}\n```\n\n## needs.yaml (frozen)\n\n```yaml\n{needs}\n```\n\n## Current files\n\n{}\n\n{FORMAT}",
dump(&opts.out)? dump(&opts.out)?
) )
}; };
let before = snapshot(&opts.out)?;
match parse_files(&llm(&opts.llm, &prompt)?) { match parse_files(&llm(&opts.llm, &prompt)?) {
Ok(files) => apply(&opts.out, files)?, Ok(files) => apply(&opts.out, files)?,
Err(_) => eprintln!("structure round {round}: the reply had no file blocks"), Err(_) => eprintln!("structure round {round}: the reply had no file blocks"),
} }
let (ok, r) = lint(&opts, &[])?; let (ok, r, found) = lint_findings(&opts, &opts.out.join(format!("findings/round-{round}.jsonl")))?;
report = r; report = r;
let Some(found) = found else {
tracked = false;
let fails = report.lines().filter(|l| l.starts_with("FAIL")).count(); let fails = report.lines().filter(|l| l.starts_with("FAIL")).count();
eprintln!("structure round {round}: {}", if ok { "green".to_string() } else { format!("{fails} FAIL line(s)") }); eprintln!("structure round {round}: {}", if ok { "green".to_string() } else { format!("{fails} FAIL line(s)") });
log.push(format!("structure round {round}: ok={ok} fails={fails}")); log.push(format!("structure round {round}: ok={ok} fails={fails}"));
@@ -156,10 +176,47 @@ pub fn run(argv: Vec<String>) -> anyhow::Result<()> {
green = true; green = true;
break; break;
} }
continue;
};
let previous = ledger.clone();
let progress = ledger.update(round, found.into_iter().filter(|f| f.level == "fail").collect());
let line = format!("structure round {round}: {}", progress.summary(ledger.open.len()));
eprintln!("{line}");
log.push(line);
undone.clear();
if ok {
green = true;
break;
}
if round == 1 {
continue;
}
// A fix that reopens what an earlier round closed is undone,
// and the model is told what it broke, rather than chasing the
// same findings round after round.
if !progress.reopened.is_empty() {
restore(&opts.out, before)?;
ledger = previous;
undone = format!(
"## Your last change was undone\n\nIt reopened findings an earlier round had closed:\n\n{}\nClose the open findings without touching what made these pass.\n\n",
progress.reopened.iter().map(|f| format!("- [{}] {}\n", f.id, f.message)).collect::<String>()
);
eprintln!("structure round {round}: reopened {} closed finding(s), undone", progress.reopened.len());
log.push(format!("structure round {round}: undone (reopened {})", progress.reopened.len()));
continue;
}
// No progress, no more rounds: another pass at the same findings
// is how a site gets over-trained. What is left is reported.
if progress.fixed.is_empty() || ledger.open.len() >= previous.open.len() {
eprintln!("structure round {round}: no finding closed, stopping");
log.push(format!("structure round {round}: stalled"));
break;
}
} }
if !green { if !green {
std::fs::write(opts.out.join("run.log"), format!("{}\n\n{report}", log.join("\n")))?; let open = if tracked { format!("\n\nStill open:\n{}", ledger.open_list(opts.rounds + 1)) } else { String::new() };
bail!("structure never went green in {} round(s); last verifier output is in run.log", opts.rounds); std::fs::write(opts.out.join("run.log"), format!("{}{open}\n\n{report}", log.join("\n")))?;
bail!("structure did not go green; what is still open is in run.log");
} }
// Phase 3: wording. // Phase 3: wording.
@@ -191,7 +248,10 @@ pub fn run(argv: Vec<String>) -> anyhow::Result<()> {
} }
eprintln!("wording round {round}: {}", summary.join(" | ")); eprintln!("wording round {round}: {}", summary.join(" | "));
log.push(format!("wording round {round}: {}", summary.join(" | "))); log.push(format!("wording round {round}: {}", summary.join(" | ")));
if defects.is_empty() || round == opts.wording_rounds { // A visitor who chose right, hesitantly, is not a defect worth a
// rewrite on its own: only a misroute is. Hesitations ride along
// as context when there is one.
if !defects.contains("MISROUTE") || round == opts.wording_rounds {
break; break;
} }
let personas = needs_full.split("\npersonas:").nth(1).unwrap_or_default(); let personas = needs_full.split("\npersonas:").nth(1).unwrap_or_default();
@@ -311,6 +371,152 @@ fn lint(opts: &Opts, extra: &[String]) -> anyhow::Result<(bool, String)> {
Ok((output.status.success(), report)) Ok((output.status.success(), report))
} }
/// One finding as the lint's `--findings` writes it (docs/lint.md).
#[derive(Clone, Debug, serde::Deserialize)]
struct Finding {
id: String,
level: String,
code: String,
#[serde(default)]
need: String,
message: String,
#[serde(default)]
fix: Option<String>,
}
/// The lint with `--findings` into `out`, and what it found; `None` for
/// the findings when the lint predates the flag (then it is run again
/// the old way, and only its text comes back).
fn lint_findings(opts: &Opts, out: &Path) -> anyhow::Result<(bool, String, Option<Vec<Finding>>)> {
let (ok, report) = lint(opts, &["--findings".to_string(), out.display().to_string()])?;
if report.contains("unknown argument: --findings") {
let (ok, report) = lint(opts, &[])?;
return Ok((ok, report, None));
}
let raw = std::fs::read_to_string(out).unwrap_or_default();
let found = raw
.lines()
.filter(|l| !l.trim().is_empty())
.map(|l| serde_json::from_str::<Finding>(l).with_context(|| format!("reading {}", out.display())))
.collect::<anyhow::Result<Vec<_>>>()?;
Ok((ok, report, Some(found)))
}
/// Every failing finding by id, across rounds.
#[derive(Clone, Default)]
struct Ledger {
open: BTreeMap<String, Finding>,
/// The round each open finding was first seen in.
since: BTreeMap<String, usize>,
/// Ids a round has closed, so one that comes back is a regression,
/// not news.
closed: std::collections::BTreeSet<String>,
}
struct Progress {
fixed: Vec<Finding>,
new: Vec<Finding>,
still: usize,
reopened: Vec<Finding>,
}
impl Progress {
fn summary(&self, open: usize) -> String {
if open == 0 {
return format!("green (fixed {})", self.fixed.len());
}
format!(
"{open} open - fixed {}, still open {}, new {}, reopened {}",
self.fixed.len(),
self.still,
self.new.len(),
self.reopened.len()
)
}
}
impl Ledger {
fn update(&mut self, round: usize, fails: Vec<Finding>) -> Progress {
let now: BTreeMap<String, Finding> = fails.into_iter().map(|f| (f.id.clone(), f)).collect();
let fixed: Vec<Finding> = self.open.values().filter(|f| !now.contains_key(&f.id)).cloned().collect();
let mut new = Vec::new();
let mut reopened = Vec::new();
let mut still = 0;
for (id, f) in &now {
if self.open.contains_key(id) {
still += 1;
} else if self.closed.contains(id) {
reopened.push(f.clone());
} else {
new.push(f.clone());
}
self.since.entry(id.clone()).or_insert(round);
}
for f in &fixed {
self.closed.insert(f.id.clone());
self.since.remove(&f.id);
}
self.open = now;
Progress { fixed, new, still, reopened }
}
/// The open findings for a prompt or a log: id, how long open, the
/// message and its fix.
fn open_list(&self, round: usize) -> String {
self.open
.values()
.map(|f| {
let age = match self.since.get(&f.id) {
Some(&first) if first + 1 < round => format!(", open since round {first}"),
_ => String::new(),
};
let scope = if f.need.is_empty() { String::new() } else { format!(" {:?}", f.need) };
format!("- [{}] {}{scope}{age}: {}\n", f.id, f.code, f.message)
})
.collect()
}
}
/// The files open findings point at - every desk page their fixes name,
/// plus aggregates.yaml - when that is enough to act on. `None` when a
/// finding is about getting around the site (a path, an entry page, a
/// missing desk) or names no page: then the model needs to see it all.
fn named_files<'a>(out: &Path, findings: impl Iterator<Item = &'a Finding>) -> Option<Vec<PathBuf>> {
const WHOLE_SITE: &[&str] = &["schema", "entry-missing", "entry-gated", "path-missing", "path-closed", "path-over-budget", "desk-missing", "need-duplicate"];
let mut files = vec![out.join("aggregates.yaml")];
for f in findings {
if WHOLE_SITE.contains(&f.code.as_str()) {
return None;
}
let text = format!("{} {}", f.message, f.fix.as_deref().unwrap_or_default());
for page in text.split_whitespace().filter(|w| w.starts_with('/')) {
let page = page.trim_end_matches(|c: char| !(c.is_alphanumeric() || c == '_' || c == '-' || c == '/'));
let rel = page.trim_start_matches('/');
let candidates = if rel.is_empty() {
vec![out.join("questions/index.yaml")]
} else {
vec![out.join(format!("questions/{rel}.yaml")), out.join(format!("questions/{rel}/index.yaml"))]
};
if let Some(path) = candidates.into_iter().find(|p| p.exists()) {
if !files.contains(&path) {
files.push(path);
}
}
}
}
Some(files)
}
/// `dump`, for the named files only.
fn dump_files(out: &Path, files: &[PathBuf]) -> anyhow::Result<String> {
let mut text = String::new();
for path in files.iter().filter(|p| p.exists()) {
let rel = path.strip_prefix(out).unwrap_or(path).display().to_string();
text.push_str(&format!("<<<FILE {rel}>>>\n{}<<<END>>>\n\n", std::fs::read_to_string(path)?));
}
Ok(text)
}
/// `<<<FILE path>>> ... <<<END>>>` blocks. Anything outside a block is /// `<<<FILE path>>> ... <<<END>>>` blocks. Anything outside a block is
/// ignored, so a model that adds a sentence does no harm. /// ignored, so a model that adds a sentence does no harm.
fn parse_files(text: &str) -> anyhow::Result<BTreeMap<String, String>> { fn parse_files(text: &str) -> anyhow::Result<BTreeMap<String, String>> {
@@ -591,3 +797,44 @@ mod tests {
assert!(!allowed(".gitea/workflows/deploy.yaml")); assert!(!allowed(".gitea/workflows/deploy.yaml"));
} }
} }
#[cfg(test)]
mod ledger_tests {
use super::*;
fn f(id: &str, code: &str, message: &str) -> Finding {
Finding { id: id.into(), level: "fail".into(), code: code.into(), need: "hire".into(), message: message.into(), fix: None }
}
#[test]
fn a_round_is_told_what_it_closed_kept_added_and_broke() {
let mut ledger = Ledger::default();
let p = ledger.update(1, vec![f("a", "strand", "x"), f("b", "strand", "y")]);
assert_eq!((p.fixed.len(), p.new.len(), p.still, p.reopened.len()), (0, 2, 0, 0));
let p = ledger.update(2, vec![f("b", "strand", "y"), f("c", "edge-unoffered", "z")]);
assert_eq!((p.fixed.len(), p.new.len(), p.still, p.reopened.len()), (1, 1, 1, 0));
assert_eq!(p.summary(ledger.open.len()), "2 open - fixed 1, still open 1, new 1, reopened 0");
// "a" was closed in round 2; its return is a regression.
let p = ledger.update(3, vec![f("a", "strand", "x"), f("b", "strand", "y")]);
assert_eq!(p.reopened.iter().map(|f| f.id.as_str()).collect::<Vec<_>>(), ["a"]);
assert!(ledger.open_list(4).contains("[b] strand \"hire\", open since round 1: y"));
let p = ledger.update(4, vec![]);
assert_eq!(p.summary(ledger.open.len()), "green (fixed 2)");
}
#[test]
fn a_fix_round_sees_the_pages_its_findings_name() {
let dir = std::env::temp_dir().join(format!("ledger-test-{}", std::process::id()));
std::fs::create_dir_all(dir.join("questions/desk")).unwrap();
std::fs::write(dir.join("aggregates.yaml"), "x").unwrap();
std::fs::write(dir.join("questions/desk/index.yaml"), "x").unwrap();
std::fs::write(dir.join("questions/index.yaml"), "x").unwrap();
let mut strand = f("a", "strand", "records strand in \"quoted\"");
strand.fix = Some("add `{ from: quoted, to: won, label: ... }` to the transitions on /desk".into());
let files = named_files(&dir, [&strand].into_iter()).unwrap();
assert_eq!(files, vec![dir.join("aggregates.yaml"), dir.join("questions/desk/index.yaml")]);
// A finding about getting around the site needs all of it.
assert!(named_files(&dir, [&strand, &f("b", "path-missing", "no path")].into_iter()).is_none());
std::fs::remove_dir_all(&dir).ok();
}
}
+6
View File
@@ -1294,6 +1294,12 @@ textarea:focus {
cursor: pointer; cursor: pointer;
} }
/* A releases manifest: the pick for this device, the rest folded. */
.releases .pill { display: inline-block; }
.releases pre { white-space: pre-wrap; font-size: 0.85rem; margin: 0.4rem 0 0; }
.releases-all summary { color: var(--ink-dim); font-size: 0.9rem; cursor: pointer; }
.releases-all ul { margin: 0.4rem 0 0 1rem; font-size: 0.9rem; }
.show-older:hover { .show-older:hover {
color: inherit; color: inherit;
border-color: var(--ink-dim); border-color: var(--ink-dim);