Compare commits
| Author | SHA1 | Date | |
|---|---|---|---|
|
|
c9faaeb4a6 | ||
|
|
1a2c064d3a | ||
|
|
fe7a868b18 | ||
|
|
aee5048b01 | ||
|
|
268d545843 | ||
|
|
22d086f9e1 | ||
|
|
48ae5e07c2 | ||
|
|
b619926a11 | ||
|
|
706c8108df |
@@ -4,6 +4,23 @@ One line per change, grouped by the release that shipped it. Newest
|
||||
first. A release is a Cargo.toml bump plus a `v<version>` tag; the
|
||||
content repos pin the tag they run (see the end of this file).
|
||||
|
||||
## 0.6.21 (2026-10-07)
|
||||
|
||||
- A content repo's `.well-known/` is served at the site root (`/.well-known/security.txt`, `/.well-known/assetlinks.json`, `/.well-known/matrix/server`), and a miss there is a 404 instead of the app shell answering 200 with an HTML page. RFC 8615 reserves the prefix and a question's id cannot begin with a dot, so nothing a repo ships can shadow a page. An extensionless file under it is typed as JSON, by the convention those files follow; `.txt`, `.xml`, `.webmanifest` and `.mobileconfig` are typed too, the last so an Apple configuration profile offers to install rather than download.
|
||||
|
||||
## 0.6.20 (2026-10-05)
|
||||
|
||||
- A releases card lets a platform's file carry an architecture: a Mac visitor is offered the `macos-arm64` file, not sent to TestFlight past it.
|
||||
|
||||
## 0.6.19 (2026-10-05)
|
||||
|
||||
- A `kind: url` resource whose JSON is a releases manifest (`latest`, `versions` with per-platform `assets`, `channels` such as a package repo or TestFlight) renders as a download card: the file for the visitor's own platform first, decided in the browser, the rest and the channels folded. Written for redoal's public releases at dl.redoal.com (docs: content-reference, "A releases manifest").
|
||||
|
||||
## 0.6.18 (2026-10-03)
|
||||
|
||||
- `iris check --findings <out.jsonl>` writes every finding as JSON: a stable `id` (scope, need, code and subject, nothing of the wording), `level`, `code`, `scope`, `need`, `subject`, `message` and `fix`. Every message that knows its fix now says it after "fix:". The codes are listed in docs/lint.md.
|
||||
- `portal generate` follows its findings through: each structure round is logged as fixed, still open, new and reopened; a fix round is shown the open findings with their fixes and only the files they name; a change that reopens a closed finding is undone; and the phase stops when a round closes nothing, writing what is left to run.log. A hesitant visitor no longer triggers a wording rewrite by itself, only a misroute does. A lint without `--findings` is still driven the old way.
|
||||
|
||||
## 0.6.17 (2026-10-02)
|
||||
|
||||
- A desk shows every record still moving and only the three most recently finished (a final state: no move out of it), with "Show older" for three more at a time. A desk that has run for a year is still about the work in hand.
|
||||
|
||||
Generated
+1
-1
@@ -3357,7 +3357,7 @@ dependencies = [
|
||||
|
||||
[[package]]
|
||||
name = "portal"
|
||||
version = "0.6.17"
|
||||
version = "0.6.21"
|
||||
dependencies = [
|
||||
"anyhow",
|
||||
"arc-swap",
|
||||
|
||||
+2
-1
@@ -1,6 +1,6 @@
|
||||
[package]
|
||||
name = "portal"
|
||||
version = "0.6.17"
|
||||
version = "0.6.21"
|
||||
edition = "2021"
|
||||
|
||||
[lib]
|
||||
@@ -60,6 +60,7 @@ web-sys = { version = "0.3", features = [
|
||||
"FileList",
|
||||
"FormData",
|
||||
"HtmlInputElement",
|
||||
"Navigator",
|
||||
"Request",
|
||||
"RequestInit",
|
||||
"Response",
|
||||
|
||||
@@ -255,6 +255,7 @@ reads is never a client-supplied parameter.
|
||||
states: [approved, refused] # only these; empty means all
|
||||
where: { field: case, equals: "{key}" }
|
||||
jq: ".[] | {name, url: .html_url}"
|
||||
limit: 1 # at most this many, after jq
|
||||
empty: Nothing decided yet.
|
||||
link: { to: "/notices/{id}", label: Answer this } # a page per record
|
||||
transitions: # the desk's buttons
|
||||
@@ -264,6 +265,14 @@ reads is never a client-supplied parameter.
|
||||
`source.kind` is one of `kv`, `gitea_starred`,
|
||||
`gitea_releases`, `url`.
|
||||
|
||||
`limit: 1` shows only the first item — for a feed where only the newest
|
||||
matters. A releases card wants the cut someone should run, not every
|
||||
cut there has ever been, and a page that grows by one card per release
|
||||
gets longer forever. It applies *after* `jq`, so a filter can reorder
|
||||
and the limit takes the top of that ordering. A source that yields a
|
||||
single record has no length and ignores it; `limit: 0` on a list shows
|
||||
nothing, and the `empty` line speaks instead.
|
||||
|
||||
A spec with neither `public: true` nor `requires_group` is
|
||||
unreachable: reads fail closed, and mutations always need a group
|
||||
whatever `public` says.
|
||||
@@ -293,6 +302,18 @@ stays about the work in hand however long the site has run; nothing
|
||||
is hidden from a read-only list, and `leaves_after` still takes a
|
||||
record out of every list altogether.
|
||||
|
||||
A **releases manifest** - a `kind: url` resource whose JSON is
|
||||
`{ name, latest, versions: [{ version, date, notes, assets: [{ platform,
|
||||
url, sha256, size }] }], channels: { arch: { repo, package, note },
|
||||
ios: { testflight, note } } }`, newest version first, as a release
|
||||
pipeline writes it - renders as one card: the newest version, the file
|
||||
for the visitor's own platform first (read from the browser: `ios`,
|
||||
`macos`, `linux-x86_64`, `linux-aarch64`, `windows`, `android`), the
|
||||
other files and the channels folded under "All downloads". Linux gets
|
||||
the package repo lines beside its file; iPhones and Macs get
|
||||
TestFlight. Platforms a manifest lacks fall through to the folded
|
||||
list.
|
||||
|
||||
`states` and `where` narrow a listing. They matter most for a register
|
||||
that is public by law — public about what was decided, not about what
|
||||
was withdrawn — while the desk working the same bucket is a different
|
||||
|
||||
@@ -50,6 +50,28 @@ already has the button, or the edge aggregates.yaml lacks. A schema
|
||||
error no longer hides the needs findings - both are printed, so one
|
||||
round can fix them together.
|
||||
|
||||
`--findings <out.jsonl>` writes the same findings one JSON object per
|
||||
line: `id`, `level` (`fail`/`warn`), `code`, `scope` (`need`,
|
||||
`bucket`, or `repo` for a schema error), `need` (the need's id, or the
|
||||
bucket's name), `subject` (the state, the move as `from>to`, or the
|
||||
page), `message` and `fix`. The file is written empty when the content
|
||||
is clean. `id` hashes scope, need, code and subject and nothing of the
|
||||
wording, so the same gap keeps its id from one run to the next: a
|
||||
trainer can tell what a round closed, what is still open and what it
|
||||
broke. The codes:
|
||||
|
||||
| code | |
|
||||
|---|---|
|
||||
| `schema` | content the site would refuse to load |
|
||||
| `need-duplicate`, `persona-unknown-need` | needs.yaml itself |
|
||||
| `entry-missing`, `entry-gated`, `entry-no-responsible` | the page a visitor arrives on |
|
||||
| `path-missing`, `path-closed`, `path-over-budget`, `path-no-action` | getting the answer recorded |
|
||||
| `machine-missing`, `desk-missing`, `desk-gated-twice` | who works the bucket |
|
||||
| `done-unknown`, `done-unreachable`, `success-not-done` | finishing a case |
|
||||
| `stage-unknown`, `stage-unreachable`, `stage-order` | the business's own stages |
|
||||
| `state-unreached`, `strand`, `edge-unoffered` | every state and move of the bucket |
|
||||
| `mail-unreachable`, `mail-unknown-field` | what a state mails |
|
||||
|
||||
A need marked `planned` warns instead of failing. Exit status is the
|
||||
verdict, so a content repo's CI is one line.
|
||||
|
||||
|
||||
+16
-5
@@ -243,13 +243,24 @@ against the lint. It is kept for the record of what it measured.
|
||||
visitor will tolerate.
|
||||
2. **Structure.** Pages, state machines and desks are generated, then
|
||||
revised against the lint, `iris check`, until every need is met:
|
||||
a path within budget, a desk held by the right group, every
|
||||
finished state reachable, no record stranding. The page-writing
|
||||
model sees the needs and never the personas.
|
||||
a path within budget, a desk held by the right group, every state
|
||||
of every bucket reachable and none stranding a record. The
|
||||
page-writing model sees the needs and never the personas. Each
|
||||
round the lint writes its findings with stable ids (`--findings`,
|
||||
kept in `<out>/findings/round-N.jsonl`), and the round is logged as
|
||||
what it fixed, what is still open, what is new and what it
|
||||
reopened. A fix round is shown only the open findings, each with
|
||||
the change that closes it, and the files they name (the whole repo
|
||||
when a finding is about getting around the site). A change that
|
||||
reopens a closed finding is undone and the model told what it
|
||||
broke. A round that closes nothing, or leaves as many open as it
|
||||
found, ends the phase: what is left is written to `run.log` rather
|
||||
than polished at again.
|
||||
3. **Wording.** A separate model plays each persona, blind, first with
|
||||
headings only and then with descriptions. A misroute is a wording
|
||||
defect and goes back to the generator. A rewrite that breaks
|
||||
structure is reverted.
|
||||
defect and goes back to the generator; a visitor who chose right
|
||||
but hesitated is reported and rewrites nothing by itself. A rewrite
|
||||
that breaks structure is reverted.
|
||||
|
||||
Model output is only ever written to `aggregates.yaml`, `site.yaml`
|
||||
and `questions/**.yaml` inside the output directory.
|
||||
|
||||
+163
@@ -2124,9 +2124,148 @@ fn ResourceValue(
|
||||
}
|
||||
}
|
||||
|
||||
// A releases manifest (content-reference, "A releases manifest"):
|
||||
// the newest version, its files per platform, and the standing
|
||||
// channels, with the visitor's own platform picked in the browser.
|
||||
if is_releases_manifest(&value) {
|
||||
return view! { <ReleasesCard manifest=value/> }.into_any();
|
||||
}
|
||||
|
||||
view! { <pre class="resource-raw">{value.to_string()}</pre> }.into_any()
|
||||
}
|
||||
|
||||
/// `{ name, latest, versions: [{ version, date, notes, assets: [{
|
||||
/// platform, url, sha256, size }] }], channels: {...} }` - the shape a
|
||||
/// release pipeline writes (redoal's does) for a site to offer downloads.
|
||||
pub fn is_releases_manifest(value: &serde_json::Value) -> bool {
|
||||
value.get("latest").and_then(|v| v.as_str()).is_some()
|
||||
&& value.get("versions").and_then(|v| v.as_array()).is_some_and(|v| !v.is_empty())
|
||||
}
|
||||
|
||||
/// The platform a browser says it is, in the manifest's words:
|
||||
/// `ios`, `macos`, `android`, `windows`, `linux-x86_64`,
|
||||
/// `linux-aarch64`, or `other`. Read, never trusted for anything but
|
||||
/// which download to put first.
|
||||
pub fn platform_of(user_agent: &str) -> &'static str {
|
||||
let ua = user_agent.to_ascii_lowercase();
|
||||
if ua.contains("iphone") || ua.contains("ipad") {
|
||||
"ios"
|
||||
} else if ua.contains("mac os x") || ua.contains("macintosh") {
|
||||
"macos"
|
||||
} else if ua.contains("android") {
|
||||
"android"
|
||||
} else if ua.contains("windows") {
|
||||
"windows"
|
||||
} else if ua.contains("linux") || ua.contains("x11") {
|
||||
if ua.contains("aarch64") || ua.contains("arm64") {
|
||||
"linux-aarch64"
|
||||
} else {
|
||||
"linux-x86_64"
|
||||
}
|
||||
} else {
|
||||
"other"
|
||||
}
|
||||
}
|
||||
|
||||
/// The newest release as a card: the file for this device first (the
|
||||
/// platform is read in the browser, so the server renders every file
|
||||
/// and the pick lands on hydration), the rest after, and the channels
|
||||
/// that are not files - a package repo, TestFlight.
|
||||
#[component]
|
||||
fn ReleasesCard(manifest: serde_json::Value) -> impl IntoView {
|
||||
let name = manifest.get("name").and_then(|v| v.as_str()).unwrap_or("").to_string();
|
||||
let latest = manifest.get("latest").and_then(|v| v.as_str()).unwrap_or("").to_string();
|
||||
let newest = manifest["versions"][0].clone();
|
||||
let date = newest.get("date").and_then(|v| v.as_str()).unwrap_or("").to_string();
|
||||
let notes = newest.get("notes").and_then(|v| v.as_str()).filter(|u| u.starts_with("https://")).map(str::to_string);
|
||||
let assets: Vec<(String, String, String)> = newest
|
||||
.get("assets")
|
||||
.and_then(|a| a.as_array())
|
||||
.map(|a| {
|
||||
a.iter()
|
||||
.filter_map(|x| {
|
||||
let platform = x.get("platform")?.as_str()?.to_string();
|
||||
let url = x.get("url")?.as_str()?.to_string();
|
||||
let sha = x.get("sha256").and_then(|s| s.as_str()).unwrap_or("").to_string();
|
||||
Some((platform, url, sha))
|
||||
})
|
||||
.collect()
|
||||
})
|
||||
.unwrap_or_default();
|
||||
let channels = manifest.get("channels").cloned().unwrap_or(serde_json::Value::Null);
|
||||
let arch = channels.get("arch").cloned();
|
||||
let ios = channels.get("ios").cloned();
|
||||
|
||||
let platform = RwSignal::new(None::<&'static str>);
|
||||
Effect::new(move |_| {
|
||||
#[cfg(feature = "hydrate")]
|
||||
if let Some(ua) = web_sys::window().and_then(|w| w.navigator().user_agent().ok()) {
|
||||
platform.set(Some(platform_of(&ua)));
|
||||
}
|
||||
});
|
||||
let label = |p: &str| match p {
|
||||
"linux-x86_64" => "Linux (x86_64)",
|
||||
"linux-aarch64" => "Linux (aarch64)",
|
||||
"macos" | "macos-arm64" => "macOS",
|
||||
"windows" | "windows-x86_64" => "Windows",
|
||||
other => other,
|
||||
}.to_string();
|
||||
let assets_for_list = assets.clone();
|
||||
let arch_for_pick = arch.clone();
|
||||
let ios_for_pick = ios.clone();
|
||||
let assets_for_pick = assets.clone();
|
||||
view! {
|
||||
<article class="item-card releases">
|
||||
<div class="item-card-title">
|
||||
<strong>{name.clone()} " " {latest.clone()}</strong>
|
||||
{(!date.is_empty()).then(|| view! { <span class="dim">" · " {date.clone()}</span> })}
|
||||
{notes.clone().map(|u| view! { " · " <a href=u target="_blank" rel="noopener noreferrer">{move || t(&lang().get(), "release_notes")}</a> })}
|
||||
</div>
|
||||
// What this device should take, decided in the browser.
|
||||
{move || {
|
||||
let p = platform.get()?;
|
||||
let arch = arch_for_pick.clone();
|
||||
let ios = ios_for_pick.clone();
|
||||
// "macos" is satisfied by a "macos-arm64" file, and so on.
|
||||
let own = assets_for_pick.iter().find(|(pl, _, _)| pl == p || pl.starts_with(&format!("{p}-"))).cloned();
|
||||
let pick: Option<AnyView> = match (p, own) {
|
||||
(_, Some((pl, url, _))) => Some(view! {
|
||||
<p><a class="pill" href=url download>{t(&lang().get(), "for_this_device")} ": " {label(&pl)}</a></p>
|
||||
{(p.starts_with("linux") && arch.is_some()).then(|| {
|
||||
let a = arch.clone().unwrap();
|
||||
let repo = a.get("repo").and_then(|v| v.as_str()).unwrap_or("").to_string();
|
||||
let pkg = a.get("package").and_then(|v| v.as_str()).unwrap_or("").to_string();
|
||||
view! { <pre class="monospace dim">{repo} "\n\npacman -Syu " {pkg}</pre> }
|
||||
})}
|
||||
}.into_any()),
|
||||
("ios" | "macos", None) => ios.as_ref().and_then(|i| i.get("testflight").and_then(|v| v.as_str()).map(str::to_string)).map(|tf| {
|
||||
let note = ios.as_ref().and_then(|i| i.get("note").and_then(|v| v.as_str())).unwrap_or("").to_string();
|
||||
view! { <p><a class="pill" href=tf target="_blank" rel="noopener noreferrer">{t(&lang().get(), "for_this_device")} ": TestFlight"</a> <span class="dim">" " {note}</span></p> }.into_any()
|
||||
}),
|
||||
_ => None,
|
||||
};
|
||||
pick
|
||||
}}
|
||||
<details class="releases-all">
|
||||
<summary>{move || t(&lang().get(), "all_downloads")}</summary>
|
||||
<ul>
|
||||
{assets_for_list.iter().map(|(pl, url, sha)| view! {
|
||||
<li><a href=url.clone() download>{label(pl)}</a> {(!sha.is_empty()).then(|| view! { <span class="dim monospace">" sha256 " {sha[..sha.len().min(12)].to_string()} "…"</span> })}</li>
|
||||
}).collect::<Vec<_>>()}
|
||||
{arch.as_ref().map(|a| {
|
||||
let note = a.get("note").and_then(|v| v.as_str()).unwrap_or("").to_string();
|
||||
view! { <li>{note}</li> }
|
||||
})}
|
||||
{ios.as_ref().and_then(|i| i.get("testflight").and_then(|v| v.as_str()).map(|tf| {
|
||||
let note = i.get("note").and_then(|v| v.as_str()).unwrap_or("").to_string();
|
||||
view! { <li><a href=tf.to_string() target="_blank" rel="noopener noreferrer">"TestFlight"</a> " " {note}</li> }
|
||||
}))}
|
||||
</ul>
|
||||
</details>
|
||||
</article>
|
||||
}
|
||||
}
|
||||
|
||||
/// Pulls the first present, non-null string field out of a JSON object
|
||||
/// matching one of `keys`, tried in order - shared between `ItemCard`
|
||||
/// and `SelectField`'s option rendering, since both need "guess a
|
||||
@@ -3365,3 +3504,27 @@ mod picture_tests {
|
||||
assert_eq!(picture_of(&answer, None), None);
|
||||
}
|
||||
}
|
||||
|
||||
#[cfg(test)]
|
||||
mod releases_tests {
|
||||
use super::*;
|
||||
|
||||
#[test]
|
||||
fn a_browser_names_its_platform_in_the_manifests_words() {
|
||||
assert_eq!(platform_of("Mozilla/5.0 (iPhone; CPU iPhone OS 19_0 like Mac OS X)"), "ios");
|
||||
assert_eq!(platform_of("Mozilla/5.0 (Macintosh; Intel Mac OS X 15_0)"), "macos");
|
||||
assert_eq!(platform_of("Mozilla/5.0 (X11; Linux x86_64) Firefox"), "linux-x86_64");
|
||||
assert_eq!(platform_of("Mozilla/5.0 (X11; Linux aarch64) Firefox"), "linux-aarch64");
|
||||
assert_eq!(platform_of("Mozilla/5.0 (Windows NT 10.0; Win64; x64)"), "windows");
|
||||
assert_eq!(platform_of("Mozilla/5.0 (Linux; Android 15)"), "android");
|
||||
assert_eq!(platform_of("curl/8"), "other");
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn only_a_manifest_shape_is_a_manifest() {
|
||||
let m = serde_json::json!({"name":"redoal","latest":"0.1.17","versions":[{"version":"0.1.17","assets":[]}]});
|
||||
assert!(is_releases_manifest(&m));
|
||||
assert!(!is_releases_manifest(&serde_json::json!({"latest":"1","versions":[]})));
|
||||
assert!(!is_releases_manifest(&serde_json::json!([{"name":"x"}])));
|
||||
}
|
||||
}
|
||||
|
||||
+115
-15
@@ -49,15 +49,56 @@ pub struct GiteaRepoQuery {
|
||||
pub async fn site_asset_handler(
|
||||
axum::extract::State(state): axum::extract::State<crate::server::AppState>,
|
||||
axum::extract::Path(path): axum::extract::Path<String>,
|
||||
) -> axum::response::Response {
|
||||
serve_repo_file(&state, &path, false).await
|
||||
}
|
||||
|
||||
/// `GET /.well-known/{*path}` - the content repo overlaid on the one part
|
||||
/// of the URL space that can never collide with a question: RFC 8615
|
||||
/// reserves `/.well-known/`, and a question's id is derived from a file
|
||||
/// name under `questions/`, which cannot begin with a dot. A repo ships
|
||||
/// `.well-known/security.txt`, `.well-known/assetlinks.json`,
|
||||
/// `.well-known/matrix/server` and so on at the path they are served at,
|
||||
/// and a miss is a real 404 rather than the app shell answering 200 with
|
||||
/// an HTML page - which is what a client probing for one of these would
|
||||
/// otherwise have to parse.
|
||||
///
|
||||
/// Paths Caddy claims in front of portal (mail autoconfiguration, the
|
||||
/// CalDAV and CardDAV redirects) never reach here.
|
||||
#[cfg(feature = "ssr")]
|
||||
pub async fn well_known_handler(
|
||||
axum::extract::State(state): axum::extract::State<crate::server::AppState>,
|
||||
axum::extract::Path(path): axum::extract::Path<String>,
|
||||
) -> axum::response::Response {
|
||||
serve_repo_file(&state, &format!(".well-known/{path}"), true).await
|
||||
}
|
||||
|
||||
/// Proxies one repo-relative file from the content repo's default branch.
|
||||
/// `well_known` only changes what an extensionless file is typed as: under
|
||||
/// `/.well-known/` those are JSON by convention (`matrix/server`,
|
||||
/// `nodeinfo`), while elsewhere a file with no extension is an unknown.
|
||||
#[cfg(feature = "ssr")]
|
||||
async fn serve_repo_file(
|
||||
state: &crate::server::AppState,
|
||||
path: &str,
|
||||
well_known: bool,
|
||||
) -> axum::response::Response {
|
||||
use axum::http::{header, StatusCode};
|
||||
use axum::response::IntoResponse;
|
||||
|
||||
if !is_safe_site_path(&path) {
|
||||
if !is_safe_site_path(path) {
|
||||
return (StatusCode::BAD_REQUEST, "invalid asset path").into_response();
|
||||
}
|
||||
let url = format!("{}/{path}?ref={}", state.content_raw_base, state.content_branch);
|
||||
let resp = match crate::http::client().get(&url).send().await.and_then(|r| r.error_for_status()) {
|
||||
let url = format!(
|
||||
"{}/{path}?ref={}",
|
||||
state.content_raw_base, state.content_branch
|
||||
);
|
||||
let resp = match crate::http::client()
|
||||
.get(&url)
|
||||
.send()
|
||||
.await
|
||||
.and_then(|r| r.error_for_status())
|
||||
{
|
||||
Ok(r) => r,
|
||||
Err(e) => {
|
||||
tracing::warn!("site asset {path}: {e}");
|
||||
@@ -79,12 +120,26 @@ pub async fn site_asset_handler(
|
||||
Some("avif") => "image/avif",
|
||||
Some("ico") => "image/x-icon",
|
||||
Some("woff2") => "font/woff2",
|
||||
Some("txt") => "text/plain; charset=utf-8",
|
||||
Some("xml") => "application/xml; charset=utf-8",
|
||||
Some("webmanifest") => "application/manifest+json",
|
||||
// An Apple configuration profile only offers to install when it
|
||||
// arrives under this type; anything else is just a download.
|
||||
Some("mobileconfig") => "application/x-apple-aspen-config",
|
||||
// A file with no extension at all: rsplit yields the whole name.
|
||||
_ if well_known && !path.rsplit('/').next().unwrap_or_default().contains('.') => {
|
||||
"application/json; charset=utf-8"
|
||||
}
|
||||
_ => "application/octet-stream",
|
||||
};
|
||||
// Photographs are large and do not change under a name, so a
|
||||
// browser may keep them for an hour; everything else revalidates,
|
||||
// so a content push shows on reload.
|
||||
let cache = if matches!(mime, "image/webp" | "image/avif" | "image/jpeg") { "public, max-age=3600" } else { "no-cache" };
|
||||
let cache = if matches!(mime, "image/webp" | "image/avif" | "image/jpeg") {
|
||||
"public, max-age=3600"
|
||||
} else {
|
||||
"no-cache"
|
||||
};
|
||||
(
|
||||
[(header::CONTENT_TYPE, mime), (header::CACHE_CONTROL, cache)],
|
||||
bytes,
|
||||
@@ -114,14 +169,22 @@ pub async fn gitea_repo_handler(
|
||||
.all(|c| c.is_ascii_alphanumeric() || c == '-' || c == '_' || c == '.')
|
||||
};
|
||||
if !is_safe_segment(&query.owner) || !is_safe_segment(&query.repo) {
|
||||
return Err((axum::http::StatusCode::BAD_REQUEST, "invalid owner/repo".to_string()));
|
||||
return Err((
|
||||
axum::http::StatusCode::BAD_REQUEST,
|
||||
"invalid owner/repo".to_string(),
|
||||
));
|
||||
}
|
||||
let GiteaRepoQuery { owner, repo } = query;
|
||||
|
||||
let api_url = format!("{}/api/v1/repos/{owner}/{repo}", state.gitea_base);
|
||||
let body = crate::http::get_text(&api_url, crate::http::GITEA_CAP)
|
||||
.await
|
||||
.map_err(|e| (axum::http::StatusCode::BAD_GATEWAY, format!("fetching {api_url}: {e}")))?;
|
||||
.map_err(|e| {
|
||||
(
|
||||
axum::http::StatusCode::BAD_GATEWAY,
|
||||
format!("fetching {api_url}: {e}"),
|
||||
)
|
||||
})?;
|
||||
let json: serde_json::Value = serde_json::from_str(&body).map_err(|e| {
|
||||
(
|
||||
axum::http::StatusCode::BAD_GATEWAY,
|
||||
@@ -168,27 +231,38 @@ pub async fn automation_kv_handler(
|
||||
.and_then(|v| v.strip_prefix("Bearer "))
|
||||
.unwrap_or("");
|
||||
if expected.is_empty() || presented != expected {
|
||||
return Err((axum::http::StatusCode::UNAUTHORIZED, "unauthorized".to_string()));
|
||||
return Err((
|
||||
axum::http::StatusCode::UNAUTHORIZED,
|
||||
"unauthorized".to_string(),
|
||||
));
|
||||
}
|
||||
// The token authenticates the automation service; what it may read
|
||||
// is the policy's to say, like every other principal.
|
||||
let decision = crate::access::decide_and_publish(
|
||||
&state.nats,
|
||||
&state.access.load(),
|
||||
&crate::access::Principal::Service { name: "automation".to_string() },
|
||||
&crate::access::Principal::Service {
|
||||
name: "automation".to_string(),
|
||||
},
|
||||
crate::access::Action::AutomationRead,
|
||||
&crate::access::Resource::Bucket { name: bucket.clone() },
|
||||
&crate::access::Resource::Bucket {
|
||||
name: bucket.clone(),
|
||||
},
|
||||
serde_json::json!({}),
|
||||
);
|
||||
if !decision.allow {
|
||||
return Err((axum::http::StatusCode::FORBIDDEN, "not authorized".to_string()));
|
||||
return Err((
|
||||
axum::http::StatusCode::FORBIDDEN,
|
||||
"not authorized".to_string(),
|
||||
));
|
||||
}
|
||||
|
||||
let store = state
|
||||
.jetstream
|
||||
.get_key_value(&bucket)
|
||||
.await
|
||||
.map_err(|e| (axum::http::StatusCode::BAD_GATEWAY, format!("bucket unavailable: {e}")))?;
|
||||
let store = state.jetstream.get_key_value(&bucket).await.map_err(|e| {
|
||||
(
|
||||
axum::http::StatusCode::BAD_GATEWAY,
|
||||
format!("bucket unavailable: {e}"),
|
||||
)
|
||||
})?;
|
||||
|
||||
use futures::TryStreamExt;
|
||||
let keys: Vec<String> = store
|
||||
@@ -209,3 +283,29 @@ pub async fn automation_kv_handler(
|
||||
Ok(axum::Json(serde_json::Value::Array(items)))
|
||||
}
|
||||
|
||||
#[cfg(test)]
|
||||
mod tests {
|
||||
use super::*;
|
||||
|
||||
/// The overlay's whole safety argument: `/.well-known/` is reserved by
|
||||
/// RFC 8615 and a question's id comes from a file under `questions/`, so
|
||||
/// a repo file served at the site root cannot shadow a page - and the
|
||||
/// path it is fetched by still cannot climb out of the repo.
|
||||
#[test]
|
||||
fn well_known_paths_stay_inside_the_repo() {
|
||||
for path in [
|
||||
".well-known/security.txt",
|
||||
".well-known/matrix/server",
|
||||
".well-known/assetlinks.json",
|
||||
] {
|
||||
assert!(is_safe_site_path(path), "{path} should be servable");
|
||||
}
|
||||
for path in [
|
||||
".well-known/../questions/index.yaml",
|
||||
".well-known//escape",
|
||||
"/.well-known/security.txt",
|
||||
] {
|
||||
assert!(!is_safe_site_path(path), "{path} should be refused");
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
@@ -519,6 +519,18 @@ pub struct ResourceSpec {
|
||||
/// it where the silence needs a voice ("Nothing released yet").
|
||||
#[serde(default)]
|
||||
pub empty: Option<String>,
|
||||
/// Show at most this many items, applied after `jq`. For a feed
|
||||
/// where only the newest matters: a releases card wants the cut you
|
||||
/// should run, not every cut there has ever been, and a page that
|
||||
/// grows by one card per release gets longer forever.
|
||||
///
|
||||
/// After `jq` rather than before, so it caps what the page will
|
||||
/// actually render whatever the filter did — and so content can
|
||||
/// reorder with jq and still take the top of its own ordering. A
|
||||
/// source that yields a single record has no length and passes
|
||||
/// through untouched. `None` shows everything.
|
||||
#[serde(default)]
|
||||
pub limit: Option<usize>,
|
||||
}
|
||||
|
||||
/// Where a resource's live data comes from. `Kv` (a NATS KV bucket
|
||||
|
||||
@@ -25,6 +25,9 @@ fn lookup(lang: &str, key: &str) -> Option<&'static str> {
|
||||
("en", "sign_out") => "Sign out",
|
||||
("en", "nothing_here_yet") => "Nothing here yet.",
|
||||
("en", "show_older") => "Show older",
|
||||
("en", "for_this_device") => "For this device",
|
||||
("en", "all_downloads") => "All downloads",
|
||||
("en", "release_notes") => "release notes",
|
||||
("en", "upload_failed") => "The file did not arrive, so nothing was sent. Try again.",
|
||||
("en", "upload_too_large") => "The file is larger than 20 MB, so nothing was sent. Choose a smaller one, or a picture taken at a lower resolution.",
|
||||
("en", "upload_unavailable") => "This site cannot take files at the moment, so nothing was sent. Let whoever runs the site know.",
|
||||
@@ -62,6 +65,9 @@ fn lookup(lang: &str, key: &str) -> Option<&'static str> {
|
||||
("no", "sign_out") => "Logg ut",
|
||||
("no", "nothing_here_yet") => "Ingenting her ennå.",
|
||||
("no", "show_older") => "Vis eldre",
|
||||
("no", "for_this_device") => "Til denne enheten",
|
||||
("no", "all_downloads") => "Alle nedlastinger",
|
||||
("no", "release_notes") => "utgivelsesnotater",
|
||||
("no", "upload_failed") => "Filen kom ikke fram, så ingenting ble sendt. Prøv igjen.",
|
||||
("no", "upload_too_large") => "Filen er større enn 20 MB, så ingenting ble sendt. Velg en mindre fil, eller et bilde med lavere oppløsning.",
|
||||
("no", "upload_unavailable") => "Siden kan ikke ta imot filer akkurat nå, så ingenting ble sendt. Si fra til den som har siden.",
|
||||
@@ -117,6 +123,9 @@ mod tests {
|
||||
"sign_out",
|
||||
"nothing_here_yet",
|
||||
"show_older",
|
||||
"for_this_device",
|
||||
"all_downloads",
|
||||
"release_notes",
|
||||
"upload_failed",
|
||||
"upload_too_large",
|
||||
"upload_unavailable",
|
||||
|
||||
+41
-3
@@ -24,6 +24,7 @@ pub async fn run(argv: Vec<String>) -> anyhow::Result<()> {
|
||||
let mut skim = false;
|
||||
let mut sim_out: Option<String> = None;
|
||||
let mut show_access = false;
|
||||
let mut findings_out: Option<String> = None;
|
||||
|
||||
while let Some(arg) = args.next() {
|
||||
match arg.as_str() {
|
||||
@@ -35,6 +36,7 @@ pub async fn run(argv: Vec<String>) -> anyhow::Result<()> {
|
||||
"--skim" => skim = true,
|
||||
"--needs-sim" => sim_out = args.next(),
|
||||
"--access" => show_access = true,
|
||||
"--findings" => findings_out = args.next(),
|
||||
"--needs-score" => answers_in = args.next(),
|
||||
"--min-accuracy" => {
|
||||
min_accuracy = args.next().and_then(|v| v.parse().ok()).unwrap_or(min_accuracy)
|
||||
@@ -66,13 +68,14 @@ pub async fn run(argv: Vec<String>) -> anyhow::Result<()> {
|
||||
(questions, aggregates_map, site, needs_raw)
|
||||
} else {
|
||||
eprintln!(
|
||||
"usage: iris check (--repo <gitea-url> [--branch main] [--subdir questions] | --path <local-dir>) [--needs-tasks <out.jsonl> [--skim]] [--needs-sim <model.json>] [--access] [--needs-score <answers.jsonl> [--min-accuracy 0.8]]"
|
||||
"usage: iris check (--repo <gitea-url> [--branch main] [--subdir questions] | --path <local-dir>) [--findings <out.jsonl>] [--needs-tasks <out.jsonl> [--skim]] [--needs-sim <model.json>] [--access] [--needs-score <answers.jsonl> [--min-accuracy 0.8]]"
|
||||
);
|
||||
std::process::exit(2);
|
||||
};
|
||||
|
||||
if let Err(e) = site.validate() {
|
||||
eprintln!("FAIL: {e}");
|
||||
write_findings(findings_out.as_deref(), &[schema_finding(&e.to_string())]);
|
||||
std::process::exit(1);
|
||||
}
|
||||
let aggregates_map = content::with_builtin_aggregates(aggregates_map);
|
||||
@@ -114,6 +117,7 @@ pub async fn run(argv: Vec<String>) -> anyhow::Result<()> {
|
||||
println!();
|
||||
}
|
||||
check_needs(
|
||||
findings_out.as_deref(),
|
||||
needs_raw.as_deref(),
|
||||
&questions,
|
||||
&aggregates_map,
|
||||
@@ -129,14 +133,43 @@ pub async fn run(argv: Vec<String>) -> anyhow::Result<()> {
|
||||
// One schema error is no reason to hide the rest: the needs
|
||||
// pass reads the same content and says what else is wrong,
|
||||
// so a fix round can take them all at once.
|
||||
let mut all = vec![schema_finding(&e.to_string())];
|
||||
if let Some(file) = needs_raw.as_deref().and_then(|raw| needs::parse(raw).ok()) {
|
||||
print_findings(&needs::check(&file, &questions, &aggregates_map));
|
||||
let findings = needs::check(&file, &questions, &aggregates_map);
|
||||
print_findings(&findings);
|
||||
all.extend(findings);
|
||||
}
|
||||
write_findings(findings_out.as_deref(), &all);
|
||||
std::process::exit(1);
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
/// A content error the schema pass stopped at, as a finding: the repo
|
||||
/// is its scope and the message its subject, since it has no other.
|
||||
fn schema_finding(message: &str) -> needs::Finding {
|
||||
needs::Finding {
|
||||
need: String::new(),
|
||||
level: needs::Level::Fail,
|
||||
message: message.to_string(),
|
||||
scope: "repo",
|
||||
code: "schema",
|
||||
subject: message.to_string(),
|
||||
}
|
||||
}
|
||||
|
||||
/// `--findings`: every finding as one JSON line (id, level, code,
|
||||
/// scope, need, subject, message, fix), the same ones the text output
|
||||
/// prints - an empty file when there are none, so a reader can tell
|
||||
/// "clean" from "not run".
|
||||
fn write_findings(out: Option<&str>, findings: &[needs::Finding]) {
|
||||
let Some(out) = out else { return };
|
||||
let lines: String = findings.iter().map(|f| format!("{}\n", f.to_json())).collect();
|
||||
if let Err(e) = std::fs::write(out, lines) {
|
||||
eprintln!("WARN: could not write the findings to {out}: {e}");
|
||||
}
|
||||
}
|
||||
|
||||
/// Each finding on its own line, FAIL or WARN first; the count of FAILs.
|
||||
fn print_findings(findings: &[needs::Finding]) -> usize {
|
||||
let mut failed = 0;
|
||||
@@ -157,6 +190,7 @@ fn print_findings(findings: &[needs::Finding]) -> usize {
|
||||
/// then the wording tasks written out and/or an engine's answers
|
||||
/// scored, when asked. A repo without `needs.yaml` skips all of it.
|
||||
fn check_needs(
|
||||
findings_out: Option<&str>,
|
||||
raw: Option<&str>,
|
||||
questions: &std::collections::HashMap<String, content::Question>,
|
||||
aggregates_map: &std::collections::HashMap<String, aggregates::AggregateSchema>,
|
||||
@@ -167,6 +201,7 @@ fn check_needs(
|
||||
min_accuracy: f64,
|
||||
) -> anyhow::Result<()> {
|
||||
let Some(raw) = raw else {
|
||||
write_findings(findings_out, &[]);
|
||||
if tasks_out.is_some() || answers_in.is_some() {
|
||||
eprintln!("FAIL: --needs-tasks/--needs-score given, but the repo has no needs.yaml");
|
||||
std::process::exit(1);
|
||||
@@ -177,10 +212,13 @@ fn check_needs(
|
||||
Ok(file) => file,
|
||||
Err(e) => {
|
||||
eprintln!("FAIL: {e}");
|
||||
write_findings(findings_out, &[schema_finding(&e.to_string())]);
|
||||
std::process::exit(1);
|
||||
}
|
||||
};
|
||||
let failed = print_findings(&needs::check(&file, questions, aggregates_map));
|
||||
let findings = needs::check(&file, questions, aggregates_map);
|
||||
let failed = print_findings(&findings);
|
||||
write_findings(findings_out, &findings);
|
||||
for need in &file.needs {
|
||||
if let Some(path) = needs::path_for(questions, need) {
|
||||
let steps: Vec<String> = path
|
||||
|
||||
+124
-15
@@ -144,6 +144,48 @@ pub struct Finding {
|
||||
/// "need" for a need's own path; "bucket" for what holds for every
|
||||
/// record in a bucket whichever need brought it in.
|
||||
pub scope: &'static str,
|
||||
/// What kind of gap it is, stable across releases: `strand`,
|
||||
/// `edge-unoffered`, `done-unreachable`, ... (docs/lint.md).
|
||||
pub code: &'static str,
|
||||
/// What in the content it is about: a state, a move (`from>to`), a
|
||||
/// page. With the scope, need and code it makes the finding's id.
|
||||
pub subject: String,
|
||||
}
|
||||
|
||||
impl Finding {
|
||||
/// The same gap keeps the same id however its message is worded,
|
||||
/// so a fix round can tell what it closed, what is still open and
|
||||
/// what it broke. FNV-1a over scope, need, code and subject.
|
||||
pub fn id(&self) -> String {
|
||||
let mut hash: u64 = 0xcbf2_9ce4_8422_2325;
|
||||
for part in [self.scope, &self.need, self.code, &self.subject] {
|
||||
for byte in part.bytes().chain(std::iter::once(0)) {
|
||||
hash ^= u64::from(byte);
|
||||
hash = hash.wrapping_mul(0x0100_0000_01b3);
|
||||
}
|
||||
}
|
||||
format!("{:08x}", hash >> 32)
|
||||
}
|
||||
|
||||
/// The change that closes it, when the lint knows one: what follows
|
||||
/// "fix:" in the message.
|
||||
pub fn fix(&self) -> Option<&str> {
|
||||
self.message.split_once(" - fix: ").map(|(_, fix)| fix)
|
||||
}
|
||||
|
||||
/// One line of `--findings` JSONL.
|
||||
pub fn to_json(&self) -> serde_json::Value {
|
||||
serde_json::json!({
|
||||
"id": self.id(),
|
||||
"level": match self.level { Level::Fail => "fail", Level::Warn => "warn" },
|
||||
"code": self.code,
|
||||
"scope": self.scope,
|
||||
"need": self.need,
|
||||
"subject": self.subject,
|
||||
"message": self.message,
|
||||
"fix": self.fix(),
|
||||
})
|
||||
}
|
||||
}
|
||||
|
||||
#[derive(Clone, Debug, PartialEq, Eq)]
|
||||
@@ -354,10 +396,12 @@ pub fn check(
|
||||
let mut strands_reported: HashSet<(String, String)> = HashSet::new();
|
||||
let mut unreached_reported: HashSet<(String, String)> = HashSet::new();
|
||||
for need in &file.needs {
|
||||
let mut report = |level: Level, message: String| {
|
||||
let mut report = |level: Level, code: &'static str, subject: String, message: String| {
|
||||
findings.push(Finding {
|
||||
need: need.id.clone(),
|
||||
scope: "need",
|
||||
code,
|
||||
subject,
|
||||
level: if need.planned || need.paused { Level::Warn } else { level },
|
||||
message: match (need.paused, need.planned, level == Level::Fail) {
|
||||
(true, _, true) => format!("paused, not open now: {message}"),
|
||||
@@ -367,15 +411,17 @@ pub fn check(
|
||||
})
|
||||
};
|
||||
if !seen.insert(need.id.as_str()) {
|
||||
report(Level::Fail, "declared more than once".to_string());
|
||||
report(Level::Fail, "need-duplicate", String::new(), "declared more than once".to_string());
|
||||
continue;
|
||||
}
|
||||
|
||||
// The visitor's side: can they get their answer recorded?
|
||||
match questions.get(&need.enters) {
|
||||
None => report(Level::Fail, format!("entry page {:?} does not exist", need.enters)),
|
||||
None => report(Level::Fail, "entry-missing", need.enters.clone(), format!("entry page {:?} does not exist", need.enters)),
|
||||
Some(page) if !visible(page, need.enters_as.as_deref()) => report(
|
||||
Level::Fail,
|
||||
"entry-gated",
|
||||
need.enters.clone(),
|
||||
format!(
|
||||
"entry page {:?} is gated on {:?}, which this visitor does not hold",
|
||||
need.enters, page.qualifies
|
||||
@@ -385,6 +431,8 @@ pub fn check(
|
||||
if page.responsible.is_none() {
|
||||
report(
|
||||
Level::Warn,
|
||||
"entry-no-responsible",
|
||||
need.enters.clone(),
|
||||
format!("entry page {:?} names no one responsible to turn to", need.enters),
|
||||
);
|
||||
}
|
||||
@@ -412,6 +460,8 @@ pub fn check(
|
||||
.unwrap_or_default();
|
||||
report(
|
||||
Level::Fail,
|
||||
"path-closed",
|
||||
at.clone(),
|
||||
format!(
|
||||
"the path exists but is closed: {at} is `disabled`, so the answer is advertised and refused"
|
||||
),
|
||||
@@ -419,6 +469,8 @@ pub fn check(
|
||||
}
|
||||
None => report(
|
||||
Level::Fail,
|
||||
"path-missing",
|
||||
String::new(),
|
||||
format!(
|
||||
"no path from {:?}{} records into {:?} within {} submission(s)",
|
||||
need.enters,
|
||||
@@ -435,6 +487,8 @@ pub fn check(
|
||||
if best.fields > budget {
|
||||
report(
|
||||
Level::Fail,
|
||||
"path-over-budget",
|
||||
String::new(),
|
||||
format!(
|
||||
"the path asks for {} required field(s), over the budget of {budget}",
|
||||
best.fields
|
||||
@@ -445,6 +499,8 @@ pub fn check(
|
||||
if best.records_with_action == Some(false) {
|
||||
report(
|
||||
Level::Warn,
|
||||
"path-no-action",
|
||||
String::new(),
|
||||
"the recording alternative has no `action` - the visitor is never told what happens next".to_string(),
|
||||
);
|
||||
}
|
||||
@@ -457,6 +513,8 @@ pub fn check(
|
||||
let Some(schema) = aggregates.get(&need.lands_in) else {
|
||||
report(
|
||||
Level::Fail,
|
||||
"machine-missing",
|
||||
need.lands_in.clone(),
|
||||
format!("aggregates.yaml declares no state machine for {:?}", need.lands_in),
|
||||
);
|
||||
continue;
|
||||
@@ -475,6 +533,8 @@ pub fn check(
|
||||
.collect();
|
||||
report(
|
||||
Level::Fail,
|
||||
"desk-missing",
|
||||
need.handled_by.clone(),
|
||||
format!(
|
||||
"no desk gated on {:?} reads {:?}{}",
|
||||
need.handled_by,
|
||||
@@ -556,7 +616,7 @@ pub fn check(
|
||||
match ways_in(state).first() {
|
||||
Some((from, to)) => format!(" - fix: {}", fix_for(from, to)),
|
||||
None => format!(
|
||||
" - and no state a record reaches has a declared edge into it: add one in aggregates.yaml under {:?}",
|
||||
" - fix: no state a record reaches has a declared edge into it; add one in aggregates.yaml under {:?}",
|
||||
need.lands_in
|
||||
),
|
||||
}
|
||||
@@ -565,12 +625,16 @@ pub fn check(
|
||||
if !schema.has_state(state) {
|
||||
report(
|
||||
Level::Fail,
|
||||
"done-unknown",
|
||||
state.clone(),
|
||||
format!("done_when names {state:?}, which {:?} has no such state", need.lands_in),
|
||||
);
|
||||
} else if !reached.contains(state) {
|
||||
unreached_reported.insert((need.lands_in.clone(), state.clone()));
|
||||
report(
|
||||
Level::Fail,
|
||||
"done-unreachable",
|
||||
state.clone(),
|
||||
format!(
|
||||
"{:?} cannot move a record to {state:?} - no desk of theirs offers the moves{}",
|
||||
need.handled_by,
|
||||
@@ -583,6 +647,8 @@ pub fn check(
|
||||
if !schema.has_state(stage) {
|
||||
report(
|
||||
Level::Fail,
|
||||
"stage-unknown",
|
||||
stage.clone(),
|
||||
format!(
|
||||
"the business describes a stage {stage:?} that {:?} has no state for",
|
||||
need.lands_in
|
||||
@@ -592,6 +658,8 @@ pub fn check(
|
||||
unreached_reported.insert((need.lands_in.clone(), stage.clone()));
|
||||
report(
|
||||
Level::Fail,
|
||||
"stage-unreachable",
|
||||
stage.clone(),
|
||||
format!(
|
||||
"stage {stage:?} exists but {:?} is offered no moves that reach it{}",
|
||||
need.handled_by,
|
||||
@@ -613,11 +681,13 @@ pub fn check(
|
||||
if reached.contains(before) && reached.contains(after) && !reachable(before, &moves).contains(after) {
|
||||
report(
|
||||
Level::Fail,
|
||||
"stage-order",
|
||||
format!("{before}>{after}"),
|
||||
format!(
|
||||
"stage {after:?} comes after {before:?} for the business, but no move leads on from {before:?} to it{}",
|
||||
match schema.allowed(before).iter().find(|to| reachable(to, &moves).contains(after) || *to == after) {
|
||||
Some(to) => format!(" - fix: {}", fix_for(before, to)),
|
||||
None => format!(" - add an edge from {before:?} toward {after:?} in aggregates.yaml under {:?}", need.lands_in),
|
||||
None => format!(" - fix: add an edge from {before:?} toward {after:?} in aggregates.yaml under {:?}", need.lands_in),
|
||||
}
|
||||
),
|
||||
);
|
||||
@@ -627,6 +697,8 @@ pub fn check(
|
||||
if !need.done_when.contains(state) {
|
||||
report(
|
||||
Level::Fail,
|
||||
"success-not-done",
|
||||
state.clone(),
|
||||
format!("success names {state:?}, which is not one of done_when"),
|
||||
);
|
||||
}
|
||||
@@ -639,12 +711,14 @@ pub fn check(
|
||||
let fix = match schema.allowed(&state).first() {
|
||||
Some(to) => format!(" - fix: {}", fix_for(&state, to)),
|
||||
None => format!(
|
||||
" - and {:?} declares no way out of it: add {state:?} to done_when if it is finished, or an edge out in aggregates.yaml",
|
||||
" - fix: {:?} declares no way out of it; add {state:?} to done_when if it is finished, or an edge out in aggregates.yaml",
|
||||
need.lands_in
|
||||
),
|
||||
};
|
||||
report(
|
||||
Level::Fail,
|
||||
"strand",
|
||||
state.clone(),
|
||||
format!("records strand in {state:?}: {movers} is offered no move out of it{fix}"),
|
||||
);
|
||||
}
|
||||
@@ -658,6 +732,8 @@ pub fn check(
|
||||
findings.push(Finding {
|
||||
need: persona.need.clone(),
|
||||
scope: "need",
|
||||
code: "persona-unknown-need",
|
||||
subject: persona.id.clone(),
|
||||
level: Level::Fail,
|
||||
message: format!("persona {:?} points at a need that is not declared", persona.id),
|
||||
});
|
||||
@@ -690,10 +766,12 @@ fn operations(
|
||||
// Only open needs make a gap fail; a bucket that only planned
|
||||
// or paused needs use reports it as a warning.
|
||||
let open = needs.iter().any(|n| !n.planned && !n.paused);
|
||||
let mut report = |level: Level, message: String| {
|
||||
let mut report = |level: Level, code: &'static str, subject: String, message: String| {
|
||||
findings.push(Finding {
|
||||
need: bucket.to_string(),
|
||||
scope: "bucket",
|
||||
code,
|
||||
subject,
|
||||
level: if open { level } else { Level::Warn },
|
||||
message,
|
||||
})
|
||||
@@ -728,6 +806,8 @@ fn operations(
|
||||
ways.sort();
|
||||
report(
|
||||
Level::Fail,
|
||||
"state-unreached",
|
||||
(*state).clone(),
|
||||
format!(
|
||||
"state {state:?} is never reached: no desk, link or clock moves a record there - fix: {}",
|
||||
match ways.first() {
|
||||
@@ -747,6 +827,8 @@ fn operations(
|
||||
if !strands_reported.contains(&(bucket.to_string(), (*state).clone())) {
|
||||
report(
|
||||
Level::Fail,
|
||||
"strand",
|
||||
(*state).clone(),
|
||||
format!(
|
||||
"records strand in {state:?}: {bucket:?} declares moves on to {} but no desk, link or clock offers one - fix: {}",
|
||||
out.join(", "),
|
||||
@@ -759,8 +841,10 @@ fn operations(
|
||||
for to in out.iter().filter(|to| !offered(state, to)) {
|
||||
report(
|
||||
Level::Warn,
|
||||
"edge-unoffered",
|
||||
format!("{state}>{to}"),
|
||||
format!(
|
||||
"the move {state:?} -> {to:?} is declared but no desk, link or clock offers it - {}, or drop the edge from aggregates.yaml",
|
||||
"the move {state:?} -> {to:?} is declared but no desk, link or clock offers it - fix: {}, or drop the edge from aggregates.yaml",
|
||||
button(state, to)
|
||||
),
|
||||
);
|
||||
@@ -771,8 +855,10 @@ fn operations(
|
||||
if page_group != group {
|
||||
report(
|
||||
Level::Warn,
|
||||
"desk-gated-twice",
|
||||
desk.page.clone(),
|
||||
format!(
|
||||
"the desk on {} is gated twice: the page qualifies {page_group:?}, its list requires {group:?} - only members of both see the buttons; make them the same group",
|
||||
"the desk on {} is gated twice: the page qualifies {page_group:?}, its list requires {group:?}, so only members of both see the buttons - fix: make them the same group",
|
||||
desk.page
|
||||
),
|
||||
);
|
||||
@@ -793,7 +879,7 @@ fn operations(
|
||||
mail_states.sort_by(|a, b| a.0.cmp(b.0));
|
||||
for (state, mail) in mail_states {
|
||||
if !reached.contains(state) {
|
||||
report(Level::Warn, format!("the mail on entering {state:?} never goes out: no record reaches {state:?}"));
|
||||
report(Level::Warn, "mail-unreachable", state.clone(), format!("the mail on entering {state:?} never goes out: no record reaches {state:?} - fix: make {state:?} reachable, or drop its mail"));
|
||||
}
|
||||
let mut unknown: Vec<String> = Vec::new();
|
||||
for key in placeholders(&mail.subject).into_iter().chain(placeholders(&mail.body)) {
|
||||
@@ -811,8 +897,10 @@ fn operations(
|
||||
names.dedup();
|
||||
report(
|
||||
Level::Fail,
|
||||
"mail-unknown-field",
|
||||
state.clone(),
|
||||
format!(
|
||||
"the mail on entering {state:?} uses {}, which no form recording into {bucket:?} collects - it would go out blank there; the forms collect: {}",
|
||||
"the mail on entering {state:?} uses {}, which no form recording into {bucket:?} collects, so it would go out blank there - fix: use one of the fields the forms collect ({}), or ask for it on the form",
|
||||
unknown.iter().map(|k| format!("{{{k}}}")).collect::<Vec<_>>().join(", "),
|
||||
if names.is_empty() { "nothing".to_string() } else { names.join(", ") }
|
||||
),
|
||||
@@ -1407,7 +1495,7 @@ mod tests {
|
||||
assert!(fails(&findings).is_empty(), "{findings:?}");
|
||||
assert_eq!(
|
||||
warns(&findings),
|
||||
vec!["the move \"open\" -> \"lost\" is declared but no desk, link or clock offers it - add `{ from: open, to: lost, label: ... }` to the transitions on /desk, or drop the edge from aggregates.yaml"]
|
||||
vec!["the move \"open\" -> \"lost\" is declared but no desk, link or clock offers it - fix: add `{ from: open, to: lost, label: ... }` to the transitions on /desk, or drop the edge from aggregates.yaml"]
|
||||
);
|
||||
}
|
||||
|
||||
@@ -1422,7 +1510,7 @@ mod tests {
|
||||
let findings = check(&needs(" stages: [open, surveyed, quoted, won, lost]\n"), &questions, &aggregates);
|
||||
assert_eq!(
|
||||
fails(&findings),
|
||||
vec!["stage \"quoted\" comes after \"surveyed\" for the business, but no move leads on from \"surveyed\" to it - add an edge from \"surveyed\" toward \"quoted\" in aggregates.yaml under \"projects\""]
|
||||
vec!["stage \"quoted\" comes after \"surveyed\" for the business, but no move leads on from \"surveyed\" to it - fix: add an edge from \"surveyed\" toward \"quoted\" in aggregates.yaml under \"projects\""]
|
||||
);
|
||||
}
|
||||
|
||||
@@ -1435,7 +1523,7 @@ mod tests {
|
||||
let findings = check(&needs(""), &questions, &aggregates);
|
||||
assert_eq!(
|
||||
fails(&findings),
|
||||
vec!["the mail on entering \"open\" uses {refnr}, which no form recording into \"projects\" collects - it would go out blank there; the forms collect: brief, budget, email"]
|
||||
vec!["the mail on entering \"open\" uses {refnr}, which no form recording into \"projects\" collects, so it would go out blank there - fix: use one of the fields the forms collect (brief, budget, email), or ask for it on the form"]
|
||||
);
|
||||
}
|
||||
|
||||
@@ -1445,8 +1533,29 @@ mod tests {
|
||||
questions.get_mut("/desk").unwrap().qualifies = Some("staff".into());
|
||||
let findings = check(&needs(""), &questions, &aggregates);
|
||||
assert!(
|
||||
warns(&findings).contains(&"the desk on /desk is gated twice: the page qualifies \"staff\", its list requires \"owners\" - only members of both see the buttons; make them the same group"),
|
||||
warns(&findings).contains(&"the desk on /desk is gated twice: the page qualifies \"staff\", its list requires \"owners\", so only members of both see the buttons - fix: make them the same group"),
|
||||
"{findings:?}"
|
||||
);
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn a_finding_keeps_its_id_when_reworded_and_says_its_fix() {
|
||||
let (questions, aggregates) = site(" - { to: quoted, label: Quote }\n - { to: lost, label: Lose }\n");
|
||||
let findings = check(&needs(""), &questions, &aggregates);
|
||||
let strand = findings.iter().find(|f| f.code == "strand").unwrap();
|
||||
assert_eq!(strand.subject, "quoted");
|
||||
assert_eq!(strand.fix(), Some("add `{ from: quoted, to: won, label: ... }` to the transitions on /desk"));
|
||||
let reworded = Finding { message: "something else entirely".into(), ..clone_of(strand) };
|
||||
assert_eq!(reworded.id(), strand.id());
|
||||
let elsewhere = Finding { subject: "open".into(), ..clone_of(strand) };
|
||||
assert_ne!(elsewhere.id(), strand.id());
|
||||
let json = strand.to_json();
|
||||
assert_eq!(json["code"], "strand");
|
||||
assert_eq!(json["level"], "fail");
|
||||
assert_eq!(json["id"], strand.id());
|
||||
}
|
||||
|
||||
fn clone_of(f: &Finding) -> Finding {
|
||||
Finding { need: f.need.clone(), level: f.level, message: f.message.clone(), scope: f.scope, code: f.code, subject: f.subject.clone() }
|
||||
}
|
||||
}
|
||||
|
||||
+49
-19
@@ -27,7 +27,14 @@ async fn main() -> anyhow::Result<()> {
|
||||
|
||||
#[cfg(feature = "ssr")]
|
||||
async fn serve() -> anyhow::Result<()> {
|
||||
use axum::{body::Body, extract::State, http::Request, response::IntoResponse, routing::{any, get, post}, Router};
|
||||
use axum::{
|
||||
body::Body,
|
||||
extract::State,
|
||||
http::Request,
|
||||
response::IntoResponse,
|
||||
routing::{any, get, post},
|
||||
Router,
|
||||
};
|
||||
use leptos::prelude::*;
|
||||
use leptos_axum::{generate_route_list, LeptosRoutes};
|
||||
use portal::app::{shell, App};
|
||||
@@ -57,12 +64,15 @@ async fn serve() -> anyhow::Result<()> {
|
||||
let gitea_base = content::gitea_api_base(&content_repo)?;
|
||||
let content_raw_base = content::gitea_raw_base(&content_repo)?;
|
||||
let aggregates = content::load_aggregates_from_gitea(&content_repo, &content_branch).await?;
|
||||
let questions = content::load_questions_from_gitea(&content_repo, &content_branch, "questions").await?;
|
||||
let questions =
|
||||
content::load_questions_from_gitea(&content_repo, &content_branch, "questions").await?;
|
||||
content::validate_questions(&questions, &aggregates)?;
|
||||
let site = content::load_site_from_gitea(&content_repo, &content_branch).await?;
|
||||
let storylines = content::same_arity(questions.arity(), aggregates.arity(), site.arity())?;
|
||||
tracing::info!(count = questions.len(), storylines, repo = %content_repo, branch = %content_branch, "loaded content");
|
||||
let access = Arc::new(arc_swap::ArcSwap::from_pointee(portal::access::Policy::from_content(&questions, &aggregates)));
|
||||
let access = Arc::new(arc_swap::ArcSwap::from_pointee(
|
||||
portal::access::Policy::from_content(&questions, &aggregates),
|
||||
));
|
||||
let questions = Arc::new(arc_swap::ArcSwap::from_pointee(questions));
|
||||
let aggregates = Arc::new(arc_swap::ArcSwap::from_pointee(aggregates));
|
||||
let site = Arc::new(arc_swap::ArcSwap::from_pointee(site));
|
||||
@@ -128,7 +138,12 @@ async fn serve() -> anyhow::Result<()> {
|
||||
|
||||
// The same decisions, answerable by other apps over NATS, for a
|
||||
// person named by phone number or address as well as by groups.
|
||||
tokio::spawn(portal::access::serve(state.nats.clone(), site_name_for_subjects(), state.access.clone(), Some(state.clone())));
|
||||
tokio::spawn(portal::access::serve(
|
||||
state.nats.clone(),
|
||||
site_name_for_subjects(),
|
||||
state.access.clone(),
|
||||
Some(state.clone()),
|
||||
));
|
||||
// Announced pages: keep their desk records current, close ended
|
||||
// windows (see announce.rs).
|
||||
tokio::spawn(portal::announce::run(state.clone()));
|
||||
@@ -202,18 +217,17 @@ async fn serve() -> anyhow::Result<()> {
|
||||
});
|
||||
match &governor {
|
||||
Some(_) => tracing::info!(rate_per_second, rate_burst, "rate limiting per caller IP"),
|
||||
None => tracing::warn!("rate limiting is off (PORTAL_RATE_PER_SECOND or PORTAL_RATE_BURST is 0)"),
|
||||
None => tracing::warn!(
|
||||
"rate limiting is off (PORTAL_RATE_PER_SECOND or PORTAL_RATE_BURST is 0)"
|
||||
),
|
||||
}
|
||||
|
||||
async fn server_fn_handler(
|
||||
State(state): State<AppState>,
|
||||
request: Request<Body>,
|
||||
) -> impl IntoResponse {
|
||||
leptos_axum::handle_server_fns_with_context(
|
||||
move || provide_context(state.clone()),
|
||||
request,
|
||||
)
|
||||
.await
|
||||
leptos_axum::handle_server_fns_with_context(move || provide_context(state.clone()), request)
|
||||
.await
|
||||
}
|
||||
|
||||
// The app's own Leptos route list includes a root-level wildcard
|
||||
@@ -264,10 +278,7 @@ async fn serve() -> anyhow::Result<()> {
|
||||
.route_service("/favicon-light.svg", ServeFile::new(favicon_light_path))
|
||||
.route_service("/favicon-dark.svg", ServeFile::new(favicon_dark_path))
|
||||
.route_service("/wordmark.svg", ServeFile::new(wordmark_path))
|
||||
.route_service(
|
||||
"/swiper-element-bundle.min.js",
|
||||
ServeFile::new(swiper_path),
|
||||
)
|
||||
.route_service("/swiper-element-bundle.min.js", ServeFile::new(swiper_path))
|
||||
// Same name every release, so revalidated like /pkg.
|
||||
.route_service(
|
||||
"/page-motion.js",
|
||||
@@ -291,7 +302,9 @@ async fn serve() -> anyhow::Result<()> {
|
||||
// with room for the multipart framing.
|
||||
.route(
|
||||
"/upload",
|
||||
post(upload::upload).layer(axum::extract::DefaultBodyLimit::max(upload::MAX_UPLOAD_BYTES + 64 * 1024)),
|
||||
post(upload::upload).layer(axum::extract::DefaultBodyLimit::max(
|
||||
upload::MAX_UPLOAD_BYTES + 64 * 1024,
|
||||
)),
|
||||
)
|
||||
.route(
|
||||
"/attachment/{bucket}/{record}/{field}",
|
||||
@@ -302,7 +315,11 @@ async fn serve() -> anyhow::Result<()> {
|
||||
.route("/report", get(portal::report::handler))
|
||||
// Content-shipped assets (hero module etc.), same-origin.
|
||||
.route("/site/{*path}", get(content::site_asset_handler))
|
||||
.route("/automation/kv/{bucket}", get(content::automation_kv_handler))
|
||||
.route("/.well-known/{*path}", get(content::well_known_handler))
|
||||
.route(
|
||||
"/automation/kv/{bucket}",
|
||||
get(content::automation_kv_handler),
|
||||
)
|
||||
// Maps, proxied: the browser asks this origin for every tile
|
||||
// and every still, so nothing third-party loads in a page and
|
||||
// the token stays here (see maps.rs).
|
||||
@@ -324,7 +341,10 @@ async fn serve() -> anyhow::Result<()> {
|
||||
.fallback(leptos_axum::file_and_error_handler::<AppState, _>(shell))
|
||||
.layer(session_layer)
|
||||
// Which storyline this visitor reads, before anything renders.
|
||||
.layer(axum::middleware::from_fn_with_state(state.clone(), portal::server::storyline::assign));
|
||||
.layer(axum::middleware::from_fn_with_state(
|
||||
state.clone(),
|
||||
portal::server::storyline::assign,
|
||||
));
|
||||
// Outermost, so a caller who is over the limit is turned away
|
||||
// before a session is loaded or a handler is entered.
|
||||
let app = match governor {
|
||||
@@ -362,7 +382,17 @@ fn site_name_for_subjects() -> String {
|
||||
let raw = std::env::var("SITE_NAME").unwrap_or_else(|_| "portal".to_string());
|
||||
let folded: String = raw
|
||||
.chars()
|
||||
.map(|c| if c.is_ascii_alphanumeric() || c == '_' || c == '-' { c.to_ascii_lowercase() } else { '_' })
|
||||
.map(|c| {
|
||||
if c.is_ascii_alphanumeric() || c == '_' || c == '-' {
|
||||
c.to_ascii_lowercase()
|
||||
} else {
|
||||
'_'
|
||||
}
|
||||
})
|
||||
.collect();
|
||||
if folded.is_empty() { "portal".to_string() } else { folded }
|
||||
if folded.is_empty() {
|
||||
"portal".to_string()
|
||||
} else {
|
||||
folded
|
||||
}
|
||||
}
|
||||
|
||||
+68
-3
@@ -347,9 +347,29 @@ async fn fetch_resource_value(
|
||||
ResourceSource::Url { url } => fetch_url_json(url, params).await?,
|
||||
};
|
||||
|
||||
match &resource.jq {
|
||||
Some(filter) => apply_jq(filter, &value),
|
||||
None => Ok(value),
|
||||
let shaped = match &resource.jq {
|
||||
Some(filter) => apply_jq(filter, &value)?,
|
||||
None => value,
|
||||
};
|
||||
Ok(apply_limit(resource.limit, shaped))
|
||||
}
|
||||
|
||||
/// Keep at most `n` items, after `jq` has had its say.
|
||||
///
|
||||
/// Anything that is not a list passes through: a single record has no
|
||||
/// length to cap, and silently turning one into an empty list would be
|
||||
/// worse than ignoring a limit that cannot apply. A limit of zero does
|
||||
/// mean nothing — the resource's `empty` line speaks instead, which is
|
||||
/// a coherent thing for content to ask for.
|
||||
#[cfg(feature = "ssr")]
|
||||
fn apply_limit(limit: Option<usize>, value: serde_json::Value) -> serde_json::Value {
|
||||
let Some(n) = limit else { return value };
|
||||
match value {
|
||||
serde_json::Value::Array(mut items) => {
|
||||
items.truncate(n);
|
||||
serde_json::Value::Array(items)
|
||||
}
|
||||
other => other,
|
||||
}
|
||||
}
|
||||
|
||||
@@ -664,6 +684,51 @@ mod tests {
|
||||
assert!(!matches_filter(&fixed, &record("open", serde_json::json!({ "which_authority": "Fire service" })), ¶ms));
|
||||
}
|
||||
|
||||
/// `limit` caps what the page renders, and does it after `jq` so
|
||||
/// content can order with the filter and take the top of its own
|
||||
/// ordering. The releases card is the case: the newest cut is the
|
||||
/// one you should run, and a page that grows a card per release
|
||||
/// gets longer forever.
|
||||
#[test]
|
||||
fn limit_caps_a_list_after_the_filter() {
|
||||
let releases = serde_json::json!([
|
||||
{"name": "redoal 0.1.17", "tag_name": "v0.1.17", "body": "newest"},
|
||||
{"name": "redoal 0.1.16", "tag_name": "v0.1.16", "body": "older"},
|
||||
{"name": "redoal 0.1.15", "tag_name": "v0.1.15", "body": "oldest"}
|
||||
]);
|
||||
// The filter redoal/questions actually declares: bare objects,
|
||||
// never a wrapped array — the frontend re-wraps.
|
||||
let filter = ".[] | {name: .name, description: .body, tag: .tag_name, url: null}";
|
||||
let shaped = apply_jq(filter, &releases).expect("filter runs");
|
||||
|
||||
let all = apply_limit(None, shaped.clone());
|
||||
assert_eq!(all.as_array().map(Vec::len), Some(3), "no limit shows everything");
|
||||
|
||||
let latest = apply_limit(Some(1), shaped.clone());
|
||||
let rows = latest.as_array().expect("still a list");
|
||||
assert_eq!(rows.len(), 1);
|
||||
assert_eq!(rows[0]["tag"], "v0.1.17", "and it is the newest, which Gitea returns first");
|
||||
|
||||
let two = apply_limit(Some(2), shaped.clone());
|
||||
assert_eq!(two.as_array().map(Vec::len), Some(2));
|
||||
|
||||
// Asking for more than there is is not an error.
|
||||
let plenty = apply_limit(Some(99), shaped);
|
||||
assert_eq!(plenty.as_array().map(Vec::len), Some(3));
|
||||
}
|
||||
|
||||
/// A limit that cannot apply is ignored rather than guessed at.
|
||||
#[test]
|
||||
fn limit_leaves_a_single_record_alone() {
|
||||
let one = serde_json::json!({"name": "a record", "state": "open"});
|
||||
assert_eq!(apply_limit(Some(1), one.clone()), one);
|
||||
assert_eq!(apply_limit(Some(0), one.clone()), one, "no length to cap");
|
||||
|
||||
// Zero on a list does mean nothing — the `empty` line speaks.
|
||||
let list = serde_json::json!([{"a": 1}, {"a": 2}]);
|
||||
assert_eq!(apply_limit(Some(0), list).as_array().map(Vec::len), Some(0));
|
||||
}
|
||||
|
||||
/// A canned, Gitea-API-shaped fixture - the same fields the real
|
||||
/// `/users/{username}/starred` endpoint returns (Gitea 1.27:
|
||||
/// `stars_count`, not GitHub's `stargazers_count`; `website` is ""
|
||||
|
||||
+254
-7
@@ -132,34 +132,91 @@ pub fn run(argv: Vec<String>) -> anyhow::Result<()> {
|
||||
.to_string();
|
||||
let mut green = false;
|
||||
let mut report = String::new();
|
||||
// What the lint found, by id, round over round (`--findings`): what
|
||||
// a round closed, what it left, what it broke. `None` while the
|
||||
// lint is one that predates `--findings`; then the whole report
|
||||
// goes back, as it always did.
|
||||
let mut ledger = Ledger::default();
|
||||
let mut tracked = true;
|
||||
let mut undone = String::new();
|
||||
std::fs::create_dir_all(opts.out.join("findings"))?;
|
||||
for round in 1..=opts.rounds {
|
||||
let prompt = if round == 1 {
|
||||
format!(
|
||||
"{REFERENCE}\n\n# Task\n\nWrite the content repo that meets this frozen `needs.yaml`: `aggregates.yaml`, `site.yaml`, and every page under `questions/` - the front door, followup pages, role-gated pages, one desk per handling group. Do not return needs.yaml.\n\n## Intake\n\n{intake}\n\n## needs.yaml (frozen)\n\n```yaml\n{needs}\n```\n\n{FORMAT}"
|
||||
)
|
||||
} else if tracked {
|
||||
let (heading, files) = match named_files(&opts.out, ledger.open.values()) {
|
||||
Some(files) => ("Files the findings name", dump_files(&opts.out, &files)?),
|
||||
None => ("Current files", dump(&opts.out)?),
|
||||
};
|
||||
format!(
|
||||
"{REFERENCE}\n\n# Task\n\nThe verifier rejected the repo. Below is every finding still open, each with the change that closes it. Make those changes and nothing else: findings already closed stay closed. Return only the files you change, each complete. To remove a file, return it with the single line DELETE. needs.yaml is frozen; do not return it.\n\n## Open findings\n\n{}\n{undone}## needs.yaml (frozen)\n\n```yaml\n{needs}\n```\n\n## {heading}\n\n{files}\n\n{FORMAT}",
|
||||
ledger.open_list(round)
|
||||
)
|
||||
} else {
|
||||
format!(
|
||||
"{REFERENCE}\n\n# Task\n\nThe verifier rejected the repo below. Fix it. Return only the files you change, each complete. To remove a file, return it with the single line DELETE. needs.yaml is frozen; do not return it.\n\n## Verifier output\n\n```\n{report}\n```\n\n## needs.yaml (frozen)\n\n```yaml\n{needs}\n```\n\n## Current files\n\n{}\n\n{FORMAT}",
|
||||
dump(&opts.out)?
|
||||
)
|
||||
};
|
||||
let before = snapshot(&opts.out)?;
|
||||
match parse_files(&llm(&opts.llm, &prompt)?) {
|
||||
Ok(files) => apply(&opts.out, files)?,
|
||||
Err(_) => eprintln!("structure round {round}: the reply had no file blocks"),
|
||||
}
|
||||
let (ok, r) = lint(&opts, &[])?;
|
||||
let (ok, r, found) = lint_findings(&opts, &opts.out.join(format!("findings/round-{round}.jsonl")))?;
|
||||
report = r;
|
||||
let fails = report.lines().filter(|l| l.starts_with("FAIL")).count();
|
||||
eprintln!("structure round {round}: {}", if ok { "green".to_string() } else { format!("{fails} FAIL line(s)") });
|
||||
log.push(format!("structure round {round}: ok={ok} fails={fails}"));
|
||||
let Some(found) = found else {
|
||||
tracked = false;
|
||||
let fails = report.lines().filter(|l| l.starts_with("FAIL")).count();
|
||||
eprintln!("structure round {round}: {}", if ok { "green".to_string() } else { format!("{fails} FAIL line(s)") });
|
||||
log.push(format!("structure round {round}: ok={ok} fails={fails}"));
|
||||
if ok {
|
||||
green = true;
|
||||
break;
|
||||
}
|
||||
continue;
|
||||
};
|
||||
let previous = ledger.clone();
|
||||
let progress = ledger.update(round, found.into_iter().filter(|f| f.level == "fail").collect());
|
||||
let line = format!("structure round {round}: {}", progress.summary(ledger.open.len()));
|
||||
eprintln!("{line}");
|
||||
log.push(line);
|
||||
undone.clear();
|
||||
if ok {
|
||||
green = true;
|
||||
break;
|
||||
}
|
||||
if round == 1 {
|
||||
continue;
|
||||
}
|
||||
// A fix that reopens what an earlier round closed is undone,
|
||||
// and the model is told what it broke, rather than chasing the
|
||||
// same findings round after round.
|
||||
if !progress.reopened.is_empty() {
|
||||
restore(&opts.out, before)?;
|
||||
ledger = previous;
|
||||
undone = format!(
|
||||
"## Your last change was undone\n\nIt reopened findings an earlier round had closed:\n\n{}\nClose the open findings without touching what made these pass.\n\n",
|
||||
progress.reopened.iter().map(|f| format!("- [{}] {}\n", f.id, f.message)).collect::<String>()
|
||||
);
|
||||
eprintln!("structure round {round}: reopened {} closed finding(s), undone", progress.reopened.len());
|
||||
log.push(format!("structure round {round}: undone (reopened {})", progress.reopened.len()));
|
||||
continue;
|
||||
}
|
||||
// No progress, no more rounds: another pass at the same findings
|
||||
// is how a site gets over-trained. What is left is reported.
|
||||
if progress.fixed.is_empty() || ledger.open.len() >= previous.open.len() {
|
||||
eprintln!("structure round {round}: no finding closed, stopping");
|
||||
log.push(format!("structure round {round}: stalled"));
|
||||
break;
|
||||
}
|
||||
}
|
||||
if !green {
|
||||
std::fs::write(opts.out.join("run.log"), format!("{}\n\n{report}", log.join("\n")))?;
|
||||
bail!("structure never went green in {} round(s); last verifier output is in run.log", opts.rounds);
|
||||
let open = if tracked { format!("\n\nStill open:\n{}", ledger.open_list(opts.rounds + 1)) } else { String::new() };
|
||||
std::fs::write(opts.out.join("run.log"), format!("{}{open}\n\n{report}", log.join("\n")))?;
|
||||
bail!("structure did not go green; what is still open is in run.log");
|
||||
}
|
||||
|
||||
// Phase 3: wording.
|
||||
@@ -191,7 +248,10 @@ pub fn run(argv: Vec<String>) -> anyhow::Result<()> {
|
||||
}
|
||||
eprintln!("wording round {round}: {}", summary.join(" | "));
|
||||
log.push(format!("wording round {round}: {}", summary.join(" | ")));
|
||||
if defects.is_empty() || round == opts.wording_rounds {
|
||||
// A visitor who chose right, hesitantly, is not a defect worth a
|
||||
// rewrite on its own: only a misroute is. Hesitations ride along
|
||||
// as context when there is one.
|
||||
if !defects.contains("MISROUTE") || round == opts.wording_rounds {
|
||||
break;
|
||||
}
|
||||
let personas = needs_full.split("\npersonas:").nth(1).unwrap_or_default();
|
||||
@@ -311,6 +371,152 @@ fn lint(opts: &Opts, extra: &[String]) -> anyhow::Result<(bool, String)> {
|
||||
Ok((output.status.success(), report))
|
||||
}
|
||||
|
||||
/// One finding as the lint's `--findings` writes it (docs/lint.md).
|
||||
#[derive(Clone, Debug, serde::Deserialize)]
|
||||
struct Finding {
|
||||
id: String,
|
||||
level: String,
|
||||
code: String,
|
||||
#[serde(default)]
|
||||
need: String,
|
||||
message: String,
|
||||
#[serde(default)]
|
||||
fix: Option<String>,
|
||||
}
|
||||
|
||||
/// The lint with `--findings` into `out`, and what it found; `None` for
|
||||
/// the findings when the lint predates the flag (then it is run again
|
||||
/// the old way, and only its text comes back).
|
||||
fn lint_findings(opts: &Opts, out: &Path) -> anyhow::Result<(bool, String, Option<Vec<Finding>>)> {
|
||||
let (ok, report) = lint(opts, &["--findings".to_string(), out.display().to_string()])?;
|
||||
if report.contains("unknown argument: --findings") {
|
||||
let (ok, report) = lint(opts, &[])?;
|
||||
return Ok((ok, report, None));
|
||||
}
|
||||
let raw = std::fs::read_to_string(out).unwrap_or_default();
|
||||
let found = raw
|
||||
.lines()
|
||||
.filter(|l| !l.trim().is_empty())
|
||||
.map(|l| serde_json::from_str::<Finding>(l).with_context(|| format!("reading {}", out.display())))
|
||||
.collect::<anyhow::Result<Vec<_>>>()?;
|
||||
Ok((ok, report, Some(found)))
|
||||
}
|
||||
|
||||
/// Every failing finding by id, across rounds.
|
||||
#[derive(Clone, Default)]
|
||||
struct Ledger {
|
||||
open: BTreeMap<String, Finding>,
|
||||
/// The round each open finding was first seen in.
|
||||
since: BTreeMap<String, usize>,
|
||||
/// Ids a round has closed, so one that comes back is a regression,
|
||||
/// not news.
|
||||
closed: std::collections::BTreeSet<String>,
|
||||
}
|
||||
|
||||
struct Progress {
|
||||
fixed: Vec<Finding>,
|
||||
new: Vec<Finding>,
|
||||
still: usize,
|
||||
reopened: Vec<Finding>,
|
||||
}
|
||||
|
||||
impl Progress {
|
||||
fn summary(&self, open: usize) -> String {
|
||||
if open == 0 {
|
||||
return format!("green (fixed {})", self.fixed.len());
|
||||
}
|
||||
format!(
|
||||
"{open} open - fixed {}, still open {}, new {}, reopened {}",
|
||||
self.fixed.len(),
|
||||
self.still,
|
||||
self.new.len(),
|
||||
self.reopened.len()
|
||||
)
|
||||
}
|
||||
}
|
||||
|
||||
impl Ledger {
|
||||
fn update(&mut self, round: usize, fails: Vec<Finding>) -> Progress {
|
||||
let now: BTreeMap<String, Finding> = fails.into_iter().map(|f| (f.id.clone(), f)).collect();
|
||||
let fixed: Vec<Finding> = self.open.values().filter(|f| !now.contains_key(&f.id)).cloned().collect();
|
||||
let mut new = Vec::new();
|
||||
let mut reopened = Vec::new();
|
||||
let mut still = 0;
|
||||
for (id, f) in &now {
|
||||
if self.open.contains_key(id) {
|
||||
still += 1;
|
||||
} else if self.closed.contains(id) {
|
||||
reopened.push(f.clone());
|
||||
} else {
|
||||
new.push(f.clone());
|
||||
}
|
||||
self.since.entry(id.clone()).or_insert(round);
|
||||
}
|
||||
for f in &fixed {
|
||||
self.closed.insert(f.id.clone());
|
||||
self.since.remove(&f.id);
|
||||
}
|
||||
self.open = now;
|
||||
Progress { fixed, new, still, reopened }
|
||||
}
|
||||
|
||||
/// The open findings for a prompt or a log: id, how long open, the
|
||||
/// message and its fix.
|
||||
fn open_list(&self, round: usize) -> String {
|
||||
self.open
|
||||
.values()
|
||||
.map(|f| {
|
||||
let age = match self.since.get(&f.id) {
|
||||
Some(&first) if first + 1 < round => format!(", open since round {first}"),
|
||||
_ => String::new(),
|
||||
};
|
||||
let scope = if f.need.is_empty() { String::new() } else { format!(" {:?}", f.need) };
|
||||
format!("- [{}] {}{scope}{age}: {}\n", f.id, f.code, f.message)
|
||||
})
|
||||
.collect()
|
||||
}
|
||||
}
|
||||
|
||||
/// The files open findings point at - every desk page their fixes name,
|
||||
/// plus aggregates.yaml - when that is enough to act on. `None` when a
|
||||
/// finding is about getting around the site (a path, an entry page, a
|
||||
/// missing desk) or names no page: then the model needs to see it all.
|
||||
fn named_files<'a>(out: &Path, findings: impl Iterator<Item = &'a Finding>) -> Option<Vec<PathBuf>> {
|
||||
const WHOLE_SITE: &[&str] = &["schema", "entry-missing", "entry-gated", "path-missing", "path-closed", "path-over-budget", "desk-missing", "need-duplicate"];
|
||||
let mut files = vec![out.join("aggregates.yaml")];
|
||||
for f in findings {
|
||||
if WHOLE_SITE.contains(&f.code.as_str()) {
|
||||
return None;
|
||||
}
|
||||
let text = format!("{} {}", f.message, f.fix.as_deref().unwrap_or_default());
|
||||
for page in text.split_whitespace().filter(|w| w.starts_with('/')) {
|
||||
let page = page.trim_end_matches(|c: char| !(c.is_alphanumeric() || c == '_' || c == '-' || c == '/'));
|
||||
let rel = page.trim_start_matches('/');
|
||||
let candidates = if rel.is_empty() {
|
||||
vec![out.join("questions/index.yaml")]
|
||||
} else {
|
||||
vec![out.join(format!("questions/{rel}.yaml")), out.join(format!("questions/{rel}/index.yaml"))]
|
||||
};
|
||||
if let Some(path) = candidates.into_iter().find(|p| p.exists()) {
|
||||
if !files.contains(&path) {
|
||||
files.push(path);
|
||||
}
|
||||
}
|
||||
}
|
||||
}
|
||||
Some(files)
|
||||
}
|
||||
|
||||
/// `dump`, for the named files only.
|
||||
fn dump_files(out: &Path, files: &[PathBuf]) -> anyhow::Result<String> {
|
||||
let mut text = String::new();
|
||||
for path in files.iter().filter(|p| p.exists()) {
|
||||
let rel = path.strip_prefix(out).unwrap_or(path).display().to_string();
|
||||
text.push_str(&format!("<<<FILE {rel}>>>\n{}<<<END>>>\n\n", std::fs::read_to_string(path)?));
|
||||
}
|
||||
Ok(text)
|
||||
}
|
||||
|
||||
/// `<<<FILE path>>> ... <<<END>>>` blocks. Anything outside a block is
|
||||
/// ignored, so a model that adds a sentence does no harm.
|
||||
fn parse_files(text: &str) -> anyhow::Result<BTreeMap<String, String>> {
|
||||
@@ -591,3 +797,44 @@ mod tests {
|
||||
assert!(!allowed(".gitea/workflows/deploy.yaml"));
|
||||
}
|
||||
}
|
||||
|
||||
#[cfg(test)]
|
||||
mod ledger_tests {
|
||||
use super::*;
|
||||
|
||||
fn f(id: &str, code: &str, message: &str) -> Finding {
|
||||
Finding { id: id.into(), level: "fail".into(), code: code.into(), need: "hire".into(), message: message.into(), fix: None }
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn a_round_is_told_what_it_closed_kept_added_and_broke() {
|
||||
let mut ledger = Ledger::default();
|
||||
let p = ledger.update(1, vec![f("a", "strand", "x"), f("b", "strand", "y")]);
|
||||
assert_eq!((p.fixed.len(), p.new.len(), p.still, p.reopened.len()), (0, 2, 0, 0));
|
||||
let p = ledger.update(2, vec![f("b", "strand", "y"), f("c", "edge-unoffered", "z")]);
|
||||
assert_eq!((p.fixed.len(), p.new.len(), p.still, p.reopened.len()), (1, 1, 1, 0));
|
||||
assert_eq!(p.summary(ledger.open.len()), "2 open - fixed 1, still open 1, new 1, reopened 0");
|
||||
// "a" was closed in round 2; its return is a regression.
|
||||
let p = ledger.update(3, vec![f("a", "strand", "x"), f("b", "strand", "y")]);
|
||||
assert_eq!(p.reopened.iter().map(|f| f.id.as_str()).collect::<Vec<_>>(), ["a"]);
|
||||
assert!(ledger.open_list(4).contains("[b] strand \"hire\", open since round 1: y"));
|
||||
let p = ledger.update(4, vec![]);
|
||||
assert_eq!(p.summary(ledger.open.len()), "green (fixed 2)");
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn a_fix_round_sees_the_pages_its_findings_name() {
|
||||
let dir = std::env::temp_dir().join(format!("ledger-test-{}", std::process::id()));
|
||||
std::fs::create_dir_all(dir.join("questions/desk")).unwrap();
|
||||
std::fs::write(dir.join("aggregates.yaml"), "x").unwrap();
|
||||
std::fs::write(dir.join("questions/desk/index.yaml"), "x").unwrap();
|
||||
std::fs::write(dir.join("questions/index.yaml"), "x").unwrap();
|
||||
let mut strand = f("a", "strand", "records strand in \"quoted\"");
|
||||
strand.fix = Some("add `{ from: quoted, to: won, label: ... }` to the transitions on /desk".into());
|
||||
let files = named_files(&dir, [&strand].into_iter()).unwrap();
|
||||
assert_eq!(files, vec![dir.join("aggregates.yaml"), dir.join("questions/desk/index.yaml")]);
|
||||
// A finding about getting around the site needs all of it.
|
||||
assert!(named_files(&dir, [&strand, &f("b", "path-missing", "no path")].into_iter()).is_none());
|
||||
std::fs::remove_dir_all(&dir).ok();
|
||||
}
|
||||
}
|
||||
|
||||
@@ -1294,6 +1294,12 @@ textarea:focus {
|
||||
cursor: pointer;
|
||||
}
|
||||
|
||||
/* A releases manifest: the pick for this device, the rest folded. */
|
||||
.releases .pill { display: inline-block; }
|
||||
.releases pre { white-space: pre-wrap; font-size: 0.85rem; margin: 0.4rem 0 0; }
|
||||
.releases-all summary { color: var(--ink-dim); font-size: 0.9rem; cursor: pointer; }
|
||||
.releases-all ul { margin: 0.4rem 0 0 1rem; font-size: 0.9rem; }
|
||||
|
||||
.show-older:hover {
|
||||
color: inherit;
|
||||
border-color: var(--ink-dim);
|
||||
|
||||
Reference in New Issue
Block a user