Docs: the skill file says what the trainer does now

iris instead of question_lint everywhere; the skill's procedure is
plan, build, score: the opener and the look-around, mails and the
sender, moves for the person, doors only when measured, the lift
through a rebuild, the language and link guards, the descriptions
score over the headings score, and the spread a change has to beat.
README documents --llm-for.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
This commit is contained in:
Bendik Aagaard Lynghaug
2026-09-24 00:45:16 +02:00
co-authored by Claude Fable 5.1
parent deee27d280
commit 07da3b40f7
2 changed files with 83 additions and 74 deletions
+9 -6
View File
@@ -12,7 +12,7 @@ produced is content, and every line of it is held to a verifier.
``` ```
portal-trainer build --plan plans/contractor.yaml --out out/nordvik \ portal-trainer build --plan plans/contractor.yaml --out out/nordvik \
--lint question_lint --llm "MODEL=nemotron-3.5-lightning:30b-a3b engines/ollama.sh" --lint iris --llm "MODEL=sonnet engines/claude.sh"
``` ```
A `plan.yaml` is a needs spec that also says what each form collects A `plan.yaml` is a needs spec that also says what each form collects
@@ -30,7 +30,10 @@ result by construction; what it still checks is the budget.
The model writes words only, one page at a time, into a JSON object The model writes words only, one page at a time, into a JSON object
whose shape is fixed. Its strings are merged over built-in fallbacks whose shape is fixed. Its strings are merged over built-in fallbacks
key by key, so a bad reply costs wording, never a working site. key by key, so a bad reply costs wording, never a working site. `--llm-for <path prefix>` points the model at some pages only
(`aggregates.yaml` for the mails alone); a reply in the wrong language
for the plan is refused, and a mail that lost the submitter's link
gets it back.
### Onboarding from the plan ### Onboarding from the plan
@@ -195,7 +198,7 @@ against the lint. It is kept for the record of what it measured.
▼ phase 1 frozen after this phase: the generator ▼ phase 1 frozen after this phase: the generator
needs.yaml ─────────────────────── may not weaken the spec to pass needs.yaml ─────────────────────── may not weaken the spec to pass
│ │
▼ phase 2 ◄── question_lint findings, until it exits 0 ▼ phase 2 ◄── iris findings, until it exits 0
aggregates.yaml + questions/** + desks aggregates.yaml + questions/** + desks
│ │
▼ phase 3 ◄── misroutes from a blind visitor model ▼ phase 3 ◄── misroutes from a blind visitor model
@@ -208,7 +211,7 @@ against the lint. It is kept for the record of what it measured.
handles it, the states that count as finished, and the effort the handles it, the states that count as finished, and the effort the
visitor will tolerate. visitor will tolerate.
2. **Structure.** Pages, state machines and desks are generated, then 2. **Structure.** Pages, state machines and desks are generated, then
revised against portal's `question_lint` until every need is met: revised against the lint, `iris check`, until every need is met:
a path within budget, a desk held by the right group, every a path within budget, a desk held by the right group, every
finished state reachable, no record stranding. The page-writing finished state reachable, no record stranding. The page-writing
model sees the needs and never the personas. model sees the needs and never the personas.
@@ -222,7 +225,7 @@ and `questions/**.yaml` inside the output directory.
``` ```
portal-trainer generate --intake intakes/contractor.yaml --out out/nordvik \ portal-trainer generate --intake intakes/contractor.yaml --out out/nordvik \
--lint /srv/app/<inst>/current/question_lint --llm engines/ollama.sh --lint iris --llm engines/ollama.sh
portal-trainer answer --tasks tasks.jsonl --llm engines/ollama.sh > answers.jsonl portal-trainer answer --tasks tasks.jsonl --llm engines/ollama.sh > answers.jsonl
``` ```
@@ -232,7 +235,7 @@ sees another visitor's situation, a heading's place on the page cannot
carry the answer, and a rerun on the same site is the same experiment. carry the answer, and a rerun on the same site is the same experiment.
The earlier runs under `runs/` answered all tasks in one prompt. The earlier runs under `runs/` answered all tasks in one prompt.
The verifier is portal's `question_lint`, not code in this repo: it The verifier is uhhm/iris, not code in this repo: it
reads portal's own content types and ships in every portal release, so reads portal's own content types and ships in every portal release, so
a generated repo's CI keeps enforcing its needs after handover. a generated repo's CI keeps enforcing its needs after handover.
+74 -68
View File
@@ -1,6 +1,6 @@
--- ---
name: portal-trainer name: portal-trainer
description: Onboard a business onto portal - interview them, write their needs.yaml, then write the content repo (pages, aggregates.yaml, desks) until question_lint proves every need is met and an answering engine routes the personas right. Use when someone wants a new portal site, a new flow on an existing one, or asks why visitors pick the wrong alternative. description: Onboard a business onto portal - interview them, write a plan, build the content repo (pages, aggregates.yaml, desks, mails) with the words written by a model, and score it until iris proves every need is met and a blind visitor routes the personas right. Use when someone wants a new portal site, a new flow on an existing one, a rebuild of a live one, or asks why visitors pick the wrong alternative.
--- ---
# Portal trainer # Portal trainer
@@ -8,66 +8,64 @@ description: Onboard a business onto portal - interview them, write their needs.
You are onboarding a business onto [portal](https://project.uhhm.no/uhhm/portal): You are onboarding a business onto [portal](https://project.uhhm.no/uhhm/portal):
a site that is one question per page, a few alternatives per question, a site that is one question per page, a few alternatives per question,
each alternative optionally a form whose answer is recorded into a each alternative optionally a form whose answer is recorded into a
bucket with a state machine, worked from a group-gated desk. bucket with a state machine, worked from a group-gated desk, with a
mail to the person when their case moves.
You have a verifier. Use it instead of your judgment wherever it can You have a verifier. Use it instead of your judgment wherever it can
speak: `question_lint --path <repo>/questions` exits non-zero until speak: `iris check --path <repo>/questions` (uhhm/iris, the release
the structure meets every need in `<repo>/needs.yaml`. matching the portal the site runs) exits non-zero until the structure
meets every need in `<repo>/needs.yaml`.
## The order of work ## The order of work
The short path, when a business has filled in an intake: `plan` 1. **Interview, then write the plan.** Never pages first. `plan`
reads it into a `plan.yaml` (a model decides needs, stages, forms and reads a filled-in intake into `plan.yaml` (a model decides needs,
budgets; a person reads the result), `build` turns the plan into every stages, forms and budgets; a person reads the result before
file of the site with no model deciding anything, and a model writes anything is built). By hand, the same questions: who comes to the
only the words. See the repo README. The steps below are the same business and what each of them wants, who inside handles it, what
procedure done by hand, and what to do when the numbers come back. "handled" means (the states a case ends in), what it costs the
visitor to ask (`max_fields`, from the person, never from counting
1. **Interview, then write `needs.yaml` first.** Never pages first. the form you would like). Start from a `sectors/` file when there
Ask who comes to the business and what each of them wants, who in is one.
the business handles it, and what "handled" means - the states a 2. **Say what the plan can say.** `does:` (one or two features, one
case ends in. Ask what it costs the visitor to ask: how many role each, the front door's opener that asks for nothing);
submissions, how many required fields they will tolerate. That `shows:` (things to look at before asking); `doors:` only when the
number is `max_fields`, and it comes from the person, never from offers are more than a page holds - measure it, a flat page with
counting the form you would like to write. If the business is in a ten offers scored worse than four doors; `mails:` per need (who
sector under `sectors/`, start from that file and cut it down with hears when a case enters a stage) with `mail: { from }` on the
them; do not start blank. plan; `moves_by: submitter` so the person has something to play
2. **Write personas in the visitor's words, not the business's.** Two after sending; `invites_by` for onboarding; `events:` when
or three per need, including one who would not use the trade's own rebuilding a live site, so its records keep replaying.
vocabulary. Never reuse a phrase you intend to put on a page - a 3. **Write personas in the visitor's words, not the business's.** Two
persona that quotes the page tests nothing. or three per need, one who would not use the trade's vocabulary.
3. **Run the lint. It fails. Read every line.** Each failure names Never reuse a phrase you intend to put on a page. They are held
what is missing: no path, a closed (`disabled`) path, a field budget out from every model that writes words.
blown, no desk held by the handling group, a finished state no desk 4. **Build.** `build --plan ... --lint iris` derives every file with
move reaches, a state records strand in. no model deciding structure; `--llm` lets a model write the words,
4. **Write the least content that turns it green**: `aggregates.yaml` `--llm-for <path prefix>` only some of them (the mails alone);
(one aggregate per bucket, states the business named, only the `--words-from <live repo>` keeps a live site's hand-edited words
transitions they described), the visitor pages, one followup page through a rebuild, following each card across pages; `--words
per recording alternative saying what happens next, and one desk <json>` overrides slots. The builder refuses a reply in the wrong
page per handling group with exactly the moves the state machine language and puts back a submitter's link a model dropped.
allows. Mark a need `planned: true` only when the business says 5. **Score.** `iris check --needs-tasks` then `answer`, then
"later" - it keeps warning. `--needs-score`, with and without descriptions; `rate --repo` for
5. **Then word it.** `question_lint ... --needs-tasks tasks.jsonl`, readability and the four understanding questions, mails included.
and again with `--skim` (headings only - most visitors read nothing A card with no button is context, never a choice. Trust the
else). Give the tasks to an answering engine that never sees descriptions score; the headings-only score swings by a third
`expect`; grade with `--needs-score answers.jsonl`. between seeds. Before believing a change, `runs/spread.sh` it over
- A **misroute** is a wording defect. Fix the heading before the three seeds and require it to beat the spread. A change that does
description: a heading that needs its description has failed. not is rejected and kept as a run.
- Two alternatives a persona confuses are usually one distinction - A **misroute** is a wording defect until the same one survives
the business makes and the visitor does not. Name the visitor's every rewording; then it is a seam in the plan.
side of it. - Change the page or the plan, never the persona.
- **Hesitant** (right, under 0.6) means it works and is still a coin 6. **Keep the run.** `runs/<business>-<date>/` with the words as used
flip. Same treatment. (`words.json`), tasks, answers, scores per seed, and a README with
- Change the page, never the persona, to make a score go up. what moved and why. Runs are the record of what worked.
6. **Keep the run.** Copy tasks, answers and the score into
`runs/<business>-<date>/` with a line on which engine answered.
Runs are the record of which wording worked, per sector.
## What a page is ## What a page is
Full reference: portal's `README.md` and Full reference: portal's `README.md`; a live example is
`docs/design/filesystem-routes.md`; a live example is `uhhm/questions`. The tree under `questions/` is the router
`uhhm/questions`. The file tree under `questions/` is the router
(`index.yaml` is `/`, `_section.yaml` applies `qualifies` to a (`index.yaml` is `/`, `_section.yaml` applies `qualifies` to a
directory, nested non-index files are followups). directory, nested non-index files are followups).
@@ -76,6 +74,9 @@ directory, nested non-index files are followups).
name: What brings you here? # the question - always a question name: What brings you here? # the question - always a question
responsible: { name: ..., contact: ... } responsible: { name: ..., contact: ... }
alternatives: alternatives:
- name: What we do # the opener: features, no button
features:
- { name: <a role's win>, description: <what we do for them> }
- name: <heading the visitor picks> # graded alone under --skim - name: <heading the visitor picks> # graded alone under --skim
description: <one or two sentences> description: <one or two sentences>
action: /received # where the submission lands them action: /received # where the submission lands them
@@ -86,7 +87,6 @@ alternatives:
requirements: requirements:
- { name: email, label: Email, type: email } - { name: email, label: Email, type: email }
- { name: problem, label: What's wrong, type: textarea } - { name: problem, label: What's wrong, type: textarea }
- { name: address, label: Address, optional: true }
``` ```
```yaml ```yaml
@@ -95,15 +95,20 @@ aggregates:
- bucket: leads - bucket: leads
initial: open initial: open
states: states:
open: { event: received } open: { event: received, mail: { to: submitter, subject: "We have it", body: "..." } }
quoted: { event: quoted } quoted: { event: quoted }
won: { event: won } won: { event: won, mail: { to: submitter, subject: "...", body: "..." } }
lost: { event: lost } lost: { event: lost }
transitions: transitions:
open: [quoted, lost] open: [quoted, lost]
quoted: [won, lost] quoted: [won, lost]
``` ```
```yaml
# site.yaml
mail: { from: "Nordvik <post@nordvik.example>", reply_to: ola@nordvik.example }
```
```yaml ```yaml
# questions/desk/_section.yaml # questions/desk/_section.yaml
qualifies: project_managers qualifies: project_managers
@@ -120,26 +125,27 @@ alternatives:
requires_group: project_managers requires_group: project_managers
transitions: transitions:
- { to: quoted, label: Quote sent } # from defaults to open - { to: quoted, label: Quote sent } # from defaults to open
- { to: lost, label: Not for us }
- { from: quoted, to: won, label: Won } - { from: quoted, to: won, label: Won }
- { from: quoted, to: lost, label: Lost }
``` ```
## House rules for the copy ## House rules for the copy
- The page name is a question the visitor could ask themselves. - The page name is a question the visitor could ask themselves.
- The front door opens modestly: what the business does for one or
two roles, then what there is to look at, then the offers. No
second tier.
- Headings say what the visitor gets, in the visitor's words. Lead - Headings say what the visitor gets, in the visitor's words. Lead
with what a thing enables; never define it by what it isn't. with what a thing enables; never define it by what it isn't.
- Every required field must be one the business will actually use to - Every required field must be one the business will use to answer.
answer. If they can't say what they do with it, it's optional or
gone.
- A followup page says who reads it, when they answer, and what the - A followup page says who reads it, when they answer, and what the
visitor can do meanwhile. visitor can do meanwhile. A mail says what happened and, where the
person has a move, the link to make it.
- Code in English; the site in the business's language.
## Engines ## Engines
Any engine that reads a task's `state` and `questions` and returns Any engine that reads a task's `state` (`body` is the persona,
`{"id","choice","confidence"}` per line works. Compare at least two `page` what the page says before its options) and `questions` and
before trusting a score, and re-run the same engine before trusting a returns `{"id","choice","confidence"}` per line works. The visitor
change: one run of eight tasks is an anecdote. See the repo README for and the writer must be different models. See the repo README for what
what has been measured about each. has been measured about each, and for the noise floor.