Docs: the skill file says what the trainer does now
iris instead of question_lint everywhere; the skill's procedure is plan, build, score: the opener and the look-around, mails and the sender, moves for the person, doors only when measured, the lift through a rebuild, the language and link guards, the descriptions score over the headings score, and the spread a change has to beat. README documents --llm-for. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
This commit is contained in:
co-authored by
Claude Fable 5.1
parent
deee27d280
commit
07da3b40f7
@@ -12,7 +12,7 @@ produced is content, and every line of it is held to a verifier.
|
|||||||
|
|
||||||
```
|
```
|
||||||
portal-trainer build --plan plans/contractor.yaml --out out/nordvik \
|
portal-trainer build --plan plans/contractor.yaml --out out/nordvik \
|
||||||
--lint question_lint --llm "MODEL=nemotron-3.5-lightning:30b-a3b engines/ollama.sh"
|
--lint iris --llm "MODEL=sonnet engines/claude.sh"
|
||||||
```
|
```
|
||||||
|
|
||||||
A `plan.yaml` is a needs spec that also says what each form collects
|
A `plan.yaml` is a needs spec that also says what each form collects
|
||||||
@@ -30,7 +30,10 @@ result by construction; what it still checks is the budget.
|
|||||||
|
|
||||||
The model writes words only, one page at a time, into a JSON object
|
The model writes words only, one page at a time, into a JSON object
|
||||||
whose shape is fixed. Its strings are merged over built-in fallbacks
|
whose shape is fixed. Its strings are merged over built-in fallbacks
|
||||||
key by key, so a bad reply costs wording, never a working site.
|
key by key, so a bad reply costs wording, never a working site. `--llm-for <path prefix>` points the model at some pages only
|
||||||
|
(`aggregates.yaml` for the mails alone); a reply in the wrong language
|
||||||
|
for the plan is refused, and a mail that lost the submitter's link
|
||||||
|
gets it back.
|
||||||
|
|
||||||
### Onboarding from the plan
|
### Onboarding from the plan
|
||||||
|
|
||||||
@@ -195,7 +198,7 @@ against the lint. It is kept for the record of what it measured.
|
|||||||
▼ phase 1 frozen after this phase: the generator
|
▼ phase 1 frozen after this phase: the generator
|
||||||
needs.yaml ─────────────────────── may not weaken the spec to pass
|
needs.yaml ─────────────────────── may not weaken the spec to pass
|
||||||
│
|
│
|
||||||
▼ phase 2 ◄── question_lint findings, until it exits 0
|
▼ phase 2 ◄── iris findings, until it exits 0
|
||||||
aggregates.yaml + questions/** + desks
|
aggregates.yaml + questions/** + desks
|
||||||
│
|
│
|
||||||
▼ phase 3 ◄── misroutes from a blind visitor model
|
▼ phase 3 ◄── misroutes from a blind visitor model
|
||||||
@@ -208,7 +211,7 @@ against the lint. It is kept for the record of what it measured.
|
|||||||
handles it, the states that count as finished, and the effort the
|
handles it, the states that count as finished, and the effort the
|
||||||
visitor will tolerate.
|
visitor will tolerate.
|
||||||
2. **Structure.** Pages, state machines and desks are generated, then
|
2. **Structure.** Pages, state machines and desks are generated, then
|
||||||
revised against portal's `question_lint` until every need is met:
|
revised against the lint, `iris check`, until every need is met:
|
||||||
a path within budget, a desk held by the right group, every
|
a path within budget, a desk held by the right group, every
|
||||||
finished state reachable, no record stranding. The page-writing
|
finished state reachable, no record stranding. The page-writing
|
||||||
model sees the needs and never the personas.
|
model sees the needs and never the personas.
|
||||||
@@ -222,7 +225,7 @@ and `questions/**.yaml` inside the output directory.
|
|||||||
|
|
||||||
```
|
```
|
||||||
portal-trainer generate --intake intakes/contractor.yaml --out out/nordvik \
|
portal-trainer generate --intake intakes/contractor.yaml --out out/nordvik \
|
||||||
--lint /srv/app/<inst>/current/question_lint --llm engines/ollama.sh
|
--lint iris --llm engines/ollama.sh
|
||||||
portal-trainer answer --tasks tasks.jsonl --llm engines/ollama.sh > answers.jsonl
|
portal-trainer answer --tasks tasks.jsonl --llm engines/ollama.sh > answers.jsonl
|
||||||
```
|
```
|
||||||
|
|
||||||
@@ -232,7 +235,7 @@ sees another visitor's situation, a heading's place on the page cannot
|
|||||||
carry the answer, and a rerun on the same site is the same experiment.
|
carry the answer, and a rerun on the same site is the same experiment.
|
||||||
The earlier runs under `runs/` answered all tasks in one prompt.
|
The earlier runs under `runs/` answered all tasks in one prompt.
|
||||||
|
|
||||||
The verifier is portal's `question_lint`, not code in this repo: it
|
The verifier is uhhm/iris, not code in this repo: it
|
||||||
reads portal's own content types and ships in every portal release, so
|
reads portal's own content types and ships in every portal release, so
|
||||||
a generated repo's CI keeps enforcing its needs after handover.
|
a generated repo's CI keeps enforcing its needs after handover.
|
||||||
|
|
||||||
|
|||||||
+74
-68
@@ -1,6 +1,6 @@
|
|||||||
---
|
---
|
||||||
name: portal-trainer
|
name: portal-trainer
|
||||||
description: Onboard a business onto portal - interview them, write their needs.yaml, then write the content repo (pages, aggregates.yaml, desks) until question_lint proves every need is met and an answering engine routes the personas right. Use when someone wants a new portal site, a new flow on an existing one, or asks why visitors pick the wrong alternative.
|
description: Onboard a business onto portal - interview them, write a plan, build the content repo (pages, aggregates.yaml, desks, mails) with the words written by a model, and score it until iris proves every need is met and a blind visitor routes the personas right. Use when someone wants a new portal site, a new flow on an existing one, a rebuild of a live one, or asks why visitors pick the wrong alternative.
|
||||||
---
|
---
|
||||||
|
|
||||||
# Portal trainer
|
# Portal trainer
|
||||||
@@ -8,66 +8,64 @@ description: Onboard a business onto portal - interview them, write their needs.
|
|||||||
You are onboarding a business onto [portal](https://project.uhhm.no/uhhm/portal):
|
You are onboarding a business onto [portal](https://project.uhhm.no/uhhm/portal):
|
||||||
a site that is one question per page, a few alternatives per question,
|
a site that is one question per page, a few alternatives per question,
|
||||||
each alternative optionally a form whose answer is recorded into a
|
each alternative optionally a form whose answer is recorded into a
|
||||||
bucket with a state machine, worked from a group-gated desk.
|
bucket with a state machine, worked from a group-gated desk, with a
|
||||||
|
mail to the person when their case moves.
|
||||||
|
|
||||||
You have a verifier. Use it instead of your judgment wherever it can
|
You have a verifier. Use it instead of your judgment wherever it can
|
||||||
speak: `question_lint --path <repo>/questions` exits non-zero until
|
speak: `iris check --path <repo>/questions` (uhhm/iris, the release
|
||||||
the structure meets every need in `<repo>/needs.yaml`.
|
matching the portal the site runs) exits non-zero until the structure
|
||||||
|
meets every need in `<repo>/needs.yaml`.
|
||||||
|
|
||||||
## The order of work
|
## The order of work
|
||||||
|
|
||||||
The short path, when a business has filled in an intake: `plan`
|
1. **Interview, then write the plan.** Never pages first. `plan`
|
||||||
reads it into a `plan.yaml` (a model decides needs, stages, forms and
|
reads a filled-in intake into `plan.yaml` (a model decides needs,
|
||||||
budgets; a person reads the result), `build` turns the plan into every
|
stages, forms and budgets; a person reads the result before
|
||||||
file of the site with no model deciding anything, and a model writes
|
anything is built). By hand, the same questions: who comes to the
|
||||||
only the words. See the repo README. The steps below are the same
|
business and what each of them wants, who inside handles it, what
|
||||||
procedure done by hand, and what to do when the numbers come back.
|
"handled" means (the states a case ends in), what it costs the
|
||||||
|
visitor to ask (`max_fields`, from the person, never from counting
|
||||||
1. **Interview, then write `needs.yaml` first.** Never pages first.
|
the form you would like). Start from a `sectors/` file when there
|
||||||
Ask who comes to the business and what each of them wants, who in
|
is one.
|
||||||
the business handles it, and what "handled" means - the states a
|
2. **Say what the plan can say.** `does:` (one or two features, one
|
||||||
case ends in. Ask what it costs the visitor to ask: how many
|
role each, the front door's opener that asks for nothing);
|
||||||
submissions, how many required fields they will tolerate. That
|
`shows:` (things to look at before asking); `doors:` only when the
|
||||||
number is `max_fields`, and it comes from the person, never from
|
offers are more than a page holds - measure it, a flat page with
|
||||||
counting the form you would like to write. If the business is in a
|
ten offers scored worse than four doors; `mails:` per need (who
|
||||||
sector under `sectors/`, start from that file and cut it down with
|
hears when a case enters a stage) with `mail: { from }` on the
|
||||||
them; do not start blank.
|
plan; `moves_by: submitter` so the person has something to play
|
||||||
2. **Write personas in the visitor's words, not the business's.** Two
|
after sending; `invites_by` for onboarding; `events:` when
|
||||||
or three per need, including one who would not use the trade's own
|
rebuilding a live site, so its records keep replaying.
|
||||||
vocabulary. Never reuse a phrase you intend to put on a page - a
|
3. **Write personas in the visitor's words, not the business's.** Two
|
||||||
persona that quotes the page tests nothing.
|
or three per need, one who would not use the trade's vocabulary.
|
||||||
3. **Run the lint. It fails. Read every line.** Each failure names
|
Never reuse a phrase you intend to put on a page. They are held
|
||||||
what is missing: no path, a closed (`disabled`) path, a field budget
|
out from every model that writes words.
|
||||||
blown, no desk held by the handling group, a finished state no desk
|
4. **Build.** `build --plan ... --lint iris` derives every file with
|
||||||
move reaches, a state records strand in.
|
no model deciding structure; `--llm` lets a model write the words,
|
||||||
4. **Write the least content that turns it green**: `aggregates.yaml`
|
`--llm-for <path prefix>` only some of them (the mails alone);
|
||||||
(one aggregate per bucket, states the business named, only the
|
`--words-from <live repo>` keeps a live site's hand-edited words
|
||||||
transitions they described), the visitor pages, one followup page
|
through a rebuild, following each card across pages; `--words
|
||||||
per recording alternative saying what happens next, and one desk
|
<json>` overrides slots. The builder refuses a reply in the wrong
|
||||||
page per handling group with exactly the moves the state machine
|
language and puts back a submitter's link a model dropped.
|
||||||
allows. Mark a need `planned: true` only when the business says
|
5. **Score.** `iris check --needs-tasks` then `answer`, then
|
||||||
"later" - it keeps warning.
|
`--needs-score`, with and without descriptions; `rate --repo` for
|
||||||
5. **Then word it.** `question_lint ... --needs-tasks tasks.jsonl`,
|
readability and the four understanding questions, mails included.
|
||||||
and again with `--skim` (headings only - most visitors read nothing
|
A card with no button is context, never a choice. Trust the
|
||||||
else). Give the tasks to an answering engine that never sees
|
descriptions score; the headings-only score swings by a third
|
||||||
`expect`; grade with `--needs-score answers.jsonl`.
|
between seeds. Before believing a change, `runs/spread.sh` it over
|
||||||
- A **misroute** is a wording defect. Fix the heading before the
|
three seeds and require it to beat the spread. A change that does
|
||||||
description: a heading that needs its description has failed.
|
not is rejected and kept as a run.
|
||||||
- Two alternatives a persona confuses are usually one distinction
|
- A **misroute** is a wording defect until the same one survives
|
||||||
the business makes and the visitor does not. Name the visitor's
|
every rewording; then it is a seam in the plan.
|
||||||
side of it.
|
- Change the page or the plan, never the persona.
|
||||||
- **Hesitant** (right, under 0.6) means it works and is still a coin
|
6. **Keep the run.** `runs/<business>-<date>/` with the words as used
|
||||||
flip. Same treatment.
|
(`words.json`), tasks, answers, scores per seed, and a README with
|
||||||
- Change the page, never the persona, to make a score go up.
|
what moved and why. Runs are the record of what worked.
|
||||||
6. **Keep the run.** Copy tasks, answers and the score into
|
|
||||||
`runs/<business>-<date>/` with a line on which engine answered.
|
|
||||||
Runs are the record of which wording worked, per sector.
|
|
||||||
|
|
||||||
## What a page is
|
## What a page is
|
||||||
|
|
||||||
Full reference: portal's `README.md` and
|
Full reference: portal's `README.md`; a live example is
|
||||||
`docs/design/filesystem-routes.md`; a live example is
|
`uhhm/questions`. The tree under `questions/` is the router
|
||||||
`uhhm/questions`. The file tree under `questions/` is the router
|
|
||||||
(`index.yaml` is `/`, `_section.yaml` applies `qualifies` to a
|
(`index.yaml` is `/`, `_section.yaml` applies `qualifies` to a
|
||||||
directory, nested non-index files are followups).
|
directory, nested non-index files are followups).
|
||||||
|
|
||||||
@@ -76,6 +74,9 @@ directory, nested non-index files are followups).
|
|||||||
name: What brings you here? # the question - always a question
|
name: What brings you here? # the question - always a question
|
||||||
responsible: { name: ..., contact: ... }
|
responsible: { name: ..., contact: ... }
|
||||||
alternatives:
|
alternatives:
|
||||||
|
- name: What we do # the opener: features, no button
|
||||||
|
features:
|
||||||
|
- { name: <a role's win>, description: <what we do for them> }
|
||||||
- name: <heading the visitor picks> # graded alone under --skim
|
- name: <heading the visitor picks> # graded alone under --skim
|
||||||
description: <one or two sentences>
|
description: <one or two sentences>
|
||||||
action: /received # where the submission lands them
|
action: /received # where the submission lands them
|
||||||
@@ -86,7 +87,6 @@ alternatives:
|
|||||||
requirements:
|
requirements:
|
||||||
- { name: email, label: Email, type: email }
|
- { name: email, label: Email, type: email }
|
||||||
- { name: problem, label: What's wrong, type: textarea }
|
- { name: problem, label: What's wrong, type: textarea }
|
||||||
- { name: address, label: Address, optional: true }
|
|
||||||
```
|
```
|
||||||
|
|
||||||
```yaml
|
```yaml
|
||||||
@@ -95,15 +95,20 @@ aggregates:
|
|||||||
- bucket: leads
|
- bucket: leads
|
||||||
initial: open
|
initial: open
|
||||||
states:
|
states:
|
||||||
open: { event: received }
|
open: { event: received, mail: { to: submitter, subject: "We have it", body: "..." } }
|
||||||
quoted: { event: quoted }
|
quoted: { event: quoted }
|
||||||
won: { event: won }
|
won: { event: won, mail: { to: submitter, subject: "...", body: "..." } }
|
||||||
lost: { event: lost }
|
lost: { event: lost }
|
||||||
transitions:
|
transitions:
|
||||||
open: [quoted, lost]
|
open: [quoted, lost]
|
||||||
quoted: [won, lost]
|
quoted: [won, lost]
|
||||||
```
|
```
|
||||||
|
|
||||||
|
```yaml
|
||||||
|
# site.yaml
|
||||||
|
mail: { from: "Nordvik <post@nordvik.example>", reply_to: ola@nordvik.example }
|
||||||
|
```
|
||||||
|
|
||||||
```yaml
|
```yaml
|
||||||
# questions/desk/_section.yaml
|
# questions/desk/_section.yaml
|
||||||
qualifies: project_managers
|
qualifies: project_managers
|
||||||
@@ -120,26 +125,27 @@ alternatives:
|
|||||||
requires_group: project_managers
|
requires_group: project_managers
|
||||||
transitions:
|
transitions:
|
||||||
- { to: quoted, label: Quote sent } # from defaults to open
|
- { to: quoted, label: Quote sent } # from defaults to open
|
||||||
- { to: lost, label: Not for us }
|
|
||||||
- { from: quoted, to: won, label: Won }
|
- { from: quoted, to: won, label: Won }
|
||||||
- { from: quoted, to: lost, label: Lost }
|
|
||||||
```
|
```
|
||||||
|
|
||||||
## House rules for the copy
|
## House rules for the copy
|
||||||
|
|
||||||
- The page name is a question the visitor could ask themselves.
|
- The page name is a question the visitor could ask themselves.
|
||||||
|
- The front door opens modestly: what the business does for one or
|
||||||
|
two roles, then what there is to look at, then the offers. No
|
||||||
|
second tier.
|
||||||
- Headings say what the visitor gets, in the visitor's words. Lead
|
- Headings say what the visitor gets, in the visitor's words. Lead
|
||||||
with what a thing enables; never define it by what it isn't.
|
with what a thing enables; never define it by what it isn't.
|
||||||
- Every required field must be one the business will actually use to
|
- Every required field must be one the business will use to answer.
|
||||||
answer. If they can't say what they do with it, it's optional or
|
|
||||||
gone.
|
|
||||||
- A followup page says who reads it, when they answer, and what the
|
- A followup page says who reads it, when they answer, and what the
|
||||||
visitor can do meanwhile.
|
visitor can do meanwhile. A mail says what happened and, where the
|
||||||
|
person has a move, the link to make it.
|
||||||
|
- Code in English; the site in the business's language.
|
||||||
|
|
||||||
## Engines
|
## Engines
|
||||||
|
|
||||||
Any engine that reads a task's `state` and `questions` and returns
|
Any engine that reads a task's `state` (`body` is the persona,
|
||||||
`{"id","choice","confidence"}` per line works. Compare at least two
|
`page` what the page says before its options) and `questions` and
|
||||||
before trusting a score, and re-run the same engine before trusting a
|
returns `{"id","choice","confidence"}` per line works. The visitor
|
||||||
change: one run of eight tasks is an anecdote. See the repo README for
|
and the writer must be different models. See the repo README for what
|
||||||
what has been measured about each.
|
has been measured about each, and for the noise floor.
|
||||||
|
|||||||
Reference in New Issue
Block a user