uhhm/portal
1
1
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases 89 Wiki Activity

plan: intake to plan by a model; doors, labelled groups, the whole card

Browse Source 
Work from another session, committed as found (tests green):

- `plan` reads an intake into a plan.yaml (a model decides needs,
  stages, forms and budgets; a person reads the result before build).
- Plans gain `doors` (a first question before the forms, when a front
  door would carry more than five needs), `groups` with labels that a
  select may draw its options from (`options_from: groups`), `private`
  fields kept off public pages, `multiple` selects, and `max_fields`
  as what the intake said the person will tolerate.
- `card.rs`: an alternative as the visitor meets it, in text, so the
  answering visitor and `rate` read the served card, not the heading.
- `rate` reads followup pages from the repo and reports "what happens
  next" as not measured rather than zero when it cannot.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01L4jrCgLiKKHAEFuZUJjckH
This commit is contained in:
Bendik Aagaard Lynghaug
2026-09-23 12:31:51 +02:00
co-authored by Claude Fable 5.1
parent baf36319b7
commit 255aeb4f4b
570 changed files with 40795 additions and 181 deletions
Download Patch File Download Diff File Expand all files Collapse all files
README.md
+181 -12
View File
@@ -15,13 +15,18 @@ portal-trainer build --plan plans/contractor.yaml --out out/nordvik \
--lint question_lint --llm "MODEL=nemotron-3.5-lightning:30b-a3b engines/ollama.sh"
```
A `plan.yaml` is a needs spec that also says what each form collects.
Everything structural is derived from it with no model involved: the
state machines (read off `stages`: walk the line, be turned down at
any point, succeed only from the end, pass over `skippable` stages),
the front door, a followup page per need, a gated directory per group
with its desk and the pages only that group may open, and every button
the machine allows. The verifier accepts the result by construction.
A `plan.yaml` is a needs spec that also says what each form collects
and, per need, `max_fields`: how many required fields that person
will tolerate, taken from what the business said about them, never
from the form. `build` refuses a form over its budget, and hands the
budget to the verifier, so the check still means something after
handover. Everything structural is derived from the plan with no
model involved: the state machines (read off `stages`: walk the line,
be turned down at any point, succeed only from the end, pass over
`skippable` stages), the front door, a followup page per need, a gated
directory per group with its desk and the pages only that group may
open, and every button the machine allows. The verifier accepts the
result by construction; what it still checks is the budget.
The model writes words only, one page at a time, into a JSON object
whose shape is fixed. Its strings are merged over built-in fallbacks
@@ -31,14 +36,89 @@ key by key, so a bad reply costs wording, never a working site.
|---|---|---|
| A site the verifier accepts | never, in 6 rounds and 16 min | first try, 130 s |
| Held-out personas, with descriptions | - | 8 of 8 |
| Held-out personas, headings only | - | 8 of 8 |
| Held-out personas, headings only | - | 7 of 8 |
The wording numbers are from the re-score of 2026-09-22, one task per
call (`runs/contractor-build-2026-09-21/rescore-2026-09-22/`). The
first score, all tasks in one prompt, said 8 of 8 on headings only;
the apprentice picking the building survey over "Join our carpentry
team" only showed once no other visitor was in the prompt.
For comparison, a Sonnet-class model writing everything free-form
scored 8 of 10 on headings only and silently dropped a stage the owner
had described. `build` cannot drop a stage: it never decides one.
had described. `build` cannot drop a stage: it never decides one. The
denominators differ: the `build` site gives the two crew personas a
one-option page, so they generate no task, and 8 of 8 is eight public
personas where the `generate` number is ten. Held out means held out
from the model: the plan's `who` and `wants` lines and the personas
were written by the same person about the same needs, and the fallback
heading is the need's `wants`, so the two are cousins.
`generate` (below) remains for turning a plain-words intake into a
first plan; what it should hand over is the plan, not the pages.
### Doors: a first question before the forms
Five or six needs on one page is more than a visitor sorts through
(the full contractor plan had six). A plan can put public needs
behind `doors`: each door is one alternative on the front door
leading to a page of its own, where the needs that name it sit.
```yaml
doors:
- id: patients
who: someone with pain or an injury, or a patient who already has a time here
needs:
- id: new-patient
door: patients
...
```
The door's page is a nav page, public, holding the forms. The spec
gives such a need `max_steps: 2` and pins `via` to the door's heading,
so the verifier walks both pages. A door nobody is behind, a door on
an internal need, or a door named like a group is refused, and a
front door over five alternatives is warned about. The visitor is
scored on both pages: a persona who picks the wrong door is a
misroute on `/`, one who picks the wrong form is a misroute on the
door's page.
Cases with doors: `plans/physio.yaml` (a physiotherapy clinic,
patients and referrers), `plans/fotballklubb.yaml` (a football club,
in Norwegian, parents and helpers), `plans/hire.yaml` (a machine hire
yard, hiring and firms), and `plans/contractor-full.yaml` (three
doors over six public needs). Each has an intake under `intakes/` in
the business's own words and held-out personas.
### What a plan can say about a form and its readers
- `multiple: true` on a `select`: pick any number (portal renders
toggles). What a volunteer can do, which events, which committees.
- `groups:` at the top, a list of `{ id, label }`: the groups with a
name the public knows, such as a vel's committees. A select with
`options_from: groups` offers exactly those labels, so the
application form and the desks cannot drift apart.
- `watched_by: [public]`: an ungated read-only page of the bucket at
`/status/<bucket>`, linked from the followup page. Everyone sees
what was sent in and where it stands. `email` and `tel` fields, and
any field marked `private: true`, are projected away before it is
shown.
## Plan: the intake, read by a model, checked by a person
```
portal-trainer plan --intake intakes/contractor.yaml --out out/nordvik \
--personas sectors/construction/needs.yaml \
--llm "MODEL=nemotron-3.5-lightning:30b-a3b engines/ollama.sh"
```
This is the one step where a model decides structure: which needs the
intake implies, the stages, who handles what, the form, the budget.
Its YAML is parsed and validated exactly as `build` will, the errors go
back to it, and what passes is written verbatim for a person to read
before building from it. Personas are never asked of this model; they
are attached from a file, and where their `need:` names an id the plan
does not have, `plan` says so and leaves the mapping to the reader.
`generate` (below) is the older path, a model writing pages free-form
against the lint. It is kept for the record of what it measured.
## How it works
@@ -79,6 +159,12 @@ portal-trainer generate --intake intakes/contractor.yaml --out out/nordvik \
portal-trainer answer --tasks tasks.jsonl --llm engines/ollama.sh > answers.jsonl
```
`answer` puts one task per call to the visitor model, with the options
shuffled per task and the call seeded from the task id: no visitor
sees another visitor's situation, a heading's place on the page cannot
carry the answer, and a rerun on the same site is the same experiment.
The earlier runs under `runs/` answered all tasks in one prompt.
The verifier is portal's `question_lint`, not code in this repo: it
reads portal's own content types and ships in every portal release, so
a generated repo's CI keeps enforcing its needs after handover.
@@ -90,7 +176,8 @@ A backend is any command that takes a prompt on stdin and prints text.
| Backend | Use |
|---|---|
| `engines/ollama.sh` | Local Ollama. `MODEL=qwen3:8b` answers a task in about two seconds on ergo's GPU and is the default visitor. Generation wants a larger model (`MODEL=qwen3.6:27b`), which runs mostly on CPU there. |
| `engines/anthropic.sh` | The Anthropic API, given a working `ANTHROPIC_API_KEY`. |
| `engines/claude.sh` | Claude through Claude Code's own login (a Max or Pro subscription), no API key. `MODEL=sonnet` or `opus`, `EFFORT=low` by default. It drops any `ANTHROPIC_API_KEY` from the environment, because a key outranks the login and a stale one fails every call. Log in once with `claude auth login`. |
| `engines/anthropic.sh` | The Anthropic API, given a working `ANTHROPIC_API_KEY` from the Console. That is separate, prepaid billing; a subscription does not cover it. |
| `engines/mailbox.sh` | Writes each prompt to a directory and waits for a reply file: drive the generator from an agent session, or by hand. |
The visitor and the generator should be different models. A model
@@ -107,6 +194,51 @@ every field the generator reads, so the step that remains is a
listener on `portal.answers.submitted` that runs it and pushes the
draft to a new repo. It passes its own `needs.yaml`.
## The whole card, not the heading
A visitor never sees a bare heading and description. Portal renders a
card: a button, an encouragement, one or more sections each with an
icon, a title and a line of guidance, and the fields with their
labels and placeholders. Since 2026-09-23 the builder fills all of it
and the scorers read all of it.
- **Slots.** Beside the heading and description, each form alternative
gets a button, an encouragement, an icon, a form title and guidance
line, and a placeholder per field. A door gets an icon and a "for
whom" line. Icons are picked from a fixed list of Iconify lucide
names; anything else is dropped.
- **Context from the plan.** Every form alternative also carries two
sections built from what the plan already knows, before the form:
what happens next (the stages, in order, as one sentence) and who
reads it (the handling group and the contact). Their fallbacks are
true before a model has written a word; the model rewords them.
- **Scoring.** `answer --repo <content dir>` shows the visitor each
option as its whole card (headings-only tasks stay headings only).
`rate --repo` does the same and asks a fourth question, "where": do
you know where you are, who this is for, and what the button does.
All four are controlled against the gutted twin.
## Rating wording
```
question_lint --path out/nordvik/questions --needs-sim model.json
portal-trainer rate --model model.json --out out/rated --repo out/nordvik \
--llm "MODEL=qwen3:8b engines/ollama.sh"
```
Four yes-or-no questions per persona (which option is mine, where am
I and what does the button do, what will I be asked, what happens
after I send it), each also put to a gutted
twin of the page with headings, descriptions, the form and the
followup removed. A question the twin still gets a yes on is not
measuring the page, and `rate` says which. The followup page is read
from the repo (`--repo`); without it "what happens next" is reported
as not measured rather than as zero. LIX is over the prose only
(question, descriptions, followup): headings and labels are not
sentences and would pull every page down to "very easy". The
`rate.json` in `runs/contractor-build-2026-09-21/` predates this and
its `asked` and `next` columns are the old, pinned ones.
## Measured so far
**Generated: a building contractor** (`runs/contractor-2026-09-21/`),
@@ -133,6 +265,43 @@ from `intakes/contractor.yaml`, a Sonnet-class model generating.
chose "Get someone to look at a building". The generator has since
been changed to hide personas from the page-writing phase.
**Built: the same contractor from a plan** (`runs/contractor-build-2026-09-21/`),
nemotron writing words only, and re-scored one task per call the day
after (`rescore-2026-09-22/` in it): 8 of 8 with descriptions, 7 of 8
on headings alone, the miss being an apprentice bricklayer reading
"Join our carpentry team" as not for them. `rate` on the same site,
with the followup shown and every question controlled, passes its
controls and puts the pages at LIX 18 to 35.
**Built behind doors, 2026-09-22.** Five plans with a first question
before the forms, each built and then scored by qwen3:8b one task per
call, against held-out personas. Behind a door a persona is asked
twice, on the front door and on the door's page.
| Case | Words by | With descriptions | Headings only | Notes |
|---|---|---|---|---|
| Machine hire (`hire`) | nemotron | 10 of 10 | 10 of 10 | doors that ask a question the visitor already knows the answer to |
| Physio clinic (`physio`) | nemotron | 8 of 10 | 9 of 10 | "Book your first appointment" fails a daughter booking for her mother |
| Football club, Norwegian (`fotballklubb`) | nemotron | 14 of 16 | 11 of 16 | three parents walk past "Er det på tide å melde inn?" |
| Football club, Norwegian | Claude Sonnet | 13 of 16 | 12 of 16 | better sentences, same driver/sponsor blur |
| Whole contractor (`contractor-full`) | nemotron | 18 of 22 | 15 of 22 | "Apply to work with us" and "Find a job with us" cross over for all four |
| Tomter Vel, Norwegian (`tomtervel`), first plan | Claude Sonnet | 25 of 30 | 22 of 30 | "utstyr" for a swing, hearing input behind the wrong door |
| Tomter Vel, plan revised 2026-09-23, whole card | Claude Sonnet | 31 of 38 | 30 of 38 | committees as groups, public lists, hearing input on the front door |
Each run's README names the misroutes and what in the plan or the
heading caused them. A stronger word model did not route better on
the one plan built with both: routing is decided by which distinctions
sit on one page, and the visitor grades that the same whoever wrote
the words.
**Planned: the contractor intake read by a local model**
(`runs/contractor-plan-2026-09-22/`), nemotron, one round, 41 s. The
plan passed every check `build` makes and a reader finds three
structural misses in it the checks cannot: the crew's report made
public, the "papers missing" stage dropped, and no need for the
project manager logging a call. The person reading the plan is not
optional.
**Existing site: uhhm.no** (`runs/uhhm-2026-09-21/`), eight tasks.
| Visitor model | With descriptions | Headings only |