uhhm/portal
1
1
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases 89 Wiki Activity

One binary: the lint and the trainer are commands of portal
Test / test (pull_request) Successful in 1m7s
Details

Browse Source 
portal serve runs a site; portal lint is what iris was; portal plan,
build, report and the rest are what portal-trainer was. One source at
one version, so the three never disagree about what content means.
Both histories are merged in (subtree). The release attaches the bare
binary for a content repo's CI. The trainer's plans, intakes, engines
and reference prompt come along; its runs stay in the old repo.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01GT37Z1Xtfd9pUuQtMTg6Yt
This commit is contained in:
Bendik Aagaard Lynghaug
2026-09-30 15:29:30 +02:00
co-authored by Claude Fable 5.1
parent d0d1cbf23d
commit 96b3746643
1733 changed files with 183 additions and 150652 deletions
Download Patch File Download Diff File Expand all files Collapse all files
docs/README.md
+7 -4
View File
@@ -14,12 +14,15 @@ engine does with it.
| [Storylines](storylines.md) | The same site told several ways: lists of texts, one storyline per visitor, counted per storyline. |
| [Memberships](memberships.md) | Groups kept by portal itself: a membership as a case, a person as a phone number first, and the provider only saying who. |
| [Operations](operations.md) | Releasing, deploying, adding a site, environment variables. |
| [Training a site](training.md) | `portal plan` and `portal build`: a business in plain words becomes a content repo, held to the lint. |
| [The lint](lint.md) | `portal lint`: what a content repo's CI runs before anything reaches a site. |
| [Filesystem routes](design/filesystem-routes.md) | Why the `questions/` tree is the router. |
Two repos in the same family carry their own docs:
One repo in the same family carries its own docs:
- [uhhm/iris](https://project.uhhm.no/uhhm/iris) — the lint. Versioned
as portal and released by portal's publish job; it is the thing a
content repo's CI runs before anything reaches a site.
- [uhhm/gdo](https://project.uhhm.no/uhhm/gdo) — the mailer. Delivers
what content asked to be sent, and carries replies back.
The lint (once uhhm/iris) and the trainer (once uhhm/portal-trainer)
are commands of the `portal` binary since 0.6; their chapters are
above, and the trainer's runs stay in the old repo.
docs/SUMMARY.md
+6
View File
@@ -12,6 +12,11 @@
- [Memberships](memberships.md)
- [Operations](operations.md)
# The trainer and the lint
- [Training a site](training.md)
- [The lint](lint.md)
# How it is made
- [Architecture](architecture.md)
@@ -20,3 +25,4 @@
---
[Changelog](changelog.md)
[The lint's changelog, to 0.5.11](lint-changelog.md)
docs/lint-changelog.md
+104
View File
@@ -0,0 +1,104 @@
# Changelog
One line per change, grouped by the release that shipped it. Newest
first. Each release names the portal tag it is built against.
## Unreleased
- `needs.yaml` may name the site's `tones`, one storyline each; the checks read the first storyline, which is what `build_questions` gives.
## 0.5.10 (2026-09-30) - portal v0.5.10
- Matches portal v0.5.10 (released by portal's publish job).
## 0.5.9 (2026-09-30) - portal v0.5.9
- Matches portal v0.5.9 (released by portal's publish job).
## 0.5.8 (2026-09-30) - portal v0.5.8
- Matches portal v0.5.8 (released by portal's publish job).
## 0.5.7 (2026-09-30) - portal v0.5.7
- Matches portal v0.5.7 (released by portal's publish job).
## Unreleased
- The effort budget counts what a person fills in: a `type: record` field (the case an answer is about, carried on the link) and a `hidden` preset are not asked and no longer count. An objection page whose form carried the application's id used to be one field over.
## 0.5.6 (2026-09-30) - portal v0.5.6
- Matches portal v0.5.6 (released by portal's publish job).
## 0.5.5 (2026-09-30) - portal v0.5.5
- Matches portal v0.5.5 (released by portal's publish job).
## 0.5.4 (2026-09-29) - portal v0.5.4
- Matches portal v0.5.4 (released by portal's publish job).
## 0.5.3 (2026-09-29) - portal v0.5.3
- Matches portal v0.5.3 (released by portal's publish job).
## 0.5.2 (2026-09-28) - portal v0.5.2
- Matches portal v0.5.2 (released by portal's publish job).
## 0.5.1 (2026-09-28) - portal v0.5.1
- Matches portal v0.5.1 (released by portal's publish job).
## 0.5.0 (2026-09-28) - portal v0.5.0
- Matches portal v0.5.0 (released by portal's publish job).
## 0.4.0 (2026-09-24) - portal v0.4.0
- Matches portal v0.4.0 (released by portal's publish job).
## 0.3.47 (2026-09-24) - portal v0.3.47
- Matches portal v0.3.47 (released by portal's publish job).
## 0.3.46 (2026-09-23) - portal v0.3.46
- Matches portal v0.3.46 (released by portal's publish job).
## 0.3.45 (2026-09-23) - portal v0.3.45
- Matches portal v0.3.45 (released by portal's publish job).
## 0.3.44 (2026-09-23) - portal v0.3.44
- Matches portal v0.3.44 (released by portal's publish job).
## 0.3.43 (2026-09-23) - portal v0.3.43
- Matches portal v0.3.43 (released by portal's publish job).
## 0.3.42 (2026-09-23) - portal v0.3.42
- Matches portal v0.3.42 (released by portal's publish job).
## 0.3.41 (2026-09-23) - portal v0.3.41
- Matches portal v0.3.41 (released by portal's publish job).
## 0.3.39 (2026-09-23) - portal v0.3.39
- Matches portal v0.3.39 (released by portal's publish job).
## 0.1.1 (2026-09-22) - portal v0.3.36
- `check --repo` fetches needs.yaml itself; the v0.1.0 tag never published (a cache permission on the runner, since fixed).
## 0.1.0 (2026-09-22) - portal v0.3.36
- First release: the lint moved out of portal (`question_lint` becomes
`iris check`, `needs_replay` becomes `iris replay`), with the
business-needs pass, the wording task export and scoring, and the
simulation model export. Portal is a library dependency pinned to
the release iris matches.
docs/lint.md
+54
View File
@@ -0,0 +1,54 @@
# The lint
The iris over a site. A content push is an incoming wormhole; nothing
comes through until it checks out.
```
portal lint --path questions # a local checkout
portal lint --repo https://project.uhhm.no/uhhm/questions
portal lint replay --path questions --nats nats://127.0.0.1:14222
```
It was its own binary, iris, until portal 0.6: the same code, released
as portal's twin. Now it is the `portal` binary's `lint` command, so a
content repo's CI downloads the release and runs it:
```yaml
- run: |
curl -sfL "https://project.uhhm.no/uhhm/portal/releases/download/$PORTAL_RELEASE/portal" -o portal
chmod +x portal && ./portal lint --path questions
```
`check` loads a content repo exactly the way the running site does,
validates it (every action lands on a page, every transition is one
the state machine allows, every recorded bucket is read back
somewhere), and then holds it to the business needs the repo declares
in `needs.yaml`, when it has one:
- a path of submissions gets each kind of visitor's answer into the
bucket it names, within a submission and required-field budget;
- the handling group holds a desk over that bucket, and every
finished state is reachable through the buttons it (and any
`also_moved_by` group) offers; no record strands on the way;
- every stage the business named is a state of the machine.
A need marked `planned` warns instead of failing. Exit status is the
verdict, so a content repo's CI is one line.
Beyond the verdict, `check` exports what a trainer needs to grade the
wording: `--needs-tasks` writes the personas as typed choice tasks
(`--skim` for headings only), `--needs-score` grades an engine's
answers, and `--needs-sim` writes a resolved model of the whole site.
`portal lint replay` runs simulated cases through the site's real aggregate
engine on a throwaway JetStream and reports every bucket by state;
`--report-only` takes the same scorecard from a live site's buckets.
See [Training a site](training.md) for what sits around those.
## Matching the site
The lint is the site: `portal lint` and `portal serve` are one binary
built from one source, so a content shape the site reads is a shape
the lint knows, at the same version. A content repo pins
`PORTAL_RELEASE` to what its site runs and lints with that release's
`portal`. The `IRIS_RELEASE` pins from before portal 0.6 keep working
against the iris releases that exist; nothing new is published there.
docs/training.md
+490
View File
@@ -0,0 +1,490 @@
# Training a site
A business describes itself in plain words: what it does, who comes to
it, what it keeps track of and through which stages, and who handles
what. The trainer, `portal plan` and `portal build`, turns that into a
working content repo: a question series for every role, the state
machines behind them, and a desk per handling role. No software is
written for the business; the only thing produced is content, and
every line of it is held to the lint (`portal lint`), which is the
same code as the site, at the same version.
The trainer was its own repo, uhhm/portal-trainer, until portal 0.6;
its runs (`runs/`, the lab notebook of every site trained so far)
stay there.
## Build: structure from the plan, words from the model
```
portal build --plan plans/contractor.yaml --out out/nordvik \
--lint "portal lint" --llm "MODEL=sonnet engines/claude.sh"
```
A `plan.yaml` is a needs spec that also says what each form collects
and, per need, `max_fields`: how many required fields that person
will tolerate, taken from what the business said about them, never
from the form. `build` refuses a form over its budget, and hands the
budget to the verifier, so the check still means something after
handover. Everything structural is derived from the plan with no
model involved: the state machines (read off `stages`: walk the line,
be turned down at any point, succeed only from the end, pass over
`skippable` stages), the front door, a followup page per need, a gated
directory per group with its desk and the pages only that group may
open, and every button the machine allows. The verifier accepts the
result by construction; what it still checks is the budget.
The model writes words only, one page at a time, into a JSON object
whose shape is fixed. Its strings are merged over built-in fallbacks
key by key, so a bad reply costs wording, never a working site. `--llm-for <path prefix>` points the model at some pages only
(`aggregates.yaml` for the mails alone); a reply in the wrong language
for the plan is refused, and a mail that lost the submitter's link
gets it back.
### Onboarding from the plan
`invites_by: [styret]` on a plan, or `invited_by: [...]` on a group,
puts an invite form on the inviting group's desk for each group it may
fill, and a list of the invites sent. Portal does the rest (an account
in Kanidm, the group, a one-time credential link) through `invite:` on
the alternative. Every group with a desk invites into itself by
default.
### Rebuilding a live site
`--words-from <repo>` lifts the words a repo already has back into the
words map by what each card is (its bucket, the page it leads to, the
group it invites into), across the whole repo, so a rebuild keeps
hand-edited wording even when a card moves page (a door taken away, a
need brought onto the front door) and
adds only the new slots. Cards are matched by what they are (the
bucket they record into, the page they lead to, a field's name, the
group they invite into), not by position, so a rebuild that reorders
or reshapes a page still hands each card its own words; on uhhm.no
the live front door's three headings follow their buckets onto the
new page. `--words
<json>` overrides slots by page and may repeat, each file layered over
the last (the words a build used, then a round of hand changes);
`words.json` in the output is the words as used. See
`runs/tomtervel-invites-2026-09-23` and `runs/uhhm-2026-09-23`.
| Same contractor, same local model (nemotron 30B-A3B on ergo) | Free-form `generate` | `build` |
|---|---|---|
| A site the verifier accepts | never, in 6 rounds and 16 min | first try, 130 s |
| Held-out personas, with descriptions | - | 8 of 8 |
| Held-out personas, headings only | - | 7 of 8 |
The wording numbers are from the re-score of 2026-09-22, one task per
call (`runs/contractor-build-2026-09-21/rescore-2026-09-22/`). The
first score, all tasks in one prompt, said 8 of 8 on headings only;
the apprentice picking the building survey over "Join our carpentry
team" only showed once no other visitor was in the prompt.
For comparison, a Sonnet-class model writing everything free-form
scored 8 of 10 on headings only and silently dropped a stage the owner
had described. `build` cannot drop a stage: it never decides one. The
denominators differ: the `build` site gives the two crew personas a
one-option page, so they generate no task, and 8 of 8 is eight public
personas where the `generate` number is ten. Held out means held out
from the model: the plan's `who` and `wants` lines and the personas
were written by the same person about the same needs, and the fallback
heading is the need's `wants`, so the two are cousins.
### The front door opens modestly
A front door with more buttons than text leaves a visitor paralysed.
So the built front door opens with what the business does, in one or
two features and no button at all (`does:` on the plan, at most two,
each `for:` a need so it speaks to one role and is headed by what
that role gets); then a look-around card, things a visitor may
inspect before asking for anything (`shows:`, each a portal resource
block passed through verbatim, plus every public view the plan
opens); and only then the offers, as more alternatives on the same
page. Nothing is layered: doors still exist for a plan that wants
them, but a second tier is what this shape is there to avoid, and on
uhhm.no a flat front door with an opener routed better than the
doored one.
The house and its players: a role the opener invites in has to see
its win, and the machine has to give it something to do after it
sits down. `build` warns when a featured need gives the submitter
neither a move (`moves_by: {<stage>: [submitter, ...]}`) nor a public
view; a player with no move is a spectator.
### Doors: a first question before the forms
Five or six needs on one page is more than a visitor sorts through
(the full contractor plan had six). A plan can put public needs
behind `doors`: each door is one alternative on the front door
leading to a page of its own, where the needs that name it sit.
```yaml
doors:
- id: patients
who: someone with pain or an injury, or a patient who already has a time here
needs:
- id: new-patient
door: patients
...
```
The door's page is a nav page, public, holding the forms. The spec
gives such a need `max_steps: 2` and pins `via` to the door's heading,
so the verifier walks both pages. A door nobody is behind, a door on
an internal need, or a door named like a group is refused, and a
front door over five alternatives is warned about. The visitor is
scored on both pages: a persona who picks the wrong door is a
misroute on `/`, one who picks the wrong form is a misroute on the
door's page.
Cases with doors: `plans/physio.yaml` (a physiotherapy clinic,
patients and referrers), `plans/fotballklubb.yaml` (a football club,
in Norwegian, parents and helpers), `plans/hire.yaml` (a machine hire
yard, hiring and firms), and `plans/contractor-full.yaml` (three
doors over six public needs). Each has an intake under `intakes/` in
the business's own words and held-out personas.
### What a plan can say about a form and its readers
- `multiple: true` on a `select`: pick any number (portal renders
toggles). What a volunteer can do, which events, which committees.
- `groups:` at the top, a list of `{ id, label }`: the groups with a
name the public knows, such as a vel's committees. A select with
`options_from: groups` offers exactly those labels, so the
application form and the desks cannot drift apart.
- `watched_by: [public]`: an ungated read-only page of the bucket at
`/status/<bucket>`, linked from the followup page. Everyone sees
what was sent in and where it stands. `email` and `tel` fields, and
any field marked `private: true`, are projected away before it is
shown.
- `events: { open: applied, active: onboarded }` on a need: the event
name a stage is entered by, where the builder's default (the stage's
own name, `received` for the first) is not what a live site already
has. Portal replays every record by event name, so a rebuild of a
site that holds records keeps the names its aggregates.yaml has, or
every record vanishes from its desk. `plans/uhhm.yaml` rebuilds
uhhm.no this way.
- `mails: { <stage>: submitter | <address> }` on a need: who hears
when a case enters that stage, sent by portal the moment it moves
(portal 0.3.43, delivered by uhhm/gdo on the host). `submitter` goes
to the email the form collected, so the form must collect a required
`email`; an address is the treasurer, the board. The plan then needs
`mail: { from, reply_to }` at the top, the sender every mail carries
into `site.yaml`. The words model writes each mail's subject and
body (`aggregates.yaml` in the words map); the fallback says what
happened and, where the submitter has a move from that stage, the
link to make it.
- `stages_text: { <stage>: <words> }` on a need: each stage in the
plan's language, where the ASCII id will not do (`ikke_lost` reads
as English). Used by the fallback mails, the "what happens next"
line, the desk buttons' fallback labels, and written into needs.yaml
as `stage_words` so the desk score describes an outcome in the
site's own words. With them, the vel's desks score 26 of 26.
- `turned_down_from: [<stage>, ...]` on a need: the unfinished stages
a case can still be turned down from; omitted, every one. An
applicant is declined before they start and departs after, which
is what uhhm.no's live machine says and what two near-synonymous
desk buttons had been standing in for.
- `reached_from: { <stage>: [<stage>, ...] }` on a need: where a stage
is entered from, when the line a case walks does not say it. Two
shapes need it and nothing else does - a case SENT BACK to a stage it
was already past (a building application returned for the drawing it
was missing, after the hearing turned something up), and a GOOD ENDING
PARTWAY ALONG (an authority that changes its own decision while it is
looking at the appeal, rather than passing it up to the board). A
stage named there is entered from exactly those stages and nowhere
else, so saying where a refusal may come from also says where it may
not: an applicant may withdraw from anywhere, and be refused only once
the papers are in, which `turned_down_from` could not express because
it was one list for every bad ending at once. Found by replaying a
building authority's real cases through portal's engine, which
refused three legal paths and accepted two illegal ones until the
plan could say this; see runs/permits-2026-09-25/README.md.
- `attended_by:` on a need: who moves cases that no desk button moves,
an automation listening on the bucket's events, written into
aggregates.yaml as portal's own annotation for the reader.
## Plan: the intake, read by a model, checked by a person
```
portal plan --intake intakes/contractor.yaml --out out/nordvik \
--personas sectors/construction/needs.yaml \
--llm "MODEL=nemotron-3.5-lightning:30b-a3b engines/ollama.sh"
```
This is the one step where a model decides structure: which needs the
intake implies, the stages, who handles what, the form, the budget.
Its YAML is parsed and validated exactly as `build` will, the errors go
back to it, and what passes is written verbatim for a person to read
before building from it. Personas are never asked of this model; they
are attached from a file, and where their `need:` names an id the plan
does not have, `plan` says so and leaves the mapping to the reader.
`generate` (below) is the older path, a model writing pages free-form
against the lint. It is kept for the record of what it measured.
## How it works
```
intake (plain words)
│
▼ phase 1 frozen after this phase: the generator
needs.yaml ─────────────────────── may not weaken the spec to pass
│
▼ phase 2 ◄── iris findings, until it exits 0
aggregates.yaml + questions/** + desks
│
▼ phase 3 ◄── misroutes from a blind visitor model
wording
```
1. **Needs.** The intake becomes `needs.yaml`: one need per kind of
person and thing they want, including the roles inside the
business. Each names the bucket the answer lands in, the group that
handles it, the states that count as finished, and the effort the
visitor will tolerate.
2. **Structure.** Pages, state machines and desks are generated, then
revised against the lint, `iris check`, until every need is met:
a path within budget, a desk held by the right group, every
finished state reachable, no record stranding. The page-writing
model sees the needs and never the personas.
3. **Wording.** A separate model plays each persona, blind, first with
headings only and then with descriptions. A misroute is a wording
defect and goes back to the generator. A rewrite that breaks
structure is reverted.
Model output is only ever written to `aggregates.yaml`, `site.yaml`
and `questions/**.yaml` inside the output directory.
```
portal generate --intake intakes/contractor.yaml --out out/nordvik \
--lint "portal lint" --llm engines/ollama.sh
portal answer --tasks tasks.jsonl --llm engines/ollama.sh > answers.jsonl
```
`answer` puts one task per call to the visitor model, with the options
shuffled per task and the call seeded from the task id: no visitor
sees another visitor's situation, a heading's place on the page cannot
carry the answer, and a rerun on the same site is the same experiment.
The earlier runs under `runs/` answered all tasks in one prompt.
The verifier is uhhm/iris, not code in this repo: it
reads portal's own content types and ships in every portal release, so
a generated repo's CI keeps enforcing its needs after handover.
## LLM backends
A backend is any command that takes a prompt on stdin and prints text.
| Backend | Use |
|---|---|
| `engines/ollama.sh` | Local Ollama. `MODEL=qwen3:8b` answers a task in about two seconds on ergo's GPU and is the default visitor. Generation wants a larger model (`MODEL=qwen3.6:27b`), which runs mostly on CPU there. |
| `engines/claude.sh` | Claude through Claude Code's own login (a Max or Pro subscription), no API key. `MODEL=sonnet` or `opus`, `EFFORT=low` by default. It drops any `ANTHROPIC_API_KEY` from the environment, because a key outranks the login and a stale one fails every call. Log in once with `claude auth login`. |
| `engines/anthropic.sh` | The Anthropic API, given a working `ANTHROPIC_API_KEY` from the Console. That is separate, prepaid billing; a subscription does not cover it. |
| `engines/mailbox.sh` | Writes each prompt to a directory and waits for a reply file: drive the generator from an agent session, or by hand. |
The visitor and the generator should be different models. A model
grading its own wording agrees with itself.
## The onboarding site
`portal-portal/` is the front end of this: a portal content repo whose
front door is the intake interview. A submission lands in
`onboardings` (`open → drafted ⇄ revising → live | declined`) and is
worked from an owners' desk. Today the desk step is a person running
`generate` and sending the link; the intake record already carries
every field the generator reads, so the step that remains is a
listener on `portal.answers.submitted` that runs it and pushes the
draft to a new repo. It passes its own `needs.yaml`.
## The whole card, not the heading
A visitor never sees a bare heading and description. Portal renders a
card: a button, an encouragement, one or more sections each with an
icon, a title and a line of guidance, and the fields with their
labels and placeholders. Since 2026-09-23 the builder fills all of it
and the scorers read all of it.
- **Slots.** Beside the heading and description, each form alternative
gets a button, an encouragement, an icon, a form title and guidance
line, and a placeholder per field. A door gets an icon and a "for
whom" line. Icons are picked from a fixed list of Iconify lucide
names; anything else is dropped.
- **Context from the plan.** Every form alternative also carries two
sections built from what the plan already knows, before the form:
what happens next (the stages, in order, as one sentence) and who
reads it (the handling group and the contact). Their fallbacks are
true before a model has written a word; the model rewords them.
- **Scoring.** `answer --repo <content dir>` shows the visitor each
option as its whole card (headings-only tasks stay headings only).
`rate --repo` does the same and asks a fourth question, "where": do
you know where you are, who this is for, and what the button does.
All four are controlled against the gutted twin.
## Scoring a built site
```
portal score --site out/nordvik --out out/scored \
--iris /var/local/cargo-target/release/iris \
--llm "MODEL=qwen3:8b engines/ollama.sh"
```
One command for the whole grading, which every run under `runs/`
before 2026-09-23 carried as its own `score.sh`: iris writes the
visitor tasks and the simulation model, the blind visitor answers
them with descriptions and with headings only, iris scores the
answers (the misroutes are printed), and `rate` reads the cards. The
two routing numbers land in `summary.json`, the rating in
`rated/rate.json`. The pieces still run alone: `answer --tasks` for
one visitor model against tasks iris wrote, and:
```
iris check --path out/nordvik/questions --needs-sim model.json
portal rate --model model.json --out out/rated --repo out/nordvik \
--llm "MODEL=qwen3:8b engines/ollama.sh"
```
Four yes-or-no questions per persona (which option is mine, where am
I and what does the button do, what will I be asked, what happens
after I send it), each also put to a gutted
twin of the page with headings, descriptions, the form and the
followup removed. A question the twin still gets a yes on is not
measuring the page, and `rate` says which. The followup page is read
from the repo (`--repo`); without it "what happens next" is reported
as not measured rather than as zero. LIX is over the prose only
(question, descriptions, followup): headings and labels are not
sentences and would pull every page down to "very easy". The
`rate.json` in `runs/contractor-build-2026-09-21/` predates this and
its `asked` and `next` columns are the old, pinned ones.
`rate --repo` also scores the **desks**, the inside of the business,
which reads button labels and nothing else: for every desk and bucket
with two or more buttons, a worker is told what has just happened to
a case (in the state's own words, never the label's) and picks the
button among the ones a row in that state shows, in a rotated order.
Scored per desk as right of total, with what each miss was mistaken
for. Measured on 2026-09-24 on the live vel: 24 of 26, the misses
one button each on the neighbour-help desk and the notice board. Two
limits: the outcome is put from the state id, and a Norwegian id with
its accents stripped (`ikke_lost`) reads as English even to a strong
model; and a first version that offered every button of the bucket,
not the row's, scored the same desks 17 of 26 and sent a rewrite
chasing an instrument fault. A desk that scores full marks has labels
a stranger to the business could press right.
`prefer` has a positive control on record
(`runs/tomtervel-opener-2026-09-23`, "The friendliness judge"): the
live vel front door against a bureaucratic twin, friendlier 9 of 10
to the live page and 0 to the twin. A pair that comes back "no
preference" throughout is alike in tone, not an instrument that
cannot see; and the rejected door rewrite that was judged clearer on
5 of 10 needs routed worse across three seeds, so clarity as judged
and routing as measured are different things, and routing decides.
## Measured so far
**Generated: a building contractor** (`runs/contractor-2026-09-21/`),
from `intakes/contractor.yaml`, a Sonnet-class model generating.
- Phase 1 derived six needs from the owner's prose, including one
nobody asked for: a project manager logging a phone enquiry, taken
from "enquiries arrive by phone, email and text and get lost".
- Phase 2 was green on the first round: 14 files, four state machines
(`enquiries: open → quoted → won | lost`, `subcontractors` with an
`awaiting_papers` loop, `applicants`, `site_issues`), two desks.
- Phase 3, its own personas: 10 of 10 in both modes. That number is
flattering, because in that run the pages were written with the
personas in view. Against ten **held-out** personas, with qwen3:8b
as the visitor:
| Visitor could read | Routed right |
|---|---|
| Headings and descriptions | 10 of 10 |
| Headings only | 8 of 10 |
A scaffolding firm asking "how do I get to do jobs for you?" chose
"Ask about working with us" (the hiring door). An attic conversion
chose "Get someone to look at a building". The generator has since
been changed to hide personas from the page-writing phase.
**Built: the same contractor from a plan** (`runs/contractor-build-2026-09-21/`),
nemotron writing words only, and re-scored one task per call the day
after (`rescore-2026-09-22/` in it): 8 of 8 with descriptions, 7 of 8
on headings alone, the miss being an apprentice bricklayer reading
"Join our carpentry team" as not for them. `rate` on the same site,
with the followup shown and every question controlled, passes its
controls and puts the pages at LIX 18 to 35.
**Built behind doors, 2026-09-22.** Five plans with a first question
before the forms, each built and then scored by qwen3:8b one task per
call, against held-out personas. Behind a door a persona is asked
twice, on the front door and on the door's page.
| Case | Words by | With descriptions | Headings only | Notes |
|---|---|---|---|---|
| Machine hire (`hire`) | nemotron | 10 of 10 | 10 of 10 | doors that ask a question the visitor already knows the answer to |
| Physio clinic (`physio`) | nemotron | 8 of 10 | 9 of 10 | "Book your first appointment" fails a daughter booking for her mother |
| Football club, Norwegian (`fotballklubb`) | nemotron | 14 of 16 | 11 of 16 | three parents walk past "Er det på tide å melde inn?" |
| Football club, Norwegian | Claude Sonnet | 13 of 16 | 12 of 16 | better sentences, same driver/sponsor blur |
| Whole contractor (`contractor-full`) | nemotron | 18 of 22 | 15 of 22 | "Apply to work with us" and "Find a job with us" cross over for all four |
| Tomter Vel, Norwegian (`tomtervel`), first plan | Claude Sonnet | 25 of 30 | 22 of 30 | "utstyr" for a swing, hearing input behind the wrong door |
| Tomter Vel, plan revised 2026-09-23, whole card | Claude Sonnet | 31 of 38 | 30 of 38 | committees as groups, public lists, hearing input on the front door |
Each run's README names the misroutes and what in the plan or the
heading caused them. A stronger word model did not route better on
the one plan built with both: routing is decided by which distinctions
sit on one page, and the visitor grades that the same whoever wrote
the words.
**Planned: the contractor intake read by a local model**
(`runs/contractor-plan-2026-09-22/`), nemotron, one round, 41 s. The
plan passed every check `build` makes and a reader finds three
structural misses in it the checks cannot: the crew's report made
public, the "papers missing" stage dropped, and no need for the
project manager logging a call. The person reading the plan is not
optional.
**Existing site: uhhm.no** (`runs/uhhm-2026-09-21/`), eight tasks.
| Visitor model | With descriptions | Headings only |
|---|---|---|
| small LLM, blind | 7 of 8 | 5 of 8 |
| qwen3:8b (Ollama) | 7 of 8 | 7 of 8 |
| granite4.1:8b (Ollama) | 7 of 8 | 6 of 8 |
All three send a paying client through **"Work happens through
dialogue, not tickets"**, the collaborator door. Different models,
same defect: that is what a wording problem looks like.
**The noise floor, 2026-09-23** (`runs/spread.sh`, `answer --seed`).
The same tasks answered by qwen3:8b with four seeds (0 is the run
every score above used):
| Run | With descriptions | Headings only |
|---|---|---|
| uhhm.no round 6, 15 tasks | 11, 10, 10, 10 | 12, 12, 7, 7 |
| Tomter Vel with the opener, 38 tasks | 30, 29, 29, 31 | 26, 24, 26, 23 |
With descriptions the visitor is steady: one task in fifteen, two in
thirty-eight. On headings alone it is not: five of fifteen on uhhm.no
between seeds, three of thirty-eight on the vel. A headings-only
difference smaller than that is the visitor's mood. Compare
descriptions-mode scores; read headings-only as a sanity check, and
run several seeds before believing it.
Small numbers, one run each. Leads, not measurements.
## What else is here
- `skill/SKILL.md` - the same procedure for an agent or a person
working by hand.
- `sectors/` - needs specs to start from. `construction/` doubles as
the held-out persona set above.
- `generator/reference.md` - everything the generating model is told
about portal content. When the lint rejects something the reference
did not warn about, the fix belongs there.
## Toward a trained model
Nothing here trains weights yet. What would make it possible is
accumulating: every run leaves the intake, the needs, the pages at
each revision and the scores. With a fast local visitor as the scorer,
generating many variants per need and keeping the winners is cheap,
and those winners are the fine-tuning set for a small generator. The
verifier has to be trusted before any of that: a generator trained
against a judge that is often wrong learns the judge.