portal/README.md

# portal-trainer

A business describes itself in plain words: what it does, who comes to
it, what it keeps track of and through which stages, and who handles
what. `portal-trainer` turns that into a working
[portal](https://project.uhhm.no/uhhm/portal) content repo: a question
series for every role, the state machines behind them, and a desk per
handling role. No software is written for the business; the only thing
produced is content, and every line of it is held to a verifier.

## Build: structure from the plan, words from the model

```
portal-trainer build --plan plans/contractor.yaml --out out/nordvik \
    --lint question_lint --llm "MODEL=nemotron-3.5-lightning:30b-a3b engines/ollama.sh"
```

A `plan.yaml` is a needs spec that also says what each form collects.
Everything structural is derived from it with no model involved: the
state machines (read off `stages`: walk the line, be turned down at
any point, succeed only from the end, pass over `skippable` stages),
the front door, a followup page per need, a gated directory per group
with its desk and the pages only that group may open, and every button
the machine allows. The verifier accepts the result by construction.

The model writes words only, one page at a time, into a JSON object
whose shape is fixed. Its strings are merged over built-in fallbacks
key by key, so a bad reply costs wording, never a working site.

| Same contractor, same local model (nemotron 30B-A3B on ergo) | Free-form `generate` | `build` |
|---|---|---|
| A site the verifier accepts | never, in 6 rounds and 16 min | first try, 130 s |
| Held-out personas, with descriptions | - | 8 of 8 |
| Held-out personas, headings only | - | 8 of 8 |

For comparison, a Sonnet-class model writing everything free-form
scored 8 of 10 on headings only and silently dropped a stage the owner
had described. `build` cannot drop a stage: it never decides one.

`generate` (below) remains for turning a plain-words intake into a
first plan; what it should hand over is the plan, not the pages.

## How it works

```
 intake (plain words)
    │
    ▼  phase 1                       frozen after this phase: the generator
 needs.yaml ───────────────────────  may not weaken the spec to pass
    │
    ▼  phase 2   ◄── question_lint findings, until it exits 0
 aggregates.yaml + questions/** + desks
    │
    ▼  phase 3   ◄── misroutes from a blind visitor model
 wording
```

1. **Needs.** The intake becomes `needs.yaml`: one need per kind of
   person and thing they want, including the roles inside the
   business. Each names the bucket the answer lands in, the group that
   handles it, the states that count as finished, and the effort the
   visitor will tolerate.
2. **Structure.** Pages, state machines and desks are generated, then
   revised against portal's `question_lint` until every need is met:
   a path within budget, a desk held by the right group, every
   finished state reachable, no record stranding. The page-writing
   model sees the needs and never the personas.
3. **Wording.** A separate model plays each persona, blind, first with
   headings only and then with descriptions. A misroute is a wording
   defect and goes back to the generator. A rewrite that breaks
   structure is reverted.

Model output is only ever written to `aggregates.yaml`, `site.yaml`
and `questions/**.yaml` inside the output directory.

```
portal-trainer generate --intake intakes/contractor.yaml --out out/nordvik \
    --lint /srv/app/<inst>/current/question_lint --llm engines/ollama.sh
portal-trainer answer --tasks tasks.jsonl --llm engines/ollama.sh > answers.jsonl
```

The verifier is portal's `question_lint`, not code in this repo: it
reads portal's own content types and ships in every portal release, so
a generated repo's CI keeps enforcing its needs after handover.

## LLM backends

A backend is any command that takes a prompt on stdin and prints text.

| Backend | Use |
|---|---|
| `engines/ollama.sh` | Local Ollama. `MODEL=qwen3:8b` answers a task in about two seconds on ergo's GPU and is the default visitor. Generation wants a larger model (`MODEL=qwen3.6:27b`), which runs mostly on CPU there. |
| `engines/anthropic.sh` | The Anthropic API, given a working `ANTHROPIC_API_KEY`. |
| `engines/mailbox.sh` | Writes each prompt to a directory and waits for a reply file: drive the generator from an agent session, or by hand. |

The visitor and the generator should be different models. A model
grading its own wording agrees with itself.

## The onboarding site

`portal-portal/` is the front end of this: a portal content repo whose
front door is the intake interview. A submission lands in
`onboardings` (`open → drafted ⇄ revising → live | declined`) and is
worked from an owners' desk. Today the desk step is a person running
`generate` and sending the link; the intake record already carries
every field the generator reads, so the step that remains is a
listener on `portal.answers.submitted` that runs it and pushes the
draft to a new repo. It passes its own `needs.yaml`.

## Measured so far

**Generated: a building contractor** (`runs/contractor-2026-09-21/`),
from `intakes/contractor.yaml`, a Sonnet-class model generating.

- Phase 1 derived six needs from the owner's prose, including one
  nobody asked for: a project manager logging a phone enquiry, taken
  from "enquiries arrive by phone, email and text and get lost".
- Phase 2 was green on the first round: 14 files, four state machines
  (`enquiries: open → quoted → won | lost`, `subcontractors` with an
  `awaiting_papers` loop, `applicants`, `site_issues`), two desks.
- Phase 3, its own personas: 10 of 10 in both modes. That number is
  flattering, because in that run the pages were written with the
  personas in view. Against ten **held-out** personas, with qwen3:8b
  as the visitor:

| Visitor could read | Routed right |
|---|---|
| Headings and descriptions | 10 of 10 |
| Headings only | 8 of 10 |

  A scaffolding firm asking "how do I get to do jobs for you?" chose
  "Ask about working with us" (the hiring door). An attic conversion
  chose "Get someone to look at a building". The generator has since
  been changed to hide personas from the page-writing phase.

**Existing site: uhhm.no** (`runs/uhhm-2026-09-21/`), eight tasks.

| Visitor model | With descriptions | Headings only |
|---|---|---|
| small LLM, blind | 7 of 8 | 5 of 8 |
| qwen3:8b (Ollama) | 7 of 8 | 7 of 8 |
| granite4.1:8b (Ollama) | 7 of 8 | 6 of 8 |

All three send a paying client through **"Work happens through
dialogue, not tickets"**, the collaborator door. Different models,
same defect: that is what a wording problem looks like.

Small numbers, one run each. Leads, not measurements.

## What else is here

- `skill/SKILL.md` - the same procedure for an agent or a person
  working by hand.
- `sectors/` - needs specs to start from. `construction/` doubles as
  the held-out persona set above.
- `generator/reference.md` - everything the generating model is told
  about portal content. When the lint rejects something the reference
  did not warn about, the fix belongs there.

## Toward a trained model

Nothing here trains weights yet. What would make it possible is
accumulating: every run leaves the intake, the needs, the pages at
each revision and the scores. With a fast local visitor as the scorer,
generating many variants per need and keeping the winners is cheap,
and those winners are the fine-tuning set for a small generator. The
verifier has to be trusted before any of that: a generator trained
against a judge that is often wrong learns the judge.