portal/README.md

# portal-trainer

Onboarding a business onto [portal](https://project.uhhm.no/uhhm/portal)
is mostly a wording problem: the engine is generic, and whether it
works for a given business depends on whether its pages say, in the
visitor's words, the thing the business needs them to answer. This
repo is where that gets tested and, eventually, learned.

## The loop

```
interview ─► needs.yaml ─► pages + aggregates + desks
                 │                    │
                 │          question_lint (structure: pass/fail)
                 │                    │
                 └─► personas ─► tasks.jsonl ─► answering engine ─► answers.jsonl
                                                                       │
                                        question_lint --needs-score ◄──┘
                                                  │
                                    misroutes = wording defects ─► revise pages
```

- **`needs.yaml`** lives in the business's own content repo. It says
  who wants what, which bucket the answer lands in, which group
  carries it to a finished state, and what the visitor will tolerate
  in submissions and required fields.
- **The verifier is portal's `question_lint`**, not code in this repo.
  It reads portal's own content types and ships in every portal
  release, so a content repo's CI enforces its needs with no new
  dependency. Structure is deterministic and gates CI. Wording is
  graded, and gates only if you pass `--min-accuracy`.
- **This repo holds what sits around the verifier**: the trainer
  procedure (`skill/SKILL.md`), sector specs to start from
  (`sectors/`), and the runs (`runs/`).

```
question_lint --path <repo>/questions                                  # structure
question_lint --path <repo>/questions --needs-tasks tasks.jsonl        # full wording
question_lint --path <repo>/questions --needs-tasks skim.jsonl --skim  # headings only
question_lint --path <repo>/questions --needs-score answers.jsonl [--min-accuracy 0.8]
```

## The engine contract

A task is one page on one persona's path, as a typed `choice`:

```json
{"id":"architecture-firm@/","persona":"architecture-firm","page":"/",
 "state":{"body":"We run a small architecture firm and ..."},
 "questions":{"pick":{"type":"choice",
   "instructions":"A visitor reads the page question \"What are you here for?\". Which alternative do they choose?",
   "criteria":{"<alternative heading>":"<its description>", "...":"..."}}},
 "expect":["<every alternative that still meets the need>"]}
```

Strip `expect` (and `need`) before an engine sees it. An answer is one
line: `{"id": "...", "choice": "<heading, verbatim>", "confidence": 0.0-1.0}`.
`state` and `questions` are exactly the arguments of
[laya](https://github.com/NandhaKishorM/laya)'s
`router.predict(state, questions)`, so a laya adapter is a loop over
the file; an LLM adapter is a prompt around the same two fields.

## Engines, and what is known about them

| Engine | Status | What it is good for |
|---|---|---|
| LLM, blind (no `expect`) | used for the first run | Reads like a visitor; slow and costs per task, so use it to find defects, not to gate CI. |
| laya `english` (ModernBERT-large, 421M) | not run yet | 33 ms per question, so it can gate CI and score thousands of generated variants. Its authors measure 0.78 intent accuracy in English. |
| laya `multilingual` | not run yet | Its authors' own sweep puts Norwegian Bokmål at **0.53 accuracy with 0.86 mean confidence**. It is confidently wrong too often to grade Norwegian wording without fine-tuning. Test a Norwegian site in English translation, or with an LLM. |

An engine is a measuring instrument. Before trusting one on a sector,
score it on tasks whose right answer nobody disputes; an engine that
misses those is measuring itself, not the page.

## First run: uhhm.no, 2026-09-21

Nine personas against the live uhhm.no front door, answered by a small
LLM that never saw `expect`. One run each; eight tasks; treat as a
lead, not a measurement.

| What the engine could read | Routed right |
|---|---|
| Headings and descriptions | 7 of 8 |
| Headings only (`--skim`) | 5 of 8 |

- A founder who wants a first version built picked **"Work happens
  through dialogue, not tickets"** - the collaborator door - in both
  modes. With headings only, so did a firm wanting to pay for storage
  and backups. The heading reads as how UHHM works with clients.
- With headings only, a student seeking an internship picked
  **"Attention is earned"**.
- Structure: four needs met. The site-improvement flow is advertised
  and `disabled`, so it is declared `planned`.

## Sectors

A sector file is a needs spec with no pages behind it: a failing test
suite for the trainer to make pass, carrying what the sector demands
(why a lead is a pipeline, why a deviation report gets three fields).

- `sectors/software-studio/needs.yaml` - UHHM's own, and green against
  `uhhm/questions`.
- `sectors/construction/needs.yaml` - a contractor with project
  managers, a safety officer, crew and subcontractors: winning survey
  and design work, vetting subcontractors, requesting and answering
  bids, hiring, deviation reporting. No pages yet.

## Toward a trained model

Nothing here trains weights yet, on purpose. The order that makes
training possible:

1. The verifier has to be trusted first. It is the reward. A generator
   trained against a judge that is wrong half the time learns the
   judge.
2. Every onboarding leaves a run: needs, the pages at each revision,
   the scores. That corpus - what wording routed which persona right,
   per sector - is the training data, and it does not exist until the
   loop has been run by hand a few dozen times.
3. With a fast engine (laya-class) as the scorer, generating and
   scoring many page variants per need is cheap, which is what makes
   search, and later fine-tuning a small generator on the winners,
   practical.