portal/README.md

portal-trainer

Onboarding a business onto portal is mostly a wording problem: the engine is generic, and whether it works for a given business depends on whether its pages say, in the visitor's words, the thing the business needs them to answer. This repo is where that gets tested and, eventually, learned.

The loop

interview ─► needs.yaml ─► pages + aggregates + desks
                 │                    │
                 │          question_lint (structure: pass/fail)
                 │                    │
                 └─► personas ─► tasks.jsonl ─► answering engine ─► answers.jsonl
                                                                       │
                                        question_lint --needs-score ◄──┘
                                                  │
                                    misroutes = wording defects ─► revise pages

• needs.yaml lives in the business's own content repo. It says who wants what, which bucket the answer lands in, which group carries it to a finished state, and what the visitor will tolerate in submissions and required fields.
• The verifier is portal's question_lint, not code in this repo. It reads portal's own content types and ships in every portal release, so a content repo's CI enforces its needs with no new dependency. Structure is deterministic and gates CI. Wording is graded, and gates only if you pass --min-accuracy.
• This repo holds what sits around the verifier: the trainer procedure (skill/SKILL.md), sector specs to start from (sectors/), and the runs (runs/).

question_lint --path <repo>/questions                                  # structure
question_lint --path <repo>/questions --needs-tasks tasks.jsonl        # full wording
question_lint --path <repo>/questions --needs-tasks skim.jsonl --skim  # headings only
question_lint --path <repo>/questions --needs-score answers.jsonl [--min-accuracy 0.8]

The engine contract

A task is one page on one persona's path, as a typed choice:

{"id":"architecture-firm@/","persona":"architecture-firm","page":"/",
 "state":{"body":"We run a small architecture firm and ..."},
 "questions":{"pick":{"type":"choice",
   "instructions":"A visitor reads the page question \"What are you here for?\". Which alternative do they choose?",
   "criteria":{"<alternative heading>":"<its description>", "...":"..."}}},
 "expect":["<every alternative that still meets the need>"]}

Strip expect (and need) before an engine sees it. An answer is one line: {"id": "...", "choice": "<heading, verbatim>", "confidence": 0.0-1.0}. state and questions are exactly the arguments of laya's router.predict(state, questions), so a laya adapter is a loop over the file; an LLM adapter is a prompt around the same two fields.

Engines, and what is known about them

Engine	Status	What it is good for
LLM, blind (no expect)	used for the first run	Reads like a visitor; slow and costs per task, so use it to find defects, not to gate CI.
laya english (ModernBERT-large, 421M)	not run yet	33 ms per question, so it can gate CI and score thousands of generated variants. Its authors measure 0.78 intent accuracy in English.
laya multilingual	not run yet	Its authors' own sweep puts Norwegian Bokmål at 0.53 accuracy with 0.86 mean confidence. It is confidently wrong too often to grade Norwegian wording without fine-tuning. Test a Norwegian site in English translation, or with an LLM.

An engine is a measuring instrument. Before trusting one on a sector, score it on tasks whose right answer nobody disputes; an engine that misses those is measuring itself, not the page.

First run: uhhm.no, 2026-09-21

Nine personas against the live uhhm.no front door, answered by a small LLM that never saw expect. One run each; eight tasks; treat as a lead, not a measurement.

What the engine could read	Routed right
Headings and descriptions	7 of 8
Headings only (--skim)	5 of 8

• A founder who wants a first version built picked "Work happens through dialogue, not tickets" - the collaborator door - in both modes. With headings only, so did a firm wanting to pay for storage and backups. The heading reads as how UHHM works with clients.
• With headings only, a student seeking an internship picked "Attention is earned".
• Structure: four needs met. The site-improvement flow is advertised and disabled, so it is declared planned.

Sectors

A sector file is a needs spec with no pages behind it: a failing test suite for the trainer to make pass, carrying what the sector demands (why a lead is a pipeline, why a deviation report gets three fields).

• sectors/software-studio/needs.yaml - UHHM's own, and green against uhhm/questions.
• sectors/construction/needs.yaml - a contractor with project managers, a safety officer, crew and subcontractors: winning survey and design work, vetting subcontractors, requesting and answering bids, hiring, deviation reporting. No pages yet.

Toward a trained model

Nothing here trains weights yet, on purpose. The order that makes training possible:

1. The verifier has to be trusted first. It is the reward. A generator trained against a judge that is wrong half the time learns the judge.
2. Every onboarding leaves a run: needs, the pages at each revision, the scores. That corpus - what wording routed which persona right, per sector - is the training data, and it does not exist until the loop has been run by hand a few dozen times.
3. With a fast engine (laya-class) as the scorer, generating and scoring many page variants per need is cheap, which is what makes search, and later fine-tuning a small generator on the winners, practical.