# portal-trainer Onboarding a business onto [portal](https://project.uhhm.no/uhhm/portal) is mostly a wording problem: the engine is generic, and whether it works for a given business depends on whether its pages say, in the visitor's words, the thing the business needs them to answer. This repo is where that gets tested and, eventually, learned. ## The loop ``` interview ─► needs.yaml ─► pages + aggregates + desks │ │ │ question_lint (structure: pass/fail) │ │ └─► personas ─► tasks.jsonl ─► answering engine ─► answers.jsonl │ question_lint --needs-score ◄──┘ │ misroutes = wording defects ─► revise pages ``` - **`needs.yaml`** lives in the business's own content repo. It says who wants what, which bucket the answer lands in, which group carries it to a finished state, and what the visitor will tolerate in submissions and required fields. - **The verifier is portal's `question_lint`**, not code in this repo. It reads portal's own content types and ships in every portal release, so a content repo's CI enforces its needs with no new dependency. Structure is deterministic and gates CI. Wording is graded, and gates only if you pass `--min-accuracy`. - **This repo holds what sits around the verifier**: the trainer procedure (`skill/SKILL.md`), sector specs to start from (`sectors/`), and the runs (`runs/`). ``` question_lint --path /questions # structure question_lint --path /questions --needs-tasks tasks.jsonl # full wording question_lint --path /questions --needs-tasks skim.jsonl --skim # headings only question_lint --path /questions --needs-score answers.jsonl [--min-accuracy 0.8] ``` ## The engine contract A task is one page on one persona's path, as a typed `choice`: ```json {"id":"architecture-firm@/","persona":"architecture-firm","page":"/", "state":{"body":"We run a small architecture firm and ..."}, "questions":{"pick":{"type":"choice", "instructions":"A visitor reads the page question \"What are you here for?\". Which alternative do they choose?", "criteria":{"":"", "...":"..."}}}, "expect":[""]} ``` Strip `expect` (and `need`) before an engine sees it. An answer is one line: `{"id": "...", "choice": "", "confidence": 0.0-1.0}`. `state` and `questions` are exactly the arguments of [laya](https://github.com/NandhaKishorM/laya)'s `router.predict(state, questions)`, so a laya adapter is a loop over the file; an LLM adapter is a prompt around the same two fields. ## Engines, and what is known about them | Engine | Status | What it is good for | |---|---|---| | LLM, blind (no `expect`) | used for the first run | Reads like a visitor; slow and costs per task, so use it to find defects, not to gate CI. | | laya `english` (ModernBERT-large, 421M) | not run yet | 33 ms per question, so it can gate CI and score thousands of generated variants. Its authors measure 0.78 intent accuracy in English. | | laya `multilingual` | not run yet | Its authors' own sweep puts Norwegian Bokmål at **0.53 accuracy with 0.86 mean confidence**. It is confidently wrong too often to grade Norwegian wording without fine-tuning. Test a Norwegian site in English translation, or with an LLM. | An engine is a measuring instrument. Before trusting one on a sector, score it on tasks whose right answer nobody disputes; an engine that misses those is measuring itself, not the page. ## First run: uhhm.no, 2026-09-21 Nine personas against the live uhhm.no front door, answered by a small LLM that never saw `expect`. One run each; eight tasks; treat as a lead, not a measurement. | What the engine could read | Routed right | |---|---| | Headings and descriptions | 7 of 8 | | Headings only (`--skim`) | 5 of 8 | - A founder who wants a first version built picked **"Work happens through dialogue, not tickets"** - the collaborator door - in both modes. With headings only, so did a firm wanting to pay for storage and backups. The heading reads as how UHHM works with clients. - With headings only, a student seeking an internship picked **"Attention is earned"**. - Structure: four needs met. The site-improvement flow is advertised and `disabled`, so it is declared `planned`. ## Sectors A sector file is a needs spec with no pages behind it: a failing test suite for the trainer to make pass, carrying what the sector demands (why a lead is a pipeline, why a deviation report gets three fields). - `sectors/software-studio/needs.yaml` - UHHM's own, and green against `uhhm/questions`. - `sectors/construction/needs.yaml` - a contractor with project managers, a safety officer, crew and subcontractors: winning survey and design work, vetting subcontractors, requesting and answering bids, hiring, deviation reporting. No pages yet. ## Toward a trained model Nothing here trains weights yet, on purpose. The order that makes training possible: 1. The verifier has to be trusted first. It is the reward. A generator trained against a judge that is wrong half the time learns the judge. 2. Every onboarding leaves a run: needs, the pages at each revision, the scores. That corpus - what wording routed which persona right, per sector - is the training data, and it does not exist until the loop has been run by hand a few dozen times. 3. With a fast engine (laya-class) as the scorer, generating and scoring many page variants per need is cheap, which is what makes search, and later fine-tuning a small generator on the winners, practical.