Commit Graph
3 Commits
Author SHA1 Message Date
Bendik Aagaard LynghaugandClaude Fable 5.1 baf36319b7 Overlapping roles on one aggregate, relay routes, and a dry run
plan.yaml: `moves_by` says who presses the button into a stage when it
is not `handled_by` - other groups, several groups, or `submitter`,
the person who sent the form, through their own link (the form must
then collect a required email). `watched_by` puts a bucket on a
group's desk with no buttons. Each group's desk carries its own buttons
only; a submitter's moves become a followup page of self_transitions;
needs.yaml gains `also_moved_by` for portal's verifier.

simulate: a route is now a relay across every role that holds a legal
button, the submitter included, and each step is judged as the role
that presses it. `--dry` asks no model anything and walks every simple
route of every aggregate, for needs_replay to put through the real
engine.

plans/contractor-full.yaml: 13 needs, 6 roles, 10 aggregates. Verifier
green by construction; 40 routes, 78 transitions accepted by the real
engine, none refused. What it exposed is in its run README.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01L4jrCgLiKKHAEFuZUJjckH
2026-09-21 21:46:12 +02:00
Bendik Aagaard LynghaugandClaude Fable 5.1 b9f2acb7b8 rate and prefer: understanding, readability, friendliness, page load
Readability is counted (LIX), not judged. Understanding is three
yes-or-no questions per persona, with a gutted twin of every page as
the control: an instrument that still says yes to "Option 1, Option 2"
is not measuring the page. Friendliness is a preference between two
versions shown in both orders; a pick that flips with the order counts
as no preference. Page load (options offered, required fields in the
step) flags a page that should be split into a further question or
have fields moved to a later stage.

First run on the generated contractor site, qwen3:8b as the visitor:
control passed on every need; "what happens next" was 0/2 everywhere
and 2/2 everywhere once the page said who reads it and when (LIX 28 to
30); an official-voice control reads LIX 63 against 27. Friendliness
mostly came back "no preference" at two personas per need: directionally
right (never preferred the cold version), too thin to rely on.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01L4jrCgLiKKHAEFuZUJjckH
2026-09-21 19:48:25 +02:00
Bendik Aagaard LynghaugandClaude Fable 5.1 02672547ee simulate: play every case through every state, judge only what a model can
`portal-trainer simulate` reads the model `question_lint --needs-sim`
exports. A visitor model picks an alternative and fills the form in a
manner that varies by run; the case is then walked along the handling
group's own desk buttons to a finished state, a different one each
run, so every state and button is exercised. At each button the
handler model is asked one thing: with only what the form collected,
could this role really do what the button claims? Its question goes
back to the visitor, who may not know.

Who wins is deliberately not simulated. The first version let the
handler decide: qwen3:8b sent every case to "won", and after one
sentence of prompt change, every case to "lost". Outcomes belong to
the live bucket; `success` in needs.yaml is what a report over real
records counts.

`compare` sets two score files side by side. engines/ollama.sh honours
SEED, FORMAT=json and TEMPERATURE so the same case replays the same
way against two versions of a site. generator/reference.md documents
`success` and `stages`.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01L4jrCgLiKKHAEFuZUJjckH
2026-09-21 19:16:28 +02:00