portal/vendor/trainer/runs/permits-2026-09-25

Vestmark kommune - Building and Planning, 2026-09-25

The hardest sector the trainer has been given, chosen to find where portal stops rather than to flatter it: a municipal building authority. Applications that pause and resume, a decision no case officer may sign, a group outside the building that moves records and nothing else, objections that belong to other cases, a six-week deadline nobody presses a button for, and an appeal that reopens something already finished.

Built with no model in the loop: the words are the builder's own fallbacks. What is measured here is the machinery, not the prose.

• plan: plans/building-permits.yaml (14 needs, 7 groups, 12 buckets)
• intake: intakes/building-permits.yaml
• 41 questions, 14 aggregates, 198 access rules, 14 needs structurally met

What was proved

iris replay runs cases through portal's own answers::store_answer and aggregates::transition, against a throwaway JetStream in docker - the same code the running site uses, with its event log and its refusal of undeclared moves. Fifteen cases, each one a path a real building case walks (cases.jsonl).

	transitions accepted	refused by the engine
as first built	36	3, all of them legal paths
hand-corrected aggregates.yaml	45	2, both of them illegal paths
rebuilt from the plan, after the fix	45	2, both of them illegal paths

The three refusals in the first row were the sector, not the site:

REFUSED: asked-twice -> incomplete            (complete -> incomplete)
REFUSED: hearing-sent-it-back -> incomplete   (on_hearing -> incomplete)
REFUSED: we-changed-our-mind -> changed_by_us (reconsidered -> changed_by_us)

and meanwhile the engine happily accepted open -> refused (refusing an application on the day it arrives, before anyone has read it) and open -> overturned (the appeals board overturning a case that never reached them).

Portal's aggregate engine needed no change at all. Hand-writing the correct graph into aggregates.yaml - a cycle back to incomplete, twice, and a success state reachable from the middle of the line - was accepted as-is and enforced exactly. The whole gap was in the trainer's plan language, which could only describe a line that goes forward.

What was fixed in the trainer

reached_from (src/build.rs). Where a stage is entered from, when the line does not say it:

reached_from:
  incomplete: [open, complete, on_hearing]        # sent back for papers
  refused: [complete, on_hearing, recommended]    # never on arrival

A stage named there is entered from exactly those stages and nowhere else, so saying where a refusal may come from also says where it may not. It covers both of the sector's shapes - a case sent back to a stage it was already past, and a good ending partway along - and it subsumes turned_down_from, which was one list for every bad end at once and so could not say that an applicant may withdraw from anywhere while the authority may refuse only once the papers are in. The rebuilt site now produces exactly the hand-corrected graph, and the access policy follows it: section_head gets complete -> refused and no longer open -> refused; appeals_board gets only before_the_board -> upheld and -> overturned.

The public-view rule is now per bucket, not per need. A permit register holds the owner's small application and the approved firm's large one, and is one page either way; the old rule refused to publish a bucket the moment any role filed into it, which made "nobody outside this building can see a case" - the intake's own complaint - unfixable. A bucket only a role ever writes to is still refused, and that is why the hearing register here is internal.

What portal cannot do yet

Each of these is a change proposed, not made. They are ordered by how much this sector needed them.

1. A submitter may only act while the record is untouched

src/access.rs:223 pins a self_transition's from-state to open:

if let Some(st) = &alt.self_transition {
    let from = crate::answers::OPEN_STATE;

The runtime is already general - self_transition_answer passes the record's actual state as context.from and asks the policy - so this is the compiler alone. The effect is that the applicant who is written to for a missing drawing cannot send it: incomplete -> complete is in the machine and in the plan, and the policy never emits it. Everything about staying in dialogue with a person across a case dies here.

Proposed: self_transition takes an optional from: [state, ...], defaulting to [open]; access.rs emits one rule per from-state, and the lint checks each named state exists and that from -> to is declared in the bucket's schema. The trainer already computes the set (moves_of(need, SUBMITTER)) and would just write it down.

2. A record cannot belong to another record

An objection belongs to an application. An exemption rides on one. An appeal is against a decision; an inspection is of a case; a hearing answer answers a hearing request. Six of this site's twelve buckets hold records that are about a record in another bucket, and the only thing joining them is a case_number the person types in from a letter, which nothing validates, nothing resolves, and nothing can list.

This is the largest structural gap, and it is why the case officer here still cannot be shown "this application and its four objections" on one page - the thing the intake asks for twice.

Proposed: a field type: record with of: <bucket>, storing a record id, validated by the lint against the bucket and at submission against the store; the parent's desk row then offers its children, and a public case page can be assembled from the parent plus everything pointing at it. This is a real piece of work and should be its own release, not a rider.

3. Nothing moves a stage except a person

attended_by lets a plan document a stage no desk button moves - here, the hearing's six weeks - but nothing actually moves it. The record sits in open until someone notices. The intake names this as its number one problem: "the twelve-week clock, and the fact that stopping it and starting it again is done by hand in a spreadsheet."

Proposed: a state in aggregates.yaml may carry after: { days: 42, to: no_answer } beside its event: and mail:, and something with a timer fires it. gdo already holds a durable consumer, already reads content, and already sends what a state tells it to - this is the same shape as mail:, one field along.

4. A public view publishes the whole bucket

watched_by: [public] opens every record in the bucket, including the ones that were refused and withdrawn. For a permit register that is roughly right and for a register of firms it is not, which is why firm-approval here is not public though the list of approved firms genuinely is.

Proposed: public: { states: [...] } on a bucket resource.

5. A need whose success is a membership cannot say so

firm-approval ending in approved is exactly what makes a firm a member of approved_firms, and therefore what lets it file a permit-application at all. Today somebody invites by hand afterwards.

Proposed: grants: <group> on a state, next to mail:, using the invite machinery that already exists.

Also found

• The planner (portal-trainer plan) fails with exited with exit status: 1 whenever ANTHROPIC_API_KEY is set to anything the nested CLI rejects. --llm 'env -u ANTHROPIC_API_KEY claude -p' is the fix. This plan was written by hand, so the planner has not yet been asked to produce anything this size.
• Two needs sharing a bucket emit the submitter's rule twice; harmless, but the access listing has duplicate rows.

Reproducing

docker run -d --rm --name iris-replay -p 127.0.0.1:14222:4222 nats:2.14.6-alpine -js
portal-trainer build --plan plans/building-permits.yaml --out runs/permits-2026-09-25 --lint iris
cd runs/permits-2026-09-25
iris check --path questions --access
iris replay --path questions --nats nats://127.0.0.1:14222 --cases cases.jsonl

fixed/ holds the hand-corrected aggregates.yaml from before the trainer could produce it, kept as the proof that portal's engine was never the thing in the way. Point replay at a throwaway NATS, never a site's own: it writes.

Routing: one stable failure, and a leak that turned out not to cost anything

Twenty-five routing tasks, three seeds, qwen3:8b as the visitor, with the builder's own fallback wording and no model having touched the prose:

seed 1: 18/25 (72%)    seed 2: 20/25 (80%)    seed 3: 22/25 (88%)

A sixteen-point spread is wider than this harness's usual floor, so the misroutes were counted per task rather than per run. One fails in every seed:

• shop-to-flats@/want-to-build (3 of 3), and garage-owner the same way (2 of 3): someone who wants to sit down with us before paying an architect picks "permission to build it" instead. The pre-application meeting does not read as distinguishable from applying. A wording fault on the door page, and the kind a wording round exists to fix.

Everything else failed once - but the shape was not random. Of the eight front-door misroutes across all three seeds, six went to one card: a neighbour who got a notice, an owner asking for a completion certificate and someone reporting a building behind a fence all picked "the decision looked at again, by us and then by the board".

The front door held two doors and two bare needs - "Want to build", "Building near me", the appeal, and firm approval. A door is an abstract category; a bare need is a concrete sentence, and the concrete sentence was out-pulling the categories, collecting everyone with any sort of grievance whatever it was about.

So the obvious fix was measured: four doors, nothing bare, everything on the front door at the same level (runs/permits-doors-2026-09-25).

	seed 1	seed 2	seed 3	mean
two doors, two bare needs	72%	80%	88%	80.0%
four doors	76%	72%	84%	77.3%

The rewrite does not beat the spread, and is rejected. The diagnosis was right and the fix was still not worth making: with four doors the appeal card stopped collecting misroutes completely - 0 of 10, down from 6 of 8 - and the site got no better, because two offers went from findable to hidden. firm-insurance-lapsed ("our certificate has expired, we have renewed it, here it is") found "to be on the list, and then to file without asking again" when it was a sentence on the front door, and fails 3 of 3 once it is behind a door called "Work as a firm". owner-explains got worse the same way.

What that says, as a measurement rather than a taste: a concrete offer standing among categories does leak, and hiding it costs more than the leak. The earlier finding - four doors beat ten offers, six offers beat doors - was about how many things a page holds. This one is about what a door costs the person who was going to pick correctly: one more page between them and the only thing they came for. The mixed front door stays.