uhhm/portal
1
1
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases 89 Wiki Activity

score: the whole grading of a built site as one command

Browse Source 
iris writes the tasks and the model, the blind visitor answers with
descriptions and with headings only, iris scores, rate reads the
cards; the routing numbers land in summary.json. Replaces the
score.sh every run carried. Proven on runs/uhhm-2026-09-23/site.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
This commit is contained in:
Bendik Aagaard Lynghaug
2026-09-23 20:25:55 +02:00
co-authored by Claude Fable 5.1
parent 9ff9924374
commit bf60879de9
3 changed files with 120 additions and 3 deletions
Download Patch File Download Diff File Expand all files Collapse all files
README.md
+17 -2
View File
@@ -269,10 +269,25 @@ and the scorers read all of it.
you know where you are, who this is for, and what the button does.
All four are controlled against the gutted twin.
## Rating wording
## Scoring a built site
```
question_lint --path out/nordvik/questions --needs-sim model.json
portal-trainer score --site out/nordvik --out out/scored \
--iris /var/local/cargo-target/release/iris \
--llm "MODEL=qwen3:8b engines/ollama.sh"
```
One command for the whole grading, which every run under `runs/`
before 2026-09-23 carried as its own `score.sh`: iris writes the
visitor tasks and the simulation model, the blind visitor answers
them with descriptions and with headings only, iris scores the
answers (the misroutes are printed), and `rate` reads the cards. The
two routing numbers land in `summary.json`, the rating in
`rated/rate.json`. The pieces still run alone: `answer --tasks` for
one visitor model against tasks iris wrote, and:
```
iris check --path out/nordvik/questions --needs-sim model.json
portal-trainer rate --model model.json --out out/rated --repo out/nordvik \
--llm "MODEL=qwen3:8b engines/ollama.sh"
```