rate: a desk row offers only the buttons that start from its state
The first desk score offered every button of the bucket, where portal shows a row only the moves from its own state; that scored the live vel 17 of 26 and sent a rewrite of the neighbour-help desk chasing an instrument fault. With the row's buttons only: 24 of 26, and the rewrite loses on the fair test too. Noted: an accent-stripped state id (ikke_lost) reads as English even to a strong picker. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
This commit is contained in:
co-authored by
Claude Fable 5.1
parent
9089503524
commit
8e6a93ebb7
@@ -328,13 +328,16 @@ its `asked` and `next` columns are the old, pinned ones.
|
|||||||
which reads button labels and nothing else: for every desk and bucket
|
which reads button labels and nothing else: for every desk and bucket
|
||||||
with two or more buttons, a worker is told what has just happened to
|
with two or more buttons, a worker is told what has just happened to
|
||||||
a case (in the state's own words, never the label's) and picks the
|
a case (in the state's own words, never the label's) and picks the
|
||||||
button among that desk's, in a rotated order. Scored per desk as
|
button among the ones a row in that state shows, in a rotated order.
|
||||||
right of total, with what each miss was mistaken for. Measured on
|
Scored per desk as right of total, with what each miss was mistaken
|
||||||
2026-09-24: the vel's neighbour-help desk 2 of 4 ("not solved" taken
|
for. Measured on 2026-09-24 on the live vel: 24 of 26, the misses
|
||||||
for "connect a neighbour"), the notice board 1 of 2, the planned
|
one button each on the neighbour-help desk and the notice board. Two
|
||||||
contractor's subcontractor desk 4 of 7 (checking, approved and
|
limits: the outcome is put from the state id, and a Norwegian id with
|
||||||
rejected all crossed). A desk that scores full marks has labels a
|
its accents stripped (`ikke_lost`) reads as English even to a strong
|
||||||
stranger to the business could press right.
|
model; and a first version that offered every button of the bucket,
|
||||||
|
not the row's, scored the same desks 17 of 26 and sent a rewrite
|
||||||
|
chasing an instrument fault. A desk that scores full marks has labels
|
||||||
|
a stranger to the business could press right.
|
||||||
## Measured so far
|
## Measured so far
|
||||||
|
|
||||||
**Generated: a building contractor** (`runs/contractor-2026-09-21/`),
|
**Generated: a building contractor** (`runs/contractor-2026-09-21/`),
|
||||||
|
|||||||
@@ -47,3 +47,14 @@ reads as volunteering. The live doors stay. What survives every round
|
|||||||
is the same four: hearing input read as a report, a committee read as
|
is the same four: hearing input read as a report, a committee read as
|
||||||
a hearing, and volunteering read as neighbour help - which says the
|
a hearing, and volunteering read as neighbour help - which says the
|
||||||
doors' seams are the plan's, not the words'.
|
doors' seams are the plan's, not the words'.
|
||||||
|
|
||||||
|
## Round 4: the neighbour-help desk reworded, and rejected
|
||||||
|
|
||||||
|
The first desk score said this desk got 2 of 4 buttons right. Sonnet
|
||||||
|
reworded its labels (`/tmp` only); the score got worse with both a
|
||||||
|
small and a strong picker, and the strong one showed why: told a case
|
||||||
|
"has just become ikke lost", it pressed "Meld ordnet" - it read the
|
||||||
|
accent-stripped state id as English. The scorer also offered every
|
||||||
|
button of the bucket where a row shows only its own state's. With the
|
||||||
|
row's buttons only, the live desk scores 3 of 4 and the vel 24 of 26
|
||||||
|
over all desks. The labels stay as they were.
|
||||||
|
|||||||
+15
-4
@@ -307,19 +307,30 @@ pub fn rate(argv: &[String]) -> anyhow::Result<()> {
|
|||||||
if !desks.is_empty() {
|
if !desks.is_empty() {
|
||||||
println!("{:<20} {:<24} {:>8} {}", "desk", "bucket", "buttons", "mistaken for");
|
println!("{:<20} {:<24} {:>8} {}", "desk", "bucket", "buttons", "mistaken for");
|
||||||
for (page, group, bucket, moves) in &desks {
|
for (page, group, bucket, moves) in &desks {
|
||||||
let labels: Vec<&str> = moves.iter().map(|(_, _, l)| l.as_str()).collect();
|
// A desk row shows only the buttons that start from the
|
||||||
|
// row's state, so those are the ones the worker chooses
|
||||||
|
// among; a state with one button has nothing to confuse.
|
||||||
let mut right = 0;
|
let mut right = 0;
|
||||||
|
let mut asked = 0;
|
||||||
let mut mistaken = Vec::new();
|
let mut mistaken = Vec::new();
|
||||||
for (i, (from, to, label)) in moves.iter().enumerate() {
|
for (i, (from, to, label)) in moves.iter().enumerate() {
|
||||||
|
let labels: Vec<&str> = moves.iter().filter(|(f, _, _)| f == from).map(|(_, _, l)| l.as_str()).collect();
|
||||||
|
if labels.len() < 2 {
|
||||||
|
continue;
|
||||||
|
}
|
||||||
|
asked += 1;
|
||||||
let picked = button_pick(&cmd, group, bucket, from, to, &labels, i)?;
|
let picked = button_pick(&cmd, group, bucket, from, to, &labels, i)?;
|
||||||
if picked == *label {
|
if picked == *label {
|
||||||
right += 1;
|
right += 1;
|
||||||
} else {
|
} else {
|
||||||
mistaken.push(format!("{to} as {picked:?}"));
|
mistaken.push(format!("{from}>{to} as {picked:?}"));
|
||||||
}
|
}
|
||||||
}
|
}
|
||||||
println!("{:<20} {:<24} {:>3} of {:<2} {}", page, bucket, right, moves.len(), mistaken.join("; "));
|
if asked == 0 {
|
||||||
report.insert(format!("desk {page} {bucket}"), json!({ "right": right, "of": moves.len(), "mistaken": mistaken }));
|
continue;
|
||||||
|
}
|
||||||
|
println!("{:<20} {:<24} {:>3} of {:<2} {}", page, bucket, right, asked, mistaken.join("; "));
|
||||||
|
report.insert(format!("desk {page} {bucket}"), json!({ "right": right, "of": asked, "mistaken": mistaken }));
|
||||||
}
|
}
|
||||||
println!();
|
println!();
|
||||||
}
|
}
|
||||||
|
|||||||
Reference in New Issue
Block a user