rate and prefer: understanding, readability, friendliness, page load
Readability is counted (LIX), not judged. Understanding is three yes-or-no questions per persona, with a gutted twin of every page as the control: an instrument that still says yes to "Option 1, Option 2" is not measuring the page. Friendliness is a preference between two versions shown in both orders; a pick that flips with the order counts as no preference. Page load (options offered, required fields in the step) flags a page that should be split into a further question or have fields moved to a later stage. First run on the generated contractor site, qwen3:8b as the visitor: control passed on every need; "what happens next" was 0/2 everywhere and 2/2 everywhere once the page said who reads it and when (LIX 28 to 30); an official-voice control reads LIX 63 against 27. Friendliness mostly came back "no preference" at two personas per need: directionally right (never preferred the cold version), too thin to rely on. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01L4jrCgLiKKHAEFuZUJjckH
This commit is contained in:
co-authored by
Claude Fable 5.1
parent
612745bc66
commit
b9f2acb7b8
@@ -22,6 +22,7 @@ use std::process::{Command, Stdio};
|
||||
|
||||
use anyhow::{anyhow, bail, Context};
|
||||
|
||||
mod rate;
|
||||
mod simulate;
|
||||
|
||||
const REFERENCE: &str = include_str!("../generator/reference.md");
|
||||
@@ -43,6 +44,8 @@ fn main() -> anyhow::Result<()> {
|
||||
match argv.first().map(String::as_str) {
|
||||
Some("simulate") => return simulate::run(&argv),
|
||||
Some("compare") => return simulate::compare(&argv),
|
||||
Some("rate") => return rate::rate(&argv),
|
||||
Some("prefer") => return rate::prefer(&argv),
|
||||
_ => {}
|
||||
}
|
||||
if argv.first().map(String::as_str) == Some("answer") {
|
||||
|
||||
Reference in New Issue
Block a user