plan: intake to plan by a model; doors, labelled groups, the whole card
Work from another session, committed as found (tests green): - `plan` reads an intake into a plan.yaml (a model decides needs, stages, forms and budgets; a person reads the result before build). - Plans gain `doors` (a first question before the forms, when a front door would carry more than five needs), `groups` with labels that a select may draw its options from (`options_from: groups`), `private` fields kept off public pages, `multiple` selects, and `max_fields` as what the intake said the person will tolerate. - `card.rs`: an alternative as the visitor meets it, in text, so the answering visitor and `rate` read the served card, not the heading. - `rate` reads followup pages from the repo and reports "what happens next" as not measured rather than zero when it cannot. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01L4jrCgLiKKHAEFuZUJjckH
This commit is contained in:
Bendik Aagaard Lynghaug
co-authored by
Claude Fable 5.1
Claude Fable 5.1
parent
baf36319b7
commit
255aeb4f4b
+301
@@ -0,0 +1,301 @@
|
||||
//! From an intake, in plain words, to a `plan.yaml` a person can check.
|
||||
//!
|
||||
//! This is the one step where a model decides structure: which needs
|
||||
//! the intake implies, the stages each goes through, who handles it,
|
||||
//! what the form collects and how much that person will put up with.
|
||||
//! Everything after it is deterministic (`build`), so this is where a
|
||||
//! misreading of the business has to be caught, and it is caught by a
|
||||
//! person reading the plan, not by a verifier. The model's YAML is
|
||||
//! parsed and validated exactly as `build` would, and the errors go
|
||||
//! back to it for another round; what passes is written verbatim, so
|
||||
//! what the person reads is what the model wrote.
|
||||
//!
|
||||
//! Personas are never asked of this model: a persona written by the
|
||||
//! same model that phrased the needs tests nothing. They come from a
|
||||
//! person or a sector file, attached with `--personas`.
|
||||
|
||||
use anyhow::{anyhow, bail, Context};
|
||||
|
||||
use crate::build::Plan;
|
||||
use crate::simulate::first_object;
|
||||
|
||||
const REFERENCE: &str = r#"# plan.yaml
|
||||
|
||||
A plan is a needs spec that also says what each form collects. A
|
||||
builder turns it into a working site with no further decisions, so
|
||||
every structural fact about the business has to be in it.
|
||||
|
||||
```yaml
|
||||
business: <name>
|
||||
language: en # the intake's language code
|
||||
contact: { name: <who answers for the site>, email: <their email> }
|
||||
about: >
|
||||
What the business does, in its own words.
|
||||
|
||||
doors: # a first question before the forms, when more than four or
|
||||
- id: kebab-case # five needs would otherwise sit on the front door: one door
|
||||
who: who comes through it # per kind of visitor, two or three needs behind each
|
||||
|
||||
groups: # groups with a label the public may see: the committees
|
||||
- { id: trafikk, label: Trafikkforhold } # a person can ask to join (optional)
|
||||
|
||||
needs:
|
||||
- id: kebab-case # ASCII only
|
||||
who: the kind of person, in plain words, in the intake's language
|
||||
wants: what they want from the business
|
||||
door: <door id> # the door this public need sits behind; omit to put it on the front door
|
||||
enters_as: <group> # ONLY for a role inside the business; omit for the public
|
||||
lands_in: <bucket> # snake_case, ASCII
|
||||
handled_by: <group> # the role that must see and move the record
|
||||
stages: [open, <stage>, ..., <end>, <end>]
|
||||
# EVERY stage the business said a case goes through, first to
|
||||
# last, read off their own sentences one verb at a time: "they
|
||||
# come in, we go and look, we send a quote, we win or lose it"
|
||||
# is [open, surveyed, quoted, won, lost]. A pipeline is not an
|
||||
# inbox: "answered" is rarely the real outcome.
|
||||
done_when: [<end>, <end>] # every stage that counts as finished, good or bad
|
||||
success: [<end>] # the finished stages that are a good outcome for the business
|
||||
skippable: [<stage>] # stages a case may pass over (papers already complete)
|
||||
moves_by: { <stage>: [<group>, ...] }
|
||||
# only where someone other than handled_by moves a case INTO a
|
||||
# stage: another group, or `submitter` (the person who sent the
|
||||
# form; the form must then collect a required `email`)
|
||||
watched_by: [<group>] # groups that see these cases and move nothing; `public` opens a
|
||||
# read-only page of the bucket to everyone (contact details hidden)
|
||||
max_fields: 4 # how many required fields THIS person will tolerate, from what
|
||||
# the intake says about them: someone on a phone with gloves
|
||||
# on gets 3, a first enquiry 4 to 6, a firm's paperwork 5 to 6
|
||||
collects: # the form; name is snake_case ASCII, type one of
|
||||
# text (default), email, tel, textarea, date, select
|
||||
- { name: name }
|
||||
- { name: phone, type: tel }
|
||||
- { name: problem, type: textarea }
|
||||
- { name: urgency, type: select, options: [This week, This month, No rush] }
|
||||
- { name: can_do, type: select, multiple: true, options: [Drive, Cook, Carry] } # any number
|
||||
- { name: committee, type: select, options_from: groups } # the labelled groups above
|
||||
- { name: address, optional: true, private: true } # never on a public page
|
||||
```
|
||||
|
||||
Rules:
|
||||
- One need per (kind of person, thing they want). Work inside the
|
||||
business counts: a role recording or posting something is a need
|
||||
with `enters_as`. The intake's "what slows us down" usually names one.
|
||||
- Groups are the roles the intake names, as snake_case identifiers.
|
||||
- The front door holds at most five alternatives: doors plus needs
|
||||
with no door. A door with no need behind it, or a door on a need with
|
||||
`enters_as`, is refused.
|
||||
- Needs that land in the same bucket must have the same stages,
|
||||
done_when, skippable, moves_by, watched_by and handled_by.
|
||||
- Every required field is one the business will use to act on the
|
||||
record. If the intake gives no reason to need it, make it optional or
|
||||
leave it out. The required fields must not exceed max_fields.
|
||||
- Only `select` lists `options`, and it must list them.
|
||||
- Everything a person reads (who, wants, about, select options) is in
|
||||
the intake's language. Everything a machine reads (ids, buckets,
|
||||
groups, stages, field names) is plain ASCII: no æ, ø, å.
|
||||
- Write no personas.
|
||||
"#;
|
||||
|
||||
pub fn run(argv: &[String]) -> anyhow::Result<()> {
|
||||
let value = |flag: &str| {
|
||||
argv.iter()
|
||||
.position(|a| a == flag)
|
||||
.and_then(|i| argv.get(i + 1))
|
||||
.cloned()
|
||||
};
|
||||
let usage = "usage: portal-trainer plan --intake <file> --out <dir> [--llm 'cmd'] [--personas <yaml with personas:>] [--rounds 4]";
|
||||
let intake_path = value("--intake").ok_or_else(|| anyhow!(usage))?;
|
||||
let out = std::path::PathBuf::from(value("--out").ok_or_else(|| anyhow!(usage))?);
|
||||
let cmd = value("--llm").unwrap_or_else(|| "claude -p".to_string());
|
||||
let rounds: usize = value("--rounds")
|
||||
.map(|v| v.parse())
|
||||
.transpose()?
|
||||
.unwrap_or(4);
|
||||
let intake =
|
||||
std::fs::read_to_string(&intake_path).with_context(|| format!("reading {intake_path}"))?;
|
||||
let personas = match value("--personas") {
|
||||
Some(path) => {
|
||||
let text = std::fs::read_to_string(&path).with_context(|| format!("reading {path}"))?;
|
||||
let doc: serde_yaml::Value =
|
||||
serde_yaml::from_str(&text).with_context(|| format!("parsing {path}"))?;
|
||||
let list = doc
|
||||
.get("personas")
|
||||
.cloned()
|
||||
.ok_or_else(|| anyhow!("{path} has no `personas:`"))?;
|
||||
Some(serde_yaml::to_string(&serde_yaml::Mapping::from_iter([(
|
||||
serde_yaml::Value::from("personas"),
|
||||
list,
|
||||
)]))?)
|
||||
}
|
||||
None => None,
|
||||
};
|
||||
std::fs::create_dir_all(out.join("llm"))?;
|
||||
|
||||
let mut feedback = String::new();
|
||||
for round in 1..=rounds {
|
||||
let prompt = format!(
|
||||
"{REFERENCE}\n\n# Task\n\nA business filled in this intake. Write its plan.yaml: every need the intake implies, for the public and for each role inside the business, with the stages read off how the business says things progress, the form each need collects, and how much each person will tolerate.\n\n## Intake\n\n{intake}\n{feedback}\nReply with the YAML only, no prose, no code fences."
|
||||
);
|
||||
let reply = crate::llm_with(&cmd, &prompt, &[("TEMPERATURE", "0.2")])?;
|
||||
std::fs::write(out.join(format!("llm/{round:02}-prompt.md")), &prompt)?;
|
||||
std::fs::write(out.join(format!("llm/{round:02}-reply.md")), &reply)?;
|
||||
let yaml = extract_yaml(&reply);
|
||||
match check(&yaml) {
|
||||
Ok(plan) => {
|
||||
let mut text = yaml.trim_end().to_string();
|
||||
text.push('\n');
|
||||
if let Some(p) = &personas {
|
||||
text.push_str("\n# Held out from every model that writes words.\n");
|
||||
text.push_str(p);
|
||||
}
|
||||
// The attached personas were written against someone
|
||||
// else's need ids. The person reading the plan maps
|
||||
// them; the mismatches are listed so they know which.
|
||||
let whole: Plan =
|
||||
serde_yaml::from_str(&text).context("the plan with its personas attached")?;
|
||||
let strangers: Vec<String> = whole
|
||||
.personas
|
||||
.iter()
|
||||
.filter(|p| {
|
||||
!whole
|
||||
.needs
|
||||
.iter()
|
||||
.any(|n| n.id == p["need"].as_str().unwrap_or_default())
|
||||
})
|
||||
.map(|p| {
|
||||
format!(
|
||||
"{} (need: {})",
|
||||
p["id"].as_str().unwrap_or_default(),
|
||||
p["need"].as_str().unwrap_or_default()
|
||||
)
|
||||
})
|
||||
.collect();
|
||||
std::fs::write(out.join("plan.yaml"), &text)?;
|
||||
let groups: std::collections::BTreeSet<&str> = plan
|
||||
.needs
|
||||
.iter()
|
||||
.flat_map(|n| [Some(n.handled_by.as_str()), n.enters_as.as_deref()])
|
||||
.flatten()
|
||||
.collect();
|
||||
let buckets: std::collections::BTreeSet<&str> =
|
||||
plan.needs.iter().map(|n| n.lands_in.as_str()).collect();
|
||||
eprintln!("plan round {round}: {} need(s), {} group(s), {} bucket(s), {} persona(s) attached", plan.needs.len(), groups.len(), buckets.len(), whole.personas.len());
|
||||
for n in &plan.needs {
|
||||
eprintln!(
|
||||
" {}: {} -> {} ({}), {} of {} field(s) required",
|
||||
n.id,
|
||||
n.stages.join(" -> "),
|
||||
n.lands_in,
|
||||
n.handled_by,
|
||||
n.collects.iter().filter(|f| !f.optional).count(),
|
||||
n.max_fields
|
||||
);
|
||||
}
|
||||
if !strangers.is_empty() {
|
||||
eprintln!(" {} persona(s) name a need this plan does not have; set their `need:` to one of the ids above before building:\n {}", strangers.len(), strangers.join("\n "));
|
||||
}
|
||||
println!(
|
||||
"plan written to {}; read it before building from it",
|
||||
out.join("plan.yaml").display()
|
||||
);
|
||||
return Ok(());
|
||||
}
|
||||
Err(e) => {
|
||||
eprintln!("plan round {round}: {e}");
|
||||
feedback = format!("\n## Your last attempt was refused\n\n{e}\n\nFix that and return the whole plan again.\n\n```yaml\n{yaml}\n```\n");
|
||||
}
|
||||
}
|
||||
}
|
||||
bail!(
|
||||
"no plan passed in {rounds} round(s); the replies are in {}/llm/",
|
||||
out.display()
|
||||
)
|
||||
}
|
||||
|
||||
/// Parsed and validated exactly as `build` will.
|
||||
fn check(yaml: &str) -> anyhow::Result<Plan> {
|
||||
let plan: Plan =
|
||||
serde_yaml::from_str(yaml).map_err(|e| anyhow!("does not parse as a plan: {e}"))?;
|
||||
if !plan.personas.is_empty() {
|
||||
bail!("contains personas; write none");
|
||||
}
|
||||
if plan.needs.is_empty() {
|
||||
bail!("has no needs");
|
||||
}
|
||||
plan.validate()?;
|
||||
Ok(plan)
|
||||
}
|
||||
|
||||
/// The YAML in a reply: inside code fences if the model used them,
|
||||
/// otherwise from the first top-level key on. A stray JSON object
|
||||
/// (some models answer `{"business": ...}`) is turned into YAML.
|
||||
fn extract_yaml(reply: &str) -> String {
|
||||
let text = reply.trim();
|
||||
if let Some(start) = text.find("```") {
|
||||
let after = &text[start + 3..];
|
||||
let after = after.find('\n').map_or(after, |i| &after[i + 1..]);
|
||||
let body = after.find("```").map_or(after, |i| &after[..i]);
|
||||
return body.to_string();
|
||||
}
|
||||
if text.starts_with('{') {
|
||||
if let Some(v) = first_object(text) {
|
||||
return serde_yaml::to_string(&v).unwrap_or_default();
|
||||
}
|
||||
}
|
||||
let start = text
|
||||
.lines()
|
||||
.position(|l| {
|
||||
l.starts_with("business:") || l.starts_with("needs:") || l.starts_with("contact:")
|
||||
})
|
||||
.unwrap_or(0);
|
||||
text.lines().skip(start).collect::<Vec<_>>().join("\n")
|
||||
}
|
||||
|
||||
#[cfg(test)]
|
||||
mod tests {
|
||||
use super::*;
|
||||
|
||||
const GOOD: &str = "business: B\ncontact: { name: I, email: i@b.example }\nneeds:\n - id: enquiry\n who: an owner\n wants: a look\n lands_in: enquiries\n handled_by: project_managers\n stages: [open, quoted, won, lost]\n done_when: [won, lost]\n success: [won]\n max_fields: 3\n collects:\n - { name: name }\n - { name: phone, type: tel }\n";
|
||||
|
||||
#[test]
|
||||
fn yaml_is_taken_from_fences_prose_or_json() {
|
||||
assert_eq!(
|
||||
extract_yaml(&format!("Here you go:\n```yaml\n{GOOD}```\nDone.")),
|
||||
GOOD
|
||||
);
|
||||
assert_eq!(
|
||||
extract_yaml(&format!("Sure, the plan:\n\n{GOOD}")),
|
||||
GOOD.trim_end()
|
||||
);
|
||||
let from_json = extract_yaml("{\"business\": \"B\", \"needs\": []}");
|
||||
assert!(from_json.contains("business: B"));
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn a_plan_is_held_to_the_same_checks_as_build() {
|
||||
assert!(check(GOOD).is_ok());
|
||||
assert!(
|
||||
check("business: B\ncontact: { name: I, email: i@b.example }\nneeds: []\n")
|
||||
.unwrap_err()
|
||||
.to_string()
|
||||
.contains("no needs")
|
||||
);
|
||||
assert!(check(&GOOD.replace("max_fields: 3", "max_fields: 1"))
|
||||
.unwrap_err()
|
||||
.to_string()
|
||||
.contains("tolerate 1"));
|
||||
assert!(check(&format!(
|
||||
"{GOOD}personas:\n - {{ id: p, need: enquiry, text: hi }}\n"
|
||||
))
|
||||
.unwrap_err()
|
||||
.to_string()
|
||||
.contains("personas"));
|
||||
assert!(check(&GOOD.replace(
|
||||
"lands_in: enquiries",
|
||||
"lands_in: henvendelser\n extra: 1"
|
||||
))
|
||||
.unwrap_err()
|
||||
.to_string()
|
||||
.contains("does not parse"));
|
||||
}
|
||||
}
|
||||
Reference in New Issue
Block a user