> ## Documentation Index > Fetch the complete documentation index at: https://docs.attensira.com/llms.txt > Use this file to discover all available pages before exploring further. # How playbooks improve > A playbook that falls short gets a new version, tested on sample businesses against the current one, and used only if it wins. ## The short version A playbook is the written procedure an agent follows for one kind of work, such as an answer block or a comparison page. Every week we check each playbook that is allowed to [run on its own](https://docs.attensira.com/agent/approval.md#the-drafts-it-actually-writes). If it is falling short, an agent writes a new version of its procedure. We test that version **in shadow**, and it replaces the current one only if it does better. This is house learning: it improves the playbook for everyone. Learning from your own declines and edits stays in your workspace and is described separately. ## When a playbook gets a new version A playbook gets a candidate version when either of these is true: * **Its win rate is provably low.** Its published win rate (`GET /v1/public/playbooks/win-rates`) across every engine over 30 days is under **25%**, and its whole 95% interval is under 25% too. If the interval reaches 25%, the difference is inside the noise, so we do not call the rate low and nothing is drafted. * **Its drafts miss.** On the sample businesses, a draft it wrote failed the never-say, claim-source or safety check, or its drafts covered less than 70% of what a useful draft covers. The candidate comes with a written rationale. The rationale says why the playbook is under its band, names each draft that missed and why, and says what the new version changes. The agent writing it sees the current procedure, those misses and a few drafts that passed. It never sees anything from a customer's workspace: the drafts belong to the made-up sample businesses, and the win rate is the published one, which never names a workspace. ## Tested in shadow, never shipped Both versions write two drafts for each of the same 9 sample businesses, and the same checks score both: never-say, claim source (the same claim tracer that checks your drafts), safety and value. A playbook with no win rate yet is judged on these drafts alone. Within 30 days of a version change, the win rate is not used to draft a new version, because it still mixes changes made under the old one. **A shadow draft is never shipped and never shown as your change.** It exists only while it is being scored, and nothing is stored but the scores. The candidate replaces the current version only if both of these are true: * **It is better, beyond chance.** Its eval score is at least 0.05 higher. The eval score is the average of its pass rate on the checks and its value score. It must also beat the current version draft for draft (same business, same sample) more often than chance would allow: a sign test at the 5% level. A gain that one or two lucky drafts could explain is inside the noise and promotes nothing. * **It is no less safe.** It misses the never-say, claim-source and safety checks no more often than the current version does. Otherwise it is **rejected**, and the reason is recorded. Once a version is promoted, every run follows it from the next run on, in every workspace. Its scored drafts also count as the drafts it is [checked on](https://docs.attensira.com/agent/approval.md#the-drafts-it-actually-writes), so a lane does not wait at L3 for the new version. ## Confirmed, or rolled back if it does worse After a promoted version has run for 30 days, we compare its win rate with the rate when it was promoted. A version promoted before the playbook had a published win rate is compared with the 25% band instead. * If the new 95% interval sits entirely below that line, the version is **rolled back**. The version it replaced comes back with a fresh 30-day window of its own, and both steps are recorded. * Otherwise it is recorded once as **confirmed**. A smaller drop is inside the noise and changes nothing. If someone edits a playbook's file after a version was drafted from it, the edited file wins at once. The version stops running and is recorded as retired. ## Reading a playbook's versions [`GET /v1/playbooks/{name}`](https://docs.attensira.com/api/endpoints), or `get` on a playbook over MCP, returns these fields: | Field | What it holds | | - | - | | `version` | The version runs follow now. | | `why_changed` | Why the current version became current, if it is not the original. | | `versions` | Every version beyond the original, newest first. | Each entry in `versions` has these fields: | Field | What it holds | | - | - | | `state` | One of `candidate`, `live`, `rejected`, `retired` or `rolled_back`. | | `rationale` | Why the version was written and what it changes. | | `misses` | The misses it was written to fix. | | `incumbent`, `candidate` and `comparison` | The two shadow scores it was decided on, and the paired sign test between them. | | `outcome` | One sentence on why it is in its state. | `GET /v1/playbooks` and `list` with `type=playbook` show each playbook's current `version`. The procedure text itself is not returned. This documentation is built and hosted on [Mintlify](https://mintlify.com), a developer documentation platform.