{"id": "TTC-110", "slug": "examples-counterexamples-and-corrections", "title": "Examples, counterexamples, and corrections", "level": "practitioner", "summary": "Teach behavior with paired examples and counterexamples, then turn each observed mistake into a scoped correction and a new test.", "learning_outcome": "Build a contrast set and convert a specific failure into a general, testable correction without overgeneralizing it.", "explanation": "An example reveals sequence and judgment that a general rule may hide. Pair it with a near-miss showing what must not happen and why. Vary irrelevant details so the agent does not memorize surface wording. A useful correction records the situation, observed output, expected output, reason, scope, and test; \u2018be more careful\u2019 does none of these. Check that a local correction does not damage another case. Keep examples fictional or properly minimized, and retain their review state and provenance alongside the rule they illustrate.", "worked_example": "Fictional case: the Pine Support Apprentice may say, \u2018I can draft a refund request for manager review.\u2019 A counterexample promises, \u2018Your refund has been approved.\u2019 The correction states that the apprentice never claims approval, applies to every refund channel, and is retested with a damaged-item case.", "exercise": "Create two positive examples, two close counterexamples, and one ambiguous case for a bounded role. Have the agent or a peer classify them with reasons. Turn the first failure into a correction record and add one new case that tests transfer plus one old case that guards against regression.", "success_criteria": ["Examples vary surface details while preserving the behavior under test.", "The correction states trigger, wrong behavior, right behavior, reason, scope, and owner.", "The new transfer test passes and previously correct behavior remains correct."], "limitations": ["A small example set cannot represent every edge case and may encode the author's blind spots.", "Examples guide behavior but do not technically enforce permissions or guarantee generalization."], "prerequisites": ["TTC-108", "TTC-109"], "next_lessons": ["TTC-111", "TTC-117"], "copyable_material": "# TTC-110 \u2014 Examples, counterexamples, and corrections\nObjective: Teach one behavior through contrasts and preserve mistakes as testable corrections.\nProcedure: Provide varied positive, negative, and ambiguous cases; record trigger, failure, expected behavior, reason, scope, and owner.\nRequired evidence: Keep the contrast set, correction diff, transfer test, and regression result.\nBoundaries: Do not generalize beyond tested scope or use private real cases when a fictional equivalent will work.\nCompletion test: The correction fixes the new transfer case without breaking the established positive cases.\nReview rule: Treat generated work as a draft until the named human reviewer accepts it.", "sources": [{"title": "Rules of Machine Learning: Best Practices for ML Engineering", "publisher": "Google", "url": "https://developers.google.com/machine-learning/guides/rules-of-ml"}, {"title": "AI RMF Playbook: Measure", "publisher": "National Institute of Standards and Technology", "url": "https://airc.nist.gov/airmf-resources/playbook/measure/"}], "version": "1.0.0", "reviewed_on": "2026-10-03", "review_status": "reviewed", "next_review_criteria": "Evaluation results reveal that a published example teaches an unintended shortcut or biased pattern.; A cited primary source is materially revised, replaced, or becomes unavailable.; Repeated learner results show that the exercise or success criteria are ambiguous.", "canonical_aliases": ["/train-an-ai-agent/"], "canonical_url": "https://teachthecompany.com/school/examples-counterexamples-and-corrections/"}