Writing
Engineering lessons from production AI systems
A short list of lessons that survive contact with production. No invented metrics.
Lessons I am willing to stand behind without dressing them up as case-study numbers:
- Evidence before interpretation. Store what happened. Then generate.
- Inconsistency is usually timing or retrieval. Debug those before you fine-tune a story.
- Idempotency is a product feature. Users experience duplicate actions, not your worker’s retry policy.
- Eval the path, not the paragraph. Tool traces and retrieved ids belong in the score.
- A prototype is a liability the day it meets retries.
This page will grow only with lessons that can be explained without confidential detail.