Developer Kit

Model Evaluation Harness

Generates a full ML model evaluation framework with task-appropriate metrics, evaluation datasets, baseline comparisons, and a populated model card. Useful for making ML evaluation a default step instead of a skipped one. ML engineers shipping classifiers, regressors, or generative models; AI engineers shipping LLM-backed features; applied researchers who need a consistent evaluation story across experiments. Teams ship models on vibes, and the production regression shows up weeks later as a support ticket, a churn spike, or an angry Slack from sales. The reasons evaluation gets skipped are mundane: choosing the right metrics takes expertise, assembling an evaluation dataset is tedious, statistical significance is easy to get wrong, and model cards live in a drawer. A structured harness flips that — evaluation becomes a `pnpm eval` command with a results summary that pastes into a PR.