AI automation
78% of AI spendre-scoped to three bands
The flagship AI deal score was worse than a coin flip
The AI lead score ate most of the AI budget across three prompts. Back-tested on 1,171 closed deals it looked fine, AUC 0.642. Limited to deals that reached a demo, the ones reps actually triage, it dropped to 0.492. It had been detecting whether a deal moved at all, not whether it would close. What survived was coarse triage: the top band won 20.8% against a 13.8% baseline, the bottom band 3.9%.
- Re-scoped from ten tiers to three bands, with the caveat inside the API payload so nobody can read the number without it.
- We recommended against spending more to fix it.