Des Traynor on reinventing Intercom twice and the 'four horsemen' of good AI companies

Key takeaways

1. Traynor says Fin's resolution rate rose from 25% at launch to 65% mostly through many small gains in architecture, prompts and handovers, not model upgrades. He calls it a thousand lead bullets, not one silver one.
2. Before adopting a new model, Intercom runs it against a test set of at least a thousand customer service scenarios. A higher resolution rate can hide more hallucinations, so Traynor says to find out why a score went up.
3. When every AI vendor claims to be the best, Traynor says to back claims with proof: a $1 million guarantee, named customer references, and metrics like resolution rate. He also suggests teaching buyers how to test products and spot weak ones.
4. Traynor's first question in every product review is what users said when shown the work. He warns that chasing prospects while current users are unhappy produces a messy product and churn.
5. Traynor says to raise money in line with your market size. He sees many businesses with fine ideas that raised on the assumption of a path to hundreds of millions in revenue.