He Built a $200M AI Agent 10 Years Before ChatGPT

Summary:In this conversation, I talked to Ashish Shubham (VP of Engineering), who's been at ThoughtSpot for 10 years, about AI agents in enterprise analytics. ThoughtSpot started as a search-based analytics company trying to make data accessible to regular business users. In 2019, they tried building natural language interfaces using BERT, but only hit about 50% accuracy. For a product where enterprise customers make billion-dollar decisions, that wasn't good enough. They shelved the project.When ChatGPT came out, ThoughtSpot was ready. Ashish walked me through how they pivoted: they built a 25-30 person team, decided to use prompting instead of fine-tuning, and leveraged their existing semantic data modeling layer to get accuracy into the high 90s. We got into the technical evolution from monolithic systems to agent architectures with tools, how they went from manual human judges to using LLMs to evaluate their outputs, and how enterprise security requirements shaped what they built.We also talked about how software engineering is changing. Ashish said 50-60% of his code is AI-generated now, and he thinks system design is becoming the critical skill, even for junior engineers. He had an interesting take on the "95% of AI deployments fail" stat too.Chapters:0:00 Intro and Ashish's journey to ThoughtSpot from GoDaddy0:13 ThoughtSpot's mission to democratize data analytics for business users1:26 Early search-based analytics before natural language processing2:36 ThoughtSpot vs Tableau and the promise of self-service analytics4:40 The analyst bottleneck problem and how ThoughtSpot aimed to solve it5:49 Early technical challenges with in-memory databases and data migration8:11 Semantic data models, joins, and creating abstraction layers for users11:39 Who builds the data models and the role of analysts12:22 Pre-LLM natural language processing using BERT and word2vec in 2018-201914:43 The accuracy problem and ambiguity in translating user queries16:58 Trust challenges and why the early NLP product never became core19:59 Competition with Tableau, Looker, and Power BI22:44 How analyst roles changed with self-service analytics tools25:30 The ChatGPT moment and pivoting to LLM-powered natural language27:48 Early prompt engineering days and generating SQL with LLMs31:09 Training vs prompting debate and why fine-tuning was eventually abandoned34:28 Organizational changes and building the NLS team37:16 Coaching systems for company-specific terminology vs training models39:02 Evolution of evaluation methods from human judges to LLM-as-judge43:23 Moving to LangFuse and GCP for agent infrastructure46:29 How LLM context windows and capabilities evolved their product50:07 From 30-column limits to agentic systems with 90%+ accuracy52:52 RAG, column selection, and using proprietary data indexes54:59 Multi-model support and enterprise data security concerns59:14 How AI has changed Ashish's personal engineering workflow1:02:42 Impact of AI on the broader engineering organization1:04:15 Measuring AI productivity and the challenge of metrics1:07:26 50-60% AI-generated code and the changing nature of coding1:09:18 System design skills becoming more important than coding1:13:00 Junior engineers doing senior-level work and interview changes1:14:37 Customer conversations about Gen AI adoption across industries1:17:26 The MIT report on 95% agent failures and why it misses the point1:22:12 Agent architecture with LangGraph vs Google ADK and building internal agent platform1:24:26 Where value lies in the next two years: tools, skills, and optimization1:28:05 Startup opportunities in making AI accessible to non-technical users1:29:26 Closing remarks