Gaming Tool Preferences in Agentic LLMs

What this episode covers

This academic paper explores a significant vulnerability in how large language models (LLMs) select and use external tools, which are crucial for their agentic capabilities. The research demonstrates that subtle modifications to a tool's natural language description, without altering its function, can dramatically influence whether an LLM chooses to use it, sometimes by a factor of over 10 times. Through experiments testing various descriptive changes, including assertive cues, claims of active maintenance, and usage examples, the authors show that tool selection is surprisingly fragile and easily manipulated across different LLMs. These findings highlight the critical need for more reliable methods for LLMs to evaluate tools, suggesting that relying solely on text descriptions is insufficient and exploitable, and propose that verifiable information about a tool's actual performance history is needed.

PodParley-generated summary based on available episode metadata and transcript content.