Gaming Tool Preferences in Agentic LLMs
from Best AI papers explained Hosted by Enoch H. Kang
This academic paper explores a significant vulnerability in how large language models (LLMs) select and use external tools, which are crucial for their agentic capabilities. The research demonstrates that subtle modifications to a tool's natural language description, without altering its function, can dramatically influence whether an LLM chooses to use it, sometimes by a factor of over 10 times. Through experiments testing various descriptive changes, including assertive cues, claims of active maintenance, and usage examples, the authors show that tool selection is surprisingly fragile and easily manipulated across different LLMs. These findings highlight the critical need for more reliable methods for LLMs to evaluate tools, suggesting that relying solely on text descriptions is insufficient and exploitable, and propose that verifiable information about a tool's actual performance history is needed.
Embed this episode
No transcript yet.