Trace is the Next AutoDiff: Generative Optimization with Rich Feedback, Execution Traces, and LLMs

What this episode covers

This paper introduces Trace, a novel framework designed to optimize complex computational workflows end-to-end. Trace extends the concept of automatic differentiation by propagating the execution trace of a workflow as feedback, enabling the optimization of both differentiable and non-differentiable operations. The authors propose a general setup called OPTO (Output-Feedback Optimization), where an optimizer iteratively updates parameters based on rich feedback and the computational graph. They also present OptoPrime, an optimizer that leverages large language models to interpret the execution trace and feedback for parameter updates. Experimental results across various tasks, including game playing, traffic control, LLM prompt and code optimization, and robot manipulation, demonstrate that Trace with OptoPrime can efficiently learn complex strategies and outperform baseline methods, highlighting the value of using execution traces in optimization.

PodParley-generated summary based on available episode metadata and transcript content.