“So the unpalacy thing and also much of RL has kind of…” — Dwarkesh Podcast at 2:10:58
“So the unpalacy thing and also much of RL has kind of converged to a much more unpalacy setup, where they don't really try to like directly train on off-policy data. At best, they use off-policy data as a way to reduce variance, but not directly influence the objective. Sorry, I have to converge to that?”
0:00
2:37:29