“off policy part here comes where, um, what if you're…” — Dwarkesh Podcast at 2:01:56
“off policy part here comes where, um, what if you're relabeling states that your new policy would never visit? Like what's the point that you're kind of wasting capacity and in extreme limit, imagine your distribution of states and your training buffer are all states that you would never visit.”
0:00
2:37:29