Preference Learning with Response Time

This academic paper introduces a new approach to preference learning by incorporating response time data alongside traditional binary choices. The authors highlight that while standard preference learning relies solely on which option a user prefers, the speed of the decision can provide valuable information about the strength of that preference. They propose novel methodologies, including a Neyman-orthogonal loss function, to leverage response time information based on the Evidence Accumulation Drift Diffusion model. Their theoretical analysis and experiments, including those on image-based preference tasks, demonstrate that this response time-augmented method significantly improves the sample efficiency and accuracy of learning human preferences compared to using only binary choice data. The research shows improved performance for both linear and non-linear reward models.