Counterfactual Evaluation and Learning for Interactive Systems
- Other
Prof. Dr. Thorsten Joachims, Full professor for machine learning at the Department of Computer Science and the Department of Information Science at Cornell University
Abstract: This tutorial provides an introduction to a rapidly growing area of new methods for training and evaluating intelligent systems that act autonomously, including applications from recommendation and search engines to automation and robotics. At the core of these methods are counterfactual estimators that enable the use of existing log data to estimate how some new target policy would have performed, if it had been used instead of the policy that logged the data. We say that those estimators work "off-policy", since the policy that logged the data is different from the target policy. In this way, counterfactual estimators enable Off-policy Evaluation (OPE) akin to an unbiased offline A/B test, as well as learning new decision-making policies through Off-policy Learning (OPL). The goal of this tutorial is to summarize the foundations of OPE and OPL, and provide an overview of activity and future directions in this field.
Prof. Dr. Thorsten Joachims
Thorsten Joachims is a Professor in the Department of Computer Science and in the Department of Information Science at Cornell University, and he is an Amazon Scholar. His research interests center on a synthesis of theory and system building in machine learning, with applications in information access, language technology, and recommendation. His past research focused on counterfactual and causal inference, learning to rank, structured output prediction, support vector machines, text classification, learning with preferences, and learning from implicit feedback. He is an ACM Fellow, AAAI Fellow, KDD Innovations Award recipient, and member of the SIGIR Academy.