Optimizing recommender systems based on user interaction data is mainly seen as a problem of dealing with selection bias, where most existing work assumes that interactions from different users are independent. However, it has been shown that in reality user feedback is often influenced by earlier interactions of other users, e.g. via average ratings, number of views or sales per item, etc. This phenomenon is known as the bandwagon effect. In contrast with previous literature, we argue that the bandwagon effect should not be seen as a problem of statistical bias. In fact, we prove that this effect leaves both individual interactions and their sample mean unbiased. Nevertheless, we show that it can make estimators inconsistent, introducing a distinct set of problems for convergence in relevance estimation. Our theoretical analysis investigates the conditions under which the bandwagon effect poses a consistency problem and explores several approaches for mitigating these issues. This work aims to show that the bandwagon effect poses an underinvestigated open problem that is fundamentally distinct from the well-studied selection bias in recommendation.

When I was young, I took a packed lunch to school every day, and since I grew up in Australia, my packed lunch would include a couple of Vegemite sandwiches. Unless you grew up in Australia, you've probably never tasted it. And judging by this American's first taste reaction of "Oh, that's bad!", you probably wouldn't like the taste if you tried it out. But I loved my Vegemite sandwiches, and they were my one-and-only lunchtime choice, no matter what the circumstances. While this blog isn't about Vegemite, it is related to lunch, specifically the no free lunch theorem.