Reviews: Hybrid-MST: A Hybrid Active Sampling Strategy for Pairwise Preference Aggregation

Neural Information Processing Systems 

The technique used for aggregation is the Bradley-Terry model with computational saving techniques. The pairs are queried with Expected Information Gain (from the Bradley-Terry model) and either choosing the most informative pairs or choosing a batch of pairs corresponding to a MST built on the graph with edges based on the most informative pairs. Questions: Something that I didn't quite understand is that this work claimed to run the preferences in batches, however, it doesn't appear that they are run in batches for the first standard trial number. Can the authors please clarify this? The runtime for small problems (n 10-20) show that the algorithm runs relatively slowly and quadratically.