Finding specific preference-guided Pareto solutions that represent different tradeoffs among multiple objectives is critical yet challenging in multi-objective problems. Existing methods are restrictive in preference definitions and/or their theoretical guarantees.

However, these methods suffer from high computational overhead because they require numerically solving a demanding optimization problem to determine an approximately optimal factorization prior to the actual model training. In this work, we present a new matrix factorization approach, BSR, which overcomes this computational bottleneck. By exploiting properties of the standard matrix square root, BSR allows to efficiently handle also large-scale problems.

We study the preference-based pure exploration problem for bandits with vector-valued rewards.

However, safe RL often suffers from sample inefficiency, requiring extensive interactions with the environment to learn a safe policy.

The convergence rates of PGA are also provided.