FedPC: Federated Learning for Language Generation with Personal and Context Preference Embeddings

As conversational agents and dialog systems are deployed to real-world scenarios, these systems require data-efficient personalization paradigms such that language systems such as conversational agents can be effectively adapted on-device. The benefits of on-device optimization are two-fold; (1) Swift adaptation of model-behavior based on human-interactions [Dudy et al., 2021], (2) Privacy protection by means of retaining all data related to the user on-device [Li et al., 2020a]. One of the prevailing paradigms for learning from and engaging with end-users is federated learning. Federated learning is an inherently decentralized learning paradigm that assumes no access to a large labeled dataset and instead leverages averaged parameter updates across all users of the system [McMahan et al., 2017]. Such averaged updates invariably dilute individual preferences or deviations from the mean, resulting in a model that works well for the average user while failing to appropriately capture under-represented preferences or sub-groups within the data. In this work, we present a novel approach (FedPC) to personalizing federated learning with personal and context embeddings (collectively called "preference embeddings"), adapting more efficiently and effectively than prior work with respect to both data and compute on-device. We leverage the insight that a client's data distribution is informed by both individual preferences and additional contextual information. For example, while each user may have their own individual style, there may be more general population-wide trends that inform the style of personalized predictions (e.g., dialogue assistants helping patients with cognitive disorders, whereby agents can personalize to individual patients and broader condition-wide trends). While individual preferences may be unique to each client (e.g. a user's taste or affect), we can more accurately personalize to client preferences with the addition of context, as shared-context parameters carry beneficial stylistic information