Probabilistic circuits (PCs) enable us to learn joint distributions over a set of random variables and to perform various probabilistic queries in a tractable fashion. Though the tractability property allows PCs to scale beyond non-tractable models such as Bayesian Networks, scaling training and inference of PCs to larger, real-world datasets remains challenging. To remedy the situation, we show how PCs can be learned across multiple machines by recursively partitioning a distributed dataset, thereby unveiling a deep connection between PCs and federated learning (FL). This leads to federated circuits (FCs) -- a novel and flexible federated learning (FL) framework that (1) allows one to scale PCs on distributed learning environments (2) train PCs faster and (3) unifies for the first time horizontal, vertical, and hybrid FL in one framework by re-framing FL as a density estimation problem over distributed datasets. We demonstrate FC's capability to scale PCs on various large-scale datasets. Also, we show FC's versatility in handling horizontal, vertical, and hybrid FL within a unified framework on multiple classification tasks.

Naturalistic driving action recognition (NDAR) has proven to be an effective method for detecting driver distraction and reducing the risk of traffic accidents. However, the intrusive design of in-cabin cameras raises concerns about driver privacy. To address this issue, we propose a novel peer-to-peer (P2P) federated learning (FL) framework with continual learning, namely FedPC, which ensures privacy and enhances learning efficiency while reducing communication, computational, and storage overheads. Our framework focuses on addressing the clients' objectives within a serverless FL framework, with the goal of delivering personalized and accurate NDAR models. We demonstrate and evaluate the performance of FedPC on two real-world NDAR datasets, including the State Farm Distracted Driver Detection and Track 3 NDAR dataset in the 2023 AICity Challenge. The results of our experiments highlight the strong competitiveness of FedPC compared to the conventional client-to-server (C2S) FLs in terms of performance, knowledge dissemination rate, and compatibility with new clients.

As conversational agents and dialog systems are deployed to real-world scenarios, these systems require data-efficient personalization paradigms such that language systems such as conversational agents can be effectively adapted on-device. The benefits of on-device optimization are two-fold; (1) Swift adaptation of model-behavior based on human-interactions [Dudy et al., 2021], (2) Privacy protection by means of retaining all data related to the user on-device [Li et al., 2020a]. One of the prevailing paradigms for learning from and engaging with end-users is federated learning. Federated learning is an inherently decentralized learning paradigm that assumes no access to a large labeled dataset and instead leverages averaged parameter updates across all users of the system [McMahan et al., 2017]. Such averaged updates invariably dilute individual preferences or deviations from the mean, resulting in a model that works well for the average user while failing to appropriately capture under-represented preferences or sub-groups within the data. In this work, we present a novel approach (FedPC) to personalizing federated learning with personal and context embeddings (collectively called "preference embeddings"), adapting more efficiently and effectively than prior work with respect to both data and compute on-device. We leverage the insight that a client's data distribution is informed by both individual preferences and additional contextual information. For example, while each user may have their own individual style, there may be more general population-wide trends that inform the style of personalized predictions (e.g., dialogue assistants helping patients with cognitive disorders, whereby agents can personalize to individual patients and broader condition-wide trends). While individual preferences may be unique to each client (e.g. a user's taste or affect), we can more accurately personalize to client preferences with the addition of context, as shared-context parameters carry beneficial stylistic information