Electroencephalography (EEG) provides access to neuronal dynamics noninvasively with millisecond resolution, rendering it a viable method in neuroscience and healthcare. However, its utility is limited as current EEG technology does not generalize well across domains (i.e., sessions and subjects) without expensive supervised re-calibration. Contemporary methods cast this transfer learning (TL) problem as a multi-source/-target unsupervised domain adaptation (UDA) problem and address it with deep learning or shallow, Riemannian geometry aware alignment methods. Both directions have, so far, failed to consistently close the performance gap to state-of-the-art domain-specific methods based on tangent space mapping (TSM) on the symmetric, positive definite (SPD) manifold. Here, we propose a machine learning framework that enables, for the first time, learning domain-invariant TSM models in an end-to-end fashion. To achieve this, we propose a new building block for geometric deep learning, which we denote SPD domain-specific momentum batch normalization (SPDDSMBN). ASPDDSMBN layer can transform domain-specific SPD inputs into domain-invariant SPD outputs, and can be readily applied to multi-source/-target and online UDA scenarios. In extensive experiments with 6 diverse EEG brain-computer interface (BCI) datasets, we obtain state-of-the-art performance in inter-session and -subject TL with a simple, intrinsically interpretable network architecture, which we denote TSMNet.

A.1 Secret Sharing CRYPTEN uses two different types of secret sharing: (1) arithmetic secret sharing [9] and (2) binary secret sharing [11]. Below, we describe the secret sharing methods for single values xbut they can trivially be extended to real-valued vectors x. A.1.1 Arithmetic Secret Sharing CRYPTEN uses arithmetic secret sharing to perform most MPC computations. In arithmetic secret sharing, a scalar value x Z/QZ (where Z/QZ denotes a ring with Qelements) is shared across |P| parties in such a way that the sum of the shares reconstructs the original value x. We denote the sharing of x by [x] = {[x]p}p P, where [x]p Z/QZ indicates party p's share of x. The representation has the property that P p P[x]p mod Q=x. We use a fixed-point encoding to obtain xfrom a floating-point value xR. To do so, we multiply xR with a large scaling factor B and round to the nearest integer: x = bBxRe, where B = 2L for some precision parameter, L. To decode a value, x, we compute xR x/B. Encoding real-valued numbers this way incurs a precision loss that is inversely proportional to L. Since we scale by a factor B to encode numbers, we must scale down by a factor B after every multiplication.

Secure multi-party computation (MPC) allows parties to perform computations on data while keeping that data private. This capability has great potential for machine-learning applications: it facilitates training of machine-learning models on private data sets owned by different parties, evaluation of one party's private model using another party's private data, etc. Although a range of studies implement machine-learning models via secure MPC, such implementations are not yet mainstream. Adoption of secure MPC is hampered by the absence of flexible software frameworks that "speak the language" of machine-learning researchers and engineers. To foster adoption of secure MPC in machine learning, we present CRYPTEN: a software framework that exposes popular secure MPC primitives via abstractions that are common in modern machine-learning frameworks, such as tensor computations, automatic differentiation, and modular neural networks. This paper describes the design of CRYPTEN and measure its performance on state-ofthe-art models for text classification, speech recognition, and image classification. Our benchmarks show that CRYPTEN's GPU support and high-performance communication between (an arbitrary number of) parties allows it to perform efficient private evaluation of modern machine-learning models under a semi-honest threat model. For example, two parties using CRYPTEN can securely predict phonemes in speech recordings using Wav2Letter [17] faster than real-time. We hope that CRYPTEN will spur adoption of secure MPC in the machine-learning community.

Current self-supervised learning (SSL) methods (e.g., SimCLR, DINO, VICReg, MOCOv3) target primarily on representations at instance level and do not generalize well to dense prediction tasks, such as object detection and segmentation. Towards aligning SSL with dense predictions, this paper demonstrates for the first time the underlying mean-shift clustering process of Vision Transformers (ViT), which aligns well with natural image semantics (e.g., a world of objects and stuffs). By employing transformer for joint embedding and clustering, we propose a bi-level feature clustering SSL method, coined Feature-Level Self-supervised Learning (FLSL). We present the formal definition of the FLSL problem and construct the objectives from the mean-shift and k-means perspectives. We show that FLSL promotes remarkable semantic cluster representations and learns an encoding scheme amenable to intra-view and inter-view feature clustering. Experiments show that FLSL yields significant improvements in dense prediction tasks, achieving 44.9 (+2.8)% AP and 46.5% AP in object detection, as well as 40.8 (+2.3)%