Weaddress these limitations byproposing a Probabilistic Transformer1 (ProbTransformer) that models a hierarchical latent distribution and performs sampling in the latent space.

We demonstrate the algorithm's effectiveness

Our initial investigation identifies which distributions accurately characterize the feature space, subsequently leveraging this priori to guide the alignment of the weakly supervised embeddings. Specifically, we analyze the superiority of the mixture of von Mises-Fisher distributions (moVMF) among several common distribution candidates.