hyperboloid
More Than A Shortcut: A Hyperbolic Approach To Early-Exit Networks
Bhosale, Swapnil, Frateanu, Cosmin, Clark, Camilla, Jasonas, Arnoldas, Mitchell, Chris, Zhu, Xiatian, Ithapu, Vamsi Krishna, Ferroni, Giacomo, Bilen, Cagdas, Parekh, Sanjeel
Deploying accurate event detection on resource-constrained devices is challenged by the trade-off between performance and computational cost. While Early-Exit (EE) networks offer a solution through adaptive computation, they often fail to enforce a coherent hierarchical structure, limiting the reliability of their early predictions. To address this, we propose Hyperbolic Early-Exit networks (HypEE), a novel framework that learns EE representations in the hyperbolic space. Our core contribution is a hierarchical training objective with a novel entailment loss, which enforces a partial-ordering constraint to ensure that deeper network layers geometrically refine the representations of shallower ones. Experiments on multiple audio event detection tasks and backbone architectures show that HypEE significantly outperforms standard Euclidean EE baselines, especially at the earliest, most computationally-critical exits. The learned geometry also provides a principled measure of uncertainty, enabling a novel triggering mechanism that makes the overall system both more efficient and more accurate than a conventional EE and standard backbone models without early-exits.
A geometrical approach to solve the proximity of a point to an axisymmetric quadric in space
Patra, Bibekananda, Kolte, Aditya Mahesh, Bandyopadhyay, Sandipan
This paper presents the classification of a general quadric into an axisymmetric quadric (AQ) and the solution to the problem of the proximity of a given point to an AQ. The problem of proximity in $R^3$ is reduced to the same in $R^2$, which is not found in the literature. A new method to solve the problem in $R^2$ is used based on the geometrical properties of the conics, such as sub-normal, length of the semi-major axis, eccentricity, slope and radius. Furthermore, the problem in $R^2$ is categorised into two and three more sub-cases for parabola and ellipse/hyperbola, respectively, depending on the location of the point, which is a novel approach as per the authors' knowledge. The proposed method is suitable for implementation in a common programming language, such as C and proved to be faster than a commercial library, namely, Bullet.
Hypo-paradoxical Linkages: Linkages That Should Move-But Don't
Classical mobility criteria, such as the Chebyshev-Grรผbler-Kutzbach formula (1904), offer a quick estimate for the number of degrees-of-freedom in linkage systems based on link and joint counts. However, several historical linkages defy these predictions, exhibiting unexpected motion despite appearing overconstrained. Such linkages are known as paradoxical linkages. In addition to Grรผbler's formula, rigidity criteria such as Laman's theorem provide a somewhat more complex condition for the rigidity of planar linkages. The theorem states that a graph with n vertices is minimally rigid in the plane if it has exactly 2n 3 edges, and no subset of k vertices spans more than 2k 3 edges. However, Laman's criterion also breaks down in the presence of non generic linkages necessitating deeper analysis of configuration spaces and infinitesimal rigidity (cf.
Hyperbolic Image-Text Representations
Desai, Karan, Nickel, Maximilian, Rajpurohit, Tanmay, Johnson, Justin, Vedantam, Ramakrishna
Visual and linguistic concepts naturally organize themselves in a hierarchy, where a textual concept "dog" entails all images that contain dogs. Despite being intuitive, current large-scale vision and language models such as CLIP do not explicitly capture such hierarchy. We propose MERU, a contrastive model that yields hyperbolic representations of images and text. Hyperbolic spaces have suitable geometric properties to embed tree-like data, so MERU can better capture the underlying hierarchy in image-text datasets. Our results show that MERU learns a highly interpretable and structured representation space while being competitive with CLIP's performance on standard multi-modal tasks like image classification and image-text retrieval.
A Model Restoration
Glancing at Barcelona's still-unfinished Sagrada Famรญlia Roman Catholic basilica, with its famous sandcastle-like exterior, it is easy to get the wrong idea about its architect, Antoni Gaudรญ, as a carefree, loosey-goosey artist. The whimsical exterior hides a geometrically sophisticated, structurally advanced design--a big part of the reason this grand basilica, begun in 1882, has taken so many decades to build, remaining the world's longest-running ongoing architectural project. This complexity required an utterly different approach to modeling than what architects had typically deployed. Instead of using two-dimensional drawings to guide builders, Gaudรญ relied heavily on large, high-fidelity plaster models--models that needed to be reverse engineered and rebuilt after extensive damage during the Spanish Civil War. In a separate project, Gaudรญ pioneered the use of hanging-chain models that enable changes in real time; though he did not use these interactive models on the Sagrada Famรญlia, they guided his thinking and prefigured the so-called parametric design software that has been instrumental to the acceleration of the project's pace in recent years.
Metric Learning for Dynamic Text Classification
Wohlwend, Jeremy, Elenberg, Ethan R., Altschul, Samuel, Henry, Shawn, Lei, Tao
Traditional text classifiers are limited to predicting over a fixed set of labels. However, in many real-world applications the label set is frequently changing. For example, in intent classification, new intents may be added over time while others are removed. We propose to address the problem of dynamic text classification by replacing the traditional, fixed-size output layer with a learned, semantically meaningful metric space. Here the distances between textual inputs are optimized to perform nearest-neighbor classification across overlapping label sets. Changing the label set does not involve removing parameters, but rather simply adding or removing support points in the metric space. Then the learned metric can be fine-tuned with only a few additional training examples. We demonstrate that this simple strategy is robust to changes in the label space. Furthermore, our results show that learning a non-Euclidean metric can improve performance in the low data regime, suggesting that further work on metric spaces may benefit low-resource research.
Statistical embedding for directed graphs
Funke, Thorben, Guo, Tian, Lancic, Alen, Antulov-Fantulin, Nino
We propose a novel statistical node embedding of directed graphs, which is based on a global minimization of pairwise relative entropy and graph geodesics in a non-linear way. Each node is encoded with a probability density function over a measurable real n-dimensional space. Furthermore, we analyze the connection to the geometrical properties of such embedding and characterize the curvature of the statistical manifolds. Extensive experiments show that our proposed embedding is better preserving the global geodesic information of graphs, as well as outperforming existing embedding models on directed graphs in a variety of evaluation metrics, in an unsupervised setting.
Scalable Hyperbolic Recommender Systems
Chamberlain, Benjamin Paul, Hardwick, Stephen R., Wardrope, David R., Dzogang, Fabon, Daolio, Fabio, Vargas, Saรบl
We present a large scale hyperbolic recommender system. We discuss why hyperbolic geometry is a more suitable underlying geometry for many recommendation systems and cover the fundamental milestones and insights that we have gained from its development. In doing so, we demonstrate the viability of hyperbolic geometry for recommender systems, showing that they significantly outperform Euclidean models on datasets with the properties of complex networks. Key to the success of our approach are the novel choice of underlying hyperbolic model and the use of the Einstein midpoint to define an asymmetric recommender system in hyperbolic space. These choices allow us to scale to millions of users and hundreds of thousands of items.
Metric Learning on Manifolds
Recent literature has shown that symbolic data, such as text and graphs, is often better represented by points on a curved manifold, rather than in Euclidean space. However, geometrical operations on manifolds are generally more complicated than in Euclidean space, and thus many techniques for processing and analysis taken for granted in Euclidean space are difficult on manifolds. A priori, it is not obvious how we may generalize such methods to manifolds. We consider specifically the problem of distance metric learning, and present a framework that solves it on a large class of manifolds, such that similar data are located in closer proximity with respect to the manifold distance function. In particular, we extend the existing metric learning algorithms, and derive the corresponding sample complexity rates for the case of manifolds. Additionally, we demonstrate an improvement of performance in $k$-means clustering and $k$-nearest neighbor classification on real-world complex networks using our methods.