Webly supervised learning has attracted increasing attention for its effectiveness in exploring publicly accessible data at scale without manual annotation. However, most existing methods of learning with web datasets are faced with challenges from label noise, and they have limited assumptions on clean samples under various noise. For instance, web images retrieved with queries of tiger cat (a cat species) and drumstick (a musical instrument) are almost dominated by images of tigers and chickens, which exacerbates the challenge of fine-grained visual concept learning. In this case, exploiting both web images and their associated texts is a requisite solution to combat real-world noise. In this paper, we propose Cross-modality Aligned Prototypes (CAPro), a unified prototypical contrastive learning framework to learn visual representations with correct semantics. For one thing, we leverage textual prototypes, which stem from the distinct concept definition of classes, to select clean images by text matching and thus disambiguate the formation of visual prototypes. For another, to handle missing and mismatched noisy texts, we resort to the visual feature space to complete and enhance individual texts and thereafter improve text matching. Such semantically aligned visual prototypes are further polished up with high-quality samples, and engaged in both cluster regularization and noise removal. Besides, we propose collective bootstrapping to encourage smoother and wiser label reference from appearance-similar instances in a manner of dictionary look-up. Extensive experiments on WebVision1k and NUS-WIDE (Web) demonstrate that CAPro well handles realistic noise under both single-label and multi-label scenarios. CAPro achieves new state-of-the-art performance and exhibits robustness to open-set recognition. Codes are available at https://github.com/yuleiqin/capro.

Accessing high-quality video content can be challenging due to insufficient and unstable network bandwidth. Recent advances in neural enhancement have shown promising results in improving the quality of degraded videos through deep learning. Neural-Enhanced Streaming (NES) incorporates this new approach into video streaming, allowing users to download low-quality video segments and then enhance them to obtain high-quality content without violating the playback of the video stream. We introduce BONES, an NES control algorithm that jointly manages the network and computational resources to maximize the quality of experience (QoE) of the user. BONES formulates NES as a Lyapunov optimization problem and solves it in an online manner with near-optimal performance, making it the first NES algorithm to provide a theoretical performance guarantee. Our comprehensive experimental results indicate that BONES increases QoE by 4% to 13% over state-of-the-art algorithms, demonstrating its potential to enhance the video streaming experience for users. Our code and data will be released to the public.

Recent advances in deep learning and automatic speech recognition have improved the accuracy of end-to-end speech recognition systems, but recognition of personal content such as contact names remains a challenge. In this work, we describe our personalization solution for an end-to-end speech recognition system based on connectionist temporal classification. Building on previous work, we present a novel method for generating additional subword tokenizations for personal entities from their pronunciations. We show that using this technique in combination with two established techniques, contextual biasing and wordpiece prior normalization, we are able to achieve personal named entity accuracy on par with a competitive hybrid system.

Transformers have become a key architecture in speech processing, but our understanding of how they build up representations of acoustic and linguistic structure is limited. In this study, we address this gap by investigating how measures of 'context-mixing' developed for text models can be adapted and applied to models of spoken language. We identify a linguistic phenomenon that is ideal for such a case study: homophony in French (e.g. livre vs livres), where a speech recognition model has to attend to syntactic cues such as determiners and pronouns in order to disambiguate spoken words with identical pronunciations and transcribe them while respecting grammatical agreement. We perform a series of controlled experiments and probing analyses on Transformer-based speech models. Our findings reveal that representations in encoder-only models effectively incorporate these cues to identify the correct transcription, whereas encoders in encoder-decoder models mainly relegate the task of capturing contextual dependencies to decoder modules.

Dementia is a neuro-degenerative disease affecting Early work in NLP for dementia relied on manual millions worldwide and is associated with cognitive engineered features based on specific lexical, decline, including language impairment (Forbes-acoustic and syntactic features stemming from description McKay and Venneri, 2005). Language dysfunction tasks (such as CTP), to detect linguistic may be difficult to detect in the early stages of dementia signs of cognitive decline (Fraser et al., 2016; Beltrami (Nestor et al., 2004); however, as the disease et al., 2018; Yeung et al., 2021). Recent progresses, a gradual decline of semantic knowledge work uses naive neural approaches to classify and ensues, and eventually, all linguistic functions analyse linguistic and acoustic characteristics so can be lost (Tang-Wai and Graham, 2008; Klimova as to either predict cognitive scores or achieve binary et al., 2015). Recognizing language disorders as classification of participants (Alzheimer's Disease prodromal symptoms in people with dementia may (AD) vs non-AD) (Karlekar et al., 2018; Balagopalan help with earlier diagnosis and improve disease et al., 2020; Nasreen et al., 2021b; Rohanian management.

Autonomous flight of flapping-wing robots is a major challenge for robot perception. Most of the previous sense-and-avoid works have studied the problem of obstacle avoidance for flapping-wing robots considering only static obstacles. This paper presents a fully onboard dynamic sense-and-avoid scheme for large-scale ornithopters using event cameras. These sensors trigger pixel information due to changes of illumination in the scene such as those produced by dynamic objects. The method performs event-by-event processing in low-cost hardware such as those onboard small aerial vehicles. The proposed scheme detects obstacles and evaluates possible collisions with the robot body. The onboard controller actuates over the horizontal and vertical tail deflections to execute the avoidance maneuver. The scheme is validated in both indoor and outdoor scenarios using obstacles of different shapes and sizes. To the best of the authors' knowledge, this is the first event-based method for dynamic obstacle avoidance in a flapping-wing robot.

Over the past few years, the division of gait phases has emerged as a complex area of research that carries significant importance for various applications in the field of gait technologies. The accurate partitioning of gait phases plays a crucial role in advancing these applications. Researchers have been exploring a range of sensors that can be employed to provide data for algorithms involved in gait phase partitioning. These sensors can be broadly categorized into two types: wearable and non-wearable, each offering unique advantages and capabilities. In our study aimed at examining the current approaches to gait analysis and detection specifically designed for implementation in ambulatory rehabilitation systems, we conducted a comprehensive meta-analysis of existing research studies. Our analysis revealed a diverse range of sensors and sensor combinations that demonstrate the ability to analyze gait patterns in ambulatory settings. These sensor options vary from basic force-based binary switches to more intricate setups incorporating multiple inertial sensors and sophisticated algorithms. The findings highlight the wide spectrum of available technologies and methodologies used in gait analysis for ambulatory applications. To conduct an extensive review, we systematically examined two prominent databases, IEEE and Scopus, with the aim of identifying relevant studies pertaining to gait analysis. The search criteria were limited to 189 papers published between 1999 and 2023. From this pool, we identified and included five papers that specifically focused on various techniques including Thresholding, Quasi-static method, adaptive classifier, and SVM-based approaches. These selected papers provided valuable insights for our review.

This paper focuses on the analysis of human gait cycle dynamics and presents a mathematical model to determine the torque exerted on the lower limb joints throughout the complete gait cycle, including its various phases. The study involved a healthy subject who participated in a series of initial walking experiments. The development of a mathematical model that accurately represents the natural motion of the human lower limb has garnered significant attention in the field of lower limb prosthetics design. In this study, the researchers incorporated the functional relationship between the limb joints and the end effector of the lower extremity. This knowledge is crucial for rehabilitation purposes as it helps in understanding the connectivity of joints, links, and the overall body orientation required to effectively control the motion of the actuators. When analysing physical activities, measurements of human strength play a crucial role. Traditionally, these measurements have focused on determining the maximum voluntary torque at a single joint angle and angular velocity. However, it is important to consider that the available strength varies significantly with joint position and velocity.

The quadratic complexity of the attention module makes it gradually become the bulk of compute in Transformer-based LLMs during generation. Moreover, the excessive key-value cache that arises when dealing with long inputs also brings severe issues on memory footprint and inference latency. In this work, we propose a plug-and-play approach that is able to incrementally compress the intermediate activation of a specified span of tokens into compact ones, thereby reducing both memory and computational cost when processing subsequent context. Experiments on both in-domain language modeling and zero-shot open-ended document generation demonstrate the advantage of our approach over sparse attention baselines in terms of fluency, n-gram matching, and semantic similarity. At last, we comprehensively profile the benefit of context compression on improving the system throughout. Code is available at https://github.com/DRSY/KV_Compression.

The development of deep learning software libraries enabled significant progress in the field by allowing users to focus on modeling, while letting the library to take care of the tedious and time-consuming task of optimizing execution for modern hardware accelerators. However, this has benefited only particular types of deep learning models, such as Transformers, whose primitives map easily to the vectorized computation. The models that explicitly account for structured objects, such as trees and segmentations, did not benefit equally because they require custom algorithms that are difficult to implement in a vectorized form. SynJax directly addresses this problem by providing an efficient vectorized implementation of inference algorithms for structured distributions covering alignment, tagging, segmentation, constituency trees and spanning trees. This is done by exploiting the connection between algorithms for automatic differentiation and probabilistic inference. With SynJax we can build large-scale differentiable models that explicitly model structure in the data. The code is available at https://github.com/google-deepmind/synjax