speller
Phonetic Reconstruction of the Consonant System of Middle Chinese via Mixed Integer Optimization
This paper is concerned with phonetic reconstruction of the consonant system of Middle Chinese. We propose to cast the problem as a Mixed Integer Programming problem, which is able to automatically explore homophonic information from ancient rhyme dictionaries and phonetic information from modern Chinese dialects, the descendants of Middle Chinese. Numerical evaluation on a wide range of synthetic and real data demonstrates the effectiveness and robustness of the new method. We apply the method to information from Guangyun and 20 modern Chinese dialects to obtain a new phonetic reconstruction result. A linguistically-motivated discussion of this result is also provided.
Towards Predictive Communication with Brain-Computer Interfaces integrating Large Language Models
This perspective article aims at providing an outline of the state of the art and future developments towards the integration of cutting-edge predictive language models with BCI. A synthetic overview of early and more recent linguistic models, from natural language processing (NLP) models to recent LLM, that to a varying extent improved predictive writing systems, is first provided. Second, a summary of previous BCI implementations integrating language models is presented. The few preliminary studies investigating the possible combination of LLM with BCI spellers to efficiently support fast communication and control are then described. Finally, current challenges and limitations towards the full integration of LLM with BCI systems are discussed. Recent investigations suggest that the combination of LLM with BCI might drastically improve human-computer interaction in patients with motor or language disorders as well as in healthy individuals. In particular, the pretrained autoregressive transformer models, such as GPT, that capitalize from parallelization, learning through pre-training and fine-tuning, promise a substantial improvement of BCI for communication with respect to previous systems incorporating simpler language models. Indeed, among various models, the GPT-2 was shown to represent an excellent candidate for its integration into BCI although testing was only perfomed on simulated conversations and not on real BCI scenarios. Prospectively, the full integration of LLM with advanced BCI systems might lead to a big leap forward towards fast, efficient and user-adaptive neurotechnology.
Evaluation Of P300 Speller Performance Using Large Language Models Along With Cross-Subject Training
Parthasarathy, Nithin, Soetedjo, James, Panchavati, Saarang, Parthasarathy, Nitya, Arnold, Corey, Pouratian, Nader, Speier, William
Amyotrophic lateral sclerosis (ALS), a progressive neuromuscular degenerative disease, severely restricts patient communication capacity within a few years of onset, resulting in a significant deterioration of quality of life. The P300 speller brain computer interface (BCI) offers an alternative communication medium by leveraging a subject's EEG response to characters traditionally highlighted on a character grid on a graphical user interface (GUI). A recurring theme in P300-based research is enhancing performance to enable faster subject interaction. This study builds on that theme by addressing key limitations, particularly in the training of multi-subject classifiers, and by integrating advanced language models to optimize stimuli presentation and word prediction, thereby improving communication efficiency. Furthermore, various advanced large language models such as Generative Pre-Trained Transformer (GPT2), BERT, and BART, alongside Dijkstra's algorithm, are utilized to optimize stimuli and provide word completion choices based on the spelling history. In addition, a multi-layered smoothing approach is applied to allow for out-of-vocabulary (OOV) words. By conducting extensive simulations based on randomly sampled EEG data from subjects, we show substantial speed improvements in typing passages that include rare and out-of-vocabulary (OOV) words, with the extent of improvement varying depending on the language model utilized. The gains through such character-level interface optimizations are approximately 10%, and GPT2 for multi-word prediction provides gains of around 40%. In particular, some large language models achieve performance levels within 10% of the theoretical performance limits established in this study. In addition, both within and across subjects, training techniques are explored, and speed improvements are shown to hold in both cases.
High Performance P300 Spellers Using GPT2 Word Prediction With Cross-Subject Training
Parthasarathy, Nithin, Soetedjo, James, Panchavati, Saarang, Parthasarathy, Nitya, Arnold, Corey, Pouratian, Nader, Speier, William
Amyotrophic lateral sclerosis (ALS) severely impairs patients' ability to communicate, often leading to a decline in their quality of life within a few years of diagnosis. The P300 speller brain-computer interface (BCI) offers an alternative communication method by interpreting a subject's EEG response to characters presented on a grid interface. This paper addresses the common speed limitations encountered in training efficient P300-based multi-subject classifiers by introducing innovative "across-subject" classifiers. We leverage a combination of the second-generation Generative Pre-Trained Transformer (GPT2) and Dijkstra's algorithm to optimize stimuli and suggest word completion choices based on typing history. Additionally, we employ a multi-layered smoothing technique to accommodate out-of-vocabulary (OOV) words. Through extensive simulations involving random sampling of EEG data from subjects, we demonstrate significant speed enhancements in typing passages containing rare and OOV words. These optimizations result in approximately 10% improvement in character-level typing speed and up to 40% improvement in multi-word prediction. We demonstrate that augmenting standard row/column highlighting techniques with layered word prediction yields close-to-optimal performance. Furthermore, we explore both "within-subject" and "across-subject" training techniques, showing that speed improvements are consistent across both approaches.
Towards gaze-independent c-VEP BCI: A pilot study
Narayanan, S., Ahmadi, S., Desain, P., Thielen, J.
A limitation of brain-computer interface (BCI) spellers is that they require the user to be able to move the eyes to fixate on targets. This poses an issue for users who cannot voluntarily control their eye movements, for instance, people living with late-stage amyotrophic lateral sclerosis (ALS). This pilot study makes the first step towards a gaze-independent speller based on the code-modulated visual evoked potential (c-VEP). Participants were presented with two bi-laterally located stimuli, one of which was flashing, and were tasked to attend to one of these stimuli either by directly looking at the stimuli (overt condition) or by using spatial attention, eliminating the need for eye movement (covert condition). The attended stimuli were decoded from electroencephalography (EEG) and classification accuracies of 88% and 100% were obtained for the covert and overt conditions, respectively. These fundamental insights show the promising feasibility of utilizing the c-VEP protocol for gaze-independent BCIs that use covert spatial attention when both stimuli flash simultaneously.
A P300 BCI for the Masses: Prior Information Enables Instant Unsupervised Spelling
The usability of Brain Computer Interfaces (BCI) based on the P300 speller is severely hindered by the need for long training times and many repetitions of the same stimulus. In this contribution we introduce a set of unsupervised hierarchical probabilistic models that tackle both problems simultaneously by incorporating prior knowledge from two sources: information from other training subjects (through transfer learning) and information about the words being spelled (through language models). We show, that due to this prior knowledge, the performance of the unsupervised models parallels and in some cases even surpasses that of supervised models, while eliminating the tedious training session.
Contextual Multilingual Spellchecker for User Queries
Sharma, Sanat, Valls-Vargas, Josep, King, Tracy Holloway, Guerin, Francois, Arora, Chirag
Spellchecking is one of the most fundamental and widely used search features. Correcting incorrectly spelled user queries not only enhances the user experience but is expected by the user. However, most widely available spellchecking solutions are either lower accuracy than state-of-the-art solutions or too slow to be used for search use cases where latency is a key requirement. Furthermore, most innovative recent architectures focus on English and are not trained in a multilingual fashion and are trained for spell correction in longer text, which is a different paradigm from spell correction for user queries, where context is sparse (most queries are 1-2 words long). Finally, since most enterprises have unique vocabularies such as product names, off-the-shelf spelling solutions fall short of users' needs. In this work, we build a multilingual spellchecker that is extremely fast and scalable and that adapts its vocabulary and hence speller output based on a specific product's needs. Furthermore, our speller out-performs general purpose spellers by a wide margin on in-domain datasets. Our multilingual speller is used in search in Adobe products, powering autocomplete in various applications.
Tiny noise, big mistakes: Adversarial perturbations induce errors in Brain-Computer Interface spellers
Zhang, Xiao, Wu, Dongrui, Ding, Lieyun, Luo, Hanbin, Lin, Chin-Teng, Jung, Tzyy-Ping, Chavarriaga, Ricardo
An electroencephalogram (EEG) based brain-computer interface (BCI) speller allows a user to input text to a computer by thought. It is particularly useful to severely disabled individuals, e.g., amyotrophic lateral sclerosis patients, who have no other effective means of communication with another person or a computer. Most studies so far focused on making EEG-based BCI spellers faster and more reliable; however, few have considered their security. This study, for the first time, shows that P300 and steady-state visual evoked potential BCI spellers are very vulnerable, i.e., they can be severely attacked by adversarial perturbations, which are too tiny to be noticed when added to EEG signals, but can mislead the spellers to spell anything the attacker wants. The consequence could range from merely user frustration to severe misdiagnosis in clinical applications. We hope our research can attract more attention to the security of EEG-based BCI spellers, and more broadly, EEG-based BCIs, which has received little attention before.
My Daughter's Spelling Is Atrocious
Care and Feeding is Slate's parenting advice column. In addition to our traditional advice, every Thursday we feature an assortment of teachers from across the country answering your education questions. Have a question for our teachers? Email askateacher@slate.com or post it in the Slate Parenting Facebook group. This week's Ask a Teacher panel: Matthew Dicks, fifth grade, Connecticut Cassy Sarnell, preschool special education, New York Carrie Bauer, middle and high school, New York Amy Scott, eighth grade, North Carolina My fourth-grade daughter is a joy to be around, a good friend, and a well-behaved student.
The Limits of "Grit"
Angela Duckworth, in her best-selling book, "Grit: The Power of Passion and Perseverance," celebrates a man whom she calls a "grit paragon": Pete Carroll, the coach of the Seattle Seahawks, who led the team to a Super Bowl victory in 2014. It seems that Carroll had seen Duckworth's TED talk nine months earlier and got in touch, eager to reassure her that building grit was exactly what the Seahawks culture was all about. Two years later, Duckworth visited the Seahawks training camp. She lectured to the team's players and coaching staff. The subject was . . . Duckworth was impressed by the Seahawks, and she quotes sentiments that are characteristic of the Carroll ethos: "Compete in everything you do. Since the team trains ferociously all the time--going all out, for instance, in bone-crunching intra-squad practice sessions--this conversation may not have been entirely necessary. Duckworth, a professor of psychology at the University of Pennsylvania, finds grit in the best possible places. Her grit obsession, as she recounts, began at least a decade earlier. As a graduate student, she visited West Point, where each year twelve hundred new cadets go through a gruelling seven-week training regimen ("Barracks Beast") before entering freshman year. Most make it through, though some do not. Duckworth could make some guesses. In this same period, eager to find out what made top people successful, she was interviewing "leaders in business, art, athletics, journalism, academia, medicine and law." She discovered that "the highly successful had a kind of ferocious determination that played out in two ways.