Goto

Collaborating Authors

 second type



Arbitrary-Length Generalization for Addition in a Tiny Transformer

arXiv.org Machine Learning

The Transformer architecture, as introduced by Vaswani et al. (2017), appears sufficiently robust to learn how to generalize addition, a fundamental operation (a+b=c) taught in elementary school. However, Nogueira et al. (2021) demonstrated that Transformers struggle to generalize this simple procedure effectively. Although some researchers have explored the use of both simplified and complex scratchpads to aid in training Transformers (Nye et al., 2021; Lee et al., 2024), they have not achieved generalization to numbers with arbitrary digit lengths. Recently, McLeish et al. (2024) argue that, by integrating an embedding for each digit that encodes its position relative to the start of the number, it is possible to train Transformers on 20-digit numbers and achieve approximately 99% accuracy on addition problems involving up to 100 digits. However, the authors do not study the accuracy for numbers exceeding 100 digits, which leaves an open question about the scalability of this approach to even larger numbers. This gap presents a significant opportunity for future research to explore the limits of Transformer generalization in arithmetic operations. I would like to thank Fernanda Cristiane de Oliveira for helping me to make parts of this work clearer.


Screening blood samples for COVID-19 using artificial intelligence

#artificialintelligence

A promising new study published in the preprint online journal medRxiv in April 2020 shows the potential of artificial intelligence (AI) for developing a patient classifier that can separate patients likely to be negative for COVID-19 from among a pool of suspected patients visiting an emergency room (ER). This would reduce the rate of spread significantly, by making it possible to immediately separate the patients most likely to be positive from others with similar symptoms of respiratory illness. It would protect both patients and healthcare providers from severe acute respiratory syndrome coronavirus 2 (SARS-CoV-2) infection. The novel coronavirus (SARS-CoV-2) has spread across the world at unprecedented speed, placing a heavy and, in some cases, practically unsustainable, load on healthcare systems. Despite government aid, many healthcare providers find themselves requiring many more beds, intensive care units (ICU), and Personal Protective Equipment (PPE) than can be provided.


Japan considers permanent residency for skilled blue-collar workers

The Japan Times

The government is considering allowing blue-collar foreign workers with certain skills to live permanently in the country with their families, as Japan struggles with a serious labor shortage amid a declining population, sources said Thursday. In what would represent a turning point for the country's immigration policy, which more or less sanctions only the entry of highly skilled professionals, the government of Prime Minister Shinzo Abe is seeking to open the door to blue-collar foreign workers by introducing a new system next April. The government is studying two new types of residence status for foreign workers, who must have Japanese language proficiency as well as knowledge and experience in one of more than 10 fields. Those sectors include nursing care, agriculture and construction, according to the sources. Those who qualify for the first type of residence status will be issued a visa valid up to five years but will not be allowed to bring their family members. Those who qualify for the second type -- namely, highly skilled laborers -- will be offered permanent residence status and allowed to bring their family members to Japan.


AI can create images based on pictures you are looking at

#artificialintelligence

Japanese scientists have create a creepy machine that can peer into your mind's eye with incredible accuracy. The AI studies electrical signals in the brain to work out exactly what images someone is looking at, and even thinking about. The technology opens the door to strange future scenarios, such as those portrayed in the series'Black Mirror', where anyone can record and playback their memories. Scientists in Japan have developed an AI that can decode patterns in the brain to recreate what a person is seeing or imagining. The breakthrough relies on neural networks, which try to simulate the way the brain works in order to learn.


AI can create images based on pictures you are looking at

Daily Mail - Science & tech

Japanese scientists have create a creepy machine that can peer into your mind's eye with incredible accuracy. The AI can study electrical signals in the brain to work out exactly what images someone is looking at, and even thinking about. The technique could theoretically be used to create footage of daydreams and to help patients in permanent vegetative states to communicate with their loved ones. Scientists in Japan have developed an AI that can decode patterns in the brain to recreate what a person is seeing or imagining. The breakthrough relies on neural networks, which try to simulate the way the brain works in order to learn.


Who Cares About Evidence?

@machinelearnbot

A close contact of mine who studies the sociology of science recently commented that, when the data do not support the researcher's hypothesis, all too often it is the data that are rejected. In the political realm, logic and evidence are routinely subordinated to belief and ideology or, in the more elegant words of Mary Wollstonecraft: "But what a weak barrier is truth when it stands in the way of an hypothesis!" Science, by the usual definitions, does not appear to come naturally to human beings and most of us struggle with it in school. While we've put men on the moon, it took us tens of thousands of years to accomplish this. Moreover, we want the world to be like…the way we want it to be. This is understandable, especially when science is shrouded in mumbo-jumbo and math, which most of us also hate.


Got It Pro uses machine learning to find a human solver for your techie problem

#artificialintelligence

Can AI be used to intelligently rank expertise? Serial entrepreneur Peter Relan thinks it can, and has today launched what he's billing as a "knowledge-as-service" (KaaS) web platform to connect (human) experts with professionals seeking the answer to specific tech tool-related problems. In the first instance the target users are professionals wrangling Excel and Google Sheets, with the platform aiming to link them (via their spreadsheet-related problem) to a relevant expert -- with the help session taking place via the medium of web-based text chat. But the wider ambition is to be able to use its expertise-ranking algorithms to intelligently assess all sorts of techie knowledge to be able to connect relevant experts with different professional "knowledge based problems". Relan likens the concept to how Google's pagerank dynamically orders online data.