kanter
The Story of British Billionaire Mike Lynch's Tragic Boat Sinking
The last night of tech mogul Mike Lynch's life has become fodder for conspiracy theories. For the first time, the whole story can be told. In the predawn hours of August 19, 2024, bolts of lightning began to fork through the purple-black clouds above the Mediterranean. From the rail of a 184-foot vessel, a 22-year-old named Matthew Griffiths took out his phone to record a video. The British deckhand was just a week and a half into his first official yacht job, and he wasn't on just any boat. The yacht, the $40 million, was a star of the superyacht world, considered to be a feat of minimal design and precision engineering. As thunder rolled toward the anchored vessel, Griffiths set the video to AC/DC's "Thunderstruck" and posted it to Instagram. In the video, the's aluminum mast, one of the tallest in the world, is briefly visible against the roiling sky. Below deck, the yacht's owner, Michael Lynch, had every reason to be sleeping soundly. The boat trip had been organized as a celebration. Months earlier, Lynch had walked out of a San Francisco federal courthouse a free man, acquitted of all charges in one of the largest fraud cases in Silicon Valley history. Lynch had built his fortune on understanding probability, on turning the unlikely into the possible. He had named his yacht in honor of the statistical theorem that made him a billionaire, after the sale, in 2011, of his company Autonomy. The British tech giant sold software that could find meaningful signals amid the flood of unstructured data in emails, videos, and phone calls, but it would be better known as the company that allegedly defrauded, and nearly destroyed, Hewlett-Packard. The cabins aboard the contained the people who had stood by Lynch through his 13-year-long legal ordeal. Beside him in the master suite was his wife of 22 years, Angela Bacares, a former vice president in the investment division of Deutsche Bank who had caught his eye while working an Autonomy deal. Other cabins housed the Clifford Chance attorneys who had orchestrated Lynch's legal victory, as well as longtime colleagues, their partners, and a 1-year-old baby, all supported by 10 crew members. Also onboard was Lynch's younger daughter, Hannah, 18, who was about to begin her studies at Oxford.
Learning Mechanism Underlying NLP Pre-Training and Fine-Tuning
Tzach, Yarden, Gross, Ronit D., Koresh, Ella, Rosner, Shalom, Shpringer, Or, Halevi, Tal, Kanter, Ido
Natural language processing (NLP) enables the understanding and generation of meaningful human language, typically using a pre-trained complex architecture on a large dataset to learn the language and next fine-tune its weights to implement a specific task. Twofold goals are examined; to understand the mechanism underlying successful pre-training and to determine the interplay between the pre-training accuracy and the fine-tuning of classification tasks. The following main results were obtained; the accuracy per token (APT) increased with its appearance frequency in the dataset, and its average over all tokens served as an order parameter to quantify pre-training success, which increased along the transformer blocks. Pre-training broke the symmetry among tokens and grouped them into finite, small, strong match token clusters, as inferred from the presented token confusion matrix. This feature was sharpened along the transformer blocks toward the output layer, enhancing its performance considerably compared with that of the embedding layer. Consequently, higher-order language structures were generated by pre-training, even though the learning cost function was directed solely at identifying a single token. These pre-training findings were reflected by the improved fine-tuning accuracy along the transformer blocks. Additionally, the output label prediction confidence was found to be independent of the average input APT, as the input meaning was preserved since the tokens are replaced primarily by strong match tokens. Finally, although pre-training is commonly absent in image classification tasks, its underlying mechanism is similar to that used in fine-tuning NLP classification tasks, hinting at its universality. The results were based on the BERT-6 architecture pre-trained on the Wikipedia dataset and fine-tuned on the FewRel and DBpedia classification tasks.
Role of Delay in Brain Dynamics
Meir, Yuval, Tevet, Ofek, Tzach, Yarden, Hodassman, Shiri, Kanter, Ido
Significant variations of delays among connecting neurons cause an inevitable disadvantage of asynchronous brain dynamics compared to synchronous deep learning. However, this study demonstrates that this disadvantage can be converted into a computational advantage using a network with a single output and M multiple delays between successive layers, thereby generating a polynomial time-series outputs with M. The proposed role of delay in brain dynamics (RoDiB) model, is capable of learning increasing number of classified labels using a fixed architecture, and overcomes the inflexibility of the brain to update the learning architecture using additional neurons and connections. Moreover, the achievable accuracies of the RoDiB system are comparable with those of its counterpart tunable single delay architectures with M outputs. Further, the accuracies are significantly enhanced when the number of output labels exceeds its fully connected input size. The results are mainly obtained using simulations of VGG-6 on CIFAR datasets and also include multiple label inputs. However, currently only a small fraction of the abundant number of RoDiB outputs is utilized, thereby suggesting its potential for advanced computational power yet to be discovered.
Microsoft, OpenAI and Nvidia investigated over possible breach of antitrust laws
Microsoft, OpenAI and Nvidia face increased antitrust scrutiny of their roles in the artificial intelligence industry after a report that US regulators have reached an agreement on investigating the companies. The New York Times reported that the US justice department and the Federal Trade Commission (FTC) have reached an agreement on investigations into the main protagonists in the AI market. The deal is expected to be completed in the coming days, according to the report. The justice department will lead on investigating whether Nvidia, the leading maker of chips that train and operate AI systems, has broken antitrust laws that oversee fair competition in business and aim to prevent monopolies, said the NYT on Wednesday. The Wall Street Journal also reported on Thursday that the FTC is investigating whether Microsoft structured a recent deal with startup Inflection AI to avoid an antitrust inquiry.
In latest benchmark test of AI, it's mostly Nvidia competing against Nvidia
For lack of rich competition, some of Nvidia's most significant results in the latest MLPerf were against itself, comparing its newest GPU, H100 "Hopper," to its existing product, the A100. Although chip giant Nvidia tends to cast a long shadow over the world of artificial intelligence, its ability to simply drive competition out of the market may be increasing, if the latest benchmark test results are any indication. Did you miss out on Black Friday 2022? No problem: Cyber Monday deals are here, with internet retailers offering their lowest prices of the year. ZDNET is surfacing the latest and best sales online in real time for you to check out now.
Nvidia and Intel show machine learning performance gains on latest MLPerf Training 2.1 results
Join us on November 9 to learn how to successfully innovate and achieve efficiency by upskilling and scaling citizen developers at the Low-Code/No-Code Summit. MLCommons is out today with its latest set of machine learning (ML) MLPerf benchmarks, once again showing how hardware and software for artificial intelligence (AI) are getting faster. MLCommons is a vendor-neutral organization that aims to provide standardized testing and benchmarks to help evaluate the state of ML software and hardware. Under the MLPerf testing name, MLCommons collects different ML benchmarks multiple times throughout the year. In September, the MLPerf Inference results were released, showing gains in how different technologies have improved inference performance.
The brain learns completely differently than we've assumed since the 20th century
The brain is a complex network containing billions of neurons, where each of these neurons communicates simultaneously with thousands of other via their synapses (links). However, the neuron actually collects its many synaptic incoming signals through several extremely long ramified "arms" only, called dendritic trees. In 1949 Donald Hebb's pioneering work suggested that learning occurs in the brain by modifying the strength of the synapses, whereas neurons function as the computational elements in the brain. This has remained the common assumption until today. Using new theoretical results and experiments on neuronal cultures, a group of scientists, led by Prof. Ido Kanter, of the Department of Physics and the Gonda (Goldschmied) Multidisciplinary Brain Research Center at Bar-Ilan University, has demonstrated that the central assumption for nearly 70 years that learning occurs only in the synapses is mistaken.
More public data key to democratizing ML, says MLCommons
Unless you're an English speaker, and one with as neutral an American accent as possible, you've probably butted heads with a digital assistant that couldn't understand you. With any luck, a couple of open-source datasets from MLCommons could help future systems grok your voice. The two datasets, which were made generally available in December, are the People's Speech Dataset (PSD), a 30,000-hour database of spontaneous English speech; and the Multilingual Spoken Words Corpus (MSWC), a dataset of some 340,000 keywords in 50 languages. By making both datasets publicly available under CC-BY and CC-BY-SA licenses, MLCommons hopes to democratize machine learning โ that is to say, make it available to everyone โ and help push the industry toward data-centric AI. David Kanter, executive director and founder of MLCommons, told Nvidia in a podcast this week that he sees data-centric AI as a conceptual pivot from "which model is the most accurate," to "what can we do with data to improve model accuracy."
Brain-Inspired Hardware Could Boost AI's Ability to Learn
Artificial intelligence (AI) could soon get a boost from a new type of computer chips inspired by the human brain. Researchers at Purdue University have built a new piece of hardware that can be reprogrammed on demand through electrical pulses. The team claims that this adaptability would allow the device to take on all of the necessary functions to build a brain-inspired computer. It's part of an ongoing effort to build AI systems that can learn continuously. "When AI systems learn continually in the environment, they can adapt to a world that changes over time," Stevens Institute of Technology AI expert Jordan Suchow told Lifewire in an email interview.
MLCommons debuts with public 86,000-hour speech data set for AI researchers โ TechCrunch
If you want to make a machine learning system, you need data for it, but that data isn't always easy to come by. MLCommons aims to unite disparate companies and organizations in the creation of large public databases for AI training, so that researchers around the world can work together at higher levels, and in doing so advance the nascent field as a whole. Its first effort, the People's Speech Dataset, is many times the size of others like it, and aims to be more diverse as well. MLCommons is a new nonprofit related to MLPerf, which has collected input from dozens of companies and academic institutions to create industry-standard benchmarks for machine learning performance. The endeavor has met with success, but in the process the team encountered a paucity of open data sets that everyone could use.