Despite the empirical success of Low-Rank Adaptation (LoRA) in fine-tuning pre-trained models, there is little theoretical understanding of how first-order methods with carefully crafted initialization adapt models to new tasks. In this work, we take the first step towards bridging this gap by theoretically analyzing the learning dynamics of LoRA for matrix factorization (MF) under gradient flow (GF), emphasizing the crucial role of initialization. For small initialization, we theoretically show that GF converges to a neighborhood of the optimal solution, with smaller initialization leading to lower final error. Our analysis shows that the final error is affected by the misalignment between the singular spaces of the pre-trained model and the target matrix, and reducing the initialization scale improves alignment. To address this misalignment, we propose a spectral initialization for LoRA in MF and theoretically prove that GF with small spectral initialization converges to the fine-tuning task with arbitrary precision. Numerical experiments from MF and image classification validate our findings.

It is widely agreed that 2023 was a stellar year for video games. The Legend of Zelda: Tears of the Kingdom, Baldur's Gate 3, Alan Wake 2, Marvel's Spider-Man 2 … barely a week passed without some blockbuster hit or independent gem. But beneath these accolades there is a sadder, more worrying story: it was also a year of widespread industry redundancies, and the trend is continuing into the opening weeks of 2024. Microsoft laid off 1,900 Activision Blizzard staff after its 69bn purchase of the company. Publisher Embracer Group let at least 900 staff go across its many studios, as well as closing veteran UK developer Free Radical Design. Epic Games, the creator of Fortnite, one of the most successful titles of the decade, laid off 830 employees; Electronic Arts shed 6% of its workforce, amounting to approximately 780 jobs.

This work presents a novel deep-learning-based pipeline for the inverse problem of image deblurring, leveraging augmentation and pre-training with synthetic data. Our results build on our winning submission to the recent Helsinki Deblur Challenge 2021, whose goal was to explore the limits of state-of-the-art deblurring algorithms in a real-world data setting. The task of the challenge was to deblur out-of-focus images of random text, thereby in a downstream task, maximizing an optical-character-recognition-based score function. A key step of our solution is the data-driven estimation of the physical forward model describing the blur process. This enables a stream of synthetic data, generating pairs of ground-truth and blurry images on-the-fly, which is used for an extensive augmentation of the small amount of challenge data provided. The actual deblurring pipeline consists of an approximate inversion of the radial lens distortion (determined by the estimated forward model) and a U-Net architecture, which is trained end-to-end. Our algorithm was the only one passing the hardest challenge level, achieving over $70\%$ character recognition accuracy. Our findings are well in line with the paradigm of data-centric machine learning, and we demonstrate its effectiveness in the context of inverse problems. Apart from a detailed presentation of our methodology, we also analyze the importance of several design choices in a series of ablation studies. The code of our challenge submission is available under https://github.com/theophil-trippe/HDC_TUBerlin_version_1.

As artificial intelligence (AI) increasingly affects the performance of products consumers use every day, discussions about who owns the AI risk is likely to become more intricate. For companies making these products, it's important to know the risks involved, be they reputational, business-related or consumer risks. "Sometimes we're having to talk to our clients about the AI they use -- where they could be sued, for example, by a third party who interacts with it…or it could end up being a first-party loss because they're using it for themselves," said Kelly MacDonald, Aon's regional sales director and senior vice president, Commercial Risk Solutions. AI could fall under a host of liabilities, programs or policies, but many risks associated with AI are only partly insurable, says MacDonald. "It probably isn't going to all be picked up under one specific placement, it will be a combination," adds Katharine Hall, Aon's senior vice president and cyber practice leader of commercial risk solutions. "How does your general liability placement work with your tech E&O, work with your cyber?"

The founder of Google Brain research lab, Andrew Ng, has raised $57 million in Series A funding for his startup -- Landing AI, led by McRock Capital, the first investment firm focused exclusively on the Industrial IoT. AI pioneer Andrew Ng's startup provides tools that make building and deploying AI systems in manufacturing faster and easier. "I'm thrilled LANDING AI has closed a $57M Series A for our Data-centric MLOps platform for computer vision. Industrial AI needs a different recipe than Internet companies. It's time to make cutting-edge AI fast and easy for anyone to use," said Andrew Ng, Founder & CEO of Landing AI in a post.

A Boston nonprofit is putting the finishing touches on the world's first affordable "tablet" for the blind, an Android-based device that is part of an innovative campaign to turn around a little-known literacy crisis among the visually impaired. "If only 12 percent of children could read today, it'd be the biggest discussion in the world," said Brian A. MacDonald, the president of National Braille Press, located in the Fenway. "But because the blind are such a small population, it's not very well known." Literacy among the blind has plummeted in the past four decades to that astonishing number -- 12 percent -- due in part to the lack of qualified Braille instructors in regular classrooms, the flipside of the mainstreaming movement. MacDonald and his team of techies hope their Braille tablet for the blind -- dubbed the B2G-20 -- will fill this void, eventually leveling the playing field for a population increasingly mired in unemployment and poverty.

Landing AI, which provides tools that make building and deploying AI systems in manufacturing faster and easier than ever, announced Series A funding of $57 million led by McRock Capital, the first investment firm focused exclusively on the Industrial IoT. In addition, New York-based global private equity and venture capital firm Insight Partners, Taiwania Capital, Canada Pension Plan Investment Board (CPP Investments), Intel Capital, Samsung Catalyst Fund, Far Eastern Group's DRIVE Catalyst, Walsin Lihwa, and AI Fund all participated in the round. Landing AI, led by artificial intelligence visionary, Andrew Ng, developed LandingLens, a fast, easy to use enterprise MLOps platform. It applies AI and deep learning to help manufacturers solve visual inspection problems, find product defects more reliably, and generate business value. Landing AI sees the next era of AI as one in which all companies access the benefits of AI--not just consumer internet companies like Google and Facebook--but legacy industries such as manufacturing, healthcare, and agriculture.

PALO ALTO, Calif., Nov. 8, 2021 -- Landing AI, which provides tools that make building and deploying AI systems in manufacturing faster and easier than ever, today announced Series A funding of $57 million led by McRock Capital, the first investment firm focused exclusively on the Industrial IoT. In addition, New York-based global private equity and venture capital firm Insight Partners, Taiwania Capital, Canada Pension Plan Investment Board (CPP Investments), Intel Capital, Samsung Catalyst Fund, Far Eastern Group's DRIVE Catalyst, Walsin Lihwa, and AI Fund all participated in the round. "Landing AI will unleash the power of the Industrial IoT one company, one factory, and one manufacturing line at a time." Landing AI, led by artificial intelligence visionary, Andrew Ng, developed LandingLens, a fast, easy-to-use enterprise MLOps platform. It applies AI and deep learning to help manufacturers solve visual inspection problems, find product defects more reliably, and generate business value.

For a d-ary Boolean function Φ: {0, 1}d → {0, 1} and an assignment to its variables x = (x1, x2, . . . , xd) we consider the problem of finding those subsets of the variables that are sufficient to determine the function value with a given probability δ. This is motivated by the task of interpreting predictions of binary classifiers described as Boolean circuits, which can be seen as special cases of neural networks. We show that the problem of deciding whether such subsets of relevant variables of limited size k ≤ d exist is complete for the complexity class NPPP and thus, generally, unfeasible to solve. We then introduce a variant, in which it suffices to check whether a subset determines the function value with probability at least δ or at most δ − γ for 0 < γ < δ. This promise of a probability gap reduces the complexity to the class NPBPP. Finally, we show that finding the minimal set of relevant variables cannot be reasonably approximated, i.e. with an approximation factor d1−α for α > 0, by a polynomial time algorithm unless P = NP. This holds even with the promise of a probability gap.

George Orwell's Animal Farm: A Fairy Story is a well-loved parable set on a farm in England, where rebellious animals stand in as critique for the corruption and downfall of the Communist Revolution in Russia. It is also a story that has often been made to serve different meanings for different groups of people. In 1946, Orwell received a letter (documented in the book George Orwell: A Life in Letters) from a colleague, Dwight Macdonald, who reported that anti-Stalinists in his circle "claimed that the parable of Animal Farm meant that revolution always ended badly for the underdog, 'hence to hell with it and hail the status quo.'" In his response, Orwell made sure to clarify his thoughts, writing: "If people think I am defending the status quo, that is, I think, because they have grown pessimistic and assume that there is no alternative except dictatorship or laissez-faire capitalism." He emphasized that if there was one lesson behind his parable, it was "you can't have a revolution unless you make it for yourself; there is no such thing as a benevolent dictatorship."