Goto

Collaborating Authors

 star


Theoretical guarantees in KL for Diffusion Flow Matching

Neural Information Processing Systems

Flow Matching (FM) (also referred to as stochastic interpolants or rectified flows) stands out as a class of generative models that aims to bridge in finite time the target distribution $\nu^\star$ with an auxiliary distribution $\mu$ leveraging a fixed coupling $\pi$ and a bridge which can either be deterministic or stochastic. These two ingredients define a path measure which can then be approximated by learning the drift of its Markovian projection. The main contribution of this paper is to provide relatively mild assumption on $\nu^\star$, $\mu$ and $\pi$ to obtain non-asymptotics guarantees for Diffusion Flow Matching (DFM) models using as bridge the conditional distribution associated with the Brownian motion. More precisely, it establishes bounds on the Kullback-Leibler divergence between the target distribution and the one generated by such DFM models under moment conditions on the score of $\nu^\star$, $\mu$ and $\pi$, and a standard $\mathrm{L}^2$-drift-approximation error assumption.


Stars: Tera-Scale Graph Building for Clustering and Learning

Neural Information Processing Systems

A fundamental procedure in the analysis of massive datasets is the construction of similarity graphs. Such graphs play a key role for many downstream tasks, including clustering, classification, graph learning, and nearest neighbor search. For these tasks, it is critical to build graphs which are sparse yet still representative of the underlying data. The benefits of sparsity are twofold: firstly, constructing dense graphs is infeasible in practice for large datasets, and secondly, the runtime of downstream tasks is directly influenced by the sparsity of the similarity graph. In this work, we present Stars: a highly scalable method for building extremely sparse graphs via two-hop spanners, which are graphs where similar points are connected by a path of length at most two. Stars can construct two-hop spanners with significantly fewer similarity comparisons, which are a major bottleneck for learning based models where comparisons are expensive to evaluate. Theoretically, we demonstrate that Stars builds a graph in nearly-linear time, where approximate nearest neighbors are contained within two-hop neighborhoods. In practice, we have deployed Stars for multiple data sets allowing for graph building at the Tera-Scale, i.e., for graphs with hundreds of billions of nodes and tens of trillions of edges. We evaluate the performance of Stars for clustering and graph learning, and demonstrate 10~1000-fold improvements in pairwise similarity comparisons and significant running time speedups with negligible quality loss.


STaR: Bootstrapping Reasoning With Reasoning

Neural Information Processing Systems

Generating step-by-step chain-of-thought rationales improves language model performance on complex reasoning tasks like mathematics or commonsense question-answering. However, inducing language model rationale generation currently requires either constructing massive rationale datasets or sacrificing accuracy by using only few-shot inference. We propose a technique to iteratively leverage a small number of rationale examples and a large dataset without rationales, to bootstrap the ability to perform successively more complex reasoning. This technique, the Self-Taught Reasoner (STaR), relies on a simple loop: generate rationales to answer many questions, prompted with a few rationale examples; if the generated answers are wrong, try again to generate a rationale given the correct answer; fine-tune on all the rationales that ultimately yielded correct answers; repeat. We show that STaR significantly improves performance on multiple datasets compared to a model fine-tuned to directly predict final answers, and performs comparably to fine-tuning a 30$\times$ larger state-of-the-art language model on CommensenseQA. Thus, STaR lets a model improve itself by learning from its own generated reasoning.


Automatic doubly robust inference for linear functionals via calibrated debiased machine learning

van der Laan, Lars, Luedtke, Alex, Carone, Marco

arXiv.org Machine Learning

In causal inference, many estimands of interest can be expressed as a linear functional of the outcome regression function; this includes, for example, average causal effects of static, dynamic and stochastic interventions. For learning such estimands, in this work, we propose novel debiased machine learning estimators that are doubly robust asymptotically linear, thus providing not only doubly robust consistency but also facilitating doubly robust inference (e.g., confidence intervals and hypothesis tests). To do so, we first establish a key link between calibration, a machine learning technique typically used in prediction and classification tasks, and the conditions needed to achieve doubly robust asymptotic linearity. We then introduce calibrated debiased machine learning (C-DML), a unified framework for doubly robust inference, and propose a specific C-DML estimator that integrates cross-fitting, isotonic calibration, and debiased machine learning estimation. A C-DML estimator maintains asymptotic linearity when either the outcome regression or the Riesz representer of the linear functional is estimated sufficiently well, allowing the other to be estimated at arbitrarily slow rates or even inconsistently. We propose a simple bootstrap-assisted approach for constructing doubly robust confidence intervals. Our theoretical and empirical results support the use of C-DML to mitigate bias arising from the inconsistent or slow estimation of nuisance functions.


Robotics: Nicolas Mansard, coordinator of the MEMMO project, winner of the Stars of Europe - Actu IA

#artificialintelligence

Created in 2013, the Stars of Europe awards recognize the coordinators of European collaborative research projects. On December 6, Sylvie Retailleau, Minister of Higher Education and Research, presented trophies to twelve winners at a ceremony at the Quai Branly Museum. Among them, Nicolas Mansard, CNRS researcher in robotics at LAAS-CNRS, holder of the ANITI chair " Artificial and natural movement", rewarded for the coordination of the MEMMO (memory of motion) project. Funded by the Horizon 2020 program over a four-year period, MEMMO (Memory of Motion) is a collaborative project initiated in 2018 that brought together a consortium of 10 European partners for a budget of €4 million: LAAS-CNRS (France), IDIAP (Switzerland), University of Edinburgh (UK), Max Planck Institute (Germany), Oxford University (UK), Trento University (IT), PAL-Robotics (Spain), Wandercraft (France), Airbus (France), Costain (UK) and APAJH (France). "I would like to thank the people who helped me coordinate this project. It is a project put together by a consortium of young researchers. It was a great pride for me to be chosen to coordinate this project. "We wanted to prove that it was possible to generate complex motions for arbitrary robots with arms and legs interacting in a dynamic environment in real time.


Sustainability applications for artificial intelligence

#artificialintelligence

Artificial intelligence (AI) systems today are already transforming industries and becoming an indispensable part of our daily lives. Such systems, which leverage machines to process and analyse large amounts of data, have vastly changed how humans work and play, and are being used today in many sectors, from banking to energy to agriculture. But AI systems can be energy-intensive, and there is a pressing need for those working in the field of AI to address the potentially large environmental impacts. This is especially as demand for data and intelligent devices continues to proliferate. Singapore has committed itself to environmental sustainability, underlined by its ratification of the Paris Agreement and recent plans to reach net-zero by or around mid-century.


Novel AI chip design platform to give the semiconductor industry a boost in productivity and quality

#artificialintelligence

A*STAR researchers have developed an AI chip design platform that has the potential to transform the multibillion-dollar global integrated circuit (IC) design industry by accelerating design optimisation, reducing IC design turnaround time, and improving productivity significantly by twofold. The traditional way of designing integrated circuits is a complicated process that requires experienced engineers with domain knowledge. It is a manual, laborious process where designers rely on trial-and-error to achieve their design goals, slowing down productivity in the process. As technology advances, the complexity of chip design is ever increasing. To combat these challenges, A*STAR's Smart IC Design with Learning Enablement (SMILE) is an AI platform that uses machine learning to automate these complex processes.


Island In The Sand, Chapter XXII

#artificialintelligence

Star Black walked the short distance back to the living room where her band was once more gathering. "The lower story is not a story at all," the house intoned. I must assume that the defective Ninety-One machine was referring to the extended unit set into the base of the canyon face. That would place it lower than the house, which is accurate, but it is a thousand meters down, connected only by a small express lift." "So, there's something at the bottom of the canyon and there's a way to get down to it," Jameson said, with no doubt from his tone that he was eager to find out more, and immediately go exploring. "That's correct and accurate," the house stated. "So, where's the lift and when can we go down?" "The lift is rising, but it's of an old variety and will take some time to reach the dwelling." "Rising?" Star said, her voice piercing in tone, stopping all activity in the room. "Why would it be rising?" "There were no humans in the vicinity, as loosely defined when you pointedly used the word'outside' in your question," the house shot back, it's tone almost one of faint or vague petulance. "How could anyone get in the lift and use it if they aren't an administrator?" she asked, realizing when she got the words out that the answer to that question was right there among them. So much was happening at once Star couldn't figure out what to do first. They had members of Sly's band, who had somehow gotten to the bottom of the canyon and were riding up to arrive at the dwelling at any moment, as a possible and vital life or death issue. "The nodes and the lifts do not require the code to be provided by an administrator," the house said. "Unlike accessing the main complex entity and myself, they only require a spoken or written code." "Give me the device," Jameson said across the room to True, "and you better pray that we don't find any more on you when there's time to search you." The boy slunk against the wall, standing with his back against the stone that extended out from the fireplace, which still exuded heat even though the flames were gone. He held out his open hand, palm up, with a small black electronic device laying upon it. The device had a short antenna sticking out of it. "I don't have anymore, that's the last one," he murmured in the silence, as everyone waited to see what was going to happen next. "I didn't know who was going to win," True said, his voice soft but faintly rebellious. "If Star won I knew it would be okay, but if Sly won then what was I to do, die with everyone else?" Jameson stepped quickly forward and swept the device off True's exposed palm. He threw it down and smashed it into bits with the butt of his rifle before anything could be said or other action taken. Once done, Jameson raised his rifle up to point at True. "We're out of time, Jameson," Star said loudly, demanding the boy's full attention. We've got to get to the top of the lift before it gets up here. Where are the lift doors, house?"


jamesmullenbach.github.io by jamesmullenbach

@machinelearnbot

Over the break between semesters, I've spent a lot of time with family playing a popular board game called Codenames. If you haven't played, the gist is that one player from each team, the'spymaster', tries to get their team members to select their team's assigned words from a group of 25 while avoiding the other team's words and a game-ending'assassin' word, using one word clues. It's like the game show Password, except clues can apply to any number of words. It's a fun language based game and makes for an interesting testbed for simple experiments like the one I'm about to talk about. Naturally, I thought about how a computer might play this game.


Google artificial intelligence helps NASA discover solar system like ours

#artificialintelligence

WASHINGTON: NASA has used Google's artificial intelligence (AI) to discover a record-tying eighth exoplanet circling a Sun-like star 2,545 light-years from Earth, marking the first finding of an eight-planet solar system like ours. Kepler-90i - a sizzling hot, rocky planet that orbits its star once every 14.4 days - was found using machine learning from Google to scour data from NASA's planet-hunting Kepler Telescope. "The Kepler-90 star system is like a mini version of our solar system. You have small planets inside and big planets outside, but everything is scrunched in much closer," said Andrew Vanderburg, a NASA Sagan Postdoctoral Fellow and astronomer at the University of Texas at Austin. Machine learning is an approach to artificial intelligence in which computers "learn."