Table 3 shows the full results with unlikelihood training and length normalization.COP A H-Swag StoryCloze Winogrande WSC WiC FT 78 .0 PEFT methods we considered and ablate the losses. We use "Question:" and "Answer:" as Since T0 is unable to perform ICL on its own, we also compare to T5+LM, the next-step-prediction language model upon which T0 is based. Due to memory constraints and because of its improved performance, we use ensemble ICL for Table table 10 shows the T-Few ablation results. Per-dataset results of T-Few and the other top-5 methods on RAFT are shown in table 11. 18 # of Param COP A H-Swag StoryCloze WinograndeFull Model Fine-tuning 3B 81 .0

Table 3 shows the full results with unlikelihood training and length normalization.COP A H-Swag StoryCloze Winogrande WSC WiC FT 78 .0 PEFT methods we considered and ablate the losses. We use "Question:" and "Answer:" as Since T0 is unable to perform ICL on its own, we also compare to T5+LM, the next-step-prediction language model upon which T0 is based. Due to memory constraints and because of its improved performance, we use ensemble ICL for Table table 10 shows the T-Few ablation results. Per-dataset results of T-Few and the other top-5 methods on RAFT are shown in table 11. 18 # of Param COP A H-Swag StoryCloze WinograndeFull Model Fine-tuning 3B 81 .0

With the advent of deep learning over the last decade, a considerable amount of effort has gone into better understanding and enhancing Stochastic Gradient Descent so as to improve the performance and stability of artificial neural network training. Active research fields in this area include exploiting second order information of the loss landscape and improving the understanding of chaotic dynamics in optimization. This paper exploits the theoretical connection between the curvature of the loss landscape and chaotic dynamics in neural network training to propose a modified SGD ensuring non-chaotic training dynamics to study the importance thereof in NN training. Building on this, we present empirical evidence suggesting that the negative eigenspectrum - and thus directions of local chaos - cannot be removed from SGD without hurting training performance. Extending our empirical analysis to long-term chaos dynamics, we challenge the widespread understanding of convergence against a confined region in parameter space.

The Open Source Robotics Foundation (OSRF) is pleased to announce the creation of the Open Source Robotics Alliance (OSRA), a new initiative to strengthen the governance of our open-source robotics software projects and ensure the health of the Robot Operating System (ROS) Suite community for many years to come. The OSRA will use a mixed membership and meritocratic model, following other successful foundations for open-source projects, including The Linux Foundation and the Eclipse Foundation. The OSRA is extending an open invitation to all community stakeholders to participate in the technical oversight, direction, development, and support of the OSRF's open source projects – ROS, Gazebo, Open-RMF, and their infrastructure. Involvement across the robotics ecosystem is crucial to this initiative. The center of the OSRA will be the Technical Governance Committee (TGC), which will oversee the activities of various Project Management Committees, Technical Committees, Special Interest Groups, and Working Groups.

IROS was held in Detroit MI Oct 1-5 and not only showcased research but the latest commercialization in robotics, particularly robotics providers selling into robotics for research or as part of the hardware/software stack. The conference focuses on future directions in robotics, and the latest approaches, designs, and outcomes. It also provides an opportunity to network with the world's leading roboticists. Highlights included seeing Silicon Valley Robotics members; Foxglove, Hello Robot, Anyware Robotics and Tangram Vision, also Open Robotics and Intrinsic talking up ROS 2 and the upcoming ROSCon 23. Intrinsic sponsored a ROS/IROS meetup and Clearpath Robotics sponsored the Diversity Cocktails event.

Finally, Intrinsic (a spin-off of Google-X) has revealed the product they have been working with the help of the Open Source Robotics Corporation team (among others): Flowstate! Flowstate is a web-based software designed to simplify the creation of software applications for industrial robots. The application provides a user-friendly desktop environment where blocks can be combined to define the desired behavior of an industrial robot for specific tasks. Based on the official post and the keynote released on Monday, May 15, 2023 (available here), this is the information we have gathered so far. However, we currently lack a comprehensive understanding of how the software works, its complete feature set, and any potential limitations.

It's a new year, but the industry's struggles are showing no signs of abating. Big firms are as susceptible -- if not more so. This week, Alphabet joined the growing list of tech giants making staff cuts amid ongoing economic struggles. Following a wave of layoffs from the likes of Amazon, Meta and Salesforce, Alphabet has begun letting people go. The company's "Other Bets" division is the first to see impact.

I'm excited by the boost this newsletter has been getting in recent months and wanted to keep the light on while I was out. Three weeks is the longest break I've taken for work in…ever, really. Went to a bunch of museums (do yourself a favor and check out Edward Hopper at the Whitney and Morris Hirshfield at the American Folk Art Museum -- can't recommend them enough) and spent a few days in Aruba, of all places. Still not sure why flights were so cheap, but if you're ever looking for a nice place to stay on the island for $150 a night, let me know. Go make friends with a miniature donkey.

Few-shot in-context learning (ICL) enables pre-trained language models to perform a previously-unseen task without any gradient-based training by feeding a small number of training examples as part of the input. ICL incurs substantial computational, memory, and storage costs because it involves processing all of the training examples every time a prediction is made. Parameter-efficient fine-tuning (PEFT) (e.g. adapter modules, prompt tuning, sparse update methods, etc.) offers an alternative paradigm where a small set of parameters are trained to enable a model to perform the new task. In this paper, we rigorously compare few-shot ICL and PEFT and demonstrate that the latter offers better accuracy as well as dramatically lower computational costs. Along the way, we introduce a new PEFT method called (IA)$^3$ that scales activations by learned vectors, attaining stronger performance while only introducing a relatively tiny amount of new parameters. We also propose a simple recipe based on the T0 model called T-Few that can be applied to new tasks without task-specific tuning or modifications. We validate the effectiveness of T-Few on completely unseen tasks by applying it to the RAFT benchmark, attaining super-human performance for the first time and outperforming the state-of-the-art by 6% absolute. All of the code used in our experiments is publicly available.

Alphabet X-birthed Intrinsic made its big debut last September. The subsidiary looks to buck its parent company's somewhat spotty robotics record with a software-first approach. Specifically, the company is looking to make manufacturing robots more intelligent -- a concept that no doubt excites many in the space amid pandemic-fueled demand for automation. Today it made its second big piece of news: an acquisition. Intrinsic is acquiring fellow AI/robotic intelligence firm Vicarious.