Goto

Collaborating Authors

 adams


Seedless Blackberries and Cherries That Grow on Bushes Vie to Be the Future of Food

WIRED

Startups and Big Ag are using Crispr gene editing to create crops that taste better and grow on a hotter planet. But will they find a market? If you wanted to create better blackberries, you could try to breed a variety without the seeds that get stuck in your teeth. Or one without the thorns that make the berries hard to pick. Or you could aim for more environmentally efficient plants that produce more berries per acre with less water and fewer chemicals.


Breaking Down the Bloody Finale of Cape Fear

TIME - Tech

Follow this section to personalize your feed and get instant alerts. Follow Go to your personalized feed WHY FOLLOW? Smart Alerts: Get notified about major news as it happens. Follow this tag to personalize your feed and get instant alerts. Follow Go to your personalized feed WHY FOLLOW? Smart Alerts: Get notified about major news as it happens. Follow this author to personalize your feed and get instant alerts. Follow Go to your personalized feed WHY FOLLOW? Smart Alerts: Get notified about major news as it happens. Follow this author to personalize your feed and get instant alerts.


The New (And Slightly Smelly) Center of the AI Boom

The Atlantic - Technology

San Francisco's brightest minds are stuffing themselves into hacker houses. The living room of the Accler8 hacker house in San Francisco, where the author stayed for a week. O n a Friday in April, I hopped into an Uber to a fish market in San Francisco with a couple of tech founders on a mission to buy lobsters. Not for dinner, but for science: The duo dreamed of one day altering human consciousness, but they would start by toying around with some crustaceans. They intended to perform neurosurgery on the lobsters in the hopes of controlling them with an AI bot. Leading the way was Elliot Roth, a bearded 32-year-old wearing a black T-shirt with Longevity printed across the chest and a silver chain with a double-helix pendant. To push the boundaries of the five senses, Roth has implanted a magnet in his left ring finger.


Democracy and the Declaration of Independence

TIME - Tech

Follow this section to personalize your feed and get instant alerts. Follow Go to your personalized feed WHY FOLLOW? Smart Alerts: Get notified about major news as it happens. Follow this tag to personalize your feed and get instant alerts. Follow Go to your personalized feed WHY FOLLOW?



Your guide to the California state controller race: Democrat Malia Cohen faces challengers

Los Angeles Times

Things to Do in L.A. From left, Meghann Adams, Malia Cohen and Herb Morgan are running for state controller in the California primary election. California voters will choose who oversees the state's finances as incumbent Malia Cohen faces Republican Herb Morgan, a finance executive, and Meghann Adams, a school bus driver and Peace and Freedom Party member. Morgan proposes using blockchain and AI technology for real-time spending transparency, while Adams advocates corporate audits and redirecting billions toward education, housing and healthcare for working-class Californians. Cohen improved financial report timeliness but fell short on promised audits of homelessness programs, the DMV and Employment Development Department. The state's fiscal watchdog oversees the intake and outtake of public funds and audits departments across the state.


Appendix: On the Overlooked Pitfalls of Weight Decay and How to Mitigate Them

Neural Information Processing Systems

Suppose we have a non-zero solution ฮธ which is a stationary point of f(ฮธ,t) at t-th step and SGD finds ฮธt = ฮธ at t-th step. Theorem 2.2 of Shapiro and Wardi [9] told us that the learning rate should be small enough for convergence. Obviously, we have ฮท < in practice. As ฮทt = ฮทt+1 does not hold, SGD cannot converging to any non-zero stationary point. The proof is now complete.




RedPajama: an Open Dataset for Training Large Language Models

Neural Information Processing Systems

Large language models are increasingly becoming a cornerstone technology in artificial intelligence, the sciences, and society as a whole, yet the optimal strategies for dataset composition and filtering remain largely elusive. Many of the top-performing models lack transparency in their dataset curation and model development processes, posing an obstacle to the development of fully open language models. In this paper, we identify three core data-related challenges that must be addressed to advance open-source language models. These include (1) transparency in model development, including the data curation process, (2) access to large quantities of high-quality data, and (3) availability of artifacts and metadata for dataset curation and analysis. To address these challenges, we release RedPajama-V1, an open reproduction of the LLaMA training dataset. In addition, we release RedPajama-V2, a massive web-only dataset consisting of raw, unfiltered text data together with quality signals and metadata.Together, the RedPajama datasets comprise over 100 trillion tokens spanning multiple domains and with their quality signals facilitate the filtering of data, aiming to inspire the development of numerous new datasets. To date, these datasets have already been used in the training of strong language models used in production, such as Snowflake Arctic, Salesforce's XGen and AI2's OLMo. To provide insight into the quality of RedPajama, we present a series of analyses and ablation studies with decoder-only language models with up to 1.6B parameters. Our findings demonstrate how quality signals for web data can be effectively leveraged to curate high-quality subsets of the dataset, underscoring the potential of RedPajama to advance the development of transparent and high-performing language models at scale.