Large Language Model
Plans for Future Maintenance of Gym · Issue #2259 · openai/gym
Fixes to code style (use the same style tests as either PettingZoo does or SB3 does and that to CI tests once they're properly functioning) (Thanks @cclauss!) Removal of old and entirely unused code Bug fixes (they'll actually be merged now!) Useful non-breaking extensions to or entirely new action/observation spaces Built in API compliance testing (Similar to what PettingZoo has for environments and what SB3 added for Gym environments) Nonbreaking and useful additions of environment arguments, similar to what most third party Gym environments now have or what PettingZoo environments generally have by default (e.g. Lycon is a Python library that's just took the C image resizing logic from OpenCV and put it in it's own repo. This makes it run slightly faster, and more importantly it gets rid of all the horrifying installation issues associated with OpenCV (and most RL libraries only depended on OpenCV for this functionality). However, Lycon is no longer maintained and does not generate wheels with the C already compiled (though Ben Black added the logic for this- ethereon/lycon#25). Dealing with all flavors of MuJuCo problems (I am objectively not qualified for this) Create a new, in depth, documentation website.
DeepMind releases database with AI predictions for every human protein shape
DeepMind released a free, open-source, big-deal database last week containing AI predictions for the shapes of every protein in the human body. Not only is it the most complete picture of the human proteome (full set of proteins) to date, according to the London-based AI lab--it's also "doubling humanity's accumulated knowledge of high-accuracy human protein structures." Deepening our understanding of protein structures can lead to major leaps forward in understanding diseases, as well as in drug and vaccine development. That could help allay anything from neglected diseases to the next pandemic. Recap: In December 2020, AlphaFold, DeepMind's neural network, made a breakthrough in protein folding--a biological mystery that's puzzled scientists for 50 years.
DeepMind will soon publish the structure of every protein known to science
DeepMind, a sister company of Google, is giving the world access to a massive protein structure database -- a gift that has the potential to revolutionize scientific research. "This will be one of the most important datasets since the mapping of the Human Genome," Ewan Birney, deputy director general of the European Molecular Biology Laboratory, which partnered with DeepMind on the database, said in a press release. Protein structure: Proteins are molecules that are hugely important to the functioning of living organisms, including humans -- practically everything we're made of and everything our cells do is determined by our proteins. "It's the most significant contribution AI has made to advancing scientific knowledge to date." Every protein is made up of a long string of hundreds or even thousands of chemical compounds called amino acids, and the way that ribbon folds on itself determines the protein's function.
Improving Social Meaning Detection with Pragmatic Masking and Surrogate Fine-Tuning
Zhang, Chiyu, Abdul-Mageed, Muhammad, Elmadany, AbdelRahim, Nagoudi, El Moatez Billah
Masked language models (MLMs) are pretrained with a denoising objective that, while useful, is in a mismatch with the objective of downstream fine-tuning. We propose pragmatic masking and surrogate fine-tuning as two strategies that exploit social cues to drive pre-trained representations toward a broad set of concepts useful for a wide class of social meaning tasks. To test our methods, we introduce a new benchmark of 15 different Twitter datasets for social meaning detection. Our methods achieve 2.34% F1 over a competitive baseline, while outperforming other transfer learning methods such as multi-task learning and domain-specific language models pretrained on large datasets. With only 5% of training data (severely few-shot), our methods enable an impressive 68.74% average F1, and we observe promising results in a zero-shot setting involving six datasets from three different languages.
An endlessly changing playground teaches AIs how to multitask
They advance to more complex multiplayer games like hide and seek or capture the flag, where teams compete to be the first to find and grab their opponent's flag. The playground manager has no specific goal but aims to improve the general capability of the players over time. AIs like DeepMind's AlphaZero have beaten the world's best human players at chess and Go. But they can only learn one game at a time. As DeepMind cofounder Shane Legg put it when I spoke to him last year, it's like having to swap out your chess brain for your Go brain each time you want to switch games.
SEO.co Announces Release Artificial Intelligence Copywriting Tool for SEO
SEO.co, a search engine optimization (SEO) agency, in collaboration with DEV.co, a custom software development company, has launched AI.DEV.co, a tool that makes it easier for businesses and individuals to generate web copy. Businesses of all sizes and in all industries face a similar dilemma online: getting attention and standing out. Most brands use a combination of different marketing and advertising strategies to get this attention and differentiate themselves from their competitors. For a campaign to succeed, it needs a set of compelling, unique copy – persuasive writing that concisely makes a point and motivates a web user to take an action (such as clicking a link, buying a product, or watching a video). Generating copy is challenging for several reasons, even if one is experienced in the field.
DeepMind AI Says Will Release Structure of Every Protein Known
DeepMind, an artificial intelligence (AI) subsidiary of Google parent Alphabet, said it has been successful in predicting the shape of nearly every protein in the human body as well as thousands of other proteins found in 20 additional organisms that scientists rely on for their research, including yeast, fruit flies, and mice. This breakthrough is likely to assist researchers to understand human diseases better and find new drugs to treat or cure them. Some scientists have compared the DeepMind project to the international effort to map every human gene. DeepMind said in a blog post it is releasing the database for free. To set up and run the database, it has partnered with the European Molecular Biology Laboratory.
Huge protein structure database could transform biology
Earlier this month, two groups unveiled the culmination of years of work by computer scientists, biologists, and physicists: advanced modeling programs that can predict the precise 3D atomic structures of proteins. Last week, the biggest payoff of that work arrived. One team used its newly minted artificial intelligence (AI) programs to solve the structures of 350,000 proteins from humans and 20 model organisms, such as Escherichia coli bacteria, yeast, and fruit flies, all mainstays of biological research. In the coming months, the group says it plans to expand its efforts to all cataloged proteins—some 100 million molecules. “It's pretty overwhelming,” says John Moult, a protein folding expert at the University of Maryland, Shady Grove, who runs a biennial competition called the Critical Assessment of protein Structure Prediction (CASP). Moult says structural biologists have dreamed for decades that accurate computer models would one day augment slow, painstaking experimental methods, such as x-ray crystallography, that map protein shapes with extreme precision. “I never thought the dream would come true,” Moult says. The computer model, called AlphaFold, is the work of researchers at DeepMind, a U.K. AI company owned by Alphabet, the parent company of Google. In fall of 2020, AlphaFold swept the CASP competition, tallying a median accuracy score of 92.4 out of 100 for its predicted structures, well ahead of the next closest competitor ( Science , 4 December 2020, p. [1144][1]). But because DeepMind researchers didn't reveal AlphaFold's underlying computer code, other teams were left frustrated, unable to build on the progress. That began to change this month ( Science , 16 July, p. [262][2]). On 15 July, researchers led by Minkyung Baek and David Baker at the University of Washington, Seattle, reported online in Science that they had created a competing system: a highly accurate protein structure prediction program called RoseTTAFold, which they released publicly. The same day, Nature rushed out details of AlphaFold in a paper by DeepMind researchers led by Demis Hassabis and John Jumper. Both programs use AI to spot folding patterns in vast databases of solved protein structures. The programs compute the most likely structure of unknown proteins by applying those patterns and also considering basic physical and biological rules governing how neighboring amino acids in a protein interact. In their paper, Baek and Baker used RoseTTAFold to create a structure database of hundreds of G-protein coupled receptors, a class of common drug targets. Now, DeepMind researchers report in Nature that they have amassed 350,000 predicted structures—more than twice as many as experimenters have solved in many decades of work. AlphaFold's structures for which the researchers say they have high confidence cover nearly 44% of all human proteins. AlphaFold determined that many of the remaining human proteins were “disordered,” meaning their shape doesn't adopt a single structure. Such disordered proteins may ultimately adopt a structure when they bind to a protein partner, Baker says. They may also naturally adopt multiple conformations, says David Agard, a structural biologist at the University of California, San Francisco. A database of DeepMind's new protein predictions, assembled with collaborators at the European Molecular Biology Laboratory (EMBL), is freely accessible online. “It's fantastic they have made this available,” Baker says. “It will really increase the pace of research.” Because the 3D structure of a protein largely dictates its function, the DeepMind library is apt to help biologists sort out how thousands of unknown proteins do their jobs. “We at EMBL believe this will be transformative to understanding how life works,” says the lab's director general, Edith Heard. “This will be one of the most important data sets since the mapping of the human genome,” adds Ewan Birney, director of EMBL's European Bioinformatics Institute. DeepMind collaborators say that by making it possible to quickly assess how a change in a protein's sequence alters its structure and function, AlphaFold has already spurred the development of novel enzymes for breaking down plastic waste. It has also prompted efforts to better target parasitic diseases. The impacts aren't likely to stop there. The predictions will help experimentalists who solve structures, Baek says. Data from x-ray crystallography and cryo–electron microscopy experiments can be difficult to interpret, Baek and others say, and having a model can help pinpoint the correct structure. “In the short term, it will boost structure determination efforts,” she predicts. “And over time it will also slowly replace [experimental] structural determination efforts.” If that happens, structural biologists won't find themselves out of work. Baker notes that both experimental and computational scientists are already beginning to turn their efforts to the more complex challenge of understanding exactly which proteins interact with one another and what molecular changes happen during these interactions. The new tools will “reset the field,” Baker says. “It's a very exciting time.” [1]: http://www.sciencemag.org/content/370/6521/1144 [2]: http://www.sciencemag.org/content/373/6552/262