dataset



Better Language Models and Their Implications

#artificialintelligence

Our model, called GPT-2 (a successor to GPT), was trained simply to predict the next word in 40GB of Internet text. Due to our concerns about malicious applications of the technology, we are not releasing the trained model. As an experiment in responsible disclosure, we are instead releasing a much smaller model for researchers to experiment with, as well as a technical paper. GPT-2 is a large transformer-based language model with 1.5 billion parameters, trained on a dataset[1] of 8 million web pages. GPT-2 is trained with a simple objective: predict the next word, given all of the previous words within some text. The diversity of the dataset causes this simple goal to contain naturally occurring demonstrations of many tasks across diverse domains. GPT-2 is a direct scale-up of GPT, with more than 10X the parameters and trained on more than 10X the amount of data. GPT-2 displays a broad set of capabilities, including the ability to generate conditional synthetic text samples of unprecedented quality, where we prime the model with an input and have it generate a lengthy continuation. In addition, GPT-2 outperforms other language models trained on specific domains (like Wikipedia, news, or books) without needing to use these domain-specific training datasets. On language tasks like question answering, reading comprehension, summarization, and translation, GPT-2 begins to learn these tasks from the raw text, using no task-specific training data.


How Healthy Is Your Machine Learning Model? - Analytics India Magazine

#artificialintelligence

Traders follow a simple motto: buy low and sell high. But when the opposite happens, the stock market goes berserk. On a fine morning in 2012, the NYSE had to step in and cancel numerous trades due to erroneous trading by Knight Capital which saw the biggest drop ever since it went public. Instead of at least attempting to provide liquidity via limit trades, Knight's algorithm acted as a market order. Naturally, when the entire logic of trading was perverted courtesy of Knight's busted algorithm, everything went chaos, and stocks went higher because they went higher, and the higher they went, the greater the incentive for the algorithm to keep pushing the stock higher.


A Crucial Step for Averting AI Disasters

#artificialintelligence

The expanding use of AI is attracting new attention to the importance of workforce diversity. Data from the U.S. Bureau of Labor Statistics shows that, although technology companies have increased efforts to recruit women and minorities, computer and software professionals who write artificial intelligence (AI) programs remain largely white and male. A byproduct of this lack of diversity is that datasets often lack adequate representation of women or minority groups. For example, one widely used dataset is more than 74% male and 83% white, meaning algorithms based on this data could have blind spots or biases built in. Biases in algorithms can skew decision-making, and many companies have realized that eliminating bias upfront among those who write code is essential.


Data Versioning · EmilyGorcenski.com

#artificialintelligence

Productionizing machine learning/AI/data science is a challenge. Not only are the outputs of machine-learning algorithms often compiled artifacts that need to be incorporated into existing production services, the languages and techniques used to develop these models are usually very different than those used in building the actual service. In this post, I want to explore how the degrees of freedom in versioning machine learning systems poses a unique challenge. I'll identify four key axes on which machine learning systems have a notion of version, along with some brief recommendations for how to simplify this a bit. Consider the following: you run a large webservice on a JVM-based stack, and now you want to incorporate a machine learning model. You have data scientists, and they have spent some time doing the research, and now they are ready to deliver their work product: a proof-of-concept model built in R, and you have to implement this somehow. When none of your data scientists are backend engineers and none of your backend engineers speak R. Most of these questions don't have an accepted answer in the way that we have accepted answers in the world of app development, for instance.


Finding answers in big data when you don't know what questions to ask

#artificialintelligence

But we are still looking for a specific result set. It's not that we already know the answer; it's just that we are looking for answers to specific questions. Another data analytics expert interviewed points out that data in its own right "is dumb." It is the variety of data that creates the value, when you can find ways to link different datasets. This linking process comes on top of the regular data preparation tasks to create a curated, clean, and interoperable set.


Artificial Intelligence and Data Science Advances in 2018 and Trends for 2019

#artificialintelligence

The insane pre-holiday shopping is behind us, along with celebrations, and personal to-do lists for the next 12 months. So, let's analyze the data science and artificial intelligence accomplishments and events of the past year. We talked with experts from Booking.com, Wolfram Research, BetConstruct, and other data science specialists who shared their thoughts about opportunities as well as their influence on business, research, and everyday lives for both industries. Experts have different points of view on whether 2018 was rich in important achievements and events. No recent achievements can compete with inventions of a multilayer perceptron (MLP), neural net training techniques like backpropagation and backpropagation through time (BPTT), residual networks, the introduction of Generative Adversarial Networks (GANs), and deep Q-learning networks (DQN). "So, looking back to memorable ones I listed before, there weren't'brand new' accomplishments in 2018," summarizes Oleksandr.


Using AI to avert 'environmental catastrophe'

#artificialintelligence

Funded by UK Research and Innovation (UKRI), the Centre for Doctoral Training in Application of Artificial Intelligence to the study of Environmental Risks (AI4ER) is one of 16 new Centres for Doctoral Training (CDTs) announced today. The Cambridge Centre will be led by Professor Simon Redfern, Head of the Department of Earth Sciences. Climate risk, environmental change and environmental hazards pose some of the most significant threats we face in the 21st century. At the same time, we have increasingly larger datasets available to observe the planet, from the atomic scale all the way through to global satellite observations. "These datasets represent a transformation in the way we can study and understand the Earth and environment, as we assess and find solutions to environmental risk," said Redfern.


5 AI Myths Debunked

#artificialintelligence

"AI will automate everything and put people out of work." "AI is a science-fiction technology." "Robots will take over the world." The hype around artificial intelligence (AI) has produced many myths, in mainstream media, in board meetings and across organizations. Some worry about an "almighty" AI that will take over the world, and some think that AI is nothing more than a buzzword.


How IBM Wants to Defend Neural Networks Against Adversarial Attacks

#artificialintelligence

The security and robustness of deep neural networks(DNNs) architectures is one of the most important areas of research in the deep learning field. The native complexity of neural networks and its lack of interpretability makes them vulnerable to many forms of attacks. Some of the most sophisticated and scariest forms of attacks on DNNs are generated using other neural networks. Adversarial neural networks(ANNs) are often used to generate numerous attack vectors on DNNs by manipulating aspects such as the input dataset of the training policy. Protecting against adversarial attacks is far from being an easy endeavor as the attackers are always mutating and evolving.