We're releasing an analysis showing that since 2012, the amount of compute used in the largest AI training runs has been increasing exponentially with a 3.4-month doubling time (by comparison, Moore's Law had a 2-year doubling period).[1] Since 2012, this metric has grown by more than 300,000x (a 2-year doubling period would yield only a 7x increase). Improvements in compute have been a key component of AI progress, so as long as this trend continues, it's worth preparing for the implications of systems far outside today's capabilities. The total amount of compute, in petaflop/s-days, used to train selected results that are relatively well known, used a lot of compute for their time, and gave enough information to estimate the compute used. A petaflop/s-day (pfs-day) consists of performing 1015 neural net operations per second for one day, or a total of about 1020 operations.

Microsoft's recent victory in landing a $10 billion Pentagon cloud-computing contract called JEDI could make life more complicated for one of the software giant's partners: the independent artificial-intelligence research lab OpenAI. OpenAI was created in 2015 by Silicon Valley luminaries including Elon Musk to look to the far horizon, and save the world. The newborn nonprofit said it had commitments totaling $1 billion and would work on AI "to benefit humanity as a whole, unconstrained by a need to generate financial return." But OpenAI restructured into a for-profit this year, saying it needed more money to fulfill its goals, and took $1 billion from Microsoft in a deal that involves helping the company's cloud division develop new AI technology. Now Microsoft's JEDI win raises the possibility that OpenAI's work for the benefit of humanity may also serve the US military.

Microsoft's recent victory in landing a $10 billion Pentagon cloud-computing contract called JEDI could make life more complicated for one of the software giant's partners: the independent artificial-intelligence research lab OpenAI. OpenAI was created in 2015 by Silicon Valley luminaries including Elon Musk to look to the far horizon, and save the world. The newborn nonprofit said it had commitments totaling $1 billion and would work on AI "to benefit humanity as a whole, unconstrained by a need to generate financial return." But OpenAI restructured into a for-profit this year, saying it needed more money to fulfill its goals, and took $1 billion from Microsoft in a deal that involves helping the company's cloud division develop new AI technology. Now Microsoft's JEDI win raises the possibility that OpenAI's work for the benefit of humanity may also serve the US military.

The deluge of fake news was first called out in the wake of the 2016 election when shady websites run by foreign interests spread misinformation, much of which gained a foothold on Facebook. OpenAI worried releasing a bot that could pump out fake news in large quantities would be dangerous for society. Although, some AI researchers felt the firm was just looking for attention. This technology or something like it would be available eventually, they said, so why not release the bot so other teams could develop ways to detect its output.

A BSTRACT In anomaly detection (AD), one seeks to identify whether a test sample is abnormal, given a data set of normal samples. A recent and promising approach to AD relies on deep generative models, such as variational autoencoders (V AEs), for unsupervised learning of the normal data distribution. In semi-supervised AD (SSAD), the data also includes a small sample of labeled anomalies. In this work, we propose two variational methods for training V AEs for SSAD. The intuitive idea in both methods is to train the encoder to'separate' between latent vectors for normal and outlier data. We show that this idea can be derived from principled probabilistic formulations of the problem, and propose simple and effective algorithms. Our methods can be applied to various data types, as we demonstrate on SSAD datasets ranging from natural images to astronomy and medicine, and can be combined with any V AE model architecture. When comparing to state-of-the-art SSAD methods that are not specific to particular data types, we obtain marked improvement in outlier detection. In its common formulation, training data is provided only for normal samples, while at test time, anomalous samples need to be detected. In the probabilistic AD approach, a model of the normal data distribution is learned, and the likelihood of a test sample under this model is thresholded for classification as normal or not. Recently, deep generative models such as variational autoencoders (V AEs, Kingma & Welling 2013) and generative adversarial networks (Goodfellow et al., 2014) have shown promise for learning data distributions in AD (An & Cho, 2015; Suh et al., 2016; Schlegl et al., 2017; Wang et al., 2017). Here, we consider the setting of semi-supervised AD (SSAD), where in addition to the normal samples, a small sample of labeled anomalies is provided (G ornitz et al., 2013). Most importantly, this set is too small to represent the range of possible anomalies, making classification methods (either supervised or semi-supervised) unsuitable. Instead, most approaches are based on'fixing' an unsupervised AD method to correctly classify the labeled anomalies, while still maintaining AD capabilities for unseen outliers (e.g., G ornitz et al., 2013; Mu noz-Mar ı et al., 2010; Ruff et al., 2019).

Advances in deep generative and density models have shown impressive capacity to model complex probability density functions in lower-dimensional space. Also, applying such models to high-dimensional image data to model the PDF has shown poor generalization, with out-of-distribution data being assigned equal or higher likelihood than in-sample data. Methods to deal with this have been proposed that deviate from a fully unsupervised approach, requiring large ensembles or additional knowledge about the data, not commonly available in the real-world. In this work, the previously offered reasoning behind these issues is challenged empirically, and it is shown that data-sets such as MNIST fashion/digits and CIFAR10/SVHN are trivially separable and have no overlap on their respective data manifolds that explains the higher OoD likelihood. Models like masked autoregressive flows and block neural autoregressive flows are shown to not suffer from OoD likelihood issues to the extent of GLOW, PixelCNN++, and real NVP. A new avenue is also explored which involves a change of basis to a new space of the same dimension with an orthonormal unitary basis of eigenvectors before modeling. In the test data-sets and models, this aids in pushing down the relative likelihood of the contrastive OoD data set and improve discrimination results. The significance of the density of the original space is maintained, while invertibility remains tractable. Finally, a look to the previous generation of generative models in the form of probabilistic principal component analysis is inspired, and revisited for the same data-sets and shown to work really well for discriminating anomalies based on likelihood in a fully unsupervised fashion compared with pixelCNN++, GLOW, and real NVP with less complexity and faster training. Also, dimensionality reduction using PCA is shown to improve anomaly detection in generative models.

A text-generating artificial intelligence (AI) algorithm whose creators initially deemed too dangerous to release – given its ability to churn out fake news, spam and misinformation after feasting on a mere headline – has been unleashed. So far, so good, says the research lab, OpenAI. In a blog post last week, the lab said that the researchers have seen "no strong evidence of misuse" of the machine-learning language model, which is called GPT-2… at least, not yet. While we've seen some discussion around GPT-2's potential to augment high-volume/low-yield operations like spam and phishing, we haven't seen evidence of writing code, documentation, or instances of misuse […] We acknowledge that we cannot be aware of all threats, and that motivated actors can replicate language models without model release. Exactly how convincing is the output?

Through this partnership, the companies will accelerate breakthroughs in AI and power OpenAI's efforts to create artificial general intelligence (AGI). The resulting enhancements to the Azure platform will also help developers build the next generation of AI applications. The companies will focus on building a computational platform in Azure of unprecedented scale, which will train and run increasingly advanced AI models, include hardware technologies that build on Microsoft's supercomputing technology, and adhere to the two companies' shared principles on ethics and trust. This will create the foundation for advancements in AI to be implemented in a safe, secure and trustworthy way and is a critical reason the companies chose to partner together. Over the past decade, innovative applications of deep neural networks coupled with increasing computational power have led to continuous AI breakthroughs in areas such as vision, speech, language processing, translation, robotic control and even gaming.

Microsoft has invested $1 billion in the Elon Musk-founded artificial intelligence venture that plans to mimic the human brain using computers. OpenAI said the investment would go towards its efforts of building artificial general intelligence (AGI) that can rival and surpass the cognitive capabilities of humans. "The creation of AGI will be the most important technological development in human history, with the potential to shape the trajectory of humanity," said OpenAI CEO Sam Altman. "Our mission is to ensure that AGI technology benefits all of humanity, and we're working with Microsoft to build the supercomputing foundation on which we'll build AGI." The two firms will jointly build AI supercomputing technologies, which OpenAI plans to commercialise through Microsoft and its Azure cloud computing business.

OpenAI, the AI research lab has finally published the GPT2 -- the text generating AI tool which the lab once said was too "dangerous" to share. In a blog post, OpenAI said that despite the arguments of GPT-2 potential in creating synthetic propaganda, fake news, and online phishing campaigns, "we've seen no strong evidence of misuse so far" Back in February, OpenAI announced the GPT2, a language model based upon 1.5 billion parameters and trained by analyzing over 8 million web pages. The main objective of GPT2 is to create coherent text from a few words. The text generating AI tool can be used for many tasks such as translation, chatbots, coming up with unprecedented answers and more. But citing concerns that it could be used for malicious intent, the company withheld the release of the full version.