AI Safety is a major concern in many deep learning applications such as autonomous driving. Given a trained deep learning model, an important natural problem is how to reliably verify the model's prediction. In this paper, we propose a novel framework --- deep verifier networks (DVN) to verify the inputs and outputs of deep discriminative models with deep generative models. Our proposed model is based on conditional variational auto-encoders with disentanglement constraints. We give both intuitive and theoretical justifications of the model. Our verifier network is trained independently with the prediction model, which eliminates the need of retraining the verifier network for a new model. We test the verifier network on out-of-distribution detection and adversarial example detection problems, as well as anomaly detection problems in structured prediction tasks such as image caption generation. We achieve state-of-the-art results in all of these problems.

In what might only be perceived as a win for Facebook, OpenAI today announced that it will migrate to the social network's PyTorch machine learning framework in future projects, eschewing Google's long-in-the-tooth TensorFlow platform. OpenAI is the San Francisco-based AI research firm cofounded by CTO Greg Brockman, chief scientist Ilya Sutskever, Elon Musk, and others, with backing from luminaries like LinkedIn cofounder Reid Hoffman and former Y Combinator president Sam Altman. In a blog post, the company cited PyTorch's efficiency, scalability, and adoption as the reasons for its decision. "Going forward we'll primarily use PyTorch as our deep learning framework but sometimes use other ones when there's a specific technical reason to do so," said the company in a statement. "We're … excited to be joining a rapidly-growing developer community, including organizations like Facebook and Microsoft, in pushing scale and performance on [graphics cards]." OpenAI says that many of its teams have already migrated their work to PyTorch and that they'll contribute to the PyTorch community in the coming months.

In a massive move, Elon Musk co-founded OpenAI, standardised there primary framework for development as PyTorch. The announcement came in January 30, 2020 in companies blog post which emphasis on the fact that the move will provide the team with trouble-free path to create and share optimized implementations of machine learning models internally. Open AI is research organisation focused on discovering and enacting the path to safe artificial general intelligence, and it was founded by Elon Musk, Sam Altman, Ilya Sutskever and Greg Brockman on December 11, 2015. Open AI is currently based in San Francisco, California. Open AI's mission is to ensure that artificial general intelligence benefits all of humanity and it tries to empower as many humans as possible withe power of AI so that not one or a group of individuals will have AI superpower.

In a massive move, Elon Musk co-founded OpenAI, standardised there primary framework for development as PyTorch. The announcement came in January 30, 2020 in companies blog post which emphasis on the fact that the move will provide the team with trouble-free path to create and share optimized implementations of machine learning models internally.

Roundup Let's catch you up on the latest goings on in the world of AI beyond what we've already written about.

In the last few years AI researchers have had increasing success in speeding up neural network training through data-parallelism, which splits large batches of data across many machines. Researchers have successfully used batch sizes of tens of thousands for image classification and language modeling, and even millions for RL agents that play the game Dota 2. These large batches allow increasing amounts of compute to be efficiently poured into the training of a single model, and are an important enabler of the fast growth in AI training compute. However, batch sizes that are too large show rapidly diminishing algorithmic returns, and it's not clear why these limits are larger for some tasks and smaller for others.[1] We have found that by measuring the gradient noise scale, a simple statistic that quantifies the signal-to-noise ratio of the network gradients,[2] we can approximately predict the maximum useful batch size. Heuristically, the noise scale measures the variation in the data as seen by the model (at a given stage in training).

Sign in to report inappropriate content. For the past 60 years, despite all the setbacks, neural networks have steadily improved. This talk from Greg Brockman of OpenAI will explore the forgotten history of the field, and discuss why we must must dare to dream about dramatic near-term progress.

Artificial intelligence routinely produces startling achievements, but those advances require staggering amounts of computing power and electricity. Last month, researchers at OpenAI in San Francisco revealed an algorithm capable of learning, through trial and error, how to manipulate the pieces of a Rubik's Cube using a robotic hand. It was a remarkable research feat, but it required more than 1,000 desktop computers plus a dozen machines running specialized graphics chips crunching intensive calculations for several months. The effort may have consumed about 2.8 gigawatt-hours of electricity, estimates Evan Sparks, CEO of Determined AI, a startup that provides software to help companies manage AI projects. A spokesperson for OpenAI questioned the calculation, noting that it makes several assumptions. But OpenAI declined to disclose further details of the project or offer an estimate of the electricity it consumed.

Last month, researchers at OpenAI in San Francisco revealed an algorithm capable of learning, through trial and error, how to manipulate the pieces of a Rubik's Cube using a robotic hand. It was a remarkable research feat, but it required more than 1,000 desktop computers plus a dozen machines running specialized graphics chips crunching intensive calculations for several months. The effort may have consumed about 2.8 gigawatt-hours of electricity, estimates Evan Sparks, CEO of Determined AI, a startup that provides software to help companies manage AI projects. A spokesperson for OpenAI questioned the calculation, noting that it makes several assumptions. But OpenAI declined to disclose further details of the project or offer an estimate of the electricity it consumed.

Learning generative models that span multiple data modalities, such as vision and language, is often motivated by the desire to learn more useful, generalisable representations that faithfully capture common underlying factors between the modalities. In this work, we characterise successful learning of such models as the fulfilment of four criteria: i) implicit latent decomposition into shared and private subspaces, ii) coherent joint generation over all modalities, iii) coherent cross-generation across individual modalities, and iv) improved model learning for individual modalities through multi-modal integration. Here, we propose a mixture-of-experts multi-modal variational autoencoder (MMVAE) for learning of generative models on different sets of modalities, including a challenging image - language dataset, and demonstrate its ability to satisfy all four criteria, both qualitatively and quantitatively. Papers published at the Neural Information Processing Systems Conference.