This paper aims to instigate discussion, answering a call for papers from the ICML Machine Learning Debates workshop. While we stand by the points represented here, we do not purport to offer a full or balanced viewpoint or to discuss the overall quality of science in ML. In many aspects, such as reproducibility, the community has advanced standards far beyond what sufficed a decade ago. We note that these arguments are made by us, against us, by insiders offering a critical introspective look, not as sniping outsiders. The ills that we identify are not specific to any individual or institution. We ourselves have fallen into these patterns, and likely will again in the future. Exhibiting one of these patterns doesn't make a paper bad nor does it indict the paper's authors, however we believe that all papers could be made stronger by avoiding these patterns. While we provide concrete examples, our guiding principles are to (i) implicate ourselves, and (ii) to preferentially select from the work of better-established researchers and institutions that we admire, to avoid singling out junior students for whom inclusion in this discussion might have consequences and who lack the opportunity to reply symmetrically. We are grateful to belong to a community that provides sufficient intellectual freedom to allow us to express critical perspectives. In each subsection below, we (i) describe a trend; (ii) provide several examples (as well as positive examples that resist the trend); and (iii) explain the consequences. Pointing to weaknesses in individual papers can be a sensitive topic. To minimize this, we keep examples short and specific.

Over the last years, companies have reaped the benefits of embracing data-driven methods. The data 2.0 strategy will be driven by new AI techniques like deep learning, reinforcement learning, and Bayesian machine learning. Deep learning is the state-of-the-art approach to AI across machine translation, computer vision, and natural language processing. Not using these techniques hurts bottom lines and limits the end-user experience. Approaches using deep learning, solve problems end-to-end.

A team of researchers from NVIDIA, MIT and Aalto University have found a way to fix pixelated photographs using AI -- even if the AI has never seen a clean example of the target photo. The group used deep learning -- a type of machine learning that can teach AI to piece together images, text or video -- to restore images with noisy input. While previous work trained AI to reconstruct photos with missing facial features by showing it complete photos, the current method means AI can rebuild a clean photo by only using "corrupted data", or two tarnished images. And surprisingly, its ability to clean up artifacts, remove text and beautify photos occasionally produced a better outcome than methods requiring cleaner reference material. The AI does this by utilizing a neural network that's been trained using corrupt photos.

We have a lot of exciting things rolling off people's sleeves at Freshworks all the time. But there is a lot more work that goes into bringing out all that awesomeness that everyone sees. We thought we would give you a glimpse of what happens behind the curtains. We are introducing'Behind the Scenes', a fortnightly series that talks about all the effort that is put into making Freshworks work for you. It's an open secret that AI is making inroads into every industry in the quest to offer simple solutions to complicated problems.

We introduce Glow, a reversible generative model which uses invertible 1x1 convolutions. It extends previous work on reversible generative models and simplifies the architecture. Our model can generate realistic high resolution images, supports efficient sampling, and discovers features that can be used to manipulate attributes of data. We're releasing code for the model and an online visualization tool so people can explore and build on these results. The model isn't given attribute labels at training time, yet it learns a latent space where certain directions correspond to changes in attributes like beard density, age, hair color, and so on.

In this special guest feature, Eliya Elon, Director of Product and Business Development at Razor Labs, discusses a new technology that is starting to trickle from the purely theoretical academic world to the business world, one that aligns with your company objectives, that draws a clear line between your business question and the insights generated. This new technology is called Deep Reinforcement-Learning, and it is gaining significant success in different use-cases. Eliya is the VP of Product and strategic partnerships at Razor Labs. He is an experienced tech entrepreneur, selling his last AI company in 2017. Since joining Razor Labs he is focused on creating AI products that bridge the multi-dimensional gap of business needs, user experience, and academic research, hopefully at scale.

Early last year, a Microsoft research project dubbed DeepCoder announced that it had made progress creating AI that could write its own programs. Such a feat has long captured the imagination of technology optimists and pessimists alike, who might consider software that creates its own software as the next paradigm in technology – or perhaps the direct route to building the evil Skynet. As with most machine learning or deep learning approaches that make up the bulk of today's AI, DeepCoder was creating code that it based on large numbers of examples of existing code that researchers used to train the system. The result: software that ended up assembling bits of human-created programs, a feat Wired Magazine referred to as'looting other software.' And yet, in spite of DeepCoder's PR faux pas, the idea of software smart enough to create its own applications remains an area of active research, as well as an exciting prospect for the digital world at large.

In recent years, state-of-the-art methods in computer vision have utilized increasingly deep convolutional neural network architectures (CNNs), with some of the most successful models employing hundreds or even thousands of layers. A variety of pathologies such as vanishing/exploding gradients make training such deep networks challenging. While residual connections and batch normalization do enable training at these depths, it has remained unclear whether such specialized architecture designs are truly necessary to train deep CNNs. In this work, we demonstrate that it is possible to train vanilla CNNs with ten thousand layers or more simply by using an appropriate initialization scheme. We derive this initialization scheme theoretically by developing a mean field theory for signal propagation and by characterizing the conditions for dynamical isometry, the equilibration of singular values of the input-output Jacobian matrix. These conditions require that the convolution operator be an orthogonal transformation in the sense that it is norm-preserving. We present an algorithm for generating such random initial orthogonal convolution kernels and demonstrate empirically that they enable efficient training of extremely deep architectures.

In this paper we study the real time deployment of deep learning algorithms in low resource computational environments. As the use case, we compare the accuracy and speed of neural networks for smile detection using different neural network architectures and their system level implementation on NVidia Jetson embedded platform. We also propose an asynchronous multithreading scheme for parallelizing the pipeline. Within this framework, we experimentally compare thirteen widely used network topologies. The experiments show that low complexity architectures can achieve almost equal performance as larger ones, with a fraction of computation required.

C. Robustness to external noise The sound samples used in all experiments are already recorded under real-world conditions in an office environment. We took care that people in the office are not talking, but background noise like typing on a keyboard and people walking around are clearly identifiable on the sample data. In addition, there is a significant amount of ego noise coming from the robot's servos. Hence, the results depicted in the previous Sections VA and V-B already involve a realistic amount of noise. However, in order to make more precise statements about robustness to noise, we also perform experiments where we simulate an environment with other external sound sources at varying levels. Therefore, we use six randomly selected samples from different background noises including traffic, people speaking, airport, etc.