It has been common to argue or imply that a regularizer can be used to alter a statistical property of a hidden layer's representation and thus improve generalization or performance of deep networks. For instance, dropout has been known to improve performance by reducing co-adaptation, and representational sparsity has been argued as a good characteristic because many data-generation processes have a small number of factors that are independent. In this work, we analytically and empirically investigate the popular characteristics of learned representations, including correlation, sparsity, dead unit, rank, and mutual information, and disprove many of the \textit{conventional wisdom}. We first show that infinitely many Identical Output Networks (IONs) can be constructed for any deep network with a linear layer, where any invertible affine transformation can be applied to alter the layer's representation characteristics. The existence of ION proves that the correlation characteristics of representation is irrelevant to the performance. Extensions to ReLU layers are provided, too. Then, we consider sparsity, dead unit, and rank to show that only loose relationships exist among the three characteristics. It is shown that a higher sparsity or additional dead units do not imply a better or worse performance when the rank of representation is fixed. We also develop a rank regularizer and show that neither representation sparsity nor lower rank is helpful for improving performance even when the data-generation process has a small number of independent factors. Mutual information $I(\mathbf{z}_l;\mathbf{x})$ and $I(\mathbf{z}_l;\mathbf{y})$ are investigated, and we show that regularizers can affect $I(\mathbf{z}_l;\mathbf{x})$ and thus indirectly influence the performance. Finally, we explain how a rich set of regularizers can be used as a powerful tool for performance tuning.

After the success of deep generative models in image generation tasks, learning disentangled latent variable of data has become a major part of deep learning research. Many models have been proposed to learn an interpretable and factorized representation of latent variable by modifying their objective function or model architecture. While disentangling the latent variable, some models show lower quality of reconstructed images and others increase the model complexity which is hard to train. In this paper, we propose a simple disentangling method with traditional principle component analysis (PCA) which is applied to the latent variables of variational auto-encoder (VAE). Our method can be applied to any generative models. In experiment, we apply our proposed method to simple VAE models and experimental results confirm that our method finds more interpretable factors from the latent space while keeping the reconstruction error the same.

Convolutional neural networks (CNNs) have achieved remarkable performance in many applications, especially image recognition. As a crucial component of CNNs, sub-sampling plays an important role, and max pooling and arithmetic average pooling are commonly used sub-sampling methods. In addition to the two pooling methods, however, there could be many other pooling types, such as geometric average, harmonic average, and so on. Since it is not easy for algorithms to find the best pooling method, human experts choose types of pooling, which might not be optimal for different tasks. Following deep learning philosophy, the type of pooling can be driven by data for a given task. In this paper, we propose {\em alpha-pooling}, which has a trainable parameter $\alpha$ to decide the type of pooling. Alpha-pooling is a general pooling method including max pooling and arithmetic average pooling as a special case, depending on the parameter $\alpha$. In experiments, alpha-pooling improves the accuracy of image recognition tasks, and we found that max pooling is not the optimal pooling scheme. Moreover each layer has different optimal pooling types.

Automl is the key technology for machine learning problem. Current state of art hyperparameter optimization methods are based on traditional black-box optimization methods like SMBO (SMAC, TPE). The objective function of black-box optimization is non-smooth, or time-consuming to evaluate, or in some way noisy. Recent years, many researchers offered the work about the properties of hyperparameters. However, traditional hyperparameter optimization methods do not take those information into consideration. In this paper, we use gradient information and machine learning model analysis information to accelerate traditional hyperparameter optimization methods SMBO. In our L2 norm experiments, our method yielded state-of-the-art performance, and in many cases outperformed the previous best configuration approach.

A Stockholm-based startup has developed a talking AI assistant with a face. Furhat combines digital assistant technology akin to Alexa or Siri and humanoid robots like SoftBank's Pepper to create a device that's creepily lifelike. It's essentially a disembodied head that can be customized with different faces - even characters from the sci-fi film Avatar. Furhat is a social robot created by Stockholm-based startup Furhat Robotics. It weighs just over 7lbs and is essentially a disembodied head with a face projected onto it.

China has begun rolling out new surveillance software capable of recognising people simply by the way that they walk. The "gait recognition" technology, developed by Chinese artificial intelligence firm Watrix, is capable of identifying individuals from the shape and movement of their silhouette from up to 50 metres away, even if their face is hidden. The system is currently being used by police in Beijing and Shanghai and adds to the country's formidable surveillance network that includes an estimated 170 million CCTV cameras. The software can be used on footage from standard surveillance cameras, however it does not currently work in real-time. Instead, the footage is analysed once it is recorded, which takes approximately 10 minutes.

Samsung has finally unveiled its foldable phone, after years of teasing and rumours. The handset, with a screen known as the Infinity Flex Display, can be folded out into a tablet or bent in to make it smaller and suitable for use as a phone or fitting into people's pockets. The company says the technology powering the new phone will serve as the "foundation of the smartphone of tomorrow". The I.F.O. is fuelled by eight electric engines, which is able to push the flying object to an estimated top speed of about 120mph. The giant human-like robot bears a striking resemblance to the military robots starring in the movie'Avatar' and is claimed as a world first by its creators from a South Korean robotic company Waseda University's saxophonist robot WAS-5, developed by professor Atsuo Takanishi and Kaptain Rock playing one string light saber guitar perform jam session A man looks at an exhibit entitled'Mimus' a giant industrial robot which has been reprogrammed to interact with humans during a photocall at the new Design Museum in South Kensington, London Electrification Guru Dr. Wolfgang Ziebart talks about the electric Jaguar I-PACE concept SUV before it was unveiled before the Los Angeles Auto Show in Los Angeles, California, U.S The Jaguar I-PACE Concept car is the start of a new era for Jaguar.

India is a country in South Asia. It is the seventh-largest country by area, the second-most populous country (with over 1.2 billion people), and the most populous democracy in the world. It is bounded by the Indian Ocean on the south, the Arabian Sea on the southwest, and the Bay of Bengal on the southeast. It shares land borders with Pakistan to the west; China, Nepal, and Bhutan to the northeast; and Bangladesh and Myanmar to the east. In the Indian Ocean, India is in the vicinity of Sri Lanka and the Maldives. According to the International Monetary Fund (IMF), the Indian economy in 2017 was nominally worth US$2.611

If you've been paying attention in the past year, it seems that all anyone can talk about is the coming artificial intelligence boom on the horizon. Whether it's the Amazon, Google, or Facebook, everyone seems to be getting in on the AI game as fast as they can. And with good reason--they're having to play catchup with the rapid growth of artificial intelligence in China. For the past few decades, China has developed a reputation as being the undisputed source of manufacturing for a whole host of well-established Western companies. Whether this bred Western complacency is debatable, but what is indisputable is that China has been diligently laying the groundwork to breakout into the tech world in its own right--with the power to start calling the shots on the world stage.

A conference here to gather American business and military experts to discuss the coming revolution in artificial intelligence was a good Election Day measure of the challenges ahead to maintain the U.S. competitive edge. Corporate and government leaders agree that China's rapid application of AI to business and military problems should be a "Sputnik moment" to propel change in America. As a top-down command economy, China is directing money and its best brains to develop the smart systems that will operate cars, planes, offices and information -- along with the transformation of warfare. The United States is struggling to respond to this world-changing challenge. What's underway is frail and exists mostly on paper.