Due to its efficiency and ease to implement, stochastic gradient descent (SGD) has been widely used in machine learning. In particular, SGD is one of the most popular optimization methods for distributed learning. Recently, quantized SGD (QSGD), which adopts quantization to reduce the communication cost in SGD-based distributed learning, has attracted much attention. Although several QSGD methods have been proposed, some of them are heuristic without theoretical guarantee, and others have high quantization variance which makes the convergence become slow. In this paper, we propose a new method, called Quantized Epoch-SGD (QESGD), for communication-efficient distributed learning. QESGD compresses (quantizes) the parameter with variance reduction, so that it can get almost the same performance as that of SGD with less communication cost. QESGD is implemented on the Parameter Server framework, and empirical results on distributed deep learning show that QESGD can outperform other state-of-the-art quantization methods to achieve the best performance.

Convex clustering is a promising new approach to the classical problem of clustering, combining strong performance in empirical studies with rigorous theoretical foundations. Despite these advantages, convex clustering has not been widely adopted, due to its computationally intensive nature and its lack of compelling visualizations. To address these impediments, we introduce Algorithmic Regularization, an innovative technique for obtaining high-quality estimates of regularization paths using an iterative one-step approximation scheme. We justify our approach with a novel theoretical result, guaranteeing global convergence of the approximate path to the exact solution under easily-checked non-data-dependent assumptions. The application of algorithmic regularization to convex clustering yields the Convex Clustering via Algorithmic Regularization Paths (CARP) algorithm for computing the clustering solution path. On example data sets from genomics and text analysis, CARP delivers over a 100-fold speed-up over existing methods, while attaining a finer approximation grid than standard methods. Furthermore, CARP enables improved visualization of clustering solutions: the fine solution grid returned by CARP can be used to construct a convex clustering-based dendrogram, as well as forming the basis of a dynamic path-wise visualization based on modern web technologies. Our methods are implemented in the open-source R package clustRviz, available at https://github.com/DataSlingers/clustRviz.

Natural human-computer interaction and audio-visual human behaviour sensing systems, which would achieve robust performance in-the-wild are more needed than ever as digital devices are becoming indispensable part of our life more and more. Accurately annotated real-world data are the crux in devising such systems. However, existing databases usually consider controlled settings, low demographic variability, and a single task. In this paper, we introduce the SEWA database of more than 2000 minutes of audio-visual data of 398 people coming from six cultures, 50% female, and uniformly spanning the age range of 18 to 65 years old. Subjects were recorded in two different contexts: while watching adverts and while discussing adverts in a video chat. The database includes rich annotations of the recordings in terms of facial landmarks, facial action units (FAU), various vocalisations, mirroring, and continuously valued valence, arousal, liking, agreement, and prototypic examples of (dis)liking. This database aims to be an extremely valuable resource for researchers in affective computing and automatic human sensing and is expected to push forward the research in human behaviour analysis, including cultural studies. Along with the database, we provide extensive baseline experiments for automatic FAU detection and automatic valence, arousal and (dis)liking intensity estimation.

State of the art deep generative networks are capable of producing images with such incredible realism that they can be suspected of memorizing training images. It is why it is not uncommon to include visualizations of training set nearest neighbors, to suggest generated images are not simply memorized. We demonstrate this is not sufficient and motivates the need to study memorization/overfitting of deep generators with more scrutiny. This paper addresses this question by i) showing how simple losses are highly effective at reconstructing images for deep generators ii) analyzing the statistics of reconstruction errors when reconstructing training and validation images, which is the standard way to analyze overfitting in machine learning. Using this methodology, this paper shows that overfitting is not detectable in the pure GAN models proposed in the literature, in contrast with those using hybrid adversarial losses, which are amongst the most widely applied generative methods. The paper also shows that standard GAN evaluation metrics fail to capture memorization for some deep generators. Finally, the paper also shows how off-the-shelf GAN generators can be successfully applied to face inpainting and face super-resolution using the proposed reconstruction method, without hybrid adversarial losses.

Quantification of the stationary points and the associated basins of attraction of neural network loss surfaces is an important step towards a better understanding of neural network loss surfaces at large. This work proposes a novel method to visualise basins of attraction together with the associated stationary points via gradient-based random sampling. The proposed technique is used to perform an empirical study of the loss surfaces generated by two different error metrics: quadratic loss and entropic loss. The empirical observations confirm the theoretical hypothesis regarding the nature of neural network attraction basins. Entropic loss is shown to exhibit stronger gradients and fewer stationary points than quadratic loss, indicating that entropic loss has a more searchable landscape. Quadratic loss is shown to be more resilient to overfitting than entropic loss. Both losses are shown to exhibit local minima, but the number of local minima is shown to decrease with an increase in dimensionality. Thus, the proposed visualisation technique successfully captures the local minima properties exhibited by the neural network loss surfaces, and can be used for the purpose of fitness landscape analysis of neural networks.

Automatic human affect recognition is a key step towards more natural human-computer interaction. Recent trends include recognition in the wild using a fusion of audiovisual and physiological sensors, a challenging setting for conventional machine learning algorithms. Since 2010, novel deep learning algorithms have been applied increasingly in this field. In this paper, we review the literature on human affect recognition between 2010 and 2017, with a special focus on approaches using deep neural networks. By classifying a total of 950 studies according to their usage of shallow or deep architectures, we are able to show a trend towards deep learning. Reviewing a subset of 233 studies that employ deep neural networks, we comprehensively quantify their applications in this field. We find that deep learning is used for learning of (i) spatial feature representations, (ii) temporal feature representations, and (iii) joint feature representations for multimodal sensor data. Exemplary state-of-the-art architectures illustrate the progress. Our findings show the role deep architectures will play in human affect recognition, and can serve as a reference point for researchers working on related applications.

Cars are getting smarter - and while many focus on seeing the road ahead, they are also set to begin analyzing drivers and passengers. This week at CES, the international consumer electronics show in Las Vegas, a host of startup companies are showing off inward facing cameras that watch and analyze drivers, passengers and objects in cars. Carmakers say they will boost safety - but privacy campaigners warn they could be used to make money by analyzing every movement - even being able to track a passenger's gaze to see what ads they are looking at, and monitor the emotions of people through their facial expressions. Occupants, inside a car, are seen on a monitor using technology by Silicon Valley company Eyeris, which uses cameras and AI to track drivers and passengers for safety benefits, shown during an interview in San Jose, California, U.S., December 28, 2018. Carmakers could gather anonymized data and sell it.

When reviewing 2018's retail landscape, there have certainly been ongoing challenges and opportunities that have pushed the evolution of retail to match the demands of a changing consumer. Mobile technology, speed of service/delivery, and low prices are just the tip of the iceberg. In recent discussions with a variety of retailers and retail analysts in Australia and New Zealand, artificial intelligence (AI) and machine learning (ML) are high on everyone's list of toys for the New Year to generate efficiencies across the retail enterprise. The question retailers are asking is whether to build an AI/ML engine themselves or, more probable, turn to a specialised software company already operating in the AI/ML space. Consumers want their product and they want it now -- in their size, flavour, length, shape, brand, weight.

Testers should be involved in the requirements collection phase of the software development lifecycle because it benefits both the QA and business teams to understand the requirements better. In test management, we analyze those requirements, prepare test cases, execute test cases, do bug tracking, and get QA to sign off on test coverage. At my company, a digital commerce agency, we work on e-commerce technologies and serve enterprise clients. Test coverage is an essential metric to measure the quality delivery of our projects. Our test management was initially handled in spreadsheets, but multiple versions of different files ended up underused.

Traditional photo-id involves a laborious manual process of matching each dolphin fin photograph captured in the field to a catalogue of known individuals. Weexamine this problem in the context of open-set recognition andutilise a triplet loss function to learn a compact representation of fin images in a Euclidean embedding, where the Euclidean distance metric represents fin similarity. We show that this compact representation can be successfully learnt from a fairly small (in deep learning context) training set and still generalise well to out-of-sample identities (completely new dolphin individuals), with top-1 and top-5 test set (37 individuals) accuracy of 90.5 2 and 93.6 1 percent. In the presence of 1200 distractors, top-1 accuracy dropped by 12%; however, top-5 accuracy saw only a 2.8% drop. I. INTRODUCTION Dolphin photo-identification (photo-ID) studies involve photographing dolphindorsal fins during field work and then having a human categorise images into unique individual animals andmatching them with an existing catalogue of known individuals. Individuals are identified by natural features that can be observed on the fins--these features vary between species, but typically include the pattern of nicks and notches on the trailing edge of the fin, the scratches/rake marks/scars on the fin and (for some species) the pigmentation patterns [1]. Matchingof new images from the field against a large catalogue is time consuming, because it requires a human to compare each candidate image to every fin in the catalogue, taking O(mn) average time (where m is the number of new images and n is the number of catalogued individuals). Moreover, as n increases over time as more individuals are added to the catalogue, so does the likelihood of making a mistake, putting catalogue integrity at risk.