We are really pleased to announce the date of our first Meetup for the IBM PowerAI Melbourne group. Please note - this meetup is being hosted by our awesome sponsors Advent One Pty Ltd (https://www.adventone.com). Our Presenters.... Dr Adam Makarucha is a data scientist within the IBM Systems team where he is developing deep learning use cases and demonstrations for clients on IBM's PowerAI platform. He recently completed his postdoctorate at IBM Research Australia, working within the Cognitive Analytics team on deep learning applications for the financial services industry. He has worked on machine and deep learning since joining IBM in 2015, primarily focused on natural language processing as well as risk and compliance.

Machine learning (ML) and Artificial Intelligence (AI) represent one of the biggest disruptions to your career according to John Roese, CTO of Dell Technologies. During the Dell Technology World keynote, Roese made this bold but accurate statement. Despite the hype, AI is real and can't be ignored. Leading businesses are using machine learning to deliver quantifiable business value today. For example, Google used the AI knowledge gathered from its DeepMind acquisition to improve its cooling systems, saving the company of hundreds of millions of dollars.

NVIDIA is expanding its Deep Learning Institute (DLI) with new partnerships and educational courses. DLI, which trains thousands of students, developers and data scientists with critical skills needed to apply artificial intelligence, has joined hands with Booz Allen Hamilton and deeplearning.ai DLI and Booz Allen Hamilton will provide hands-on training for data scientists to solve challenging problems in healthcare, cybersecurity and defense. NVIDIA is also expanding its reach with the new NVIDIA University Ambassador Program that enables instructors worldwide to teach students critical job skills and practical applications of AI at no cost. The graphics processing designer is already working with professors at several universities, including Arizona State, Harvard, Hong Kong University of Science and Technology and UCLA.

Note: The popularity of this post has inspired me to write a machine learning test library. Over the past year, I've spent most of my working time doing deep learning research and internships. And a lot of that year was making very big mistakes that helped me learn not just about ML, but about how to engineer these systems correctly and soundly. One of the main principles I learned during my time at Google Brain was that unit tests can make or break your algorithm and can save you weeks of debugging and training time. However, there doesn't seem to be a solid tutorial online on how to actually write unit tests for neural network code.

The ability to generate natural language sequences from source code snippets can be used for code summarization, documentation, and retrieval. Sequence-to-sequence (seq2seq) models, adopted from neural machine translation (NMT), have achieved state-of-the-art performance on these tasks by treating source code as a sequence of tokens. We present ${\rm {\scriptsize CODE2SEQ}}$: an alternative approach that leverages the syntactic structure of programming languages to better encode source code. Our model represents a code snippet as the set of paths in its abstract syntax tree (AST) and uses attention to select the relevant paths during decoding, much like contemporary NMT models. We demonstrate the effectiveness of our approach for two tasks, two programming languages, and four datasets of up to 16M examples. Our model significantly outperforms previous models that were specifically designed for programming languages, as well as general state-of-the-art NMT models.

Topology design optimization offers tremendous opportunity in design and manufacturing freedoms by designing and producing a part from the ground-up without a meaningful initial design as required by conventional shape design optimization approaches. Ideally, with adequate problem statements, to formulate and solve the topology design problem using a standard topology optimization process, such as SIMP (Simplified Isotropic Material with Penalization) is possible. In reality, an estimated over thousands of design iterations is often required for just a few design variables, the conventional optimization approach is in general impractical or computationally unachievable for real world applications significantly diluting the development of the topology optimization technology. There is, therefore, a need for a different approach that will be able to optimize the initial design topology effectively and rapidly. Therefore, this work presents a new topology design procedure to generate optimal structures using an integrated Generative Adversarial Networks (GANs) and convolutional neural network architecture.

Machine learning software accounts for a significant amount of energy consumed in data centers. These algorithms are usually optimized towards predictive performance, i.e. accuracy, and scalability. This is the case of data stream mining algorithms. Although these algorithms are adaptive to the incoming data, they have fixed parameters from the beginning of the execution. We have observed that having fixed parameters lead to unnecessary computations, thus making the algorithm energy inefficient. In this paper we present the nmin adaptation method for Hoeffding trees. This method adapts the value of the nmin parameter, which significantly affects the energy consumption of the algorithm. The method reduces unnecessary computations and memory accesses, thus reducing the energy, while the accuracy is only marginally affected. We experimentally compared VFDT (Very Fast Decision Tree, the first Hoeffding tree algorithm) and CVFDT (Concept-adapting VFDT) with the VFDT-nmin (VFDT with nmin adaptation). The results show that VFDT-nmin consumes up to 27% less energy than the standard VFDT, and up to 92% less energy than CVFDT, trading off a few percent of accuracy in a few datasets.

Deep learning models have lately shown great performance in various fields such as computer vision, speech recognition, speech translation, and natural language processing. However, alongside their state-of-the-art performance, it is still generally unclear what is the source of their generalization ability. Thus, an important question is what makes deep neural networks able to generalize well from the training set to new data. In this article, we provide an overview of the existing theory and bounds for the characterization of the generalization error of deep neural networks, combining both classical and more recent theoretical and empirical results.

Confidentiality of patient information is an essential part of Electronic Health Record System. Patient information, if exposed, can cause a serious damage to the privacy of individuals receiving healthcare. Hence it is important to remove such details from physician notes. A system is proposed which consists of a deep learning model where a de-convolutional neural network and bi-directional LSTM-CNN is used along with regular expressions to recognize and eliminate the individually identifiable information. This information is then removed from a medical practitioner's data which further allows the fair usage of such information among researchers and in clinical trials.

Global Style Tokens (GSTs) are a recently-proposed method to learn latent disentangled representations of high-dimensional data. GSTs can be used within Tacotron, a state-of-the-art end-to-end text-to-speech synthesis system, to uncover expressive factors of variation in speaking style. In this work, we introduce the Text-Predicted Global Style Token (TP-GST) architecture, which treats GST combination weights or style embeddings as "virtual" speaking style labels within Tacotron. TP-GST learns to predict stylistic renderings from text alone, requiring neither explicit labels during training nor auxiliary inputs for inference. We show that, when trained on a dataset of expressive speech, our system generates audio with more pitch and energy variation than two state-of-the-art baseline models. We further demonstrate that TP-GSTs can synthesize speech with background noise removed, and corroborate these analyses with positive results on human-rated listener preference audiobook tasks. Finally, we demonstrate that multi-speaker TP-GST models successfully factorize speaker identity and speaking style. We provide a website with audio samples for each of our findings.