Convolutional Neural Networks (CNN) for image recognition tasks are seeing rapid advances in the available architectures and how networks are trained based on large computational infrastructure and standard datasets with millions of images. In contrast, performance and time constraints for example, of small devices and free cloud GPUs necessitate efficient network training (i.e., highest accuracy in the shortest inference time possible), often on small datasets. Here, we hypothesize that initially decreasing image size during training makes the training process more efficient, because pre-shaping weights with small images and later utilizing these weights with larger images reduces initial network parameters and total inference time. We test this Efficient Network TRaining (ENTR) Hypothesis by training pre-trained Residual Network (ResNet) models (ResNet18, 34, & 50) on three small datasets (steel microstructures, bee images, and geographic aerial images) with a free cloud GPU. Based on three training regimes of i) not, ii) gradually or iii) in one step increasing image size over the training process, we show that initially reducing image size increases training efficiency consistently across datasets and networks. We interpret these results mechanistically in the framework of regularization theory. Support for the ENTR hypothesis is an important contribution, because network efficiency improvements for image recognition tasks are needed for practical applications. In the future, it will be exciting to see how the ENTR hypothesis holds for large standard datasets like ImageNet or CIFAR, to better understand the underlying mechanisms, and how these results compare to other fields such as structural learning.

Despite transitional growing pains, the promise of artificial intelligence (AI) in innovation and decision-making offers a future with better decisions made at the command of but not by humans. That's what Pradeep Dubey, director of the Parallel Computing Laboratory at Intel, told attendees of a plenary talk at the PEARC18 conference in Pittsburgh, Pa., on July 25. "Humans and machines have had this very nice separation of labor," Dubey said. "Humans make decisions; machines crunch numbers … but humans are terrible decision makers." The annual Practice and Experience in Advanced Research Computing (PEARC) conference--with the theme Seamless Creativity--stresses key objectives for those who manage, develop and use advanced research computing throughout the U.S. and the world. This year's program offered tutorials, plenary and contributed talks, workshops, panels, poster sessions and a visualization showcase.

Microsoft Research has a recent paper (Machine Teaching: A New Paradigm for Building Machine Learning Systems) that explores the eventual evolution of Machine Learning. The paper makes a clear distinction between Machine Learning and Machine Teaching. The authors explain that Machine Learning is what is practiced in research organizations and Machine Teaching is what will eventually practiced by engineering organizations. If you have been actively following this blog, it should be apparent by now that it has a distinctly software engineering spin towards the application of Deep Learning technology. We are inundated on a daily basis with plenty of astonishing discoveries in Deep Learning.

Google is an advertising company. In spite of its dominance in mobile operating systems, productivity tools like Gmail, and forays into media with subscription services such as Google Play Music and YouTube TV, the company still makes nearly 90% of its money from advertisements and its advertising platform. Advertising is also the primary driver of revenue for its holding company, Alphabet, which has a large portfolio of other businesses including AI research lab DeepMind, autonomous driving company Waymo, life sciences company Verily, and investment groups GV and CapitalG. Many of those other companies are long-term bets that aren't expected to pay off in the near term--or maybe ever. There are grandiose objectives like DeepMind's quest to "solve intelligence," or Calico's mission to cure death.

The term "Artificial Intelligence" has been floating around for a while. We see it in sci-fi movies, "AI" game bots we play against, Google search, and, oh yeah, those robots that are some day going to take over the world. Off late, though, "Machine Learning" and "Deep Learning" have surfaced, with many asking what exactly each of these are. Artificial Intelligence is the general category, common to all three. In a diagram, Artificial Intelligence would be the bigger, encapsulating circle that contains Machine and Deep Learning.

Deep learning is a relatively new term, although it has existed prior to the dramatic uptick in online searches of late. Enjoying a surge in research and industry, due mainly to its incredible successes in a number of different areas, deep learning is the process of applying deep neural network technologies - that is, neural network architectures with multiple hidden layers - to solve problems. Deep learning is a process, like data mining, which employs deep neural network architectures, which are particular types of machine learning algorithms. Deep learning has racked up an impressive collection of accomplishments of late. In light of this, it's important to keep a few things in mind, at least in my opinion: As shown in the image above, deep learning is to data mining as (deep) neural networks are to machine learning (process versus architecture).

Abstract--Cancer is among the leading causes of death worldwide. Risk stratification of cancer tumors in radiology images can be improved with computer-aided diagnosis (CAD) tools which can be made faster and more accurate. Tumor characterization through CADs can enable noninvasive cancer staging and prognosis, and foster personalized treatment planning as a part of precision medicine. In this study, we propose both supervised and unsupervised machine learning strategies to improve tumor characterization. Our first approach is based on supervised learning for which we demonstrate significant gains in deep learning algorithms, particularly by utilizing a 3D Convolutional Neural Network along with transfer learning. Motivated by the radiologists' interpretations of the scans, we then show how to incorporate task dependent feature representations into a CAD system via a "graph-regularized sparse Multi-Task Learning (MTL)" framework. In the second approach, we explore an unsupervised scheme to address the limited availability of labeled training data, a common problem in medical imaging applications. Inspired by learning from label proportion (LLP) approaches, we propose a new algorithm, proportion-SVM, to characterize tumor types. We also seek the answer to the fundamental question about the goodness of "deep features" for unsupervised tumor classification. We evaluate our proposed approaches (both supervised and unsupervised) on two different tumor diagnosis challenges: lung and pancreas with 1018 CT and 171 MRI scans respectively. Cancer affects a significant number of the world's population; approximately 40% of people will be diagnosed with cancer at some point during their lifetime with an overall mortality of 171.2 per 100,000 people per year (based on 2008-2012 deaths) [1]. Lung and pancreatic cancers are two of the most common cancers. While lung cancer is the largest cause of cancer-related deaths in the world, pancreatic cancer has the poorest prognosis with a 5-year survival rate of just 7% in the United States [1]. With regards to pancreatic cancer, specifically in this work, we focus on the challenging problem of automatic diagnosis of Intraductal Papillary Mucinous Neoplasms (IPMN). IPMN is a pre-malignant condition and if left untreated, it can progress to invasive cancer.

Transformationally invariant processors constructed by transformed input vectors or operators have been suggested and applied to many applications. In this study, transformationally identical processing based on combining results of all sub-processes with corresponding transformations either at the final processing step or at the beginning step were found to be equivalent through a special algebraical operation property. This technique can be applied to most convolutional neural network (CNN) systems. Specifically, a transformationally identical CNN system can be constructed by running internally symmetric operations in parallel with the same transformation family followed by a flatten layer with weights sharing among their corresponding transformation elements. Such a CNN can output the same result with any transformation version of the original input vector. Interestingly, we found that this type of transformationally identical CNN system by combining symmetric operations at the flatten layer is mathematically equivalent to an ordinary CNN but combining all transformation versions of the input vector at the input layer. Since the former is computationally demanding, its equivalent with greatly simplified implementation is suggested

Convolutional neural networks have led to significant breakthroughs in the domain of medical image analysis. However, the task of breast cancer segmentation in whole-slide images (WSIs) is still underexplored. WSIs are large histopathological images with extremely high resolution. Constrained by the hardware and field of view, using high-magnification patches can slow down the inference process and using low-magnification patches can cause the loss of information. In this paper, we aim to achieve two seemingly conflicting goals for breast cancer segmentation: accurate and fast prediction. We propose a simple yet efficient framework Reinforced Auto-Zoom Net (RAZN) to tackle this task. Motivated by the zoom-in operation of a pathologist using a digital microscope, RAZN learns a policy network to decide whether zooming is required in a given region of interest. Because the zoom-in action is selective, RAZN is robust to unbalanced and noisy ground truth labels and can efficiently reduce overfitting. We evaluate our method on a public breast cancer dataset. RAZN outperforms both single-scale and multi-scale baseline approaches, achieving better accuracy at low inference cost.

Research on automatic music generation has seen great progress due to the development of deep neural networks. However, the generation of multi-instrument music of arbitrary genres still remains a challenge. Existing research either works on lead sheets or multi-track piano-rolls found in MIDIs, but both musical notations have their limits. In this work, we propose a new task called lead sheet arrangement to avoid such limits. A new recurrent convolutional generative model for the task is proposed, along with three new symbolic-domain harmonic features to facilitate learning from unpaired lead sheets and MIDIs. Our model can generate lead sheets and their arrangements of eight-bar long. Audio samples of the generated result can be found at https://drive.google.com/open?id=1c0FfODTpudmLvuKBbc23VBCgQizY6-Rk