Deep Learning
Deep Neural Networks and Neuro-Fuzzy Networks for Intellectual Analysis of Economic Systems
Averkin, Alexey, Yarushev, Sergey
In tis paper we consider approaches for time series forecasting based on deep neural networks and neuro-fuzzy nets. Also, we make short review of researches in forecasting based on various models of ANFIS models. Deep Learning has proven to be an effective method for making highly accurate predictions from complex data sources. Also, we propose our models of DL and Neuro-Fuzzy Networks for this task. Finally, we show possibility of using these models for data science tasks. This paper presents also an overview of approaches for incorporating rule-based methodology into deep learning neural networks.
Multi-modal, multi-task, multi-attention (M3) deep learning detection of reticular pseudodrusen: towards automated and accessible classification of age-related macular degeneration
Chen, Qingyu, Keenan, Tiarnan D. L., Allot, Alexis, Peng, Yifan, Agrรณn, Elvira, Domalpally, Amitha, Klaver, Caroline C. W., Luttikhuizen, Daniel T., Colyer, Marcus H., Cukras, Catherine A., Wiley, Henry E., Magone, M. Teresa, Cousineau-Krieger, Chantal, Wong, Wai T., Zhu, Yingying, Chew, Emily Y., Lu, Zhiyong
Objective Reticular pseudodrusen (RPD), a key feature of age-related macular degeneration (AMD), are poorly detected by human experts on standard color fundus photography (CFP) and typically require advanced imaging modalities such as fundus autofluorescence (FAF). The objective was to develop and evaluate the performance of a novel'M3' deep learning framework on RPD detection. Materials and Methods A deep learning framework M3 was developed to detect RPD presence accurately using CFP alone, FAF alone, or both, employing 8000 CFP-FAF image pairs obtained prospectively (Age-Related Eye Disease Study 2). The M3 framework includes multi-modal (detection from single or multiple image modalities), multi-task (training different tasks simultaneously to improve generalizability), and multi-attention (improving ensembled feature representation) operation. Performance on RPD detection was compared with state-of-the-art deep learning models and 13 ophthalmologists; performance on detection of two other AMD features (geographic atrophy and pigmentary abnormalities) was also evaluated. Results For RPD detection, M3 achieved area under receiver operating characteristic (AUROC) 0.832, 0.931, and 0.933 for CFP alone, FAF alone, and both, respectively. M3 performance on CFP was very substantially superior to human retinal specialists (median F1-score 0.644 versus 0.350). External validation (on Rotterdam Study, Netherlands) demonstrated high accuracy on CFP alone (AUROC 0.965). The M3 framework also accurately detected geographic atrophy and pigmentary abnormalities (AUROC 0.909 and 0.912, respectively), demonstrating its generalizability. Conclusion This study demonstrates the successful development, robust evaluation, and external validation of a novel deep learning framework that enables accessible, accurate, and automated AMD diagnosis and prognosis. INTRODUCTION Age-related macular degeneration (AMD) is the leading cause of legal blindness in developed countries [1 2]. Late AMD is the stage with the potential for severe visual loss; it takes two forms, geographic atrophy and neovascular AMD. AMD is traditionally diagnosed and classified using color fundus photography (CFP) [3], the most widely used and accessible imaging modality in ophthalmology. In the absence of late disease, two main features (macular drusen and pigmentary abnormalities) are used to classify disease and stratify risk of progression to late AMD [3]. More recently, additional imaging modalities have become available in specialist centers, particularly fundus autofluorescence (FAF) imaging [4 5]. Following these developments in retinal imaging, a third macular feature (reticular pseudodrusen, RPD) is now recognized as a key AMD lesion [6 7].
Graph Contrastive Learning with Augmentations
You, Yuning, Chen, Tianlong, Sui, Yongduo, Chen, Ting, Wang, Zhangyang, Shen, Yang
Generalizable, transferrable, and robust representation learning on graph-structured data remains a challenge for current graph neural networks (GNNs). Unlike what has been developed for convolutional neural networks (CNNs) for image data, self-supervised learning and pre-training are less explored for GNNs. In this paper, we propose a graph contrastive learning (GraphCL) framework for learning unsupervised representations of graph data. We first design four types of graph augmentations to incorporate various priors. We then systematically study the impact of various combinations of graph augmentations on multiple datasets, in four different settings: semi-supervised, unsupervised, and transfer learning as well as adversarial attacks. The results show that, even without tuning augmentation extents nor using sophisticated GNN architectures, our GraphCL framework can produce graph representations of similar or better generalizability, transferrability, and robustness compared to state-of-the-art methods. We also investigate the impact of parameterized graph augmentation extents and patterns, and observe further performance gains in preliminary experiments.
Getting Rid of the Deep Learning Silo in the Data Center
With an electrical engineering education from Purdue University (PhD) and the Indian Institute of Technology Bombay (BS, MS), and nearly 25 years of experience in the semiconductor, systems, and hyperscale service provider industries, he has a broad perspective on hardware design and deployment. The elasticity of cloud infrastructure is a key enabler for enterprises and internet services, creating a shared pool of compute resources that various tenants can draw from as their workloads ebb and flow. Cloud tenants are spared the details of capacity and supply planning. This worked well because modern server systems are very efficient at a multitude of general computing tasks. Deep learning, however, creates new complexities for this model.
Can AI Send Cryptic Messages: Deep Learning For Steganography
"An advantage to concealing speech and not text is preservation of non-lexical content such as speaker identity." Steganography is derived from the word "steganos" meaning concealed or covered. It is the science of concealing messages inside other messages, which are referred to as'carrier.' Steganography techniques date back to the 15th century when messages were physically hidden. In modern steganography, the goal is to covertly communicate a digital message.
It's time to talk about the carbon footprint of artificial intelligence
Artificial intelligence is an increasingly important element of science, medicine, and even the minutiae of our daily lives. Chatbots, digital assistants, and movie and music recommendations from streaming services all depend on "deep learning"--a process by which computer models are trained to recognize patterns in data. That training requires powerful computers and lots and lots of energy--and associated carbon emissions. One of the most elaborate deep learning models, designed to produce human-like language and known as GPT-3, requires an amount of energy equivalent to the yearly consumption of 126 Danish homes and creates a carbon footprint equivalent to traveling 700,000 kilometers by car for a single training session. Still, the computing power used in deep learning grew 300,000-fold between 2012 and 2018, and if that pace of growth continues it's not hard to see how artificial intelligence could have a major climate impact.
Is AutoML ready for Business?
Do (will) we still need Data Scientists? AutoML tools have been gaining traction for the last couple of years, both due to technological advancements and their potential to be leveraged by'Citizen Data Scientists'. Citizen Data Science, is an interesting (often controversial) aspect of Data Science (DS) that aims to automate the design of Machine Learning (ML)/Deep Learning (DL) models, making it more accessible to people without the specialized skills of a Data Scientist. In this article, we will try to understand AutoML, its promise, what is possible today?, where AutoML fails (today)?, is it meant only for Citizen Data Scientists, or does it hold some value for skilled Data Scientists as well? Let us start with a very high-level primer on Machine Learning (ML).
Creating my First Deep Learning + Data Science Workstation
Creating my workstation has been a dream for me, if nothing else. I knew the process involved, yet I somehow never got to it. It might have been time or money. But this time I just had to do it. I was just fed up with setting up a server on AWS for any small personal project and fiddling with all the installations.
AI Jukebox creates 'deepfake' songs, imitating dead pop stars
Artificial intelligence (AI) is being used to create new'deepfake' pop songs that sound like they're being performed by dead musicians, including Elvis Presley, Frank Sinatra, David Bowie and Michael Jackson. Jukebox, created by California-based company OpenAI, is a neural network that generates eerie approximates of pop songs in the style of multiple artists. The neural network generates music, including rudimentary singing complete with lyrics in English and a variety of instruments like guitar and piano. OpenAI has created a expansive library of new tracks, imitating a diverse selection of artists, including the Beatles, Nirvana, Katy Perry, Simon and Garfunkel, Stevie Wonder, Elton John and Ed Sheeran, as well as deceased heroes that almost appear to be brought back to life. Most of the samples have a bizarre, faraway quality to them, as if they're poorly produced demos from the 1950s that haven't seen the light of day until now.
AI-enabled Data Science for COVID-19
COVID-19 is a pandemic that has spread all over the world. With the US now projected at over 6 million cases, and a lot more people are assumed to be exposed and asymptomatic, based on the seroprevalence studies. With the many COVID-19 related datasets that have been collected, AI is helping us fight this virus with applications such as early detection and diagnosis, contact tracing, projection of cases and mortality, development of drugs and vaccines, etc. We invite submission of papers describing timely and innovative research on all aspects of using AI in the fight against COVID.We invite submission of papers describing timely and innovative research on fighting COVID-19 using AI. Some examples that have been delivered in our BIOKDD 2020 workshop (http://home.biokdd.org/biokdd20/program.html) include: (i) bioinformatics (e.g., SARS-CoV-2 study using signature mutations and human leukocyte antigen)(ii) data curation (e.g., COVID-19 knowledge graph and knowledge base, gene signature database, 1Point3Acres CovidNet, COVID-19 literature curation), (iii) deep learning models (e.g., for case projection, COVID-19 detection using chest X-ray), and (iv) statistical methods (e.g., analysis using Bayesian inference and virtual reality). We welcome papers in all aspects of using AI in the fight against COVID-19, such as clinical, epidemiological, data-driven machine learning, statistical research in developing AI for COVID-19, as well as application-oriented papers that make innova...