Deep Learning
Will AI coding assistants like GitHub's Copilot transform developers' jobs?
OpenAI has once again made the headlines, this time with Copilot, an AI-powered programming tool jointly built with GitHub. Built on top of GPT-3, OpenAI's famous language model, Copilot is an autocomplete tool that provides relevant (and sometimes lengthy) suggestions as you write code. Copilot is currently available to select applicants as an extension in Visual Studio Code, the flagship programming tool of Microsoft, GitHub's parent company. While the AI-powered code generator is still a work in progress, it provides some interesting hints about the business of large language models and the future directions of the software industry. Attend the tech festival of the year and get your super early bird ticket now!
AI is transforming the coding of computer programs
GPT-3 IS quite a beast. The Generative Pre-Trained Transformer 3, to give its full name, is a language model developed by OpenAI, a part-commercial, part not-for-profit artificial-intelligence (AI) laboratory in San Francisco. GPT-3 was trained on an unprecedented mass of text to teach it the probability that a given word will follow preceding words. When fed a short text "prompt", it cranks out astonishingly coherent prose written in a similar style. Access to GPT-3 is restricted.
Chest X-ray Interpretation Better with AI
A new deep learning model could help radiologists in any facility interpret chest X-rays. In a new study published in The Lancet Digital Health, investigators from Australia outlined their new tool. It is designed to alleviate heavy workloads and make it easier for providers who do not have specialty thoracic training to read these scans while reducing errors. Chest X-rays are already the most common imaging study worldwide, and that number is growing, said the team from annalise.ai, the company that created the AI model. Developing a tool to help shoulder the weight and process the workload will be critical. "The ability of the AI model to identify findings on chest X-rays is very encouraging," said Catherine Jones, MBBS, thoracic radiologist, chest lead at annalise.ai,
Unpopular Opinion: We'll Abandon Machine Learning as Main AI Paradigm
AI can be looked at from two very different perspectives. We're solving problems with deep learning methods we thought impossible just a few years ago. From this point of view AI looks fine. No one knows which is the right path to reach artificial general intelligence (AGI). Deep learning-based solutions are okay, but the ultimate goal -- which could hypothetically solve everything -- is, and always has been, AGI.
FarsTail: A Persian Natural Language Inference Dataset
Amirkhani, Hossein, AzariJafari, Mohammad, Pourjafari, Zohreh, Faridan-Jahromi, Soroush, Kouhkan, Zeinab, Amirak, Azadeh
Natural language inference (NLI) is known as one of the central tasks in natural language processing (NLP) which encapsulates many fundamental aspects of language understanding. With the considerable achievements of data-hungry deep learning methods in NLP tasks, a great amount of effort has been devoted to develop more diverse datasets for different languages. In this paper, we present a new dataset for the NLI task in the Persian language, also known as Farsi, which is one of the dominant languages in the Middle East. This dataset, named FarsTail, includes 10,367 samples which are provided in both the Persian language as well as the indexed format to be useful for non-Persian researchers. The samples are generated from 3,539 multiple-choice questions with the least amount of annotator interventions in a way similar to the SciTail dataset. A carefully designed multi-step process is adopted to ensure the quality of the dataset. We also present the results of traditional and state-of-the-art methods on FarsTail including different embedding methods such as word2vec, fastText, ELMo, BERT, and LASER, as well as different modeling approaches such as DecompAtt, ESIM, HBMP, and ULMFiT to provide a solid baseline for the future research. The best obtained test accuracy is 83.38% which shows that there is a big room for improving the current methods to be useful for real-world NLP applications in different languages. We also investigate the extent to which the models exploit superficial clues, also known as dataset biases, in FarsTail, and partition the test set into easy and hard subsets according to the success of biased models. The dataset is available at https://github.com/dml-qom/FarsTail
A Dataset and Method for Hallux Valgus Angle Estimation Based on Deep Learing
Xu, Ningyuan, Zhuang, Jiayan, Wu, Yaojun, Xiao, Jiangjian
Angular measurements is essential to make a resonable treatment for Hallux valgus (HV), a common forefoot deformity. However, it still depends on manual labeling and measurement, which is time-consuming and sometimes unreliable. Automating this process is a thing of concern. However, it lack of dataset and the keypoints based method which made a great success in pose estimation is not suitable for this field.To solve the problems, we made a dataset and developed an algorithm based on deep learning and linear regression. It shows great fitting ability to the ground truth.
End-to-end Malaria Diagnosis and 3D Cell Rendering with Deep Learning
Malaria is a parasitic infection that poses a significant burden on global health. It kills one child every 30 seconds and over one million people annually. If diagnosed in a timely manner, however, most people can be effectively treated with antimalarial therapy. Several deaths due to malaria are byproducts of disparities in the social determinants of health; the current gold standard for diagnosing malaria requires microscopes, reagents, and other equipment that most patients of low socioeconomic brackets do not have access to. In this paper, we propose a convolutional neural network (CNN) architecture that allows for rapid automated diagnosis of malaria (achieving a high classification accuracy of 98%), as well as a deep neural network (DNN) based three-dimensional (3D) modeling algorithm that renders 3D models of parasitic cells in augmented reality (AR). This creates an opportunity to optimize the current workflow for malaria diagnosis and demonstrates potential for deep learning models to improve telemedicine practices and patient health literacy on a global scale. Our website is accessible here.
Ensembles of Randomized NNs for Pattern-based Time Series Forecasting
In this work, we propose an ensemble forecasting approach based on randomized neural networks. Improved randomized learning streamlines the fitting abilities of individual learners by generating network parameters in accordance with the data and target function features. A pattern-based representation of time series makes the proposed approach suitable for forecasting time series with multiple seasonality. We propose six strategies for controlling the diversity of ensemble members. Case studies conducted on four real-world forecasting problems verified the effectiveness and superior performance of the proposed ensemble forecasting approach. It outperformed statistical models as well as state-of-the-art machine learning models in terms of forecasting accuracy. The proposed approach has several advantages: fast and easy training, simple architecture, ease of implementation, high accuracy and the ability to deal with nonstationarity and multiple seasonality in time series.
Does Form Follow Function? An Empirical Exploration of the Impact of Deep Neural Network Architecture Design on Hardware-Specific Acceleration
Abbasi, Saad, Shafiee, Mohammad Javad, Chan, Ellick, Wong, Alexander
The fine-grained relationship between form and function with respect to deep neural network architecture design and hardware-specific acceleration is one area that is not well studied in the research literature, with form often dictated by accuracy as opposed to hardware function. In this study, a comprehensive empirical exploration is conducted to investigate the impact of deep neural network architecture design on the degree of inference speedup that can be achieved via hardware-specific acceleration. More specifically, we empirically study the impact of a variety of commonly used macro-architecture design patterns across different architectural depths through the lens of OpenVINO microprocessor-specific and GPU-specific acceleration. Experimental results showed that while leveraging hardware-specific acceleration achieved an average inference speed-up of 380%, the degree of inference speed-up varied drastically depending on the macro-architecture design pattern, with the greatest speedup achieved on the depthwise bottleneck convolution design pattern at 550%. Furthermore, we conduct an in-depth exploration of the correlation between FLOPs requirement, level 3 cache efficacy, and network latency with increasing architectural depth and width. Finally, we analyze the inference time reductions using hardware-specific acceleration when compared to native deep learning frameworks across a wide variety of hand-crafted deep convolutional neural network architecture designs as well as ones found via neural architecture search strategies. We found that the DARTS-derived architecture to benefit from the greatest improvement from hardware-specific software acceleration (1200%) while the depthwise bottleneck convolution-based MobileNet-V2 to have the lowest overall inference time of around 2.4 ms.
Learning to Delegate for Large-scale Vehicle Routing
Li, Sirui, Yan, Zhongxia, Wu, Cathy
Vehicle routing problems (VRPs) are a class of combinatorial problems with wide practical applications. While previous heuristic or learning-based works achieve decent solutions on small problem instances of up to 100 customers, their performance does not scale to large problems. This article presents a novel learning-augmented local search algorithm to solve large-scale VRP. The method iteratively improves the solution by identifying appropriate subproblems and $\textit{delegating}$ their improvement to a black box subsolver. At each step, we leverage spatial locality to consider only a linear number of subproblems, rather than exponential. We frame subproblem selection as a regression problem and train a Transformer on a generated training set of problem instances. We show that our method achieves state-of-the-art performance, with a speed-up of up to 15 times over strong baselines, on VRPs with sizes ranging from 500 to 3000.