Deep Learning
Need Artificial Intelligence Training? Try these Resources.
This artificial intelligence training includes machine learning and deep learning. If you don't know it already you are most likely using artificial intelligence. If you use Alexa, Siri or talk into any machine and get a response, that is most likely artificial intelligence. Netflix and Amazon use artificial intelligence to recommend items you may like. Facebook is using artificial intelligence for facial recognition.
Deep Learning is UnAmerican
The fundamental principle of Deep Learning is that truth, maybe even Truth, lies in a big puddle of unparsed data, and a sufficiently speedy computer that can zip through and turn the data into other data comprised of relationships will eventuate into some reflection of this t/Truth. Seems to make enough sense on the face of it, but nobody has asked the question: Can human-sized truths be extracted from planet-sized datasets? Or: What is the relationship between the dataset and the individual? In the currently-fashionable model, the individual does not exist except as, at best, a cluster of datapoints, and then, only in and as a relationship. All fine and well, maybe even quite metaphysically Buddhist, but since when did we have Buddhist metaphysician computers influencing public thinking?
Classifier Chains: A Review and Perspectives
Read, Jesse, Pfahringer, Bernhard, Holmes, Geoff, Frank, Eibe
The family of methods collectively known as classifier chains has become a popular approach to multi-label learning problems. This approach involves linking together off-the-shelf binary classifiers in a chain structure, such that class label predictions become features for other classifiers. Such methods have proved flexible and effective and have obtained state-of-the-art empirical performance across many datasets and multi-label evaluation metrics. This performance led to further studies of how exactly it works, and how it could be improved, and in the recent decade numerous studies have explored classifier chains mechanisms on a theoretical level, and many improvements have been made to the training and inference procedures, such that this method remains among the state-of-the-art options for multi-label learning. Given this past and ongoing interest, which covers a broad range of applications and research themes, the goal of this work is to provide a review of classifier chains, a survey of the techniques and extensions provided in the literature, as well as perspectives for this approach in the domain of multi-label classification in the future. We conclude positively, with a number of recommendations for researchers and practitioners, as well as outlining a number of areas for future research.
A Survey of Deep Reinforcement Learning in Video Games
Shao, Kun, Tang, Zhentao, Zhu, Yuanheng, Li, Nannan, Zhao, Dongbin
Deep reinforcement learning (DRL) has made great achievements since proposed. Generally, DRL agents receive high-dimensional inputs at each step, and make actions according to deep-neural-network-based policies. This learning mechanism updates the policy to maximize the return with an end-to-end method. In this paper, we survey the progress of DRL methods, including value-based, policy gradient, and model-based algorithms, and compare their main techniques and properties. Besides, DRL plays an important role in game artificial intelligence (AI). We also take a review of the achievements of DRL in various video games, including classical Arcade games, first-person perspective games and multi-agent real-time strategy games, from 2D to 3D, and from single-agent to multi-agent. A large number of video game AIs with DRL have achieved super-human performance, while there are still some challenges in this domain. Therefore, we also discuss some key points when applying DRL methods to this field, including exploration-exploitation, sample efficiency, generalization and transfer, multi-agent learning, imperfect information, and delayed spare rewards, as well as some research directions.
Academic Performance Estimation with Attention-based Graph Convolutional Networks
Student's academic performance prediction empowers educational technologies including academic trajectory and degree planning, course recommender systems, early warning and advising systems. Given a student's past data (such as grades in prior courses), the task of student's performance prediction is to predict a student's grades in future courses. Academic programs are structured in a way that prior courses lay the foundation for future courses. The knowledge required by courses is obtained by taking multiple prior courses, which exhibits complex relationships modeled by graph structures. Traditional methods for student's performance prediction usually neglect the underlying relationships between multiple courses; and how students acquire knowledge across them. In addition, traditional methods do not provide interpretation for predictions needed for decision making. In this work, we propose a novel attention-based graph convolutional networks model for student's performance prediction. We conduct extensive experiments on a real-world dataset obtained from a large public university. The experimental results show that our proposed model outperforms state-of-the-art approaches in terms of grade prediction. The proposed model also shows strong accuracy in identifying students who are at-risk of failing or dropping out so that timely intervention and feedback can be provided to the student.
Deep Learning Training with Simulated Approximate Multipliers
Hammad, Issam, El-Sankary, Kamal, Gu, Jason
This paper presents by simulation how approximate multipliers can be utilized to enhance the training performance of convolutional neural networks (CNNs). Approximate multipliers have significantly better performance in terms of speed, power, and area compared to exact multipliers. However, approximate multipliers have an inaccuracy which is defined in terms of the Mean Relative Error (MRE). To assess the applicability of approximate multipliers in enhancing CNN training performance, a simulation for the impact of approximate multipliers error on CNN training is presented. The paper demonstrates that using approximate multipliers for CNN training can significantly enhance the performance in terms of speed, power, and area at the cost of a small negative impact on the achieved accuracy. Additionally, the paper proposes a hybrid training method which mitigates this negative impact on the accuracy. Using the proposed hybrid method, the training can start using approximate multipliers then switches to exact multipliers for the last few epochs. Using this method, the performance benefits of approximate multipliers in terms of speed, power, and area can be attained for a large portion of the training stage. On the other hand, the negative impact on the accuracy is diminished by using the exact multipliers for the last epochs of training.
DeGAN : Data-Enriching GAN for Retrieving Representative Samples from a Trained Classifier
Addepalli, Sravanti, Nayak, Gaurav Kumar, Chakraborty, Anirban, Babu, R. Venkatesh
DeGAN: Data-Enriching GAN for Retrieving Representative Samples from a Trained Classifier Sravanti Addepalli, Gaurav Kumar Nayak, Anirban Chakraborty, R. V enkatesh Babu Department of Computational and Data Sciences Indian Institute of Science, Bangalore, India {sravantia, gauravnayak, anirban, venky } @iisc.ac.in Abstract In this era of digital information explosion, an abundance of data from numerous modalities is being generated as well as archived everyday. However, most problems associated with training Deep Neural Networks still revolve around lack of data that is rich enough for a given task. Data is required not only for training an initial model, but also for future learning tasks such as Model Compression and Incremental Learning. A diverse dataset may be used for training an initial model, but it may not be feasible to store it throughout the product life cycle due to data privacy issues or memory constraints. We propose to bridge the gap between the abundance of available data and lack of relevant data, for the future learning tasks of a given trained network. We use the available data, that may be an imbalanced subset of the original training dataset, or a related domain dataset, to retrieve representative samples from a trained classifier, using a novel Data-enriching GAN (DeGAN) framework. We demonstrate that data from a related domain can be leveraged to achieve state-of-the-art performance for the tasks of Data-free Knowledge Distillation and Incremental Learning on benchmark datasets. We further demonstrate that our proposed framework can enrich any data, even from unrelated domains, to make it more useful for the future learning tasks of a given network. 1 Introduction The performance and generalizability of Deep Neural Networks largely depend on the amount and quality of training data available. Several successful implementations of tasks such as classification, object detection and segmentation leverage very large, class-balanced and diverse datasets. In addition to training an initial network, data is also required for future updates to the model. This makes it important for training data to be available throughout the life cycle of a product. While data collection is a challenge in itself, storing the data for future use could also be a concern due to data confidentiality constraints, privacy issues, or memory costs.
Efficient Training of Deep Classifiers for Wireless Source Identification using Test SNR Estimates
Wang, Xingchen, Ju, Shengtai, Zhang, Xiwen, Ramjee, Sharan, Gamal, Aly El
We investigate the potential of training time reduction for deep learning algorithms that process received wireless signals, if an accurate test Signal to Noise Ratio (SNR) estimate is available. Our focus is on two tasks that facilitate source identification: 1- Identifying the modulation type, 2- Identifying the wireless technology and channel index in the 2.4 GHZ ISM band. For benchmarking, we rely on a fast growing recent literature on testing deep learning algorithms against two well-known synthetic datasets. We first demonstrate that using training data corresponding only to the test SNR value leads to dramatic reductions in training time - that can reach up to 35x - while incurring a small loss in average test accuracy, as it improves the accuracy for low test SNR values. Further, we show that an erroneous test SNR estimate with a small positive offset is better for training than another having the same error magnitude with a negative offset. Secondly, we introduce a greedy training SNR Boosting algorithm that leads to uniform improvement in test accuracy across all tested SNR values, while using only a small subset of training SNR values at each test SNR. Finally, we discuss, with empirical evidence, the potential of bootstrap aggregating (Bagging) based on training SNR values to improve generalization at low test SNR
A Comparative Study on Machine Learning Algorithms for the Control of a Wall Following Robot
Hammad, Issam, El-Sankary, Kamal, Gu, Jason
A comparison of the performance of various machine learning models to predict the direction of a wall following robot is presented in this paper. The models were trained using an open-source dataset that contains 24 ultrasound sensors readings and the corresponding direction for each sample. This dataset was captured using SCITOS G5 mobile robot by placing the sensors on the robot waist. In addition to the full format with 24 sensors per record, the dataset has two simplified formats with 4 and 2 input sensor readings per record. Several control models were proposed previously for this dataset using all three dataset formats. In this paper, two primary research contributions are presented. First, presenting machine learning models with accuracies higher than all previously proposed models for this dataset using all three formats. A perfect solution for the 4 and 2 inputs sensors formats is presented using Decision Tree Classifier by achieving a mean accuracy of 100%. On the other hand, a mean accuracy of 99.82% was achieves using the 24 sensor inputs by employing the Gradient Boost Classifier. Second, presenting a comparative study on the performance of different machine learning and deep learning algorithms on this dataset. Therefore, providing an overall insight on the performance of these algorithms for similar sensor fusion problems. All the models in this paper were evaluated using Monte-Carlo cross-validation.