Education
On Evaluating the Generalization of LSTM Models in Formal Languages
Suzgun, Mirac, Belinkov, Yonatan, Shieber, Stuart M.
Recurrent Neural Networks (RNNs) are theoretically Turing-complete and established themselves as a dominant model for language processing. Yet, there still remains an uncertainty regarding their language learning capabilities. In this paper, we empirically evaluate the inductive learning capabilities of Long Short-Term Memory networks, a popular extension of simple RNNs, to learn simple formal languages, in particular $a^nb^n$, $a^nb^nc^n$, and $a^nb^nc^nd^n$. We investigate the influence of various aspects of learning, such as training data regimes and model capacity, on the generalization to unobserved samples. We find striking differences in model performances under different training settings and highlight the need for careful analysis and assessment when making claims about the learning capabilities of neural network models.
CommonsenseQA: A Question Answering Challenge Targeting Commonsense Knowledge
Talmor, Alon, Herzig, Jonathan, Lourie, Nicholas, Berant, Jonathan
When answering a question, people often draw upon their rich world knowledge in addition to some task-specific context. Recent work has focused primarily on answering questions based on some relevant document or content, and required very little general background. To investigate question answering with prior knowledge, we present CommonsenseQA: a difficult new dataset for commonsense question answering. To capture common sense beyond associations, each question discriminates between three target concepts that all share the same relationship to a single source drawn from ConceptNet (Speer et al., 2017). This constraint encourages crowd workers to author multiple-choice questions with complex semantics, in which all candidates relate to the subject in a similar way. We create 9,500 questions through this procedure and demonstrate the dataset's difficulty with a large number of strong baselines. Our best baseline, the OpenAI GPT (Radford et al., 2018), obtains 54.8% accuracy, well below human performance, which is 95.3%.
Artificial intelligence, or the end of the world as we know it DW 26.10.2018
That's one of the surprising -- and unsettling -- questions Israeli historian Yuval Noah Harari asks in his much-quoted new book, 21 Lessons for the 21st Century. Whereas 20th-century technology favored democracies as they were able to distribute power to make decisions among many people and institutions, according to Harari, artificial intelligence (AI) might make centralized systems that concentrate all information and power far more efficient as machine learning works better with more information to analyze. "If you disregard all privacy concerns and concentrate all the information relating to a billion people in one database," Harari writes, "you'll wind up with much better algorithms than if you respect individual privacy and have in your database only partial information on a million people." The rise of AI swinging the pendulum from democracies toward authoritarian regimes is just one of the feared adverse impacts of technologies: Others include job displacement, concentration of power, diminishing privacy, rising income inequality and losing our "free will." Yet most people have little or no knowledge about how AI, blockchain, the Internet of Things or genetic engineering could affect their lives.
Machine Learning โ Intelligent Decisions based on Data โ Witan World
While artificial intelligence (AI) is the broad science of mimicking human abilities, machine learning is a specific subset of AI that trains a machine how to learn. Watch this video to better understand the relationship between AI and machine learning. The rising popularity of Machine learning is because of the same factors that have made data mining and Bayesian analysis more popular than ever. Things like growing volumes and varieties of available data, computational processing that is cheaper and more powerful, and affordable data storage. With Machine learning, it's possible to quickly and automatically produce models that can analyze bigger, more complex data and deliver faster, more accurate results even on a very large scale.
Bots in learning - AI and personalized learning experience
Instructors often teach a class of 30 or more students, making it difficult to give proper help and attention to each student. Every student learns at a different pace and in their own way. Many instructors use computer programs to allow students to work at a pace they find comfortable. However, these programs make it easy for students to "beat the system," not truly learn. Students often develop an extrinsic attitude toward learning in this setting.
Bots in learning - AI and personalized learning experience
Instructors often teach a class of 30 or more students, making it difficult to give proper help and attention to each student. Every student learns at a different pace and in their own way. Many instructors use computer programs to allow students to work at a pace they find comfortable. However, these programs make it easy for students to "beat the system," not truly learn. Students often develop an extrinsic attitude toward learning in this setting.
Machine learning with Python: Essential hacks and tricks
It's never been easier to get started with machine learning. In addition to structured massive open online courses (MOOCs), there are a huge number of incredible, free resources available around the web. Here are a few that have helped me. Familiarity and moderate expertise in at least one high-level programming language is useful for beginners in machine learning. Unless you are a Ph.D. researcher working on a purely theoretical proof of some complex algorithm, you are expected to mostly use the existing machine learning algorithms and apply them in solving novel problems. This requires you to put on a programming hat.
On Exploration, Exploitation and Learning in Adaptive Importance Sampling
Lu, Xiaoyu, Rainforth, Tom, Zhou, Yuan, van de Meent, Jan-Willem, Teh, Yee Whye
We study adaptive importance sampling (AIS) as an online learning problem and argue for the importance of the trade-off between exploration and exploitation in this adaptation. Borrowing ideas from the bandits literature, we propose Daisee, a partition-based AIS algorithm. We further introduce a notion of regret for AIS and show that Daisee has $\mathcal{O}(\sqrt{T}(\log T)^{\frac{3}{4}})$ cumulative pseudo-regret, where $T$ is the number of iterations. We then extend Daisee to adaptively learn a hierarchical partitioning of the sample space for more efficient sampling and confirm the performance of both algorithms empirically.
Online Learning Algorithms for Statistical Arbitrage
Arbitrage is the risk-free method of making profit from exploiting price differences in different markets. For example, if one stock is trading at a higher price in one market than another, one could buy the stock for the lower price on one market and sell it for the higher price on the other, thereby making profit without taking risks. These pricing disparities have become increasingly hard to capitalize on as they only appear for very short periods of time with the advancements in technology and highfrequency trading. Only those who can recognize and take advantage of arbitrage opportunities first can benefit, turning it into a winner-takes-all situation. This has made it difficult to make consistent profit from price discrepancies, as one needs to recognize them quickly and be the first to leverage them. Yet, arbitrage is a necessary tool in the marketplace as it quickly eliminates market inefficiencies and keeps prices uniform across markets [2, 5, 11, 6, 3, 17].