“Data is like garbage! You’d better know what you’re going to do with it before you collect it.” ~ Mark TwainThis is not a typical reference book for descriptive statistics. Instead, we take an in-depth look at what methods you would be foolish not to use in data science or machine learning. In this book, we dive into descriptive statistics with Python and an overview of what is crucial to know to obtain the most advantage of what we show you. Descriptive statistics is essential for decision making based on data. Using descriptive statistics will give you a way to make straightforward decisions on your decision making without complex methodology. Descriptive statistics form the fundamental platform for every quantitative data analysis."Diving into Descriptive Statistics with Python" is a book by Pratik Shukla and Roberto Iriondo. Between us, we have worked together for the past year to create this material and prepare you for straightforward, data-driven decision making. Please know that this is an experience-based, opinionated book. We only cover topics that will help you make the most out of using descriptive statistics for data-driven decision making.Book Sample:You can access a sample of this book through this article or this PDF. No email address needed.Contents:Population and SampleProbability Sampling TechniquesSimple Random SamplingSystematic SamplingStratified SamplingClustered SamplingNon-probability Sampling TechniqueConvenience SamplingQuota SamplingJudgemental SamplingSnowball SamplingWhat is Statistics?Importance of Statistics In Data-science and Machine LearningTypes of StatisticsMeasure of Central TendencyArithmetic MeanWeighted MeanMean of Categorical DataGeometric MeanHarmonic MeanMedianModeMeasure of Spread/DispersionQuantilesPercentile in DetailRangeInterquartile Range(IQR)Box and Whisker PlotVarianceStandard DeviationDegree of FreedomMean DeviationCoefficient of VariationSkewnessKurtosisCovariance and CorrelationMomentsStandard ErrorConfidence IntervalStem and Leaf diagramDot PlotsFrequency DistributionRelative FrequencyCumulative Relative FrequencyAbout the Authors:Pratik Shukla is a machine learning engineer with Towards AI. He is pursuing his master's degree in computer science in the US starting in 2021. His current goals are motivated by the purpose of learning something new and remarkable every day. His research interests lie in machine learning and its applications, especially in astronomy and astrophysics. Previously, he received his B.Tech. from Gujarat Technological University, and his work has been featured in many places, from KDNuggets, Nightingale, and others. Roberto Iriondo is the founder of Towards AI, a globally recognized publication and software company, and a front-end engineer at Carnegie Mellon University. As a builder and strategist by heart, his work has helped several companies to achieve their business goals and needs, from Anyscale, Superb AI, Determined AI, Lambda, Udacity, and many others.FAQ:What's the refund policy?We thank you for supporting Towards AI. If what you see is not what you expected, please reply to the download email within 30 days, and you'll get a full refund.Can I share this book with my team?This version is for individual use only, but we are working on a team license to share with your team, class, or organization.Resources:Google Colab. Github.DISCLAIMER: The views expressed in this book are those of the author(s) and do not represent the views of any company (directly or indirectly) associated with the author(s). This book does not intend to be a final product, yet rather a reflection of current thinking along with being a catalyst for discussion and improvement.

Artificial intelligence (AI) is here, and its benefits are seemingly limitless. There is a flip side though--there always is. AI experts, and those involved with AI, are concerned that if we do not proceed with caution, some of the strange things predicted in science-fiction movies such as 2001: A Space Odyssey may be more truth than fiction. Elon Musk told The New York Times that his experience with AI at Tesla allows him to say with confidence, "We're headed toward a situation where AI is vastly smarter than humans." He adds, "That doesn't mean everything goes to hell in five years. It just means that things get unstable or weird."

An artificial intelligence taught itself to manipulate blocks by setting increasingly difficult goals. Unlike human students, computers don't seem to get bored or frustrated when a lesson is too easy or too hard. But just like humans, they do better when a lesson plan is "just right" for their level of skill. Coming up with the right curricula isn't easy, though, so computer scientists wondered: What if they could make machines design their own? That's what researchers have done in several new studies, creating artificial intelligence (AI) that can figure out how best to teach itself.

Tabatha Plew quit her good-paying construction job in August, pulled her kids out of a Central Valley school they loved and moved seven hours north to this tiny town in Trinity County. Like a lot of rural communities, Weaverville in recent years has seen more people leaving than arriving, but it had a golden commodity Plew couldn't find at home in Fresno County for her three children: open classrooms that promised a desk in front of a teacher. "I packed them up, and I told my husband, 'We love you. See you on the weekends,'" said Plew, who moved into her in-laws' home in Weaverville. "This was the highest-paying job I've ever had, and, you know, the money didn't mean anything when my kids were struggling."

Pretrained language models have significantly improved the performance of down-stream language understanding tasks, including extractive question answering, by providing high-quality contextualized word embeddings. However, learning question answering models still need large-scaled data annotation in specific domains. In this work, we propose a cooperative, self-play learning framework, REGEX, for question generation and answering. REGEX is built upon a masked answer extraction task with an interactive learning environment containing an answer entity REcognizer, a question Generator, and an answer EXtractor. Given a passage with a masked entity, the generator generates a question around the entity, and the extractor is trained to extract the masked entity with the generated question and raw texts. The framework allows the training of question generation and answering models on any text corpora without annotation. We further leverage a reinforcement learning technique to reward generating high-quality questions and to improve the answer extraction model's performance. Experiment results show that REGEX outperforms the state-of-the-art (SOTA) pretrained language models and zero-shot approaches on standard question-answering benchmarks, and yields the new SOTA performance under the zero-shot setting.

Training an agent to navigate with DRL is data-hungry, which requires millions of training steps. Besides, the DRL agents performing well in training scenarios are found to perform poorly in some unseen real-world scenarios. In this paper, we discuss why the DRL agent fails in such unseen scenarios and find the representation of LiDAR readings is the key factor behind the agent's performance degradation. Moreover, we propose an easy, but efficient input pre-processing (IP) approach to accelerate training and enhance the performance of the DRL agent in such scenarios. The proposed IP functions can highlight the important short-distance values of laser scans and compress the range of less-important long-distance values. Extensive comparative experiments are carried out, and the experimental results demonstrate the high performance of the proposed IP approaches.

Reducing the variance of the gradient estimator is known to improve the convergence rate of stochastic gradient-based optimization and sampling algorithms. One way of achieving variance reduction is to design importance sampling strategies. Recently, the problem of designing such schemes was formulated as an online learning problem with bandit feedback, and algorithms with sub-linear static regret were designed. In this work, we build on this framework and propose Avare, a simple and efficient algorithm for adaptive importance sampling for finite-sum optimization and sampling with decreasing step-sizes. Under standard technical conditions, we show that Avare achieves $\mathcal{O}(T^{2/3})$ and $\mathcal{O}(T^{5/6})$ dynamic regret for SGD and SGLD respectively when run with $\mathcal{O}(1/t)$ step sizes. We achieve this dynamic regret bound by leveraging our knowledge of the dynamics defined by the algorithm, and combining ideas from online learning and variance-reduced stochastic optimization. We validate empirically the performance of our algorithm and identify settings in which it leads to significant improvements.

Algorithmic decision-making (ADM) increasingly shapes people's daily lives. Given that such autonomous systems can cause severe harm to individuals and social groups, fairness concerns have arisen. A human-centric approach demanded by scholars and policymakers requires taking people's fairness perceptions into account when designing and implementing ADM. We provide a comprehensive, systematic literature review synthesizing the existing empirical insights on perceptions of algorithmic fairness from 39 empirical studies spanning multiple domains and scientific disciplines. Through thorough coding, we systemize the current empirical literature along four dimensions: (a) algorithmic predictors, (b) human predictors, (c) comparative effects (human decision-making vs. algorithmic decision-making), and (d) consequences of ADM. While we identify much heterogeneity around the theoretical concepts and empirical measurements of algorithmic fairness, the insights come almost exclusively from Western-democratic contexts. By advocating for more interdisciplinary research adopting a society-in-the-loop framework, we hope our work will contribute to fairer and more responsible ADM.

Employing paraphrasing tools to conceal plagiarized text is a severe threat to academic integrity. To enable the detection of machine-paraphrased text, we evaluate the effectiveness of five pre-trained word embedding models combined with machine learning classifiers and state-of-the-art neural language models. We analyze preprints of research papers, graduation theses, and Wikipedia articles, which we paraphrased using different configurations of the tools SpinBot and SpinnerChief. The best performing technique, Longformer, achieved an average F1 score of 80.99% (F1=99.68% for SpinBot and F1=71.64% for SpinnerChief cases), while human evaluators achieved F1=78.4% for SpinBot and F1=65.6% for SpinnerChief cases. We show that the automated classification alleviates shortcomings of widely-used text-matching systems, such as Turnitin and PlagScan. To facilitate future research, all data, code, and two web applications showcasing our contributions are openly available.

In this work, we study the phenomenon of catastrophic forgetting in the graph representation learning scenario. The primary objective of the analysis is to understand whether classical continual learning techniques for flat and sequential data have a tangible impact on performances when applied to graph data. To do so, we experiment with a structure-agnostic model and a deep graph network in a robust and controlled environment on three different datasets. The benchmark is complemented by an investigation on the effect of structure-preserving regularization techniques on catastrophic forgetting. We find that replay is the most effective strategy in so far, which also benefits the most from the use of regularization. Our findings suggest interesting future research at the intersection of the continual and graph representation learning fields. Finally, we provide researchers with a flexible software framework to reproduce our results and carry out further experiments.