Education
How Continued Learning Can Help Data Scientists Solve Industry-Specific Challenges
Data scientists are, first and foremost, problem solvers. But new problems can't always be solved with old tricks. Currently organizations in every industry are experiencing overwhelming challenges, many of them emerging from shifts to digital, the need to instantly deliver new service models, or better meet changing customer priorities. This demands a commitment to lifelong learning, arguably one of the most important factors shaping any data scientist's career. Ronald van Loon is a SAS partner, and is an advocate for the power of ongoing education in helping to expedite solution development for data practitioners like data scientists, data analysts, data engineers, or ML engineers.
Can artificial intelligence solve the corporate knowledge problem?
In this new era of remote working, our reliance on technology is greater than ever, with many businesses having migrated processes from offline to online to accommodate the transition. But amongst the success stories of companies adapting to this change, working in the current climate has raised a pertinent issue for employers: how can they deliver effective learning and development (L&D) to employees when they are scattered across different workplaces? Irrespective of the shift to working from home, providing personalised and efficient L&D has been an issue for quite some time. We are in an age where we are more informed than ever, and while this is no bad thing, one of the drawbacks of this knowledge explosion is that employers are now faced with the prospect of ever-changing policies, procedures and regulations. It is vital that staff are kept up-to-date with industry developments, but this is becoming an increasingly unattainable goal.
Question Generation using Natural Language processing
Auto generate assessments in edtech like MCQs, True/False, Fill-in-the-blanks etc using state-of-the-art NLP techniques. This course focuses on using state-of-the-art Natural Language processing techniques to solve the problem of question generation in edtech. If we pick up any middle school textbook, at the end of every chapter we see assessment questions like MCQs, True/False questions, Fill-in-the-blanks, Match the following, etc. In this course, we will see how we can take any text content and generate these assessment questions using NLP techniques. This course will be a very practical use case of NLP where we put basic algorithms like word vectors (word2vec, Glove, etc) to recent advancements like BERT, openAI GPT-2, and T5 transformers to real-world use.
Deep Learning was Top In-demand Skill of 2020, List of Most Popular Nanodegree Programs
Udacity has released the list of the most popular Nanodegree programs in India in 2020. The data is based on the number of enrollments during the year, showing the demand across different states and union territories. Deep learning and data engineering were the top Nanodegree programs showing the country's growing interest towards artificial intelligence and data. While deep learning is driving advances in artificial intelligence that are changing our world, data engineering is the foundation for the new world of Big Data. There is no doubt about the fact that COVID-19 has changed the global job landscape, said a statement from the company.
Announcing Intro to Embedded Machine Learning on Coursera
Embedded machine learning applications power every modern aspect of our modern life. From Siri to fitness wearables, space rovers to medical implants, you will find tiny instructions embedded on micronized hardware that delivers magical and important experiences. At Edge Impulse, our mission is to enable any developer to use machine learning to build better products while making it easier for them to get started and be successful. We teamed up with instructor Shawn Hymel, Coursera, Arm, Arduino and the tinyML Foundation to bring you the ultimate course to get you started with embedded machine learning. This course is an important piece of our journey and mission, and we aim to reach millions of developers so they can make machine learning part of their stack.
Student-Teacher Learning from Clean Inputs to Noisy Inputs
Hong, Guanzhe, Mao, Zhiyuan, Lin, Xiaojun, Chan, Stanley H.
Feature-based student-teacher learning, a training method that encourages the student's hidden features to mimic those of the teacher network, is empirically successful in transferring the knowledge from a pre-trained teacher network to the student network. Furthermore, recent empirical results demonstrate that, the teacher's features can boost the student network's generalization even when the student's input sample is corrupted by noise. However, there is a lack of theoretical insights into why and when this method of transferring knowledge can be successful between such heterogeneous tasks. We analyze this method theoretically using deep linear networks, and experimentally using nonlinear networks. We identify three vital factors to the success of the method: (1) whether the student is trained to zero training loss; (2) how knowledgeable the teacher is on the clean-input problem; (3) how the teacher decomposes its knowledge in its hidden features. Lack of proper control in any of the three factors leads to failure of the student-teacher learning method.
Escaping Saddle Points with Stochastically Controlled Stochastic Gradient Methods
Liang, Guannan, Tong, Qianqian, Zhu, Chunjiang, Bi, Jinbo
Stochastically controlled stochastic gradient (SCSG) methods have been proved to converge efficiently to first-order stationary points which, however, can be saddle points in nonconvex optimization. It has been observed that a stochastic gradient descent (SGD) step introduces anistropic noise around saddle points for deep learning and non-convex half space learning problems, which indicates that SGD satisfies the correlated negative curvature (CNC) condition for these problems. Therefore, we propose to use a separate SGD step to help the SCSG method escape from strict saddle points, resulting in the CNC-SCSG method. The SGD step plays a role similar to noise injection but is more stable. We prove that the resultant algorithm converges to a second-order stationary point with a convergence rate of $\tilde{O}( \epsilon^{-2} log( 1/\epsilon))$ where $\epsilon$ is the pre-specified error tolerance. This convergence rate is independent of the problem dimension, and is faster than that of CNC-SGD. A more general framework is further designed to incorporate the proposed CNC-SCSG into any first-order method for the method to escape saddle points. Simulation studies illustrate that the proposed algorithm can escape saddle points in much fewer epochs than the gradient descent methods perturbed by either noise injection or a SGD step.