Knowledge distillation (KD) is a popular method for reducing the computational overhead of deep network inference, in which the output of a teacher model is used to train a smaller, faster student model. Hint training (i.e., FitNets) extends KD by regressing a student model's intermediate representation to a teacher model's intermediate representation. In this work, we introduce bLock-wise Intermediate representation Training (LIT), a novel model compression technique that extends the use of intermediate representations in deep network compression, outperforming KD and hint training. LIT has two key ideas: 1) LIT trains a student of the same width (but shallower depth) as the teacher by directly comparing the intermediate representations, and 2) LIT uses the intermediate representation from the previous block in the teacher model as an input to the current student block during training, avoiding unstable intermediate representations in the student network. We show that LIT provides substantial reductions in network depth without loss in accuracy -- for example, LIT can compress a ResNeXt-110 to a ResNeXt-20 (5.5) on CIFAR10 and a VDCNN-29 to a VDCNN-9 (3.2) on Amazon Reviews without loss in accuracy, outperforming KD and hint training in network size for a given accuracy. We also show that applying LIT to identical student/teacher architectures increases the accuracy of the student model above the teacher model, outperforming the recently-proposed Born Again Networks procedure on ResNet, ResNeXt, and VDCNN. Finally, we show that LIT can effectively compress GAN generators, which are not supported in the KD framework because GANs output pixels as opposed to probabilities. Modern deep networks have achieved increased accuracy by continuing to introduce more layers (Ioffe & Szegedy, 2015; He et al., 2016) at the cost of higher computational overhead. In response, researchers have proposed many techniques to reduce this computational overhead at inference time, which broadly fall under two categories. First, in deep compression (Han et al., 2015; Zhu et al., 2016; Li et al., 2016; Hubara et al., 2017), parts of a model are removed or quantized to reduce the number of weights and/or the computational footprint.

Machine learning libraries are becoming faster and more accessible with each passing year, showing no signs of slowing down. While traditionally Python has been the go-to language for machine learning, nowadays neural networks can run in any language, including JavaScript! The web ecosystem has made a lot of progress in recent times and although JavaScript and Node.js are still less performant than Python and Java, they are now powerful enough to handle many machine learning problems. Web languages also have the advantage of being super accessible - all you need to run a JavaScript ML project is your web browser. Most JavaScript machine learning libraries are fairly new and still in development, but they do exist and are ready for you to try them.

It's amazing to think that, in barely more than a decade, we've gone from the Keystone Cops to self-driving cars safely navigating the streets of many cities. A big part of that story is the amazing power of simply setting a clear goal and attaching a prize to it. I remember how graduate students and professors on the MIT team became so engrossed in the DARPA Grand Challenge, they forgot to eat and sleep. Humans can be incredibly dedicated and successful when we have a target in mind.

At kindergarten, 3-year-old Seven Kong has his schoolmates to play with, but at home his best friend is a kidney-shaped, lime-coloured android named BeanQ. The two spend hours together, with Seven peppering the robot with a continuous stream of questions. The green android responds with similarly simple words and phrases, alongside an array of different emoji facial expressions displayed on a large screen, which serves as its face. Recommended to the family by a friend, the android is intended to be an early educator, sharing some of the parental burden. "When we get really busy, BeanQ can be there keeping him entertained," said Seven's mother Liu Qian, 33, who is a work-at-home mom living in Beijing.

During the spring semester of my junior year in college, I had the opportunity to study abroad in Copenhagen, Denmark. I had never been to Europe before that, so I was incredibly excited to immerse into a new culture, meet new people, travel to new places, and, most important, encounter a new language. Now although English is not my native language (Vietnamese is), I have learned and spoken it since early childhood that it has become second-nature. Danish, on the other hand, is an incredibly complicated language with very different sentence structure and grammatical made-ups. Before my trip, I tried to learn a bit of Danish using the app Duolingo; however, I only got a hold of simple phrases such as Hello (Hej) and Good Morning (God Morgen).

BINA48, a robot owned by lawyer, author, and entrepreneur Martine Aliana Rothblatt's Terrasem Movement Foundation (TMF) is living the life of any normal college student in America: She attends classes at Notre Dame de Namur University in Belmont, California, and every once in a while gets excused from those classes when she needs to get a facelift at Hanson Robotics. She also got excused from class that one time she was invited to ring the bell at the stock exchange, so her schedule isn't exactly like other students'. BINA48, who looks like a live flesh-and-blood woman, at least from the head to the shoulders, is now pursuing a rather interesting subject at the university, given that she is a work of artificial intelligence (AI): a Philosophy of Love course. For a final project, she and a human student presented philosophical perspectives on love, showing the world that a robot can have thoughts and views on the subject. As of the fall of 2017, BINA48 became the first robot to complete a college class.

While everywhere else there is a fear of technology taking over people's job, China, the world leader in the digital economy, thinks it has the capacity to bring a change. The latest research from Dentsu Aegis Network (DAN) shows that people in China are the most optimistic about the impact of digital technology and artificial intelligence (AI) on work and life. "The answer may lie in the belief of people in China that their education system is providing them with the tools and knowledge to succeed in the future, however uncertain or disruptive that may be," says Susana Tsui, the chief executive officer of Dentsu Aegis Network China, in an email interview. The report highlights that 68 per cent of people in China think that their formal education (e.g. Tsui gives the credit of formalized education system to the Chinese government.

Suky Kang and Daniel Jun Lee were married Sept. 29 at Volunteer Park in Seattle. Gholamreza Assadi, the bride's brother-in-law who became a Universal Life minister for this event, officiated. Ms. Kang, 29, is the director of international programs at Code.org, a nonprofit organization in Seattle that champions computer science education in kindergarten through Grade 12. She graduated from Harvard and received an master's of arts degree from the Bard Graduate Center in New York. She is a daughter of Ki Ok Kang and Shin Woo Kang of Greensboro, N.C.

Batch Normalization (BN) improves both convergence and generalization in training neural networks. This work understands these phenomena theoretically. We analyze BN by using a basic block of neural networks, consisting of a kernel layer, a BN layer, and a nonlinear activation function. This basic network helps us understand the impacts of BN in three aspects. First, by viewing BN as an implicit regularizer, BN can be decomposed into population normalization (PN) and gamma decay as an explicit regularization. Second, learning dynamics of BN and the regularization show that training converged with large maximum and effective learning rate. Third, generalization of BN is explored by using statistical mechanics. Experiments demonstrate that BN in convolutional neural networks share the same traits of regularization as the above analyses.

Deep Reinforcement Learning (DRL) has obtained unprecedented results in decisionmaking problems, such as playing Atari games [1], or beating the world champion in GO [2]. Nevertheless, in robotic problems, DRL is still limited in applications with real-world systems [3]. Most of the tasks that have been successfully addressed with DRL have two common characteristics: 1) they have well-specified reward functions, and 2) they require large amounts of trials, which means long training periods (or powerful computers) to obtain a satisfying behavior. These two characteristics can be problematic in cases where 1) the goals of the tasks are poorly defined or hard to specify/model (reward function does not exist), 2) the execution of many trials is not feasible (real systems case) and/or not much computational power or time is available, and 3) sometimes additional external perception is necessary for computing the reward/cost function. On the other hand, Machine Learning methods that rely on transfer of human knowledge, Interactive Machine Learning (IML) methods, have shown to be time efficient for obtaining good performance policies and may not require a well-specified reward function; moreover, some methods do not need expert human teachers for training high performance agents [4-6].