Deep Learning: Instructional Materials
University of San Francisco School of Law becomes first law program to fully integrate AI, as Anthropic goes big on education
Anthropic, the mind behind ChatGPT competitor Claude, is joining the industry-wide charge into education, as the tech company announces a new university and classroom partnerships that will put their educational chatbot into the hands of students of all ages. Announced today, Claude for Education will be entering more classrooms and boosting its peer-reviewed knowledge bank, as it integrates with teaching and learning software Canvas, textbook and courseware company Wiley, and video learning tool Panopto. "We're building toward a future where students can reference readings, lecture recordings, visualizations, and textbook content directly within their conversations," the company explained. Students and educators can connect Wiley and Panopto materials to Claude's data base using pre-built MCP servers, says Anthropic, and access Claude directly in the Canvas coursework platform. In summary: students can use Claude like a personal study partner.
The AI Industry is Funding A Massive AI Training Initiative for Teachers
AI tools have become deeply embedded in how many students learn and complete schoolwork--and that usage is only poised to increase. On Tuesday, the American Federation of Teachers announced an AI training hub for educators, backed by 23 million from Microsoft, OpenAI, and Anthropic. The AFT is the second-largest teachers' union, representing 1.8 million teachers and educational staffers across the country. Their training hub will open in New York City this fall, featuring workshops that will educate teachers on how to use AI tools for tasks like generating lesson plans and quizzes, or writing emails to parents. Microsoft is providing 12.5 million for AI teacher training over the next five years.
Microsoft, OpenAI, Anthropic announce free AI academy with national teachers union
The nation's largest teachers' union -- representing millions of staff within America's education system -- has joined forces with some of the world's top players in AI to ready another generation of tech-savvy educators. Announced Tuesday, July 8, by the American Federation of Teachers (AFT) and New York City-based affiliate United Federation of Teachers, along with tech giants Microsoft, OpenAI, and Anthropic, the new National Academy for AI Instruction will funnel 23 million toward free AI training and curriculum for all 1.8 million union members. The goal of the program and its brick-and-mortar Manhattan facility -- the brainchild of venture capitalist Roy Bahat and modeled after other high-tech training centers -- is to create a "national model for AI-integrated curriculum," according to the coalition, focused on skills-based workshops, online courses, and hands-on training. Microsoft will invest 12.5 million into the training program, with an additional 8 million in funding from OpenAI and 500,000 from Anthropic, the New York Times reports. OpenAI will also provide 2 million in technical resources.
EnsIR: An Ensemble Algorithm for Image Restoration via Gaussian Mixture Models Shangquan Sun 1,2 Hyunhee Park 6
Image restoration has experienced significant advancements due to the development of deep learning. Nevertheless, it encounters challenges related to ill-posed problems, resulting in deviations between single model predictions and ground-truths. Ensemble learning, as a powerful machine learning technique, aims to address these deviations by combining the predictions of multiple base models. Most existing works adopt ensemble learning during the design of restoration models, while only limited research focuses on the inference-stage ensemble of pre-trained restoration models. Regression-based methods fail to enable efficient inference, leading researchers in academia and industry to prefer averaging as their choice for post-training ensemble.
DeepMath - Deep Sequence Models for Premise Selection
Geoffrey Irving, Christian Szegedy, Alexander A. Alemi, Niklas Een, Francois Chollet, Josef Urban
We study the effectiveness of neural sequence models for premise selection in automated theorem proving, one of the main bottlenecks in the formalization of mathematics. We propose a two stage approach for this task that yields good results for the premise selection task on the Mizar corpus while avoiding the handengineered features of existing state-of-the-art models. To our knowledge, this is the first time deep learning has been applied to theorem proving on a large scale.
Direct Feedback Alignment Provides Learning in Deep Neural Networks
Artificial neural networks are most commonly trained with the back-propagation algorithm, where the gradient for learning is provided by back-propagating the error, layer by layer, from the output layer to the hidden layers. A recently discovered method called feedback-alignment shows that the weights used for propagating the error backward don't have to be symmetric with the weights used for propagation the activation forward. In fact, random feedback weights work evenly well, because the network learns how to make the feedback useful. In this work, the feedback alignment principle is used for training hidden layers more independently from the rest of the network, and from a zero initial condition. The error is propagated through fixed random feedback connections directly from the output layer to each hidden layer. This simple method is able to achieve zero training error even in convolutional networks and very deep networks, completely without error backpropagation. The method is a step towards biologically plausible machine learning because the error signal is almost local, and no symmetric or reciprocal weights are required. Experiments show that the test performance on MNIST and CIFAR is almost as good as those obtained with back-propagation for fully connected networks. If combined with dropout, the method achieves 1.45% error on the permutation invariant MNIST task.
Online Adaptation of Language Models with a Memory of Amortized Contexts Jihoon Tack, Eric Mitchell
Due to the rapid generation and dissemination of information, large language models (LLMs) quickly run out of date despite enormous development costs. To address the crucial need to keep models updated, online learning has emerged as a critical tool when utilizing LLMs for real-world applications. However, given the ever-expanding corpus of unseen documents and the large parameter space of modern LLMs, efficient adaptation is essential. To address these challenges, we propose Memory of Amortized Contexts (MAC), an efficient and effective online adaptation framework for LLMs with strong knowledge retention. We propose a feature extraction and memory-augmentation approach to compress and extract information from new documents into compact modulations stored in a memory bank.
Towards Multi-dimensional Explanation Alignment for Medical Classification
The lack of interpretability in the field of medical image analysis has significant ethical and legal implications. Existing interpretable methods in this domain encounter several challenges, including dependency on specific models, difficulties in understanding and visualization, as well as issues related to efficiency. To address these limitations, we propose a novel framework called Med-MICN (Medical Multidimensional Interpretable Concept Network). Med-MICN provides interpretability alignment for various angles, including neural symbolic reasoning, concept semantics, and saliency maps, which are superior to current interpretable methods. Its advantages include high prediction accuracy, interpretability across multiple dimensions, and automation through an end-to-end concept labeling process that reduces the need for extensive human training effort when working with new datasets. To demonstrate the effectiveness and interpretability of Med-MICN, we apply it to four benchmark datasets and compare it with baselines. The results clearly demonstrate the superior performance and interpretability of our Med-MICN.
Towards General Loop Invariant Generation: A Benchmark of Programs with Memory Manipulation Chang Liu
Program verification is vital for ensuring software reliability, especially in the context of increasingly complex systems. Loop invariants, remaining true before and after each iteration of loops, are crucial for this verification process. Traditional provers and machine learning based methods for generating loop invariants often require expert intervention or extensive labeled data, and typically only handle numerical property verification.