This paper addresses learning end-to-end models for time series data that include a temporal alignment step via dynamic time warping (DTW). Existing approaches to differentiable DTW either differentiate through a fixed warping path or apply a differentiable relaxation to the min operator found in the recursive steps used to solve the DTW problem. We instead propose a DTW layer based around bi-level optimisation and deep declarative networks, which we name DecDTW. By formulating DTW as a continuous, inequality constrained optimisation problem, we can compute gradients for the solution of the optimal alignment (with respect to the underlying time series) using implicit differentiation. An interesting byproduct of this formulation is that DecDTW outputs the optimal warping path between two time series as opposed to a soft approximation, recoverable from Soft-DTW. We show that this property is particularly useful for applications where downstream loss functions are defined on the optimal alignment path itself. This naturally occurs, for instance, when learning to improve the accuracy of predicted alignments against ground truth alignments. We evaluate DecDTW on two such applications, namely the audio-to-score alignment task in music information retrieval and the visual place recognition task in robotics, demonstrating state-of-the-art results in both.

The traditional framework of federated learning (FL) requires each client to re-train their models in every iteration, making it infeasible for resource-constrained mobile devices to train deep-learning (DL) models. Split learning (SL) provides an alternative by using a centralized server to offload the computation of activations and gradients for a subset of the model but suffers from problems of slow convergence and lower accuracy. In this paper, we implement PFSL, a new framework of distributed split learning where a large number of thin clients perform transfer learning in parallel, starting with a pre-trained DL model without sharing their data or labels with a central server. We implement a lightweight step of personalization of client models to provide high performance for their respective data distributions. Furthermore, we evaluate performance fairness amongst clients under a work fairness constraint for various scenarios of non-i.i.d. data distributions and unequal sample sizes. Our accuracy far exceeds that of current SL algorithms and is very close to that of centralized learning on several real-life benchmarks. It has a very low computation cost compared to FL variants and promises to deliver the full benefits of DL to extremely thin, resource-constrained clients.

Building systems that are good for society in the face of complex societal effects requires a dynamic approach. Recent approaches to machine learning (ML) documentation have demonstrated the promise of discursive frameworks for deliberation about these complexities. However, these developments have been grounded in a static ML paradigm, leaving the role of feedback and post-deployment performance unexamined. Meanwhile, recent work in reinforcement learning has shown that the effects of feedback and optimization objectives on system behavior can be wide-ranging and unpredictable. In this paper we sketch a framework for documenting deployed and iteratively updated learning systems, which we call Reward Reports. Taking inspiration from various contributions to the technical literature on reinforcement learning, we outline Reward Reports as living documents that track updates to design choices and assumptions behind what a particular automated system is optimizing for. They are intended to track dynamic phenomena arising from system deployment, rather than merely static properties of models or data. After presenting the elements of a Reward Report, we discuss a concrete example: Meta's BlenderBot 3 chatbot. Several others for game-playing (DeepMind's MuZero), content recommendation (MovieLens), and traffic control (Project Flow) are included in the appendix.

Improving automated analysis of medical imaging will provide clinicians more options in providing care for patients. The 2023 AI-enabled Medical Image Analysis Workshop and Covid-19 Diagnosis Competition (AI-MIA-COV19D) provides an opportunity to test and refine machine learning methods for detecting the presence and severity of COVID-19 in patients from CT scans. This paper presents version 2 of Cov3d, a deep learning model submitted in the 2022 competition. The model has been improved through a preprocessing step which segments the lungs in the CT scan and crops the input to this region. It results in a validation macro F1 score for predicting the presence of COVID-19 in the CT scans at 93.2% which is significantly above the baseline of 74\%. It gives a macro F1 score for predicting the severity of COVID-19 on the validation set for task 2 as 72.8% which is above the baseline of 38%.

Survivors of sexual harassment frequently share their experiences on social media, revealing their feelings and emotions and seeking advice. We observed that on Reddit, survivors regularly share long posts that describe a combination of (i) a sexual harassment incident, (ii) its effect on the survivor, including their feelings and emotions, and (iii) the advice being sought. We term such posts MeToo posts, even though they may not be so tagged and may appear in diverse subreddits. A prospective helper (such as a counselor or even a casual reader) must understand a survivor's needs from such posts. But long posts can be time-consuming to read and respond to. Accordingly, we address the problem of extracting key information from a long MeToo post. We develop a natural language-based model to identify sentences from a post that describe any of the above three categories. On ten-fold cross-validation of a dataset, our model achieves a macro F1 score of 0.82. In addition, we contribute MeThree, a dataset comprising 8,947 labeled sentences extracted from Reddit posts. We apply the LIWC-22 toolkit on MeThree to understand how different language patterns in sentences of the three categories can reveal differences in emotional tone, authenticity, and other aspects.

The scaling of large language models has greatly improved natural language understanding, generation, and reasoning. In this work, we develop a system that trained a trillion-parameter language model on a cluster of Ascend 910 AI processors and MindSpore framework, and present the language model with 1.085T parameters named PanGu-{\Sigma}. With parameter inherent from PanGu-{\alpha}, we extend the dense Transformer model to sparse one with Random Routed Experts (RRE), and efficiently train the model over 329B tokens by using Expert Computation and Storage Separation(ECSS). This resulted in a 6.3x increase in training throughput through heterogeneous computing. Our experimental findings show that PanGu-{\Sigma} provides state-of-the-art performance in zero-shot learning of various Chinese NLP downstream tasks. Moreover, it demonstrates strong abilities when fine-tuned in application data of open-domain dialogue, question answering, machine translation and code generation.

When covering investment and personal finance stories, we aim to inform our readers rather than recommend specific financial product or asset classes. While we may highlight certain positives of a financial product or asset class, there is no guarantee that readers will benefit from the product or investment approach and may, in fact, make a loss if they acquire the product or adopt the approach. To the extent any recommendations or statements of opinion or fact made in a story may constitute financial advice, they constitute general information and not personal financial advice in any form. As such, any recommendations or statements do not take into account the financial circumstances, investment objectives, tax implications, or any specific requirements of readers. Readers of our stories should not act on any recommendation without first taking appropriate steps to verify the information in the stories consulting their independent financial adviser in order to ascertain whether the recommendation (if any) is appropriate, having regard to their investment objectives, financial situation and particular needs.

This week: TikTok challenges Google and Microsoft with search ads, GPT-4 is on the way, and social media engagement rates are dropping. Here's what happened this week in digital marketing. OpenAI hasn't been in the news enough lately so it's time for a fresh update. The next version of GPT, unimaginatively called GPT-4, will go live soon. In fact, it might already be live by the time you read this. As far as the updates that make it more worthwhile than GPT-3, it's got multimodal functionality. That means it supports text, speech, images, and even video. GPT-4 also works across multiple languages. If you've noticed that your social media engagement rates are on the decline, you're not alone.

In this post, I compute each batsman's or bowler's Win Probability Contribution (WPC) in a T20 match. This metric captures by how much the player (batsman or bowler) changed/impacted the Win Probability of the T20 match. For this computation I use my machine learning models, I had created earlier, which predicts the ball-by-ball win probability as the T20 match progresses through the 2 innings of the match. In my previous posts I had created several Machine Learning models. In order to compute the player's Win Probability contribution in this post, I have used the following ML models The player's contribution is calculated as the difference in win probability when the batsman faces the 1st ball in his innings and the last ball either when is out or the innings comes to an end.

Former U.S. Amb. to NATO Kurt Volker says the Russian fighter jet collision was'intentional' and requires a'firm response' from the U.S. The United States has resumed its normal flights through international waters over the Black Sea following the crash of a drone due to Russian interference. U.S. officials said Friday that a RQ-4 Global Hawk flew through the region -- the first U.S. aircraft to do so since the skirmish, according to Reuters. An RQ-4 Global Hawk takes off from Andersen Air Force Base, Guam (U.S. Air Force photo/Senior Airman Nichelle Anderson) Military officials assured the public that the Russian harassment of the US drone on Tuesday would not affect regular operations in the region. Defense Secretary Lloyd Austin summarized the incident Wednesday in a press conference, saying, "Two Russian jets dumped fuel on an unmanned U.S. MQ-9 aircraft conducting routine operations in international airspace. And one Russian jet intercepted and hit our MQ-9 aircraft, resulting in a crash."