Convolutional architectures have proven extremely successful for vision tasks. Their hard inductive biases enable sample-efficient learning, but come at the cost of a potentially lower performance ceiling. Vision Transformers (ViTs) rely on more flexible self-attention layers, and have recently outperformed CNNs for image classification. However, they require costly pre-training on large external datasets or distillation from pre-trained convolutional networks. In this paper, we ask the following question: is it possible to combine the strengths of these two architectures while avoiding their respective limitations? To this end, we introduce gated positional self-attention (GPSA), a form of positional self-attention which can be equipped with a "soft" convolutional inductive bias. We initialize the GPSA layers to mimic the locality of convolutional layers, then give each attention head the freedom to escape locality by adjusting a gating parameter regulating the attention paid to position versus content information. The resulting convolutional-like ViT architecture, ConViT, outperforms the DeiT on ImageNet, while offering a much improved sample efficiency. We further investigate the role of locality in learning by first quantifying how it is encouraged in vanilla self-attention layers, then analyzing how it is escaped in GPSA layers. We conclude by presenting various ablations to better understand the success of the ConViT. Our code and models are released publicly.

Survival analysis is a technique to predict the times of specific outcomes, and is widely used in predicting the outcomes for intensive care unit (ICU) trauma patients. Recently, deep learning models have drawn increasing attention in healthcare. However, there is a lack of deep learning methods that can model the relationship between measurements, clinical notes and mortality outcomes. In this paper we introduce BERTSurv, a deep learning survival framework which applies Bidirectional Encoder Representations from Transformers (BERT) as a language representation model on unstructured clinical notes, for mortality prediction and survival analysis. We also incorporate clinical measurements in BERTSurv. With binary cross-entropy (BCE) loss, BERTSurv can predict mortality as a binary outcome (mortality prediction). With partial log-likelihood (PLL) loss, BERTSurv predicts the probability of mortality as a time-to-event outcome (survival analysis). We apply BERTSurv on Medical Information Mart for Intensive Care III (MIMIC III) trauma patient data. For mortality prediction, BERTSurv obtained an area under the curve of receiver operating characteristic curve (AUC-ROC) of 0.86, which is an improvement of 3.6% over baseline of multilayer perceptron (MLP) without notes. For survival analysis, BERTSurv achieved a concordance index (C-index) of 0.7. In addition, visualizations of BERT's attention heads help to extract patterns in clinical notes and improve model interpretability by showing how the model assigns weights to different inputs.

A composable infrastructure is defined as resources, such as compute, storage, accelerators and networking, that are shared in a pool and that can be grouped in various configurations to meet application requirements. This freedom to 'mix and match' resources dynamically allows for experimentation early in the design cycle, prior to the final architectural design or hardware implementation of a system. This design provides flexibility to serve a variety of workloads and provides a dynamic co-design platform that allows experiments and measurements in a controlled manner. For instance, key performance bottlenecks can be revealed early on in the experimentation phase thus avoiding costly and time consuming mistakes. Additionally, various system-level topologies can be evaluated when experimenting with new System on Chip (SoCs) and new accelerator types. This paper details the design of an enterprise composable infrastructure that we have implemented and made available to our partners in the IBM Research AI Hardware Center (AIHC). Our experimental evaluations on the composable system give insights into how the system works and evaluates the impact of various resource aggregations and reconfigurations on representative deep learning benchmarks.

Understanding the information contained in the increasing repository of data is of vital importance to behavior sciences [34], which aim to predict human decision making and enable wide applications, such as mental health evaluation [14], business recommendation [33], opinion mining [54], and entertainment assistance [78]. Analyzing media data on an affective (emotional) level belongs to affective computing, which is defined as "the computing that relates to, arises from, or influences emotions" [38]. The importance of emotions has been emphasized for decades since Minsky introduced the relationship between intelligence and emotion [31]. One famous claim is "The question is not whether intelligent machines can have any emotions, but whether machines can be intelligent without emotions." Based on the types of media data, the research on affective computing can be classified into different categories, such as text [13, 72], image [75], speech [45], music [64], facial expression [24], video [56, 79], physiological signals [2], and multi-modal data [52, 41, 80]. The adage "a picture is worth a thousand words" indicates that images can convey rich semantics. Therefore, images are used as an important channel to express emotions. Image emotion analysis (IEA) has recently been paid much attention. As compared to analyzing the images' cognitive aspect that is related with objective content [15], such as object classification and semantic segmentation, IEA focuses on understanding what emotions can be induced by the images in viewers.

Skeletal motion plays a vital role in human activity recognition as either an independent data source or a complement. The robustness of skeleton-based activity recognizers has been questioned recently, which shows that they are vulnerable to adversarial attacks when the full-knowledge of the recognizer is accessible to the attacker. However, this white-box requirement is overly restrictive in most scenarios and the attack is not truly threatening. In this paper, we show that such threats do exist under black-box settings too. To this end, we propose the first black-box adversarial attack method BASAR. Through BASAR, we show that adversarial attack is not only truly a threat but also can be extremely deceitful, because on-manifold adversarial samples are rather common in skeletal motions, in contrast to the common belief that adversarial samples only exist off-manifold. Through exhaustive evaluation and comparison, we show that BASAR can deliver successful attacks across models, data, and attack modes. Through harsh perceptual studies, we show that it achieves effective yet imperceptible attacks. By analyzing the attack on different activity recognizers, BASAR helps identify the potential causes of their vulnerability and provides insights on what classifiers are likely to be more robust against attack.

Canadian boffins are testing semi-autonomous exoskeletons that could help people with limited mobility walk again without the need for implanted sensors. Researchers at the University of Waterloo, Ontario, are hard at work trying to combine modern deep-learning systems with robotic prostheses. They hope to give disabled patients who have suffered spinal cord injuries or strokes, or are inflicted with conditions including multiple sclerosis, spinal, cerebral palsy, and osteoarthritis, the ability to get back on their feet and move freely. The project differs from other efforts for amputees that involve trying to control the movement of machines using electrodes implanted in nerves and muscles in the limbs and brain, explained Brock Laschowski, a PhD student at the university who is leading the ExoNet study. "Our control approach wouldn't necessarily require human thought. Similar to autonomous cars that drive themselves, we're designing autonomous exoskeletons that walk for themselves."

Artificial Intelligence has changed our lives for better. Be it in the form of robots, automated cars, or voice based applications like Alexa and Siri, we have seen it all. Without a doubt, AI is that one technology that makes the best use of human intelligence to take up tasks that earlier could only be performed by humans. Machines now stand the potential to learn and put the knowledge gained in the best possible use. All the human-like tasks are now performed using AI.

Two disciplines familiar to econometricians, factor analysis of equities returns and machine learning, have grown up alongside each other. Used in tandem, these fields of study can build effective investment-management tools, according to City University of Hong Kong's Guanhao Feng (a graduate of Chicago Booth's PhD Program), Booth's Nicholas Polson, and Booth PhD candidate Jianeng Xu. The researchers set out to determine whether they could create a deep-learning model to automate the management of a portfolio built on buying stocks that are expected to rise and short selling those that are expected to fall, known as a long-short strategy. They created a machine-learning algorithm that built a long-short equity portfolio from the top and bottom 20 percent of a 3,000-stock universe. They ranked the equities using the five-factor model of Chicago Booth's Eugene F. Fama and Dartmouth's Kenneth R. French.

Adversarial examples are a hot research topic in deep learning nowadays. Subtle, often invisible changes in the data can push our networks to make terrible mistakes. We, as human beings, seem to be way more resilient to these perturbations in our sensory inputs (though not totally immune). There is a certain pattern in our deep learning systems. They achieve remarkable things, but they are also at times delicate and brittle.

One morning in the spring of 2019, I entered a pastry shop in the Ueno train station, in Tokyo. After taking a tray and tongs at the front, you browsed, plucking what you liked from heaps of baked goods. What first struck me was the selection, which seemed endless: there were croissants, turnovers, Danishes, pies, cakes, and open-faced sandwiches piled up everywhere, sometimes in dozens of varieties. But I was most surprised when I got to the register. At the urging of an attendant, I slid my items onto a glowing rectangle on the counter. A nearby screen displayed an image, shot from above, of my doughnuts and Danish.