Since visual perception can give rich information beyond text descriptions for world understanding, there has been increasing interest in leveraging visual grounding for language learning. Recently, vokenization (Tan and Bansal, 2020) has attracted attention by using the predictions of a text-to-image retrieval model as labels for language model supervision. Despite its success, the method suffers from approximation error of using finite image labels and the lack of vocabulary diversity of a small image-text dataset. To overcome these limitations, we present VidLanKD, a video-language knowledge distillation method for improving language understanding. We train a multi-modal teacher model on a video-text dataset, and then transfer its knowledge to a student language model with a text dataset.

Question answering (QA) models for reading comprehension have been demonstrated to exploit unintended dataset biases such as question-context lexical overlap. This hinders QA models from generalizing to under-represented samples such as questions with low lexical overlap. Question generation (QG), a method for augmenting QA datasets, can be a solution for such performance degradation if QG can properly debias QA datasets. However, we discover that recent neural QG models are biased towards generating questions with high lexical overlap, which can amplify the dataset bias. Moreover, our analysis reveals that data augmentation with these QG models frequently impairs the performance on questions with low lexical overlap, while improving that on questions with high lexical overlap. To address this problem, we use a synonym replacement-based approach to augment questions with low lexical overlap. We demonstrate that the proposed data augmentation approach is simple yet effective to mitigate the degradation problem with only 70k synthetic examples. Our data is publicly available at https://github.com/KazutoshiShinoda/Synonym-Replacement.

Operations in many essential industries including finance and banking are often characterized by the need to perform repetitive sequential tasks. Despite their criticality to the business, workflows are rarely fully automated or even formally specified, though there may exist a number of natural language documents describing these procedures for the employees of the company. Plan extraction methods provide us with the possibility of extracting structure plans from such natural language descriptions of the plans/workflows, which could then be leveraged by an automated system. In this paper, we investigate the utility of generalized language models in performing such extractions directly from such texts. Such models have already been shown to be quite effective in multiple translation tasks, and our initial results seem to point to their effectiveness also in the context of plan extractions. Particularly, we show that GPT-3 is able to generate plan extraction results that are comparable to many of the current state of the art plan extraction methods.

Existing methods for vision-and-language learning typically require designing task-specific architectures and objectives for each task. For example, a multi-label answer classifier for visual question answering, a region scorer for referring expression comprehension, and a language decoder for image captioning, etc. To alleviate these hassles, in this work, we propose a unified framework that learns different tasks in a single architecture with the same language modeling objective, i.e., multimodal conditional text generation, where our models learn to generate labels in text based on the visual and textual inputs. On 7 popular vision-and-language benchmarks, including visual question answering, referring expression comprehension, visual commonsense reasoning, most of which have been previously modeled as discriminative tasks, our generative approach (with a single unified architecture) reaches comparable performance to recent task-specific state-of-the-art vision-and-language models. Moreover, our generative approach shows better generalization ability on answering questions that have rare answers. In addition, we show that our framework allows multi-task learning in a single architecture with a single set of parameters, which achieves similar performance to separately optimized single-task models. Our code will be publicly available at: https://github.com/j-min/VL-T5

Pull requests are a key part of the collaborative software development and code review process today. However, pull requests can also slow down the software development process when the reviewer(s) or the author do not actively engage with the pull request. In this work, we design an end-to-end service, Nudge, for accelerating overdue pull requests towards completion by reminding the author or the reviewer(s) to engage with their overdue pull requests. First, we use models based on effort estimation and machine learning to predict the completion time for a given pull request. Second, we use activity detection to reduce false positives. Lastly, we use dependency determination to understand the blocker of the pull request and nudge the appropriate actor(author or reviewer(s)). We also do a correlation analysis to understand the statistical relationship between the pull request completion times and various pull request and developer related attributes. Nudge has been deployed on 147 repositories at Microsoft since 2019. We do a large scale evaluation based on the implicit and explicit feedback we received from sending the Nudge notifications on 8,500 pull requests. We observe significant reduction in completion time, by over 60%, for pull requests which were nudged thus increasing the efficiency of the code review process and accelerating the pull request progression.

Policy gradients-based reinforcement learning has proven to be a promising approach for directly optimizing non-differentiable evaluation metrics for language generation tasks. However, optimizing for a specific metric reward leads to improvements in mostly that metric only, suggesting that the model is gaming the formulation of that metric in a particular way without often achieving real qualitative improvements. Hence, it is more beneficial to make the model optimize multiple diverse metric rewards jointly. While appealing, this is challenging because one needs to manually decide the importance and scaling weights of these metric rewards. Further, it is important to consider using a dynamic combination and curriculum of metric rewards that flexibly changes over time. Considering the above aspects, in our work, we automate the optimization of multiple metric rewards simultaneously via a multi-armed bandit approach (DORB), where at each round, the bandit chooses which metric reward to optimize next, based on expected arm gains. We use the Exp3 algorithm for bandits and formulate two approaches for bandit rewards: (1) Single Multi-reward Bandit (SM-Bandit); (2) Hierarchical Multi-reward Bandit (HM-Bandit). We empirically show the effectiveness of our approaches via various automatic metrics and human evaluation on two important NLG tasks: question generation and data-to-text generation, including on an unseen-test transfer setup. Finally, we present interpretable analyses of the learned bandit curriculum over the optimized rewards.

Traceable, a startup developing an end-to-end cloud app security solution, today emerged from stealth with $20 million in venture equity financing. Newly flush with capital, CEO Jyoti Bansal intends to focus on acquiring customers globally while growing Traceable's team and accelerating R&D. Cloud-native apps are often built with hundreds or even thousands of API microservices (i.e., loosely coupled services), making them difficult to protect at scale. Gartner predicts that by 2022, API abuses will be the most frequent attack vector, which isn't surprising considering API calls represented 83% of web traffic as of 2018. Traceable ostensibly protects these APIs with machine learning algorithms that analyze app activity from the user and the session all the way down to the code.

This ebook, based on the latest ZDNet / TechRepublic special feature, explores how infrastructure around the world is being linked together via sensors, machine learning and analytics. The Industrial Internet of Things (IIoT) revolution is no longer coming -- it's here. While the regular IoT brings connectivity to consumer gadgets like smartphones, wearables, and appliances, the IIoT connects machines and devices in industries such as manufacturing, healthcare, retail, and more. By joining big data and analytics with machine learning technology, the IIoT can help industrial employees gain insights from data collected by their machines. This information can help business cut costs, increase safety, eliminate inefficiencies, and improve customer service -- particularly in the manufacturing sector.

Walk into any one of the many start-up events organized across India, and inevitably the image of an Indian bazaar comes to mind: people rushing around, shouting, bargaining, answering phones with great excitement, laughing loudly, boasting, blushing, and generally being optimistic, as if they are at the beginning of a rising trend of well-being. Such optimism might seem justified. According to data compiled by Fortune magazine,a from just eight'unicorns' in 2015, the number of start-ups in India valued at more than $1 billion has grown to 26. What is interesting is that in 2018 alone, India added eight unicorns to the club. These include diverse entities such as Ola, started in India as a competitor to Uber and has since expanded its footprint into the U.K. (and is eyeing Australia); an insurance aggregator called PolicyBazaar; the e-commerce site Paytm Mall; an eyewear retailer called Lenskart; food technology aggregators such as Swiggy and Zomato, and hotel-room aggregators like OYO and FabHotels. Thousands of entrepreneurs start up every year and aspire to become one of the new unicorns.

As companies shift to CI/CD (continuous integration/continuous delivery), they face a problem around monitoring and fixing problems in builds that have been deployed. How do you deal with an issue after moving onto the next delivery milestone? Harness, the startup launched last year by AppDynamics founder Jyoti Bansal, wants to fix that with a new tool called 24 7 Service Guard. The new tool is designed to help companies working with a continuous delivery process by monitoring all of the builds, regardless of when they were launched. What's more, the company claims that using AI and machine learning, it can dial back a problematic build to one that worked in an automated fashion, freeing developers and operations to keep working without worry.