South America
Extracting or Guessing? Improving Faithfulness of Event Temporal Relation Extraction
Wang, Haoyu, Zhang, Hongming, Deng, Yuqian, Gardner, Jacob R., Roth, Dan, Chen, Muhao
In this paper, we seek to improve the faithfulness of TempRel extraction models from two perspectives. The first perspective is to extract genuinely based on contextual description. To achieve this, we propose to conduct counterfactual analysis to attenuate the effects of two significant types of training biases: the event trigger bias and the frequent label bias. We also add tense information into event representations to explicitly place an emphasis on the contextual description. The second perspective is to provide proper uncertainty estimation and abstain from extraction when no relation is described in the text. By parameterization of Dirichlet Prior over the model-predicted categorical distribution, we improve the model estimates of the correctness likelihood and make TempRel predictions more selective. We also employ temperature scaling to recalibrate the model confidence measure after bias mitigation. Through experimental analysis on MATRES, MATRES-DS, and TDDiscourse, we demonstrate that our model extracts TempRel and timelines more faithfully compared to SOTA methods, especially under distribution shifts.
Continual Training of Language Models for Few-Shot Learning
Ke, Zixuan, Lin, Haowei, Shao, Yijia, Xu, Hu, Shu, Lei, Liu, Bing
Recent work on applying large language models (LMs) achieves impressive performance in many NLP applications. Adapting or posttraining an LM using an unlabeled domain corpus can produce even better performance for end-tasks in the domain. This paper proposes the problem of continually extending an LM by incrementally post-train the LM with a sequence of unlabeled domain corpora to expand its knowledge without forgetting its previous skills. The goal is to improve the few-shot end-task learning in these domains. The resulting system is called CPT (Continual PostTraining), which to our knowledge, is the first continual post-training system. Experimental results verify its effectiveness.
Semantic Segmentation under Adverse Conditions: A Weather and Nighttime-aware Synthetic Data-based Approach
Kerim, Abdulrahman, Chamone, Felipe, Ramos, Washington, Marcolino, Leandro Soriano, Nascimento, Erickson R., Jiang, Richard
Recent semantic segmentation models perform well under standard weather conditions and sufficient illumination but struggle with adverse weather conditions and nighttime. Collecting and annotating training data under these conditions is expensive, time-consuming, error-prone, and not always practical. Usually, synthetic data is used as a feasible data source to increase the amount of training data. However, just directly using synthetic data may actually harm the model's performance under normal weather conditions while getting only small gains in adverse situations. Therefore, we present a novel architecture specifically designed for using synthetic training data for domain adaptation. We propose a simple yet powerful addition to DeepLabV3+ by using weather and time-of-the-day supervisors trained with multi-task learning, making it both weather and nighttime aware, which improves its mIoU accuracy by $14$ percentage points on the ACDC dataset while maintaining a score of $75\%$ mIoU on the Cityscapes dataset. Our code is available at https://github.com/lsmcolab/Semantic-Segmentation-under-Adverse-Conditions.
FusionDeepMF: A Dual Embedding based Deep Fusion Model for Recommendation
Mandal, Supriyo, Maiti, Abyayananda
Traditional Collaborative Filtering (CF) based methods are applied to understand the personal preferences of users/customers for items or products from the rating matrix. Usually, the rating matrix is sparse in nature. So there are some improved variants of the CF method that apply the increasing amount of side information to handle the sparsity problem. Only linear kernel or only non-linear kernel is applied in most of the available recommendation-related work to understand user-item latent feature embeddings from data. Only linear kernel or only non-linear kernel is not sufficient to learn complex user-item features from side information of users. Recently, some researchers have focused on hybrid models that learn some features with non-linear kernels and some other features with linear kernels. But it is very difficult to understand which features can be learned accurately with linear kernels or with non-linear kernels. To overcome this problem, we propose a novel deep fusion model named FusionDeepMF and the novel attempts of this model are i) learning user-item rating matrix and side information through linear and non-linear kernel simultaneously, ii) application of a tuning parameter determining the trade-off between the dual embeddings that are generated from linear and non-linear kernels. Extensive experiments on online review datasets establish that FusionDeepMF can be remarkably futuristic compared to other baseline approaches. Empirical evidence also shows that FusionDeepMF achieves better performances compared to the linear kernels of Matrix Factorization (MF) and the non-linear kernels of Multi-layer Perceptron (MLP).
Patching open-vocabulary models by interpolating weights
Ilharco, Gabriel, Wortsman, Mitchell, Gadre, Samir Yitzhak, Song, Shuran, Hajishirzi, Hannaneh, Kornblith, Simon, Farhadi, Ali, Schmidt, Ludwig
Open-vocabulary models like CLIP achieve high accuracy across many image classification tasks. However, there are still settings where their zero-shot performance is far from optimal. We study model patching, where the goal is to improve accuracy on specific tasks without degrading accuracy on tasks where performance is already adequate. Towards this goal, we introduce PAINT, a patching method that uses interpolations between the weights of a model before fine-tuning and the weights after fine-tuning on a task to be patched. On nine tasks where zero-shot CLIP performs poorly, PAINT increases accuracy by 15 to 60 percentage points while preserving accuracy on ImageNet within one percentage point of the zero-shot model. PAINT also allows a single model to be patched on multiple tasks and improves with model scale. Furthermore, we identify cases of broad transfer, where patching on one task increases accuracy on other tasks even when the tasks have disjoint classes. Finally, we investigate applications beyond common benchmarks such as counting or reducing the impact of typographic attacks on CLIP. Our findings demonstrate that it is possible to expand the set of tasks on which open-vocabulary models achieve high accuracy without re-training them from scratch.
Transfer Learning with Joint Fine-Tuning for Multimodal Sentiment Analysis
de Toledo, Guilherme Lourenรงo, Marcacini, Ricardo Marcondes
Most existing methods focus on sentiment analysis of textual data. However, recently there has been a massive use of images and videos on social platforms, motivating sentiment analysis from other modalities. Current studies show that exploring other modalities (e.g., images) increases sentiment analysis performance. State-of-the-art multimodal models, such as CLIP and VisualBERT, are pre-trained on datasets with the text paired with images. Although the results obtained by these models are promising, pre-training and sentiment analysis fine-tuning tasks of these models are computationally expensive. This paper introduces a transfer learning approach using joint fine-tuning for sentiment analysis. Our proposal achieved competitive results using a more straightforward alternative fine-tuning strategy that leverages different pre-trained unimodal models and efficiently combines them in a multimodal space. Moreover, our proposal allows flexibility when incorporating any pre-trained model for texts and images during the joint fine-tuning stage, being especially interesting for sentiment classification in low-resource scenarios.
TropeTwist: Trope-based Narrative Structure Generation
Games are complex, multi-faceted systems that share common elements This paper presents TropeTwist, a preliminar system that uses and underlying narratives, such as the conflict between a Tropes [21, 54] extracted from TvTropes [26, 46] as patterns and fundamental hero and a big bad enemy or pursuing a goal that requires overcoming units, which when combined can compose structures further challenges. However, identifying and describing these elements representing other composed tropes. Common narrative structures together is non-trivial as they might differ in certain properties can be identified and defined using TropeTwist. TropeTwist and how players might encounter the narratives. Likewise, generating can define generic aspects of a story, leading to the identification of narratives also pose difficulties when encoding, interpreting, events, roles, and narrative elements, as well as a novel way to form and evaluating them. To address this, we present TropeTwist, a narratives. As a proof-of-concept, we built, analyzed, and described trope-based system that can describe narrative structures in games structurally three game examples shown in figure 1, top row. in a more abstract and generic level, allowing the definition of We propose graph grammars as indirect encoding of narrative games' narrative structures and their generation using interconnected graphs and the use of the Multi-dimensional Archive of Phenotypic tropes, called narrative graphs. To demonstrate the system, Elites (MAP-Elites) [40] to generate novel variations (shown we represent the narrative structure of three different games. in figure 1, bottom row) using the proof-of-concept examples as We use MAP-Elites to generate and evaluate novel quality-diverse roots. Simultaneously, we propose metrics to evaluate the resulting narrative graphs encoded as graph grammars, using these three narrative graphs' coherence, cohesion, and interestingness.
WinoGAViL: Gamified Association Benchmark to Challenge Vision-and-Language Models
Bitton, Yonatan, Guetta, Nitzan Bitton, Yosef, Ron, Elovici, Yuval, Bansal, Mohit, Stanovsky, Gabriel, Schwartz, Roy
While vision-and-language models perform well on tasks such as visual question answering, they struggle when it comes to basic human commonsense reasoning skills. In this work, we introduce WinoGAViL: an online game of vision-and-language associations (e.g., between werewolves and a full moon), used as a dynamic evaluation benchmark. Inspired by the popular card game Codenames, a spymaster gives a textual cue related to several visual candidates, and another player tries to identify them. Human players are rewarded for creating associations that are challenging for a rival AI model but still solvable by other human players. We use the game to collect 3.5K instances, finding that they are intuitive for humans (>90% Jaccard index) but challenging for state-of-the-art AI models, where the best model (ViLT) achieves a score of 52%, succeeding mostly where the cue is visually salient. Our analysis as well as the feedback we collect from players indicate that the collected associations require diverse reasoning skills, including general knowledge, common sense, abstraction, and more. We release the dataset, the code and the interactive game, allowing future data collection that can be used to develop models with better association abilities.
A Newton-CG based barrier method for finding a second-order stationary point of nonconvex conic optimization with complexity guarantees
In this paper we consider finding an approximate second-order stationary point (SOSP) of nonconvex conic optimization that minimizes a twice differentiable function over the intersection of an affine subspace and a convex cone. In particular, we propose a Newton-conjugate gradient (Newton-CG) based barrier method for finding an $(\epsilon,\sqrt{\epsilon})$-SOSP of this problem. Our method is not only implementable, but also achieves an iteration complexity of ${\cal O}(\epsilon^{-3/2})$, which matches the best known iteration complexity of second-order methods for finding an $(\epsilon,\sqrt{\epsilon})$-SOSP of unconstrained nonconvex optimization. The operation complexity, consisting of ${\cal O}(\epsilon^{-3/2})$ Cholesky factorizations and $\widetilde{\cal O}(\epsilon^{-3/2}\min\{n,\epsilon^{-1/4}\})$ other fundamental operations, is also established for our method.
Remote AR/VR openings near you -Updated October 10, 2022 - Remote Tech Jobs
Role requiring'No experience data provided' months of experience in None Pay if you succeed in getting hired and start work at a high-paying job first. Get Paid to Read Emails, Play Games, Search the Web, $5 Signup Bonus. Headquartered out of Germany we are a subsidiary of one of the largest global manufactures of optical components in the world. Since 1992 we have been known as a leading manufacturer of optical test equipment. After a massively successful 2021 and our recent merger with Jenoptik our business is growing faster than ever!