The ability to anticipate technical expertise and capability evolution trends globally is essential for national and global security, especially in safety-critical domains like nuclear nonproliferation (NN) and rapidly emerging fields like artificial intelligence (AI). In this work, we extend traditional statistical relational learning approaches (e.g., link prediction in collaboration networks) and formulate a problem of anticipating technical expertise and capability evolution using dynamic heterogeneous graph representations. We develop novel capabilities to forecast collaboration patterns, authorship behavior, and technical capability evolution at different granularities (e.g., scientist and institution levels) in two distinct research fields. We implement a dynamic graph transformer (DGT) neural architecture, which pushes the state-of-the-art graph neural network models by (a) forecasting heterogeneous (rather than homogeneous) nodes and edges, and (b) relying on both discrete -- and continuous -- time inputs. We demonstrate that our DGT models predict collaboration, partnership, and expertise patterns with 0.26, 0.73, and 0.53 mean reciprocal rank values for AI and 0.48, 0.93, and 0.22 for NN domains. DGT model performance exceeds the best-performing static graph baseline models by 30-80% across AI and NN domains. Our findings demonstrate that DGT models boost inductive task performance, when previously unseen nodes appear in the test data, for the domains with emerging collaboration patterns (e.g., AI). Specifically, models accurately predict which established scientists will collaborate with early career scientists and vice-versa in the AI domain.

In this paper we present new algorithms for training reduced-size nonlinear representations in the Kernel Dictionary Learning (KDL) problem. Standard KDL has the drawback of a large size of the kernel matrix when the data set is large. There are several ways of reducing the kernel size, notably Nystr\"om sampling. We propose here a method more in the spirit of dictionary learning, where the kernel vectors are obtained with a trained sparse representation of the input signals. Moreover, we optimize directly the kernel vectors in the KDL process, using gradient descent steps. We show with three data sets that our algorithms are able to provide better representations, despite using a small number of kernel vectors, and also decrease the execution time with respect to KDL.

In this paper we present a new classification method based on Dictionary Learning (DL). The main contribution consists of a kernel version of incoherent DL, derived from its standard linear counterpart. We also propose an improvement of the AK-SVD algorithm concerning the representation update. Our algorithms are tested on several popular databases of classification problems.

The age of social media has opened new opportunities for businesses. This flourishing wealth of information is outside traditional channels and frameworks of classical marketing research, including that of Marketing Mix Modeling (MMM). Textual data, in particular, poses many challenges that data analysis practitioners must tackle. Social media constitute massive, heterogeneous, and noisy document sources. Industrial data acquisition processes include some amount of ETL. However, the variability of noise in the data and the heterogeneity induced by different sources create the need for ad-hoc tools. Put otherwise, customer insight extraction in fully unsupervised, noisy contexts is an arduous task. This research addresses the challenge of fully unsupervised topic extraction in noisy, Big Data contexts. We present three approaches we built on the Variational Autoencoder framework: the Embedded Dirichlet Process, the Embedded Hierarchical Dirichlet Process, and the time-aware Dynamic Embedded Dirichlet Process. These nonparametric approaches concerning topics present the particularity of determining word embeddings and topic embeddings. These embeddings do not require transfer learning, but knowledge transfer remains possible. We test these approaches on benchmark and automotive industry-related datasets from a real-world use case. We show that our models achieve equal to better performance than state-of-the-art methods and that the field of topic modeling would benefit from improved evaluation metrics.

NATO leaders are holding their annual summit as Ukraine looks to the security alliance for support in its attempt to push back invading Russian forces. The Vilnius communique, however, while emphasising NATO's support for Ukraine, gave no clear timetable on when the country might be able to join the alliance, in a major disappointment for Ukrainian President Volodymyr Zelenskyy, who had travelled to the Lithuanian capital. "Ukraine's future is in NATO," the leaders said in the joint statement on Tuesday. "We will be in a position to extend an invitation to Ukraine to join the alliance when allies agree and conditions are met," the declaration said, without specifying the conditions. The communique also touched on the Asia Pacific, with the leaders of Australia, Japan, New Zealand and South Korea all attending as NATO allies. It said China was a challenge to NATO's interests, security and values with its "ambitions and coercive policies" triggering a furious response from Beijing. And it accused Beijing and Moscow of "mutually reinforcing attempts to undercut the rules-based international order". China has said it wants peace in Ukraine, but has not condemned Russia's full scale invasion since it began in February 2022. NATO is a defensive Alliance. It is the unique, essential and indispensable transatlantic forum to consult, coordinate and act on all matters related to our individual and collective security. We reaffirm our iron-clad commitment to defend each other and every inch of Allied territory at all times, protect our one billion citizens, and safeguard our freedom and democracy, in accordance with Article 5 of the Washington Treaty. We will continue to ensure our collective defence from all threats, no matter where they stem from, based on a 360-degree approach, to fulfil NATO's three core tasks of deterrence and defence, crisis prevention and management, and cooperative security. We adhere to international law and to the purposes and principles of the Charter of the United Nations and are committed to upholding the rules-based international order. This Summit marks a milestone in strengthening our Alliance. We look forward to our valuable exchanges with the Heads of State and Government of Australia, Japan, New Zealand, and the Republic of Korea, as well as the President of the European Council and the President of the European Commission at this Summit. We also welcome the engagements with the Foreign Ministers of Georgia and the Republic of Moldova, and with the Deputy Foreign Minister of Bosnia and Herzegovina, as we continue to consult closely on the implementation of NATO's tailored support measures. This is an historic step for Finland and for NATO. For many years, we worked closely as partners; we now stand together as Allies. NATO membership makes Finland safer, and NATO stronger. Every nation has the right to choose its own security arrangements.

In this work, we present the contribution of the BLUE team in the eRisk Lab task on searching for symptoms of depression. The task consists of retrieving and ranking Reddit social media sentences that convey symptoms of depression from the BDI-II questionnaire. Given that synthetic data provided by LLMs have been proven to be a reliable method for augmenting data and fine-tuning downstream models, we chose to generate synthetic data using ChatGPT for each of the symptoms of the BDI-II questionnaire. We designed a prompt such that the generated data contains more richness and semantic diversity than the BDI-II responses for each question and, at the same time, contains emotional and anecdotal experiences that are specific to the more intimate way of sharing experiences on Reddit. We perform semantic search and rank the sentences' relevance to the BDI-II symptoms by cosine similarity. We used two state-of-the-art transformer-based models (MentalRoBERTa and a variant of MPNet) for embedding the social media posts, the original and generated responses of the BDI-II. Our results show that using sentence embeddings from a model designed for semantic search outperforms the approach using embeddings from a model pre-trained on mental health data. Furthermore, the generated synthetic data were proved too specific for this task, the approach simply relying on the BDI-II responses had the best performance.

In this paper, we introduce a novel technique called Tree-Ring Watermarking that robustly fingerprints diffusion model outputs. Unlike existing methods that perform post-hoc modifications to images after sampling, Tree-Ring Watermarking subtly influences the entire sampling process, resulting in a model fingerprint that is invisible to humans. The watermark embeds a pattern into the initial noise vector used for sampling. These patterns are structured in Fourier space so that they are invariant to convolutions, crops, dilations, flips, and rotations. After image generation, the watermark signal is detected by inverting the diffusion process to retrieve the noise vector, which is then checked for the embedded signal. We demonstrate that this technique can be easily applied to arbitrary diffusion models, including text-conditioned Stable Diffusion, as a plug-in with negligible loss in FID. Our watermark is semantically hidden in the image space and is far more robust than watermarking alternatives that are currently deployed.

We consider minimizing a smooth function subject to a summation constraint over its variables. By exploiting a connection between the greedy 2-coordinate update for this problem and equality-constrained steepest descent in the 1-norm, we give a convergence rate for greedy selection under a proximal Polyak-Lojasiewicz assumption that is faster than random selection and independent of the problem dimension $n$. We then consider minimizing with both a summation constraint and bound constraints, as arises in the support vector machine dual problem. Existing greedy rules for this setting either guarantee trivial progress only or require $O(n^2)$ time to compute. We show that bound- and summation-constrained steepest descent in the L1-norm guarantees more progress per iteration than previous rules and can be computed in only $O(n \log n)$ time.

In contrast, transformers employ a "Scaled Dot-Product Attention" mechanism that is parallelizable. This unique attention mechanism allows for large-scale pretraining. Additionally, self-supervised pretraining paradigm such as masked language modeling onlarge unlabeled datasets enabled transformers to be trained without costly annotations. Transformer model, although originally designed for the NLP [3] domain, Transformers have witnessed adaptations in various domains such as computer vision [5, 6], remote sensing [7], time series [8], speech processing [9] and multimodal learning [10]. Consequently, modality specific surveys emerged, focusing on medical imaging [11-13] and biomedical language models [14] in the medical domain. This paper aims to provide comprehensive overview of Transformer models utilized across multiple modalities of data to address healthcare objectives. We discuss pre-training strategies to manage the lack of robust and annotated healthcare datasets. The rest of the paper is organized as follows: Section 2 discusses the strategy to search for relevant citations; Section 3 describes the architecture of the original transformer; Section 4 describes the two primary Transformer variants: the Bidirectional Encoder Representations from Transformers (BERT) and the Vision Transformer (ViT). Section 5 describes advancements in large language models (LLM), and section 6 through 12 provides a review of Transformers in healthcare.

With the rise of bidirectional encoder representations from Transformer models in natural language processing, the speech community has adopted some of their development methodologies. Therefore, the Wav2Vec models were introduced to reduce the data required to obtain state-of-the-art results. This work leverages this knowledge and improves the performance of the pre-trained speech models by simply replacing the fine-tuning dense layer with a lateral inhibition layer inspired by the biological process. Our experiments on Romanian, a low-resource language, show an average improvement of 12.5% word error rate (WER) using the lateral inhibition layer. In addition, we obtain state-of-the-art results on both the Romanian Speech Corpus and the Robin Technical Acquisition Corpus with 1.78% WER and 29.64% WER, respectively.