We introduce the algorithm Bayesian Optimization (BO) with Fictitious Play (BOFiP) for the optimization of high dimensional black box functions. BOFiP decomposes the original, high dimensional, space into several sub-spaces defined by non-overlapping sets of dimensions. These sets are randomly generated at the start of the algorithm, and they form a partition of the dimensions of the original space. BOFiP searches the original space with alternating BO, within sub-spaces, and information exchange among sub-spaces, to update the sub-space function evaluation. The basic idea is to distribute the high dimensional optimization across low dimensional sub-spaces, where each sub-space is a player in an equal interest game. At each iteration, BO produces approximate best replies that update the players belief distribution. The belief update and BO alternate until a stopping condition is met. High dimensional problems are common in real applications, and several contributions in the BO literature have highlighted the difficulty in scaling to high dimensions due to the computational complexity associated to the estimation of the model hyperparameters. Such complexity is exponential in the problem dimension, resulting in substantial loss of performance for most techniques with the increase of the input dimensionality. We compare BOFiP to several state-of-the-art approaches in the field of high dimensional black box optimization. The numerical experiments show the performance over three benchmark objective functions from 20 up to 1000 dimensions. A neural network architecture design problem is tested with 42 up to 911 nodes in 6 up to 92 layers, respectively, resulting into networks with 500 up to 10,000 weights. These sets of experiments empirically show that BOFiP outperforms its competitors, showing consistent performance across different problems and increasing problem dimensionality.

Federated learning (FL) is a privacy-preserving paradigm where multiple participants jointly solve a machine learning problem without sharing raw data. Unlike traditional distributed learning, a unique characteristic of FL is statistical heterogeneity, namely, data distributions across participants are different from each other. Meanwhile, recent advances in the interpretation of neural networks have seen a wide use of neural tangent kernel (NTK) for convergence and generalization analyses. In this paper, we propose a novel FL paradigm empowered by the NTK framework. The proposed paradigm addresses the challenge of statistical heterogeneity by transmitting update data that are more expressive than those of the traditional FL paradigms. Specifically, sample-wise Jacobian matrices, rather than model weights/gradients, are uploaded by participants. The server then constructs an empirical kernel matrix to update a global model without explicitly performing gradient descent. We further develop a variant with improved communication efficiency and enhanced privacy. Numerical results show that the proposed paradigm can achieve the same accuracy while reducing the number of communication rounds by an order of magnitude compared to federated averaging.

In this paper, we provide the first focused study on the discontinuities (aka. When investigating latent holes, existing works are exclusively centred around the encoder network and they merely explore the existence of holes. We tackle these limitations by proposing a highly efficient Tree-based Decoder-Centric (TDC) algorithm for latent hole identification, with a focal point on the text domain. In contrast to past studies, our approach pays attention to the decoder network, as a decoder has a direct impact on the model's output quality. Furthermore, we provide, for the first time, in-depth empirical analysis of the latent hole phenomenon, investigating several important aspects such as how the holes impact VAE algorithms' performance on text generation, and how the holes are distributed in the latent space.

Transformer-based models have achieved great success in various NLP, vision, and speech tasks. However, the core of Transformer, the self-attention mechanism, has a quadratic time and memory complexity with respect to the sequence length, which hinders applications of Transformer-based models to long sequences. Many approaches have been proposed to mitigate this problem, such as sparse attention mechanisms, low-rank matrix approximations and scalable kernels, and token mixing alternatives to self-attention. We propose a novel Pooling Network (PoNet) for token mixing in long sequences with linear complexity. We design multi-granularity pooling and pooling fusion to capture different levels of contextual information and combine their interactions with tokens. On the Long Range Arena benchmark, PoNet significantly outperforms Transformer and achieves competitive accuracy, while being only slightly slower than the fastest model, FNet, across all sequence lengths measured on GPUs. We also conduct systematic studies on the transfer learning capability of PoNet and observe that PoNet achieves 96.0% of the accuracy of BERT on the GLUE benchmark, outperforming FNet by 4.5% relative. Comprehensive ablation analysis demonstrates effectiveness of the designed multi-granularity pooling and pooling fusion for token mixing in long sequences and efficacy of the designed pre-training tasks for PoNet to learn transferable contextualized language representations.

We introduce SMProbLog, a generalization of the probabilistic logic programming language ProbLog. A ProbLog program defines a distribution over logic programs by specifying for each clause the probability that it belongs to a randomly sampled program, and these probabilities are mutually independent. The semantics of ProbLog is given by the success probability of a query, which corresponds to the probability that the query succeeds in a randomly sampled program. It is well-defined when each random sample uniquely determines the truth values of all logical atoms. Argumentation problems, however, represent an interesting practical application where this is not always the case. SMProbLog generalizes the semantics of ProbLog to the setting where multiple truth assignments are possible for a randomly sampled program, and implements the corresponding algorithms for both inference and learning tasks. We then show how this novel framework can be used to reason about probabilistic argumentation problems. Therefore, the key contribution of this paper are: a more general semantics for ProbLog programs, its implementation into a probabilistic programming framework for both inference and parameter learning, and a novel approach to probabilistic argumentation problems based on such framework.

We show how to apply Sobol's method of global sensitivity analysis to measure the influence exerted by a set of nodes' evidence on a quantity of interest expressed by a Bayesian network. Our method exploits the network structure so as to transform the problem of Sobol index estimation into that of marginalization inference. This way, we can efficiently compute indices for networks where brute-force or Monte Carlo based estimators for variance-based sensitivity analysis would require millions of costly samples. Moreover, our method gives exact results when exact inference is used, and also supports the case of correlated inputs. The proposed algorithm is inspired by the field of tensor networks, and generalizes earlier tensor sensitivity techniques from the acyclic to the cyclic case. We demonstrate the method on three medium to large Bayesian networks that cover the areas of project risk management and reliability engineering.

Iranian dissidents are warning of the hard-line regime's use of drones to cause instability in the region, saying it is using the technology – materials for which are being imported from China – to make up for the weaknesses of its air force. The National Council of Resistance of Iran (NCRI), an umbrella group of Iranian resistance groups that oppose the regime, released evidence in a press conference it says shows the production and utilization of unmanned aerial vehicles (UACs) for terrorist operations and for assisting its proxies in the Middle East – including aerial photographs of the alleged sites and details that have emerged from inside the country. "Our revelation today is significant because it shows that the Qods Force of the IRGC has in recent years expanded its arsenal to step up terrorism and warmongering to destabilize the region by arming its proxies with UAVs," Alireza Jafarzadeh, deputy director of the Washington office of the National Council of Resistance of Iran, told Fox News. "This is in line with the regime's nuclear defiance and its repression at home." The group alleges that the regime, which has been rocked by a slew of economic sanctions imposed by the Trump administration as well as protests at home and challenges related to its handling of the COVID-19 pandemic, has used a web of industries to spend billions of dollars to produce components or smuggle them in from foreign countries.

Leading augmented reality (AR) technology company Blippar has confirmed its commitment to putting power in the hands of creators with the launch of its WebAR SDK technology. The toolkit will empower AR creators to build their own immersive WebAR experiences from the ground up using HTML and Java coding. WebAR SDK users will have access to full 24/7 support from the Blippar team to help hone their creative campaigns, and, during its beta phase, the platform will be entirely free to use, create, and publish from – with its immersive WebAR experiences able to be accessed and shared across platforms including browsers, Facebook, TikTok, WeChat, and WhatsApp – a further step in ensuring access to AR creativity is available to everyone. Blippar's WebAR SDK includes its most advanced implementation of simultaneous location and mapping (SLAM) to date, boasting 99% accuracy on tracking when locked, with less than a 1% margin of error in angular accuracy. SLAM is a set of computer vision technologies that allow AR developers and creatives to build much more interactive, immersive, and realistic AR experiences by using the device camera to create a mesh of the user's surroundings that includes floors, walls, ceilings, and other objects.

Morocco's permanent representative to the United Nations, Ambassador Omar Hilale, highlighted on September 30 that agricultural sciences and new technologies are an important part of the country's new economic projections. Morocco's Green Plan reached a goal of strengthening localized irrigation, one of the three major components of its Irrigation Strategy. The high-level meeting addressed "the role of Artificial Intelligence (AI) in achieving post-Covid food security." "Today, these sciences and technologies are helping to increase the production of small and medium farmers," Ambassador Hilale emphasized during the meeting. He also explained the crucial role that AI plays in "helping to produce more food with less water and energy."

This transcript has been edited for clarity. This is Eric Topol with the Medscape Medicine and the Machine podcast. I'm thrilled today to welcome Kai-Fu Lee, who is one of the leading artificial intelligence (AI) experts in the world. Before we get to that, let me give our Medscape audience a little background. You were born in Taiwan. You came to the United States in 1973, went to Columbia University and then Carnegie Mellon University, one of the leading AI centers in the country. You had an amazing career at Apple, Microsoft, and Google, when you led Google in China. In many ways you have been a major force for AI around the world, so we're really interested in your perspective. You and I first converged after I read your book AI Superpowers: China, Silicon Valley, and the New World Order. I was blown away because you had a unique perspective.