Transformer-based models have achieved great success in various NLP, vision, and speech tasks. However, the core of Transformer, the self-attention mechanism, has a quadratic time and memory complexity with respect to the sequence length, which hinders applications of Transformer-based models to long sequences. Many approaches have been proposed to mitigate this problem, such as sparse attention mechanisms, low-rank matrix approximations and scalable kernels, and token mixing alternatives to self-attention. We propose a novel Pooling Network (PoNet) for token mixing in long sequences with linear complexity. We design multi-granularity pooling and pooling fusion to capture different levels of contextual information and combine their interactions with tokens. On the Long Range Arena benchmark, PoNet significantly outperforms Transformer and achieves competitive accuracy, while being only slightly slower than the fastest model, FNet, across all sequence lengths measured on GPUs. We also conduct systematic studies on the transfer learning capability of PoNet and observe that PoNet achieves 96.0% of the accuracy of BERT on the GLUE benchmark, outperforming FNet by 4.5% relative. Comprehensive ablation analysis demonstrates effectiveness of the designed multi-granularity pooling and pooling fusion for token mixing in long sequences and efficacy of the designed pre-training tasks for PoNet to learn transferable contextualized language representations.

We introduce SMProbLog, a generalization of the probabilistic logic programming language ProbLog. A ProbLog program defines a distribution over logic programs by specifying for each clause the probability that it belongs to a randomly sampled program, and these probabilities are mutually independent. The semantics of ProbLog is given by the success probability of a query, which corresponds to the probability that the query succeeds in a randomly sampled program. It is well-defined when each random sample uniquely determines the truth values of all logical atoms. Argumentation problems, however, represent an interesting practical application where this is not always the case. SMProbLog generalizes the semantics of ProbLog to the setting where multiple truth assignments are possible for a randomly sampled program, and implements the corresponding algorithms for both inference and learning tasks. We then show how this novel framework can be used to reason about probabilistic argumentation problems. Therefore, the key contribution of this paper are: a more general semantics for ProbLog programs, its implementation into a probabilistic programming framework for both inference and parameter learning, and a novel approach to probabilistic argumentation problems based on such framework.

We show how to apply Sobol's method of global sensitivity analysis to measure the influence exerted by a set of nodes' evidence on a quantity of interest expressed by a Bayesian network. Our method exploits the network structure so as to transform the problem of Sobol index estimation into that of marginalization inference. This way, we can efficiently compute indices for networks where brute-force or Monte Carlo based estimators for variance-based sensitivity analysis would require millions of costly samples. Moreover, our method gives exact results when exact inference is used, and also supports the case of correlated inputs. The proposed algorithm is inspired by the field of tensor networks, and generalizes earlier tensor sensitivity techniques from the acyclic to the cyclic case. We demonstrate the method on three medium to large Bayesian networks that cover the areas of project risk management and reliability engineering.

Iranian dissidents are warning of the hard-line regime's use of drones to cause instability in the region, saying it is using the technology – materials for which are being imported from China – to make up for the weaknesses of its air force. The National Council of Resistance of Iran (NCRI), an umbrella group of Iranian resistance groups that oppose the regime, released evidence in a press conference it says shows the production and utilization of unmanned aerial vehicles (UACs) for terrorist operations and for assisting its proxies in the Middle East – including aerial photographs of the alleged sites and details that have emerged from inside the country. "Our revelation today is significant because it shows that the Qods Force of the IRGC has in recent years expanded its arsenal to step up terrorism and warmongering to destabilize the region by arming its proxies with UAVs," Alireza Jafarzadeh, deputy director of the Washington office of the National Council of Resistance of Iran, told Fox News. "This is in line with the regime's nuclear defiance and its repression at home." The group alleges that the regime, which has been rocked by a slew of economic sanctions imposed by the Trump administration as well as protests at home and challenges related to its handling of the COVID-19 pandemic, has used a web of industries to spend billions of dollars to produce components or smuggle them in from foreign countries.

Leading augmented reality (AR) technology company Blippar has confirmed its commitment to putting power in the hands of creators with the launch of its WebAR SDK technology. The toolkit will empower AR creators to build their own immersive WebAR experiences from the ground up using HTML and Java coding. WebAR SDK users will have access to full 24/7 support from the Blippar team to help hone their creative campaigns, and, during its beta phase, the platform will be entirely free to use, create, and publish from – with its immersive WebAR experiences able to be accessed and shared across platforms including browsers, Facebook, TikTok, WeChat, and WhatsApp – a further step in ensuring access to AR creativity is available to everyone. Blippar's WebAR SDK includes its most advanced implementation of simultaneous location and mapping (SLAM) to date, boasting 99% accuracy on tracking when locked, with less than a 1% margin of error in angular accuracy. SLAM is a set of computer vision technologies that allow AR developers and creatives to build much more interactive, immersive, and realistic AR experiences by using the device camera to create a mesh of the user's surroundings that includes floors, walls, ceilings, and other objects.

Morocco's permanent representative to the United Nations, Ambassador Omar Hilale, highlighted on September 30 that agricultural sciences and new technologies are an important part of the country's new economic projections. Morocco's Green Plan reached a goal of strengthening localized irrigation, one of the three major components of its Irrigation Strategy. The high-level meeting addressed "the role of Artificial Intelligence (AI) in achieving post-Covid food security." "Today, these sciences and technologies are helping to increase the production of small and medium farmers," Ambassador Hilale emphasized during the meeting. He also explained the crucial role that AI plays in "helping to produce more food with less water and energy."

This transcript has been edited for clarity. This is Eric Topol with the Medscape Medicine and the Machine podcast. I'm thrilled today to welcome Kai-Fu Lee, who is one of the leading artificial intelligence (AI) experts in the world. Before we get to that, let me give our Medscape audience a little background. You were born in Taiwan. You came to the United States in 1973, went to Columbia University and then Carnegie Mellon University, one of the leading AI centers in the country. You had an amazing career at Apple, Microsoft, and Google, when you led Google in China. In many ways you have been a major force for AI around the world, so we're really interested in your perspective. You and I first converged after I read your book AI Superpowers: China, Silicon Valley, and the New World Order. I was blown away because you had a unique perspective.

It is true that parts of data analytics are being automated every day like visualization and reporting, and I believe the trend will continue well into the future. I can understand this statement because many professions such as call center agents are being replaced by Chat bots and aspiring data analysts are afraid that if they start learning data analytics in a few years' time they too will be out of work. What is good about AI? When used on routine and repetitive tasks AI is very efficient. Finally, when we look at latest developments in "Strong" AI such as GANs, we see that machines can really take us to places we never imagined.

Watershed models such as the Soil and Water Assessment Tool (SWAT) consist of high-dimensional physical and empirical parameters. These parameters need to be accurately calibrated for models to produce reliable predictions for streamflow, evapotranspiration, snow water equivalent, and nutrient loading. Existing parameter estimation methods are time-consuming, inefficient, and computationally intensive, with reduced accuracy when estimating high-dimensional parameters. In this paper, we present a fast, accurate, and reliable methodology to calibrate the SWAT model (i.e., 21 parameters) using deep learning (DL). We develop DL-enabled inverse models based on convolutional neural networks to ingest streamflow data and estimate the SWAT model parameters. Hyperparameter tuning is performed to identify the optimal neural network architecture and the nine next best candidates. We use ensemble SWAT simulations to train, validate, and test the above DL models. We estimated the actual parameters of the SWAT model using observational data. We test and validate the proposed DL methodology on the American River Watershed, located in the Pacific Northwest-based Yakima River basin. Our results show that the DL models-based calibration is better than traditional parameter estimation methods, such as generalized likelihood uncertainty estimation (GLUE). The behavioral parameter sets estimated by DL have narrower ranges than GLUE and produce values within the sampling range even under high relative observational errors. This narrow range of parameters shows the reliability of the proposed workflow to estimate sensitive parameters accurately even under noise. Due to its fast and reasonably accurate estimations of process parameters, the proposed DL workflow is attractive for calibrating integrated hydrologic models for large spatial-scale applications.

A "bigger is better" explosion in the number of parameters in deep neural networks has made it increasingly challenging to make state-of-the-art networks accessible in compute-restricted environments. Compression techniques have taken on renewed importance as a way to bridge the gap. However, evaluation of the trade-offs incurred by popular compression techniques has been centered on high-resource datasets. In this work, we instead consider the impact of compression in a data-limited regime. We introduce the term low-resource double bind to refer to the co-occurrence of data limitations and compute resource constraints. This is a common setting for NLP for low-resource languages, yet the trade-offs in performance are poorly studied. Our work offers surprising insights into the relationship between capacity and generalization in data-limited regimes for the task of machine translation. Our experiments on magnitude pruning for translations from English into Yoruba, Hausa, Igbo and German show that in low-resource regimes, sparsity preserves performance on frequent sentences but has a disparate impact on infrequent ones. However, it improves robustness to out-of-distribution shifts, especially for datasets that are very distinct from the training distribution. Our findings suggest that sparsity can play a beneficial role at curbing memorization of low frequency attributes, and therefore offers a promising solution to the low-resource double bind.