There is an ongoing debate in the NLP community whether modern language models contain linguistic knowledge, recovered through so-called probes. In this paper, we study whether linguistic knowledge is a necessary condition for the good performance of modern language models, which we call the rediscovery hypothesis. In the first place, we show that language models that are significantly compressed but perform well on their pretraining objectives retain good scores when probed for linguistic structures. This result supports the rediscovery hypothesis and leads to the second contribution of our paper: an information-theoretic framework that relates language modeling objectives with linguistic information. This framework also provides a metric to measure the impact of linguistic information on the word prediction task. We reinforce our analytical results with various experiments, both on synthetic and on real NLP tasks in English.

The ever-increasing amount of user-generated content online has led, in recent years, to an expansion in research and investment in automated content analysis tools. Scrutiny of automated content analysis has accelerated during the COVID-19 pandemic, as social networking services have placed a greater reliance on these tools due to concerns about health risks to their moderation staff from in-person work. At the same time, there are important policy debates around the world about how to improve content moderation while protecting free expression and privacy. In order to advance these debates, we need to understand the potential role of automated content analysis tools. This paper explains the capabilities and limitations of tools for analyzing online multimedia content and highlights the potential risks of using these tools at scale without accounting for their limitations. It focuses on two main categories of tools: matching models and computer prediction models. Matching models include cryptographic and perceptual hashing, which compare user-generated content with existing and known content. Predictive models (including computer vision and computer audition) are machine learning techniques that aim to identify characteristics of new or previously unknown content.

Hybrid methods have been shown to outperform pure statistical and pure deep learning methods at forecasting tasks and quantifying the associated uncertainty with those forecasts (prediction intervals). One example is Exponential Smoothing Recurrent Neural Network (ES-RNN), a hybrid between a statistical forecasting model and a recurrent neural network variant. ES-RNN achieves a 9.4\% improvement in absolute error in the Makridakis-4 Forecasting Competition. This improvement and similar outperformance from other hybrid models have primarily been demonstrated only on univariate datasets. Difficulties with applying hybrid forecast methods to multivariate data include ($i$) the high computational cost involved in hyperparameter tuning for models that are not parsimonious, ($ii$) challenges associated with auto-correlation inherent in the data, as well as ($iii$) complex dependency (cross-correlation) between the covariates that may be hard to capture. This paper presents Multivariate Exponential Smoothing Long Short Term Memory (MES-LSTM), a generalized multivariate extension to ES-RNN, that overcomes these challenges. MES-LSTM utilizes a vectorized implementation. We test MES-LSTM on several aggregated coronavirus disease of 2019 (COVID-19) morbidity datasets and find our hybrid approach shows consistent, significant improvement over pure statistical and deep learning methods at forecast accuracy and prediction interval construction.

Astronomical transients are stellar objects that become temporarily brighter on various timescales and have led to some of the most significant discoveries in cosmology and astronomy. Some of these transients are the explosive deaths of stars known as supernovae while others are rare, exotic, or entirely new kinds of exciting stellar explosions. New astronomical sky surveys are observing unprecedented numbers of multi-wavelength transients, making standard approaches of visually identifying new and interesting transients infeasible. To meet this demand, we present two novel methods that aim to quickly and automatically detect anomalous transient light curves in real-time. Both methods are based on the simple idea that if the light curves from a known population of transients can be accurately modelled, any deviations from model predictions are likely anomalies. The first approach is a probabilistic neural network built using Temporal Convolutional Networks (TCNs) and the second is an interpretable Bayesian parametric model of a transient. We show that the flexibility of neural networks, the attribute that makes them such a powerful tool for many regression tasks, is what makes them less suitable for anomaly detection when compared with our parametric model.

Cultural code-switching concerns how we adjust our overall behaviours, manners of speaking, and appearance in response to a perceived change in our social environment. We defend the need to investigate cultural code-switching capacities in artificial intelligence systems. We explore a series of ethical and epistemic issues that arise when bringing cultural code-switching to bear on artificial intelligence. Building upon Dotson's (2014) analysis of testimonial smothering, we discuss how emerging technologies in AI can give rise to epistemic oppression, and specifically, a form of self-silencing that we call 'cultural smothering'. By leaving the socio-dynamic features of cultural code-switching unaddressed, AI systems risk negatively impacting already-marginalised social groups by widening opportunity gaps and further entrenching social inequalities.

Many of us who have uploaded images of our faces and the faces of our friends and family to openly-licensed platforms on the Web may have inadvertently contributed to a massive and growing database for AI facial recognition. So how are our faces being used? So have we all thrown away our privacy and assumption of innocence for a selfie? The film is Web Monetized, with all streaming payments going to the Surveillance Technology Oversight Project (S.T.O.P.) On the GftW Community Forum, we have been streaming funds to S.T.O.P. since July. So far, we have generated almost $200 in micropayments to support their work.

Washington, DC – The United States is sending a "dangerous and misleading message" by failing to hold any US military personnel responsible for a Kabul drone attack that killed 10 civilians, including seven children, human rights advocates have said. Calls for accountability for the deadly bombing on August 29 grew on Tuesday, a day after US media outlets first reported that US Defense Secretary Lloyd Austin had accepted a recommendation from top commanders not to punish any members of the military. Rights groups also urged President Joe Biden's administration to do more to help the survivors of the attack in the Afghan capital to relocate to the US. The bombing targeted the car of Zemari Ahmadi, who worked for US-based aid organisation Nutrition and Education International (NEI), killing him and nine of his family members. "I've been beseeching the US government to evacuate directly-impacted family members and NEI employees for months because their security situation is so dire," Steven Kwon, founder and president of NEI, said in a statement.

Welcome to The TechCrunch Exchange, a weekly startups-and-markets newsletter. It's inspired by the daily TechCrunch column where it gets its name. If you aren't in the United States, it's a little hard to explain. In short, certain deficiencies in our policing and judicial systems flared brightly as the week came to a close. So, today's Exchange newsletter will be shorter than intended. Hug the people you love, and everyone else.

Organizations are just starting to tap the incredible computational powers of AI for creativity, human productivity, and business results. If you thought judgment, ethics, and even creativity were the unique purview of humans, think again. The latest industry analyst predictions about artificial intelligence (AI) are out, and they're certain to oust a ton of assumptions we've made to date. Read on to find out just how smart, creative, and sincere AI will become during the next few years. Noting that South Africa granted the first patent to a creative AI system in 2021, Forrester researchers predicted creative AI systems will win dozens of patents in 2022.

Enthusiastic and inquisitive healthcare professionals are significantly excited for healthtech summits taking place around the world. Perhaps when it is about rapidly evolving technologies influencing healthcare operations like robotics in healthcare. Lately, the introduction of artificial intelligence has considerably fueled surgical robots and other robotic applications in the healthcare industry. Thus, concerns relating to the future of healthcare and scrutinization of past healthtech events are analysed in these healthtech summits. Here are some of the most helpful healthtech summits listed that cordially invite healthcare enthusiasts to join.