Deep neural networks have revolutionized many machine learning tasks in power systems, ranging from pattern recognition to signal processing. The data in these tasks is typically represented in Euclidean domains. Nevertheless, there is an increasing number of applications in power systems, where data are collected from non-Euclidean domains and represented as the graph-structured data with high dimensional features and interdependency among nodes. The complexity of graph-structured data has brought significant challenges to the existing deep neural networks defined in Euclidean domains. Recently, many studies on extending deep neural networks for graph-structured data in power systems have emerged. In this paper, a comprehensive overview of graph neural networks (GNNs) in power systems is proposed. Specifically, several classical paradigms of GNNs structures (e.g., graph convolutional networks, graph recurrent neural networks, graph attention networks, graph generative networks, spatial-temporal graph convolutional networks, and hybrid forms of GNNs) are summarized, and key applications in power systems such as fault diagnosis, power prediction, power flow calculation, and data generation are reviewed in detail. Furthermore, main issues and some research trends about the applications of GNNs in power systems are discussed.

In many domains data is currently represented as graphs and therefore, the graph representation of this data becomes increasingly important in machine learning. Network data is, implicitly or explicitly, always represented using a graph shift operator (GSO) with the most common choices being the adjacency, Laplacian matrices and their normalisations. In this paper, a novel parametrised GSO (PGSO) is proposed, where specific parameter values result in the most commonly used GSOs and message-passing operators in graph neural network (GNN) frameworks. The PGSO is suggested as a replacement of the standard GSOs that are used in state-of-the-art GNN architectures and the optimisation of the PGSO parameters is seamlessly included in the model training. It is proved that the PGSO has real eigenvalues and a set of real eigenvectors independent of the parameter values and spectral bounds on the PGSO are derived. PGSO parameters are shown to adapt to the sparsity of the graph structure in a study on stochastic blockmodel networks, where they are found to automatically replicate the GSO regularisation found in the literature. On several real-world datasets the accuracy of state-of-theart GNN architectures is improved by the inclusion of the PGSO in both nodeand graph-classification tasks. Graph representation learning has attracted a significant research interest over the last years, mainly due to the structural complexity of real-world data and applications (Hamilton et al., 2017b; Wu et al., 2020). The topology of the observations plays a central role when performing machine learning tasks on graph structured data.

Expressive text encoders such as RNNs and Transformer Networks have been at the center of NLP models in recent work. Most of the effort has focused on sentence-level tasks, capturing the dependencies between words in a single sentence, or pairs of sentences. However, certain tasks, such as argumentation mining, require accounting for longer texts and complicated structural dependencies between them. Deep structured prediction is a general framework to combine the complementary strengths of expressive neural encoders and structured inference for highly structured domains. Nevertheless, when the need arises to go beyond sentences, most work relies on combining the output scores of independently trained classifiers. One of the main reasons for this is that constrained inference comes at a high computational cost. In this paper, we explore the use of randomized inference to alleviate this concern and show that we can efficiently leverage deep structured prediction and expressive neural encoders for a set of tasks involving complicated argumentative structures.

In this paper we propose an extension of the notion of deviation-based aggregation function tailored to aggregate multidimensional data. Our objective is both to improve the results obtained by other methods that try to select the best aggregation function for a particular set of data, such as penalty functions, and to reduce the temporal complexity required by such approaches. We discuss how this notion can be defined and present three illustrative examples of the applicability of our new proposal in areas where temporal constraints can be strict, such as image processing, deep learning and decision making, obtaining favourable results in the process.

There is growing evidence showing the significant role of social determinant of health (SDOH) on a wide variety of health outcomes. In the era of artificial intelligence (AI), electronic health records (EHRs) have been widely used to conduct observational studies. However, how to make the best of SDOH information from EHRs is yet to be studied. In this paper, we systematically reviewed recently published papers and provided a methodology review of AI methods using the SDOH information in EHR data. A total of 1250 articles were retrieved from the literature between 2010 and 2020, and 74 papers were included in this review after abstract and full-text screening. We summarized these papers in terms of general characteristics (including publication years, venues, countries etc.), SDOH types, disease areas, study outcomes, AI methods to extract SDOH from EHRs and AI methods using SDOH for healthcare outcomes. Finally, we conclude this paper with discussion on the current trends, challenges, and future directions on using SDOH from EHRs.

Housebound by a pandemic, humanity slowed its emissions of greenhouse gases in 2020. But Earth paid little heed: Temperatures last year tied the modern record, climate scientists reported last week. Overall, the planet was about 1.25°C warmer than in preindustrial times, a trend that puts climate targets in jeopardy, according to jointly reported assessments from NASA, Berkeley Earth, the U.K. Met Office, and the National Oceanic and Atmospheric Administration. The annual update of global surface temperatures—an average of readings from thousands of weather stations and ocean probes—shows 2020 essentially tied records set in 2016. But the years were nothing alike. Temperatures in 2016 were boosted by a strong El Niño, a weather pattern that warms the globe by blocking the rise of cold deep waters in the eastern Pacific Ocean. Last year, however, the Pacific entered La Niña, which has a cooling effect. That La Niña didn't provide more relief is an unwelcome surprise, says Nerilie Abram, a climate scientist at Australian National University. “It makes me worried about how quickly the global warming trend is growing.” The past 6 years are the six warmest on record, but the warming of the atmosphere is unsteady because of its chaotic nature. The ocean, which absorbs more than 90% of the heat from global warming, displays a steadier trend, and here, too, 2020 was a record year. The upper levels of the ocean contained 20 zettajoules (1021 joules) more heat than in 2019, and the rise was double the typical annual increase, scientists reported last week in Advances in Atmospheric Sciences . The subtropical Atlantic Ocean was particularly hot, fueling a record outbreak of hurricanes, says Lijing Cheng, a climate scientist at the Chinese Academy of Sciences's Institute of Atmospheric Physics who led the work. This heat, monitored down to 2000 meters by a fleet of 4000 robotic probes, is spreading deeper into the ocean while also migrating toward the poles. An extreme heat wave struck the northern Pacific, killing marine life. For the first time, warm Atlantic waters were seen penetrating into the Arctic Ocean, melting sea ice from below and reducing its extent nearly to a record low ( Science , 28 August 2020, p. [1043][1]). The warming ocean and melting ice sheets are raising sea levels by 4.8 millimeters per year, and the rate is accelerating ( Science , 20 November 2020, p. [901][2]). On land, 2020 was even more relentless, with temperatures rising 1.96°C above preindustrial levels, a clear record, Berkeley Earth reported. It was the warmest year ever in Asia and Europe and tied for the warmest in South America. Russia was particularly hot, breaking its previous record by 1.2°C, while swaths of Siberia were 7°C warmer than in preindustrial times, leading to large-scale fires and thawing permafrost that caused buildings to founder and set off oil spills ( Science , 7 August 2020, p. [612][3]). “Siberia was crazy,” says Zeke Hausfather, a climate scientist at the Breakthrough Institute and co-author of the Berkeley Earth analysis. “That heat would effectively be impossible without the warming we've seen.” In Australia, record-setting heat and drought fueled catastrophic bushfires at the start of 2020. Fires torched nearly one-quarter of southeastern Australia's forests and destroyed 3000 homes. Climate change was to blame for the country's “Black Summer,” Abram and co-authors concluded in a study published this month in Communications Earth & Environment . Meanwhile, in the United States, unprecedented heat came to the desert Southwest, which is already warming faster than the rest of the country. Phoenix wilted under its hottest summer ever, averaging 36°C. Arizona's Maricopa county, home to Phoenix, is a leader in addressing heat exposure, yet its heat deaths have hit a new record each year since 2016. In 2020, the number approached 300, a jump of some 50% over the previous year, says David Hondula, a climatologist who studies heat mortality at Arizona State University, Tempe. “It was just off the charts in terms of heat.” ![Figure][4] Turning up the heatCREDITS: (GRAPHIC) N. DESAI/ SCIENCE ; (DATA) MET OFFICE; NASA; BERKELEY EARTH; NOAA Although the global economic slowdown of the COVID-19 pandemic cut carbon dioxide (CO2) emissions by some 7%, atmospheric CO2 is long-lived, and warming from previous emissions is preordained. In any case, the drop in emissions is unlikely to last. Later this year, in May, before photosynthesis in the Northern Hemisphere draws down CO2, the U.K. Met Office predicts that levels of atmospheric CO2 will pass 417 parts per million for several weeks, 50% higher than preindustrial levels. Only dramatic action by the world's countries, far beyond existing efforts, can begin to halt this build up, Cheng says. Should the current rate of warming continue, the world will breach the targets set in the Paris climate agreement—limiting warming to 1.5°C or 2°C—by 2035 and 2065, respectively. But Hausfather says it's quite possible that warming, which has largely held steady for the past few decades at 0.19°C per decade, will actually speed up. The rate of warming over the past 14 years is well above the long-term trend. The debate now, he says, is whether that is an omen of an even darker future. [1]: https://www.sciencemag.org/content/369/6507/1043.full [2]: https://www.sciencemag.org/content/370/6519/901.full [3]: https://www.sciencemag.org/content/369/6504/612.full [4]: pending:yes

This thesis is a proof of concept for the potential of Variational Auto-Encoder (VAE) on representation learning of real-world Knowledge Graphs (KG). Inspired by successful approaches to the generation of molecular graphs, we evaluate the capabilities of our model, the Relational Graph Variational Auto-Encoder (RGVAE). The impact of the modular hyperparameter choices, encoding through graph convolutions, graph matching and latent space prior, is compared. The RGVAE is first evaluated on link prediction. The mean reciprocal rank (MRR) scores on the two datasets FB15K-237 and WN18RR are compared to the embedding-based model DistMult. A variational DistMult and a RGVAE without latent space prior constraint are implemented as control models. The results show that between different settings, the RGVAE with relaxed latent space, scores highest on both datasets, yet does not outperform the DistMult. Further, we investigate the latent space in a twofold experiment: first, linear interpolation between the latent representation of two triples, then the exploration of each latent dimension in a $95\%$ confidence interval. Both interpolations show that the RGVAE learns to reconstruct the adjacency matrix but fails to disentangle. For the last experiment we introduce a new validation method for the FB15K-237 data set. The relation type-constrains of generated triples are filtered and matched with entity types. The observed rate of valid generated triples is insignificantly higher than the random threshold. All generated and valid triples are unseen. A comparison between different latent space priors, using the $\delta$-VAE method, reveals a decoder collapse. Finally we analyze the limiting factors of our approach compared to molecule generation and propose solutions for the decoder collapse and successful representation learning of multi-relational KGs.

An accurate and efficient forecasting system is imperative to the prevention of emerging infectious diseases such as COVID-19 in public health. This system requires accurate transient modeling, lower computation cost, and fewer observation data. To tackle these three challenges, we propose a novel deep learning approach using black-box knowledge distillation for both accurate and efficient transmission dynamics prediction in a practical manner. First, we leverage mixture models to develop an accurate, comprehensive, yet impractical simulation system. Next, we use simulated observation sequences to query the simulation system to retrieve simulated projection sequences as knowledge. Then, with the obtained query data, sequence mixup is proposed to improve query efficiency, increase knowledge diversity, and boost distillation model accuracy. Finally, we train a student deep neural network with the retrieved and mixed observation-projection sequences for practical use. The case study on COVID-19 justifies that our approach accurately projects infections with much lower computation cost when observation data are limited.

Autonomous trading robots have been studied in ar-tificial intelligence area for quite some time. Many AI techniqueshave been tested in finance field including recent approaches likeconvolutional neural networks and deep reinforcement learning.There are many reported cases, where the developers are suc-cessful in creating robots with great performance when executingwith historical price series, so called backtesting. However, whenthese robots are used in real markets or data not used intheir training or evaluation frequently they present very poorperformance in terms of risks and return. In this paper, wediscussed some fundamental aspects of modelling autonomoustraders and the complex environment that is the financialworld. Furthermore, we presented a framework that helps thedevelopment and testing of autonomous traders. It may also beused in real or simulated operation in financial markets. Finally,we discussed some open problems in the area and pointed outsome interesting technologies that may contribute to advancein such task. We believe that mt5b3 may also contribute todevelopment of new autonomous traders.

Logical theories in the form of ontologies and similar artefacts in computing and IT are used for structuring, annotating, and querying data, among others, and therewith influence data analytics regarding what is fed into the algorithms. Algorithmic bias is a well-known notion, but what does bias mean in the context of ontologies that provide a structuring mechanism for an algorithm's input? What are the sources of bias there and how would they manifest themselves in ontologies? We examine and enumerate types of bias relevant for ontologies, and whether they are explicit or implicit. These eight types are illustrated with examples from extant production-level ontologies and samples from the literature. We then assessed three concurrently developed COVID-19 ontologies on bias and detected different subsets of types of bias in each one, to a greater or lesser extent. This first characterisation aims contribute to a sensitisation of ethical aspects of ontologies primarily regarding representation of information and knowledge.