State-of-the-art deep Q-learning methods update Q-values using state transition tuples sampled from the experience replay buffer. This strategy often uniformly and randomly samples or prioritizes data sampling based on measures such as the temporal difference (TD) error. Such sampling strategies can be inefficient at learning Q-function because a state's Q-value depends on the Q-value of successor states. If the data sampling strategy ignores the precision of the Q-value estimate of the next state, it can lead to useless and often incorrect updates to the Q-values. To mitigate this issue, we organize the agent's experience into a graph that explicitly tracks the dependency between Q-values of states. Each edge in the graph represents a transition between two states by executing a single action. We perform value backups via a breadth-first search starting from that expands vertices in the graph starting from the set of terminal states and successively moving backward. We empirically show that our method is substantially more data-efficient than several baselines on a diverse range of goal-reaching tasks. Notably, the proposed method also outperforms baselines that consume more batches of training experience and operates from high-dimensional observational data such as images.

The calibration of the EFL teaching and learning approaches with Artificial Intelligence can potentially facilitate a smart transformation, fostering a personalized and engaging experience in teaching and learning among the stakeholders. The paper focuses on developing an EFL Big Data Ecosystem that is based on Big Data, Analytics, Machine Learning and cluster domain of EFL teaching and learning contents. The framework has been developed on the basis of the theory that machine learning algorithms, when exposed to structured or semistructure data stored in the cluster domains of EFL Big Data ecosystem, can cull out the patterns, similarities, and differences existing in the contents of the domains. Later these machine learning algorithms can apply these already identified patterns to perform new tasks on open Big Data platform and identify similar contents to be stored in the respective cluster domain of EFL Bigdata Ecosystem without being supervised. Accordingly, the paper uses two membranes to construe its framework, namely (i) Open Big Data Membrane that stores random data collected from various source domains and (ii) Machine Learning Membrane that stores specially prepared structured and semi-structured data. Theoretically, the structured and semi structured data are to be prepared skill-wise, attribute-wise, method-wise, and preference-wise to accommodate the personalized preferences and diverse teaching and learning needs of different individuals. Within the machine learning membrane, the paper includes a number of stages such as knowledge building, development of cluster domain of the EFL contents, integration of skill-wise cluster domain with the CEFR attribute-wise teaching and learning approaches, machine learning of the personalized preferences, resonating, machine learning of the cluster domain for proximity development and sustainable operation. The ultimate goal is to optimize the learning experience by leveraging machine learning to create tailored content that aligns with the diverse teaching and learning needs of the EFL communities. Developing a prototype following the framework exerts the potentials to provide an'alternative to methods', transforming the process of learning into a process of acquisition.

Integrating automatic speech scoring/assessment systems has become a critical aspect of second-language speaking education. With self-supervised learning advancements, end-to-end speech scoring approaches have exhibited promising results. However, this study highlights the significant decrease in the performance of speech scoring systems in new question contexts, thereby identifying this as a cold start problem in terms of items. With the finding of cold-start phenomena, this paper seeks to alleviate the problem by following methods: 1) prompt embeddings, 2) question context embeddings using BERT or CLIP models, and 3) choice of the pretrained acoustic model. Experiments are conducted on TOEIC speaking test datasets collected from English-as-a-second-language (ESL) learners rated by professional TOEIC speaking evaluators. The results demonstrate that the proposed framework not only exhibits robustness in a cold-start environment but also outperforms the baselines for known content.

We formulate the manifold learning problem as the problem of finding an operator that maps any point to a close neighbor that lies on a ``hidden'' $k$-dimensional manifold. We call this operator the correcting function. Under this formulation, autoencoders can be viewed as a tool to approximate the correcting function. Given an autoencoder whose Jacobian has rank $k$, we deduce from the classical Constant Rank Theorem that its range has a structure of a $k$-dimensional manifold. A $k$-dimensionality of the range can be forced by the architecture of an autoencoder (by fixing the dimension of the code space), or alternatively, by an additional constraint that the rank of the autoencoder mapping is not greater than $k$. This constraint is included in the objective function as a new term, namely a squared Ky-Fan $k$-antinorm of the Jacobian function. We claim that this constraint is a factor that effectively reduces the dimension of the range of an autoencoder, additionally to the reduction defined by the architecture. We also add a new curvature term into the objective. To conclude, we experimentally compare our approach with the CAE+H method on synthetic and real-world datasets.

The traditional way of sentence-level event detection involves two important subtasks: trigger identification and trigger classifications, where the identified event trigger words are used to classify event types from sentences. However, trigger classification highly depends on abundant annotated trigger words and the accuracy of trigger identification. In a real scenario, annotating trigger words is time-consuming and laborious. For this reason, we propose a trigger-free event detection model, which transforms event detection into a two-tower model based on machine reading comprehension and prompt learning. Compared to existing trigger-based and trigger-free methods, experimental studies on two event detection benchmark datasets (ACE2005 and MAVEN) have shown that the proposed approach can achieve competitive performance.

Scientific machine reading comprehension (SMRC) aims to understand scientific texts through interactions with humans by given questions. As far as we know, there is only one dataset focused on exploring full-text scientific machine reading comprehension. However, the dataset has ignored the fact that different readers may have different levels of understanding of the text, and only includes single-perspective question-answer pairs, leading to a lack of consideration of different perspectives. To tackle the above problem, we propose a novel multi-perspective SMRC dataset, called SciMRC, which includes perspectives from beginners, students and experts. Our proposed SciMRC is constructed from 741 scientific papers and 6,057 question-answer pairs. Each perspective of beginners, students and experts contains 3,306, 1,800 and 951 QA pairs, respectively. The extensive experiments on SciMRC by utilizing pre-trained models suggest the importance of considering perspectives of SMRC, and demonstrate its challenging nature for machine comprehension.

Most existing text generation models follow the sequence-to-sequence paradigm. Generative Grammar suggests that humans generate natural language texts by learning language grammar. We propose a syntax-guided generation schema, which generates the sequence guided by a constituency parse tree in a top-down direction. The decoding process can be decomposed into two parts: (1) predicting the infilling texts for each constituent in the lexicalized syntax context given the source sentence; (2) mapping and expanding each constituent to construct the next-level syntax context. Accordingly, we propose a structural beam search method to find possible syntax structures hierarchically. Experiments on paraphrase generation and machine translation show that the proposed method outperforms autoregressive baselines, while also demonstrating effectiveness in terms of interpretability, controllability, and diversity.

Empirical studies suggest a deep intertwining between opinion formation and decision-making processes, but these have been treated as separate problems in the study of dynamical models for social networks. In this paper, we bridge the gap in the literature by proposing a novel coevolutionary model, in which each individual selects an action from a binary set and has an opinion on which action they prefer. Actions and opinions coevolve on a two-layer network. For homogeneous parameters, undirected networks, and under reasonable assumptions on the asynchronous updating mechanics, we prove that the coevolutionary dynamics is an ordinal potential game, enabling analysis via potential game theory. Specifically, we establish global convergence to the Nash equilibria of the game, proving that actions converge in a finite number of time steps, while opinions converge asymptotically. Next, we provide sufficient conditions for the existence of, and convergence to, polarized equilibria, whereby the population splits into two communities, each selecting and supporting one of the actions. Finally, we use simulations to examine the social psychological phenomenon of pluralistic ignorance.

Learning analytics (LA) as a research discipline focuses on multiple perspectives of understanding and supporting educational activities utilizing collected log data. To do so at a national and even international level, educational technology platforms that enable gathering users' interaction traces and digitally generated artifacts must store data in a standardized format. In Japan, the government initiated the GIGA School project in 2020, which installed more than nine million tablet PCs and high-speed Internet access at compulsory education institutions (elemental and middle schools). Such infrastructure enables the collection of educational data and analysis with the aim to improve educational practices in each school. With standardized data logging, it is possible to aggregate data from all schools and to generate educational Big Data that can support evidence-based policy-making and research at a national level.

Physics-informed machine learning (PIML) is a set of methods and tools that systematically integrate machine learning (ML) algorithms with physical constraints and abstract mathematical models developed in scientific and engineering domains. As opposed to purely data-driven methods, PIML models can be trained from additional information obtained by enforcing physical laws such as energy and mass conservation. More broadly, PIML models can include abstract properties and conditions such as stability, convexity, or invariance. The basic premise of PIML is that the integration of ML and physics can yield more effective, physically consistent, and data-efficient models. This paper aims to provide a tutorial-like overview of the recent advances in PIML for dynamical system modeling and control. Specifically, the paper covers an overview of the theory, fundamental concepts and methods, tools, and applications on topics of: 1) physics-informed learning for system identification; 2) physics-informed learning for control; 3) analysis and verification of PIML models; and 4) physics-informed digital twins. The paper is concluded with a perspective on open challenges and future research opportunities.