In the policy evaluation steps, the critic is updated to capture the soft Q-function. In the policy improvement steps, the actor is adjusted in accordance with the updated soft Q-function.

Subtyping, also known as subtype polymorphism, is a concept extensively studied in programming language theory, delineating the substitutability relation among datatypes. This property ensures that programs designed for supertype objects remain compatible with their subtypes. In this paper, we explore the capability of order-sorted logic for utilizing these ideas in the context of Knowledge Representation. We recognize two fundamental limitations: First, the inability of this logic to address the concept rather than the value of non-logical symbols, and second, the lack of language constructs for constraining the type of terms. Consequently, we propose guarded order-sorted intensional logic, where guards are language constructs for annotating typing information and intensional logic provides support for quantification over concepts.

Large Language Models (LLMs) can memorize sensitive information, raising concerns about potential misuse. LLM Unlearning, a post-hoc approach to remove this information from trained LLMs, offers a promising solution to mitigate these risks. However, previous practices face three key challenges: 1. Utility: successful unlearning often causes catastrophic collapse on unrelated tasks. 2. Efficiency: many methods either involve adding similarly sized models, which slows down unlearning or inference, or require retain data that are difficult to obtain. 3. Robustness: even effective methods may still leak data via extraction techniques. To address these challenges, we propose MEOW, a simple yet effective gradient descent-based unlearning method. Specifically, we use an offline LLM to generate a set of inverted facts. Then, we design a new metric, MEMO, to quantify memorization in LLMs. Finally, based on the signals provided by MEMO, we select the most appropriate set of inverted facts and finetune the model based on them. We evaluate MEOW on the commonly used unlearn benchmark, ToFU, with Llama2-7B-Chat and Phi-1.5B, and test it on both NLU and NLG tasks. Results demonstrate significant improvement of MEOW in forget quality without substantial loss in model utility. Meanwhile, MEOW does not exhibit significant degradation in NLU or NLG capabilities, and there is even a slight improvement in NLU performance.

Existing Maximum-Entropy (MaxEnt) Reinforcement Learning (RL) methods for continuous action spaces are typically formulated based on actor-critic frameworks and optimized through alternating steps of policy evaluation and policy improvement. In the policy evaluation steps, the critic is updated to capture the soft Q-function. In the policy improvement steps, the actor is adjusted in accordance with the updated soft Q-function. In this paper, we introduce a new MaxEnt RL framework modeled using Energy-Based Normalizing Flows (EBFlow). This framework integrates the policy evaluation steps and the policy improvement steps, resulting in a single objective training process. Our method enables the calculation of the soft value function used in the policy evaluation target without Monte Carlo approximation. Moreover, this design supports the modeling of multi-modal action distributions while facilitating efficient action sampling. To evaluate the performance of our method, we conducted experiments on the MuJoCo benchmark suite and a number of high-dimensional robotic tasks simulated by Omniverse Isaac Gym. The evaluation results demonstrate that our method achieves superior performance compared to widely-adopted representative baselines.

Large language models (LLMs), in conjunction with various reasoning reinforcement methodologies, have demonstrated remarkable capabilities comparable to humans in fields such as mathematics, law, coding, common sense, and world knowledge. In this paper, we delve into the reasoning abilities of LLMs within complex human systems. We propose a novel reasoning framework, termed ``Mosaic Expert Observation Wall'' (MEOW) exploiting generative-agents-based simulation technique. In the MEOW framework, simulated data are utilized to train an expert model concentrating ``experience'' about a specific task in each independent time of simulation. It is the accumulated ``experience'' through the simulation that makes for an expert on a task in a complex human system. We conduct the experiments within a communication game that mirrors real-world security scenarios. The results indicate that our proposed methodology can cooperate with existing methodologies to enhance the reasoning abilities of LLMs in complex human systems.

You can scratch up rugs, furniture and doors with alternating L2 and R2 trigger pulls on a PlayStation controller, and you'll even leaves marks behind. For an added touch of realism, the PS5 DualSense controller's adaptive triggers, which adjust the tension of the rear buttons in response to gameplay, are harder to press down during these sequences.

A few years ago this month, Portland, Oregon artist Darius Kazemi watched a flood of tweets from would-be novelists. November is National Novel Writing Month, a time when people hunker down to churn out 50,000 words in a span of weeks. To Kazemi, a computational artist whose preferred medium is the Twitter bot, the idea sounded mildly tortuous. "I was thinking I would never do that," he says. "But if a computer could do it for me, I'd give it a shot."

We work out a quantum-theoretic model in complex Hilbert space of a recently performed test on co-occurrencies of two concepts and their combination in retrieval processes on specific corpuses of documents. The test violated the Clauser-Horne-Shimony-Holt version of the Bell inequalities ('CHSH inequality'), thus indicating the presence of entanglement between the combined concepts. We make use of a recently elaborated 'entanglement scheme' and represent the collected data in the tensor product of Hilbert spaces of the individual concepts, showing that the identified violation is due to the occurrence of a strong form of entanglement, involving both states and measurements and reflecting the meaning connection between the component concepts. These results provide a significant confirmation of the presence of quantum structures in corpuses of documents, like it is the case for the entanglement identified in human cognition.

We show that data collected from corpuses of documents violate the Clauser-Horne-Shimony-Holt version of Bell's inequality (CHSH inequality) and therefore indicate the presence of quantum entanglement in their structure. We obtain this result by considering two concepts and their combination and coincidence operations consisting of searches of co-occurrences of exemplars of these concepts in specific corpuses of documents. Measuring the frequencies of these co-occurrences and calculating the relative frequencies as approximate probabilities entering in the CHSH inequality, we obtain manifest violations of the latter for all considered corpuses of documents. In comparing these violations with those analogously obtained in an earlier work for the same combined concepts in psychological coincidence experiments with human participants, also violating the CHSH inequality, we identify the entanglement as being carried by the meaning connection between the two considered concepts within the combination they form. We explain the stronger violation for the corpuses of documents, as compared to the violation in the psychology experiments, as being due to the superior meaning domain of the human mind and, on the other side, to the latter reaching a broader domain of meaning and being possibly also actively influenced during the experimentation. We mention some of the issues to be analyzed in future work such as the violations of the CHSH inequality being larger than the `Cirel'son bound' for all of the considered corpuses of documents.