Context-aware Human Activity Recognition (HAR) is a hot research area in mobile computing, and the most effective solutions in the literature are based on supervised deep learning models. However, the actual deployment of these systems is limited by the scarcity of labeled data that is required for training. Neuro-Symbolic AI (NeSy) provides an interesting research direction to mitigate this issue, by infusing common-sense knowledge about human activities and the contexts in which they can be performed into HAR deep learning classifiers. Existing NeSy methods for context-aware HAR rely on knowledge encoded in logic-based models (e.g., ontologies) whose design, implementation, and maintenance to capture new activities and contexts require significant human engineering efforts, technical knowledge, and domain expertise. Recent works show that pre-trained Large Language Models (LLMs) effectively encode common-sense knowledge about human activities. In this work, we propose ContextGPT: a novel prompt engineering approach to retrieve from LLMs common-sense knowledge about the relationship between human activities and the context in which they are performed. Unlike ontologies, ContextGPT requires limited human effort and expertise. An extensive evaluation carried out on two public datasets shows how a NeSy model obtained by infusing common-sense knowledge from ContextGPT is effective in data scarcity scenarios, leading to similar (and sometimes better) recognition rates than logic-based approaches with a fraction of the effort.

State-of-the-art approaches rely on image-based features extracted via neural networks for the deepfake detection binary classification. While these approaches trained in the supervised sense extract likely fake features, they may fall short in representing unnatural `non-physical' semantic facial attributes -- blurry hairlines, double eyebrows, rigid eye pupils, or unnatural skin shading. However, such facial attributes are generally easily perceived by humans via common sense reasoning. Furthermore, image-based feature extraction methods that provide visual explanation via saliency maps can be hard to be interpreted by humans. To address these challenges, we propose the use of common sense reasoning to model deepfake detection, and extend it to the Deepfake Detection VQA (DD-VQA) task with the aim to model human intuition in explaining the reason behind labeling an image as either real or fake. To this end, we introduce a new dataset that provides answers to the questions related to the authenticity of an image, along with its corresponding explanations. We also propose a Vision and Language Transformer-based framework for the DD-VQA task, incorporating text and image aware feature alignment formulations. Finally, we evaluate our method on both the performance of deepfake detection and the quality of the generated explanations. We hope that this task inspires researchers to explore new avenues for enhancing language-based interpretability and cross-modality applications in the realm of deepfake detection.

Service robots need common-sense knowledge to help humans in everyday situations as it enables them to understand the context of their actions. However, approaches that use ontologies face a challenge because common-sense knowledge is often implicit, i.e., it is obvious to humans but not explicitly stated. This paper investigates if Large Language Models (LLMs) can fill this gap. Our experiments reveal limited effectiveness in the selective extraction of contextual action knowledge, suggesting that LLMs may not be sufficient on their own. However, the large-scale extraction of general, actionable knowledge shows potential, indicating that LLMs can be a suitable tool for efficiently creating ontologies for robots. This paper shows that the technique used for knowledge extraction can be applied to populate a minimalist ontology, showcasing the potential of LLMs in synergy with formal knowledge representation.

Charles Simon, BSEE, MSCs, is the founder and CEO of Future AI: Technologies that Think. AI systems are often compared to the human brain, even though they have almost nothing in common. To achieve artificial general intelligence (AGI), we tend to look to the only example of general intelligence available for humans to study: the human brain. This approach has led to modern artificial intelligence (AI) often being presented as working like your brain. It is more accurate, though, to view modern AI techniques such as machine learning (ML) as powerful statistical methods.

The ongoing recalibration of AI research and development underscores a fundamental tenet of machine learning: We must learn to crawl before we can walk. Thus far, AI hype has mostly talked the talk rather than walking the walk. Returning to what appear to be engineering first principles, U.S. research efforts are attempting to move beyond current "brittle" AI models that excel at only specific tasks. The goal is developing more generalized models that can adapt much like humans do in new situations. Among those efforts is a Machine Common Sense program overseen by the Defense Advanced Research Projects Agency (DARPA) that seeks to imbue machine learning models with the kinds of commonplace reasoning displayed by among the fastest learners on the planet: infants.

Math word problem (MWP) solving is the task of transforming a sequence of natural language problem descriptions to executable math equations. An MWP solver not only needs to understand complex scenarios described in the problem texts, but also identify the key mathematical variables and associate text descriptions with math equation logic. Although recent sequence modeling MWP solvers have gained credits on the math-text contextual understanding, pre-trained language models (PLM) have not been explored for solving MWP, considering that PLM trained over free-form texts is limited in representing text references to mathematical logic. In this work, we introduce MWP-BERT to obtain pre-trained token representations that capture the alignment between text description and mathematical logic. Additionally, we introduce a keyword-based prompt matching method to address the MWPs requiring common-sense knowledge. On a benchmark Math23K dataset and a new Ape210k dataset, we show that MWP-BERT outperforms the strongest baseline model by 5-10% improvement on accuracy.

Key Takeaways: # Intelligent software agents must use common sense in order to reason. Common-sense knowledge is required before intelligent software agents can anticipate how people and the physical world react. Deep learning models do not currently understand what they produce, and have no common-sense knowledge. The Commonsense Transformers (COMET) project attempts to train models with information about the world in ways similar to how a human would acquire such knowledge. The COMET project and other similar efforts are still in the research phase.

Artificial intelligence researchers have not been successful in giving intelligent agents the common-sense knowledge they need to reason about the world. Without this knowledge, it is impossible for intelligent agents to truly interact with the world. Traditionally, there have been two unsuccessful approaches to getting computers to reason about the world--symbolic logic and deep learning. A new project, called COMET, tries to bring these two approaches together. Although it has not yet succeeded, it offers the possibility of progress.

Artificial intelligence researchers have not been successful in giving intelligent agents the common-sense knowledge they need to reason about the world. Without this knowledge, it is impossible for intelligent agents to truly interact with the world. Traditionally, there have been two unsuccessful approaches to getting computers to reason about the world--symbolic logic and deep learning. A new project, called COMET, tries to bring these two approaches together. Although it has not yet succeeded, it offers the possibility of progress.

Artificial intelligence researchers have not been successful in giving intelligent agents the common-sense knowledge they need to reason about the world. Without this knowledge, it is impossible for intelligent agents to truly interact with the world. Traditionally, there have been two unsuccessful approaches to getting computers to reason about the world--symbolic logic and deep learning. A new project, called COMET, tries to bring these two approaches together. Although it has not yet succeeded, it offers the possibility of progress.