gwt
Hypothesis on the Functional Advantages of the Selection-Broadcast Cycle Structure: Global Workspace Theory and Dealing with a Real-Time World
Nakanishi, Junya, Baba, Jun, Yoshikawa, Yuichiro, Kamide, Hiroko, Ishiguro, Hiroshi
This paper discusses the functional advantages of the Selection-Broadcast Cycle structure proposed by Global Workspace Theory (GWT), inspired by human consciousness, particularly focusing on its applicability to artificial intelligence and robotics in dynamic, real-time scenarios. While previous studies often examined the Selection and Broadcast processes independently, this research emphasizes their combined cyclic structure and the resulting benefits for real-time cognitive systems. Specifically, the paper identifies three primary benefits: Dynamic Thinking Adaptation, Experience-Based Adaptation, and Immediate Real-Time Adaptation. This work highlights GWT's potential as a cognitive architecture suitable for sophisticated decision-making and adaptive performance in unsupervised, dynamic environments. It suggests new directions for the development and implementation of robust, general-purpose AI and robotics systems capable of managing complex, real-world tasks.
Breaking Memory Limits: Gradient Wavelet Transform Enhances LLMs Training
Wen, Ziqing, Luo, Ping, Wang, Jiahuan, Deng, Xiaoge, Zou, Jinping, Yuan, Kun, Sun, Tao, Li, Dongsheng
Large language models (LLMs) have shown impressive performance across a range of natural language processing tasks. However, their vast number of parameters introduces significant memory challenges during training, particularly when using memory-intensive optimizers like Adam. Existing memory-efficient algorithms often rely on techniques such as singular value decomposition projection or weight freezing. While these approaches help alleviate memory constraints, they generally produce suboptimal results compared to full-rank updates. In this paper, we investigate the memory-efficient method beyond low-rank training, proposing a novel solution called Gradient Wavelet Transform (GWT), which applies wavelet transforms to gradients in order to significantly reduce the memory requirements for maintaining optimizer states. We demonstrate that GWT can be seamlessly integrated with memory-intensive optimizers, enabling efficient training without sacrificing performance. Through extensive experiments on both pre-training and fine-tuning tasks, we show that GWT achieves state-of-the-art performance compared with advanced memory-efficient optimizers and full-rank approaches in terms of both memory usage and training performance.
A Case for AI Consciousness: Language Agents and Global Workspace Theory
Goldstein, Simon, Kirk-Giannini, Cameron Domenico
It is generally assumed that existing artificial systems are not phenomenally conscious, and that the construction of phenomenally conscious artificial systems would require significant technological progress if it is possible at all. We challenge this assumption by arguing that if Global Workspace Theory (GWT) - a leading scientific theory of phenomenal consciousness - is correct, then instances of one widely implemented AI architecture, the artificial language agent, might easily be made phenomenally conscious if they are not already. Along the way, we articulate an explicit methodology for thinking about how to apply scientific theories of consciousness to artificial systems and employ this methodology to arrive at a set of necessary and sufficient conditions for phenomenal consciousness according to GWT.
Consciousness and Automated Reasoning
Barthelmeร, Ulrike, Furbach, Ulrich, Schon, Claudia
This paper aims at demonstrating how a first-order logic reasoning system in combination with a large knowledge base can be understood as an artificial consciousness system. For this we review some aspects from the area of philosophy of mind and in particular Baars' Global Workspace Theory. This will be applied to the reasoning system Hyper with ConceptNet as a knowledge base. Finally we demonstrate that such a system is very well able to do conscious mind wandering.
On the Singularity, emotions, and computer consciousness OUPblog
The term'artificial intelligence' was coined as long ago as 1956 to describe'the science and engineering of making intelligent machines'. The work that has happened in the subject since then has had enormous impact. Margaret Boden is a Research Professor of Cognitive Science at the University of Sussex, and one of the best known figures in the field of Artificial Intelligence. We put four key questions to her about this exciting area of research. It works according to (still largely unknown) scientific principles that could conceivably be simulated in computers. If AI could equal human intelligence, it could probably also surpass it.