The remarkable performance of large language models (LLMs) on complex linguistic tasks has sparked a lively debate on the nature of their capabilities. Unlike humans, these models learn language exclusively from textual data, without direct interaction with the real world. Nevertheless, they can generate seemingly meaningful text about a wide range of topics. This impressive accomplishment has rekindled interest in the classical 'Symbol Grounding Problem,' which questioned whether the internal representations and outputs of classical symbolic AI systems could possess intrinsic meaning. Unlike these systems, modern LLMs are artificial neural networks that compute over vectors rather than symbols. However, an analogous problem arises for such systems, which we dub the Vector Grounding Problem. This paper has two primary objectives. First, we differentiate various ways in which internal representations can be grounded in biological or artificial systems, identifying five distinct notions discussed in the literature: referential, sensorimotor, relational, communicative, and epistemic grounding. Unfortunately, these notions of grounding are often conflated. We clarify the differences between them, and argue that referential grounding is the one that lies at the heart of the Vector Grounding Problem. Second, drawing on theories of representational content in philosophy and cognitive science, we propose that certain LLMs, particularly those fine-tuned with Reinforcement Learning from Human Feedback (RLHF), possess the necessary features to overcome the Vector Grounding Problem, as they stand in the requisite causal-historical relations to the world that underpin intrinsic meaning. We also argue that, perhaps unexpectedly, multimodality and embodiment are neither necessary nor sufficient conditions for referential grounding in artificial systems.

In this paper, we explain the inference logic of large language models (LLMs) as a set of symbolic concepts. Many recent studies [4, 9, 10] have discovered that traditional DNNs usually encode sparse symbolic concepts. However, because an LLM has much more parameters than traditional DNNs, whether the LLM also encodes sparse symbolic concepts is still an open problem. Therefore, in this paper, we propose to disentangle the inference score of LLMs for dialogue tasks into a small number of symbolic concepts. We verify that we can use those sparse concepts to well estimate all inference scores of the LLM on all arbitrarily masking states of the input sentence. We also evaluate the transferability of concepts encoded by an LLM and verify that symbolic concepts usually exhibit high transferability across similar input sentences. More crucially, those symbolic concepts can be used to explain the exact reasons accountable for the LLM's prediction errors.

Elon Musk氏やSteve Wozniak氏を含む1800人以上のAIリーダーや技術者が、公開書簡を発表し、GPT-4よりも強力なAIシステムの開発を6ヶ月間一時停止するよう呼びかけた。彼らは、「社会と人類に深刻なリスクがある」としている。

I recently got into ChatGPT, i was just curious about the kind of interpretations, answers, "opinions" and text it can generate based on our conversation. As an AI language model, i don't have personal preferences, opinions or emotions… However as an objective observer i can say that… Most of the time it was able to offer its "thoughts". One of the exercise i did was asking for direct feedback about some text i wrote myself. The blue color of the sky is, in itself, a fiction of the sea; where colored birds swim, and a white pearl -signature of the strangeness of our world- adorn the necklace of a Nereid who is neither half human nor half inhuman and from whose song each one of us descends gently, without ceasing. ChatGPT feedback: The text you've written is very poetic and imaginative. It evokes a sense of mystery and beauty, with vivid imagery and a dreamlike quality. The idea of the blue color of the sky being a fiction of the sea is an interesting one, and it challenges our assumptions about the natural world.

In the last few weeks, we have been exploring ChatGPT's ability to build Software Diagrams for us. So far, In our first blog post, we covered Entity Relationship, Sequence and State Transition diagrams. In the next blog post, we looked at the C4 diagramming technique using Mermaid.js. Then, we evaluated ChatGPT's ability to work with PlantUML and create MindMaps.

Lately, I've been hearing a lot of sentiment of fear of AI potentially replacing people's jobs, sprung from scary titles like this one:

Discover The Ultimate Guide To Unlocking 40 Powerful Methods To Generate Income With ChatGPT Take Action Now and Transform Your Financial Future! This Guide Is Packed with Insider Secrets and Proven Strategies Used by Our Successful Clients. Enter Name & Email Below

Generative AI models are the hot new thing in the Big Tech world, and everyone is joining the race. The buzz really only started with OpenAI's ChatGPT chatbot, a generative AI language model that is incredibly good at predicting which words should follow one another when you feed it with prompts. Google has long been working on a similar technology, dubbed LaMDA, and with ChatGPT taking the world by storm, the company saw itself forced to release some version of its AI model to the world. That's how we got Bard, Google's first publicly available chat-based generative language model, with access to many parts of the internet. But is Google really at the same level as ChatGPT already?

“What kind of civilization is it that turns away from the challenge of dealing with more… intelligence?” That’s Tyler Cowen (GMU), writing at Marginal Revolution. He is addressing the “radical uncertainty” we should acknowledge regarding a future in which we’ve developed artificial intelligence (AI). Even if one does not believe that large language models (LLMs) could be a form of AI (recall the possible architectural limitation noted in the paper discussed last week), it does seem that at least the AI-like is here, will only get more convincing in functionality, and will likely bring substantial changes to our lives. Cowen’s targets are those who are making broad judgments about the goodness and badness of these technological developments. He thinks we’re living in a transformational period—he calls it “moving history”—and our predictions about it should be informed by an appropriate degree of epistemic humility. He says: Since we are not used to living in moving history, and indeed most of us are psychologically unable to truly imagine living in moving history, all these new AI developments pose a great conundrum. We don’t know how to respond psychologically, or for that matter substantively. And just about all of the responses I am seeing I interpret as “copes,” whether from the optimists, the pessimists, or the extreme pessimists… No matter how positive or negative the overall calculus of cost and benefit, AI is very likely to overturn most of our apple carts, most of all for the so-called chattering classes. Of course, that AI is “very likely to overturn most of our apple carts” and will ultimately be as unpredictable in its effects as the invention of fire or the printing press is itself a bold prediction. But suppose we accept it. That we can’t be certain of what might happen doesn’t render speculation random or pointless. So let’s speculate. I’m curious what changes, if any, you think we might be in for. And let’s talk about how to speculate. I’m curious about how to think about these changes. We might learn something from paleo-futurology, the study of past predictions of the future. One lesson appears to be that while some technological advances may be easy to predict, social changes are less so. Futurists..

If artificial intelligence evangelists' predictions pan out, generative AI systems like ChatGPT and DALL-E are set to transform Hollywood by developing and writing scripts for the next hit TV show, "diversifying" casts with AI-generated actors and generating imagery across multiple mediums, practically instantly, for a fraction of the cost of a real, human artist. But how long will it take for the vision to meet reality, and can a select group of companies -- similar to the rise of Facebook and social media -- be trusted to herald the way? Driving much of the current conversation around AI innovation has been OpenAI, an AI research company with both non-profit and for-profit arms. Just four months after the formal launch of OpenAI's chatbot, ChatGPT, industry titans like Bill Gates were ready to hail artificial intelligence as the most revolutionary technology of our time since the advent of cell phones and the internet. Major tech companies like Google and Microsoft have invested hundreds of millions into AI companies, including OpenAI, as executives look to the technology to steward their companies into the future amid an economic downturn that has particularly hit digital native companies hard.