Visual ChatGPT, the chatbot that communicates through images - Plugavel

One of the main weak points of conversational artificial intelligence ChatGPTChatGPT is that it is limited to text only. To solve this problem, researchers at MicrosoftMicrosoft have just released a new version of ChatGPT called Visual ChatGPT. In the associated articlethey explain how they managed to integrate image support into ChatGPT without touching the AI itself. Rather than completely rebuilding ChatGPT to support different modalities (audio, images, videos…), they decided to rely on pre-existing Visual Foundation Models (VFMs), like Stable Diffusion, BLIP, Transformers, Maskformer and ControlNet. The central module of Visual ChatGPT is the request handler (Prompt Manager).