To render each generated token in real time, the LLM server generates response tokens one by one and streams each generated token (or group of a few tokens) through the network to the user right after it is generated, which we refer to as LLM token streaming. However, under unstable network conditions, the LLM token streaming experience could suffer greatly from stalls since one packet loss could block the rendering of tokens contained in subsequent packets even if they arrive on time. With a real-world measurement study, we show that current applications including ChatGPT, Claude, and Bard all suffer from increased stall under unstable network. For this emerging token streaming problem in LLM Chatbots, we propose a novel transport layer scheme, called Chatterbox, which puts new generated tokens as well as currently unacknowledged tokens in the next outgoing packet. This ensures that each packet contains some new tokens and can be independently rendered when received, thus avoiding aforementioned stalls caused by missing packets. Through simulation under various network conditions, we show Chatterbox reduces stall ratio (proportion of token rendering wait time) by 71.0% compared to the token streaming method commonly used by real chatbot applications and by 31.6% compared to a custom packet duplication scheme. By tailoring Chatterbox to fit the token-by-token generation of LLM, we enable the Chatbots to respond like an eloquent speaker for users to better enjoy pervasive AI.

As already explored previously in this blog, I experience different reactions to webchat. Some organisations love it, others shun it, as it's not really channel shift. Sometimes the webchat experience is very good and the customer (or prospect) receives their advice in a timely way. On other occasions, it can be a bit like the disjointed one below. You can see how webchat using customer services operators, can fall well short. That's why webchat is becoming superseded by the chatbot.

Chatterbox is a build-it-yourself, program-it-yourself smart speaker that teaches kids how to program a voice-based AI system. The company is able to ensure complete privacy because it is using Mycroft, an open-source voice assistant that is not always listening, not collecting any data, and not advertising. In addition, the product is fully compliant with the Children's Online Privacy Protection Act (COPPA), which is what the Federal Trade Commission uses to regulate governing online services directed at children under 13 years of age. The company announced recently that it would be launching a Kickstarter campaign on April 30th, and will ship to consumers in schools in December 2019, with a suggested retail price of $179. We've honed in on privacy, safety, and accessibility because our mission is to provide the healthiest and safest alternative computing platform for children.

Being a chatterbox could boost your child's intelligence, according to new research. It found youngsters exposed to large amounts of speech by their parents had higher IQs and better cognitive skills. The children, aged two to four, also tended to have better non-verbal skills such as reasoning, numeracy and shape awareness. Additionally, the study found children who interacted with adults that used a diverse vocabulary knew a greater variety of words themselves. Researchers suggest greater exposure to language forces youngsters to learn what words and phrases mean.

Friday 6 October 2017 04.06 EDT Last modified on Friday 6 October 2017 04.07 EDT

The place to talk about games and other things that matter... for two more days The place to talk about games and other things that matter... for two more days The Blood Bowl is almost done.

As you may have seen I'm leaving my position as games editor this Friday – unfortunately, this also means the end of Chatterbox as I'm the only one who maintains the regular posts. I'm really sorry that our friendly, funny and long-running forum must come to an end. I'd like to thank everyone who has helped manage the box over its long life, especially Limni, who has been my community liaison officer all these years. It was great fun to meet many of you at various events. I'll never forget that time we invaded Bafta – and, to be honest, neither will they.

Friday 29 September 2017 02.00 EDT Last modified on Friday 29 September 2017 02.01 EDT

Wednesday 27 September 2017 02.00 EDT Last modified on Wednesday 27 September 2017 02.01 EDT