Optimizing language models for use in conversational agents requires large quantities of example dialogues. Increasingly, these dialogues are synthetically generated by using powerful large language models (LLMs), especially in domains with challenges to obtain authentic human data. One such domain is human resources (HR). In this context, we compare two LLM-based dialogue generation methods for the use case of generating HR job interviews, and assess whether one method generates higher-quality dialogues that are more challenging to distinguish from genuine human discourse. The first method uses a single prompt to generate the complete interview dialog. The second method uses two agents that converse with each other. To evaluate dialogue quality under each method, we ask a judge LLM to determine whether AI was used for interview generation, using pairwise interview comparisons. We demonstrate that despite a sixfold increase in token cost, interviews generated with the dual-prompt method achieve a win rate up to ten times higher than those generated with the single-prompt method. This difference remains consistent regardless of whether GPT-4o or Llama 3.3 70B is used for either interview generation or judging quality.

Conversational systems are now capable of producing impressive and generally relevant responses. However, we have no visibility nor control of the socio-emotional strategies behind state-of-the-art Large Language Models (LLMs), which poses a problem in terms of their transparency and thus their trustworthiness for critical applications. Another issue is that current automated metrics are not able to properly evaluate the quality of generated responses beyond the dataset's ground truth. In this paper, we propose a neural architecture that includes an intermediate step in planning socio-emotional strategies before response generation. We compare the performance of open-source baseline LLMs to the outputs of these same models augmented with our planning module. We also contrast the outputs obtained from automated metrics and evaluation results provided by human annotators. We describe a novel evaluation protocol that includes a coarse-grained consistency evaluation, as well as a finer-grained annotation of the responses on various social and emotional criteria. Our study shows that predicting a sequence of expected strategy labels and using this sequence to generate a response yields better results than a direct end-to-end generation scheme. It also highlights the divergences and the limits of current evaluation metrics for generated content. The code for the annotation platform and the annotated data are made publicly available for the evaluation of future models.

Large, general purpose language models have demonstrated impressive performance across many different conversational domains. While multi-domain language models achieve low overall perplexity, their outputs are not guaranteed to stay within the domain of a given input prompt. This paper proposes domain privacy as a novel way to quantify how likely a conditional language model will leak across domains. We also develop policy functions based on token-level domain classification, and propose an efficient fine-tuning method to improve the trained model's domain privacy. Experiments on membership inference attacks show that our proposed method has comparable resiliency to methods adapted from recent literature on differentially private language models.

You might have heard people talking about Natural Language Processing, but what is it exactly? We all know formal language. There are formal responses for formal statements and questions. You can easily feed such information into a system and create responses for the same. Formal languages follow grammatical rules and syntax.

From massive face-recognition-based surveillance and machine-learning-based decision systems predicting crime recidivism rates, to the move towards automated health diagnostic systems, artificial intelligence (AI) is being used in scenarios that have serious consequences in people's lives. However, this rapid permeation of AI into society has not been accompanied by a thorough investigation of the sociopolitical issues that cause certain groups of people to be harmed rather than advantaged by it. For instance, recent studies have shown that commercial face recognition systems have much higher error rates for dark skinned women while having minimal errors on light skinned men. A 2016 ProPublica investigation uncovered that machine learning based tools that assess crime recidivism rates in the US are biased against African Americans. Other studies show that natural language processing tools trained on newspapers exhibit societal biases (e.g. finishing the analogy "Man is to computer programmer as woman is to X" by homemaker). At the same time, books such as Weapons of Math Destruction and Automated Inequality detail how people in lower socioeconomic classes in the US are subjected to more automated decision making tools than those who are in the upper class. Thus, these tools are most often used on people towards whom they exhibit the most bias. While many technical solutions have been proposed to alleviate bias in machine learning systems, we have to take a holistic and multifaceted approach. This includes standardization bodies determining what types of systems can be used in which scenarios, making sure that automated decision tools are created by people from diverse backgrounds, and understanding the historical and political factors that disadvantage certain groups who are subjected to these tools.

Since Amazon's Echo arrived in late 2014, voice assistants have become increasingly important in making sense of the disparate smart home devices like speakers, light bulbs, thermostats, security cameras and more. But in the last few months, Alexa and the Google Assistant started letting users do multiple things -- like turning down the thermostat, lowering the lights and telling you what's on the calendar -- with a single command. It's been a key addition that has made managing a variety of different smart home devices easier. We tried out Amazon's implementation of routines back in October alongside new Echo hardware, but Google didn't launch its version until last month. If Google's going to keep Amazon from dominating the voice assistant market, it needs to be as good at tying a smart home together.

Google has begun rolling out Routines, a new feature of Google Assistant that allows users to string together a number of actions with a single phrase. There are six routines in the initial roll out: "Good morning," "Bedtime," Leaving home," "I'm, home," Commuting to work," and Commuting home." . To use them, go into the Google Home app, click the "hamburger" menu icon (the three horizontal bars in the top right), scroll down and click "more settings." Inside the services group, you should find Routines. If not, you'll need to update the app.

Humans are always fascinated with self-operating devices and today, it is software "Chatbots" which are becoming more human-like and are automated. The combination of immediate response and constant connectivity makes them an enticing way to extend or replace the web applications trend. But how do these automated programs work? At first, Chatbot can look like a normal app. There is an application layer, a database and APIs to call external services.

Facebook has apologised after an error in its machine-translation service saw Israeli police arrest a Palestinian man for posting "good morning" on his social media profile. The man, a construction worker in the West Bank settlement of Beitar Illit, near Jerusalem, posted a picture of himself leaning against a bulldozer with the caption "يصبحهم", or "yusbihuhum", which translates as "good morning". But Facebook's artificial intelligence-powered translation service, which it built after parting ways with Microsoft's Bing translation in 2016, instead translated the word into "hurt them" in English or "attack them" in Hebrew. Police officers arrested the man later that day, according to Israeli newspaper Haaretz, after they were notified of the post. They questioned him for several hours, suspicious he was planning to use the pictured bulldozer in a vehicle attack, before realising their mistake.

At first, Chatbot can look like a normal app. There is an application layer, a database and APIs to call external services. In a case of the chatbot, UI is replaced with chat interface. While Chatbots are easy to use for users, it adds complexity for the app to handle. There is a general worry that the bot can't understand the intent of the customer.