As Large Language Models (LLMs) play an increasingly pivotal role in natural language processing applications, their safety concerns become critical areas of NLP research. This paper presents Safety and Over-Defensiveness Evaluation (SODE) benchmark: a collection of diverse safe and unsafe prompts with carefully designed evaluation methods that facilitate systematic evaluation, comparison, and analysis over 'safety' and 'over-defensiveness.' With SODE, we study a variety of LLM defense strategies over multiple state-of-the-art LLMs, which reveals several interesting and important findings, such as (a) the widely popular 'self-checking' techniques indeed improve the safety against unsafe inputs, but this comes at the cost of extreme over-defensiveness on the safe inputs, (b) providing a safety instruction along with in-context exemplars (of both safe and unsafe inputs) consistently improves safety and also mitigates undue over-defensiveness of the models, (c) providing contextual knowledge easily breaks the safety guardrails and makes the models more vulnerable to generating unsafe responses. Overall, our work reveals numerous such critical findings that we believe will pave the way and facilitate further research in improving the safety of LLMs.

We present symbol tuning - finetuning language models on in-context input-label pairs where natural language labels (e.g., "positive/negative sentiment") are replaced with arbitrary symbols (e.g., "foo/bar"). Symbol tuning leverages the intuition that when a model cannot use instructions or natural language labels to figure out a task, it must instead do so by learning the input-label mappings. We experiment with symbol tuning across Flan-PaLM models up to 540B parameters and observe benefits across various settings. First, symbol tuning boosts performance on unseen in-context learning tasks and is much more robust to underspecified prompts, such as those without instructions or without natural language labels. Second, symbol-tuned models are much stronger at algorithmic reasoning tasks, with up to 18.2% better performance on the List Functions benchmark and up to 15.3% better performance on the Simple Turing Concepts benchmark. Finally, symbol-tuned models show large improvements in following flipped-labels presented in-context, meaning that they are more capable of using in-context information to override prior semantic knowledge.

Quantification of behavior is critical in applications ranging from neuroscience, veterinary medicine and animal conservation efforts. A common key step for behavioral analysis is first extracting relevant keypoints on animals, known as pose estimation. However, reliable inference of poses currently requires domain knowledge and manual labeling effort to build supervised models. We present a series of technical innovations that enable a new method, collectively called SuperAnimal, to develop unified foundation models that can be used on over 45 species, without additional human labels. Concretely, we introduce a method to unify the keypoint space across differently labeled datasets (via our generalized data converter) and for training these diverse datasets in a manner such that they don't catastrophically forget keypoints given the unbalanced inputs (via our keypoint gradient masking and memory replay approaches). These models show excellent performance across six pose benchmarks. Then, to ensure maximal usability for end-users, we demonstrate how to fine-tune the models on differently labeled data and provide tooling for unsupervised video adaptation to boost performance and decrease jitter across frames. If the models are fine-tuned, we show SuperAnimal models are 10-100$\times$ more data efficient than prior transfer-learning-based approaches. We illustrate the utility of our models in behavioral classification in mice and gait analysis in horses. Collectively, this presents a data-efficient solution for animal pose estimation.

In the filing, Cohen wrote that he had not kept up with "emerging trends (and related risks) in legal technology and did not realize that Google Bard was a generative text service that, like ChatGPT, could show citations and descriptions that looked real but actually were not." To him, he said, Google Bard seemed to be a "supercharged search engine."

Jack Krawczyk discusses how Google Bard helps users connect and communicate -- and what the future holds for the platform. Michael Cohen, former President Trump's onetime fixer and lawyer, admitted in a filing unsealed Friday that he inadvertently gave his lawyer fake legal case citations generated by artificial intelligence in connection with a motion to end his supervised release early. U.S. District Judge Jesse M. Furman previously called the citations into question, writing earlier this month, "In the letter brief, Mr. Cohen asserts that, "[a]s recently as 2022, there have been District Court decisions, affirmed by the Second Circuit Court, granting early termination of supervised release." Furman added, "As far as the Court can tell, none of these cases exist." Cohen said in his sworn declaration released Friday that he had found the phony citations through Google Bard, an AI service that he said he thought was a "supercharged" search engine. Michael Cohen admitted to inadvertently citing fake legal cases in a motion to end his early release in a sworn declaration released Friday. "As a non-lawyer, I have not kept up with emerging trends (and related risks) in legal technology and did not realize that Google Bard was a generative text service that, like Chat-GPT, could show citations and descriptions that looked real but actually were not," Cohen said. "Instead, I understood it to be a super-charged search engine and had repeatedly used it in other contexts to (successfully) find accurate information online." In 2018, Cohen pleaded guilty to tax evasion, campaign finance charges and lying to Congress, spending more than a year in prison before he was put on supervised release. He was also disbarred as a lawyer. "It did not occur to me then and remains surprising to me now--that Mr. Schwartz would drop the cases into his submission wholesale without even confirming that they existed," he added, citing his lawyer David Schwartz. "I deeply regret any problems Mr. Schwartz's filing may have caused." He said Schwartz's alleged mistake was "a product of inadvertence, not any intent to deceive." E. Danya Perry, who represents Cohen and discovered the citations were fake, told the judge, "Mr.

Donald Trump's former "fixer," Michael Cohen, used Google Bard to cite made-up legal cases that ended up in a federal court. The New York Times reported Friday that Cohen admitted in unsealed court papers that he passed on documents referencing bogus cases to his lawyer, who then relayed them to a federal judge. Cohen reportedly wrote in the sworn declaration he hadn't stayed on top of "emerging trends (and related risks) in legal technology." Cohen's legal team filed the paperwork in a motion asking for an early end to court supervision from his 2018 campaign finance case, for which he served three years in prison. After Cohen's attorney, David M. Schwartz, presented the legal documents to the federal court, Judge Jesse M. Furman of the Federal District Court said he was having trouble finding the three decisions cited by Schwartz (via Cohen).

Google's Chrome has long featured the ability to launch the browser in Incognito mode, offering a seemingly blank slate for your internet browsing, away from your usual cookies, forms and web history. But that seemingly didn't mean Google wasn't keeping an eye on where you were browsing. The company faced a lawsuit in 2020 that accused it of tracking Chrome users' activities even when they were using Incognito mode. Google has now agreed to settle the complaint that originally sought $5 billion in damages, after failing to get the suit dismissed. The plaintiffs said Google used tools like its Analytics product, apps and browser plug-ins to monitor users.

You can manage your newsletter subscriptions at any time. As is now tradition, the final quiz of the year is a look back at the past 12 months. It was a year fraught with discord, so grab your favorite beverage, maybe a cookie or two, and take a deep, relaxing breath before you plunge into 2023 for one last time in this week's Slate News Quiz. If this is your first time playing, read the rules here. The quiz may require you to turn on cookies in your browser for it to function properly.

America's Newsroom anchor Bill Hemmer looks back at the top headlines of the past 12 months. Supreme Court rulings, wars waged over parental rights, crackdowns on conservative school boards and scandals that imbued some districts with controversy: Education's rocky landscape showcased this year's equally tumultuous cultural climate, and the issue has taken center stage for candidates going into 2024. Republicans continued to capitalize on parents' concerns that children are being exposed to age-inappropriate content in the classroom while calling for school choice and cautioning against giving transgender students access to single-sex spaces. Democrats, meanwhile, called out the opposition for alleged "book bans" and a majority defended transgender students' access to spaces corresponding with their preferred gender. The gridlock is expected to augment the intensity of an already explosive election season next year, and the issues aren't expected to fade anytime soon.

The artificial intelligence (AI) industry began 2023 with a bang as schools and universities struggled with students using OpenAI's ChatGPT to help them with homework and essay writing. Less than a week into the year, New York City Public Schools banned ChatGPT – released weeks earlier to enormous fanfare – a move that would set the stage for much of the discussion around generative AI in 2023. As the buzz grew around Microsoft-backed ChatGPT and rivals like Google's Bard AI, Baidu's Ernie Chatbot and Meta's LLaMA, so did questions about how to handle a powerful new technology that had become accessible to the public overnight. In March, a group of more than 1,000 signatories, including Apple co-founder Steve Wozniak and billionaire tech entrepreneur Elon Musk, called for a pause in the development of more advanced AI in light of its "profound risks to society and humanity". While a pause did not happen, governments and regulatory authorities began rolling out new laws and regulations to set guardrails on the development and use of AI.