Large language models (LLMs), while memorizing parts of their training data scraped from the Internet, may also inadvertently encode societal preferences and norms. As these models are integrated into sociotechnical systems, it is crucial that the norms they encode align with societal expectations. These norms could vary across models, hyperparameters, optimization techniques, and datasets. This is especially challenging due to prompt sensitivity$-$small variations in prompts yield different responses, rendering existing assessment methodologies unreliable. There is a need for a comprehensive framework covering various models, optimization, and datasets, along with a reliable methodology to assess encoded norms. We present LLM-CI, the first open-sourced framework to assess privacy norms encoded in LLMs. LLM-CI uses a Contextual Integrity-based factorial vignette methodology to assess the encoded norms across different contexts and LLMs. We propose the multi-prompt assessment methodology to address prompt sensitivity by assessing the norms from only the prompts that yield consistent responses across multiple variants. Using LLM-CI and our proposed methodology, we comprehensively evaluate LLMs using IoT and COPPA vignettes datasets from prior work, examining the impact of model properties (e.g., hyperparameters, capacity) and optimization strategies (e.g., alignment, quantization).

Executives from Amazon, Apple, AT&T, Charter Communications, Google, and Twitter are heading to Washington Wednesday to testify before the Senate Commerce Committee on the topic of privacy. As ever, the main question will be: Are these companies doing enough to protect consumer privacy, and if not, what should Congress do about it? It has been the backdrop to just about every hearing with tech leaders over the last year--and there have been many. And yet, the threat of regulation carries new weight this time around. Over the summer, California passed the country's first data privacy bill, giving residents unprecedented control over their data.

Talking toys have come a long way since the original Furby. Now they're connected to the Internet, use speech recognition, and are raising a host of new questions about the online privacy and security of children. Hackers have already targeted toys. Late last year, Hong Kong-based digital toy maker Vtech admitted that cybercriminals accessed the personal information of 6.4 million children. Researchers have also shown how hackers can gain control of connected dolls.

In a promotional video for Amazon's Echo virtual assistant device, a young girl no older than 12 asks excitedly: "Is it for me?". The voice-controlled speaker can search the web for information, answer questions and even tell kids' jokes. An investigation by the Guardian has found that despite Amazon marketing the Echo to families with young children, the device is likely to contravene the US Children's Online Privacy Protection Act (COPPA), set up to regulate the collection and use of personal information from anyone younger than 13. Along with Google, Apple and others promoting voice-activated artificial intelligence systems to young children, the company could now face multimillion-dollar fines. "This is part of the initial wave of marketing to children using the internet of things," says Jeff Chester, executive director of the Center for Digital Democracy, a privacy advocacy group that helped write the law.

"In sum: Alexa is kinda my new best friend," writes Rebecca, an author and blogger on Mommyproof. As a mother of young children, it's not hard to see why: Rebecca details the joys of virtual assistant Amazon Echo, which functions as a kind of combined kitchen helper, child entertainer and DJ. "Is it weird that Alexa feels like my trusty little friend during that six pm witching hour, gently guiding me through dinnertime?" Rebecca's not alone: Other parent bloggers have touted the device's family-friendly uses when it comes to child care and household tasks. Many of these "mommy blog" posts are sponsored by Amazon in an ad campaign explicitly geared toward families with young children. In one recent television spot for the device, a young father excitedly purchases an Echo and watches as Alexa slowly becomes an integral part of the household.

In a promotional video for Amazon's Echo virtual assistant device, a young girl no older than 12 asks excitedly: "Is it for me?". The voice-controlled speaker can search the web for information, answer questions and even tell kids' jokes. An investigation by the Guardian has found that despite Amazon marketing the Echo to families with young children, the device is likely to contravene the US Children's Online Privacy Protection Act (COPPA), set up to regulate the collection and use of personal information from anyone younger than 13. Along with Google, Apple and others promoting voice-activated artificial intelligence systems to young children, the company could now face multimillion-dollar fines. "This is part of the initial wave of marketing to children using the internet of things," says Jeff Chester, executive director of the Center for Digital Democracy, a privacy advocacy group that helped write the law.