Goto

Collaborating Authors

 peek


PEEK: Guiding and Minimal Image Representations for Zero-Shot Generalization of Robot Manipulation Policies

Zhang, Jesse, Memmel, Marius, Kim, Kevin, Fox, Dieter, Thomason, Jesse, Ramos, Fabio, Bıyık, Erdem, Gupta, Abhishek, Li, Anqi

arXiv.org Artificial Intelligence

Robotic manipulation policies often fail to generalize because they must simultaneously learn where to attend, what actions to take, and how to execute them. We argue that high-level reasoning about where and what can be offloaded to vision-language models (VLMs), leaving policies to specialize in how to act. We present PEEK (Policy-agnostic Extraction of Essential Keypoints), which fine-tunes VLMs to predict a unified point-based intermediate representation: 1. end-effector paths specifying what actions to take, and 2. task-relevant masks indicating where to focus. These annotations are directly overlaid onto robot observations, making the representation policy-agnostic and transferable across architectures. To enable scalable training, we introduce an automatic annotation pipeline, generating labeled data across 20+ robot datasets spanning 9 embodiments. In real-world evaluations, PEEK consistently boosts zero-shot generalization, including a 41.4x real-world improvement for a 3D policy trained only in simulation, and 2-3.5x gains for both large VLAs and small manipulation policies. By letting VLMs absorb semantic and visual complexity, PEEK equips manipulation policies with the minimal cues they need--where, what, and how. Website at https://peek-robot.github.io/.


Government drones used in 'runaway spying operation' to peek into backyards in Sonoma County, lawsuit says

Los Angeles Times

Three residents filed a lawsuit this week against Sonoma County seeking to block code enforcement from using drones to take aerial images of their homes in what the American Civil Liberties Union is calling a "runaway spying operation." The lawsuit, filed by the ACLU Wednesday on behalf of the three residents, alleges that the county began using drones with high-powered cameras and zoom lenses in 2019 to track illegal cannabis cultivation, but in the years since, officials have used the devices more than 700 times to find other code violations on private property without first seeking a warrant. "For too long, Sonoma County code enforcement has used high-powered drones to warrantlessly sift through people's private affairs and initiate charges that upend lives and livelihoods. All the while, the county has hidden these unlawful searches from the people they have spied on, the community, and the media," Matt Cagle, a senior staff attorney with the ACLU Foundation of Northern California, said in a statement. A spokesperson for Sonoma County said the county is reviewing the complaint and takes "the allegations very seriously."


New Bird Buddy smart garden products let you peek into the secret lives of pollinators

Engadget

The makers of the camera-equipped bird feeder, Bird Buddy, introduced two new products at CES 2025 under a new brand called Wonder that let you spy on nature and help pollinators thrive. Petal, a solar-powered camera with changeable lenses and Nature Intelligence (aka AI), can be mounted with a clip, a flexible arm or a stem, so it can be set up pretty much wherever you want outdoors. It'll analyze everything it sees to let you know what birds, insects and other critters stopped by. The second product, Wonder Blocks, is a modular system that's kind of like an apartment building for bugs and birds. The Petal camera comes in soft, bright colors like orange, blue and yellow, so it would look right at home in a flower pot or wrapped around the thin branch of a tree.


Here's a peek at how A Minecraft Movie will handle crafting

Engadget

The team behind the upcoming Minecraft movie shared a new clip during Minecraft Live that expands on the brief crafting moment we saw in the polarizing first teaser. The scene comes in the middle of a discussion between Mojang creative director Torfi Frans Olafsson and A Minecraft Movie director Jared Hess, at 4:51. The segment also gives us our first look at the movie's interpretation of a Minecraft bee, which I'm not quite sure how to feel about yet. That you can find toward the end of the video. A Minecraft Movie is slated for release in April 2025 and stars Jack Black as Steve, alongside Jason Momoa, Danielle Brooks, Emma Myers and Sebastian Eugene Hansen.


A peek inside San Francisco's AI boom

Washington Post - Technology News

In an opulent ballroom on a Saturday night, the classic pump-up anthem "Eye of the Tiger" blared as artificial intelligence enthusiasts tapped away on their keyboards. This was a hackathon -- an event where participants have a set amount of time to collaborate on a project they present to the crowd -- at a sprawling mansion about 30 minutes south of San Francisco. As a professional freelance photographer, I've spent the past decade documenting the people and culture of Silicon Valley. Ever since OpenAI's ChatGPT debuted in November 2022, countless entrepreneurs have been inspired to make their own generative AI tools. Now, nearly every new start-up has an AI element -- technology that automates simple tasks, for example, or a chatbot that provides mental health tips.


OpenAI Offers a Peek Inside the Guts of ChatGPT

WIRED

ChatGPT developer OpenAI's approach to building artificial intelligence came under fire this week from former employees who accuse the company of taking unnecessary risks with technology that could become harmful. Today OpenAI released a new research paper apparently aimed at showing it is serious about tackling AI risk by making its models more explainable. In the paper, researchers from the company lay out a way to peer inside the AI model that powers ChatGPT. They devised a way to identify how it stores certain concepts--including those that might perhaps cause an AI system to misbehave. Although the research makes OpenAI's work on keeping AI in check more visible, it also highlights recent turmoil at the company.


From a sneak peek to Google's hotly-anticipated Pixel 9 phone to new AI features: What to expect at the tech giant's unveiling event tomorrow

Daily Mail - Science & tech

As Google's I/O conference kicks off tomorrow evening, avid tech fans can certainly expect some exciting news from the company's biggest event of the year. The event, held in Mountain View, California, will be live-streamed here from 18:00 BST (10:00 PST) tomorrow. And, following the limited success of Apple's recent iPad launch, expectations will be high for the tech giant to deliver something impressive. With Google's Pixel 8A officially shipping today, there is a chance we might get a sneak peek of the hotly anticipated Pixel 9 or even a new foldable phone. But while there might be some updates on Google's hardware, you can expect AI to be the biggest topic of the day.


BMW's Vision Neue Klasse X concept gives us a peek at BMW's future SUV

Engadget

Last year, at the IAA show in Berlin, BMW gave us a first look at what it calls the Neue Klasse, a "new class" of sedan that would not only drive BMW into its electric future, but also create a platform with greater efficiency and driving dynamics than we've yet seen from the German brand. Now, the company is showing us an evolution of that concept. A taller one at that, to see just how that same idea for future electrification would apply to an SUV. In keeping with BMW's naming conventions, this one's called the Neue Klasse X. The styling is somewhat predictable in that regard, since it is a taller version of what we saw in the Neue Klasse sedan.


Unlocking Context Constraints of LLMs: Enhancing Context Efficiency of LLMs with Self-Information-Based Content Filtering

Li, Yucheng

arXiv.org Artificial Intelligence

Large language models (LLMs) have received significant attention by achieving remarkable performance across various tasks. However, their fixed context length poses challenges when processing long documents or maintaining extended conversations. This paper proposes a method called \textit{Selective Context} that employs self-information to filter out less informative content, thereby enhancing the efficiency of the fixed context length. We demonstrate the effectiveness of our approach on tasks of summarisation and question answering across different data sources, including academic papers, news articles, and conversation transcripts.


'Vulkan' Leak Offers a Peek at Russia's Cyberwar Playbook

WIRED

Did you hear that Donald Trump got indicted this week? The first-ever indictment of a former US president had been looming for weeks. And now that it's happened, the move by a Manhattan grand jury is deepening fissures in America's already-fraught political divide. But while Trump headlines flood your feeds, there were plenty of other big stories this week, none of which have anything to do with any of that. In Germany, police are cracking down on people who post adult content to websites and platforms that lack age-verification checks, like Twitter.