Goto

Collaborating Authors

 knoll


Language-Enhanced Mobile Manipulation for Efficient Object Search in Indoor Environments

arXiv.org Artificial Intelligence

Enabling robots to efficiently search for and identify objects in complex, unstructured environments is critical for diverse applications ranging from household assistance to industrial automation. However, traditional scene representations typically capture only static semantics and lack interpretable contextual reasoning, limiting their ability to guide object search in completely unfamiliar settings. To address this challenge, we propose a language-enhanced hierarchical navigation framework that tightly integrates semantic perception and spatial reasoning. Our method, Goal-Oriented Dynamically Heuristic-Guided Hierarchical Search (GODHS), leverages large language models (LLMs) to infer scene semantics and guide the search process through a multi-level decision hierarchy. Reliability in reasoning is achieved through the use of structured prompts and logical constraints applied at each stage of the hierarchy. For the specific challenges of mobile manipulation, we introduce a heuristic-based motion planner that combines polar angle sorting with distance prioritization to efficiently generate exploration paths. Comprehensive evaluations in Isaac Sim demonstrate the feasibility of our framework, showing that GODHS can locate target objects with higher search efficiency compared to conventional, non-semantic search strategies. Website and Video are available at: https://drapandiger.github.io/GODHS


Direction Informed Trees (DIT*): Optimal Path Planning via Direction Filter and Direction Cost Heuristic

arXiv.org Artificial Intelligence

Optimal path planning requires finding a series of feasible states from the starting point to the goal to optimize objectives. Popular path planning algorithms, such as Effort Informed Trees (EIT*), employ effort heuristics to guide the search. Effective heuristics are accurate and computationally efficient, but achieving both can be challenging due to their conflicting nature. This paper proposes Direction Informed Trees (DIT*), a sampling-based planner that focuses on optimizing the search direction for each edge, resulting in goal bias during exploration. We define edges as generalized vectors and integrate similarity indexes to establish a directional filter that selects the nearest neighbors and estimates direction costs. The estimated direction cost heuristics are utilized in edge evaluation. This strategy allows the exploration to share directional information efficiently. DIT* convergence faster than existing single-query, sampling-based planners on tested problems in R^4 to R^16 and has been demonstrated in real-world environments with various planning tasks. A video showcasing our experimental results is available at: https://youtu.be/2SX6QT2NOek


GenAI for Automotive Software Development: From Requirements to Wheels

arXiv.org Artificial Intelligence

This paper introduces a GenAI-empowered approach to automated development of automotive software, with emphasis on autonomous and Advanced Driver Assistance Systems (ADAS) capabilities. The process starts with requirements as input, while the main generated outputs are test scenario code for simulation environment, together with implementation of desired ADAS capabilities targeting hardware platform of the vehicle connected to testbench. Moreover, we introduce additional steps for requirements consistency checking leveraging Model-Driven Engineering (MDE). In the proposed workflow, Large Language Models (LLMs) are used for model-based summarization of requirements (Ecore metamodel, XMI model instance and OCL constraint creation), test scenario generation, simulation code (Python) and target platform code generation (C++). Additionally, Retrieval Augmented Generation (RAG) is adopted to enhance test scenario generation from autonomous driving regulations-related documents. Our approach aims shorter compliance and re-engineering cycles, as well as reduced development and testing time when it comes to ADAS-related capabilities.


Survey of GenAI for Automotive Software Development: From Requirements to Executable Code

arXiv.org Artificial Intelligence

Adoption of state-of-art Generative Artificial Intelligence (GenAI) aims to revolutionize many industrial areas by reducing the amount of human intervention needed and effort for handling complex underlying processes. Automotive software development is considered to be a significant area for GenAI adoption, taking into account lengthy and expensive procedures, resulting from the amount of requirements and strict standardization. In this paper, we explore the adoption of GenAI for various steps of automotive software development, mainly focusing on requirements handling, compliance aspects and code generation. Three GenAI-related technologies are covered within the state-of-art: Large Language Models (LLMs), Retrieval Augmented Generation (RAG), Vision Language Models (VLMs), as well as overview of adopted prompting techniques in case of code generation. Additionally, we also derive a generalized GenAI-aided automotive software development workflow based on our findings from this literature review. Finally, we include a summary of a survey outcome, which was conducted among our automotive industry partners regarding the type of GenAI tools used for their daily work activities.


On the Convexity and Reliability of the Bethe Free Energy Approximation

arXiv.org Machine Learning

The Bethe free energy approximation provides an effective way for relaxing NP-hard problems of probabilistic inference. However, its accuracy depends on the model parameters and particularly degrades if a phase transition in the model occurs. In this work, we analyze when the Bethe approximation is reliable and how this can be verified. We argue and show by experiment that it is mostly accurate if it is convex on a submanifold of its domain, the 'Bethe box'. For verifying its convexity, we derive two sufficient conditions that are based on the definiteness properties of the Bethe Hessian matrix: the first uses the concept of diagonal dominance, and the second decomposes the Bethe Hessian matrix into a sum of sparse matrices and characterizes the definiteness properties of the individual matrices in that sum. These theoretical results provide a simple way to estimate the critical phase transition temperature of a model. As a practical contribution we propose $\texttt{BETHE-MIN}$, a projected quasi-Newton method to efficiently find a minimum of the Bethe free energy.


Israeli start-up Apply Design creates virtual interior designs with AI

#artificialintelligence

Israeli start-up Apply Design has closed a $3.3 million seed funding round for its platform, which automatically creates virtual interior designs for empty property images. The capital raised will be used to increase the size of the Apply Design team, and to develop more products that will allow homeowners to sell and advertise their properties online. The seed round was led by top US funds like Urban Innovation Fund, Cathexis Ventures, Goodwater Capital, and Y Combinator, which Apply Design joined during their latest summer batch, alongside earlier pre-seed investors like Fusion, Secret Chord, and other strategic real estate angel investors. Prior to founding Apply Design in 2018, co-founders Yaniv Knoll (CTO) and Asaf Amit (CEO) worked together in an elite intelligence unit in the IDF that specializes in 3D solutions for special operations. The duo drew upon that prior expertise in order to develop what they tout as "the next Shopify for property listings."


Insurance drones, boots on the ground, and big data

#artificialintelligence

Drones are often touted as being the answer to a great number of modern day challenges. Soon, we are told, they will be making shopping deliveries for us, dropping off pizzas, and even taxiing us around. A similar expectation about drone capability has been seen in the world of insurance. It was thought that insurers would have large drone divisions, allowing them to easily assess large or hazardous properties, or make claims assessments on otherwise difficult to view property. But this has not happened for a few reasons.


Motion capture and visual effects bring back Tarkin for 'Rogue One'

Los Angeles Times

One of the best-kept secrets of 2016 was the fact that a major character in Gareth Edwards' "Rogue One: A Star Wars Story" would be appearing on screen for the first time since the actor who portrayed him passed away over 20 years ago. Through visual effects wizardry and a live-action performance by actor Guy Henry, the commander of the first Death Star in 1977's "Star Wars," Grand Moff Tarkin, was brought back to the big screen as though the late Peter Cushing was still portraying him. For John Knoll, it was the most difficult aspect of his responsibilities as visual effects supervisor on the global blockbuster. An Oscar winner for his work on "Pirates of the Caribbean – Dead Man's Chest," Knoll believes the illusion wouldn't have succeeded without Henry's presence. The effects team's job was effectively that of someone who would be creating cosmetic or prosthetic makeup.


Meet John Knoll, the Creative Genius Who Brought Rogue One to Life

WIRED

In one corner of John Knoll's office at Lucasfilm stand three racks of imposing black computer servers. The sleek 6-foot-tall towers, complete with mechanical switches and fans, flash blue LEDs. Each bears the insignia of the Galactic Empire from Star Wars and a name--Death Star 748, Death Star 749. As impressive and menacing as the machines appear, they aren't real. They're just faceplates wired with Arduino controllers to make the lights blink and flutter like actual computers. They are, in other words, visual effects--and a look into the mind of Knoll, the 54-year-old chief creative officer of Industrial Light & Magic, Lucasfilm's famed VFX arm. They're what made the movies. They come from the machines that spent roughly 13,000 hours rendering digital effects for the three Star Wars prequels, on which Knoll was a lead effects supervisor. The march of Moore's law turned the server farm that created those movies into scrap. "It took a few weeks," Knoll says, shrugging.


Evaluating Description and Reference Strategies in a Cooperative Human-Robot Dialogue System

AAAI Conferences

We then describe In this paper, we describe a user evaluation of a humanrobot a study which assessed the responses of naïve users dialogue system that is designed to enable a humanoid to output that varied along two dimensions: the robot to cooperate with a human partner on building wooden method of describing an assembly plan (pre-order construction toys. In the evaluation, we experimentally vary or post-order), and the method of referring to objects two aspects of the output generated by the system: the way in the world (basic and full). Varying both that it describes assembly plans to the user, and the way that of these factors produced significant results: subjects it refers to objects in the world. We then measure the impact using the system that employed a pre-order of varying each of these features on the users' objective success description strategy asked for instructions to be repeated at working with the system, as well as on their subjective significantly less often than those who experienced impressions of the interaction.