segment 1
CoRI: Communication of Robot Intent for Physical Human-Robot Interaction
Wang, Junxiang, Küçüktabak, Emek Barış, Zarrin, Rana Soltani, Erickson, Zackory
We introduce CoRI, a pipeline that automatically generates natural language communication of a robot's upcoming actions directly from its motion plan and visual perception. Our pipeline first processes the robot's image view to identify human poses and key environmental features. It then encodes the planned 3D spatial trajectory (including velocity and force) onto this view, visually grounding the path and its dynamics. CoRI queries a vision-language model with this visual representation to interpret the planned action within the visual context before generating concise, user-directed statements, without relying on task-specific information. Results from a user study involving robot-assisted feeding, bathing, and shaving tasks across two different robots indicate that CoRI leads to statistically significant difference in communication clarity compared to a baseline communication strategy. Specifically, CoRI effectively conveys not only the robot's high-level intentions but also crucial details about its motion and any collaborative user action needed. Video and code of our project can be found on our project website: https://cori-phri.github.io/ .
Problem-Oriented Segmentation and Retrieval: Case Study on Tutoring Conversations
Wang, Rose E., Wirawarn, Pawan, Lam, Kenny, Khattab, Omar, Demszky, Dorottya
Many open-ended conversations (e.g., tutoring lessons or business meetings) revolve around pre-defined reference materials, like worksheets or meeting bullets. To provide a framework for studying such conversation structure, we introduce Problem-Oriented Segmentation & Retrieval (POSR), the task of jointly breaking down conversations into segments and linking each segment to the relevant reference item. As a case study, we apply POSR to education where effectively structuring lessons around problems is critical yet difficult. We present LessonLink, the first dataset of real-world tutoring lessons, featuring 3,500 segments, spanning 24,300 minutes of instruction and linked to 116 SAT math problems. We define and evaluate several joint and independent approaches for POSR, including segmentation (e.g., TextTiling), retrieval (e.g., ColBERT), and large language models (LLMs) methods. Our results highlight that modeling POSR as one joint task is essential: POSR methods outperform independent segmentation and retrieval pipelines by up to +76% on joint metrics and surpass traditional segmentation methods by up to +78% on segmentation metrics. We demonstrate POSR's practical impact on downstream education applications, deriving new insights on the language and time use in real-world lesson structures.
Hysteresis Compensation of Flexible Continuum Manipulator using RGBD Sensing and Temporal Convolutional Network
Park, Junhyun, Jang, Seonghyeok, Park, Hyojae, Bae, Seongjun, Hwang, Minho
Flexible continuum manipulators are valued for minimally invasive surgery, offering access to confined spaces through nonlinear paths. However, cable-driven manipulators face control difficulties due to hysteresis from cabling effects such as friction, elongation, and coupling. These effects are difficult to model due to nonlinearity and the difficulties become even more evident when dealing with long and coupled, multi-segmented manipulator. This paper proposes a data-driven approach based on Deep Neural Networks (DNN) to capture these nonlinear and previous states-dependent characteristics of cable actuation. We collect physical joint configurations according to command joint configurations using RGBD sensing and 7 fiducial markers to model the hysteresis of the proposed manipulator. Result on a study comparing the estimation performance of four DNN models show that the Temporal Convolution Network (TCN) demonstrates the highest predictive capability. Leveraging trained TCNs, we build a control algorithm to compensate for hysteresis. Tracking tests in task space using unseen trajectories show that the proposed control algorithm reduces the average position and orientation error by 61.39% (from 13.7mm to 5.29 mm) and 64.04% (from 31.17{\deg} to 11.21{\deg}), respectively. This result implies that the proposed calibrated controller effectively reaches the desired configurations by estimating the hysteresis of the manipulator. Applying this method in real surgical scenarios has the potential to enhance control precision and improve surgical performance.
Extracting Biomedical Factual Knowledge Using Pretrained Language Model and Electronic Health Record Context
Yao, Zonghai, Cao, Yi, Yang, Zhichao, Deshpande, Vijeta, Yu, Hong
Language Models (LMs) have performed well on biomedical natural language processing applications. In this study, we conducted some experiments to use prompt methods to extract knowledge from LMs as new knowledge Bases (LMs as KBs). However, prompting can only be used as a low bound for knowledge extraction, and perform particularly poorly on biomedical domain KBs. In order to make LMs as KBs more in line with the actual application scenarios of the biomedical domain, we specifically add EHR notes as context to the prompt to improve the low bound in the biomedical domain. We design and validate a series of experiments for our Dynamic-Context-BioLAMA task. Our experiments show that the knowledge possessed by those language models can distinguish the correct knowledge from the noise knowledge in the EHR notes, and such distinguishing ability can also be used as a new metric to evaluate the amount of knowledge possessed by the model.
Market Segmentation in the Emoji Era
Ishaan and Elizabeth, both graduate students in business, are attending a marketing strategy lecture at a business school in the Northeast. While learning about the principles of market segmentation, Ishaan texts "outdated" followed by three thinking--face emojis to Elizabeth. He wonders how demographic-, geographic-, or psychographic-based segmentation--the topic of the lecture--can help his family's franchise restaurant deal with the hundreds of sometimes-not-so-positive online reviews and social media posts. Meanwhile, Elizabeth hopes that the fast-food restaurant where she ordered her lunch understands that she now belongs to the segment of'extremely displeased' customers. Earlier, she used the restaurant's new app to order a burrito without cheese and sour cream, only to discover that the meal included both offending ingredients. Her lunch went straight into the trash can and she angrily tweeted her disappointment to the restaurant. Elizabeth replies to Ishaan's text, "that is so passé," followed by a face_with_ rolling_eyes. This simple vignette illustrates an important point. Organizations of every size are challenged with capitalizing on enormous amounts of unstructured organizational data--for instance, from social media posts--particularly for applications such as market segmentation. The purpose of this article is to give the reader an idea of the challenges and opportunities faced by businesses using market segmentation, including the impacts of big data.
MATLAB Master Class: Go from Beginner to Expert in MATLAB
MATLAB (matrix laboratory) is one of the fundamental and leading programming language and is a must learn skill for anyone who want to develop a career in engineering, science or related fields. Excellent MATLAB programming skills is therefore a crucial factor in making or breaking your career. This course is designed from a perspective of a student who has no prior knowledge of MATLAB. The course starts from the very basic concepts and then built on top of those basic concepts and move towards more advanced topics such as visualization, exporting and importing of data, advance data types and data structures and advance programming constructs. To get the real feel of MATLAB in solving and analyzing real life problems, the course includes machine learning topics in data science and data preprocessing. To convert the source codes into meaningful pieces of softwares, the course also covers topics in building GUI's using GUIDE and App Designer utilities of matlab.
10.22.19 Episode 637 Segment 1 – AI Bias
Where are we going with AI (artificial intelligence)? Peggy answers, saying AI is fundamentally changing our society, but that we need to talk about the ethical concerns that AI raises. She asks: Do we know what fairness means in machine learning outcomes? Ironically, AI can reduce bias, but it can also bake in and scale bias. Perhaps it is time to request a bias check and make sure yours is the unbiased AI that will survive.
10.01.19 Episode 634 Segment 1 – AI Augments Work
Peggy says the workplace is changing, thanks to AI (artificial intelligence), explaining that the old model of work won't apply in the future. At the same time, we need to keep in mind where jobs are lost, jobs are created, and we need to look beyond how AI is going to automate away some of the jobs, but how it is going to help people too.
09.17.19 Episode 632 Segment 1 – Delivering a Digital Experience
Peggy talks about the fact that in today's society experience is as important as the products and services, as buyers are more likely to be loyal to a company that they trust. She profiles what today's customer looks like today and how technology and AI (artificial intelligence) can help deliver a better customer experience.