arXiv ScienceSearch

arXiv subjects

Shuo Hu

Publications and source records attributed to Shuo Hu.

3 recordsLinked to original sources

Pour me a drink: Robotic Precision Pouring Carbonated Beverages into Transparent Containers

With the growing emphasis on the development and integration of service robots within household environments, we will need to endow robots with the ability to reliably pour a variety of liquids. However, liquid handling and pouring is a challenging task due to the complex dynamics and varying properties of different liquids, the exacting precision required to prevent spills and ensure accurate pouring, and the necessity for robots to adapt seamlessly to a multitude of containers in real-world scenarios. In response to these challenges, we propose a novel autonomous robotics pipeline that empowers robots to execute precision pouring tasks, encompassing both carbonated and non-carbonated liquids, as well as opaque and transparent liquids, into a variety of transparent containers. Our proposed approach maximizes the potential of RGB input alone, achieving zero-shot capability by harnessing existing pre-trained vision segmentation models. This eliminates the need for additional data collection, manual image annotations, or extensive training. Furthermore, our work integrates ChatGPT, facilitating seamless interaction between individuals without prior expertise in robotics and our pouring pipeline, this integration enables users to effortlessly request and execute pouring actions. Our experiments demonstrate the pipeline's capability to successfully pour a diverse range of carbonated and non-carbonated beverages into containers of varying sizes, relying solely on visual input.

cs.RO

MPMQA: Multimodal Question Answering on Product Manuals

Visual contents, such as illustrations and images, play a big role in product manual understanding. Existing Product Manual Question Answering (PMQA) datasets tend to ignore visual contents and only retain textual parts. In this work, to emphasize the importance of multimodal contents, we propose a Multimodal Product Manual Question Answering (MPMQA) task. For each question, MPMQA requires the model not only to process multimodal contents but also to provide multimodal answers. To support MPMQA, a large-scale dataset PM209 is constructed with human annotations, which contains 209 product manuals from 27 well-known consumer electronic brands. Human annotations include 6 types of semantic regions for manual contents and 22,021 pairs of question and answer. Especially, each answer consists of a textual sentence and related visual regions from manuals. Taking into account the length of product manuals and the fact that a question is always related to a small number of pages, MPMQA can be naturally split into two subtasks: retrieving most related pages and then generating multimodal answers. We further propose a unified model that can perform these two subtasks all together and achieve comparable performance with multiple task-specific models. The PM209 dataset is available at https://github.com/AIM3-RUC/MPMQA.

cs.CL

The Reconfiguration Pattern of Individual Brain Metabolic Connectome for Parkinson's Disease Identification

Background: Positron Emission Tomography (PET) with 18F-fluorodeoxyglucose (18F-FDG) reveals metabolic abnormalities in Parkinson's disease (PD) at a systemic level. Previous metabolic connectome studies derived from groups of patients have failed to identify the individual neurophysiological details. We aim to establish an individual metabolic connectome method to characterize the aberrant connectivity patterns and topological alterations of the individual-level brain metabolic connectome and their diagnostic value in PD. Methods: The 18F-FDG PET data of 49 PD patients and 49 healthy controls (HCs) were recruited. Each individual's metabolic brain network was ascertained using the proposed Jensen-Shannon Divergence Similarity Estimation (JSSE) method. The intergroup difference of the individual's metabolic brain network and its global and local graph metrics were analyzed to investigate the metabolic connectome's alterations. The identification of the PD from HC individuals was used by the multiple kernel support vector machine (MK-SVM) to combine the information from connection and topological metrics. The validation was conducted using the nest leave-one-out cross-validation strategy to confirm the performance of the methods. Results: The proposed JSSE metabolic connectome method showed the most involved metabolic motor networks were PUT-PCG, THA-PCG, and SMA pathways in PD, which was similar to the typical group-level method, and yielded another detailed individual pathological connectivity in ACG-PCL, DCG-PHG and ACG pathways. These aberrant functional network measures exhibited an ideal classification performance in the identifying of PD individuals from HC individuals at an accuracy of up to 91.84%.

q-bio.NC