Research Project
CHLOE-GRAPH: ClotH-Like ObjEcts Grasping, Representation, and Action Planning
Type
National Project
Start Date
01/09/2021
End Date
31/08/2024
Project Code
PID2020-118649RB-I00
Staff
-
-
Alenyà, Guillem
Principal Investigator
-
Grosch, Patrick John
Researcher
-
Torras, Carme
Researcher
-
Foix, Sergi
Researcher
-
Colomé, Adrià
Researcher
-
Jiménez, Pablo
Researcher
-
Ponsa, Sergi
PhD Student
-
Garcia, Irene
PhD Student
-
Tzelepis, Georgios
PhD Student
-
Corona, Enric
PhD Student
-
Caldarelli, Edoardo
PhD Student
-
Suárez, Alejandro
PhD Student
-
Olivares, Alberto
PhD Student
-
Delgado, Juan Antonio
Member
Project Description
Project PID2020-118649RB-I00 funded by MCIN/ AEI /10.13039/501100011033
CHLOE-GRAPH proposes a novel robotic manipulation approach for cloth manipulation that is expected to be general enough to be applied to different assistive tasks. While rigid object manipulation has achieved a decent maturity, robots are yet to make an impact in the handling of deformable objects. Cloth-like objects are extremely challenging due to the complexity of understanding their deformation often with severe self-occlusions, making learning from real data very difficult. This has led to a tendency to develop techniques focused on specific tasks, resulting in a lack of general approaches that provide solutions that are transferable to different object types and tasks. Robots in assistive environments, able to manipulate cloth-like objects, would be a major step, able to provide new business opportunities.
In CHLOE-GRAPH we argue that the solution must embrace the different robotic layers, perception, decision making, and execution. We propose to develop a novel CHLOE (ClotH-Like ObjEcts) scene state representation (SSR) carefully designed to simplify its recognition, but including detailed information on the parts that have to be manipulated to execute the next action and also semantic tags designed for high-level planning.
The 4 objectives of CHLOE-GRAPH are
(O1) - To define the CHLOE-SSR by selecting semantic tags and parameters depending on the requirements from perception, execution, and high-level planning of manipulation tasks with textiles. The representation will have a central role: it will help to determine what has to be perceived, will be used to bootstrap planning operators, and will define the desired abilities of the manipulation actions.
(O2) To develop a CHLOE-SSR recognition method based on deep learning from real data labelled with the novel representation tags. We plan to combine computational topological methods with deep learning architectures to identify only the relevant features of the 2D surface in a 3D world, such as the environment contacts, the grasp location, the deformation category, and other manipulation affordance related information. We propose that only the relevant parts for manipulation need to be understood, not the full deformation state.
(O3) To implement a set of low-level primitives with strategies to accomplish transitions from state to state using the information in CHLOE-SSR. Depending on the complexity of each transition, we will use learning from demonstration techniques, end-to-end deep learning sim-to-real methods with dynamics extending from previous results, or model predictive control, implementing the parameters of the SSR within the motion characterization to allow for trajectory adaptability to the environment.
(O4) To develop a high-level decision-making framework to decide robot actions among the alternative plans, taking into account the non-complete observability of the state and the uncertainty on the action execution. We will contribute to three challenges: the representation, the complexity related to the high-low level gap, and the use of experiences to refine the learned transition model.
We will also propose novel measures to benchmark cloth manipulation tasks related to the segmentation steamed from the new representation, and demonstrate the validity of our approach in two robotic demonstrators.
Project Publications
Journal Publications
-
A. Luque, D. Parent, A. Colomé, C. Ocampo-Martínez and C. Torras. Model predictive control for dynamic cloth manipulation: Parameter learning and experimental validation. IEEE Transactions on Control Systems Technology: 1-17, 2024, to appear.
Abstract Info PDF
-
D. Blanco-Mulero, O. Barbany, G. Alcan, A. Colomé, C. Torras and V. Kyrki. Benchmarking the sim-to-real gap in cloth manipulation. IEEE Robotics and Automation Letters, 9(3): 2981-2988, 2024.
Abstract Info PDF
-
G. Canal, C. Torras and G. Alenyà. Generating predicate suggestions based on the space of plans: an example of planning with preferences. User Modeling and User-Adapted Interaction, 33: 333-357, 2023.
Abstract Info PDF
-
F. Coltraro, J. Fontana, J. Amorós, M. Alberich-Carramiñana, J. Borràs and C. Torras. A representation of cloth states based on a derivative of the Gauss Linking Integral. Applied Mathematics and Computation, 457: 128165, 2023.
Abstract Info PDF
-
A. San-Miguel, V. Puig and G. Alenyà. Condition-based design of variable impedance controllers from user demonstrations. Control Engineering Practice, 140: 105658, 2023.
Abstract Info PDF
-
A. Olivares-Alarcos, S. Foix, S. Borgo and G. Alenyà. OCRA – An ontology for collaborative robotics and adaptation. Computers in Industry, 132: 103627, 2022.
Abstract Info PDF
-
I. Garcia-Camacho, J. Borràs, B. Calli, A. Norton and G. Alenyà. Household cloth object set: Fostering benchmarking in deformable object manipulation. IEEE Robotics and Automation Letters, 7(3): 5866-5873, 2022.
Abstract Info PDF
Conference Publications
-
G. Tzelepis, A. Eren Erdal, J. Borràs and G. Alenyà. Semantic state estimation in robot cloth manipulations using domain adaptation from human demonstrations, 19 International Conference on Computer Vision Theory and Applications, 2024, Rome, pp. 172-182.
Abstract Info PDF
-
T. Love, A. Andriella and G. Alenyà. Towards explainable proactive robot interactions for groups of people in unstructured environments, 2024 ACM/IEEE International Conference on Human-Robot Interaction, 2024, Boulder, CO, USA, pp. 697–701.
Abstract Info PDF
-
A. Civit, A. Andriella, C. Barrue, M. Antonio, C. Boqué and G. Alenyà. Introducing social robots to assess frailty in older adults, 2024 ACM/IEEE International Conference on Human-Robot Interaction, 2024, Boulder, CO, USA, pp. 342–346.
Abstract Info PDF
-
J. Borràs, A. Boix, S. Foix and C. Torras. A virtual reality framework for fast dataset creation applied to cloth manipulation with automatic semantic labelling, 2023 IEEE International Conference on Robotics and Automation, 2023, London (UK), pp. 11605-11611.
Abstract Info PDF
-
A. Olivares-Alarcos, A. Andriella, S. Foix and G. Alenyà. Robot explanatory narratives of collaborative and adaptive experiences, 2023 IEEE International Conference on Robotics and Automation, 2023, London (UK), pp. 11964-11971.
Abstract Info PDF
-
S. Izquierdo, G. Alenyà and C. Rizzo. Adaptive Human-Robot Collaboration: evolutionary learning of action costs using an action outcome simulator, 32nd IEEE International Symposium on Robot and Human Interactive Communication, 2023, Busan, Korea, pp. 1901-1907.
Abstract Info PDF
-
R. Cupec, I. Vidovic, V. Simundic, P. Pejic, S. Foix and G. Alenyà. Teaching a robot where doors and drawers are and how to handle them, 32nd IEEE International Symposium on Robot and Human Interactive Communication, 2023, Busan, Korea, pp. 2288-2294.
Abstract Info PDF
-
A. Suárez, A. Andriella, C. Torras and G. Alenyà. User interactions and negative examples to improve the learning of semantic rules in a cognitive exercise scenario, 2023 IEEE/RSJ International Conference on Intelligent Robots and Systems, 2023, Detroit, MI, USA, pp. 7953-7960.
Abstract Info PDF
-
G. Tzelepis, J. Borràs, A. Eren Erdal and G. Alenyà. Semantic state prediction in robotic cloth manipulation, 2023 Intelligent Systems Conference, 2023, Amsterdam, pp. 205-219, Springer Lecture Notes in Networks and Systems.
Abstract Info PDF
-
S. Izquierdo, G. Canal, C. Rizzo and G. Alenyà. Improved task planning through failure anticipation in human-robot collaboration, 2022 IEEE International Conference on Robotics and Automation, 2022, Philadelphia, Pennsylvania, USA, pp. 7875-7880.
Abstract Info PDF
-
I. Garcia-Camacho, J. Borràs and G. Alenyà. Knowledge representation to enable high-level planning in cloth manipulation tasks, 2022 ICAPS Workshop on Knowledge Engineering for Planning and Scheduling, 2022, Singapore (Virtual), pp. 9.
Abstract Info PDF
-
B.M. Lopes, R. Marques , T. Santos , J. Sousa and G. Alenyà. Relevant parameters identification in traditional & stretch and blow thermoplastics injection molding, 10th IEEE International Conference on Computational Intelligence and Virtual Environments for Measurement Systems and Applications, 2022, Chemnitz, Germany , pp. 1-6, IEEE.
Abstract Info PDF
-
B.M. Lopes, J. Sousa and G. Alenyà. Machine learning methods for quality prediction in thermoplastics injection molding, 2021 IEEE International Conference on Electrical, Computer and Energy Technologies, 2021, Cape Town, South Africa (Virtual), pp. 1-6.
Abstract Info PDF
Follow us!