Mightability: A Multi-State Visuo-Spatial Reasoning for Human-Robot Interaction

Amit Kumar Pandey 1 Rachid Alami 1
1 LAAS-RIS - Équipe Robotique et InteractionS
LAAS - Laboratoire d'analyse et d'architecture des systèmes [Toulouse]
Abstract : We, the Humans, are capable of estimating various abilities of ourselves and of the person we are interacting with. Visibility and reachability are among two such abilities. Studies in neuroscience and psychology suggest that from the age of 12-15 months children start to understand the occlusion of others line-of-sight and from the age of 3 years they start to develop the ability, termed as perceived reachability for self and for others. As such capabilities evolve in the children, they start showing intuitive and proactive behavior by perceiving various abilities of the human partner. Inspired from such studies, which suggest that visuo-spatial perception plays an important role in Human-Human interaction, we propose to equip our robot to perceive various types of abilities of the agents in the workspace. The robot perceives such abilities not only from the current state of the agent but also by virtually putting an agent into various achievable states, such as turn left, stand up, etc. As the robot estimates what an agent might be able to 'see' and 'reach' if will be in a particular state, we term such analyses as Mightability Analyses. Currently the robot is equipped to perform such Mightability analyses at two levels: cells in the 3D grid and objects in the space, which we termed as Mightability Maps (MM) and Object Oriented Mightabilities (OOM) respectively. We have shown the applications of Mightability analyses in performing various cooperative tasks like show and make an object accessible to the human as well as competitive tasks like hide and put away an object from the human. Such Mightability analyses equip the robot for higher-level learning and decisional capabilities as well as could facilitate the robot for better verbalize interaction and proactive behavior.
Complete list of metadatas

Contributor : Rachid Alami <>
Submitted on : Thursday, January 10, 2019 - 5:43:23 PM
Last modification on : Friday, June 14, 2019 - 6:31:22 PM
Long-term archiving on : Thursday, April 11, 2019 - 6:54:06 PM


Files produced by the author(s)



Amit Kumar Pandey, Rachid Alami. Mightability: A Multi-State Visuo-Spatial Reasoning for Human-Robot Interaction. Oussama Khatib, Vijay Kumar, Gaurav Sukhatme. Experimental Robotics. The 12th International Symposium on Experimental Robotics, 79, Springer, pp.49-63, 2014, Springer Tracts in Advanced Robotics, ⟨10.1007/978-3-642-28572-1_4⟩. ⟨hal-01977480⟩



Record views


Files downloads