GEPETTO Research Group

In order for humans and robots to cooperate in an effective manner, it must be possible for human users to efficiently communicate to the robot its role in accomplishing the coordinated task. Spoken language is an obvious candidate. The current resarch analyses how spoken language can be used by human users to communicate with the HRP-2 humanoid to program the robot's behavior in a cooperative task. The task involves the humans and the HRP-2 working together to assemble a piece of furniture. The objectives of the system are to 1. Allow the human to impart knowlefge of how to accomplish a cooperative task to the robot, i.e. to program the robot, in the form of a sensory-motor action plan. @. To do this in a semi-natural and real-time manner using spoken language. In this framework, a system for Spoken Language Programming (SLP) is presented, and experimental results are presented from this prototype system. In Experiement 1, the human programs the robots to assist in assembling a small table. In Experiment 2, the generalization of the system is demonstrated as the user programs the robot to assist in taking the table apart. The SLP is evaluated in terms of the changes in efficiency as revealed by task completion time and number of command operations required to accomplish the tasks. Lessons learned are discussed, along with plans for improving the system, including developing a richer base of robot action and perception predicateds that will allow the use of richer language.

@inproceedings\{dominey:icra07, author = \{P. F. Dominey and A. Mallet and E. Yoshida}, title = \{Progress in Programming the HRP-2 Humanoid Using Spoken Language}, crossref = \{icra07}, url = \{http://www.laas.fr/gepetto/papers/dominey-icra07.pdf}, movie = \{http://www.laas.fr/gepetto/movies/dominey-icra07.mp4} }

In this paper we address a task-driven motion generation method that allows a humanoid robot to make wholebody motions including support polygon reshaping to achieve the given tasks. In the proposed method, generalized inverse kinematics (IK) is employed to generate humanoid whole-body motions that enable the robot to accomplish the tasks according to given priorities. During the motion, several criteria such as manipulability or end-effector position error are tracked. If the desired task cannot be done because of those criteria, a geometric planner for support polygon is activated. Then the whole-body motion is computed again always using the generalized IK solver including stepping motion that realizes the deformed support polygon by using dynamic walking pattern generator. This method provides a way to perform tasks that could not be done without changing the humanoid's support state. We have verified the proposed framework through simulations humanoid robot HRP-2.

@inproceedings\{yoshida:humanoid06, author = \{E. Yoshida and O. Kanoun and C. Esteves and J-P. Laumond}, title = \{Task-driven Support Polygon Humanoids}, crossref = \{humanoid06}, url = \{http://www.laas.fr/gepetto/papers/yoshida-humanoid06.pdf} }

This paper addresses an integrated humanoid motion planning scheme including both advanced algorithmic motion planning technique and dynamic pattern generator so that the humanoid robot achieve tasks including dynamic motions. A two-stage approach is proposed for this goal. First, geometric and kinematic motion planner first computes collision-free paths for the humanoid robot. Then the dynamic pattern generator provides dynamically feasible humanoid motion including both locomotion and task execution such as object transportation or manipulation. If the generated dynamic motion causes collision due to dynamic movements, the planner go back to the planning stage to remove the collision by path reshaping. This iterative planning scheme enables robust planning against variation of task dynamics. Simulation results are provided to validate the proposed planning method.

@inproceedings\{yoshida:humanoid05, author = \{E. Yoshida and I. Belousov and C. Esteves and J-P. Laumond}, title = \{Humanoid Motion Planning for Dynamic Tasks}, crossref = \{humanoid05}, url = \{http://www.laas.fr/gepetto/papers/yoshida-humanoid05.pdf} }

In this paper we address smooth and collision-free whole-body motion planning for humanoid robots. A two-stage iterative planning framework is introduced where geometric motion planner and dynamic pattern generator interacts by exchanging the trajectory, to obtain 3D whole-body dynamic motions simultaneous tasks including locomotion, in complex environments. We propose a practical method for smooth motion reshaping to avoid collisions in generated dynamic motion. Based on motion editing techniques in computer graphics animation, smooth collision-avoiding motion is generated through trajectory deformation. The validity of the proposed reshaping method is verified by computer simulations and experiments using humanoid platform HRP-2.

@inproceedings\{yoshida|iros06, author = \{E. Yoshida and C. Esteves and T. Sakaguchi and J-P. Laumond and K. Yokoi}, title = \{Smooth Collision Avoidance: Practical Issues in Dynamic Humanoid Motion}, crossref = \{iros06}, url = \{http://www.laas.fr/gepetto/papers/yoshida-iros06.pdf}, movie = \{http://www.laas.fr/gepetto/movies/HRP2-dynamic-motion-planning.mp4} }

We present a fast GPU-based algorithm to approximate the Swept Volume (SV) boundary of arbitrary polygon soup models. Despite the extensive research on calculating the volume swept by an object along a trajectory, the efficient algorithms described have imposed constraints on both the trajectories and geometric models. By proposing a general algorithm that handles flat surfaces as well as volumes and disconnected objects, we allow SV calculation without resorting to pre-processing mesh repair. This is of particular interest in the domain of Product Lifecycle Management (PLM), which deals with industrial Computer Aided Design (CAD) models that are malformed more often than not. We incorporate the bounded distance operator used in path planning to efficiently sample the trajectory while controlling the total error. We develop a triangulation scheme that draws on the unique data set created by an advancing front level-set method to tessellate the SV boundary in linear time. We analyze its performance, and demonstrate its effectiveness both theoretically and on real cases taken from PLM.

@inproceedings\{himmelstein:icra07, author = \{J. C. Himmelstein and E. Ferré and J-P. Laumond}, title = \{Swept Volume approximation of polygon soups}, crossref = \{icra07}, annote = \{Long version submitted to IEEE T-ASE}, url = \{http://www.laas.fr/gepetto/papers/himmelstein-icra07.pdf}, movie = \{http://www.laas.fr/gepetto/movies/sv-exhaust.wmv http://www.laas.fr/gepetto/movies/sv-human.wmv http://www.laas.fr/gepetto/movies/sv-seat.wmv http://www.laas.fr/gepetto/movies/sv-flatsurface.wmv} }

Synthesizing human motion signals is difficult because of its multi-dimensional and nonlinear nature. However, the locomotion is a synchronized motion, which means that these signals are related. Using this property, we propose a new method for modeling the human locomotion and identifying this model. The input signals of our model is the trajectory of pelvis and the outputs are corresponding motion signals of whole human body. To identify this system, we considered it as black-box, for which we propose an adapted method of identification using motion capture. We show that the identified model allows to generate various human locomotion in a fast and efficient way.

@inproceedings\{suleiman:iros06, author = \{W. Suleiman and A. Monin and J-P. Laumond}, title = \{Synthesizing and modeling human locomotion using system identification}, crossref = \{iros06}, url = \{http://www.laas.fr/gepetto/papers/suleiman-iros06.pdf} }

Virtual mannequins need to navigate in order to interact with their environment. Their autonomy to accomplish navigation tasks is ensured by locomotion controllers. Control inputs can be user-defined or automatically computed to achieve high-level operations (e.g. obstacle avoidance). This paper presents a locomotion controller based on a motion capture edition technique. Controller inputs are the instantaneous linear and angular velocities of the walk. Our solution works in real time and supports at any time continuous changes of inputs. The controller combines three main components to synthesize locomotion animations in a four-stage process. First, the Motion Library stores motion capture samples. Motion captures are analysed to compute quantitative characteristics. Second, these characteristics are represented in a linear control space. This geometric representation is appropriate for selecting and weighting three motion samples with respect to the input state. Third, locomotion cycles are synthesized by blending the selected motion samples. Blending is done in the frequency domain. Lastly, successive postures are extracted from the synthesized cycles in order to complete the animation of the moving mannequin. The method is demonstrated in this paper in a locomotion-planning context.

@article\{pettre:cawv06, author = \{J. Pettré and J-P. Laumond}, title = \{A motion capture-based control-space approach for walking mannequins}, journal = cawv, year = \{2006}, volume = \{17}, pages = \{109-206}, url = \{http://www.laas.fr/gepetto/papers/pettre-cawv06.pdf}, movie = \{http://www.laas.fr/gepetto/movies/locomotion-control.avi} }

This paper presents an approach to automatically compute animations for virtual (human-like and robot) characters cooperating to move bulky objects in cluttered environments. The main challenge is to deal with 3D collision avoidance while preserving the believability of the agent's behaviors. To accomplish the coordinated task, a geometric and kinematic decoupling of the system is proposed. This decomposition enables us to plan a collision-free path for a reduced system, then to animate locomotion and grasping behaviors independently, and finally to automatically tune the animation to avoid residual collisions. These three steps are applied consecutively to synthesize an animation. The different techniques used, such as probabilistic path planning, locomotion controllers, inverse kinematics and path planning for closed kinematic chains are explained, and the way to integrate them into a single scheme is described.

@article\{esteves:tog06, author = \{C. Esteves and G. Arechavaleta and J. Pettré and J-P. Laumond}, title = \{Animation Planning for Virtual Character Cooperation}, journal = tog, year = \{2006}, volume = \{25}, number = \{2}, month = apr, url = \{http://www.laas.fr/gepetto/papers/esteves-tog06.pdf}, movie = \{http://www.laas.fr/gepetto/movies/motion-coordination.mpg} }

This paper deals with mechanical part assembly planning. The goal is to automatically compute a collision-free path for both the part to be assembled and the mannequin manipulating it. Two approaches are proposed according to the difficulty of the problem. Both are based on a general probabilistic diffusion algorithm working in the configuration space of the considered system. The first approach consists in first planning a path for the part alone and then in checking the feasibility of the solution by adding the mannequin. The second one considers the part grasped and the mannequin as a single system. While the first approach performs quickly the second one is able to solve more constrained and difficult cases. Both solutions are based on the same path planning library allowing the user to easily evaluate the proposed solutions. Experimental results based on feedback experiences in automotive industry are presented.

@inproceedings\{laumond:isatp05, author = \{J-P. Laumond and E. Ferré and G. Arechavaleta and C. Esteves}, title = \{Mechanical Part Assembly Planning with Virtual Mannequins}, crossref = \{isatp05}, url = \{http://www.laas.fr/gepetto/papers/laumond-isatp05.pdf}, movie = \{http://www.laas.fr/gepetto/movies/manikin-disassembly.mov} }

This paper introduces a framework for real-time simulation and rendering of crowds navigating in a virtual environment. The solution first consists in a specific environment preprocessing technique giving rise to navigation graphs, which are then used by the navigation and simulation tasks. Second, navigation planning interactively provides various solutions to the user queries, allowing to spread a crowd by individualizing trajectories. A scalable simulation model enables the management of large crowds, while saving computation time for rendering tasks. Pedestrian graphical models are divided into three rendering fidelities ranging from billboards to dynamic meshes, allowing close-up views of detailed digital actors with a large variety of locomotion animations. Examples illustrate our method in several environments with crowds of up to 35 000 pedestrians with real-time performance.

@article\{pettre:cawv06a, author = \{J. Pettré and P. de Heras Ciechomski and J. Maïm and B. Yersin and J-P. Laumond and D. Thalmann}, title = \{Real-time navigating crowds: scalable simulation and rendering}, journal = cawv, year = \{2006}, volume = \{17}, pages = \{445-455}, url = \{http://www.laas.fr/gepetto/papers/pettre-cawv06a.pdf}, movie = \{http://www.laas.fr/gepetto/movies/crowds-with-EPFL.avi} }

Whether the central nervous system of primates uses a body-centered or an eye-centered frame for reaching is still a controversial debate in neurosciences. Does the proprioceptive information allow to represent the visual information with respect to the body or inversely, does it transform the hand position in retinal coordinates? In this paper, we implement and test two control schemes associated to each of these two hypotheses by using a computational approach from robotics. To this end, biological models of motor control are applied to a realistic dynamic model of upper body including a 4 DOF eye-neck kinematic chain and a 6DOF arm. Important characteristics, such as proprioceptive biases and sensory delay, are considered. Simulation results allow to compare the geometry of trajectories and illustrate the better robustness of the eye-centered control scheme with respect to biases and sensory delay. Applications to the control of the humanoid robot HRP2 are finally presented.

@inproceedings\{tuan:robio09, author = \{M. T. Tuan and P. Soueres and M. Taix and B. Girard}, title = \{Eye-centered vs Body-centered reaching control : A robotics insight into the neuroscience debat}, crossref = \{robio09}, url = \{http://www.laas.fr/gepetto/papers/tuan-robio09.pdf} }

This paper presents an application of neurobiolog- ical motor principles to the control of humanoid robot reaching movements. The model is mainly based on the hypotheses that the energy of motoneurons signals is continuously minimized along the motion and that dynamic and static efforts are processed separately. This paradigm is used to control the robot HRP2 by considering a dynamic model of the arm including, for each degree of freedom, two second order low-pass filters mod- eling the neuromuscular system defined by a pair of antagonist muscles. The optimal control problem is solved as a nonlinear programming problem by using a direct transcription method coupled with the optimization software Ipopt. This approach allows to generate realistic movements with anthropomorphic features such as quasi-rectilinear trajectories and bell-shaped velocity.

@inproceedings\{tuan:biorob08, author = \{M. t. Tuan and P. Soueres and M. Taix and E. Guigon}, title = \{A principled approach to motor control for generating humanoid robot reaching movements}, crossref = \{biorob08}, url = \{http://www.laas.fr/gepetto/papers/tuan-biorob08.pdf} }

This paper addresses the problem of understanding the shape of the locomotor trajectories for a human being walking in an empty space to reach a goal defined both in position and in direction. Among all the possible trajectories reaching a given goal what are the fundamental reasons to chose one trajectory instead of another? The underlying idea to attack this question has been to relate this problem to an optimal control problem: the trajectory is chosen according to some optimization principle. This is our basic starting assumption. The subject being viewed as a controlled system, the question becomes what criteria is optimized? Is it the time to perform the trajectory? the length of the path? the minimum jerk along the path?... In this study we show that the human locomotor trajectories are well approximated by the geodesics of a differential system minimizing the L2 norm of the control. Such geodesics are made of arcs of clothoids. The study is based on an experimental protocol involving 7 subjects. They had to walk within a motion capture room from a fixed starting point, and to cross over distant porches from which both position in the room and orientation were changing over trials.

@inproceedings\{arechavaleta:humanoid06, author = \{G. Arechavaleta and J-P. Laumond and H. Hicheur and A. Berthoz}, title = \{Optimizing principles underlying the shape of trajectories in goal oriented locomotion for humans}, crossref = humanoid06, url = \{http://www.laas.fr/gepetto/papers/arechavaleta-humanoid06.pdf} }

This work presents a differential system which accurately describes the geometry of human locomotor trajectories of humans walking on the ground level, in absence of obstacles. Our approach emphasizes the close relationship between the shape of the locomotor paths in goal-directed movements and the simplified kinematic model of a wheeled mobile robot. This kind of system has been extensively studied in robotics community. From a kinematic perspective, the characteristic of this wheeled robot is the nonholonomic constraint of the wheels on the floor, which forces the vehicle to move tangentially to its main axis. Humans do not walk sideways. This obvious observation indicates that some constraints (mechanical, anatomical...) act on human bodies restricting the way humans generate locomotor trajectories. To model this, we propose a differential system that respects nonholonomic constraints. We validate this model by comparing simulated trajectories with actual (recorded) trajectories produced during goal-oriented locomotion in humans. Subjects had to start from a pre-defined position and direction to cross over a distant porch (position and orientation of the porch were the two manipulated factors). Such comparative analysis is undertaken by making use of numerical methods to compute the control inputs from actual trajectories. Three body frames have been considered : head, pelvis and trunk. It appears that the trunk can be considered as a kind of a steering wheel that steers the human body with a delay of around 0.2 second. This model has been validated on a database of 1,560 trajectories recorded from seven subjects. It opens a promising route to better understand the human locomotion via differential geometry tools successfuly experienced in mobile robotics.

@inproceedings\{arechavaleta:biorob06, author = \{G. Arechavaleta and J-P. Laumond and H. Hicheur and A. Berthoz}, title = \{The nonholonomic nature of human locomotion: a modeling study}, crossref = \{biorob06}, url = \{http://www.laas.fr/gepetto/papers/arechavaleta-biorob06.pdf} }

In this chapter we focus on the modeling of cortical activity related to planning and control of visually guided reaching hand movements in primates. We bring a new light to this problem by considering the visual-servoing framework in robotics. A review of representative theories and models describing the neuronal processes related to 3D representation of space, motor control and visuomotor integration in Neuroscience is first presented. The kinematics and dynamics of manipulators and the basics of visual-servoing techniques in robotics are then recalled. In particular, for the control of a robotic arm with a deported camera, we underline the fact that the task-Jacobian is dependent on all the joints of the kinematic chain linking the camera to the end-effector. This point suggests that the motor activity during a visually guided movement of the hand cannot be completely encoded within a body-centered reference frame as claimed by numerous models in Neuroscience. Finally, we present an experimental result showing the existence of gaze-related signals in the monkey premotor cortex during visually guided reaching. This result corroborates the idea that the eye position with respect to the head and the head position with respect to the body, which belong to the kinematic chain linking the eye to the hand, must also be coded by neurons in premotor and motor cortex.

@article\{soueres:bct07, author = \{P. Souères and C. Jouffrais and S. Celebrini and Y. Trotter}, title = \{Robotics Insights for the Modeling of Visually Guided Hand Movements in Primates}, in = \{To appear in}, journal = \{Biology and Control Theory: Current Challenges, Lecture Notes in Control and Information Sciences, Springer Verlag}, year = \{2007}, url = \{http://www.laas.fr/gepetto/papers/soueres-bct07.pdf} }