The 3D modelling and primitive extraction research is concerned with obtaining of highly detailed surface reconstruction of the object as well as extracting numerous features and properties. The first step in the process is the object 3D reconstruction. The object of interest is placed on the work area after which the robot starts generating an automated scanning path based on the object’s geometry.
Fig 1. Supervoxels and object parts
In order to estimate the forces and torques that need to be applied on the object we extract three levels of hierarchy of surface information. On the lowest level, for each point on the surface of the object, we estimate the directions of the maximum and minimum curvature towards neighboring points. This information is also called principal curvature. We also group these points into small patches called Supervoxels in order to understand the geometry of the object on a middle level. Finally, on the highest level, we extract object parts and combine all the geometrical data together with the recorded human demonstrator data in order to teach the robot how to perform an action in an humanoid way.
Fig 2. Trajectory generalization
Except for learning the forces and torques that are exerted by the demonstrator we also learn the action trajectories performed on a small set of sample objects and generalize this trajectories onto previously unseen objects. For example, if a human demonstrator has painted six doors we use the learned trajectories to paint unknown doors. This is achieved by calculating unique features for each point of the object and generalizing these features on classes of objects.