RGB-D Segmentation

Geometric data abstraction and segmentation of RGB-D images

Data abstraction and segmentation of objects is done on four levels of abstraction. The input data gets pre-segmented by calculation of surface normals and recursive clustering of neighbouring pixels to planar patches. On the primitive level planes and B-spline surfaces are fitted to get parametric models of the pre-segmented clusters. A merging algorithm uses model selection to connect neighbouring surface patches if a joint model fits better than two single single models. On the structural and assembly level are grouping rules learned to predict connectedness of neighbouring and non-neighbouring patches. Finally is graph cut used to segment the image globally optimal.


  • Richtsfeld Andreas – ari(at)acin.tuwien.ac.at
  • Mörwald Thomas – moerwald(at)acin.tuwien.ac.at
  • Prankl Johann – prankl(at)acin.tuwien.ac.at


The software is designed to use in Linux (Ubuntu 11.04 – 12.10).

We also provide a “light” version of our segmenter. Results are around 3-5% weaker, but the processing time decreases below 500ms on a state of the art computer.

Other resources

OSD, the Object Segmentation Database, consists of objects on a table top organized in several learn- and test-sets. All evaluation of the segmentation algorithms was done on that dataset.


Richtsfeld A., Mörwald T., Prankl J., Zillich M. and Vincze M. – Segmentation of Unknown Objects in Indoor Environments. IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS), 2012 (pdf)

Example images

Two examples of model abstraction (left to right): Original image, pre-segmented planar patches, merged and abstracted parametric surface models (B-splines and planes), reconstructed and refined scene.

Examples of object segmentation with the proposed structure.