Efficient extraction and tracking of surfaces patches from depth-color movies for the semantic representation of manipulations

Work default illustration


  • If you are interested in the proposal, please contact with the supervisors.


In this project, an efficient method for the extraction and tracking of parametric surface patches from depth-color movies is developed. The parametric descriptions of surfaces and their relations will provide a condensed semantic description of scenes, which is urgently needed to analyze movies showing manipulation actions and to implement learning algorithms. Real time issues (computation times) will be an important aspect of this work. The framework will be to the tracking of plant structures (leaves) within the context of the EU project GARNICS (''Gardening with a cognitive system'').

The project roughly consists of the following tasks:

1. Joint calibration of depth (PMD) camera and color (RGB) camera
2. Recording of depth – color movies of scenes involving deformable objects, e.g plants, cloth.
3. Implementation of color-segmentation algorithm
4. Surface extraction combining color segments and sparse depth
5. Tracking of surface segments
6. Semantic segment-based description of scenes.
7. Real-time implementation issues (GPU)

The work is under the scope of the following projects:

  • GARNICS: Gardening with a cognitive system (web)