Research Project

GARNICS: Gardening with a cognitive system

Type

European Project

Start Date

01/03/2010

End Date

31/12/2013

Project Code

FP7-ICT-2009-4-247947

Project illustration

Staff

Project Description

The GARNICS project aims at 3D sensing of plant growth and building perceptual representations for learning the links to actions of a robot gardener. Plants are complex, self-changing systems with increasing complexity over time. Actions performed at plants (like watering), will have strongly delayed effects. Thus, monitoring and controlling plants is a difficult perception-action problem requiring advanced predictive cognitive properties, which so far can only be provided by experienced human gardeners. Sensing and control of a plants actual properties, i.e. its phenotype, is relevant to e.g. seed production and plant breeders.

We address plant sensing and control by combining active vision with appropriate perceptual representations, which are essential for cognitive interactions. Core ingredients for these representations are channel representations, dynamic graphs and cause-effect couples (CECs). Channel representations are a wavelet-like, biologically motivated information representation, which can be generalized coherently using group theory. Using these representations, plant models (represented by dynamic graphs) will be acquired, and, by interacting with a human gardener, the system will be taught the different cause-effect relations resulting from possible treatments. Employing decision making and planning processes via CECs, our robot gardener will then be able to choose from its learned repertoire the appropriate actions for optimal plant growth. This way we will arrive at an adaptive, interactive cognitive system, which will be implemented and tested in an industrially-relevant plant-phenotyping application.

http://www.garnics.eu/

Project Publications

Journal Publications

  • A. Agostini, G. Alenyà, A. Fischbach, H. Scharr, F. Wörgötter and C. Torras. A cognitive architecture for automatic gardening. Computers and Electronics in Agriculture, 138: 69-79, 2017.

    Open/Close abstract Abstract Info Info pdf PDF
  • C. Torras. Service robots for citizens of the future. European Review, 24(1): 17-30, 2016.

    Open/Close abstract Abstract Info Info pdf PDF
  • B. Dellen, H. Scharr and C. Torras. Growth signature of rosette plants from time-lapse video. IEEE/ACM Transactions on Computational Biology and Bioinformatics, 12(6): 1470-1478, 2015.

    Open/Close abstract Abstract Info Info pdf PDF
  • A. Agostini, C. Torras and F. Wörgötter. Learning weakly correlated cause-effects for gardening with a cognitive system. Engineering Applications of Artificial Intelligence, 36: 178–194, 2014.

    Open/Close abstract Abstract Info Info pdf PDF
  • W. Kazmi, S. Foix, G. Alenyà and H.J. Andersen. Indoor and outdoor depth imaging of leaves with time-of-flight and stereo vision sensors: Analysis and comparison. ISPRS Journal of Photogrammetry and Remote Sensing, 88: 128-146, 2014.

    Open/Close abstract Abstract Info Info pdf PDF
  • G. Alenyà, S. Foix and C. Torras. ToF cameras for active vision in robotics. Sensors and Actuators A: Physical, 218: 10-22, 2014.

    Open/Close abstract Abstract Info Info pdf PDF
  • F. Moreno-Noguer and P. Fua. Stochastic exploration of ambiguities for nonrigid shape recovery. IEEE Transactions on Pattern Analysis and Machine Intelligence, 35(2): 463-475, 2013.

    Open/Close abstract Abstract Info Info pdf PDF
  • G. Alenyà, B. Dellen, S. Foix and C. Torras. Robotized plant probing: Leaf segmentation utilizing time-of-flight data. IEEE Robotics and Automation Magazine, 20(3): 50-59, 2013.

    Open/Close abstract Abstract Info Info pdf PDF
  • E. Todt and C. Torras. Outdoor view recognition based on landmark grouping and logistic regression. International Journal of Pattern Recognition and Artificial Intelligence, 27(3): 1-21, 2013.

    Open/Close abstract Abstract Info Info pdf PDF
  • S. Ulbrich, V. Ruiz de Angulo, T. Asfour, C. Torras and R. Dillman. General robot kinematics decomposition without intermediate markers. IEEE Transactions on Neural Networks, 23(4): 620-630, 2012.

    Open/Close abstract Abstract Info Info pdf PDF
  • E.E. Aksoy, A. Abramov, J. Dörr, K. Ning, B. Dellen and F. Wörgötter. Learning the semantics of object-action relations by observation. The International Journal of Robotics Research, 30(10): 1229-1249, 2011.

    Open/Close abstract Abstract Info Info pdf PDF

Conference Publications

  • C. Torras. Learning algorithms for robot manipulation of clothing and plant leaves, 2014 ICRA Workshop on Advances in Robot Manipulation of Clothes and Flexible Objects, 2014, Hong-Kong.

    Open/Close abstract Abstract Info Info pdf PDF
  • B. Dellen, F. Husain and C. Torras. Joint segmentation and tracking of object surfaces in depth movies along human/robot manipulations, 8th International Conference on Computer Vision Theory and Applications, 2013, Barcelona, pp. 244-251.

    Open/Close abstract Abstract Info Info pdf PDF
  • C. Torras. From the Turing test to science fiction: The challenges of social robotics, 16th Catalan Conference on Artificial Intelligence, 2013, Vic, Spain, in Artificial Intelligence Research and Development, Vol 256 of Frontiers in Artificial Intelligence and Applications, pp. 5-7, 2013, IOS Press.

    Open/Close abstract Abstract Info Info pdf PDF
  • A. Colomé and C. Torras. Redundant inverse kinematics: Experimental comparative review and two enhancements, 2012 IEEE/RSJ International Conference on Intelligent Robots and Systems, 2012, Vilamoura, Portugal, pp. 5333-5340, IEEE.

    Open/Close abstract Abstract Info Info pdf PDF
  • W. Kazmi, S. Foix and G. Alenyà. Plant leaf imaging using time of flight camera under sunlight, shadow and room conditions, 2012 IEEE International Symposium on Robotic and Sensors Environments, 2012, Magdeburg, Germany, pp. 192-197, IEEE.

    Open/Close abstract Abstract Info Info pdf PDF
  • G. Alenyà, B. Dellen, S. Foix and C. Torras. Robotic leaf probing via segmentation of range data into surface patches, 2012 IROS Workshop on Agricultural Robotics: Enabling Safe, Efficient, Affordable Robots for Food Production, 2012, Vilamoura, Portugal, pp. 1-6.

    Open/Close abstract Abstract Info Info pdf PDF
  • E. Serradell, P. Glowacki, K. Jan, F. Moreno-Noguer and P. Fua. Robust non-rigid registration of 2D and 3D graphs, 2012 IEEE Computer Society Conference on Computer Vision and Pattern Recognition, 2012, Providence, pp. 996-1003, IEEE Press.

    Open/Close abstract Abstract Info Info pdf PDF
  • E. Serradell, K. Jan, F. Moreno-Noguer and P. Fua. Robust elastic 2D/3D geometric graph matching, 2012 SPIE Medical Imaging: Image Processing, 2012, San Diego CA, USA, Vol 8314 of Proceedings of the SPIE, 2012, SPIE.

    Open/Close abstract Abstract Info Info pdf PDF
  • A. Alexey, K. Pauwels, J. Papon, F. Wörgötter and B. Dellen. Depth-supported real-time video segmentation with the Kinect, 2012 IEEE Workshop on Applications of Computer Vision, 2012, Breckenridge, CO, USA, pp. 457-464.

    Open/Close abstract Abstract Info Info pdf PDF
  • S. Foix, G. Alenyà and C. Torras. Towards plant monitoring through next best view, 14th Catalan Conference on Artificial Intelligence, 2011, Lleida, in Artificial Intelligence Research and Development, Vol 232 of Frontiers in Artificial Intelligence and Applications, pp. 101-109, 2011, IOS Press.

    Open/Close abstract Abstract Info Info pdf PDF
  • G. Alenyà, F. Moreno-Noguer, A. Ramisa and C. Torras. Active perception of deformable objects using 3D cameras, 2011 Workshop de Robótica Experimental, 2011, Seville, pp. 434-440.

    Open/Close abstract Abstract Info Info pdf PDF
  • B. Dellen, G. Alenyà, S. Foix and C. Torras. Segmenting color images into surface patches by exploiting sparse depth data, 2011 IEEE Workshop on Applications of Computer Vision, 2011, Kona, Hawaii, pp. 591-598.

    Open/Close abstract Abstract Info Info pdf PDF
  • E. Serradell, A. Romero, R. Leta, C. Gatta and F. Moreno-Noguer. Simultaneous correspondence and non-rigid 3D reconstruction of the coronary tree from single X-ray images, 13th International Conference on Computer Vision, 2011, Barcelona, pp. 850-857, IEEE.

    Open/Close abstract Abstract Info Info pdf PDF
  • G. Alenyà, B. Dellen and C. Torras. 3D modelling of leaves from color and ToF data for robotized plant measuring, 2011 IEEE International Conference on Robotics and Automation, 2011, Shanghai, pp. 3408-3414, IEEE.

    Open/Close abstract Abstract Info Info pdf PDF
  • L. Rozo, P. Jiménez and C. Torras. Robot learning from demonstration in the force domain, 2011 IJCAI Workshop on Agents Learning Interactively from Human Teachers, 2011, Barcelona, Spain, pp. 1-6.

    Open/Close abstract Abstract Info Info pdf PDF
  • L. Rozo, P. Jiménez and C. Torras. Robot learning from demonstration of force-based tasks with multiple solution trajectories, 15th International Conference on Advanced Robotics, 2011, Tallin, Estonia, pp. 124-129, IEEE.

    Open/Close abstract Abstract Info Info pdf PDF
  • M. Wallenberg, M. Felsberg, P. Forssen and B. Dellen. Channel coding for joint colour and depth segmentation, 33rd Annual Symposium of the German Association for Pattern Recognition, 2011, Frankfurt, Germany, in Pattern Recognition, Vol 6835 of Lecture Notes in Computer Science, pp. 306-315, 2011, Springer.

    Open/Close abstract Abstract Info Info pdf PDF
  • A. Agostini, C. Torras and F. Wörgötter. Integrating task planning and interactive learning for robots to work in human environments, 2011 International Joint Conference on Artificial Intelligence, 2011, Barcelona, pp. 2386-2391.

    Open/Close abstract Abstract Info Info pdf PDF
  • F. Moreno-Noguer and J.M. Porta. Probabilistic simultaneous pose and non-rigid shape recovery, 2011 IEEE Computer Society Conference on Computer Vision and Pattern Recognition, 2011, Colorado Springs, pp. 1289-1296, IEEE Press.

    Open/Close abstract Abstract Info Info pdf PDF
  • E. Simo-Serra, F. Moreno-Noguer and A. Perez. Design of non-anthropomorphic robotic hands for anthropomorphic tasks, 2011 ASME International Design Engineering Technical Conference, 2011, Washington, DC, USA, pp. 377-386.

    Open/Close abstract Abstract Info Info pdf PDF
  • F. Moreno-Noguer. Deformation and illumination invariant feature point descriptor, 2011 IEEE Computer Society Conference on Computer Vision and Pattern Recognition, 2011, Colorado Springs, pp. 1593-1600, IEEE Press.

    Open/Close abstract Abstract Info Info pdf PDF
  • L. Rozo, P. Jiménez and C. Torras. Learning force-based robot skills from haptic demonstration, 13th Catalan Conference on Artificial Intelligence, 2010, Espluga de Francolí, Spain, in Artificial Intelligence Research and Development, Vol 220 of Frontiers in Artificial Intelligence and Applications, pp. 331-340, 2010, IOS Press.

    Open/Close abstract Abstract Info Info pdf PDF
  • L. Rozo, P. Jiménez and C. Torras. Sharpening haptic inputs for teaching a manipulation skill to a robot, 1st IEEE International Conference on Applied Bionics and Biomechanics, 2010, Venice, pp. 331-340.

    Open/Close abstract Abstract Info Info pdf PDF
  • F. Moreno-Noguer, J.M. Porta and P. Fua. Exploring ambiguities for monocular non-rigid shape estimation, 11th European Conference on Computer Vision, 2010, Crete, in Computer Vision - ECCV 2010, Vol 6313 of Lecture Notes in Computer Science, pp. 370-383, 2010, Springer, Berlin.

    Open/Close abstract Abstract Info Info pdf PDF
  • E. Serradell, M. Özuysal, V. Lepetit, P. Fua and F. Moreno-Noguer. Combining geometric and appearance priors for robust homography estimation, 11th European Conference on Computer Vision, 2010, Crete, in Computer Vision - ECCV 2010, Vol 6313 of Lecture Notes in Computer Science, pp. 58-72, 2010, Springer, Berlin.

    Open/Close abstract Abstract Info Info pdf PDF
  • J. Sanchez, J. Ostlund, P. Fua and F. Moreno-Noguer. Simultaneous pose, correspondence and non-rigid shape, 2010 IEEE Computer Society Conference on Computer Vision and Pattern Recognition, 2010, San Francisco, pp. 1189-1196.

    Open/Close abstract Abstract Info Info pdf PDF
  • M. Villamizar, F. Moreno-Noguer, J. Andrade-Cetto and A. Sanfeliu. Efficient rotation invariant object detection using boosted random Ferns, 2010 IEEE Computer Society Conference on Computer Vision and Pattern Recognition, 2010, San Francisco, pp. 1038-1045.

    Open/Close abstract Abstract Info Info pdf PDF
  • M. Villamizar, F. Moreno-Noguer, J. Andrade-Cetto and A. Sanfeliu. Shared random Ferns for efficient detection of multiple categories, 20th International Conference on Pattern Recognition, 2010, Istanbul, Turkey, pp. 388-391.

    Open/Close abstract Abstract Info Info pdf PDF

Other Publications

  • A. Agostini, C. Torras and F. Wörgötter. A general strategy for interactive decision-making in robotic platforms. Technical Report IRI-TR-11-01, Institut de Robòtica i Informàtica Industrial, CSIC-UPC, 2011.

    Open/Close abstract Abstract Info Info pdf PDF