 |
 |
Introduction
This work is focused on the development of a system that can recognize automatically the human action and activity in sports using image sequences from monocular uncalibrated camera. The system has been developed by the collaboration of LIS-INPG and UOC. This collaboration has been started from the SIMILAR meeting in Lausanne (May 2005). Costas Panagiotakis, a PhD student from UOC and Emmanuel Ramasso, a PhD student from LIS-INPG, have worked on the development of this multimodal system and they had involved in the SIMILAR PhD exchanges on January 2006 and May 2006, respectively. Prof. Michele Rombaut and Prof. Denis Pellerin are the supervisors Emmanuel, and Prof. Georgios Tziritas is the supervisor of Costas.
Methodology
In this work, we present an automatic human shape-motion analysis method based on a fusion architecture is proposed for human action and activity recognition in videos. Robust shape-motion features are extracted from human points detection and tracking. The features are combined within the Transferable Belief Model (TBM) framework for action recognition. The TBM based modelling and fusion process allows to take into account imprecision, uncertainty and conflict inherent to the features. Action recognition is performed by a multilevel analysis. The sequencing is exploited for feedback information extraction in order to improve tracking results. The system is tested on real videos of athletics meetings to recognize four types of jumps: high jump, pole vault, triple jump and long jump.
Moreover, we present a shape based method for automatic people detection and counting without any assumption or knowledge of camera motion. The proposed method is applied to athletic videos in order to classify them to videos of individual and group sports and provides running/hurdling activity recognition for the case of group sports. The proposed features are combined within the Transferable Belief Model (TBM) framework yielding a two level (instant and global) video categorization.
System Architecture
|
|
Experimental Results
Athlete tracking using three major human points
 Original high jump video (in DivX format)
Video of three major human points tracking (in DivX format) |
 Original high jump video (in DivX format)  Video of three major human points tracking (in DivX format)
|
Athlete tracking using four major human points
Original high jump video (in DivX format)
Video of four major human points tracking (in DivX format) |
 Original high jump video (in DivX format)
Video of four major human points tracking (in DivX format) |
Original high jump video (in
DivX
format)  Video of four major human points tracking (in DivX format) |
Correction of tracking based on activity recognition in a high jump sequence
 Original high jump video (in DivX format) |
 Video of four major human points tracking before correction (in DivX format) |
 Video of four Major Human Points Tracking after correction (in DivX format) |
An example of human tracking correction based on feedback information from activity recognition. The detection of points inversion is done automatically, since during the stand action the human head was found down. The human tracking is executed on frames that corresponds to stand action under the assumption that the head should be up. |
Individual/Team Sport Recognition - People Detection and Counting
 Original high jump video (in DivX format)
Video of people detection and counting method (in
DivX
format). The small black boxes corresponds |
 Original high jump video (in DivX format)
Video of people detection and counting method
(in DivX
format). The small black boxes corresponds |
Related Publications
[1] E. Ramasso, D. Pellerin, C. Panagiotakis, M. Rombaut, G. Tziritas and W. Lim, Spatiotemporal information fusion for human action recognition in videos, European Signal Processing Conference, 2005.
[2] C. Panagiotakis, E. Ramasso, G. Tziritas, D. Pellerin and M. Rombaut, Shape-Motion Based Athlete Tracking for Multilevel Action Recognition, Intern. Conf. on Articulated Motion and Deformable Objects, 2006. (Sponsor award, certification in pdf)
[3] C. Panagiotakis, E. Ramasso, G. Tziritas, M. Rombaut and
D. Pellerin, Shape-Motion Based Athlete Tracking for Multilevel Action
Recognition, submitted to Computer Vision and Image Understanding, 2006.
Acknowledgments: This work is partially supported by SIMILAR European Network of Excellence.
