Method and system for the detection and the classification of events during motion actions

• US 20090060352A1
• Filed: 04/19/2006
• Published: 03/05/2009
• Est. Priority Date: 04/20/2005
• Status: Active Grant

• Current Assignee: Federazione Italiana Giuoco Calcio, Consiglio Nazionale Delle Ricerche (Repubblica Italiana)
• Original Assignee: Federazione Italiana Giuoco Calcio, Consiglio Nazionale Delle Ricerche (Repubblica Italiana)
• Inventors: Stella, Ettore, Mosca, Nicola, D'Orazio, Tiziana Rita, Distante, Arcangelo, Leo, Marco, Spagnolo, Paolo, Nitti, Massimiliano
• Granted Patent: US 8,300,935 B2
• Time in Patent Office: Days
• US Class Current: 382/224
• CPC Class Codes: A63B 2024/0025 Tracking the path or location of one or more users, e.g. players of a game
• CPC Class Codes: A63B 2220/13 Relative positions
• CPC Class Codes: A63B 2220/30 Speed
• CPC Class Codes: A63B 2220/40 Acceleration
• CPC Class Codes: A63B 2220/806 Video cameras
• CPC Class Codes: A63B 2243/0025 Football
• CPC Class Codes: A63B 24/0021 Tracking a path or terminating locations
• CPC Class Codes: A63B 71/0605 Decision makers and devices using detection means facilitating arbitration
• CPC Class Codes: G06T 2207/30221 Sports video; Sports image
• CPC Class Codes: G06T 7/20 Analysis of motion motion estimation for coding, decoding, compressing or decompressing digital video signals H04N19/43, H04N19/51
• CPC Class Codes: G06T 7/73 using feature-based methods
• CPC Class Codes: G06V 10/48 by mapping characteristic values of the pattern into a parameter space, e.g. Hough transformation
• CPC Class Codes: G06V 10/52 Scale-space analysis, e.g. wavelet analysis multi-scale boundary representations G06V10/42
• CPC Class Codes: G06V 20/40 in video content extracting overlay text G06V20/62; video retrieval G06F16/70; processing of video elementary streams in video servers H04N21/234; processing of video elementary streams in video clients H04N21/44

Abstract

The present invention relates to a system for detecting and classifying events during motion actions, in particular “offside” event in the football game. The system allows determining such event in a real-time and semi-automatic context, by taking into account the variability of the environmental conditions and of the dynamics of the events which can be traced back to the offside. The present invention proposes itself with a not-invasive technique, compatible with the usual course of the match.

Claims

1. Method for detecting and classifying an event occurred during motion actions of one or more subjects in an area, the occurrence or not of said event depending upon predetermined rules related to said motion and to mutual positions of said subjects in a predetermined time instant, the method comprising the following steps:

acquiring a plurality of digital images in sequence of said area during the progress of said actions;

processing each image of said plurality of images for extracting first features apt to identify each one of said subjects and second features apt to determine the instantaneous position of each one of said subjects within said area;

classifying said subjects based upon said first and/or second features; and

evaluating the occurrence of said event, by verifying said predetermined rules base upon said first and second features and upon the result of said classification.

2. Method according to claim 1, wherein said step of processing each image comprises a step of recognizing each one of said subjects inside said image.

3. Method according to claim 2, wherein said phase of recognizing said subjects comprises a sub-phase of removing the noise, based upon an analysis procedure of the point connectivity, in order to extract said first features to detect a first category of subjects and a second category of subjects, by discriminating them based upon the respective sizes.

4. Method according to claim 3, wherein said phase of recognizing said subjects comprises the sub-phases of:

extracting a region of said image candidate to contain said subjects;

verifying the presence of said subjects in said candidate region;

calculating one or more parameters identifying the dynamics of said subjects, parameters to be utilized for recognizing the same subjects during the processing of a subsequent image.

5. Method according to claim 4, wherein said phase of recognizing said subjects comprises an additional sub-phase of constructing a reference image.

6. Method according to claim 5, wherein said phase of recognizing said subjects comprises an additional sub-phase of comparing the current image to the reference image, in order to detect said subjects.

7. Method according to claim 5, wherein said sub-phase of constructing a reference image is repeated in time, so as to use always a current reference image.

8. Method according to claim 3, wherein said sub-phase of calculating one of more parameters identifying the dynamics of said subject is based upon a procedure or predictive type.

9. Method according to claim 4, wherein said phase of recognizing said subjects comprises an additional shadow removal sub-phase.

10. Method according to claim 9, wherein said shadow removal sub-phase is performed by means of the correlation between the image under examination and the reference one.

11. Method according to claim 4, wherein said sub-phase of extracting a region candidate to contain a first subject belonging to the first category of subjects, comprises a step of extraction of the contours of the subjects existing in the image, by obtaining a bordered image.

12. Method according to claim 11, wherein said sub-phase of extracting a candidate region comprises a step of constructing a correlation map by means of a bordered image convolution operation with a filter apt to characterize spatially said first subject.

13. Method according to claim 12, wherein said filter is constituted by a bidimensional convolution mask bearing dimensional and spatial information of said first subject.

14. Method according to claim 13, wherein said sub-phase of verifying the presence of said first subject in said candidate region comprises a step of calculating one or more characteristic parameters of a texture of said object.

15. Method according to claim 14, wherein said calculation is performed by means of a transform operation of Wavelet type.

16. Method according to claim 15, wherein said transformation operation is based upon a Haar transform.

17. Method according to claim 14, wherein said sub-phase of verifying said first subject in said candidate region comprises a classification step based upon said characteristic parameters.

18. Method according to claim 17, wherein said classification is performed by means of one or more artificial neural networks.

19. Method according to claim 18, wherein said neural networks are pre-trained by means of algorithms of “

back propagation”

type.

20. Method according to claim 4, wherein said sub-phase of extracting a region candidate to contain a second subject belonging to the second category of subjects, comprises a step of analyzing the intensity levels of each image dot.

21. Method according to claim 20, wherein said step of analyzing the intensity level of each image dot is performed on grey-level images, acquired by means of sensors under the form of Bayer grids.

22. Method according to claim 20, wherein said step of analyzing the intensity levels of each image dot is performed on color images, operating on RGB bands.

23. Method according to claim 20, wherein said step of analyzing the intensity levels of each image dot is performed on color images, operating on HIS bands.

24. Method according to claim 20, wherein said step of verifying the presence of said second subject in said candidate region is based upon a procedure for extracting statistical information related to said second object.

25. Method according to claim 24, wherein said statistical information comprises an average and a standard deviation calculated for each line of the image related to said candidate region.

26. Method according to claim 25, wherein said step of calculating the average and the standard deviation is performed for each band of RGB color.

27. Method according to claim 25, wherein said step of calculating the average and standard deviation is performed for each HIS band.

28. Method according to claim 24, wherein said sub-phase of verifying the presence of said second subject in said candidate region comprises a classification step based upon said statistical parameters.

29. Method according to claim 28, wherein said step of classifying the objects utilizes a not supervised classifier (SOM).

30. Method according to claim 28, wherein said step of classifying the objects utilizes a not supervised clustering algorithm (k-means).

31. Method according to claim 29, wherein said classifier is trained by means of normalized examples with respect to said statistical parameters.

39. Program for computers, apt to implement a method according to claim 1 when operating on a processor.

32. System for the detection and the classification of an event occurred during motion actions of one of more subjects in an area, the occurrence or not of said event depending upon predetermined rules related to said motion and to mutual positions of said subjects in a predetermined time moment, the system comprising one or more units for acquiring digital images of said area during the course of said actions, wherein each one of said units comprises:

means for processing each image of said plurality of images for extracting first features apt to identify each one of said subjects and second features apt to determine the instantaneous position of each one of said subject within said area; and

means for classifying said subjects based upon said first and/or second features;

the system further comprising a supervision unit to evaluate the occurrence of said event, by verifying said predetermined rules based upon said first and second features and upon the result of said classification.

33. System according to claim 32, wherein each one of said acquisition units comprises a high resolution camera.

34. System according to claim 33, wherein said camera operates with a resolution of 1920×

1080, according to a CAMERALINK®

reading standard.

35. System according to claim 32, further comprising one or more units for storing in real time the acquired images.

36. System according to claim 32, further comprising means for connecting said acquiring units to said supervision unit.

37. System according to claim 32, wherein said supervision unit further comprises interface means for the display and/or analysis of the results.

38. System according to claim 32, apt to implement a method.