METHOD AND APPARATUS FOR DETECTING OBJECTS OF INTEREST IN SOCCER VIDEO BY COLOR

• US 20100098307A1
• Filed: 11/06/2007
• Published: 04/22/2010
• Est. Priority Date: 03/26/2007
• Status: Active Grant

• Current Assignee: InterDigital Madison Patent Holdings (InterDigital, Inc.)
• Original Assignee: Thomson Licensing (Vantiva SA)
• Inventors: Huang, Yu, Bhagavathy, Sitaram, Llach, Joan
• Granted Patent: US 8,391,547 B2
• Time in Patent Office: Days
• US Class Current: 382/128
• CPC Class Codes: G06V 10/34 Smoothing or thinning of the pattern; Morphological operations; Skeletonisation
• CPC Class Codes: G06V 10/56 relating to colour
• CPC Class Codes: G06V 20/40 in video content extracting overlay text G06V20/62; video retrieval G06F16/70; processing of video elementary streams in video servers H04N21/234; processing of video elementary streams in video clients H04N21/44

Abstract

A method is disclosed for detecting and locating players in soccer video frames without errors caused by artifacts by a shape analysis-based approach to identify the players and the ball from roughly extracted foregrounds obtained by color segmentation and connected component analysis, by performing a Euclidean distance transform to extract skeletons for every foreground blob, by performing a shape analysis to remove false alarms (non-players and non-ball), and then by performing skeleton pruning and a reverse Euclidean distance transform to cut-off the artifacts primarily caused by playing field lines.

Claims

1. A method for identifying an object of interest in an image comprising background pixels and foreground pixels, said background pixels including pixels of a first color, the method comprising:

extracting the foreground pixels from the image;

identifying a region among the foreground pixels;

if an area of the region is greater than a first area threshold and if a ratio of pixels of the first color in the region is less than or equal to a first color pixel ratio threshold, processing the region to produce a skeleton; and

if elongation of the skeleton is smaller than an elongation threshold, determining the region as the object of interest.

2. The method of claim 1, wherein the object of interest includes a human object.

3. The method of claim 2, wherein the determining step includes determining that the region is the human object when the elongation of the skeleton is smaller than the elongation threshold and an aspect ratio of the skeleton is smaller than a aspect ratio threshold.

4. The method of claim 2, further comprising the step of pruning the skeleton to cut off artifacts.

5. The method defined in claim 4, wherein the step of pruning includes:

if a thickness of the region at a point on the skeleton is less than a threshold based, at least in part, on a maximum thickness of the skeleton, pruning said point and removing associated pixels for said point.

6. The method defined in claim 4, wherein the step of pruning includes:

if a distance from a point on the skeleton to another point in the skeleton associating with a maximal number of pixels is less than each of a vertical and horizontal threshold based, at least in part, on a maximum thickness of the skeleton, pruning said point and removing associated pixels for said point.

7. The method defined in claim 5, wherein the step of pruning further includes:

if a distance from said point on the skeleton to another point in the skeleton associating with a maximal number of pixels is less than each of a vertical and horizontal threshold based, at least in part, on said maximum thickness of the skeleton, pruning said point and removing associated pixels for said point.

8. The method defined in claim 7, wherein the step of pruning further includes:

if a pixel proportion of pixels of the first color in the associated pixels of said point exhibits sufficient intensity and normalized intensity of other color components exhibits insufficient intensity, pruning said point and removing said associated pixels for said point.

9. The method defined in claim 2, wherein the method further includes:

if an area of the region is greater than a first area threshold and less than a second area threshold and if a ratio of pixels of the first color in the region is greater than a second first pixel ratio threshold and if a roundness characteristic of said region is less than a roundness threshold and if an eccentricity characteristic of said region is less than an eccentricity threshold, determining the region as the object of interest including at least a ball.

10. The method defined in claim 2, wherein the extracting step further includes:

detecting a pixel as a background pixel classifier by using a likelihood ratio approach that is satisfied when;

$\frac{P<br><br>\left(\mathrm{rgb}/\mathrm{playfield}\right)}{P<br><br>\left(\mathrm{rgb}/\mathrm{non\_playfield}\right)}\ge <br><br>\mathrm{\phi <br><br>},$

where 0≦

φ

≦

1 and where

$P<br><br>\left(\mathrm{rgb}/\mathrm{playfield}\right)=\frac{f<br><br>\left(\mathrm{rgb}\right)}{T_f},<br><br>\mathrm{and}$ $P<br><br>\left(\mathrm{rgb}/\mathrm{non\_playfield}\right)=\frac{n<br><br>\left(\mathrm{rgb}\right)}{T_n},$

where f(rgb) gives the count in bin rgb of a background histogram, n(rgb) is the count in bin rgb from a foreground histogram, each histogram including N bins, and T_f, and T_n are total counts in the background and foreground histograms, respectively.

11. A method for identifying a ball object in an image comprising background pixels and foreground pixels, said background pixels including pixels of a first color, the method comprising:

converting a pixel in the image into one of background and foreground pixel to form a mask image;

identifying a region enclosed by connected foreground pixels in the image;

filling the region with foreground pixels;

producing a completed region by comparing corresponding pixels in the identified region and the filled region;

if an area of the completed region is greater than a first area threshold and less than a second area threshold, and if a ratio of pixels of the first color in the completed region is greater than a first color pixel ratio threshold, and if roundness of the completed region is less than a roundness threshold, and if eccentricity of the competed region is less than an eccentricity threshold, determining the competed region as said ball object.

12. A machine readable medium having stored thereon machine executable instructions that, when executed, implement a method for identifying an object of interest in an image comprising background pixels and foreground pixels, said background pixels including pixels of a first color, the method comprising:

extracting the foreground pixels from the image;

identifying a region among the foreground pixels;

if an area of the region is greater than a first area threshold and if a ratio of pixels of the first color in the region is less than or equal to a first color pixel ratio threshold, processing the region to produce a skeleton; and

if elongation of the skeleton is smaller than an elongation threshold, determining the region as the object of interest including at least a human object.

13. The machine readable medium of claim 12, wherein the object of interest includes a human object.

14. The machine readable medium of claim 13, wherein the determining step performed via execution of the instructions includes determining that the region is the human object when the elongation of the skeleton is smaller than the elongation threshold and an aspect ratio of the skeleton is smaller than an aspect ratio threshold.

15. The machine readable medium of claim 13, wherein the method performed via execution of the instructions further comprises the step of pruning the skeleton to cut off artifacts.

16. The machine readable medium defined in claim 15, wherein the step of pruning performed via execution of the instructions includes:

if a thickness at a point on the skeleton is less than a threshold based, at least in part, on a maximum thickness of the skeleton, pruning said point and removing associated pixels for said point.

17. The machine readable medium defined in claim 15, wherein the step of pruning performed via execution of the instructions includes:

if a distance from a point on the skeleton is less than each of a vertical and horizontal threshold based, at least in part, on a maximum thickness of the skeleton, pruning said point and removing associated pixels for said point.

18. The machine readable medium defined in claim 16, wherein the step of pruning performed via execution of the instructions further includes:

if a distance from said point on the skeleton is less than each of a vertical and horizontal threshold based, at least in part, on said maximum thickness of the skeleton, pruning at said point and removing said associated pixels for said point.

19. The machine readable medium defined in claim 18, wherein the step of pruning performed via execution of the instructions further includes:

if a pixel proportion of pixels of the first color in the associated pixels of said point exhibits sufficient intensity and normalized intensity of other color components exhibits insufficient intensity, pruning said point and removing said associated pixels for said point.

20. The machine readable medium defined in claim 13, wherein the method performed via execution of the instructions further includes:

if an area of the region is greater than a first area threshold and less than a second area threshold and if a ratio of pixels of the first color in the region is greater than a second first color pixel ratio threshold and if a roundness characteristic of said region is less than a roundness threshold and if an eccentricity characteristic of said region is less than an eccentricity threshold, determining the region as the object of interest including at least a ball.

21. The machine readable medium defined in claim 13, wherein the extracting step performed via execution of the instructions further includes:

detecting a pixel as a background pixel classifier by using a likelihood ratio approach that is satisfied when;

$\frac{P<br><br>\left(\mathrm{rgb}/\mathrm{playfield}\right)}{P<br><br>\left(\mathrm{rgb}/\mathrm{non\_playfield}\right)}\ge <br><br>\mathrm{\phi <br><br>},$

where 0≦

φ

≦

1 and where

$P<br><br>\left(\mathrm{rgb}/\mathrm{playfield}\right)=\frac{f<br><br>\left(\mathrm{rgb}\right)}{T_f},<br><br>\mathrm{and}$ $P<br><br>\left(\mathrm{rgb}/\mathrm{non\_playfield}\right)=\frac{n<br><br>\left(\mathrm{rgb}\right)}{T_n},$

where f(rgb) gives the count in bin rgb of a background histogram, n(rgb) is the count in bin rgb from a foreground histogram, each histogram including N bins, and T_f, and T_n are total counts in the background and foreground histograms, respectively.

22. A machine readable medium having stored thereon machine executable instructions that, when executed, implement a method for identifying a ball object in an image comprising background pixels and foreground pixels, said background pixels including pixels of a first color, the method comprising:

converting a pixel in the image into one of background and foreground pixel to form a mask image;

identifying a region enclosed by connected foreground pixels in the image;

filling the region with foreground pixels;

producing a completed region by comparing corresponding pixels in the identified region and the filled region;

if an area of the completed region is greater than a first area threshold and less than a second area threshold, and if a ratio of pixels of the first color in the completed region is greater than a first color pixel ratio threshold, and if roundness of the completed region is less than a roundness threshold, and if eccentricity of the completed region is less than an eccentricity threshold, determining the completed region as said ball object.