Method and apparatus for detecting and separating objects of interest in soccer video by color segmentation and shape analysis

• US 8,805,004 B2
• Filed: 01/05/2010
• Issued: 08/12/2014
• Est. Priority Date: 01/09/2009
• Status: Expired due to Fees

• Current Assignee: Magnolia Licensing LLC (IP Edge LLC)
• Original Assignee: Thomson Licensing (Vantiva SA)
• Inventors: Huang, Yu, Llach, Joan
• Primary Examiner(s): Desire, Gregory M
• Application Number:
  US13/138,091
• Publication Number:
  US 20110268320A1
• Time in Patent Office: 1,680 Days
• Field of Search: 382/103, 382/190, 382/266, 348/169
• US Class Current: 382/103
• CPC Class Codes: G06V 20/41 Higher-level, semantic clustering, classification or understanding of video scenes, e.g. detection, labelling or Markovian modelling of sport events or news items segmenting video sequences G06V20/49
• CPC Class Codes: G06V 20/42 of sport video content
• CPC Class Codes: G06V 20/46 Extracting features or characteristics from the video content, e.g. video fingerprints, representative shots or key frames

Abstract

Substantial elimination of errors in the detection and location of overlapping human objects in an image of a playfield is achieved, in accordance with at least one aspect of the invention, by performing a predominately shape-based analysis of one or more characteristics obtained from a specified portion of the candidate non-playfield object, by positioning a human object model substantially over the specified portion of the candidate non-playfield object in accordance with information based at least in part on information from the shape-based analysis, and removing an overlapping human object from the portion of the candidate non-playfield object identified by the human object model. In one exemplary embodiment, the human object model is an ellipse whose major and minor axes are variable in relation to one or more characteristics identified from the specified portion of the candidate non-playfield object.

Claims

1. A method for detecting and isolating overlapping human objects in an image of a playfield, the image comprising a plurality of playfield and non-playfield pixels, the method comprising:

extracting, by a processor, a plurality of substantially non-playfield pixels from the image as a candidate object;

generating, by the processor, shape information from the candidate object;

detecting, by the processor, in at least a portion of the candidate object presence of at least one human characteristic at a position by analyzing one or more features of the shape information for the candidate object;

identifying, by the processor, a subset of the plurality of image pixels included in said candidate object as representative of the human object, wherein the subset is positioned based at least in part on the position of the at least one characteristic detected in at least the portion of the candidate object; and

removing, by the processor, from the candidate object the subset of pixels associated with the human object.

2. The method as defined in claim 1, wherein the step of generating shape information comprises:

generating, by the processor, a convex hull from the candidate object to include substantially all the pixels associated with the candidate object.

3. The method as defined in claim 2, wherein the at least one characteristic is a human head, wherein the one or more features of the shape information includes at least one concave vertex on the convex hull, and wherein the detecting step further comprises detecting, by the processor, the at least one concave vertex on the convex hull in order to determine the position for said human head.

4. The method as defined in claim 1, wherein the step of generating shape formation comprises:

generating, by the processor, a convex hull from the candidate object to include substantially all the pixels associated with the candidate object;

generating, by the processor, a vertical projection profile of the candidate object, wherein the vertical projection profile represents an amount of non-playfield pixels present in each vertical slice through said candidate object.

5. The method as defined in claim 4, wherein the at least one characteristic is a human head, wherein the one or more features of the shape information includes at least one concave vertex on the convex hull and at least one peak along the vertical projection profile, and wherein the detecting step further comprises:

detecting, by the processor, the at least one concave vertex on the convex hull in order to identify a first position for said human head in said candidate object;

detecting, by the processor, at least one peak along the vertical projection profile in order to identify at least a second location for said human head in said candidate object; and

selecting, by the processor, the position for the human head based at least in part on the first and second positions.

6. The method as defined in claim 5, wherein the selecting step comprises:

applying, by the processor, one or more constraints to the selecting so that the position of the human head appears in a substantially upper region of the candidate object corresponding to a head position for an upright human object.

7. The method as defined in claim 5, wherein the step of identifying the subset of pixels further comprises:

positioning, by the processor, one end of a major axis of an elliptical model substantially at the position selected for the human head;

positioning, by the processor, another end of the major axis of the elliptical model at a first pixel that represents a non-playfield pixel substantially along the major axis, wherein the major axis is aligned in a substantially vertical direction; and

adjusting, by the processor, a minor axis of the elliptical model to have a length, which when combined in a ratio with the major axis length corresponds substantially to a desired width and height ratio for the human object.

8. The method as defined in claim 7, wherein the desired width and height ratio is selected from a group consisting of an average width/height ratio, a maximum width/height ratio, and a minimum width height ratio.

9. The method as defined in claim 7, wherein the step of identifying the subset of pixels further comprises:

determining, by the processor, a number of pixels in the subset of pixels included within a perimeter of the elliptical model; and

when the number of pixels is less than a prescribed threshold, ceasing, by the processor, all positioning and formation of the elliptical model.

10. The method as defined in claim 1, the method further comprising:

determining, by the processor, a number of pixels remaining in the candidate object upon removal of the subset of pixels associated with the human object; and

when the number of pixels is less than a prescribed threshold, removing, by the processor, the candidate object from further processing.

11. The method as defined in claim 10, the method further comprising:

when the number of pixels is at least greater than the prescribed threshold, repeating, by the processor, the method steps beginning at the step of generating shape information.

12. The method as defined in claim 10, the method further comprising:

when the number of pixels is at least greater than the prescribed threshold, determining, by the processor, whether the overlapping human objects overlap in a substantially vertical manner by generating the shape information of the candidate object with respect to at least vertical elongation of the candidate object.

13. The method as defined in claim 12, wherein the at least one characteristic is a human torso, wherein the one or more features of the shape information includes at least one flat peak region along a horizontal projection profile, the method further comprises:

when the candidate object is determined to include vertically overlapping human objects, generating, by the processor, a horizontal projection profile of the candidate object, wherein the horizontal projection profile represents an amount of non-playfield pixels present in each horizontal slice through said candidate object; and

detecting, by the processor, the at least one flat peak region along the horizontal projection profile in order to identify a location for said human torso in said candidate object.

14. The method of claim 13, wherein the method further comprises:

second identifying, by the processor, a second subset of the plurality of image pixels included in said candidate object as representative of the human object, wherein the second subset is positioned based at least in part on the position of the human torso detected in the candidate object, said second identifying comprising;

positioning a major axis of an elliptical model to be aligned substantially vertically through substantially the position for the human torso;

positioning one end of the major axis of the elliptical model at a first pixel that represents a non-playfield pixel substantially along the major axis; and

adjusting a length of the major axis so that with respect to said torso location, another end of the major axis is positioned substantially at a position for a human head; and

adjusting a minor axis of the elliptical model to have a length, which when combined in a ratio with the major axis length corresponds substantially to a desired width and height ratio for the human object.

15. The method as defined claim 14, wherein the method further comprises:

removing, by the processor, from the candidate object the second subset of pixels associated with the human object.

16. The method as defined in claim 12, wherein the method further comprises:

when the candidate object is determined to include vertically overlapping human objects, performing, by the processor, a color-based analysis of said candidate object to determine a first location for a human object; and

second identifying, by the processor, a second subset of the plurality of image pixels included in said candidate object as representative of the human object, wherein the second subset is positioned based at least in part on the position of the human object detected in the candidate object, said second identifying comprising

positioning an elliptical model to be aligned with its major axis substantially vertical through substantially the location determined for the human object;

adjusting a length of the major axis and a length of the minor axis to correspond substantially to a desired height and width ratio for the human object.

17. The method as defined claim 16, wherein the method further comprises:

removing, by the processor, from the candidate object the second subset of pixels associated with the human object.

18. The method as defined in claim 1, further comprising:

rotating, by the processor, the candidate object by a desired number of degrees before generating the shape information.