Detecting an object in an image using edge detection and morphological processing

• US 8,401,304 B2
• Filed: 12/19/2007
• Issued: 03/19/2013
• Est. Priority Date: 07/27/2007
• Status: Active Grant

• Current Assignee: Sportsmedia Technology Corporation
• Original Assignee: Sportvision, Inc. (Sportsmedia Technology Corporation)
• Inventors: Cavallaro, Richard H., Elangovan, Vidya
• Primary Examiner(s): Bhatnagar, Anand
• Application Number:
  US11/960,565
• Publication Number:
  US 20090028385A1
• Time in Patent Office: 1,917 Days
• Field of Search: 382/100, 382/181, 382/190, 382/195, 382/199, 382/203, 348/92, 348/125, 348/126, 348/135, 715/247
• US Class Current: 382/199
• CPC Class Codes: A63F 2300/663 for simulating liquid objects, e.g. water, gas, fog, snow, clouds
• CPC Class Codes: A63F 2300/6692 using special effects, generally involving post-processing, e.g. blooming
• CPC Class Codes: A63F 2300/69 Involving elements of the real world in the game world, e.g. measurement in live races, real video
• CPC Class Codes: A63F 2300/8017 Driving on land or water; Flying
• CPC Class Codes: G06T 2207/30221 Sports video; Sports image
• CPC Class Codes: G06T 2207/30236 Traffic on road, railway or crossing
• CPC Class Codes: G06T 7/248 involving reference images or patches
• CPC Class Codes: G06T 7/74 involving reference images or patches
• CPC Class Codes: G06V 10/751 Comparing pixel values or logical combinations thereof, or feature values having positional relevance, e.g. template matching
• CPC Class Codes: G06V 2201/08 Detecting or categorising vehicles
• CPC Class Codes: H04N 5/265 Mixing
• CPC Class Codes: H04N 5/272 Means for inserting a foreground image in a background image, i.e. inlay, outlay
• CPC Class Codes: H04N 5/2723 Insertion of virtual advertisement; Replacing advertisements physical present in the scene by virtual advertisement data processing systems or methods specially adapted for marketing G06Q30/02

Abstract

A representation of an object in a live event is detected in an image of the event. A location of the object in the live event is translated to an estimated location in the image based on camera sensor and/or registration data. A search area is determined around the estimated location in the image. A direction of motion of the object in the image is also determined. A representation of the object is identified in the search area by detecting edges of the object, e.g., perpendicular to the direction of motion and parallel to the direction of motion, performing morphological processing, and matching against a model or other template of the object. Based on the position of the representation of the object, the camera sensor and/or registration data can be updated, and a graphic can be located in the image substantially in real time.

Claims

1. A method for processing image data, comprising:

obtaining an image of a live event which is captured by at least one camera, the live event comprising an object;

performing first edge detection to detect first edges which extend in a first direction in the image;

performing second edge detection to detect second edges which extend in a second direction in the image;

detecting which of the second edges are near the first edges and which of the second edges are not near the first edges; and

performing morphological processing on the first edges and the second edges which are detected to be near the first edges to obtain a representation of the object, the second edges which are detected to be not near the first edges are not used to obtain the representation of the object.

2. The method of claim 1, further comprising:

providing an estimate of a position of the object in the live event based on a position of the representation of the object in the image.

3. The method of claim 1, further comprising:

updating registration data of the at least one camera based on a position of the representation of the object in the image.

4. The method of claim 1, further comprising:

updating sensor data of the at least one camera based on a position of the representation of the object in the image.

5. The method of claim 1, further comprising:

determining whether a position of the representation of the object in the image is offset from an estimated position due to at least one of registration data or sensor data of the at least one camera, and updating the at least one of registration data or sensor data of the at least one camera if the offset is caused thereby.

6. The method of claim 1, wherein:

the detecting which of the second edges are near the first edges comprises dilating the first edges to provide dilated first edges, and identifying which of the second edges are a specified number of pixels away from the dilated first edges, the second edges which are not the specified number of pixels away from the dilated first edges are not used to obtain the representation of the object.

7. The method of claim 1, wherein:

the first direction is perpendicular to a direction of motion of the object in the image, and the second direction is parallel to the direction of motion.

8. The method of claim 7, further comprising:

identifying the direction of motion of the object based on sensor data of the at least one camera.

9. The method of claim 7, further comprising:

identifying the direction of motion of the object based on movement of the object in multiple images of the live event.

10. The method of claim 1, further comprising:

determining a search area in the image based on a direction of motion of the object in the image.

11. The method of claim 1, wherein:

the detecting which of the second edges are near the first edges comprises dilating the first edges to provide dilated first edges, and identifying which of the second edges intersect with the dilated first edges, the second edges which do not intersect the dilated first edges are not used to obtain the representation of the object.

12. The method of claim 1, further comprising:

correlating a template of the object with different portions of the image to determine a position of the representation of the object in the image.

13. The method of claim 12, further comprising:

determining an estimated size of the representation of the object in the image, a size of the template corresponds to the estimated size of the representation of the object in the image.

14. The method of claim 12, wherein:

the template comprises a model of the object; and

the method further comprises determining an orientation of the object, an orientation of the model corresponds to the orientation of the object.

15. The method of claim 1, wherein:

the object moves relative to a background object in the live event; and

the second edges which are not near the first edges comprise an edge of the background object.

16. The method of claim 1, further comprising:

obtaining data identifying a location of the object in the live event;

transforming the data identifying the location to an estimated position of the representation of the object in the image; and

determining a search area in the image for performing the first and second edge detection based on the estimated position of the representation of the object in the image.

17. The method of claim 16, wherein:

the data identifying the location comprises data which indicates at least one of a time or a distance since the object passed a known location in the live event.

18. The method of claim 1, further comprising:

combining a graphic with the image, the graphic is located in the image based on at least one of an orientation of the representation of the object in the image or a perspective of the representation of the object in the image.

19. The method of claim 1, further comprising:

combining a graphic with the image, the graphic is located in the image based on a position of the representation of the object in the image, the image is part of a video broadcast of the live event, and the graphic is combined with the image substantially in real time.

20. The method of claim 1, wherein:

the object comprises a vehicle and the live event comprises a race.

21. A system for processing image data, comprising:

at least one camera for capturing an image of a live event, the live event comprising an object; and

at least one processing facility which receives the image, the at least one processing facility;

performs first edge detection to detect first edges which extend in a first direction in the image, performs second edge detection to detect second edges which extend in a second direction in the image, detects which of the second edges are near the first edges and which of the second edges are not near the first edges, discards the second edges which are detected to be not near the first edges and performs morphological processing on the first edges and the second edges which are detected to be near the first edges to obtain a representation of the object.

22. The system of claim 21, wherein:

the at least one processing facility identifies a direction of motion of the object, the first direction is perpendicular to a direction of motion of the object in the image, and the second direction is parallel to the direction of motion;

the object moves in the direction of motion relative to a background object in the live event; and

the second edges which are not near the first edges comprise an edge of the background object.

23. A method for processing image data, comprising:

obtaining an image of a live event which is captured by at least one camera, the live event comprising an object;

determining a direction of motion of the object in the image;

detecting first edges in the image which extend perpendicular to the direction of motion;

detecting second edges in the image which extend parallel to the direction of motion;

dilating the first edges to provide dilated first edges;

identifying which of the second edges intersect with the dilated first edges and which of the second edges do not intersect with the dilated first edges; and

performing morphological processing on the first edges and the second edges which intersect with the dilated first edges, but not with the second edges which do not intersect with the dilated first edges, to obtain a representation of the object.

24. The method of claim 23, further comprising:

updating at least one of registration or sensor data of the at least one camera based on a position of the representation of the object in the image.

25. The method of claim 23, further comprising:

combining a graphic with the image, the graphic is located in the image based on a position of the representation of the object in the image, the image is part of a video broadcast of the live event, and the graphic is combined with the image substantially in real time.