DETECTING AN OBJECT IN AN IMAGE USING TEMPLATES INDEXED TO LOCATION OR CAMERA SENSORS

• US 20090027500A1
• Filed: 12/19/2007
• Published: 01/29/2009
• Est. Priority Date: 07/27/2007
• Status: Active Grant

• Current Assignee: Sportsmedia Technology Corporation
• Original Assignee: Sportvision, Inc. (Sportsmedia Technology Corporation)
• Inventors: White, Marvin S., Cavallaro, Richard H., Milnes, Kenneth A., Heidmann, Timothy P., Elangovan, Vidya
• Granted Patent: US 8,456,527 B2
• Time in Patent Office: Days
• US Class Current: 348/159
• CPC Class Codes: A63F 2300/663 for simulating liquid objects, e.g. water, gas, fog, snow, clouds
• CPC Class Codes: A63F 2300/6692 using special effects, generally involving post-processing, e.g. blooming
• CPC Class Codes: A63F 2300/69 Involving elements of the real world in the game world, e.g. measurement in live races, real video
• CPC Class Codes: A63F 2300/8017 Driving on land or water; Flying
• CPC Class Codes: G06T 2207/30221 Sports video; Sports image
• CPC Class Codes: G06T 2207/30236 Traffic on road, railway or crossing
• CPC Class Codes: G06T 7/248 involving reference images or patches
• CPC Class Codes: G06T 7/74 involving reference images or patches
• CPC Class Codes: G06V 10/751 Comparing pixel values or logical combinations thereof, or feature values having positional relevance, e.g. template matching
• CPC Class Codes: G06V 2201/08 Detecting or categorising vehicles
• CPC Class Codes: H04N 5/265 Mixing
• CPC Class Codes: H04N 5/272 Means for inserting a foreground image in a background image, i.e. inlay, outlay
• CPC Class Codes: H04N 5/2723 Insertion of virtual advertisement; Replacing advertisements physical present in the scene by virtual advertisement data processing systems or methods specially adapted for marketing G06Q30/02

Abstract

An object is detected in images of a live event by storing and indexing templates based on representations of the object from previous images. For example, the object may be a vehicle which repeatedly traverses a course. A first set of images of the live event is captured when the object is at different locations in the live event. A representation of the object in each image is obtained, such as by image recognition techniques, and a corresponding template is stored. When the object again traverses the course, for each location, the stored template which is indexed to the location can be retrieved for use in detecting the object in a current image. The object'"'"'s current location may be obtained from GPS data from the object, for instance, or from camera sensor data, e.g., pan, tilt and zoom, which indicates a direction in which the camera is pointed.

Claims

1. A method for processing image data, comprising:

obtaining a first set of images of a live event including an object when the object is at different locations in the live event, the first set of images are captured by at least one camera;

storing and indexing templates based on representations of the object in images of the first set of images;

obtaining at least a first additional image of the live event which is captured by the at least one camera;

obtaining, from the stored templates, based on the indexing, at least one template; and

detecting a representation of the object in the at least a first additional image by correlation with the at least one template.

2. The method of claim 1, wherein:

the at least one template comprises monochrome data.

3. The method of claim 1, wherein:

the at least one template comprises at least one of chrominance data and luminance data.

4. The method of claim 1, wherein:

the indexing is based on sensor data of the at least one camera.

5. The method of claim 4, wherein:

the sensor data comprises at least one of pan, tilt and zoom.

6. The method of claim 1, wherein:

the indexing is based on location data of the different locations of the object in the live event.

7. The method of claim 6, wherein:

the location data comprises GPS data.

8. The method of claim 6, wherein:

the location data comprises radio data, the radio data is used to identify the different locations by at least one of multi-angulation and multi-lateration.

9. The method of claim 6, wherein:

the location data comprises data which indicates a time and/or distance since the object passed a known location in the live event.

10. The method of claim 1, wherein:

the different locations are along a course which the object traverses, and the indexing is to the different locations.

11. The method of claim 1, further comprising:

updating registration and/or sensor data of the at least one camera based on a position of the representation of the object in the at least a first additional image.

12. The method of claim 1, wherein:

the object comprises a vehicle and the live event comprises a race.

13. The method of claim 1, further comprising:

obtaining at least a second additional image of the live event which is captured by the at least one camera; and

detecting a representation of the object in the at least a second additional image by correlation with the at least one template.

14. The method of claim 1, further comprising:

combining a graphic with the at least a first additional image, the graphic is located in the at least a first additional image based on a position of the representation of the object in the at least a first additional image, the at least a first additional image is part of a video broadcast of the live event, and the graphic is combined with the at least a first additional image substantially in real time.

15. The method of claim 14, wherein:

the graphic is located in the at least a first additional image based on an orientation of the object.

16. The method of claim 14, wherein:

the graphic is located in the at least a first additional image based on a perspective of the object.

17. The method of claim 16, wherein:

the template comprises pixel data.

18. The method of claim 1, wherein:

the representation of the object in the at least a first additional image is determined by correlating the obtained templates with pixel data of the at least a first additional image.

19. The method of claim 1, wherein:

the representation of the object in the at least a first additional image is determined by correlating the obtained templates with an edge detected and morphologically processed image which is obtained from the at least a first additional image.

20. A system for processing image data, comprising:

at least one camera for capturing a first set of images of a live event, the live event including an object; and

at least one processing facility which receives the first set of images, the at least one processing facility;

a) stores and indexes templates based on representations of the object in images of the first set of images, b) obtains at least a first additional image of the live event which is captured by the at least one camera, c) obtains, from the stored templates, based on the indexing, at least one template, and d) detects a representation of the object in the at least a first additional image by correlation with the at least one template.

21. At least one processor readable storage device having processor readable code embodied thereon for programming at least one processor to perform a method, the method comprising:

determining templates based on representations of an object in images of a first set of images, the first set of images is of a live event and includes the object when the object is at different locations in the live event;

storing and indexing the templates;

obtaining at least a first additional image of the live event;

obtaining, from the stored templates, based on the indexing, at least one template; and

detecting a representation of the object in the at least a first additional image by correlation with the at least one template.

22. The at least one processor readable storage device of claim 21, wherein:

the indexing is based on sensor data of at least one camera which captures the first set of images and the at least a first additional image.

23. The at least one processor readable storage device of claim 22, wherein:

the sensor data comprises at least one of pan, tilt and zoom.

24. The at least one processor readable storage device of claim 21, wherein:

the indexing is based on location data of the different locations of the object in the live event.