Real world/virtual world correlation system using 3D graphics pipeline

• US 6,778,171 B1
• Filed: 10/31/2000
• Issued: 08/17/2004
• Est. Priority Date: 04/05/2000
• Status: Expired due to Fees

• Current Assignee: Gracenote Media Services LLC (RR Donnelley & Sons Company)
• Original Assignee: Eagle New Media Investments, LLC (Nexstar Media Group, Inc.)
• Inventors: Kikinis, Dan
• Primary Examiner(s): Sealey, Lance W.
• Application Number:
  US09/704,387
• Time in Patent Office: 1,386 Days
• Field of Search: 345/419, 345/425, 345/64, 345/162, 345/169, 345/972, 382/103, 382/107, 382/276, 348/169, 348/14.05, 348/14.07, 348/155, 348/170, 342/357.06, 342/357.07, 342/357.13, 342/457, 707/500.1, 707/501.1
• US Class Current: 345/419
• CPC Class Codes: G06T 19/00 Manipulating 3D models or images for computer graphics
• CPC Class Codes: G06T 2207/30221 Sports video; Sports image

Abstract

A method of tracking objects that allows objects to be tracked across multiple scene changes, with different camera positions, without losing track of the selected object.

In one embodiment, a method of tracking an object using a computer, a display device, a camera, and a camera tracking device, the computer being coupled to the display device, the camera and the camera tracking device is disclosed. The method includes: A first image from within a field-of-view of the camera is captured. The first image, which includes an actual object with a tracking device, is displayed on the display device. Information about the tracking device'"'"'s location is received. The information is used to create a virtual world reflecting the actual object'"'"'s position within the field-of-view of the camera as a shape in the virtual world. Information about the camera tracking device is received. A virtual-camera position in the virtual world is created. A 3D graphics pipeline is used to create a second image, the second image presenting the shape in the virtual world. The second image is used to obtain the actual object'"'"'s position.

In another embodiment, the method includes using the virtual-camera'"'"'s position to compute a new position for the camera to track the actual object.

Claims

1. A method comprising:

receiving data identifying a real-time physical location of an object, the object including a tracking device;

receiving data identifying a real-time field of view for a camera, the data includes a length vector within the view of the camera indicating a distance between the camera and the object, the camera including a tracking device;

generating a three-dimensional (3D) image representing the object as viewed within the field of view of the camera, the 3D image generated based on the data identifying the real-time location of the object and the real-time field of view of the camera;

adding an object ID, associated with a link, to the object represented in the 3D image; and

joining data from the 3D image including the object ID with a video image that includes the object as received from the camera, wherein the video when displayed thereafter includes the object ID, associated with the link, to the object represented in the video.

2. The method of claim 1, wherein the object ID, associated with the link, represented in the video is not displayed.

3. The method of claim 1, wherein the object ID, associated with the link, represented in the video is displayed as an icon, the icon when activated retrieves data which is displayed.

4. The method of claim 1, wherein the data identifying the real-time field of view for the camera, includes focus information relating to the field of view for the camera.

5. The method of claim 4, wherein the focus information relating to the field of view for the camera, includes a length vector within the view of the camera indicating a distance between the camera and the object.

6. The method of claim 1, wherein the receiving data identifying the real-time physical location of the object and the receiving data identifying the real-time field of view for the camera, includes receiving the information via Global Positioning Satellite.

7. A computer readable storage medium containing executable computer program instructions which when executed perform a method comprising:

receiving data identifying a real-time physical location of an object, the object including a tracking device;

receiving data identifying a real-time field of view for a camera, the data includes a length vector within the view of the camera indicating a distance between the camera and the object, the camera including a tracking device;

generating a three-dimensional (3D) image representing the object as viewed within the field of view of the camera, the 3D image generated based on the data identifying the real-time location of the object and the real-time field of view of the camera;

adding an object ID, associated with a link, to the object represented in the 3D image; and

joining data from the 3D image including the object ID with a video image that includes the object as received from the camera, wherein the video when displayed thereafter includes the object ID, associated with the link, to the object represented in the video.

8. The computer readable storage medium of claim 7, wherein the object ID, associated with the link, represented in the video is not displayed.

9. The computer readable storage medium of claim 7, wherein the object ID, associated with the link, represented in the video is displayed as an icon, the icon when activated retrieves data which is displayed.

10. The computer readable storage medium of claim 7, wherein the data identifying the real-time field of view for the camera, includes focus information relating to the field of view for the camera.

11. The computer readable storage medium of claim 10, wherein the focus information relating to the field of view for the camera, includes a length vector within the view of the camera indicating a distance between the camera and the object.

12. The computer readable storage medium of claim 7, wherein the receiving data identifying the real-time physical location of the object and the receiving data identifying the real-time field of view for the camera, includes receiving the information via Global Positioning Satellite.