System and method for transitioning between real images and virtual images

• US 20030030734A1
• Filed: 08/08/2002
• Published: 02/13/2003
• Est. Priority Date: 08/10/2001
• Status: Active Grant

• Current Assignee: Sony Corporation (Sony Group Corp.), Sony Electronics Inc. (Sony Group Corp.)
• Original Assignee: Sony Corporation (Sony Group Corp.), Sony Electronics Inc. (Sony Group Corp.)
• Inventors: Gibbs, Simon, Hoch, Michael
• Granted Patent: US 7,173,672 B2
• Time in Patent Office: Days
• US Class Current: 348/239
• CPC Class Codes: A63F 2300/69 Involving elements of the real world in the game world, e.g. measurement in live races, real video
• CPC Class Codes: A63F 2300/8017 Driving on land or water; Flying
• CPC Class Codes: H04N 21/21805 enabling multiple viewpoints, e.g. using a plurality of cameras
• CPC Class Codes: H04N 21/2187 Live feed
• CPC Class Codes: H04N 21/23617 by inserting additional data into a data carousel, e.g. inserting software modules into a DVB carousel arrangements for broadcast or for distribution of identical information repeatedly in broadcast distribution systems H04H20/16
• CPC Class Codes: H04N 21/44012 involving rendering scenes according to scene graphs, e.g. MPEG-4 scene graphs
• CPC Class Codes: H04N 21/8126 involving additional data, e.g. news, sports, stocks, weather forecasts
• CPC Class Codes: H04N 21/8146 involving graphical data, e.g. 3D object, 2D graphics
• CPC Class Codes: H04N 5/2224 related to virtual studio applications
• CPC Class Codes: H04N 5/262 Studio circuits, e.g. for mixing, switching-over, change of character of image, other special effects ; Cameras specially adapted for the electronic generation of special effects

Abstract

The invention illustrates a system and method of blending a virtual scene and a real scene comprising: capturing a real image of a real event from a real camera; receiving an instrumentation data based on the real event; and rendering a virtual image wherein the virtual image is based on the instrumentation data and the real image.

Claims

1. A system comprising:

a. a real camera device for capturing a real image and a corresponding instrumentation data associated with the real image; and

b. a rendering module for generating a virtual image wherein the virtual image is generated from a virtual perspective based upon the instrumentation data for blending the virtual image and the real image into a resultant image.

2. The system according to claim 1 further comprising a compositing module configured for receiving the virtual image and the real image and displaying the resultant image.

3. The system according to claim 1 wherein the resultant image includes the real image and the virtual image.

4. The system according to claim 1 wherein the instrumentation data includes one of a camera position, a camera zoom, a camera pan, a camera tilt, a camera field-of-view, and an object location.

5. The system according to claim 1 wherein the rendering module further comprising a depth map module configured for ordering a blending priority for an object within the real image is based on the instrumentation data.

6. The system according to claim 5 wherein the blending priority is based on a perspective angle change of the object.

7. The system according to claim 1 wherein the rendering module further comprising an alignment module configured for aligning the virtual image with a viewpoint of the real camera device wherein the alignment module utilizes the instrumentation data.

8. The system according to claim 1 further comprising a sensor for generating the instrumentation data for measuring a physical parameter of a real event.

9. The system according to claim 8 wherein the physical parameter may be one of a force exerted on an object and a location of the object.

10. A method comprising:

a. capturing a real image of a real event from a real camera;

b. receiving an instrumentation data based on the real event; and

c. rendering a virtual image wherein the virtual image is based on the instrumentation data and the real image.

11. The method according to claim 10 wherein the instrumentation data includes one of a camera position, a camera zoom, a camera pan, a camera tilt, a camera field-of-view, and an object location.

12. The method according to claim 10 further comprising tracking a parameter of the real camera via the instrumentation data.

13. The method according to claim 10 further comprising aligning a viewpoint of the real camera with a viewpoint of the virtual image via the instrumentation data.

14. The method according to claim 10 further comprising blending the real image and the virtual image into a resultant image.

15. The method according to claim 10 further comprising calculating a perspective angle change of an object within the real image via the instrumentation data.

16. The method according to claim 10 further comprising positioning the real image relative to the virtual image in response to the instrumentation data.

17. A computer-readable medium having computer executable instructions for performing a method comprising:

a. capturing a real image of a real event from a real camera;

b. receiving an instrumentation data based on the real event; and

c. rendering a virtual image wherein the virtual image is based on the instrumentation data and the real image.