System and method for data assisted chroma-keying

• US 20030030651A1
• Filed: 08/08/2002
• Published: 02/13/2003
• Est. Priority Date: 08/10/2001
• Status: Active Grant

• Current Assignee: Sony Corporation (Sony Group Corp.), Sony Electronics Inc. (Sony Group Corp.)
• Original Assignee: Sony Corporation (Sony Group Corp.), Sony Electronics Inc. (Sony Group Corp.)
• Inventors: Gibbs, Simon, Hoch, Michael
• Granted Patent: US 7,091,989 B2
• Time in Patent Office: Days
• US Class Current: 345/619
• CPC Class Codes: A63F 2300/69 Involving elements of the real world in the game world, e.g. measurement in live races, real video
• CPC Class Codes: A63F 2300/8017 Driving on land or water; Flying
• CPC Class Codes: G09G 5/026 Control of mixing and/or overlay of colours in general G09G5/022 and G09G5/024 take precedence
• CPC Class Codes: G09G 5/377 for mixing or overlaying two or more graphic patterns G09G5/02, G09G5/397 take precedence
• CPC Class Codes: H04N 21/21805 enabling multiple viewpoints, e.g. using a plurality of cameras
• CPC Class Codes: H04N 21/2187 Live feed
• CPC Class Codes: H04N 21/23617 by inserting additional data into a data carousel, e.g. inserting software modules into a DVB carousel arrangements for broadcast or for distribution of identical information repeatedly in broadcast distribution systems H04H20/16
• CPC Class Codes: H04N 21/44012 involving rendering scenes according to scene graphs, e.g. MPEG-4 scene graphs
• CPC Class Codes: H04N 21/8126 involving additional data, e.g. news, sports, stocks, weather forecasts
• CPC Class Codes: H04N 21/8146 involving graphical data, e.g. 3D object, 2D graphics
• CPC Class Codes: H04N 5/2224 related to virtual studio applications
• CPC Class Codes: H04N 9/75 Chroma key

Abstract

The invention illustrates a system and method of displaying a base image and an overlay image comprising: capturing a base image of a real event; receiving an instrumentation data based on the real event; identifying a visual segment within the base image based on the instrumentation data; and rendering an overlay image within the visual segment.

Claims

1. A system comprising:

a. a camera device for capturing a base image and a corresponding instrumentation data associated with the base image; and

b. a segment identifier module for defining a visual segment within the base image based upon the instrumentation data wherein the visual segment is configured to accept an overlay image.

2. The system according to claim 1 further comprising a compositing module configured for receiving the base image and the overlay image and simultaneously displaying the base image and the overlay image.

3. The system according to claim 1 further comprising a rendering module configured for rendering the overlay image within the visual segment of the base image.

4. The system according to claim 1 wherein the instrumentation data includes one of a camera position, a camera zoom, a camera pan, a camera tilt, a camera field-of-view, and an object location.

5. The system according to claim 1 further comprising a depth map module configured for ordering an object relative to the overlay image based on the instrumentation data and defining a boundary for the visual segment.

6. The system according to claim 1 further comprising a color histogram module for analyzing a color distribution of the visual segment and updating a key color value.

7. The system according to claim 1 wherein the segment identifier is configured to track a location of the visual segment within the base image based on the instrumentation data.

8. The system according to claim 1 further comprising a sensor for generating the instrumentation data for measuring a physical parameter of a real event.

9. The system according to claim 8 wherein the physical parameter may be one of a force exerted on an object and a location of the object.

10. A method comprising:

a. capturing a base image of a real event;

b. receiving an instrumentation data based on the real event;

c. identifying a visual segment within the base image based on the instrumentation data; and

d. rendering an overlay image within the visual segment.

11. The method according to claim 10 wherein the instrumentation data includes one of a camera position, a camera zoom, a camera pan, a camera tilt, a camera field-of-view, and an object location.

12. The method according to claim 10 further comprising tracking the visual segment via the instrumentation data.

13. The method according to claim 10 further comprising analyzing a color distribution of the visual segment.

14. The method according to claim 13 further comprising updating a key color value based on the color distribution.

15. The method according to claim 10 further comprising calculating a depth map of an object within the base image relative to the visual segment based on the instrumentation data.

16. The method according to claim 15 further refining the visual segment based on the depth map.

17. The method according to claim 10 further comprising simultaneously displaying the base image and the overlay image within the visual segment.

18. The method according to claim 10 wherein the overlay image is one of a graphic and a base image captured by a device.

19. The method according to claim 10 wherein the overlay image is an advertisement.

20. A system comprising:

a. a camera device for capturing a base image;

a. sensing means for capturing an instrumentation data associated with a base image; and

b. a segment identifier module for defining a visual segment within the base image based upon the instrumentation data wherein the visual segment is configured to accept an overlay image.

21. The system according to claim 20 further comprising a camera device for capturing a base image;

22. A computer-readable medium having computer executable instructions for performing a method comprising:

a. capturing a base image of a real event;

b. receiving an instrumentation data based on the real event;

c. identifying a visual segment within the base image based on the instrumentation data; and

d. rendering an overlay image within the visual segment.