Method and apparatus for optimizing capture device settings through depth information

• US 20050157204A1
• Filed: 01/16/2004
• Published: 07/21/2005
• Est. Priority Date: 01/16/2004
• Status: Active Grant

• Current Assignee: Sony Interactive Entertainment Inc. (Sony Group Corp.)
• Original Assignee: Sony Computer Entertainment Incorporated (Sony Group Corp.)
• Inventors: Marks, Richard L.
• Granted Patent: US 7,663,689 B2
• Time in Patent Office: Days
• US Class Current: 348/370
• CPC Class Codes: A63F 2300/1012 involving biosensors worn by the player, e.g. for measuring heart beat, limb activity
• CPC Class Codes: A63F 2300/1093 using visible light
• CPC Class Codes: A63F 2300/69 Involving elements of the real world in the game world, e.g. measurement in live races, real video
• CPC Class Codes: H04N 23/635 Region indicators; Field of view indicators
• CPC Class Codes: H04N 23/72 Combination of two or more compensation controls
• CPC Class Codes: H04N 5/2226 Determination of depth image, e.g. for foreground/background separation determining depth by image analysis in general G06T7/50; segmentation by image analysis in general G06T7/10
• CPC Class Codes: H04N 5/272 Means for inserting a foreground image in a background image, i.e. inlay, outlay

Abstract

A method for adjusting image capture settings for an image capture device is provided. The method initiates with identifying a scene. Then, an image of the scene is captured. The method includes generating a depth mask of the scene from data defining the image of the scene. Then, pixel values corresponding to objects within any one or both of a foreground region and a background region of the captured image are adjusted based upon bit values of the depth mask. An image capture device and a system are provided.

Claims

1. A method for differentiating between foreground objects and background objects within a scene being captured through an image capture device, comprising:

emitting a ray of light from a light source toward an object of the scene;

opening an aperture cover allowing access to a sensor of the image capture device for reflected light from the light source;

closing the aperture cover after a set time, the predefined amount of time corresponding to a maximum distance traveled by the light;

generating a depth mask identifying objects within a foreground region of the scene based upon the light captured during the set time; and

adjusting image capture device parameters according to bit values of the depth mask prior to capturing a subsequent corresponding image of the scene.

2. The method of claim 1, further comprising:

storing the depth mask in memory of the image capture device.

3. The method of claim 1, wherein the light source is configured to emit infrared light.

4. The method of claim 1, wherein the method operation of opening an aperture cover allowing access to a sensor of the image capture device includes, receiving reflected light from the objects within the foreground region.

5. The method of claim 1, wherein the method operation of generating a depth mask identifying objects within a foreground region of the scene based upon the light captured during the predefined time includes,

identifying objects within the foreground region with a first bit value;

and

identifying objects within a background region with a second bit value.

6. The method of claim 1, wherein the method operation of adjusting image capture device parameters according to bit values of the depth mask prior to capturing a subsequent corresponding image of the scene includes,

determining an optimal amount of light based upon the depth mask;

and

adjusting the aperture cover to allow the optimal amount of light into the image capture device.

7. The method of claim 1, wherein the image capture device parameters are selected from the group consisting of focus, brightness, exposure, and gain.

8. The method operation of claim 1, wherein the method operation of emitting a ray of light from a light source toward an object of the scene includes,

pulsing infrared light from the light source.

9. A method for adjusting image capture settings for an image capture device, comprising:

identifying a scene;

capturing an image of the scene;

generating a depth mask of the scene from data defining the image of the scene; and

adjusting pixel values corresponding to objects within any one or both of a foreground region and a background region of the captured image.

10. The method of claim 9, wherein the method operation of generating a depth mask of the scene from data defining the image of the scene includes,

segmenting the foreground and background regions of the scene.

11. The method of claim 9, wherein the data defining the image of the scene includes pixel data where each pixel is tagged with distance information.

12. The method of claim 9, wherein the method operation of adjusting pixel values corresponding to objects within any one or both of a foreground region and a background region of the captured image based upon bit values of the depth mask includes,

independently adjusting pixel values associated with the foreground region from pixel values associated with the background region.

13. The method of claim 9, wherein the image capture device is selected from the group consisting of a digital camera, a web cam, and a camcorder.

14. The method of claim 9, further comprising:

displaying a portion of the image of the scene having adjusted pixel values.

15. The method of claim 14, wherein the portion of the image of the scene is an image of a participant for use in an interactive gaming application.

16. The method of claim 9, wherein the method operation of adjusting pixel values corresponding to objects within any one or both of a foreground region and a background region of the captured image includes,

adjusting the pixel values according to bit values of the depth mask.

17. An image capture device configured to provide an image of a scene, comprising:

depth logic configured to provide a depth mask associated with the scene, the depth mask configured to distinguish between foreground objects and background objects within the scene; and

image capture logic configured to adjust a characteristic associated with the image based upon a corresponding bit value of the depth mask, wherein the bit value determines whether the respective pixel is associated with one of the foreground objects and the background objects.

18. The image capture device of claim 17, wherein the depth mask is a bit mask having a first logical value assigned to represent the foreground objects and a second logical value assigned to represent the background objects.

19. The image capture device of claim 17, further comprising:

a sensor in communication with the depth logic, the sensor configured to receive a light signal reflected from one of the foreground objects, the receipt of the light signal indicating a location corresponding to one of the foreground objects.

20. The image capture device of claim 17, wherein each logic element is one or a combination of hardware and software.

21. The image capture device of claim 17, wherein the image capture device is a video capture device.

22. The image capture device of claim 21, wherein the depth logic is further configured to periodically provide a depth mask for a sequence of video frames captured by the video capture device.

23. The image capture device of claim 17, wherein the characteristic is selected from the group consisting of exposure, gain, focus and brightness.

24. The image capture device of claim 17, wherein the image capture logic is further configured to adjust each pixel of image data of the scene.

25. A system, comprising:

a computing device;

a display screen in communication with the computing device, the display screen configured to display an image of a scene;

a video capture device in communication with the computing device, the video capture device providing scene image data to the computing device for presentation on the display screen, the video capture device including,

depth logic configured to provide a depth mask associated with the scene, the depth mask configured to distinguish between foreground objects and background objects within the scene; and

image capture logic configured to adjust a characteristic associated with each pixel of the image data based upon depth information.

26. The system of claim 25, wherein the computing device is a game console.

27. The system of claim 25, wherein the depth logic is further configured to periodically provide a single depth mask for a sequence of video frames captured by the video capture device.

28. The system of claim 25, wherein the characteristic is selected from the group consisting of exposure, gain, focus, and brightness.

29. The system of claim 25, wherein the video capture device is a webcam.

30. The system of claim 25, wherein the image data defines data for each pixel, the data for each pixel including distance information.

31. The system of claim 26, wherein the scene image data includes an image of a person, the image of the person being incorporated into a video game for interaction therein.

32. The system of claim 25, wherein the depth information is obtained from a depth mask, the depth mask defining a relative distance between an object associated with the corresponding pixel and the video capture device.