METHOD, APPARATUS AND SYSTEM FOR PROCESSING DEPTH-RELATED INFORMATION

• US 20100220920A1
• Filed: 05/05/2008
• Published: 09/02/2010
• Est. Priority Date: 05/11/2007
• Status: Active Grant

• Current Assignee: Koninklijke Philips N.V.
• Original Assignee: Koninklijke Philips Electronics N.V. (Koninklijke Philips N.V.)
• Inventors: Barenbrug, Bart Gerard Bernard
• Granted Patent: US 8,588,514 B2
• Time in Patent Office: Days
• US Class Current: 382/154
• CPC Class Codes: G06T 2207/10012 Stereo images
• CPC Class Codes: G06T 7/55 from multiple images
• CPC Class Codes: H04N 13/10 Processing, recording or transmission of stereoscopic or multi-view image signals
• CPC Class Codes: H04N 13/122 Improving the 3D impression of stereoscopic images by modifying image signal contents, e.g. by filtering or adding monoscopic depth cues H04N13/128 takes precedence
• CPC Class Codes: H04N 13/128 Adjusting depth or disparity
• CPC Class Codes: H04N 13/144 for flicker reduction
• CPC Class Codes: H04N 2213/002 Eyestrain reduction by processing stereoscopic signals or controlling stereoscopic devices

Abstract

The invention relates to a method, apparatus and system for processing first depth-related information associated with an image sequence. The method of processing comprises mapping first depth-related information of respective images of a shot of the image sequence on corresponding second depth-related information using a first estimate of a characteristic of the distribution of first depth-related information associated with at least one image from the shot, the mapping adapting the first depth-related information by enhancing the dynamic range of a range of interest of first depth-related information defined at least in part by the first estimate, and the amount of variation in the mapping for respective images in temporal proximity in the shot being limited.

Claims

1. Method of processing first depth-related information (110) associated with an image sequence (105), comprising mapping (140) first depth-related information (110) of respective images of a shot of the image sequence on corresponding second depth-related information (150) using a first estimate (135) of a characteristic of the distribution of first depth-related information associated with at least one image from the shot, the mapping (140) adapting the first depth-related information by enhancing the dynamic range of a range of interest of first depth-related information defined at least in part by the first estimate, and wherein the amount of variation in the mapping (140) for respective images in temporal proximity in the shot is limited.

2. The method of claim 1, wherein the second depth-related information is first depth information and wherein enhancing the dynamic range of the range of interest of first depth-related information comprises allocating a larger fraction of a first depth range to the range of interest than according to the relationship between the first depth-related information (110) and depth of objects in the image.

3. The method of claim 1, wherein the second depth-related information is first disparity information and wherein enhancing the dynamic range of the range of interest of first depth-related information comprises allocating a larger fraction of a first disparity range to the range of interest than according to the relationship between the first depth-related information (110) and disparity of objects in the image.

4. The method of claim 1, wherein the first depth-related information (110) is one of second disparity information and second depth information.

5. The method of claim 1, wherein the maximum amount of variation in the respective mapping (140) used for adjacent images within the shot is upper-bounded by a predetermined threshold.

6. The method of claim 5, wherein the predetermined threshold is zero.

7. The method of claim 1, wherein the first estimate (135) represents an estimate of a characteristic of the distribution of first depth-related information for all images in the shot.

8. The method of claim 1, wherein the first estimate is based on temporally filtered first depth-related information associated with at least one image from the shot.

9. The method of claim 1, wherein the first estimate is based on weighted first depth-related information associated with at least one image from the shot, the weighting factors based on reliability information indicative of the reliability of the first depth-related information.

10. The method of claim 1, the method further comprising:

obtaining classifying information for the shot; and

adapting the mapping for respective images of the shot based on the classifying information.

11. The method of claim 1, wherein the first depth-related information used for determining the first estimate are located in a first image region;

and wherein the method further comprises reducing the probability of window violations using a second estimate, the second estimate being an estimate of a characteristic of the distribution of first depth-related information associated with at least one image from the shot, the first depth-related information used for determining the second estimate located in a second image region near an image border.

12. The method of claim 11, wherein reducing the probability of window violations comprises contracting the range of first depth-related information produced by the mapping.

16. Computer program product to be loaded by a computer arrangement, comprising instructions for processing first depth-related information (110) associated with an image sequence (105), comprising mapping first depth-related information (110) of respective images of a shot of the image sequence on corresponding depth-related information (150) as claimed claim 1, when run on the computer arrangement.

13. Apparatus for processing first depth-related information associated with an image sequence (105), comprising mapping first depth-related information (110) of respective images of a shot of the image sequence on corresponding second depth-related information (150), the apparatus comprising:

processing means (730) arranged to map the first depth-related information (110) of respective images of a shot of the image sequence (105) on the corresponding second depth-related information (150) using a first estimate (135) of a characteristic of the distribution of first depth-related information associated with at least one image from the shot, the mapping (140) adapting the first depth-related information by enhancing the dynamic range of a range of interest of first depth-related information defined at least in part by the first estimate, and wherein the amount of variation in the mapping (140) for respective images in temporal proximity in the shot is limited.

14. System (800) for processing first depth-related information associated with an image sequence (105), comprising mapping first depth-related information (110) of respective images of a shot of the image sequence on corresponding second depth-related information (150), the system comprising:

a first apparatus (805) comprising;

obtaining means (710) arranged to obtain shot information (125) classifying multiple sequential images from the image sequence (105) as a shot

obtaining means (720) arranged to obtain a first estimate (135) of a characteristic of the distribution of first depth-related information associated with at least one image from the shot;

transmitter means (815) arranged to send the shot information (125) and the first estimate (135) over a network (850); and

a second apparatus (810) comprising;

receiver means (820) arranged to receive the shot information (125) and the first estimate (135) from a network (850) and

processing means (730) arranged to map the first depth-related information (110) of respective images of the shot on the corresponding second depth-related information (150) using the first estimate (135), the mapping (140) adapting the first depth-related information by enhancing the dynamic range of a range of interest of first depth-related information defined at least in part by the first estimate, and wherein the amount of variation in the mapping (140) for respective images in temporal proximity in the shot is limited.

15. Signal (825) comprising first depth-related information (110) associated with an image-sequence (105), the signal further comprising:

shot information (125) classifying multiple images from the image sequence as a shot;

a first estimate (135) of a characteristic of the distribution of first depth-related information values associated with at least one image from the shot.