Integration of monocular cues to improve depth perception

• US 6,157,733 A
• Filed: 04/18/1997
• Issued: 12/05/2000
• Est. Priority Date: 04/18/1997
• Status: Expired due to Term

• Current Assignee: AT&T Corporation (AT&T, Inc.)
• Original Assignee: AT&T Corporation (AT&T, Inc.)
• Inventors: Swain, Cassandra Turner
• Primary Examiner(s): ROGERS, SCOTT A
• Application Number:
  US08/844,422
• Time in Patent Office: 1,327 Days
• Field of Search: 382/151, 382/152, 382/154, 382/173, 382/190, 382/274, 382/282, 382/284, 382/285, 382/255, 382/199, 382/203, 382/277, 382/298
• US Class Current: 382/154
• CPC Class Codes: A63F 2300/6646 for the computation and display of the shadow of an object or character
• CPC Class Codes: G06T 15/00 3D [Three Dimensional] image rendering
• CPC Class Codes: G06T 15/02 Non-photorealistic rendering

Abstract

One or more monocular cues are extracted from an original image and combined to enhance depth effect. An original image is acquired and segmented into one or more objects. The objects are identified as being either in the foreground or the background, and an object of interest is identified. One or more depth cues are then extracted from the original image, including shading, brightness, blur and occlusion. The depth cues may be in the form of one or more intermediate images having an improved depth effect. The depth cues are then combined or applied to create an image with enhanced depth effect.

Claims

1. A method of using monocular depth cues in a single two dimensional image to improve depth perception in the image, said method comprising the steps of:

obtaining an image, said image comprising one or more objects, each of said objects in the image being either in the foreground or the background of the image;

segmenting said one or more objects in the image;

identifying which of said objects are in the foreground and which of said objects are in the background of the image;

increasing the brightness of one or more said foreground objects relative to the brightness of one or more of said background objects in the image;

blurring one or more said background objects in said image, said steps of increasing and blurring operating to improve depth perception of the image;

generating a shadow of one or more said foreground objects in the image; and

casting the generated shadow on one or more said background objects in the image.

2. A method of using monocular depth cues in a single image to improve depth perception in the image, said method comprising the steps of:

obtaining an image, said image comprising one or more objects, each of said objects in the image being either in the foreground or the background of the image;

segmenting said one or more objects in the image;

identifying which of said objects are in the foreground and which of said objects are in the background;

increasing the brightness of one or more said foreground objects relative to the brightness of one or more of said background objects in the image;

generating a shadow of one or more said foreground objects in the image; and

casting the generated shadow on one or more said background objects in the image, said steps of increasing, generating and casting operating to improve depth perception of the image.

3. The method of claim 2 and further comprising the step of increasing the size of one or more said foreground objects relative to the size of one or more said background objects in said image.

4. A method of using monocular depth cues in a single image to improve depth perception in the image, said method comprising the steps of:

obtaining an image, said image comprising one or more objects, each of said objects in the image being either in the foreground or the background of the image;

segmenting said one or more objects in the image;

identifying which of said objects are in the foreground and which of said objects are in the background;

blurring one or more said background objects in said image;

generating a shadow of one or more said foreground objects in the image;

and casting the generated shadow on one or more said background objects in the image, said steps of blurring, generating and casting operating to improve depth perception of the image.

5. The method of claim 4 and further comprising the step of increasing the size of one or more said foreground objects relative to the size of one or more said background objects in said image.

6. A method of using monocular depth cues in a single image to improve depth perception in the image, said method comprising the steps of:

obtaining an image, said image comprising one or more objects, each of said objects in the image being either in the foreground or the background of the image;

segmenting said one or more objects in the image;

identifying which of said objects are in the foreground and which of said objects are in the background;

generating a shadow of one or more said foreground objects in the image;

casting the generated shadow on one or more said background objects in the image; and

increasing the size of one or more said foreground objects relative to the size of one or more said background objects in said image, said steps of generating, casting and increasing operating to enhance depth perception of the image.

7. A method of using monocular depth cues in a single image to improve depth perception in the image, said method comprising the steps of:

obtaining an image, said image comprising one or more objects, each of said objects in the image being either in the foreground or the background of the image;

segmenting said one or more objects in the image;

identifying which of said objects are in the foreground and which of said objects are in the background;

increasing the brightness of one or more said foreground objects relative to the brightness of one or more of said background objects in the image;

blurring one or more said background objects in said image;

generating a shadow of one or more said foreground objects in the image;

casting the generated shadow on one or more said background objects in the image; and

increasing the size of one or more said foreground objects relative to the size of one or more said background objects in said image, said steps of increasig the brightness, blurring, generating, casting and increasing the size operating to enhance depth perception of the image.

8. The method of claim 7 wherein said step of segmenting comprises the step of grouping like pixels together based on intensity which are in close proximity to one another, each said pixel group comprising a segmented object.

9. The method of claim 7 wherein said step of segmenting comprises the steps of:

obtaining a plurality of successive frames of an image from the same point;

comparing corresponding pixels in each of the plurality of frames;

identifying, based on said step of comparing, any pixels which have changed; and

grouping together pixels in close proximity which have changed, each said pixel group comprising a segmented object.

10. The method of claim 7 wherein said step of increasing the brightness of one or more said foreground objects comprises the step of decreasing the brightness only of one or more of said background objects in the image.

11. The method of claim 7 wherein said step of increasing the brightness of one or more said foreground objects comprises the step of increasing the brightness of only one or more said foreground objects.

12. The method of claim 7 wherein said step of blurring comprises the steps of:

removing one or more said foreground objects from the image leaving a remaining image;

blurring the objects in the remaining image; and

superimposing the removed foreground objects onto the blurred remaining image.

13. The method of claim 7 wherein said step of blurring comprises the step of convolving an original focused image with a point spread function to obtain a blurred image.

14. The method of claim 7 wherein said step of generating a shadow comprises the steps of:

calculating the x position of the shadow; and

calculating the y position of the shadow.

15. The method of claim 14 wherein said step of casting the generated shadow comprises the step of decreasing the brightness of pixels at the calculated position of the shadow in the image and in the shape of one or more of said foreground objects.

16. The method of claim 7 wherein said step of increasing the size of one or more said foreground objects comprises the steps of:

removing one or more said foreground objects from the image leaving a remaining image;

supersampling the pixels in the removed foreground objects to generate one or more larger foreground objects; and

superimposing the one or more larger foreground objects onto the remaining image.

17. A method of using monocular depth cues in a single image to improve depth perception in the image, said method comprising the steps of:

obtaining an original image, said original image comprising one or more objects, each of said objects in the image being either in the foreground or the background of the image;

segmenting said one or more objects in the image;

identifying which of said objects are in the foreground and which of said objects are in the background;

creating an intermediate bright image by increasing the brightness of one or more said foreground objects in the original image relative to the brightness of one or more of said background objects;

creating an intermediate blurred image by selectively blurring one or more said background objects in said original image;

creating an intermediate shadow image by performing steps a) and b);

a) generating a shadow of one or more said foreground objects in the original image; and

b) casting the generated shadow on one or more said background objects in the original image; and

combining said intermediate bright image, said intermediate blurred image and said intermediate shadow image into a single final image having enhanced depth perception.

18. The method of claim 17 and further comprising the step of increasing the size of one or more said foreground objects relative to the size of one or more said background objects in said final image, thereby causing one or more said foreground objects to at least partially occlude one or more said background objects.

19. The method of claim 17 wherein said step of combining comprises the step of averaging corresponding pixel values from the intermediate bright image, the said intermediate blurred image and the intermediate shadow image to generate a combined image.

20. The method of claim 17 wherein said step of combining comprises the steps of:

adding together corresponding pixel values from the intermediate bright image, the said intermediate blurred image and the intermediate shadow image to generate cumulative pixel values for each pixel in the image; and

dividing each cumulative pixel value by three to generate a combined image.

21. A method of using monocular depth cues in a single image to improve depth perception in the image, said method comprising the steps of:

obtaining an image, said image comprising one or more objects, at least one of said objects being of interest;

segmenting said one or more objects in the image;

identifying which of said objects are of interest and the remaining objects not of interest;

increasing the brightness of said one or more objects of interest relative to the brightness of the remaining objects in the image not of interest;

blurring said remaining objects not of interest;

generating a shadow of said one or more objects of interest in the image;

casting the generated shadow of the objects of interest onto said one or more remaining objects in the image not of interest; and

increasing the size of said one or more objects of interest relative to the size of said one or more remaining objects in said image not of interest, said steps of increasing the brightness, blurring, generating, casting and increasing the size operating to enhance depth perception of the image.

22. A method of using monocular depth cues in a single image to improve depth perception in the image, said method comprising the steps of:

obtaining an image, said image comprising one or more objects, each of said objects in the image being either in the foreground or the background of the image;

segmenting said one or more objects in the image;

identifying which of said objects are in the foreground and which of said objects are in the background;

to adjust a view of a foreground object relative to a background object in the image, performing at least two operations selected from the group consisting of;

a) increasing the brightness of one or more said foreground objects relative to the brightness of one or more of said background objects in the image;

b) blurring one or more said background objects in said image;

c) generating a shadow of one or more said foreground objects in the image and casting the generated shadow on one or more said background objects in the image; and

d) increasing the size of one or more said foreground objects relative to the size of one or more said background objects in said image.