IMAGE PROCESSING APPARATUS, IMAGE PROCESSING METHOD, RECORDING METHOD, AND RECORDING MEDIUM

• US 20110273543A1
• Filed: 07/20/2011
• Published: 11/10/2011
• Est. Priority Date: 01/21/2009
• Status: Active Grant

• Current Assignee: Nikon Corporation
• Original Assignee: Nikon Corporation
• Inventors: Ichihara, Yutaka, USHIO, Yoshijiro
• Granted Patent: US 8,791,989 B2
• Time in Patent Office: Days
• US Class Current: 348/54
• CPC Class Codes: G02B 2027/0134 of stereoscopic type
• CPC Class Codes: G02B 2027/014 comprising information/image processing systems
• CPC Class Codes: G02B 2027/0178 Eyeglass type eyeglass details G02C
• CPC Class Codes: G02B 27/0172 characterised by optical features
• CPC Class Codes: H04N 13/122 Improving the 3D impression of stereoscopic images by modifying image signal contents, e.g. by filtering or adding monoscopic depth cues H04N13/128 takes precedence
• CPC Class Codes: H04N 13/128 Adjusting depth or disparity
• CPC Class Codes: H04N 13/261 with monoscopic-to-stereoscopic image conversion
• CPC Class Codes: H04N 13/305 using lenticular lenses, e.g. arrangements of cylindrical lenses
• CPC Class Codes: H04N 13/31 using parallax barriers
• CPC Class Codes: H04N 13/383 for tracking with gaze detection, i.e. detecting the lines of sight of the viewer's eyes
• CPC Class Codes: H04N 13/398 Synchronisation thereof; Control thereof

Abstract

Provided is an image processing apparatus that stereoscopically displays a two-dimensional image, comprising a generating section that generates a left-side image and a right-side image by shifting the two-dimensional image left and right in a display region; a right-edge processing section that attaches a right-edge image, which is displayed within a prescribed range from a right edge of the display region, to a right side of the left-side image; a left-edge processing section that attaches a left-edge image, which is displayed within a prescribed range from a left edge of the display region, to a left side of the right-side image; and an output section that outputs the left-side image with the right-edge image attached thereto to a left eye of a user and outputs the right-side image with the left-edge image attached thereto to a right eye of the user.

Claims

1. An image processing apparatus that stereoscopically displays a two-dimensional image, comprising:

a generating section that generates a left-side image and a right-side image by shifting the two-dimensional image left and right in a display region;

a right-edge processing section that attaches a right-edge image, which is displayed within a prescribed range from a right edge of the display region, to a right side of the left-side image;

a left-edge processing section that attaches a left-edge image, which is displayed within a prescribed range from a left edge of the display region, to a left side of the right-side image; and

an output section that outputs the left-side image with the right-edge image attached thereto to a left eye of a user and outputs the right-side image with the left-edge image attached thereto to a right eye of the user.

2. The image processing apparatus according to claim 1, wherein

the generating section generates the left-side image and the right-side image based on the same two-dimensional image.

3. The image processing apparatus according to claim 1, wherein

the output section includes a display section that has a plurality of pixels arranged in a two-dimensional matrix and that includes the display region in which pixel lines that display an image to the left eye of the user and pixel lines that display an image to the right eye of the user are arranged in an alternating manner, and

the generating section generates the left-side image from pixels of the two-dimensional image corresponding to the pixel lines in the display region that display the image to the left eye, and generates the right-side image from pixels of the two-dimensional image corresponding to the pixel lines in the display region that display the image to the right eye.

4. The image processing apparatus according to claim 1, wherein

the generating section generates the left-side image and the right-side image by shifting the two-dimensional image to the left and right within the display region by a distance no greater than a distance between pupils of the user.

5. The image processing apparatus according to claim 4, wherein

the generating section generates the left-side image and the right-side image by shifting the two-dimensional image to the left and right by an amount whereby line of sight of the left eye is shifted left by no more than three degrees from a parallel state and line of sight of the right eye is shifted right by no more than three degrees from the parallel state.

6. The image processing apparatus according to claim 4, wherein

the generating section generates the left-side image and the right-side image by shifting the two-dimensional image to the left and right within the display region by a distance greater than 0 and less than or equal to 90 mm.

7. The image processing apparatus according to claim 1, wherein

the right-edge processing section attaches to the right side of the left-side image, as the right-edge image, a portion displayed within a prescribed range from the right edge of the display region in the right-side image, and

the left-edge processing section attaches to the left side of the right-side image, as the left-edge image, a portion displayed within a prescribed range from the left edge of the display region in the left-side image.

8. The image processing apparatus according to claim 1, wherein

the right-edge processing section attaches to the right side of the left-side image, as the right-edge image, an image obtained by blurring a portion displayed within a prescribed range from the right edge of the display region in the right-side image, and

the left-edge processing section attaches to the left side of the right-side image, as the left-edge image, an image obtained by blurring a portion displayed within a prescribed range from the left edge of the display region in the left-side image.

9. The image processing apparatus according to claim 1, wherein

the right-edge processing section generates an image that includes the left-side image and the right-edge image by expanding the left-side image to the right, and

the left-edge processing section generates an image that includes the right-side image and the left-edge image by expanding the right-side image to the left.

10. The image processing apparatus according to claim 9, wherein

the right-edge processing section generates an image that includes the left-side image and the right-edge image by expanding the left-side image to the right using a larger expansion rate closer to the left and right edges, and

the left-edge processing section generates an image that includes the right-side image and the left-edge image by expanding the right-side image to the left using a larger expansion rate closer to the left and right edges.

11. The image processing apparatus according to claim 1, wherein

the right-edge processing section attaches to the right side of the left-side image, as the right-edge image, an image obtained by repeating, toward the right, a portion of the left-side image displayed within a prescribed range from the right edge of the display region, and

the left-edge processing section attaches to the left side of the right-side image, as the left-edge image, an image obtained by repeating, toward the left, a portion of the right-side image displayed within a prescribed range from the left edge of the display region.

12. The image processing apparatus according to claim 1, wherein

the right-edge processing section selects, according to characteristics of a portion displayed in a prescribed range from the right edge of the display region in the right-side image, whether to generate the right-edge image based on an image of this displayed portion or to generate the right-edge image with predetermined display conditions, and

the left-edge processing section selects, according to characteristics of a portion displayed in a prescribed range from the left edge of the display region in the left-side image, whether to generate the left-edge image based on an image of this displayed portion or to generate the left-edge image with predetermined display conditions.

13. The image processing apparatus according to claim 12, wherein

when a spatial frequency of the portion displayed in the prescribed range from the right edge of the display region in the right-side image is greater than or equal to the reference value, the right-edge processing section generates the right-edge image based on the image of this displayed portion,

when the spatial frequency of the portion displayed in the prescribed range from the right edge of the display region in the right-side image is less than a reference value, the right-edge processing section generates the right-edge image with the predetermined display characteristics,

when the spatial frequency of the portion displayed in the prescribed range from the left edge of the display region in the left-side image is greater than or equal to a reference value, the left-edge processing section generates the left-edge image based on the image of this displayed portion, and

when the spatial frequency of the portion displayed in the prescribed range from the left edge of the display region in the left-side image is less than the reference value, the left-edge processing section generates the left-edge image with the predetermined display characteristics.

14. The image processing apparatus according to claim 12, wherein

when an average color of the portion displayed in the prescribed range from the right edge of the display region in the right-side image is outside a reference range, the right-edge processing section generates the right-edge image based on the image of this displayed portion,

when the average color of the portion displayed in the prescribed range from the right edge of the display region in the right-side image is within the reference range, the right-edge processing section generates the right-edge image with the predetermined display characteristics

when the average color of the portion displayed in the prescribed range from the left edge of the display region in the left-side image is outside the reference range, the left-edge processing section generates the left-edge image based on the image of this displayed portion, and

when the average color of the portion displayed in the prescribed range from the left edge of the display region in the left-side image is within the reference range, the left-edge processing section generates the left-edge image with the predetermined display characteristics.

15. The image processing apparatus according to claim 9, wherein

the image processing apparatus can stereoscopically display a series of two-dimensional images included in a moving image,

when a rate of change over time of a portion displayed in a prescribed range from the right edge of the display region in the right-side image is greater than or equal to a reference value, the right-edge processing section generates the right-edge image based on an image of this displayed portion,

when the rate of change over time of the portion displayed in the prescribed range from the right edge of the display region in the right-side image is less than the reference value, the right-edge processing section generates the right-edge image with predetermined display characteristics,

when the rate of change over time of a portion displayed in a prescribed range from the left edge of the display region in the left-side image is greater than or equal to a reference value, the left-edge processing section generates the left-edge image based on an image of this displayed portion, and

when the rate of change over time of the portion displayed in the prescribed range from the left edge of the display region in the left-side image is less than the reference value, the left-edge processing section generates the left-edge image with predetermined display characteristics.

16. The image processing apparatus according to claim 1, further comprising a sight line measuring section that measures a direction of a line of sight of the user, wherein

when the direction of the line of sight is to the right of a reference range including a center of the display region, the right-edge processing section generates the right-edge image based on an image of a portion displayed in a prescribed range from the right edge of the display region,

when the direction of the line of sight is not to the right of the reference range including the center of the display region, the right-edge processing section generates the right-edge image with predetermined display conditions,

when the direction of the line of sight is to the left of the reference range including the center of the display region, the left-edge processing section generates the left-edge image based on an image of a portion displayed in a prescribed range from the left edge of the display region, and

when the direction of the line of sight is not to the left of the reference range including the center of the display region, the left-edge processing section generates the left-edge image with predetermined display conditions.

17. The image processing apparatus according to claim 1, further comprising a setting color storage section that stores a color to be set for the right-edge image and the left-edge image, wherein

the right-edge processing section and the left-edge processing section respectively set colors of the right-edge image and the left-edge image to be the color stored in the setting color storage section.

18. The image processing apparatus according to claim 1, wherein

the right-edge processing section sets a color of the right-edge image to be an average color of a portion displayed in a prescribed range from the right edge of the display region in the right-side image, and

the left-edge processing section sets a color of the left-edge image to be an average color of a portion displayed in a prescribed range from the left edge of the display region in the left-side image.

19. The image processing apparatus according to claim 1, wherein

the right-edge processing section sets brightness of the right-edge image to be an average brightness of a portion displayed in a prescribed range from the right edge of the display region in the right-side image, and

the left-edge processing section sets brightness of the left-edge image to be an average brightness of a portion displayed in a prescribed range from the left edge of the display region in the left-side image.

20. The image processing apparatus according to claim 1, wherein

the right-edge processing section deletes a prescribed range on the right side of the right-side image, instead of attaching the right-edge image to the right side of the left-side image, and

the left-edge processing section deletes a prescribed range on the left side of the left-side image, instead of attaching the left-edge image to the left side of the right-side image.

21. The image processing apparatus according to claim 1, wherein

the generating section generates the left-side image and the right-side image by expanding the two-dimensional image to the left and right by prescribed distances and shifting the expanded two-dimensional image to the left and right by prescribed distances in the display region,

the right-edge processing section deletes a prescribed range on the right side of the right-side image, instead of attaching the right-edge image to the right side of the left-side image, and

the left-edge processing section deletes a prescribed range on the left side of the left-side image, instead of attaching the left-edge image to the left side of the right-side image.

22. The image processing apparatus according to claim 1, wherein

the generating section generates the left-side image and the right-side image when the display region has a width expressed by Expression 1

Expression 1 is defined as W≧

(3×

L)+(2×

D×

sin(θ

/2)),

W represents horizontal width of the display region,

D represents a distance from a viewpoint of the user to a display screen,

L represents a distance between pupils of the user, and

θ

represents an angle of an effective field of vision of the user.

23. A recording medium storing thereon a program that causes a computer to function as the image processing apparatus of claim 1.

24. An image processing method for stereoscopically displaying a two-dimensional image, comprising:

generating a left-side image and a right-side image by shifting the two-dimensional image left and right in a display region;

attaching a right-edge image, which is displayed within a prescribed range from a right edge of the display region, to a right side of the left-side image;

attaching a left-edge image, which is displayed within a prescribed range from a left edge of the display region, to a left side of the right-side image; and

outputting the left-side image with the right-edge image attached thereto to a left eye of a user and outputting the right-side image with the left-edge image attached thereto to a right eye of the user.

25. A recording method for recording an image that enables a two-dimensional image to be displayed stereoscopically, comprising:

generating image data of a left-side image and image data of a right-side image obtained by shifting the two-dimensional image left and right in a display region;

attaching a right-edge image, which is displayed within a prescribed range from a right edge of the display region, to a right side of the left-side image;

attaching a left-edge image, which is displayed within a prescribed range from a left edge of the display region, to a left side of the right-side image;

recording image data of the left-side image with the right-edge image attached thereto in a left-side image data storage region in a recording medium, the left-side image data storage region holding an image to be read by a reproducing apparatus and output to a left eye of a user; and

recording image data of the right-side image with the left-edge image attached thereto in a right-side image data storage region in the recording medium, the right-side image data storage region holding an image to be read by the reproducing apparatus and output to a right eye of the user.

26. A recording medium storing thereon an image to be read by a reproducing apparatus and displayed stereoscopically, the recording medium comprising:

a left-side image data storage region holding image data of a left-side image, from among the left-side image and a right-side image generated by shifting a two-dimensional image left and right within a display region, that has attached to a right side thereof a right-edge image displayed within a prescribed range from a right side of the display region and that is read by the reproducing apparatus and output to a left eye of a user; and

a right-side image data storage region holding image data of the right-side image, from among the left-side image and the right-side image generated by shifting the two-dimensional image left and right within the display region, that has attached to a left side thereof a left-edge image displayed within a prescribed range from a left side of the display region and that is read by the reproducing apparatus and output to a right eye of the user.

27. An image processing apparatus that generates a display image from a two-dimensional image, comprising a display image generating section that generates the display image to be displayed within a frame provided on a border thereof, the display image obtained by shifting the two-dimensional image left and right by an amount irrespective of objects displayed in the two-dimensional image.

28. The image processing apparatus according to claim 27, further comprising:

a frame section that is provided on at least a portion of the border of the display image; and

a display image displaying section that displays the display image in a display region within the frame.

29. The image processing apparatus according to claim 28, wherein

the display image generating section generates the display image by shifting the two-dimensional image left and right by a predetermined amount according to position within the two-dimensional image.

30. The image processing apparatus according to claim 28, wherein

the display image generating section generates the display image to include a right-side image and a left-side image obtained by shifting the two-dimensional image left and right by a predetermined amount.

31. The image processing apparatus according to claim 28, wherein

the display image generating section generates a left-side image and a right-side image by shifting the two-dimensional image left and right, and

the display image displaying section displays the left-side image to a left eye of a user and displays the right-side image to a right eye of the user.

32. The image processing apparatus according to claim 28, wherein the frame section includes:

a frame image generating section that generates a frame image to be displayed as the frame; and

a frame image displaying section that displays the frame image in a region corresponding to the frame.

33. The image processing apparatus according to claim 32, wherein

the frame image generating section shifts the frame image left and right by an amount smaller than the amount by which the display image is shifted.

34. The image processing apparatus according to claim 32, wherein

the frame image generating section generates the frame image with a shift amount of 0.

35. The image processing apparatus according to claim 32, wherein

the frame image displaying section displays the frame image when the image processing apparatus displays the display image, and does not display the frame image when the image processing apparatus does not display the two-dimensional image stereoscopically.

36. The image processing apparatus according to claim 32, wherein

the display image displaying section and the frame image displaying section display the display image and the frame image in a right-eye display region and a left-eye display region of a head-mounted display, and

the frame image generating section generates the frame image to be shifted left and right by an amount resulting in the frame image appearing closer to a viewpoint of a user than the right-eye display region and the left-eye display region of the head-mounted display.

37. The image processing apparatus according to claim 28, wherein

the frame section covers at least a portion of a display region in which a right-side image and a left-side image, obtained by shifting the two-dimensional image left and right, do not overlap.

38. The image processing apparatus according to claim 37, wherein

the frame section covers at least a portion of the display region in which the right-side image and the left-side image do not overlap when the image processing apparatus displays the display image, and does not cover the display region in which the right-side image and the left-side image do not overlap when the image processing apparatus does not display the two-dimensional image stereoscopically.

39. The image processing apparatus according to claim 28, wherein

the display image displaying section displays the display image in a right-eye display region and a left-eye display region of a head-mounted display, and

the frame section is provided closer to a viewpoint of the user than to the right-eye display region and the left-eye display region of the head-mounted display.

40. The image processing apparatus according to claim 28, wherein

the frame section includes only a right-side portion and a left-side portion.

42. A recording medium storing thereon a program that causes a computer to function as the image processing apparatus of claim 27.

41. An image processing apparatus that generates a display image from a two-dimensional image, comprising:

a frame image generating section that that generates a frame image to be displayed as at least a portion of a frame provided on a border of the display image; and

a display image generating section that generates the display image by shifting the two-dimensional image left and right within a display region within the frame by an amount irrespective of objects displayed in the two-dimensional image.

43. An image processing method for generating a display image from a two-dimensional image, the method comprising generating a display image to be displayed within a frame provided on a border thereof, the display image obtained by shifting the two-dimensional image left and right by an amount irrespective of objects displayed in the two-dimensional image.