IMAGE DISPLAY APPARATUS, IMAGE DISPLAY METHOD, MOBILE APPARATUS, IMAGE DISPLAY SYSTEM, AND COMPUTER PROGRAM

• US 20150293362A1
• Filed: 10/02/2013
• Published: 10/15/2015
• Est. Priority Date: 11/13/2012
• Status: Active Grant

• Current Assignee: Sony Corporation (Sony Group Corp.)
• Original Assignee: Sony Corporation (Sony Group Corp.)
• Inventors: Takahashi, Naomasa, Narahara, Tatsuya, Sako, Yoichiro, Takai, Motoyuki
• Granted Patent: US 10,108,018 B2
• Time in Patent Office: Days
• US Class Current: 1/1
• CPC Class Codes: B64U 20/87 Mounting of imaging devices, e.g. mounting of gimbals
• CPC Class Codes: B64U 2101/30 for imaging, photography or videography
• CPC Class Codes: G02B 2027/0138 comprising image capture systems, e.g. camera
• CPC Class Codes: G02B 2027/0141 characterised by the informative content of the display
• CPC Class Codes: G02B 27/017 Head mounted
• CPC Class Codes: G02B 27/02 Viewing or reading apparatus stereoscopic systems G02B30/00
• CPC Class Codes: G06F 3/147 using display panels
• CPC Class Codes: G09G 2340/045 Zooming at least part of an image, i.e. enlarging it or shrinking it
• CPC Class Codes: G09G 2340/0464 Positioning
• CPC Class Codes: G09G 2340/12 Overlay of images, i.e. displayed pixel being the result of switching between the corresponding input pixels
• CPC Class Codes: G09G 2340/14 Solving problems related to the presentation of information to be displayed
• CPC Class Codes: G09G 2354/00 Aspects of interface with display user
• CPC Class Codes: G09G 3/003 to produce spatial visual effects
• CPC Class Codes: G09G 5/00 Control arrangements or circuits for visual indicators common to cathode-ray tube indicators and other visual indicators image data processing or generation, in general G06T
• CPC Class Codes: G09G 5/14 Display of multiple viewports
• CPC Class Codes: H04N 13/239 using two 2D image sensors having a relative position equal to or related to the interocular distance H04N13/243 takes precedence
• CPC Class Codes: H04N 13/296 Synchronisation thereof; Control thereof
• CPC Class Codes: H04N 13/344 with head-mounted left-right displays
• CPC Class Codes: H04N 13/366 using viewer tracking

Abstract

A first-person viewpoint image captured by a camera mounted on a radio-controlled model is displayed on a head mount display and is enjoyed.

When a whole-sky image captured by a camera unit 605 mounted on a model apparatus 120 is reproduced and displayed with a head mount display 110, a CPU 601 moves the display field so as to offset the motion of the head of a user detected by a posture/location detecting unit 604. In this manner, an image that follows the motion of the head of the user is displayed. Meanwhile, on the side of the head mount display 110, a combined image generated by superimposing an AR image as virtual image information on the actual image in the display field is displayed.

Claims

1. An image display apparatus comprising:

a display unit mounted on a head of a user;

a posture detecting unit configured to detect posture of the head; and

a display control unit configured to control display of an image on the display unit based on the posture of the head, the image being captured by a mobile apparatus.

2. The image display apparatus according to claim 1, wherein the display control unit clips a region corresponding to the posture of the head from a wide-angle image captured by the mobile apparatus, and displays the clipped region on the display unit.

3. The image display apparatus according to claim 1, wherein, when displaying a plurality of viewpoint images captured at a plurality of viewpoints by a deep focus parallel method, the display control unit adjusts a convergence point between the viewpoint images based on a movement velocity of the moving object.

4. The image display apparatus according to claim 1, wherein the display control unit displays an image with an inter-viewpoint distance corresponding to a zooming operation on the display unit.

5. The image display apparatus according to claim 1, wherein a direction of a line of sight of a camera unit of the mobile apparatus is offset in at least one of directions of panning, tilting, and rolling with respect to the posture of the head.

6. The image display apparatus according to claim 1, further comprising

an operational feeling feedback unit configured to feed back an operational feeling to the user through tactile sensation or vibration,

wherein the feedback to the user is based on an acceleration to which the mobile apparatus is subjected while moving.

7. The image display apparatus according to claim 1, wherein the display control unit superimposes an AR image on a real-world image captured by the mobile apparatus, before displaying the image.

8. The image display apparatus according to claim 7, wherein the display control unit displays the AR image corresponding to at least one of a current location of the mobile apparatus, an object included in the captured image, and a state of the mobile apparatus.

9. The image display apparatus according to claim 1, wherein the display control unit displays location information about the mobile apparatus and the user.

10. The image display apparatus according to claim 1, wherein the display control unit further displays an image captured by an automatic tracker that captures the image while tracking the mobile apparatus.

11. The image display apparatus according to claim 1, further comprising

a self-sight-line image acquiring unit configured to acquire a self-sight-line image to be seen on a line of sight of the user,

wherein the display control unit switches the image being displayed between a moving-object sight-line image captured by the mobile apparatus and the self-sight-line image.

12. The image display apparatus according to claim 1, wherein the display control unit switches the image being displayed between images captured from a plurality of viewpoint positions of the mobile apparatus in accordance with the posture of the head.

13. The image display apparatus according to claim 1, wherein the display control unit corrects shaking in a moving image captured by the mobile apparatus, before displaying the moving image.

14. An image display method comprising:

a posture detecting step of detecting posture of ahead of a user; and

a display control step of controlling display of an image based on the posture of the head, the image being captured by a mobile apparatus.

15. An image display system comprising:

a mobile apparatus configured to capture an image while moving; and

an image display apparatus configured to display the image captured by the mobile apparatus in accordance with posture of a head of a user.

16. A mobile apparatus comprising:

a camera unit;

a camera platform configured to control a direction of a line of sight of the camera unit;

a moving unit configured to move the apparatus; and

a communication unit configured to communicate data including an image captured by the camera unit,

wherein the camera unit includes a plurality of cameras configured to perform imaging by a deep focus parallel method, the cameras having different viewpoints.

17. The mobile apparatus according to claim 16, which captures a whole-sky image while changing an inter-viewpoint distance between the cameras with different viewpoints.

18. The mobile apparatus according to claim 16, which extrapolates images outside the viewpoints of the cameras from images captured by the cameras with different viewpoints at a fixed inter-viewpoint distance from each other.

19. A computer program written in a computer-readable format,

the computer program causing a computer to function as:

a posture detecting unit configured to detect posture of a head of a user; and

a display control unit configured to control display of an image based on the posture of the head, the image being captured by a mobile apparatus.