DISPLAY APPARATUS, IMAGE PROCESSING APPARATUS AND IMAGE PROCESSING METHOD, IMAGING APPARATUS, AND PROGRAM

• US 20070188493A1
• Filed: 04/19/2007
• Published: 08/16/2007
• Est. Priority Date: 10/04/2002
• Status: Active Grant

• Current Assignee: Sony Corporation (Sony Group Corp.)
• Original Assignee: Sony Corporation (Sony Group Corp.)
• Inventors: Kondo, Tetsujiro, Tanaka, Kenji, Murakami, Tetsuya, Haga, Tsugihiko, Okumura, Yuji
• Granted Patent: US 8,102,395 B2
• Time in Patent Office: Days
• US Class Current: 345/427
• CPC Class Codes: A63F 2300/6661 for changing the position of the virtual camera
• CPC Class Codes: A63F 2300/6669 using a plurality of virtual cameras concurrently or sequentially, e.g. automatically switching between fixed virtual cameras when a character change rooms
• CPC Class Codes: A63F 2300/6676 by dedicated player input
• CPC Class Codes: A63F 2300/69 Involving elements of the real world in the game world, e.g. measurement in live races, real video
• CPC Class Codes: A63F 2300/8023 the game being played by multiple players at a common site, e.g. in an arena, theatre, shopping mall using a large public display
• CPC Class Codes: G03B 21/56 Projection screens
• CPC Class Codes: G03B 37/04 with cameras or projectors providing touching or overlapping fields of view
• CPC Class Codes: G03B 37/06 involving anamorphosis G03B37/02, G03B37/04 take precedence
• CPC Class Codes: G06F 1/1601 Constructional details related to the housing of computer displays, e.g. of CRT monitors, of flat displays constructional details related to flat displays integrated in a portable computer, e.g. laptop, handheld computer G06F1/1637; constructional details related to television receivers H04N5/64
• CPC Class Codes: G06F 3/04815 Interaction with a metaphor-based environment or interaction object displayed as three-dimensional, e.g. changing the user viewpoint with respect to the environment or object
• CPC Class Codes: G06T 15/10 Geometric effects
• CPC Class Codes: G06T 15/205 Image-based rendering
• CPC Class Codes: G09G 2340/0492 Change of orientation of the displayed image, e.g. upside-down, mirrored
• CPC Class Codes: G09G 3/002 to project the image of a two-dimensional display, such as an array of light emitting or modulating elements or a CRT
• CPC Class Codes: G09G 3/003 to produce spatial visual effects
• CPC Class Codes: H04N 21/21805 enabling multiple viewpoints, e.g. using a plurality of cameras
• CPC Class Codes: H04N 23/58 Means for changing the camera field of view without moving the camera body, e.g. nutating or panning of optics or image sensors
• CPC Class Codes: H04N 23/698 for achieving an enlarged field of view, e.g. panoramic image capture
• CPC Class Codes: H04N 5/2627 for providing spin image effect, 3D stop motion effect or temporal freeze effect 2D image animation in general G06T13/80

Abstract

A system and method displays a realistic image that allows a user to readily grasp his/her own positional relationship and also to experience a sense of presence. An outer dome screen is disposed so as to surround a user, and an immersion image such as the scenery of the surroundings of an object is displayed thereon. An inner dome screen is disposed inside the outer dome screen, and it displays a bird'"'"'s-eye image of the object as perceived by the vision of the user when the object is viewed from a viewpoint of the user. In this case, the user is allowed to readily grasp his/her own positional relationship by the bird'"'"'s-eye image and is allowed to experience a sense of presence by the immersion image.

Claims

1. An image processing apparatus for processing images captured from a plurality of positions and generating with a projector a projected image that is projected for display toward an interior display surface having a predetermined 3-dimensional shape, the image processing apparatus comprising:

an image selecting mechanism configured to select a viewpoint image of a predetermined object as viewed from a viewpoint of a user, based on the viewpoint of the user, from images captured respectively from the plurality of positions; and

a viewpoint-image converting mechanism configured to convert the viewpoint image into the projected image based on the viewpoint of the user and the predetermined 3-dimensional shape of the interior display surface.

2. An image processing apparatus according to claim 1, further comprising a detector configured to detect the viewpoint of the user.

3. An image processing apparatus according to claim 1, further comprising a plurality of imaging devices.

4. An image processing apparatus according to claim 1,

wherein:

the image selecting mechanism selects viewpoint images respectively associated with a plurality of viewpoints of the user;

the viewpoint-image converting mechanism is configured to convert the viewpoint images respectively associated with the plurality of viewpoints of the user to generate the projected image; and

the display mechanism is configured to display the projected image so as to display images of a predetermined object as viewed respectively from the plurality of viewpoints of the user.

5. An image processing apparatus according to claim 1, further comprising an integrating mechanism configured to integrate a plurality of viewpoints of the user into an integrated single viewpoint,

wherein the image selecting mechanism is configured to select viewpoint images respectively associated with the plurality of user'"'"'s viewpoints from the images respectively captured by the plurality of imaging devices, and generate the viewpoint image for the single viewpoint from the viewpoint images,

and wherein the viewpoint-image converting mechanism is configured to convert the viewpoint image for the single viewpoint into the projected image.

6. An image processing apparatus according to claim 5,

wherein the image selecting mechanism is configured to calculate a weighted sum of the viewpoint images respectively associated with the plurality of viewpoints of the user in accordance distances of the respective viewpoints of the user and the integrated single viewpoint so as to generate the viewpoint image for the single viewpoint.

7. An image processing apparatus according to claim 5,

wherein the integrating mechanism is configured to integrate the plurality of viewpoints of the user into a position of a barycenter of the plurality of viewpoints of the user.

8. An image processing apparatus according to claim 5, further comprising:

a viewpoint-distance calculator configured to calculate distances among the plurality of viewpoints of the user,

wherein the integrating mechanism is configured to integrate a plurality of viewpoints of the user having viewpoint distances not greater than a predetermined distance into an integrated single viewpoint.

9. An image processing method for processing images captured from a plurality of positions and generating a projected image projected by a projector toward a display surface, the image processing method comprising steps of:

selecting a viewpoint image of an object according to a viewpoint of a user, based on the viewpoint of the user, from the images captured respectively from the plurality of positions; and

converting the viewpoint image into the projected image based on the viewpoint of the user and a shape of the display surface.

10. A program for execution by a computer to perform image processing for processing images captured from a plurality of positions and generating a projected image projected by a projector toward a display surface configured to having an image displayed thereon, the program comprising instructions that cause the computer to implement steps of:

selecting a viewpoint image of an object according to a viewpoint of a user, based on the viewpoint of the user, from the images captured respectively from the plurality of positions; and

converting the viewpoint image into the projected image based on the viewpoint of the user and a shape of the display surface.

11. An imaging apparatus for capturing an image, comprising:

omnidirectional imaging means for obtaining an image surroundings in all directions; and

object-imaging means for obtaining an image of an object from a plurality of directions.

12. An imaging apparatus according to claim 11,

wherein an image obtained by the omnidirectional imaging means includes means for displaying an immersion image for presenting a user with a sense of immersion,

and wherein an image obtained by the object-imaging means includes means for displaying a bird'"'"'s-eye image of the object as viewed from a viewpoint of the user.