Multi-viewpoint image generation apparatus, multi-viewpoint image generation method, and multi-viewpoint image generation program

• US 7,633,528 B2
• Filed: 09/25/2006
• Issued: 12/15/2009
• Est. Priority Date: 09/29/2005
• Status: Expired due to Fees

• Current Assignee: Kabushiki Kaisha Toshiba (Toshiba Corporation)
• Original Assignee: Kabushiki Kaisha Toshiba (Toshiba Corporation)
• Inventors: Saishu, Tatsuo, Hirayama, Yuzo, Fukushima, Rieko, Numazaki, Shunichi
• Primary Examiner(s): VU, NGOC YEN T
• Application Number:
  US11/525,817
• Publication Number:
  US 20070121182A1
• Time in Patent Office: 1,177 Days
• Field of Search: None
• US Class Current: 348/211.8
• CPC Class Codes: G02B 30/27 involving lenticular arrays
• CPC Class Codes: G02B 30/30 involving parallax barriers
• CPC Class Codes: G03B 21/625 Lenticular translucent screens
• CPC Class Codes: G03B 35/08 by simultaneous recording
• CPC Class Codes: H04N 13/305 using lenticular lenses, e.g. arrangements of cylindrical lenses
• CPC Class Codes: H04N 13/398 Synchronisation thereof; Control thereof

Abstract

A multi-viewpoint image generation apparatus includes: a generation screen display unit that controls a two-dimensional image display device to display a two-dimensional image indicating an arrangement of the object in a multi-viewpoint image to be displayed on a the three-dimensional image display device; a modification command acquisition unit that acquires a modification command; a camera control unit that modifies a camera condition including a number of cameras corresponding to the multi-viewpoint image, a placement position of each of the cameras, a size of the projection plane of each of the cameras, a point of regard, and a projection method, based on the modification command acquired by the modification command acquisition unit; and a multi-viewpoint image generation unit that generates the multi-viewpoint image acquired by the cameras being configured according to the camera condition modified by the camera control unit.

Claims

1. A multi-viewpoint image generation apparatus for generating a multi-viewpoint image for displaying an object on a three-dimensional image display device, the apparatus comprising:

a generation screen display unit that controls a two-dimensional image display device to display a two-dimensional image indicating an arrangement of the object in a three-dimensional image displayed on a the three-dimensional image display device;

a modification command acquisition unit that acquires a modification command for modifying an arrangement of the object in the two-dimensional image;

a camera control unit that modifies a camera condition including a number of cameras corresponding to the multi-viewpoint image, a placement position of each of the cameras, a size of the projection plane of each of the cameras, a point of regard, and a projection method, based on the modification command acquired by the modification command acquisition unit; and

a multi-viewpoint image generation unit that generates the multi-viewpoint image acquired by the cameras being configured according to the camera condition modified by the camera control unit.

2. The apparatus according to claim 1, wherein the camera control unit modifies a distance from the point of regard to the camera as the placement position of the camera.

3. The apparatus according to claim 1, wherein the camera control unit modifies the interval between the cameras as the placement position of the camera.

4. The apparatus according to claim 1, wherein the generation screen display unit controls the two-dimensional image display device to display the object and a boundary box indicating a three-dimensionally displayable area on the three-dimensional image display device as the two-dimensional image,

wherein the modification command acquisition unit acquires the modification command for modifying a relative arrangement of the object and the boundary box, and

wherein the camera control unit modifies the camera condition further including a size and a number of the point of regard and the projection plane, based on the modification command acquired by the modification command acquisition unit.

5. The apparatus according to claim 4, wherein the generation screen display unit controls the two-dimensional image display devce to display the object and the boundary box placed in the relative arrangement indicated in the modification command, when the modification command acquisition unit acquires the modification command.

6. The apparatus according to claim 5, further comprising a boundary box processing unit that modifies the arrangement of the boundary box based on the position of the object displayed on the two-dimensional image display device based on the modification command when the modification command acquisition unit acquires the modification command,

wherein the generation screen display unit controls the two-dimensional image display device to display the object and the boundary box in the arrangement modified by the boundary box processing unit.

7. The apparatus according to claim 1, further comprising a preview screen display unit that controls the two-dimensional image display device to display an image of the object captured as perspective projection from a view camera placed at an observation reference position assumed to be a position of an observer and directed to the point of regard.

8. The apparatus according to claim 4, wherein the boundary box has the display limit in a near-side direction of the three-dimensional image display device and the display limit in a far-side direction as boundary surfaces.

9. The apparatus according to claim 4, wherein the boundary box includes the point of regard.

10. The apparatus according to claim 4, wherein the boundary box has boundary surfaces of capturing ranges of two cameras placed at both ends of the cameras.

11. The apparatus according to claim 4, wherein the modification command acquisition unit acquires a shape modification command for modifying a shape of the object to be displayed on the three-dimensional image display device, and

wherein the camera control unit modifies at least one of the size of the projection plane and the placement position of each of the cameras, based on the shape of the object indicated in the shape modification command.

12. The apparatus according to claim 4, wherein the modification command acquisition unit acquires a placement position modification command for modifying a placement position of the object, and

wherein the camera control unit translates both of the point of regard and the placement position of each of the cameras, based on the placement position of the object indicated in the placement position modification command.

13. The apparatus according to claim 4, wherein the modification command acquisition unit acquires a placement angle modification command for modifying a placement angle of the object relative to a screen of the three-dimensional image display device, and

wherein the camera control unit rotates each of the cameras with the point of regard included in the boundary box as the reference, based on the rotation angle indicated in the placement angle modification command.

14. The apparatus according to claim 4, wherein the generation screen display unit controls the two-dimensional image display device to display the two-dimensional image including a camera center vector that indicates a direction of a center position of the placement positions of the cameras,

wherein the modification command acquisition unit acquires a camera center vector modification command for modifying the direction of the camera center vector, and

wherein the camera control unit modifies the placement position of each of the cameras based on the direction of the camera center vector indicated in the camera center vector modification command, while maintaining the point of regard.

15. The apparatus according to claim 4, wherein the modification command acquisition unit acquires a screen size modification command for modifying a screen size of the three-dimensional image display device, and

wherein the camera control unit modifies the size of the projection plane of each of the cameras based on the screen size indicated in the screen size modification command acquired by the modification command acquisition unit.

16. The apparatus according to claim 4, wherein the modification command acquisition unit acquires a display system modification command for modifying a display system of the three-dimensional image display device, and

wherein the camera control unit modifies at least one of the number of cameras, the placement position of each of the cameras, the point of regard, and the projection method, based on the display system indicated in the display system modification command acquired by the modification command acquisition unit.

17. The apparatus according to claim 4, wherein the modification command acquisition unit acquires a projection method modification command for modifying the projection method, and

wherein the camera control unit modifies at least one of the projection method of each of the cameras, the placement position, the size of the projection plane, and the point of regard, based on the projection method indicated in the projection method modification command acquired by the modification command acquisition unit.

18. The apparatus according to claim 4, wherein the modification command acquisition unit acquires an object spatial frequency modification command for modifying a spatial frequency used for drawing the two-dimensional image, and

wherein the camera control unit modifies the two-dimensional image based on the spatial frequency indicated in the object spatial frequency modification command acquired by the modification command acquisition unit.

19. The apparatus according to claim 4, wherein the modification command acquisition unit acquires an observation reference viewing distance modification command for modifying an observation reference viewing distance between the object and an observation reference position of the three-dimensional image, and

wherein the camera control unit modifies at least one of the number of cameras and the placement position of each of the cameras, based on the observation reference viewing distance indicated in the observation reference viewing distance modification command acquired by the modification command acquisition unit.

20. The apparatus according to claim 4, wherein the modification command acquisition unit acquires a perspective degree modification command for modifying a perspective degree in the three-dimensional image, and

wherein the camera control unit modifies the placement position of each of the cameras, based on the perspective degree indicated in the perspective degree modification command acquired by the modification command acquisition unit.

21. The apparatus according to claim 4, further comprising:

a reproduce distortion removal application command acquisition unit that acquires a reproduce distortion removal application command for performing removal processing of reproduce distortion to be occurred in the three-dimensional image; and

a mesh deformation unit that performs mesh deformation of the object when the reproduce distortion removal application command acquisition unit acquires the reproduce distortion removal application command,

wherein the multi-viewpoint image generation unit generates the multi-viewpoint image of the object being subjected to the mesh deformation by the mesh deformation unit.

22. The apparatus according to claim 21, wherein the generation screen display unit controls the two-dimensional image display device to display the two-dimensional image indicating the arrangement of the object, before being subjected to the mesh deformation, in the multi-viewpoint image, when the reproduce distortion removal application command acquisition unit acquires the reproduce distortion removal application command.

23. The apparatus according to claim 21, further comprising:

an inverse mesh deformation unit that performs inverse mesh deformation of inverse processing to the mesh deformation for the object deformed by the mesh deformation unit; and

a preview image display unit that controls the two-dimensional image display device to display a three-dimensional shape image of the object after subjected to the inverse mesh deformation by the inverse mesh deformation unit, the object captured as perspective projection from a view camera placed at the observation reference position that is assumed to be a position of an observer and directed to the point of regard on the two-dimensional image display screen, when the object is subjected to the mesh deformation.

24. The apparatus according to claim 1, further comprising a camera resolution modification command acquisition unit that acquires a camera resolution modification command for modifying a resolution of the camera,

wherein the multi-viewpoint image generation unit generates the multi-viewpoint image in the camera resolution indicated in the camera resolution modification command acquired by the camera resolution modification command acquisition unit.

25. The apparatus according to claim 1, further comprising a camera move three-dimensional image generation command acquisition unit that acquires a camera move three-dimensional image generation command for generating a camera move three-dimensional image being captured while the camera is moved relative to the object,

wherein the camera control unit matches a locus of a view camera placed at an observer reference position of three-dimensional image display device with a locus of the move camera for capturing the camera move three-dimensional image, and matches the point of regard and the projection plane of the cameras with the point of regard and the projection plane of the move camera, when the camera move three-dimensional image generation command acquisition unit acquires the camera move three-dimensional image generation command.

26. The apparatus according to claim 4, further comprising:

a background board installation command acquisition unit that acquires a background board installation command for installing a background board in a far-side direction of the multi-viewpoint image; and

a background board handling unit that installs the background board when the background board installation command acquisition unit acquires the background board installation command,

wherein the generation screen display unit controls the two-dimensional image display device to display the background board installed by the background board handling unit.

27. The apparatus according to claim 1, further comprising a clipping command acquisition unit that acquires a clipping command for performing a clipping processing in a far-side direction or a near-side direction in a screen of the three-dimensional image display device,

wherein the generation screen display unit controls the two-dimensional image display device to display the clipping position indicated in the clipping command acquired by the clipping command acquisition unit.

28. The apparatus according to claim 27, wherein the camera control unit matches the projection plane and the clipping plane of the cameras with a screen and a clipping plane of the three-dimensional image display device, when the clipping command acquisition unit acquires the clipping processing command.

29. A method for generating a multi-viewpoint image for displaying an object on a three-dimensional image display device, the method comprising:

controlling a two-dimensional image display device to display a two-dimensional image indicating an arrangement of the object in the multi-viewpoint image to be displayed on the three-dimensional image display device;

acquiring a modification command for modifying an arrangement of the object in the two-dimensional image;

modifying a camera condition including a number of cameras corresponding to the multi-viewpoint image, a placement position of each of the cameras, a size of the projection plane of each of the cameras, a point of regard, and a projection method, based on the acquired modification command; and

generating the multi-viewpoint image acquired by the cameras being configured according to the modified camera condition.

30. A computer-readable program product for causing a computer to generating a multi-viewpoint image for displaying an object on a three-dimensional image display device, the program product causing the computer to perform procedures comprising:

controlling a two-dimensional image display device to display a two-dimensional image indicating an arrangement of the object in the multi-viewpoint image to be displayed on the three-dimensional image display device;

acquiring a modification command for modifying an arrangement of the object in the two-dimensional image;

modifying a camera condition including a number of cameras corresponding to the multi-viewpoint image, a placement position of each of the cameras, a size of the projection plane of each of the cameras, a point of regard, and a projection method, based on the acquired modification command; and

generating the multi-viewpoint image acquired by the cameras being configured according to the modified camera condition.