Image processing device, image processing method, and program for display angle determination

• US 9,307,150 B2
• Filed: 02/15/2013
• Issued: 04/05/2016
• Est. Priority Date: 02/28/2012
• Status: Active Grant

• Current Assignee: Canon Kabushiki Kaisha (Canon Inc.)
• Original Assignee: Canon Kabushiki Kaisha (Canon Inc.)
• Inventors: Kaneko, Chiaki
• Primary Examiner(s): SPINKS, ANTOINETTE T
• Application Number:
  US13/768,317
• Publication Number:
  US 20130222656A1
• Time in Patent Office: 1,145 Days
• Field of Search: 348/262, 348/333.01, 348/333.05
• US Class Current: 1/1
• CPC Class Codes: H04N 23/631 Graphical user interfaces [GUI] specially adapted for controlling image capture or setting capture parameters
• CPC Class Codes: H04N 23/90 Arrangement of cameras or camera modules, e.g. multiple cameras in TV studios or sports stadiums
• CPC Class Codes: H04N 5/268 Signal distribution or switching

Abstract

In a case of a camera array, the arrangement of image capturing units and the arrangement of captured images that are displayed do not agree with each other depending on the orientation of the camera at the time of image capturing and it is hard to grasp the correspondence relationship between both. An image processing device for processing a plurality of images represented by captured image data obtained by a camera array image capturing device including a plurality of image capturing units includes a determining unit configured to determine, on the basis of a display angle of the images in a display region, an arrangement of each image in the display region corresponding to each of the plurality of image capturing units, and the arrangement of each image in the display region is determined based on the arrangement of the plurality of image capturing units.

Claims

1. An image processing device comprising:

a first acquiring unit configured to acquire image data captured by an image capturing device, the image capturing device capturing a plurality of images corresponding to different viewpoints;

a second acquiring unit configured to acquire rotation information representing a degree of rotation of a plurality of images in displaying the plurality of images of the captured image data or a degree of inclination of the image capturing device in capturing the plurality of images;

a display unit configured to display the plurality of images of the captured image data in a side-by-side arrangement; and

a determining unit configured to determine, on the basis of the rotation information, an orientation and an arrangement of the plurality of images in displaying the plurality of images of the captured image data in the side-by-side arrangement,

wherein an arrangement of image capturing units agrees with the arrangement of captured images that are displayed.

2. The image processing device according to claim 1, wherein the determining unit determines a relationship between the orientation and the arrangement of the plurality of displayed images in a case of displaying the plurality of images in a side-by-side arrangement that is consistent with a relationship between a capturing orientation of the plurality of captured images and each viewpoint corresponding to the plurality of images.

3. The image processing device according to claim 1, wherein

the second acquiring unit acquires rotation instructions of the plurality of images including the degree of rotation of the plurality of images in displaying the plurality of images, or a degree of inclination of the image capturing device in capturing the plurality of images, the instructions being input by a user;

and

the determining unit determines the orientation and the arrangement of the plurality of images based on the rotation instructions.

4. The image processing device according to claim 1, wherein

the second acquiring unit acquires the degree of inclination of the image capturing device at the time of capturing the plurality of images;

and

the determining unit determines the orientation and the arrangement of the plurality of images based on the degree of inclination.

5. The image processing device according to claim 4, wherein the determining unit selects, from a plurality of angles set at predetermined intervals, an angle closest to an angle indicating the degree of inclination, and determines the orientation and the arrangement of the plurality of images based on the selected angle.

6. The image processing device according to claim 4, wherein

the image capturing device includes an acceleration sensor;

and

the second acquiring unit acquires the degree of inclination of the image capturing device based on an output from the acceleration sensor.

7. The image processing device according to claim 1, wherein

the second acquiring unit further acquires information about a positional relationship among a plurality of viewpoints corresponding to the plurality of images;

and

the determining unit determines the orientation and the arrangement of the plurality of images based further on the acquired positional relationship.

8. The image processing device according to claim 1, further comprising:

a third acquiring unit configured to acquire user instructions to select some of the plurality of images, the instructions being given to display the plurality of images aligned in an orientation and an arrangement determined by the determining unit; and

a combining unit configured to generate a combined image by combining images selected by the user instructions acquired by the third acquiring unit.

9. The image processing device according to claim 8, further comprising a generating unit for generating a display image displaying an image selected among the plurality of images by the user instructions and the combined image.

12. The image processing device according to claim 1, further comprising a display unit for displaying the plurality of images in the orientation and the arrangement determined by the determining unit.

13. The image processing device according to claim 1, wherein the image capturing device is a camera array image capturing device including a plurality of image capturing units.

10. An image processing device comprising:

a first acquiring unit configured to acquire image data captured by an image capturing device, the image capturing device capturing a plurality of images corresponding to different viewpoints;

a second acquiring unit configured to acquire rotation information representing rotation instructions of the plurality of images input by a user, the instructions including a degree of rotation of the plurality of images of the captured image data, or a degree of inclination of the image capturing device in capturing the plurality of images;

a display unit configured to display the plurality of images of the captured image data in a side-by-side arrangement; and

a determining unit configured to determine, on the basis of the rotation information, information indicating an orientation and an arrangement of the plurality of images in a case of displaying the plurality of images in the side-by-side arrangement,

wherein an arrangement of image capturing units agrees with the arrangement of captured images that are displayed.

11. An image processing device comprising:

a first acquiring unit configured to acquire image data captured by an image capturing device, the image capturing device capturing a plurality of images corresponding to different viewpoints;

a second acquiring unit configured to acquire rotation information representing rotation instructions of the plurality of images input by a user, the instructions including a degree of rotation of the plurality of images of the captured image data, or a degree of inclination of the image capturing device in capturing the plurality of images; and

a display unit configured to display the plurality of images of the captured image data in a side-by-side arrangement,

wherein in a case where a degree of rotation of the plurality of images in displaying the plurality of images or a degree of inclination of the image capturing device in capturing the plurality of images is changed, both an orientation and an arrangement of the plurality of images are changed in conjunction therewith in the displaying in the display unit,

wherein an arrangement of image capturing units agrees with the arrangement of captured images that are displayed.

14. An image capturing device having the image processing device, the image processing device comprising:

a first acquiring unit configured to acquire image data captured by an image capturing device, the image capturing device capturing a plurality of images corresponding to different viewpoints;

a second acquiring unit configured to acquire rotation information representing a degree of rotation of a plurality of images in displaying the plurality of images of the captured image data or a degree of inclination of the image capturing device in capturing the plurality of images;

a display unit configured to display the plurality of images of the captured image data in a side-by-side arrangement;

a determining unit configured to determine, on the basis of the rotation information, an orientation and an arrangement of the plurality of images in displaying the plurality of images of the captured image data in the side-by-side arrangement,

wherein the arrangement of image capturing units agrees with the arrangement of captured images that are displayed, and

wherein the image capturing device simultaneously captures a plurality of images corresponding to different viewpoints.

15. An image processing method comprising the steps of:

acquiring image data captured by an image capturing device, the image capturing device capturing a plurality of images corresponding to different viewpoints;

acquiring rotation information representing a degree of rotation of a plurality of images in displaying the plurality of images of the captured image data or a degree of inclination of the image capturing device in capturing the plurality of images;

displaying the plurality of images of the captured image data in a side-by-side arrangement; and

determining, on the basis of the rotation information, an orientation and an arrangement of the plurality of images in displaying the plurality of images of the captured image data in the side-by-side arrangement,

wherein an arrangement of image capturing units agrees with the arrangement of captured images that are displayed.

16. An image processing method comprising the steps of:

acquiring image data captured by an image capturing device, the image capturing device capturing a plurality of images corresponding to different viewpoints;

acquiring rotation information representing rotation instructions of the plurality of images input by a user, the instructions including a degree of rotation of the plurality of images of the captured image data, or a degree of inclination of the image capturing device in capturing the plurality of images;

displaying the plurality of images of the captured image data in a side-by-side arrangement; and

determining, on the basis of the rotation information, information indicating an orientation and an arrangement of the plurality of images in arranging the plurality of images in a case of displaying the plurality of images in the side-by-side arrangement,

wherein an arrangement of image capturing units agrees with the arrangement of captured images that are displayed.

17. An image processing method comprising the steps of:

acquiring image data captured by an image capturing device, the image capturing device capturing a plurality of images corresponding to different viewpoints;

acquiring rotation information representing rotation instructions of the plurality of images input by a user, the instructions including a degree of rotation of the plurality of images of the captured image data, or a degree of inclination of the image capturing device in capturing the plurality of images; and

displaying the plurality of images of the captured image data in a side-by-side arrangement, wherein in a case where a degree of rotation of the plurality of images in displaying the plurality of images or a degree of inclination of the image capturing device in capturing the plurality of images is changed, both an orientation and an arrangement of the plurality of images are changed in conjunction therewith in the displaying in the display unit,

wherein an arrangement of image capturing units agrees with the arrangement of captured images that are displayed.

18. A program stored in a non-transitory computer readable storage medium for causing a computer to perform the image processing method, the method comprising the steps of:

acquiring image data captured by an image capturing device, the image capturing device capturing a plurality of images corresponding to different viewpoints;

acquiring rotation information representing a degree of rotation of a plurality of images in displaying the plurality of images of the captured image data or a degree of inclination of the image capturing device in capturing the plurality of images;

displaying the plurality of images of the captured image data in a side-by-side arrangement; and

determining, on the basis of the rotation information, an orientation and an arrangement of the plurality of images in displaying the plurality of images of the captured image data in the side-by-side arrangement,

wherein the arrangement of image capturing units agrees with the arrangement of captured images that are displayed.