Multiple viewpoint imaging control device, multiple viewpoint imaging control method and computer readable medium

• US 8,933,996 B2
• Filed: 04/13/2011
• Issued: 01/13/2015
• Est. Priority Date: 06/30/2010
• Status: Expired due to Fees

• Current Assignee: Fujifilm Corporation (Fujifilm Holdings Corporation)
• Original Assignee: Fujifilm Corporation (Fujifilm Holdings Corporation)
• Inventors: Hashimoto, Takashi
• Primary Examiner(s): Czekaj, Dave
• Assistant Examiner(s): Werner, David N.
• Application Number:
  US13/203,829
• Publication Number:
  US 20120002019A1
• Time in Patent Office: 1,371 Days
• Field of Search: None
• US Class Current: 348/50
• CPC Class Codes: H04N 13/00 Stereoscopic video systems; Multi-view video systems; Details thereof
• CPC Class Codes: H04N 13/10 Processing, recording or transmission of stereoscopic or multi-view image signals
• CPC Class Codes: H04N 13/106 Processing image signals for multi-view video sequence encoding H04N19/597
• CPC Class Codes: H04N 13/117 the virtual viewpoint locations being selected by the viewers or determined by viewer tracking
• CPC Class Codes: H04N 13/20 Image signal generators
• CPC Class Codes: H04N 13/207 using a single 2D image sensor
• CPC Class Codes: H04N 13/211 using temporal multiplexing
• CPC Class Codes: H04N 13/221 using the relative movement between cameras and objects
• CPC Class Codes: H04N 13/279 the virtual viewpoint locations being selected by the viewers or determined by tracking
• CPC Class Codes: H04N 13/282 for generating image signals corresponding to three or more geometrical viewpoints, e.g. multi-view systems
• CPC Class Codes: H04N 13/296 Synchronisation thereof; Control thereof
• CPC Class Codes: H04N 23/695 Control of camera direction for changing a field of view, e.g. pan, tilt or based on tracking of objects

Abstract

When images from multiple viewpoints are required, respective imaging locations of the multiple viewpoints are specified and imaging operations themselves may be executed simply. When image data is to be imaged from plural viewpoints that are elements for generating a three-dimensional form of a specific subject, a user is appropriately guided. Messages prompting ranging directly in front of the specific subject, movement from the ranging position to an imaging start point, and movement from the imaging start point to an imaging end point are displayed in a message display region of an LCD monitor or the like. In addition, from analysis of a through-image, positions of a digital camera are automatically identified and automatic imaging is carried out on the basis of movements of feature points.

Claims

1. A multiple viewpoint imaging control device comprising:

an imaging section that images an image;

a moving image display section that displays on a display screen a moving image for determining an imaging image angle;

a ranging section that performs provisional imaging of a specific subject included in the moving image displayed at the moving image display section, and that measures at least a distance to the specific subject;

an acquisition section that, when imaging is to be performed from a plurality of imaging viewpoints, acquires a convergence angle between the imaging viewpoints;

a viewpoint number setting section that sets a number of viewpoints, including at least an imaging start point at which imaging is initially performed and an imaging end point at which imaging is finally performed;

a position information computation section that computes position information including a movement amount of a viewpoint from the imaging start point at which imaging is initially performed to the imaging end point at which imaging is finally performed, on a basis of data of the convergence angle and data of the distance from a ranging point to the specific subject measured by the ranging section, such that a viewpoint at a position of provisional imaging is at a middle of the movement amount;

a notification section that, on a basis of the position information computed by the position information computation section, executes a notification prompting that a viewpoint be moved from the ranging point to the imaging start point and a notification prompting that a viewpoint be moved from the imaging start point to the imaging end point;

an automatic imaging execution control section that automatically images the subject at an imaging viewpoint;

a judgment section that, in accordance with the movement of a viewpoint, uses the imaging start point as a reference image and judges whether or not the device is at a viewpoint that should be imaged, on a basis of a comparison between the reference image and an image imaged after movement, the automatic imaging execution control section automatically imaging the subject at an imaging viewpoint when it is judged by the judgment section that the device is at the imaging viewpoint; and

a storage section that stores image data imaged by the automatic imaging execution control section as an element data.

2. The multiple viewpoint imaging control device according to claim 1, wherein an updating is performed to set a viewpoint image that is automatically imaged by the automatic imaging execution control section at an imaging point subsequent to the imaging start point as a new reference image.

3. The multiple viewpoint imaging control device according to claim 1, wherein the judgment section judges, with respect to an image imaged after movement of a viewpoint, whether or not the imaging viewpoint is a viewpoint that should be imaged on a basis of an offset amount of a pre-specified background image when a positional offset of the specific subject between the image imaged after the movement and the reference image has been corrected.

4. The multiple viewpoint imaging control device according to claim 1, wherein an image imaged at an imaging point prior to the imaging end point is displayed on the display screen with a predetermined degree of transparency.

5. The multiple viewpoint imaging control device according to claim 1, wherein a movement path of a viewpoint from the imaging start point to the imaging end point is a circular arc path centered on the subject, and the ranging point is disposed at the middle of the movement range.

6. The multiple viewpoint imaging control device according to claim 1, wherein the imaging points, including the at least two imaging points at the imaging start point and the imaging end point, are automatically set.

7. The multiple viewpoint imaging control device according to claim 1, further comprising:

a measurement section that measures a movement speed of a viewpoint from the imaging start point to the imaging end point; and

a stopping section that, if the movement speed of a viewpoint measured by the measurement section is greater than a predetermined value, judges that continuing automatic imaging is not possible and stops the imaging by the automatic imaging execution control section.

8. The multiple viewpoint imaging control device according to claim 1, wherein the imaging points, including the at least two imaging points at the imaging start point and the imaging end point, are manually set.

9. A multiple viewpoint imaging control method comprising:

displaying a moving image for determining an imaging image angle on a display screen;

provisionally imaging a specific subject that is included in the displayed moving image;

measuring at least a distance to the specific subject;

when imaging is to be performed from a plurality of imaging viewpoints, acquiring the imaging viewpoints and a convergence angle between the imaging viewpoints;

setting a number of viewpoints, including at least an imaging start point at which imaging is initially performed and an imaging end point at which imaging is finally performed;

computing position information including a movement amount of a viewpoint from the imaging start point at which imaging is initially performed to the imaging end point at which imaging is finally performed, on a basis of data of the convergence angle and data of the distance from a ranging point to the specific subject measured by the ranging section, such that a viewpoint at a position of the provisional imaging is at a middle of the movement amount;

on a basis of the computed position information, executing a notification prompting movement of a viewpoint from the ranging point to the imaging start point and a notification prompting a movement of a viewpoint from the imaging start point to the imaging end point;

automatically imaging the subject at an imaging viewpoint;

in accordance with the movement of a viewpoint, using the imaging start point as a reference image and judging whether or not the method is at a viewpoint that should be imaged, on a basis of a comparison between the reference image and an image imaged after the movement, the automatically imaging automatically imaging the subject at an imaging viewpoint when it is judged by the judging that the method is at the imaging viewpoint; and

storing image data imaged by the automatically imaging as element data.

10. The multiple viewpoint imaging control method according to claim 9, further comprising:

measuring a movement speed of a viewpoint from the imaging start point to the imaging end point; and

if the movement speed of a viewpoint measured by the measurement section is greater than a predetermined value, judging that continuing automatic imaging is not possible and stopping the imaging by the automatic imaging execution control section.

11. The multiple viewpoint imaging control method according to claim 9, wherein an updating is performed to set a viewpoint image that is automatically imaged by the automatically imaging at an imaging point subsequent to the imaging start point as a new reference image.

12. The multiple viewpoint imaging control method according to claim 11, wherein the judging judges, with respect to an image imaged after movement of a viewpoint, whether or not the imaging viewpoint is a viewpoint that should be imaged on a basis of an offset amount of a pre-specified background image when a positional offset of the specific subject between the image imaged after the movement and the reference image has been corrected.

13. A non-transitory, computer-readable medium storing a multiple viewpoint imaging control program causing a computer to execute a process comprising:

displaying a moving image for determining an imaging image angle on a display screen;

provisionally imaging a specific subject that is included in the displayed moving image;

measuring at least a distance to the specific subject;

when imaging is to be performed from a plurality of imaging viewpoints, acquiring the imaging viewpoints and a convergence angle between the imaging viewpoints;

setting a number of viewpoints, including at least an imaging start point at which imaging is initially performed and an imaging end point at which imaging is finally performed;

computing position information including a movement amount of a viewpoint from the imaging start point at which imaging is initially performed to the imaging end point at which imaging is finally performed, on a basis of data of the convergence angle and data of the distance from a ranging point to the specific subject measured by the ranging section, such that a viewpoint at a position of the provisional imaging is at a middle of the movement amount;

on a basis of the computed position information, executing a notification prompting movement of a viewpoint from the ranging point to the imaging start point and a notification prompting movement of a viewpoint from the imaging start point to the imaging end point;

automatically imaging the subject at an imaging viewpoint;

in accordance with the movement of a viewpoint, using the imaging start point as a reference image and judging whether or not the method is at a viewpoint that should be imaged, on a basis of a comparison between the reference image and an image imaged after movement, the automatically imaging automatically imaging the subject at an imaging viewpoint when it is judged by the judging that the method is at the imaging viewpoint; and

storing image data imaged by the automatically imaging as element data.

14. The non-transitory computer-readable medium according to claim 13, the process further comprising:

measuring a movement speed of a viewpoint from the imaging start point to the imaging end point; and

if the movement speed of a viewpoint measured by the measurement section is greater than a predetermined value, judging that continuing automatic imaging is not possible and stopping the imaging by the automatic imaging execution control section.

15. The non-transitory computer-readable medium according to claim 13, wherein an updating is performed to set a viewpoint image that is automatically imaged by the automatically imaging at an imaging point subsequent to the imaging start point as a new reference image.

16. The non-transitory computer-readable medium according to claim 15, wherein the judging judges, with respect to an image imaged after the movement of a viewpoint, whether or not the imaging viewpoint is a viewpoint that should be imaged on a basis of an offset amount of a pre-specified background image when a positional offset of the specific subject between the image imaged after the movement and the reference image has been corrected.