Imaging apparatus, imaging method, and program

• US 10,200,609 B2
• Filed: 12/05/2017
• Issued: 02/05/2019
• Est. Priority Date: 11/11/2010
• Status: Expired due to Fees

• Current Assignee: Sony Corporation (Sony Group Corp.)
• Original Assignee: Sony Corporation (Sony Group Corp.)
• Inventors: Arai, Takuya, Goto, Junichiro
• Primary Examiner(s): Vo, Tung T
• Assistant Examiner(s): Becker, Joseph W
• Application Number:
  US15/832,280
• Publication Number:
  US 20180097995A1
• Time in Patent Office: 427 Days
• CPC Class Codes: G03B 17/12 with means for supporting objectives, supplementary lenses, filters, masks, or turrets
• CPC Class Codes: G03B 17/20 visible in viewfinder
• CPC Class Codes: G06T 3/4038 Image mosaicing, e.g. composing plane images from plane sub-images
• CPC Class Codes: G06T 7/70 Determining position or orientation of objects or cameras camera calibration G06T7/80
• CPC Class Codes: H04N 23/45 for generating image signals from two or more image sensors being of different type or operating in different modes, e.g. with a CMOS sensor for moving images in combination with a charge-coupled device [CCD] for still images
• CPC Class Codes: H04N 23/62 Control of parameters via user interfaces
• CPC Class Codes: H04N 23/632 for displaying or modifying preview images prior to image capturing, e.g. variety of image resolutions or capturing parameters
• CPC Class Codes: H04N 23/634 Warning indications
• CPC Class Codes: H04N 23/635 Region indicators; Field of view indicators
• CPC Class Codes: H04N 23/667 Camera operation mode switching, e.g. between still and video, sport and normal or high- and low-resolution modes
• CPC Class Codes: H04N 23/698 for achieving an enlarged field of view, e.g. panoramic image capture
• CPC Class Codes: H04N 5/2628 Alteration of picture size, shape, position or orientation, e.g. zooming, rotation, rolling, perspective, translation

Abstract

An image processing apparatus including an interface that receives an input identifying a subject, and a processor that generates a panoramic image based on captured image data so that the subject is positioned in a reference position in the panoramic image.

Claims

1. An imaging apparatus comprising:

a shutter button configured to output a signal when operated by a user;

an image sensor configured to generate an image;

a display; and

circuitry configured to

control the display to display a first instruction to the user to operate the shutter button at an initial position of the imaging apparatus;

control the display to display a second instruction to the user to change an orientation of the imaging apparatus from the initial position to a start position in a first direction and to operate the shutter button after the first instruction has been displayed;

control the display to display a third instruction to the user to change the orientation of the imaging apparatus in a second direction opposite to the first direction after the second instruction has been displayed;

start a predetermined processing procedure, before a start of recording images by the image sensor used in generating a panorama image, in response to a first operation of the shutter button by the user based on the first instruction;

record an initial image generated by the image sensor, used in generating the panorama image, in response to a second operation of the shutter button by the user based on the second instruction after the first operation of the shutter button by the user based on the first instruction, the initial image generated by the image sensor at the start position constructing one end of the panorama image in an longitudinal direction of the panorama image;

record a last image used in generating the panorama image when the imaging apparatus reaches an end position that is opposite to the start position with respect to the initial position after the second operation of the shutter button by the user based on the second instruction, the last image generated by the image sensor at the end position constructing the other end of the panorama image; and

generate the panorama image from at least the initial image and the last image.

2. The imaging apparatus of claim 1, wherein the circuitry is configured to stop the predetermined processing procedure in a case where the panorama image has been generated.

3. The imaging apparatus of claim 1, wherein the circuitry is configured to determine the end position from a size of the panorama image.

4. The imaging apparatus of claim 3, wherein the circuitry is configured to determine the end position from the size of the panorama image and from the start position.

5. The imaging apparatus of claim 1, wherein the circuitry is configured to determine the first direction and the second direction before a display of the first instruction.

6. The imaging apparatus of claim 5, wherein the circuitry is configured to

control the display to display a prompt to prompt the user to determine the second direction;

and

determine the first direction and the second direction in accordance with an input indicating the second direction from the user.

7. The imaging apparatus of claim 1, wherein the circuitry is configured to

control the display to display a prompt to the user to identify a reference subject;

identify the reference subject in accordance with an input identifying the reference subject from the user; and

control the display to display a position of the reference subject relative to the images recorded by the imaging apparatus as the orientation of the imaging apparatus is changed.

8. The imaging apparatus of claim 1, wherein the first direction and the second direction are on a same single line.

9. An imaging method performed using an imaging apparatus having a shutter button configured to output a signal when operated by a user, an image sensor configured to generate an image, a display, and circuitry, the method comprising:

controlling the display to display a first instruction to the user to operate the shutter button at an initial position of the imaging apparatus;

controlling the display to display a second instruction to the user to change an orientation of the imaging apparatus from the initial position to a start position in a first direction and to operate the shutter button after the first instruction has been displayed;

controlling the display to display a third instruction to the user to change the orientation of the imaging apparatus in a second direction opposite to the first direction after the second instruction has been displayed;

starting a predetermined processing procedure, before a start of recording images by the image sensor used in generating a panorama image, in response to an operation of the shutter button by the user based on the first instruction;

record an initial image generated by the image sensor, used in generating the panorama image, in response to a second operation of the shutter button by the user based on the second instruction after the first operation of the shutter button by the user based on the first instruction, the initial image generated by the image sensor at the start position constructing one end of the panorama image in an longitudinal direction of the panorama image;

record a last image used in generating the panorama image when the imaging apparatus reaches an end position that is opposite to the start position with respect to the initial position after the second operation of the shutter button by the user based on the second instruction, the last image generated by the image sensor at the end position constructing the other end of the panorama image; and

generating the panorama image from at least the initial image and the last image.

10. The imaging method of claim 9, further comprising stopping the predetermined processing procedure in a case where the panorama image has been generated.

11. The imaging method of claim 9, further comprising determining the end position from a size of the panorama image.

12. The imaging method of claim 11, comprising determining the end position from the size of the panorama image and from the start position.

13. The imaging method of claim 9, further comprising determining the first direction and the second direction before a display of the first instruction.

14. The imaging method of claim 13, comprising:

controlling the display to display a prompt to prompt the user to determine the second direction; and

determining the first direction and the second direction in accordance with an input indicating the second direction from the user.

15. The imaging method of claim 9, further comprising:

controlling the display to display a prompt to the user to identify a reference subject;

identifying the reference subject in accordance with an input identifying the reference subject from the user; and

controlling the display to display a position of the reference subject relative to the images recorded by the imaging apparatus as the orientation of the imaging apparatus is changed.

16. A non-transitory, computer-readable recording medium storing instructions that, when executed by an imaging apparatus having a shutter button configured to output a signal when operated by a user, an image sensor configured to generate an image, a display, and circuitry, control the imaging apparatus to carry out a method comprising:

controlling the display to display a first instruction to the user to operate the shutter button at an initial position of the imaging apparatus;

controlling the display to display a second instruction to the user to change an orientation of the imaging apparatus from the initial position to a start position in a first direction and to operate the shutter button after the first instruction has been displayed;

controlling the display to display a third instruction to the user to change the orientation of the imaging apparatus in a second direction opposite to the first direction after the second instruction has been displayed;

starting a predetermined processing procedure, before a start of recording images by the image sensor used in generating a panorama image, in response to a first operation of the shutter button by the user based on the first instruction;

record an initial image generated by the image sensor, used in generating the panorama image, in response to a second operation of the shutter button by the user based on the second instruction after the first operation of the shutter button by the user based on the first instruction, the initial image generated by the image sensor at the start position constructing one end of the panorama image in a longitudinal direction of the panorama image;

record a last image used in generating the panorama image when the imaging apparatus reaches an end position that is opposite to the start position with respect to the initial position after the second operation of the shutter button by the user based on the second instruction, the last image generated by the image sensor at the end position constructing the other end of the panorama image; and

generating the panorama image from at least the initial image and the last image.

17. The non-transitory, computer-readable recording medium of claim 16, wherein the method further comprises stopping the predetermined processing procedure in a case where the panorama image has been generated.

18. The non-transitory, computer-readable recording medium of claim 16, wherein the method further comprises determining the end position from a size of the panorama image.

19. The non-transitory, computer-readable recording medium of claim 18, wherein the method comprises determining the end position from the size of the panorama image and from the start position.

20. The non-transitory, computer-readable recording medium of claim 16, wherein the method further comprises determining the first direction and the second direction before a display of the first instruction.

21. The non-transitory, computer-readable recording medium of claim 20, wherein the method comprises:

controlling the display to display a prompt to prompt the user to determine the second direction; and

determining the first direction and the second direction in accordance with an input indicating the second direction from the user.

22. The non-transitory, computer-readable recording medium of claim 16, wherein the method further comprises:

controlling the display to display a prompt to the user to identify a reference subject;

identifying the reference subject in accordance with an input identifying the reference subject from the user; and

controlling the display to display a position of the reference subject relative to the images recorded by the imaging apparatus as the orientation of the imaging apparatus is changed.

23. An imaging apparatus comprising:

a shutter button configured to output a signal when operated by a user;

an image sensor configured to generate an image;

a display; and

circuitry configured to

control the display to display a first instruction to the user to operate the shutter button at an initial position of the imaging apparatus;

control the display to display a second instruction to the user to change an orientation of the imaging apparatus from the initial position to a start position in a first direction and to operate the shutter button after the first instruction has been displayed;

control the display to display a third instruction to the user to change the orientation of the imaging apparatus in a second direction opposite to the first direction after the second instruction has been displayed;

start a predetermined processing procedure, before a start of recording images by the image sensor used in generating a synthesized image having a field of view wider than a field of view of the image sensor, in response to a first operation of the shutter button by the user based on the first instruction;

record an initial image generated by the image sensor, used in generating the synthesized image, in response to an operation of the shutter button by the user based on the second instruction after the first operation of the shutter button by the user based on the first instruction, the initial image generated by the image sensor at the start position constructing one end of the synthesized image in an longitudinal direction of the synthesized image;

record a last image used in generating the synthesized image when the imaging apparatus reaches an end position that is opposite to the start position with respect to the initial position after the second operation of the shutter button by the user based on the second instruction, the last image generated by the image sensor at the end position constructing the other end of the panorama image; and

generate the synthesized image from at least the initial image and the last image.

24. The imaging apparatus of claim 23, wherein the circuitry is configured to stop the predetermined processing procedure in a case where the synthesized image has been generated.

25. The imaging apparatus of claim 23, wherein the circuitry is configured to determine the end position from a size of the synthesized image.

26. The imaging apparatus of claim 25, wherein the circuitry is configured to determine the end position from the size of the synthesized image and from the start position.

27. The imaging apparatus of claim 23, wherein the circuitry is configured to determine the first direction and the second direction before a display of the first instruction.

28. The imaging apparatus of claim 27, wherein the circuitry is configured to

control the display to display a prompt to prompt the user to determine the second direction;

and

determine the first direction and the second direction in accordance with an input indicating the second direction from the user.

29. The imaging apparatus of claim 23, wherein the circuitry is configured to

control the display to display a prompt to the user to identify a reference subject;

identify the reference subject in accordance with an input identifying the reference subject from the user; and

control the display to display a position of the reference subject relative to the images recorded by the imaging apparatus as the orientation of the imaging apparatus is changed.