Image processing device and associated methodology for generating panoramic images

• US 9,906,719 B2
• Filed: 10/02/2015
• Issued: 02/27/2018
• Est. Priority Date: 08/02/2011
• Status: Active Grant

• Current Assignee: Sony Group Corp.
• Original Assignee: Sony Corporation (Sony Group Corp.)
• Inventors: Sento, Kazuya
• Primary Examiner(s): Lee, Michael
• Application Number:
  US14/873,548
• Publication Number:
  US 20160028953A1
• Time in Patent Office: 879 Days
• Field of Search: 348 36
• CPC Class Codes: G03B 17/12 with means for supporting objectives, supplementary lenses, filters, masks, or turrets
• CPC Class Codes: G03B 17/20 visible in viewfinder
• CPC Class Codes: G03B 37/02 with scanning movement of lens or cameras
• CPC Class Codes: G03B 37/04 with cameras or projectors providing touching or overlapping fields of view
• CPC Class Codes: G06T 11/60 Editing figures and text; Combining figures or text
• CPC Class Codes: G06T 3/4038 Image mosaicing, e.g. composing plane images from plane sub-images
• CPC Class Codes: G06T 5/90 Dynamic range modification of images or parts thereof
• CPC Class Codes: G06T 5/92 based on global image properties
• CPC Class Codes: G06T 7/70 Determining position or orientation of objects or cameras camera calibration G06T7/80
• CPC Class Codes: H04N 23/45 for generating image signals from two or more image sensors being of different type or operating in different modes, e.g. with a CMOS sensor for moving images in combination with a charge-coupled device [CCD] for still images
• CPC Class Codes: H04N 23/62 Control of parameters via user interfaces
• CPC Class Codes: H04N 23/63 by using electronic viewfinders
• CPC Class Codes: H04N 23/632 for displaying or modifying preview images prior to image capturing, e.g. variety of image resolutions or capturing parameters
• CPC Class Codes: H04N 23/635 Region indicators; Field of view indicators
• CPC Class Codes: H04N 23/698 for achieving an enlarged field of view, e.g. panoramic image capture
• CPC Class Codes: H04N 23/90 Arrangement of cameras or camera modules, e.g. multiple cameras in TV studios or sports stadiums
• CPC Class Codes: H04N 25/41 Extracting pixel data from a plurality of image sensors simultaneously picking up an image, e.g. for increasing the field of view by combining the outputs of a plurality of sensors
• CPC Class Codes: H04N 25/46 by combining or binning pixels

Abstract

One embodiment of an apparatus includes a reference position receiving unit configured to receive intermediate or end panorama reference position information input by a user, and a control unit configured to control an imaging device to begin generating a plurality of images to be used to generate a panoramic image based on the intermediate or end panorama reference position information input by the user after the reference position receiving unit receives the intermediate or end panorama reference position information.

Claims

1. An image processing method for controlling an image capturing apparatus that captures images while the image capturing apparatus is moved such that the captured images are used for generating a panoramic image, the method comprising:

controlling, with circuitry, a display of an instruction to a user to instruct a movement of the image capturing apparatus toward a predetermined direction;

determining, with the circuitry, a center of the panoramic image as a position of the image capturing apparatus at which an instruction from the user is received;

recording, with the circuitry, image data as the image capturing apparatus is moved in a first direction relative to the center position and in a second direction relative to the center position, the second direction being opposite to the first direction, the image data including at least the captured images; and

using, with the circuitry, the image data recorded to generate the panoramic image, the panoramic image being generated based on image data recorded in the first direction and in the second direction.

2. The image processing method according to claim 1, wherein the predetermined direction is a direction with respect to the center position.

3. The image processing method according to claim 1, further comprising:

controlling display of a rectangular shape to indicate a panoramic image range.

15. The image processing method according to claim 1, wherein the predetermined start position and the predetermined end position are symmetrically arranged relative to the center position.

16. The image processing method according to claim 2, wherein the image data further includes brightness information.

17. The image processing method according to claim 16, wherein brightness information is recorded as the image capturing apparatus is moved in the first direction.

18. The image processing method according to claim 17, wherein the captured images are recorded as the image capturing apparatus is moved in the second direction.

4. An image capturing apparatus that captures images while the image capturing apparatus is moved such that the captured images are used for generating a panoramic image, the image capturing apparatus comprising:

circuitry configured to

control a display of an instruction to a user to instruct a movement of the image capturing apparatus toward a predetermined direction,

determine a center of the panoramic image as a position of the image capturing apparatus at which an instruction from the user is received,

record image data as the image capturing apparatus is moved in a first direction relative to the center position and in a second direction relative to the center position, the second direction being opposite the first direction, the image data including at least the captured images, and

use the image data recorded to generate the panoramic image, the panoramic image being based on image data recorded in the first direction and in the second direction.

5. The image capturing apparatus according to claim 4, wherein the predetermined direction is a direction with respect to the center position.

6. The image capturing apparatus according to claim 4, wherein the circuitry is further configured to control display of a rectangular shape to indicate a panoramic image range.

7. A non-transitory computer-readable medium encoded with computer-readable instructions thereon that, when executed by circuitry, cause the circuitry to perform a method for controlling an image capturing apparatus that captures images while the image capturing apparatus is moved such that the captured images are used for generating a panoramic image, the method comprising:

controlling a display of an instruction to a user to instruct a movement of the image capturing apparatus toward a predetermined direction;

determining a center of the panoramic image as a position of the image capturing apparatus at which an instruction from the user is received;

recording image data as the image capturing apparatus is moved in a first direction relative to the center position and in a second direction relative to the center position, the second direction being opposite to the first direction, the image data including at least the captured images; and

using the imaged data recorded in order to generate the panoramic image, the panoramic image being based on image data recorded in the first direction and in the second direction.

8. An image processing method for controlling an image capturing apparatus that captures images while the image capturing apparatus is moved such that the captured images are used for generating a synthesized image, the method comprising:

controlling, with circuitry, a display of an instruction to a user to instruct a movement of the image capturing apparatus toward a predetermined direction;

determining, with the circuitry, a center of the synthesized image as a position of the image capturing apparatus at which an instruction from the user is received;

recording, with the circuitry, image data as the image capturing apparatus is moved in a first direction relative to the center position and in a second direction relative to the center position, the second direction being opposite to the first direction, the image data including at least the captured images; and

using, with the circuitry, the image data recorded to generate the synthesized image, the synthesized image being based on image data recorded in the first direction and in the second direction.

9. The image processing method according to claim 8, wherein the predetermined direction is a direction with respect to the center position.

10. The image processing method according to claim 8, further comprising:

controlling display of a rectangular shape to indicate a synthesized image range.

11. An image capturing apparatus that captures images while the image capturing apparatus is moved such that the captured images are used for generating a synthesized image, the image capturing apparatus comprising:

circuitry configured to

control a display of an instruction to a user to instruct a movement of the image capturing apparatus toward a predetermined direction;

determine a center of the synthesized image as a position of the image capturing apparatus at which instruction from the user is received,

record image data as the image capturing apparatus is moved in a first direction relative to the center position and in a second direction relative to the center position, the second direction being opposite the first direction, the image data including at least the captured images, and

use the image data recorded to generate the synthesized image, the synthesized image being based on image data recorded in the first direction and in the second direction.

12. The image capturing apparatus according to claim 11, wherein the predetermined direction is a direction with respect to the center position.

13. The image capturing apparatus according to claim 11, wherein the circuitry is further configured to control display of a rectangular shape to indicate a synthesized image range.

14. A non-transitory computer-readable medium encoded with computer-readable instructions that, when executed by circuitry, cause the circuitry to perform a method for controlling an image capturing apparatus that captures images while the image capturing apparatus is moved such that the captured images are used for generating a synthesized image, the method comprising:

controlling a display of an instruction to a user to instruct a movement of the image capturing apparatus toward a predetermined direction;

determining a center of the synthesized image as a position of the image capturing apparatus at which an instruction from the user is received;

recording image data as the image capturing apparatus is moved in a first direction relative to the center position and in a second direction relative to the center position, the second direction being opposite to the first direction, the image data including at least the captured images; and

using the image data recorded to generate the synthesized image, the synthesized image being based on image data recorded in the first direction and in the second direction.