Image processing device and associated methodology for generating panoramic images

• US 9,185,287 B2
• Filed: 07/18/2012
• Issued: 11/10/2015
• Est. Priority Date: 08/02/2011
• Status: Active Grant

• Current Assignee: Sony Group Corp.
• Original Assignee: Sony Corporation (Sony Group Corp.)
• Inventors: Sento, Kazuya
• Primary Examiner(s): Kostak, Victor
• Application Number:
  US13/551,663
• Publication Number:
  US 20130033566A1
• Time in Patent Office: 1,210 Days
• Field of Search: 348/36, 348/39, 348/222.1, 348/333.02, 348/333.11, 348/333.12, 348/231.3, 348/231.5
• US Class Current: 1/1
• CPC Class Codes: G03B 17/12 with means for supporting objectives, supplementary lenses, filters, masks, or turrets
• CPC Class Codes: G03B 17/20 visible in viewfinder
• CPC Class Codes: G03B 37/02 with scanning movement of lens or cameras
• CPC Class Codes: G03B 37/04 with cameras or projectors providing touching or overlapping fields of view
• CPC Class Codes: G06T 11/60 Editing figures and text; Combining figures or text
• CPC Class Codes: G06T 3/4038 Image mosaicing, e.g. composing plane images from plane sub-images
• CPC Class Codes: G06T 5/90 Dynamic range modification of images or parts thereof
• CPC Class Codes: G06T 5/92 based on global image properties
• CPC Class Codes: G06T 7/70 Determining position or orientation of objects or cameras camera calibration G06T7/80
• CPC Class Codes: H04N 23/45 for generating image signals from two or more image sensors being of different type or operating in different modes, e.g. with a CMOS sensor for moving images in combination with a charge-coupled device [CCD] for still images
• CPC Class Codes: H04N 23/62 Control of parameters via user interfaces
• CPC Class Codes: H04N 23/63 by using electronic viewfinders
• CPC Class Codes: H04N 23/632 for displaying or modifying preview images prior to image capturing, e.g. variety of image resolutions or capturing parameters
• CPC Class Codes: H04N 23/635 Region indicators; Field of view indicators
• CPC Class Codes: H04N 23/698 for achieving an enlarged field of view, e.g. panoramic image capture
• CPC Class Codes: H04N 23/90 Arrangement of cameras or camera modules, e.g. multiple cameras in TV studios or sports stadiums
• CPC Class Codes: H04N 25/41 Extracting pixel data from a plurality of image sensors simultaneously picking up an image, e.g. for increasing the field of view by combining the outputs of a plurality of sensors
• CPC Class Codes: H04N 25/46 by combining or binning pixels

Abstract

One embodiment of an apparatus includes a reference position receiving unit configured to receive intermediate or end panorama reference position information input by a user, and a control unit configured to control an imaging device to begin generating a plurality of images to be used to generate a panoramic image based on the intermediate or end panorama reference position information input by the user after the reference position receiving unit receives the intermediate or end panorama reference position information.

Claims

1. An image capturing apparatus that captures images while being moved such that the captured images are used for generating a panoramic image, the image capturing apparatus comprising:

circuitry configured to

control a display of an instruction to instruct a movement of the image capturing apparatus toward a predetermined direction from a reference position, at the reference position, and

start to record the captured images in response to an arrival of the image capturing apparatus at a predetermined start position, the predetermined start position being a position that is a predetermined distance from the reference position in the predetermined direction.

2. The image capturing apparatus according to claim 1, wherein the reference position includes a center position of the panoramic image.

3. The image capturing apparatus according to claim 1, wherein the circuitry is further configured to generate a plurality of images.

4. The image capturing apparatus according to claim 1, wherein an end panorama reference position includes another reference image, and the circuitry is configured to end generating the plurality of images when a current image matches the other reference image.

5. The image capturing apparatus according to claim 1, wherein the circuitry is further configured to detect a current orientation of the apparatus.

6. The image capturing apparatus according to claim 4, wherein the end panorama reference position information includes an end panorama reference position orientation, and the circuitry is configured to end generating the plurality of images when the current orientation matches the end panorama reference position orientation.

7. The image capturing apparatus according to claim 4, wherein the circuitry receives an image width from the user and determines a start point orientation based on the image width and the end panorama reference position.

8. The image capturing apparatus according to claim 5, wherein the reference position information includes a reference position orientation, and the circuitry is further configured to receive an image width from the user and determine a start point orientation and an end point orientation based on the image width and the reference position orientation.

9. The image capturing apparatus according to claim 1, wherein the circuitry is configured to receive panoramic image width from the user and determine a start point and an end point of the panoramic image based on the image width and the reference position.

10. The image capturing apparatus according to claim 3, wherein the circuitry comprises a main imaging circuit and a sub imaging circuit.

11. The image capturing apparatus according to claim 10, wherein the sub imaging circuit images an end panorama reference image, and the circuitry ends generating the plurality of images when a current image matches the end panorama reference position image.

12. The image capturing apparatus according to claim 10, wherein the sub imaging circuit images a start panorama reference image, and the circuitry starts generating the plurality of images when a current image matches the start panorama reference position image.

13. The image capturing apparatus according to claim 12, wherein the sub imaging circuit images an end panorama reference image, and the circuitry ends generating the plurality of images when a current image matches the end panorama reference position image.

14. The image capturing apparatus according to claim 10, wherein the main imaging circuit images all of the plurality of images for the panoramic image.

15. The image capturing apparatus according to claim 1, wherein the apparatus is a mobile phone, the mobile phone transmits the plurality of images to a server, and receives the panoramic image from the server.

16. The image capturing apparatus according to claim 1, wherein the circuitry is configured to begin generating the plurality of images to be used to generate the panoramic image based on panorama reference position information for a plurality of reference points input by a user.

17. The image capturing apparatus according to claim 16, wherein the circuitry receives the panorama reference position information for the plurality of reference points including at least a start point and a center point.

18. The image capturing apparatus according to claim 1, wherein the circuitry is configured to begin generating the plurality of images to be used to generate the panoramic image based on an end panorama reference position image input by a user, and to control the imaging device to end generating the plurality of images when a current image matches the reference image.

19. An image processing method for controlling an image capturing apparatus that captures images while the image capturing apparatus is moved such that the captured images are used for generating a panoramic image, the method comprising:

controlling a display of an instruction to instruct a movement of the image capturing apparatus toward a predetermined direction from a reference position, at the reference position; and

starting to record the captured images in response to an arrival of the image capturing apparatus at a predetermined start position, the predetermined start position being a position that is a predetermined distance from the reference position in the predetermined direction.

20. The method according to claim 19, further comprising:

detecting a user operation to set an intermediate position of the panoramic image; and

determining the reference position as a start position based on the intermediate position.

21. The method according to claim 20, further comprising:

determining an end position corresponding to an endpoint of the panoramic image based on the intermediate position.

22. The method according to claim 21, wherein the start position is a predetermined angle from the intermediate position in a first direction, and the end position is a predetermined angle from the intermediate position in a second direction opposite the first direction.

23. The method according to claim 20, wherein the intermediate position defines a midpoint of the panoramic image.

24. The method according to claim 20, wherein the instruction includes an instruction to move the image capturing apparatus to the start position.

25. The method according to claim 20, wherein the user operation corresponds to fully depressing a shutter button of the image capturing apparatus.

26. The method according to claim 20, further comprising:

controlling display of a rectangular shape to indicate a panorama image range; and

controlling display of a solid line to indicate a center position of the panoramic image.

27. The method according to claim 20, further comprising:

storing, as reference attitude parameters, attitude parameters of the image capturing apparatus in response to the user operation to set the intermediate position of the panoramic image.

28. The method according to claim 27, wherein the attitude parameters correspond to the attitude parameters of the image capturing apparatus when the user operation is received.

29. The method according to claim 27, wherein the reference attitude parameters are used to capture other images used to form the panoramic image.

30. The method according to claim 22, wherein the predetermined angle is ninety degrees.

31. The method according to claim 19, wherein the intermediate position is a position other than a midpoint of the panoramic image.

32. A non-transitory computer readable medium encoded computer readable instructions that, when executed by a processor, causes the processor to perform an image processing method for controlling an image capturing apparatus that captures images while the ma e capturing apparatus is moved such that the captured images are used for generating a panoramic image, the method comprising:

controlling a display of an instruction to instruct a movement of the image capturing apparatus toward a predetermined direction from a reference position, at the reference position; and

starting to record the captured images in response to an arrival of the image capturing apparatus at a predetermined start position, the predetermined start position being a position that is a predetermined distance from the reference position in the predetermined direction.