Controller in a camera for creating a panoramic image

• US 8,355,042 B2
• Filed: 08/10/2009
• Issued: 01/15/2013
• Est. Priority Date: 10/16/2008
• Status: Expired due to Fees

• Current Assignee: Spatial Cam LLC
• Original Assignee: Spatial Cam LLC
• Inventors: Lablans, Peter
• Primary Examiner(s): BOUTAH, ALINA A
• Application Number:
  US12/538,401
• Publication Number:
  US 20100097443A1
• Time in Patent Office: 1,254 Days
• Field of Search: 348/36, 348/39, 348/46, 348/E13.007
• US Class Current: 348/36
• CPC Class Codes: G03B 37/00 Panoramic or wide-screen photography; Photographing extended surfaces, e.g. for surveying; Photographing internal surfaces, e.g. of pipe
• CPC Class Codes: G06T 7/80 Analysis of captured images to determine intrinsic or extrinsic camera parameters, i.e. camera calibration
• CPC Class Codes: H04N 1/00307 with a mobile telephone apparatus constructional features of portable telephone sets H04M1/0202; mobile radio systems H04W84/00
• CPC Class Codes: H04N 2101/00 Still video cameras
• CPC Class Codes: H04N 23/635 Region indicators; Field of view indicators
• CPC Class Codes: H04N 23/661 Transmitting camera control signals through networks, e.g. control via the Internet
• CPC Class Codes: H04N 23/67 Focus control based on electronic image sensor signals
• CPC Class Codes: H04N 23/698 for achieving an enlarged field of view, e.g. panoramic image capture
• CPC Class Codes: H04N 23/90 Arrangement of cameras or camera modules, e.g. multiple cameras in TV studios or sports stadiums
• CPC Class Codes: H04N 25/40 Extracting pixel data from image sensors by controlling scanning circuits, e.g. by modifying the number of pixels sampled or to be sampled
• CPC Class Codes: H04N 25/41 Extracting pixel data from a plurality of image sensors simultaneously picking up an image, e.g. for increasing the field of view by combining the outputs of a plurality of sensors
• CPC Class Codes: H04N 25/445 by skipping some contiguous pixels within the read portion of the array
• CPC Class Codes: H04N 5/2628 Alteration of picture size, shape, position or orientation, e.g. zooming, rotation, rolling, perspective, translation

Abstract

Methods and apparatus to create and display panoramic images on a mobile device are disclosed. Such a mobile device can be a mobile phone. Apparatus is provided to control the position of a lens in relation to a reference lens. Methods and apparatus are provided to generate multiple images that are combined into a panoramic image. A panoramic image may be a static image. It may also be a video image. A controller provides correct camera settings for different conditions. An image processor creates a panoramic image from the correct settings provided by the controller. A panoramic camera is applied in a computer gaming system.

Claims

1. A camera, comprising:

a first and a second imaging unit, each imaging unit including a lens and an image sensor;

a portable body;

the image sensors of the first and second imaging unit being rotationally aligned in the portable body with a misalignment angle that is determined during a calibration and that is not greater than a pre-determined value;

a memory to store setting data generated during the calibration that define first coordinates of a first active area of the image sensor of the first imaging unit that is smaller than a total active sensor area of the image sensor of the first imaging unit and second coordinates of a first active area of the image sensor of the second imaging unit that is smaller than a total active sensor area of the image sensor of the second imaging unit, wherein a combination of the first and second coordinates defines a registered panoramic image;

a controller to apply the setting data to process image data from the image sensors of the first and second imaging unit only generated by its first active areas to generate the registered panoramic image; and

a memory to store the registered panoramic image in a raw image data format prior to a demosaicing.

2. The camera as claimed in claim 1, further comprising a display to display the registered panoramic image.

3. The camera as claimed in claim 2, wherein a pixel density of the image sensor in the first imaging unit is greater than the pixel density of the display.

4. The camera as claimed in claim 1, wherein the registered panoramic image is a video image.

5. The camera as claimed in claim 1, wherein the misalignment angle is about or smaller than 0.5 degree.

6. The camera as claimed in claim 1, wherein the misalignment angle is about or smaller than 1 arcmin.

7. The camera as claimed in claim 1, wherein the misalignment angle is applied to determine a scanline angle for the image sensor of the second imaging unit.

8. The camera as claimed in claim 1, wherein the misalignment angle is applied to generate an address transformation to store image data of the first active area of the image sensor of the second imaging unit in a rectangular addressing scheme.

9. The camera as claimed in claim 1, wherein the camera is comprised in a mobile computing device.

10. The camera as claimed in claim 1, wherein the camera is comprised in a mobile phone.

11. The camera as claimed in claim 1, further comprising at least a third imaging unit including a lens and an image sensor.

12. The camera as claimed in claim 1, wherein the lens of the first imaging unit has a fixed focal length.

13. The camera as claimed in claim 1, wherein the lens of the first imaging unit has a variable focal length and the setting data depends on a focal setting.

14. The camera as claimed in claim 1, wherein the lens of the first imaging unit is a zoom lens and the setting data depends on a zoom setting.

15. The camera as claimed in claim 1, further comprising a motor mechanism that reduces the misalignment angle to negligible.

20. The camera of claim 1, further comprising a memory to store the substantially registered panoramic image in a raw image data format prior to a demosaicing.

16. A camera, comprising:

a first and a second imaging unit, each imaging unit including a lens and an image sensor; and

a controller to apply data generated during a calibration that determines a first active sensor area of the image sensor of the first imaging unit that is smaller than a total active sensor area of the image sensor of the first imaging unit and a first active sensor area of the image sensor of the second imaging unit that is smaller than a total active sensor area of the image sensor of the second imaging unit to generate a registered panoramic image during operation of the camera by processing image data of the image sensors of the first and second imaging unit only generated by their first active sensor areas, wherein

a misalignment angle between the image sensor of the first imaging unit and the image sensor of the second imaging unit is applied to determine a scanline angle for the image sensor of the second imaging unit.

17. The camera of claim 16, wherein the camera is part of a mobile phone.

18. A camera, comprising:

a portable body;

a first imaging unit including a first image sensor and a first lens and a second imaging unit including a second image sensor and a second lens, wherein during a calibration are defined an active sensor area of the first image sensor that is smaller than a total active area of the first image sensor to generate a first set of image data and an active sensor area of the second image sensor that is smaller than a total active area of the second image sensor to generate a second set of image data; and

a memory to store prior to a de-mosaicing the first and second set of image data, wherein the first and second set of image data form a substantially registered panoramic image in a raw data format.

19. The camera of claim 18, wherein image data generated by the first image sensor and image data generated by the second image sensor each generated by applying a horizontal scanline are rotated relative to each other to compensate for a misalignment angle between the first and second image sensor.