Enabling image stabilization for a panoramic camera with multiple fixed lenses

• US 9,438,800 B1
• Filed: 12/31/2013
• Issued: 09/06/2016
• Est. Priority Date: 04/10/2013
• Status: Expired due to Fees

• Current Assignee: IC Real Tech, Inc.
• Original Assignee: IC Real Tech, Inc.
• Inventors: Kozko, Dmitry
• Primary Examiner(s): Kelley, Christopher S
• Assistant Examiner(s): Walsh, Kathleen
• Application Number:
  US14/144,689
• Time in Patent Office: 980 Days
• Field of Search: 348/36
• US Class Current: 1/1
• CPC Class Codes: A63F 13/213 comprising photodetecting means, e.g. cameras, photodiodes or infrared cells A63F13/219, A63F13/655 take precedence
• CPC Class Codes: A63F 13/65 automatically by game devices or servers from real world data, e.g. measurement in live racing competition
• CPC Class Codes: A63F 13/803 Driving vehicles or craft, e.g. cars, airplanes, ships, robots or tanks
• CPC Class Codes: A63F 13/86 Watching games played by other players
• CPC Class Codes: G03B 17/02 Bodies
• CPC Class Codes: G03B 17/08 Waterproof bodies or housings
• CPC Class Codes: G06T 11/00 2D [Two Dimensional] image generation
• CPC Class Codes: G06T 15/20 Perspective computation
• CPC Class Codes: G06T 15/30 Clipping
• CPC Class Codes: G06T 17/00 Three dimensional [3D] modelling, e.g. data description of 3D objects
• CPC Class Codes: G06T 19/00 Manipulating 3D models or images for computer graphics
• CPC Class Codes: G11B 27/10 Indexing; Addressing; Timing or synchronising; Measuring tape travel
• CPC Class Codes: H01R 13/5219 Sealing means between coupling parts, e.g. interfacial seal
• CPC Class Codes: H04N 23/6812 based on additional sensors, e.g. acceleration sensors
• CPC Class Codes: H04N 23/698 for achieving an enlarged field of view, e.g. panoramic image capture
• CPC Class Codes: H05K 5/0247 Electrical details of casings, e.g. terminals, passages for cables or wiring

Abstract

An accelerometer within fixed lenses of a 360 degree camera can be polled for a first sensor reading for a motion data. The data can be changes in a magnitude and/or a direction of a yaw, pitch, and/or roll movement. A second sensor reading of a different motion data from a different accelerometer within a base of the camera affixed to a surface can be obtained. The different motion data can be changes in a magnitude and/or a direction of a yaw, pitch, and/or roll movement. A delta between the datas can be determined, when lens shake or camera shake is detected within at least one of the plurality of fixed lenses. Optics within each of the fixed lenses or camera image sensors can be adjusted based on the delta during media capture.

Claims

1. A method for image stabilization comprising:

polling an accelerometer within a plurality of fixed lenses of a 360 degree camera for a first sensor reading comprising of a motion data, wherein the motion data are changes in at least one of a yaw movement, pitch movement, and roll movement, wherein the motion data is at least one of a magnitude and a direction associated with the yaw, pitch, and roll movement;

obtaining a second sensor reading of different motion data from a different accelerometer within a base of the camera, wherein the base is affixed to a surface, wherein the different motion data is changes in at least one of a yaw movement, pitch movement, and roll movement, wherein the different motion data is at least one of a magnitude and a direction associated with the yaw, pitch, and roll movement;

determining a delta between the motion data and the different motion data, wherein the delta is a difference in at least one of the magnitude and direction of the motion data and the different motion data, when lens shake or camera shake is detected within at least one of the plurality of fixed lenses; and

responsive to the determining adjusting the at least one of a plurality of optics within each of the plurality of fixed lenses and a plurality of camera image sensors based on the delta during media capture by at least one of the plurality of fixed lenses of the 360 degree camera, wherein a plurality of actuators within each of the plurality of fixed lenses compensate for movements not exceeding a previously established motion threshold, wherein the compensation of each of the plurality of actuators is different.

2. The method of claim 1, further comprising:

stitching together video obtained from each of the plurality of fixed lenses as a panoramic video.

3. The method of claim 1, further comprising:

encoding the motion data of the accelerometer responsive to the accelerometer detecting a change in at least one of yaw and pitch; and

responsive to the encoding, conveying the motion data to an optical stabilization actuator within each of the plurality of fixed lenses.

4. The method of claim 1, further comprising:

encoding the motion data of the different accelerometer responsive to the different accelerometer detecting a change in at least one of yaw and pitch;

responsive to the encoding, conveying the motion data to an optical stabilization actuator within each of the plurality of fixed lenses.

5. The method of claim 1, wherein at least one of the plurality of fixed lenses is a stitching lens, wherein the stitching lens comprises of a six axis stabilization system, wherein the system obtains motion data from the different accelerometer to perform the six axis stabilization.

6. The method of claim 1, wherein the base comprises of a stabilization system configured to compensate for movements exceeding a previously established motion threshold, wherein the compensation significantly reduces motion affecting the plurality of fixed lenses.

7. The method of claim 1, wherein the delta is an input value for a digital stabilization algorithm, wherein the delta is applied to each of a plurality of media recorded by the camera image sensor associated with each of the plurality of lenses.

8. The method of claim 1, further comprising:

polling a plurality of accelerometers within the base for camera movement motion during media capture;

determining a delta associated with the motion for each one of the plurality of fixed lenses; and

each lens performing optical stabilization based on the delta.

9. A system for image stabilization comprising:

a stabilization system of a 360 degree camera comprising of a camera housing, a plurality of fixed lenses, and a camera base, wherein each of the plurality of fixed lenses is an assembly comprising an accelerometer, a plurality of optic lenses, and an optic lens adjustment mechanism, wherein the camera base is coupled to the camera housing, wherein the camera base comprises of an accelerometer and a mounting hardware, wherein when the camera is affected by motion during a media capture process, the stabilization system triggers the adjustment mechanism of each of the plurality of fixed lenses to compensate for the motion based on an aggregate adjustment computed from motion data collected from each of the accelerometers of the plurality of fixed lenses; and

a data store configured to persist at least one of the aggregate adjustment and the motion data, wherein the motion data is at least one of a magnitude and a direction associated with the motion.

10. The system of claim 9, further comprising:

the stabilization system, computing a delta between motion data obtained from the accelerometer of the camera base and from each accelerometer of each of the plurality of fixed lenses.

11. The system of claim 10, wherein the delta is utilized to programmatically determine the aggregate adjustment.

12. The system of claim 9, wherein the mounting hardware is physically affixed to a surface, wherein the surface is affected by an external force resulting in the surface moving within a physical environment, wherein when the surface is moving, determining a location within the physical environment.

13. The system of claim 12, further comprising:

the stabilization system, responsive to the determining, identifying a previously established aggregate adjustment associated with the location;

the stabilization system, conveying the aggregate adjustment to at least one of the plurality of fixed lenses.

14. The system of claim 9, further comprising:

the stabilization system computing a delta between motion data at a time index obtained from the accelerometer within the camera base and motion data at the time index obtained from the accelerometer within at least one of the plurality of fixed lenses, wherein the delta is at least one of a magnitude and a direction; and

the stabilization system programmatically determining an optical stabilization adjustment for the at least one of the plurality of fixed lenses for the time index.

15. The system of claim 9, further comprising:

the stabilization system computing a delta between motion data at a time index obtained from the accelerometer within the camera base and motion data at the time index obtained from the accelerometer within at least one of the plurality of fixed lenses, wherein the delta is at least one of a magnitude and a direction; and

the stabilization system programmatically determining a digital stabilization adjustment for media captured during the media capture process by at least one of the plurality of fixed lenses for the time index.

16. The system of claim 9, further comprising:

a sensor manager configured to obtain the motion data from accelerometers in the camera base and in each of the plurality of fixed lenses;

an adjustment engine able to perform an optical digital stabilization function responsive to detecting the motion; and

a media corrector configured to perform a digital stabilization function on media captured by the media capture process.

17. The system of claim 9, further comprising:

the stabilization system, selecting a plurality of media captured from the plurality of fixed lenses during the media capture process;

the stabilization system, during a media stitching process, stitching the plurality media into a panoramic media based on the aggregate adjustment, wherein the media is at least one of a video and an image.

18. A computer program product comprising a non-transitory computer readable storage medium having computer usable program code embodied therewith, the computer usable program code comprising:

computer usable program code stored in the non-transitory computer readable storage medium, when said computer usable program code is executed by a processor it polls an accelerometer within a plurality of fixed lenses of a 360 degree camera for a first sensor reading comprising of a motion data, wherein the motion data are changes in at least one of a yaw movement, pitch movement, and roll movement, wherein the motion data is at least one of a magnitude and a direction associated with the yaw, pitch, and roll movement;

computer usable program code stored in the non-transitory computer readable storage medium, when said computer usable program code is executed by the processor it obtains an second sensor reading of different motion data from a different accelerometer within a base of the camera, wherein the base is affixed to a surface, wherein the different motion data is changes in at least one of a yaw movement, pitch movement, and roll movement, wherein the motion data is at least one of a magnitude and a direction associated with the yaw, pitch, and roll movement;

computer usable program code stored in the non-transitory computer readable storage medium, when said computer usable program code is executed by the processor it, encodes the motion data of the accelerometer responsive to the accelerometer detecting a change in at least one of yaw and pitch;

computer usable program code stored in the non-transitory computer readable storage medium, when said computer usable program code is executed by the processor it, responsive to encoding the motion data, conveys the motion data to an optical stabilization actuator within each of the plurality of fixed lenses;

computer usable program code stored in the non-transitory computer readable storage medium, when said computer usable program code is executed by the processor it determines a delta between the motion data and the different motion data, wherein the delta is a difference in at least one of the magnitude and direction of the motion data and the different motion data, when lens shake or camera shake is detected within at least one of the plurality of fixed lenses; and

computer usable program code stored in the non-transitory computer readable storage medium, when said computer usable program code is executed by the processor it, responsive to determining the delta, adjusts the at least one of a plurality of optics within each of the plurality of fixed lenses and a plurality of camera image sensors based on the delta during media capture by at least one of the plurality of fixed lenses of the 360 degree camera.

19. The computer program product of claim 18, further comprising:

computer usable program code stored in the non-transitory computer readable storage medium, when said computer usable program code is executed by the processor it selects a plurality of media captured from the plurality of lenses during the media capture process;

computer usable program code stored in the non-transitory computer readable storage medium, when said computer usable program code is executed by the processor it, during a media stitching process, stitches the plurality media into a panoramic media based on the delta, wherein the media is at least one of a video and an image.