Image stabilization device using image analysis to control movement of an image recording sensor

• US 8,289,406 B2
• Filed: 09/14/2005
• Issued: 10/16/2012
• Est. Priority Date: 09/18/2004
• Status: Expired due to Fees

• Current Assignee: Deutsche Telekom AG
• Original Assignee: Deutsche Telekom AG
• Inventors: Brackmann, Ludwig
• Primary Examiner(s): CUTLER, ALBERT H
• Application Number:
  US11/226,014
• Publication Number:
  US 20060061660A1
• Time in Patent Office: 2,589 Days
• Field of Search: 348/208.99, 348/208.1, 348/208.2, 348/208.3, 348/208.4, 348/208.5, 348/208.7
• US Class Current: 348/208.7
• CPC Class Codes: H04N 2007/145 Handheld terminals
• CPC Class Codes: H04N 23/54 Mounting of pick-up tubes, electronic image sensors, deviation or focusing coils
• CPC Class Codes: H04N 23/68 for stable pick-up of the scene, e.g. compensating for camera body vibrations
• CPC Class Codes: H04N 23/6811 based on the image signal
• CPC Class Codes: H04N 23/687 by shifting the lens or sensor position

Abstract

An image stabilization device includes a number of micromechanical actuators, an image recording sensor, and a computational device. The image sense generates image data of an image and is suspendedly mounted to the micromechanical actuators. The computational device receives and processes the image data generated by the image recording sensor. The computational device includes a detection device for detecting shake so as to perform an image analysis on the image data and detect changes in a time sequence of the image data. The detected changes are used to generate control signals for controlling the micromechanical actuators so as to mechanically move the image recording sensor in a manner to counteract the shake and to stabilize the image.

Claims

1. An image stabilization device comprising:

a plurality of micromechanical actuators;

a housing having at least two walls disposed substantially perpendicular to one another, at least two of the walls having at least one of the plurality of micromechanical actuators attached thereto and extending perpendicular therefrom;

an image recording sensor operative to generate image data of an image, the image recording sensor being at least partially surrounded by the housing, the image recording sensor being suspendedly mounted to the micromechanical actuators, the image recording sensor being movably and suspendedly attached to the housing by the actuators; and

a computational device operative to receive and process the image data generated by the image recording sensor, the computational device including a detection device operative to perform a shake detection image analysis on the image data to detect changes between two images in a time sequence of the image data and determine differential angles from the time sequence of the image data by tracking a position of a reference point between the two images in a time sequence of the image data so as to describe a present movement of the image recording sensor, the computational device being operative to immediately use the detected changes and determined differential angles between the two images describing the present movement of the image recording sensor to generate control signals for controlling the micromechanical actuators so as to mechanically move the image recording sensor in a manner to actively counteract the present shake and to stabilize the image.

2. The image stabilization device as recited in claim 1 wherein the image is generated using a hand-holdable mobile video telephone.

3. The image stabilization device as recited in claim 1 wherein the plurality of actuators includes a first group of the actuators, the actuators of the first group being operative to perform a linear movement along a respective longitudinal axis, the actuators of the first group being disposed with their respective longitudinal axes transverse to an image plane of the image recording sensor so that when actuated, the actuators of the first group tilt the image recording sensor in a direction transverse to a normal of the image plane.

4. The image stabilization device as recited in claim 3 wherein the actuators of the first group are disposed on a back side of the image recording sensor.

5. The image stabilization device as recited in claim 3 wherein the image recording sensor includes a CCD chip, the actuators of the first group being integrated into the CCD chip on a back side of the CCD chip.

6. The image stabilization device as recited in claim 3 wherein the first group of actuators includes at least three actuators disposed on a back side of the image recording sensor so that when actuated, the first group of actuators effect two-dimensional timing of the image recording sensor in a direction transverse to the normal of the image plane.

7. The image stabilization device as recited in claim 1 wherein the plurality of actuators includes a second group of actuators, the actuators of the second group each being operative to perform a linear movement along a respective longitudinal axis transverse to a normal of an image plane of the image recording sensor so as to rotate the image recording sensor about the normal of the image plane.

8. The image stabilization device as recited in claim 7 wherein the image recording sensor includes a plurality of side surfaces extending transversely to the image plane, the actuators of the second group being suspendedly attached to the side surfaces.

9. The image stabilization device as recited in claim 8 wherein:

the plurality of side surfaces includes four side surfaces;

the image recording sensor includes a substantially rectangular shape in the image plane; and

the second group of actuators includes at least four actuators each suspended at a respective end of a respective one of the plurality of side surfaces, each of the actuators of the second group being disposed with their respective longitudinal axes extending parallel to a respective adjacent side surface of the plurality of side surfaces so that when actuated, the at least four actuators effect rotation of the image recording sensor about the normal of the image plane, and

wherein a direction of the effected rotation is dependent upon a direction of the linear movement performed by each of the at least four actuators.

10. The image stabilization device as recited in claim 1 wherein the plurality of actuators includes a first and second group of actuators, the actuators of the first group being disposed perpendicularly to the actuators of the second group, so that when actuated, the actuators of the first group tilt the image recording sensor in a direction transverse to a normal of an image plane, and when actuated, the actuators of the second group rotate the image recording sensor about the normal of the image plane.

11. The image stabilization device as recited in claim 1 wherein the control signals include electrical control signals, an independent control signal being generated for each actuator.

12. The image stabilization device as recited in claim 1 wherein the actuators include piezoelectric elements that are directly driven by the control signals.

13. The image stabilization device as recited in claim 1 wherein the image recording sensor is directly attached to the housing by the actuators.

14. The image stabilization device as recited in claim 1 wherein the detection device is operative to make a distinction between wanted and unwanted movements based on a size of the differential angles and predefined decision criteria so that only unwanted movements are counteracted.

15. The image stabilization device as recited in claim 1 wherein the detection device is operative to perform a face recognition.

16. The image stabilization device as recited in claim 15 wherein detection device is operative to perform the face recognition by selecting a plurality of prominent facial features from a group of facial features including an eye, a nose, and a mouth of a face so as to assign a point of reference to a recognized facial feature, at least one reference point being calculated from the point of reference, the stabilizing the image being performed by tracking the at least one reference point in each image of the image data.

17. The image stabilization device as recited in claim 1 wherein the position is measured as a position vector of the reference point.

18. The image stabilization device as recited in claim 17 wherein a measured first position vector of the reference point is taken from a first recorded image of a video stream and a measured second position vector of the reference point is taken from a second recorded image of the video stream, a displacement vector being determined by subtracting either the first position vector or the second position vector from the other of the first position vector or the second position vector, the computational device using the displacement vector to generate the control signals.

19. The image stabilization device as recited in claim 1 wherein the computational device is operative to generate, using the detected changes and determined differential angles describing the present movement of the image recording sensor, control signals for controlling the micromechanical actuators so as to mechanically move the image recording sensor in a manner to actively counteract the present shake and to stabilize the image without electronically modifying the image data.

20. A method for image stabilization comprising:

providing a plurality of micromechanical actuators;

providing a housing having at least two walls disposed substantially perpendicular to one another, at least two of the walls having at least one of the plurality of micromechanical actuators attached thereto and extending perpendicular therefrom;

generating image data of an image using an image recording sensor suspendedly mounted to the micromechanical actuators, the image recording sensor being at least partially surrounded by the housing the image recording sensor being movably and suspendedly attached to the housing the actuators;

receiving and processing the generated image data using a computational device including a detection device by performing a shake detection image analysis on the image data to detect changes between two images in a time sequence of the image data and determine differential angles from the time sequence of the image data by tracking a position of a reference point between the two images in a time sequence of the image data so as to describe a present movement of the image recording sensor; and

immediately using the detected changes and determined differential angles between the two images describing the present movement of the image recording sensor to generate control signals for controlling the micromechanical actuators so as to mechanically move the image recording sensor in a manner to actively counteract the present shake and to stabilize the image.

21. The method as recited in claim 20 wherein the plurality of micromechanical actuators and the image recording sensor are disposed in a hand-holdable mobile video telephone.