Image stabilisation system and method

• US 20060061658A1
• Filed: 12/04/2003
• Published: 03/23/2006
• Est. Priority Date: 12/13/2002
• Status: Active Grant

• Current Assignee: Qinetiq Limited (QinetiQ Group PLC)
• Original Assignee: Qinetiq Limited (QinetiQ Group PLC)
• Inventors: Kent, Philip John, Faulkner, David Andrew Alexander, Fretwell, Paul, Price, Gary Shaun
• Granted Patent: US 7,609,293 B2
• Time in Patent Office: Days
• US Class Current: 348/207.100
• CPC Class Codes: H04N 23/68 for stable pick-up of the scene, e.g. compensating for camera body vibrations
• CPC Class Codes: H04N 23/6811 based on the image signal
• CPC Class Codes: H04N 23/6815 by distinguishing pan or tilt from motion
• CPC Class Codes: H04N 23/683 performed by a processor, e.g. controlling the readout of an image memory
• CPC Class Codes: H04N 5/145 Movement estimation for video coding H04N19/51

Abstract

A system for stabilising video signals generated by a video camera which may be mounted in an unstable manner includes a digital processing means for manipulation of each incoming image that attempts to overlay features of the current image onto similar features of a previous image. A mask is used that prevents parts of the image that are likely to cause errors in the overlaying process from being used in the calculation of the required movement to be applied to the image. The mask may include areas where small movements of the image have been detected, and many also include areas where image anomalies including excess noise have been detected. The invention also discloses means for dealing with wanted movements of the camera, such as pan or zoom, and also discloses means for dealing with the edges of video signals as processed by the system. A method is also disclosed.

Claims

1. A video image stabilisation system for correction of camera motion, that is arranged to receive one or more signals representative of a plurality of images from an image source wherein, for an image n following at least an image (n−

1) and an image (n−

2) the system is arranged to estimate a Global Motion Offset (GMO) value between image n and a previous image representative of the spatial separation between the scene imaged in image n and the previous image, and apply a corrective movement to the image n based upon this GMO, characterised in that;

the system is arranged to estimate the GMO for the image n with reference to a mask that represents a region or regions of the image n not to be considered in the GMO estimation, the region(s) being region(s) estimated as likely to mislead the estimation of the GMO.

2. A stabilisation system as claimed in claim 1 wherein the system is arranged to examine one or more local regions of the image n and corresponding local regions of a previous image, and estimate a local motion offset (LMO) representative of spatial separation between like features in corresponding local regions of the current and previous images, and if the, or each, LMO is greater than a given threshold, to set area(s) of the mask that correspond to this local region or regions to indicate omission from the GMO estimation.

3. A stabilisation system as claimed in claim 2 wherein the local regions comprise an array of rectangular regions.

4. A stabilisation system as claimed in claim 1 wherein the system is arranged to estimate the GMO of an image representative of image n but having a spatial resolution lower than image n.

5. A stabilisation system as claimed in claim 4 wherein the system is arranged to iterate the estimation of the GMO on a plurality of images each representative of image n, where each of the plurality of images has a different spatial resolution.

6. A stabilisation system as claimed in claim 1 wherein the system is arranged to adjust the GMO if a stationary camera state is detected, this state being indicated by means of a plurality of contiguous GMOs including the current GMO all being below a given threshold.

7. A stabilisation system as claimed in claim 2 wherein the system is arranged to adjust the GMO if intentional adjustment of the image source viewing direction (pan) or field of view (zoom) is detected.

8. A stabilisation system as claimed in claim 7 wherein the system is arranged to detect a pan of the image source by means of low-pass filtering GMO values from at least a sequence of previous images at a cut-off frequency lower than that expected from unintentional camera movements.

9. A stabilisation system as claimed in claim 7 wherein a zoom is detected if a number x of LMOs examined for image n all show a direction of movement in towards a central region of the image n, or all show a direction of movement away from a central region of the image n, the number x being greater than some given threshold.

10. A stabilisation system as claimed in claim 9 wherein the threshold is 50% of those LMOs examined, and the number x is proportional to the absolute difference between the number of those LMOs examined showing a direction of movement in towards a central region of the image n, and those LMOs examined showing a direction of movement away from a central region of the image n.

11. A stabilisation system as claimed in claim 9 wherein the LMOs examined are taken from those local regions that are substantially adjacent the edge of image n.

12. A stabilisation system as claimed in claim 1 wherein the system is arranged to generate a border on at least one edge of the image n, the border being adjustable in size such that it covers any blank space between the edge of image n and the corresponding edge of a display area on which the image n is displayed.

13. A stabilisation system as claimed in claim 12 wherein the system is arranged to adjust the border size on at least one edge of the image n such that it also covers an area on image n corresponding to blank space present on one or more previous images.

14. A stabilisation system as claimed in claim 12 wherein the border generated by the system comprises of image data from one or more previous images.

15. A stabilisation system as claimed in claim 1 wherein the system is arranged to scale the image n, such that it covers any blank space between the edge of image n and the corresponding edge of a display area on which the image n is displayed.

16. A stabilisation system as claimed in claim 1 wherein anomalous pixels of the image n are used to set corresponding pixels of the mask such that they are excluded from the estimation of the GMO.

17. A stabilisation system as claimed in claim 16 wherein the pixels above a threshold in an image comprising the absolute difference between the image n and a previous image m, both images n and m having had corrective movements applied, are regarded as anomalous.

18. A stabilisation system as claimed in claim 1 wherein the system is arranged to multiply the calculated GMO, as adjusted in any other operation, by a decay constant factor lying between 0 and 1 before shifting the image n.

19. A method of stabilising a present image relative to at least one previous image where both current and previous images are part of a sequence of video images represented by an electronic signal, comprising the steps of:

i. estimating a global motion offset (GMO) between the current and previous image representative of the spatial separation between the scene imaged in the current image and that imaged in the previous image; and

ii. applying a corrective movement to the current image based upon the GMO;

characterised in that;

a mask image is used in estimating the GMO, the mask image representing a region or regions of the current image not to be considered in the GMO estimation, the region(s) being region(s) being estimated as likely to mislead the estimation of the GMO.

20. A method as claimed in claim 19 wherein the method further includes the step of examining one or more local regions of the current image and corresponding local regions of a previous image, and estimating a local motion offset (LMO) representing the spatial separation between like features in corresponding local regions of the current and previous images, and if the, or each, LMO is greater than a given threshold, setting area(s) of the mask that correspond to this local region or regions to indicate omission from the GMO estimation.

21. A computer program designed to run on a computer and arranged to implement a video image stabilisation system, the system being arranged to receive as an input a digital signal representative of a plurality of images from an image source wherein, for an image n following at least an image (n−

1) and an image (n−

2) the system is arranged to estimate a Global Motion Offset (GMO) value between image n and a previous image representative of the spatial separation between the scene imaged in image n and the previous image, and apply a corrective movement to the image n based upon this GMO, characterised in that;

the system is arranged to estimate the GMO for the image n with reference to a mask that represents a region or regions of the image n not to be considered in the GMO estimation, the region(s) being region(s) estimated as likely to mislead the estimation of the GMO.