Methods and System for Stabilizing Live Video in the Presence of Long-Term Image Drift

• US 20130083192A1
• Filed: 09/30/2011
• Published: 04/04/2013
• Est. Priority Date: 09/30/2011
• Status: Active Grant

• Current Assignee: Siemens Schweiz AG (Siemens AG)
• Original Assignee: Siemens Industry Incorporated (Siemens AG)
• Inventors: Bobick, Aaron, Bragg, Jeffrey, Strine, Lloyd
• Granted Patent: US 8,760,513 B2
• Time in Patent Office: Days
• US Class Current: 348/143
• CPC Class Codes: G06T 2207/30232 Surveillance
• CPC Class Codes: G06T 7/20 Analysis of motion motion estimation for coding, decoding, compressing or decompressing digital video signals H04N19/43, H04N19/51
• CPC Class Codes: H04N 23/683 performed by a processor, e.g. controlling the readout of an image memory
• CPC Class Codes: H04N 7/183 for receiving images from a single remote source

Abstract

Methods and systems stabilization of a camera image for short term or ‘pole shake’ and longer term ‘pole drift’ are provided. The camera is attached to a fixed structure. The pole drift is over periods of times long enough that the imagery can change significantly between a base reference frame and the current stabilized transient frame. A multitude of mapped frames and associated information is maintained. The slowly varying camera orientation (“pole drift”) is decoupled from the rapid motion jitter (“pole shake”), and separate transformations are calculated for each. Up to three transformations may be combined together for any one input frame. The separate transformations are combined together, so that only one full-blown image transformation computation is performed for each frame. A surveillance system applies a stabilization method.

Claims

1. A surveillance system, comprising:

one or more cameras for generating a plurality of frames of video;

a memory to store the plurality of frames of video;

a processor in communication with the memory to process each of the plurality of frames of video by;

receiving a frame in the plurality of frames of video;

determining a transient transformation which transforms the frame to a transient coordinate system related derived from a transient frame;

determining a base transformation which transforms the transient coordinate system to a base coordinate system derived from a base frame;

determining a combined transformation that combines the transient and the base transformation; and

performing the combined transformation on the frame.

2. The surveillance system of claim 1, further including the processor extracting a feature from the frame.

3. The surveillance system of claim 2, wherein the transient transformation transforms the feature in the frame to the feature in a transient frame.

4. The surveillance system of claim 1, wherein the transient frame is updated during operation of the surveillance system with the frame.

5. The surveillance system of claim 4, wherein the frame is selected as an updated transient frame based on a bias detection.

6. The surveillance system of claim 1, wherein the base frame transformation is determined during a setup phase of the surveillance system.

7. The surveillance system of claim 1, wherein the transient transformation corrects for high frequency effects such as pole shaking.

8. The surveillance system of claim 1, wherein the base frame transformation corrects for slower acting effects such as thermal variations.

9. The surveillance system of claim 5, wherein an updated transient frame cannot be transformed to the base frame and is stored in the memory for later processing.

10. The surveillance system of claim 1, wherein at least one of the one or more cameras is attached to a static structure which experiences environmentally caused movement which is not larger than a field-of-vision of the at least one of the one or more cameras.

11. The surveillance system of claim 4, wherein the combined transformation combines an updated transient transformation of the frame to the updated transient frame, a transformation of the updated transient frame to a previous transient frame and a transformation of the previous transient frame to the base frame.

12. A method of managing a surveillance system, comprising:

a processor receiving a frame in the plurality of frames of video;

the processor determining a transient transformation which transforms the frame to a transient coordinate system related derived from a transient frame;

the processor determining a base transformation which transforms the transient coordinate system to a base coordinate system derived from a base frame;

the processor determining a combined transformation that combines the transient and the base transformation; and

the processor performing the combined transformation on the frame.

13. The method of claim 12, further including the processor extracting a feature from the frame.

14. The method of claim 13, wherein the transient transformation transforms the feature in the frame to the feature in a transient frame.

15. The method of claim 12, wherein the transient frame is updated during operation of the surveillance system with the frame.

16. The method of claim 15, wherein the frame is selected as an updated transient frame based on a bias detection.

17. The method of claim 12, wherein the base frame transformation is determined during a setup phase of the surveillance system.

18. The method of claim 12, wherein the transient transformation corrects for high frequency effects such as pole shaking.

19. The method of claim 12, wherein the base frame transformation corrects for slower acting effects such as thermal variations.

20. The method system of claim 12, wherein at least one of the one or more cameras is attached to a static structure which experiences environmentally caused movement which is not larger than a field-of-vision of the at least one of the one or more cameras.