Enhanced processing for scanning video

• US 20070058717A1
• Filed: 09/09/2005
• Published: 03/15/2007
• Est. Priority Date: 09/09/2005
• Status: Abandoned Application

• Current Assignee: ObjectVideo Incorporated (Alarm.com Holdings, Inc.)
• Original Assignee: ObjectVideo Incorporated (Alarm.com Holdings, Inc.)
• Inventors: Yu, Li, Haering, Niels, Gupta, Himaanshu, Egnal, Geoffrey, Lipton, Alan, Chosak, Andrew, Brewer, Paul
• Application Number:
  US11/222,233
• Publication Number:
  US 20070058717A1
• Time in Patent Office: Days
• US Class Current: 375/240.80
• CPC Class Codes: G06T 2200/32 involving image mosaicing
• CPC Class Codes: G06T 2207/10016 Video; Image sequence
• CPC Class Codes: G06T 2207/20076 Probabilistic image processing
• CPC Class Codes: G06T 2207/30232 Surveillance
• CPC Class Codes: G06T 7/246 using feature-based methods, e.g. the tracking of corners or segments
• CPC Class Codes: G06T 7/33 using feature-based methods
• CPC Class Codes: G06V 10/16 using multiple overlapping images; Image stitching
• CPC Class Codes: G06V 10/24 Aligning, centring, orientation detection or correction of the image
• CPC Class Codes: G08B 13/19606 Discriminating between target movement or movement in an area of interest and other non-signicative movements, e.g. target movements induced by camera shake or movements of pets, falling leaves, rotating fan
• CPC Class Codes: H04N 23/698 for achieving an enlarged field of view, e.g. panoramic image capture

Abstract

A method of video processing may include registering one or more frames of input video received from a sensing unit, where the sensing unit may be capable of operating in a scanning mode. The registration process may project the frames onto a common reference. The method may further include maintaining a scene model corresponding to the sensing unit'"'"'s field of view. The method may also include processing the registered frames using the scene model, where the result of processing the registered frames includes visualization of at least one result of processing.

Claims

1. A method of video processing comprising:

registering one or more frames of input video received from a sensing unit, the sensing unit being capable of operation in a scanning mode, to project the frames onto a common reference and to obtain registered frames of the input video;

maintaining a scene model corresponding to said sensing unit'"'"'s field of view;

processing said registered frames of said input video to obtain processed video, said processing utilizing said scene model, wherein said processed video includes visualization of at least one result of said processing.

2. The method according to claim 1, further comprising:

estimating motion of said sensing unit.

3. The method according to claim 2, wherein said estimating motion is performed based on real-time telemetry data obtained from the sensing unit.

4. The method according to claim 2, wherein said estimating motion comprises:

using a translational model of motion between video frames.

5. The method according to claim 2, wherein said estimating motion comprises:

using an affine model of motion between video frames.

6. The method according to claim 2, wherein said estimating motion comprises:

using a perspective projection model of motion between video frames.

7. The method according to claim 2, wherein said estimating motion comprises performing at least two of the operations selected from the group consisting of:

using a translational model of motion between video frames;

using an affine model of motion between video frames; and

using a perspective projection model of motion between video frames.

8. The method according to claim 7, wherein said estimating motion further comprises:

downsampling video frames; and

wherein said estimating motion comprises performing at least one of said at least two selected operations upon a first set of downsampled video frames resulting from said downsampling.

9. The method according to claim 8, wherein said estimating motion comprises:

using said translational model of motion between video frames on said first set of downsampled video frames; and

using said affine model of motion between video frames on a second set of downsampled video frames that are downsampled by a factor less than said first set of downsampled video frames.

10. The method according to claim 9, wherein said using said affine model of motion between video frames utilizes as an initial estimate of sensing unit motion a result obtained from said using said translational model of motion between video frames.

11. The method according to claim 9, wherein said estimating motion further comprises:

using said perspective projection model of motion between video frames on the non-downsampled video frames.

12. The method according to claim 11, wherein said using said perspective projection model of motion between video frames utilizes as an initial estimate of sensing unit motion a result obtained from said using said affine model of motion between video frames.

13. The method according to claim 2, wherein said estimating motion of said sensing unit comprises:

computing a frame-to-frame motion estimate based on a current frame and a previous frame;

obtaining an approximation of said current frame by combining a projection of said previous frame onto a background mosaic with said frame-to-frame motion estimate; and

estimating a motion estimate correction based on said current frame and said approximation of said current frame.

14. The method according to claim 2, wherein said scene model includes statistical data about each pixel of a background model, and wherein said estimating motion of said sensing unit comprises choosing at least one reference point using said statistical data.

15. The method according to claim 2, wherein said scene model includes a scan path model, and wherein said estimating motion of said sensing unit comprises:

keeping track of at least one reference point used for estimating motion of said sensing unit; and

reusing at least one reference point previously used in estimating motion of said sensing unit when a position corresponding to said at least one reference point is reached along a scan path of said sensing unit.

16. The method according to claim 2, wherein said estimating motion of said sensing unit comprises:

selecting at least one feature of said input video frames;

matching said at least one feature between frames; and

fitting the results of said matching to a sensing unit model.

17. The method according to claim 1, wherein said scene model comprises:

a background model; and

at least one further model selected from the group consisting of;

a scan path model and a target model.

18. The method according to claim 1, further comprising:

detecting at least one target in said video based on said registered frames of said input video.

19. The method according to claim 18, wherein said detecting at least one target comprises:

segmenting said registered frames into foreground and background regions; and

performing blobization on said foreground regions to obtain one or more targets.

20. The method according to claim 19, wherein said segmenting, said performing blobization, or both use said scene model.

21. The method according to claim 19, wherein results of said segmenting, said performing blobization, or both are used to update said scene model.

22. The method according to claim 18, further comprising:

tracking at least one detected target.

23. The method according to claim 1, wherein said processing comprises:

detecting at least one of the group consisting of a scene event, a target characteristic, and a target activity.

24. The method according to claim 23, further comprising:

detecting and tracking at least one target in said video based on said registered frames of said input video; and

wherein said detecting at least one of the group consisting of a scene event, a target characteristic, and a target activity comprises;

analyzing the behavior of said at least one target.

25. The method according to claim 24, wherein said analyzing the behavior comprises:

classifying said at least one target.

26. The method according to claim 1, wherein said visualization includes at least one indication of at least one target in said processed video.

27. The method according to claim 26, wherein said indication comprises a bounding box.

28. The method according to claim 27, wherein said at least one bounding box includes a feature to indicate a characteristic of said at least one target.

29. The method according to claim 26, wherein said indication comprises an icon.

30. The method according to claim 29, wherein said icon includes a feature to indicate a characteristic of said target.

31. The method according to claim 1, wherein said visualization includes at least one indication of aging of video frames in said processed video.

32. The method according to claim 1, wherein said visualization includes at least one indication of a current view of said sensing unit relative to at least a portion of the entire field-of-view of said sensing unit.

33. A machine-accessible medium containing software that when executed by a processor causes said processor to execute the method of video processing according to claim 1.

34. The machine-accessible medium according to claim 33, further containing software that when executed by said processor causes the method to further include:

estimating motion of said sensing unit, wherein said registering uses a result of said estimating motion; and

detecting and tracking at least one target, wherein said visualization includes at least one indication of said at least one target.

35. The machine-accessible medium according to claim 33, wherein said visualization includes at least one indication of a current view of said sensing unit relative to at least a portion of the entire field-of-view of said sensing unit.

36. A method of estimating motion of a sensing unit based on video frames provided by said sensing unit, the method comprising performing at least two of the operations selected from the group consisting of:

using a translational model of motion between video frames;

using an affine model of motion between video frames; and

using a perspective projection model of motion between video frames.

37. The method according to claim 36, wherein said estimating motion further comprises:

downsampling video frames; and

wherein said estimating motion comprises performing at least one of said at least two selected operations upon a first set of downsampled video frames resulting from said downsampling.

38. The method according to claim 37, wherein said estimating motion comprises:

using said translational model of motion between video frames on said first set of downsampled video frames; and

using said affine model of motion between video frames on a second set of downsampled video frames that are downsampled by a factor less than said first set of downsampled video frames.

39. The method according to claim 38, wherein said using said affine model of motion between video frames utilizes as an initial estimate of sensing unit motion a result obtained from said using said translational model of motion between video frames.

40. The method according to claim 38, wherein said estimating motion further comprises:

using said perspective projection model of motion between video frames on the non-downsampled video frames.

41. The method according to claim 40, wherein said using said perspective projection model of motion between video frames utilizes as an initial estimate of sensing unit motion a result obtained from said using said affine model of motion between video frames.

42. The method according to claim 36, further comprising:

computing a frame-to-frame motion estimate based on a current frame and a previous frame;

obtaining an approximation of said current frame by combining a projection of said previous frame onto a background mosaic with said frame-to-frame motion estimate; and

estimating a motion estimate correction based on said current frame and said approximation of said current frame.

43. The method according to claim 36, further comprising choosing at least one reference point using statistical data about each pixel of a background model.

44. The method according to claim 36, further comprising:

keeping track of at least one reference point used for estimating motion of said sensing unit; and

reusing at least one reference point previously used in estimating motion of said sensing unit when a position corresponding to said at least one reference point is reached along a scan path of said sensing unit.

45. The method according to claim 36, further comprising:

selecting at least one feature of said input video frames;

matching said at least one feature between frames; and

fitting the results of said matching to a sensing unit model.

46. A video processing system comprising:

at least one sensing device to be operated in a scanning mode;

a video processor coupled to said at least one scanning device to receive video frames from said at least one sensing device, the video processor to register said video frames, to maintain at least one scene model corresponding to said video frames, and to process said video frames based on said at least one scene model; and

a monitoring device coupled to said video processor, wherein said video processor visualizes at least one result of processing said video frames on said monitoring device.

47. The video processing system according to claim 46, wherein said monitoring device is to perform at least one of the tasks selected from the group consisting of:

displaying video in real-time;

transmitting video across a network to enable remote viewing; and

storing video to enable delayed playback.

48. The video processing system according to claim 46, wherein said sensing device comprises means for increasing an image quality obtained by said sensing device.