Scene preset identification using quadtree decomposition analysis

• US 9,805,271 B2
• Filed: 08/18/2009
• Issued: 10/31/2017
• Est. Priority Date: 08/18/2009
• Status: Active Grant

• Current Assignee: Intellective Ai, Inc.
• Original Assignee: Omni AI, Inc.
• Inventors: Cobb, Wesley Kenneth, Blythe, Bobby Ernest, Gottumukkal, Rajkiran Kumar, Saitwal, Kishor Adinath, Xu, Gang, Yang, Tao
• Primary Examiner(s): Holder, Anner N
• Application Number:
  US12/543,223
• Publication Number:
  US 20110043625A1
• Time in Patent Office: 2,996 Days
• Field of Search: 348143, 348169, 382209, 375240
• CPC Class Codes: G06T 2207/20081 Training; Learning
• CPC Class Codes: G06T 2207/30232 Surveillance
• CPC Class Codes: G06T 7/254 involving subtraction of images
• CPC Class Codes: G06V 20/52 Surveillance or monitoring of activities, e.g. for recognising suspicious objects recognising microscopic objects G06V20/69

Abstract

Techniques are disclosed for matching a current background scene of an image received by a surveillance system with a gallery of scene presets that each represent a previously captured background scene. A quadtree decomposition analysis is used to improve the robustness of the matching operation when the scene lighting changes (including portions containing over-saturation/under-saturation) or a portion of the content changes. The current background scene is processed to generate a quadtree decomposition including a plurality of window portions. Each of the window portions is processed to generate a plurality of phase spectra. The phase spectra are then projected onto a corresponding plurality of scene preset image matrices of one or more scene preset. When a match between the current background scene and one of the scene presets is identified, the matched scene preset is updated. Otherwise a new scene preset is created based on the current background scene.

Claims

1. A computer-implemented method for identifying a scene preset for a background scene depicted in a stream of video images captured via a video camera, comprising:

receiving the background scene;

generating a quadtree decomposition of the background scene, the quadtree decomposition including a plurality of window portions of the background scene;

determining if the background scene matches one of a plurality of stored scene presets, each representing a previously captured background scene determined for a distinct field-of-view of the camera, based on the plurality of window portions of the background scene;

upon determining the background scene matches the stored scene preset that represents the previously captured background scene;

updating the stored scene preset, and

restoring a learning state associated with the stored scene preset used by a machine-learning engine;

upon determining the background scene does not match the stored scene preset that represents the previously captured background scene, creating a new scene preset;

computing a phase spectrum for each window portion of the plurality of window portions of the background scene;

projecting the phase spectra onto corresponding preset image matrices of a quadtree decomposition of the stored scene preset; and

computing a reconstruction score for the background scene that indicates how closely the background scene matches the previously captured background scene.

2. The computer-implemented method of claim 1, wherein the computing of the reconstruction score comprises:

selecting the best reconstruction scores for each of the plurality of window portions of the background scene; and

computing the reconstruction score as a weighted sum of the best reconstruction scores.

3. The computer-implemented method of claim 1, wherein the updating of the stored scene preset when the background scene matches the stored scene preset comprises:

replacing phase spectra of an oldest background scene that was used to compute the stored scene preset with the phase spectra for each window portion of the plurality of window portions of the background scene; and

recomputing the preset image matrices of a quadtree decomposition of the stored scene preset to produce an updated scene preset that incorporates the background scene.

4. The computer-implemented method of claim 1, wherein the background scene is determined to match the stored scene preset based on a minimum error threshold.

7. The computer-implemented method of claim 1, wherein the creating of the new scene preset when the background scene does not match the stored scene preset comprises:

computing a phase spectrum for each window portion of the plurality of window portions of background scenes in a sequence of background scenes that includes the background scene; and

performing an analysis to produce image matrices that represent the background scene and the additional background scenes in the sequence.

8. The computer-implemented method of claim 1, wherein each one of the plurality of window portions of the background scene has an equal pixel resolution.

5. A computer-implemented method for identifying a scene preset for a background scene depicted in a stream of video images captured via a video camera, comprising:

receiving the background scene;

generating a quadtree decomposition of the background scene, the quadtree decomposition including a plurality of window portions of the background scene;

determining that a first window portion of the quadtree decomposition of the background scene is over-saturated;

discarding the first window portion of the quadtree decomposition of the background scene;

determining if the background scene matches one of a plurality of stored scene presets, each representing a previously captured background scene determined for a distinct field-of-view of the camera, based on the plurality of window portions of the background scene without the first window portion;

upon determining the background scene matches the stored scene preset that represents the previously captured background scene;

updating the stored scene preset, and

restoring a learning state associated with the stored scene preset used by a machine-learning engine; and

upon determining the background scene does not match the stored scene preset that represents the previously captured background scene, creating a new scene preset.

6. A computer-implemented method for identifying a scene preset for a background scene depicted in a stream of video images captured via a video camera, comprising:

receiving the background scene;

generating a quadtree decomposition of the background scene, the quadtree decomposition including a plurality of window portions of the background scene;

determining that a first window portion of the quadtree decomposition of the background scene is under-saturated;

discarding the first window portion of the quadtree decomposition of the background scene;

determining if the background scene matches one of a plurality of stored scene presets, each representing a previously captured background scene determined for a distinct field-of-view of the camera, based on the plurality of window portions of the background scene without the first window portion;

upon determining the background scene matches the stored scene preset that represents the previously captured background scene;

updating the stored scene preset, and

restoring a learning state associated with the stored scene preset used by a machine-learning engine; and

upon determining the background scene does not match the stored scene preset that represents the previously captured background scene, creating a new scene preset.

9. A non-transitory computer-readable storage medium containing a program which, when executed by a processor, performs an operation for identifying a scene preset for a background scene of in a stream of video images captured via a video camera, the operation comprising:

receiving the background scene;

generating a quadtree decomposition of the background scene, the quadtree decomposition including a plurality of window portions of the background scene; and

determining if the background scene matches one of a plurality of stored scene presets, each representing a previously captured background scene determined for a distinct field-of-view of the camera, based on the plurality of window portions of the background scene;

upon determining the background scene matches the stored scene preset that represents the previously captured background scene;

updating the stored scene preset;

restoring a learning state associated with the stored scene preset used by a machine-learning engine;

upon determining the background scene does not match the stored scene preset that represents the previously captured background scene, creating a new scene preset;

computing a phase spectrum for each window portion of the plurality of window portions of the background scene;

projecting the phase spectra onto corresponding preset image matrices of a quadtree decomposition of the stored scene preset; and

computing a reconstruction score for the background scene that indicates how closely the background scene matches the stored scene preset that represents the previously captured background scene.

10. The computer-readable storage medium of claim 9, wherein the updating of the stored scene preset when the background scene matches the stored scene preset comprises:

replacing phase spectra of an oldest background scene that was used to compute the stored scene preset with the phase spectra for each window portion of the plurality of window portions of the background scene; and

recomputing the preset image matrices of a quadtree decomposition of the stored scene preset to produce an updated scene preset that incorporates the background scene.

12. The computer-readable storage medium of claim 9, wherein the creating of the new scene preset upon determining the background scene does not match the stored scene preset, comprises:

computing a phase spectrum for each window portion of the plurality of window portions of background scenes in a sequence of background scenes that includes the background scene; and

performing an analysis to produce image matrices that represent the background scene and the additional background scenes in the sequence.

11. A non-transitory computer-readable storage medium containing a program which, when executed by a processor, performs an operation for identifying a scene preset for a background scene of a stream of video images captured via a video camera, the operation comprising:

receiving the background scene;

generating a quadtree decomposition of the background scene, the quadtree decomposition including a plurality of window portions of the background scene;

determining that a first window portion of the quadtree decomposition of the background scene is over-saturated or under-saturated;

discarding the first window portion of the quadtree decomposition of the background scene;

determining if the background scene matches one of a plurality of stored scene presets, each representing a previously captured background scene determined for a distinct field-of-view of the camera, based on the plurality of window portions of the background scene without the first window portion;

upon determining the background scene matches the stored scene preset that represents the previously captured background scene;

updating the stored scene preset, and

restoring a learning state associated with the stored scene preset used by a machine-learning engine;

upon determining the background scene does not match the stored scene preset that represents the previously captured background scene, creating a new scene preset.

13. A system, comprising:

a video input source configured to capture images;

a processor; and

a memory containing a program, which, when executed on the processor is configured to perform an operation for identifying a scene preset for a background scene of a stream of video images captured by the video input source, the operation comprising;

receiving the background scene,

generating a quadtree decomposition of the background scene, the quadtree decomposition including a plurality of window portions of the background scene,

determining if the background scene matches one of a plurality of stored scene presets, each representing a previously captured background scene determined for a distinct field-of-view of the video input source, based on the plurality of window portions of the background scene,

upon determining the background scene matches the stored scene preset that represents the previously captured background scene;

updating the scene preset; and

restoring a learning state associated with the stored scene preset used by a machine-learning engine;

upon determining the background scene does not match the stored scene preset that represents the previously captured background scene, creating a new scene preset,

computing a phase spectrum for each window portion of the plurality of window portions of the background scene,

projecting the phase spectra onto corresponding preset image matrices of a quadtree decomposition of the stored scene preset, and

computing a reconstruction score for the background scene that indicates how closely the background scene matches the previously captured background scene.

14. The system of claim 13, wherein the computing of the reconstruction score comprises:

selecting the best reconstruction scores for each of the plurality of window portions of the background scene; and

computing the reconstruction score as a weighted sum of the best reconstruction scores.

15. The system of claim 13, wherein the updating of the stored scene preset when the background scene matches the stored scene preset comprises:

replacing phase spectra of an oldest background scene that was used to compute the stored scene preset with the phase spectra for each window portion of the plurality of window portions of the background scene; and

recomputing the preset image matrices of a quadtree decomposition of the stored scene preset to produce an updated scene preset that incorporates the background scene.

16. The system of claim 13, wherein the background scene is determined to match the stored scene preset based on a minimum error threshold.

17. The system of claim 13, wherein the creating of the new scene preset when the background scene does not match the background scene comprises:

computing a phase spectrum for each window portion of the plurality of window portions of background scenes in a sequence of background scenes that includes the background scene; and

performing an analysis to produce image matrices that represent the background scene and the additional background scenes in the sequence.

18. A system, comprising:

a video input source configured to capture images;

a processor; and

a memory containing a program, which, when executed on the processor is configured to perform an operation for identifying a scene preset for a background scene of a stream of video images captured by the video input source, the operation comprising;

receiving the background scene;

generating a quadtree decomposition of the background scene, the quadtree decomposition including a plurality of window portions of the background scene;

determining that a first window portion of the quadtree decomposition of the background scene is over-saturated;

discarding the first window portion of the quadtree decomposition of the background scene;

determining if the background scene matches one of a plurality of stored scene presets, each representing a previously captured background scene determined for a distinct field-of-view of the camera, based on the plurality of window portions of the background scene without the first window portion;

upon determining the background scene matches the stored scene preset that represents the previously captured background scene;

updating the scene preset, and

restoring a learning state associated with the stored scene preset used by a machine-learning engine; and

upon determining the background scene does not match the stored scene preset that represents the previously captured background scene, creating a new scene preset.

19. A system, comprising:

a video input source configured to capture images;

a processor; and

a memory containing a program, which, when executed on the processor is configured to perform an operation for identifying a scene preset for a background scene of a stream of video images captured by the video input source, the operation comprising;

receiving the background scene;

generating a quadtree decomposition of the background scene, the quadtree decomposition including a plurality of window portions of the background scene;

determining that a first window portion of the quadtree decomposition of the background scene is under-saturated;

discarding the first window portion of the quadtree decomposition of the background scene;

determining if the background scene matches one of a plurality of stored scene presets, each representing a previously captured background scene determined for a distinct field-of-view of the camera, based on the plurality of window portions of the background scene without the first window portion;

upon determining the background scene matches the stored scene preset that represents the previously captured background scene;

updating the scene preset, and

restoring a learning state associated with the stored scene preset used by a machine-learning engine; and

upon determining the background scene does not match the stored scene preset that represents the previously captured background scene, creating a new scene preset.