SEARCHING RECORDED VIDEO

• US 20120170803A1
• Filed: 12/30/2010
• Published: 07/05/2012
• Est. Priority Date: 12/30/2010
• Status: Active Grant

• Current Assignee: Pelco Incorporated (Motorola Solutions, Inc.)
• Original Assignee: Pelco Incorporated (Motorola Solutions, Inc.)
• Inventors: Millar, Greg, Aghdasi, Farzin, Wang, Lei
• Granted Patent: US 9,171,075 B2
• Time in Patent Office: Days
• US Class Current: 382/103
• CPC Class Codes: G06F 16/783 using metadata automatically derived from the content
• CPC Class Codes: G06F 16/7837 using objects detected or recognised in the video content
• CPC Class Codes: G06V 10/273 removing elements interfering with the pattern to be recognised
• CPC Class Codes: G06V 20/52 Surveillance or monitoring of activities, e.g. for recognising suspicious objects recognising microscopic objects G06V20/69

Abstract

Embodiments of the disclosure provide for systems and methods for creating metadata associated with a video data. The metadata can include data about objects viewed within a video scene and/or events that occur within the video scene. Some embodiments allow users to search for specific objects and/or events by searching the recorded metadata. In some embodiments, metadata is created by receiving a video frame and developing a background model for the video frame. Foreground object(s) can then be identified in the video frame using the background model. Once these objects are identified they can be classified and/or an event associated with the foreground object may be detected. The event and the classification of the foreground object can then be recorded as metadata.

Claims

1. A method for creating metadata associated with a video frame, the method comprising:

receiving a video frame;

developing a background model for the video frame;

separating a foreground object from the video frame using the background model;

classifying the foreground object into a classification, wherein the classification characterizes the foreground object based on the object color, aspect ratio, and position; and

recording the classification of the foreground object in metadata.

2. The method according to claim 1, wherein background model comprises a short-tern background model.

3. The method according to claim 2, wherein background model further comprises a long-term background model used in conjunction with the short-term background model.

4. The method according to claim 1, wherein background model comprises a long-term background model.

5. The method according to claim 1 further comprising detecting and removing shadows from an image frame.

6. The method according to claim 1 further comprising detecting and removing reflections from an image frame.

7. The method according to claim 1 further comprising:

detecting an event associated with the foreground object; and

recording the event in metadata.

8. The method according to claim 7, wherein the foreground object is classified by one or more characteristics selected from the group consisting of location within the video frame, object size, object color, object type, and object aspect ratio.

9. The method according to claim 1, wherein the detected event is an event selected from the group consisting of the object entering a frame, the object leaving the frame, camera sabotage, the object holding still, the object merging with another object, the object splitting into two objects, the object entering a zone, the object leaving a zone, the object crossing a tripwire, the object being removed, the object being abandoned, object counting, object loitering, and object timer.

10. The method according to claim 1, wherein the method occurs at a video camera.

11. The method according to claim 1, wherein the foreground object is further classified by object type.

12. The method according to claim 1, wherein the foreground object is further classified by object velocity.

13. The method according to claim 1, wherein the background model is constructed using a probability Gaussian model.

14. The method according to claim 1, wherein separating the foreground object from the video frame comprises subtracting the background model from the video frame to return a foreground object.

15. The method according to claim 1, wherein the classification data associated with the object are recorded as metadata separate from the video data.

16. A method for creating metadata about a video frame, the method comprising:

receiving a video frame;

developing a background model for the video frame;

separating a foreground object from the video frame using the background model;

identifying the foreground object as a previously identified foreground object;

classifying the foreground object into a classification based on characteristics of the foreground object, wherein the characteristics comprise object location within the video frame and object color; and

recording the classification of the object in metadata.

17. The method according to claim 16, wherein the video frame comprises a plurality of video frames.

18. The method according to claim 16, wherein the detecting the event of the foreground object includes tracking the foreground object from previous video frames.

19. The method according to claim 16 further comprising determining an event of the foreground object.

20. The method according to claim 19, wherein the event is determined based at least in part on the classification of the object.

21. The method according to claim 19, wherein the event is determined based at least in part on a change in the classification of the object from a classification of the object in a previous frame.

22. The method according to claim 19 further comprising detecting and removing shadows from an image frame.

23. The method according to claim 19 further comprising detecting and removing reflections from an image frame.

24. A video camera system comprising:

a video camera for capturing video data of a scene and outputting the video data as digital data;

a processor communicatively coupled to the camera, wherein the processor is configured to

receive video data from the video camera,

develop a background model for the video data,

separate a foreground object from the video data using the background model,

identify the foreground object'"'"'s location within the a video scene of the video data,

determine the foreground object'"'"'s color, and

record the foreground objects location and color in metadata; and

an output port communicatively coupled to the processor and configured to output the metadata.

25. The video camera system according to claim 24, wherein background model comprises a short-term background model.

26. The video camera system according to claim 24, wherein background model further comprises a long-term background model used in conjunction with the short-term background model.

27. The video camera system according to claim 24, wherein background model comprises a long-term background model.

28. The video camera system according to claim 24, wherein the output port is configured to output the video data separate from the metadata.

29. The video camera system according to claim 24, wherein the processor is further configured to compress the video data according to a video compression algorithm.

30. The video camera system according to claim 24, wherein the processor is further configured to determine the speed of the foreground object and record the speed of the foreground object in metadata.

31. The video camera system according to claim 24, wherein the processor is further configured to determine the aspect ratio of the foreground object and record the aspect ratio of the foreground object in metadata.

32. A video processing system comprising:

a video input port configured to receive video data;

a processor communicatively coupled to the video input port and configured to;

develop a background model for the video data,

separate a foreground object from the video data using the background model,

determine the relative location of the foreground object, and

determine the relative color of the foreground object; and

a video output port communicatively coupled to the processor and configured to output the relative location and the color of the foreground object as metadata.

33. The video processing system according to claim 32, wherein the output port is configured to output the video data separate from the metadata.