Object selective video recording

• US 20060165386A1
• Filed: 03/24/2006
• Published: 07/27/2006
• Est. Priority Date: 01/08/2002
• Status: Abandoned Application

• Current Assignee: Cernium Corporation (Kastle Systems International LLC)
• Original Assignee: Cernium Corporation (Kastle Systems International LLC)
• Inventors: Garoutte, Maurice V.
• Application Number:
  US11/388,505
• Publication Number:
  US 20060165386A1
• Time in Patent Office: Days
• US Class Current: 386/210
• CPC Class Codes: G08B 13/19604 involving reference image or background adaptation with time to compensate for changing conditions, e.g. reference image update on detection of light level change
• CPC Class Codes: G08B 13/19608 Tracking movement of a target, e.g. by detecting an object predefined as a target, using target direction and or velocity to predict its new position
• CPC Class Codes: G08B 13/19667 Details realated to data compression, encryption or encoding, e.g. resolution modes for reducing data volume to lower transmission bandwidth or memory requirements
• CPC Class Codes: G08B 13/19673 Addition of time stamp, i.e. time metadata, to video stream
• CPC Class Codes: H04N 19/132 Sampling, masking or truncation of coding units, e.g. adaptive resampling, frame skipping, frame interpolation or high-frequency transform coefficient masking
• CPC Class Codes: H04N 19/137 Motion inside a coding unit, e.g. average field, frame or block difference
• CPC Class Codes: H04N 19/146 Data rate or code amount at the encoder output
• CPC Class Codes: H04N 19/156 Availability of hardware or computational resources, e.g. encoding based on power-saving criteria
• CPC Class Codes: H04N 19/17 the unit being an image region, e.g. an object
• CPC Class Codes: H04N 19/172 the region being a picture, frame or field
• CPC Class Codes: H04N 19/20 using video object coding
• CPC Class Codes: H04N 19/23 with coding of regions that are present throughout a whole video segment, e.g. sprites, background or mosaic
• CPC Class Codes: H04N 5/915 for field- or frame-skip recording or reproducing
• CPC Class Codes: H04N 7/18 Closed-circuit television [CCTV] systems, i.e. systems in which the video signal is not broadcast
• CPC Class Codes: H04N 9/8042 involving data reduction

Abstract

Object selective video analysis and recordation system in which video camera output is recorded media with reduction of the amount of the recording media, with preservation of intelligence content of images of objects appearing against a background scene. Preset knowledge of symbolic categories of scene objects and analysis of object attributes is provided. Spatial resolution and temporal resolution of objects are automatically varied per preset criteria based on predetermined interest in object attributes while recording both background and object video. A system user can query recorded images by content to recall data according to specified symbolic content. So-called intelligent pruning allows changes in criteria for data storage or archiving to “prune” (cull or remove data) based upon such changes in criteria. Under software control, the system carries out pruning “after-the-fact,” i.e., after data has previously been identified by the system as significant enough for storage or archive.

Claims

1. In a system having video camera apparatus providing large amounts of output video which must be recorded in a useful form on recording media in order to preserve the content of such images, the video output consisting of background video and object video representing images of objects appearing against the background, said system including a video separator which separates the video output into background video and object video, and an analyzer which analyzes the object video for content according to different possible objects in the images and different possible kinds of object attributes which define the symbolic content of the object, improvement comprising:

video processing apparatus for reducing the amount of video actually recorded so as to reduce the amount of recording media used therefor,

a storage control which independently stores the background and object video while compressing both the background and object video according to at least one suitable compression algorithm,

wherein;

the object video is recorded while varying the frame rate of the recorded object video in accordance with the different possible objects or different kinds of object behavior, or both said different kinds of objects and different kinds of object behavior, the frame rate having a preselected value at any given time corresponding to the different possible objects which value is not less than will provide a useful image of the respective different possible objects when recovered from storage; and

the object video is compressed while varying the compression ratio so that it has a value at any given corresponding to the different possible objects or different kinds of object behavior, or both said different kinds of objects and different kinds of object behavior, the compression ratio at any given time having a preselected value not greater than will provide a useful image of the respective different possible objects when recovered from storage; and

video recovery and presentation provision to present the stored object and background video by reassembling the recorded background and the recorded object video for viewing.

2. In a system according to claim 1, the improvement further comprising provision for recording the background video at a frame rate or resolution different from a frame rate or resolution at which the object video is recorded.

3. In a system according to claim 2, the improvement further comprising provision for recording the background video at a frame rate which is less than a frame rate at which the object video is recorded.

4. In a system as set forth in claim 1, the analysis circuitry providing computed knowledge of any one or more of at least the following preselected possible object attributes:

(a) categorical object content;

(b) characteristic object features; and

(c) behavior of said objects.

9. The combination as set forth in claim 3 wherein said object attributes include one or more of the following:

(a) categorical object content;

(b) characteristic object features; and

(c) behavior of said objects.

5. In a system having video camera apparatus providing large amounts of output video which must be recorded in a useful form on recording media in order to preserve the content of such images, the video output consisting of background video and object video representing images of objects appearing against the background, and wherein said system comprises an analysis worker module which separates the video output into background video and object video,

the analysis worker module analyzing the object video for content according to different possible objects in the images and different possible kinds of object attributes, the improvement comprising:

video processing apparatus for reducing the amount of video actually recorded so as to reduce the amount of recording media used therefor, said apparatus including;

a storage control which independently stores the background and object video while compressing both the background and object video according to at least one suitable compression algorithm,

wherein;

the object video is recorded while varying the frame rate of the recorded object video in accordance with the different possible objects or different kinds of object attributes, or both said different kinds of objects and different kinds of object attributes, the frame rate having a preselected value at any given time corresponding to the different possible objects which value is not less than will provide a useful image of the respective different possible objects when recovered from storage; and

the object video is compressed while varying the compression ratio so that it has a value at any given corresponding to the different possible objects or different kinds of object behavior, or both said different kinds of objects and different kinds of object behavior, the compression ratio at any given time having a preselected value not greater than will provide a useful image of the respective different possible objects when recovered from storage;

video recovery and presentation provision for presentation of the stored object and background video by reassembling the recorded background and the recorded object video for viewing, the improvement further comprising provision for allowing a user of the system to query recorded video images by content by enabling the user to recall recorded data according to different possible objects in the images and different possible kinds of object attributes

6. In a system according to claim 5, the improvement further comprising provision for recording the background video at a frame rate which is less than a frame rate at which the object video is recorded.

7. In a system as set forth in claim 5, the analysis worker module providing computed knowledge of any one or more of at least the following preselected possible object attributes:

(a) categorical object content;

(b) characteristic object features; and

(c) behavior of said objects;

8. In combination, a system having video camera apparatus providing output video to be recorded in a useful form on recording media in order to preserve intelligence content of such images;

the video output consisting of background video and object video representing images of objects;

the system providing computed knowledge of one or more object attributes of said objects;

the system providing for varying either the spatial resolution or temporal resolution of the objects, or both said spatial resolution and said temporal resolution, based on predetermined interest in any one or more of said object attributes.

10. In combination, a system having video camera apparatus providing output video to be recorded in a useful form on recording media in order to preserve intelligence content of such images,

the video output consisting of background video and object video representing images of objects,

the system providing computed knowledge of any one or more of at least the following preselected possible object attributes:

(a) categorical object content;

(b) characteristic object features; and

(c) behavior of said objects;

the system providing for varying either the spatial resolution or temporal resolution of the objects, or both said spatial resolution and said temporal resolution, based on predetermined interest in any one or more of said preselected possible object attributes.

11. In a system having video camera apparatus providing output video data comprising both object video and background video, wherein the output video data is recorded in a useful form on recording media in order to preserve the content of such images, and wherein the system provides computed knowledge of any one or more of at least the following object attributes:

(a) categorical object content;

(b) characteristic object features; and

(c) object behavior;

provision for allowing a user of the system to query recorded video images by content by enabling the user to recall recorded video data according to one or more of said attributes.

12. In a system according to claim 11, further comprising provision for intelligent pruning of said video images in the recorded video data after the data is recorded.

13. In a system having video camera apparatus providing output video data comprising both object video and background video, wherein the output video data is recorded in a useful form on recording media in order to preserve the content of such images, and wherein the system provides computed knowledge of symbolic content of objects in the output video data;

provision for allowing a user of the system to query recorded video images by content by enabling the user to recall recorded video data according to symbolic content selected by the user.

14. In a system according to claim 13, further comprising provision for intelligent pruning of said video images in the recorded video data after the data is recorded.

15. In a system having video camera apparatus providing output video data comprising both object video and background video, wherein the output video data is recorded in a useful form on recording media in order to preserve the content of such images according to predetermined criteria for so recording the data, and wherein the system provides computed knowledge of symbolic content of objects in the object video:

provision for intelligent pruning of said video images n the recorded video data after the data is recorded by allowing user-selected determination for culling of recorded video data in accordance with changes predetermined criteria for so recording the data; and

provision for allowing a user of the system to query recorded video images by symbolic content by enabling the user to recall recorded video data of the objects according to specification by the user of the symbolic content.

16. In a system having video camera apparatus providing output video to be recorded in a useful form on recording media in order to preserve intelligence content of such images, the video output comprised of background video and object video representing images of one or more objects appearing against the background, wherein the system providing computed knowledge relative to said objects of any one or more of the following object attributes:

(a) categorical object content;

(b) characteristic object features; and

(c) object behavior;

the improvement comprising provision for varying either the spatial resolution or temporal resolution or both said spatial resolution and said temporal resolution of said one or more objects based on predetermined interest in any one or more of said object attributes;

while recording the background video and object video;

the improvement further comprising provision for allowing a user of the system to query recorded video images by content or behavior by enabling the user to recall recorded data according to any one or more of said object attributes.

17. For use in a system having video camera apparatus providing large amounts of output video to be recorded in a useful form on recording media to preserve intelligence content of such images, the video output consisting of background video and object video representing images of objects appearing against the background, the improvement comprising, a method for reducing an amount of video actually recorded so as to reduce an amount of recording media used therefor, the method comprising:

separating the video output into background video and object video;

analyzing the object video for content according to differing possible objects in the images, wherein the objects have different possible object attributes detected by such analysis;

independently storing the background and object video while compressing both the background and object video according to at least one suitable compression algorithm;

wherein the object video is recorded while varying the frame rate of the recorded object video in accordance with the different possible object attributes, the frame rate having a preselected value at any given time corresponding to the differing possible objects which value is not less than will provide a useful image of the respective differing possible objects when recovered from storage;

wherein the object video is compressed while varying the compression ratio so that it has a value at any given corresponding to the differing possible objects, the compression ratio at any given time having a preselected value not greater than will provide a useful image of the respective differing possible objects when recovered from storage;

recovering the stored object and background video by reassembling the recorded background and the recorded object video for viewing.

18. The method according to claim 17 wherein the background video is recorded at a frame rate which is less than a frame rate at which the object video is recorded.

19. The method according to claim 17 wherein among different possible object attributes are different possible events which, as detected, are used to revise a preset level of interest in the video at the time of the event detection, by providing boost of resolution for a predetermined time interval according to the different possible events detected.

20. For use with a system having video camera apparatus providing output video to be recorded in a useful form on recording media to preserve intelligence content of such images,

wherein the video output consists of background video and object video representing images of objects, a method comprising:

providing computed knowledge of any one or more of at least the following object attributes;

(a) categorical object content;

(b) characteristic object features; and

(c) behavior of said objects; and

recording said output video in a useful form on recording media while varying either the spatial resolution or temporal resolution of the objects in the video, or both said spatial resolution and said temporal resolution, based on predetermined interest in any one or more of said object attributes.

21. For use with a system having video camera apparatus providing output video to be recorded in a useful form on recording media to preserve intelligence content of such images, wherein the video output consists of background video and object video representing images of objects, the method according to claim 20 further comprising

providing for changes in the predetermined storage or archiving criteria of such video or data according to user-selected determination for culling of stored or archived video or data;

and

providing software-implemented pruning of the stored or archived data according to such changes in the storage or archiving criteria.

22. In a system having video camera apparatus providing large amounts of output video which must be recorded in a useful form on recording media in order to preserve the content of such images, the video output consisting of background video and object video representing images of objects appearing against the background, said system including a video separator which separates the video output into background video and object video, and an analyzer which analyzes the object video for content according to different possible objects in the images and different possible kinds of object attributes which define the symbolic content of the object, the improvement comprising:

video processing apparatus for reducing the amount of video actually recorded so as to reduce the amount of recording media used therefor; and

a storage control which, under software control, independently stores or archives the background and object video or data associated with such video according to storage or archiving criteria while providing software-implemented compressing of both the background and object video according to at least one suitable compression algorithm determined by user-established criteria for data storage or archiving;

pruning provision that, under software control, allows changes in the criteria for data storage or archiving of such video or data and permits software-implemented pruning of the stored or archived data according to such changes in the criteria,

wherein;

the object video is recorded while varying the frame rate of the recorded object video in accordance with the different possible objects or different kinds of object behavior, or both said different kinds of objects and different kinds of object behavior, the frame rate having a preselected value at any given time corresponding to the different possible objects which value is not less than will provide a useful image of the respective different possible objects when recovered from storage;

the object video is compressed while varying the compression ratio so that it has a value at any given corresponding to the different possible objects or different kinds of object behavior, or both said different kinds of objects and different kinds of object behavior, the compression ratio at any given time having a preselected value not greater than will provide a useful image of the respective different possible objects when recovered from storage;

video recovery and presentation provision to present the stored object and background video by reassembling the recorded background and the recorded object video for viewing; and

the pruning provision is operative under software control to cull or remove at least some of said stored or archived video or data based upon said changes in said criteria in response to such changes in said criteria after data has previously been identified by the system as sufficiently significant as to be stored or archived.