System and method for producing a still image representation of a motion video

• US 6,665,342 B1
• Filed: 07/02/1999
• Issued: 12/16/2003
• Est. Priority Date: 07/02/1999
• Status: Expired due to Fees

• Current Assignee: International Business Machines Corporation
• Original Assignee: International Business Machines Corporation
• Inventors: Brown, Lisa Marie, Crayne, Susan
• Primary Examiner(s): Le, Vu
• Assistant Examiner(s): Senfi, Behrooz
• Application Number:
  US09/345,024
• Time in Patent Office: 1,628 Days
• Field of Search: 348/420.1, 348/413.1, 348/169, 348/579, 348/586, 348/587, 348/699, 348/700, 375/240.16, 382/236, 382/173, 382/194, 382/283
• US Class Current: 375/240.16
• CPC Class Codes: G06T 2207/10016 Video; Image sequence
• CPC Class Codes: G06T 7/254 involving subtraction of images
• CPC Class Codes: G08B 13/19602 Image analysis to detect motion of the intruder, e.g. by frame subtraction
• CPC Class Codes: G08B 13/19604 involving reference image or background adaptation with time to compensate for changing conditions, e.g. reference image update on detection of light level change
• CPC Class Codes: G08B 13/19691 Signalling events for better perception by user, e.g. indicating alarms by making display brighter, adding text, creating a sound
• CPC Class Codes: H04N 5/144 Movement detection for video coding H04N19/503; analysis of motion in general G06T7/20
• CPC Class Codes: H04N 5/2625 for obtaining an image which is composed of images from a temporal image sequence, e.g. for a stroboscopic effect sequence generated by event triggered capturing H04N7/188

Abstract

A still image from a digital video sequence by a computer system executing a strobe process. After an initialization step, the strobe process interatively selects two of the still frames in a sampled order. An attribute differencing step is performed that determines the difference of the attribute of the selected frames using a frame to frame comparison (attribute difference) and determines if the attribute difference from one frame to the other frame exceeds an attribute sensitivity. The attribute difference is stored along with the motion statistics of the attribute difference and if the attribute difference meets certain conditions, a relevant motion is determined. If a relevant motion is determined, a segmentation mask at one or more mask locations corresponding to the frame locations is updated. A strobe photo is also updated with the relevant motion at one or more photo locations corresponding to the mask and frame locations so that a history of relevant motions is recorded on the strobe photo.

Claims

1. A video processing system having a computer with one or more central processing units (CPUs), one or more memories, and further comprising:

a digital video stream, stored in one or more of the memories, the stream having a plurality of still frames in an ordered sequence; and

a strobe process, executed by one or more of the CPUs, comprising the following steps;

a. initializing a motion history record with one or more motion statistics, a segmentation mask, an attribute, and an attribute sensitivity;

b. iteratively selecting two of the still frames in a sampled order;

c. attribute differencing of the attribute the selected frames using a frame to frame comparison to determine an attribute difference at one or more frame locations in the frames and determining if the attribute difference from one frame to the other frame exceeds the attribute sensitivity;

d. storing the attribute difference and the motion statistics of the attribute difference in the motion history record;

e. if the attribute difference exceeds the attribute sensitivity when the segmentation mask has not yet been updated, determining a relevant motion;

f. if the segmentation mask has been updated and the attribute difference exceeds one or more of the motion statistics within a statistical confidence level, determining a relevant motion;

g. if a relevant motion is determined, updating the segmentation mask at one or more mask locations corresponding to the frame locations;

h. if the segmentation mask is updated, updating a strobe photo with the relevant motion at one or more photo locations corresponding to the mask and frame locations so that a history of relevant motions is recorded on the strobe photo; and

i. repeating steps b through h for each pair of frames selected.

2. A system, as in claim 1, where the attribute includes any one or more of the follow:

an intensity, a luminosity, a color dimension, a color vector, a spatial average over a local window, and a median over the local window.

3. A system, as in claim 1, where the sensitivity includes any one or more of the following:

an intensity threshold, a luminosity threshold, an adaptive threshold, a user input threshold, a threshold based on psychophysical measures of human color sensitivity, and a statistically significant change based on the measurements taken in the initialization step.

4. A system, as in claim 1, where the frame to frame comparison includes any one or more of the following:

a pixel by pixel comparison, a window by window comparison, and a comparison of a combination of a temporal and spatial window.

5. A system, as in claim 1, where the attribute difference is an absolute magnitude.

6. A system, as in claim 1, where the sampled order is any one or more of the following:

a consecutive sampling;

a sampling every n^th frame where n is any whole number;

a reverse sampling, and a variable sampling.

7. A system, as in claim 1, where the motion history record includes a maximum attribute difference encountered in the sampled order so far, an incrementally updated mean, and a variance of the attribute differences.

8. A video processing system having a computer with one or more central processing units (CPUs), one or more memories, and further comprising:

a. means for initializing a motion history record with one or more motion statistics, a segmentation mask, an attribute, and an attribute sensitivity of a digital video stream having a plurality of still frames in an ordered sequence;

b. means for iteratively selecting two of the still frames in a sampled order;

c. means for attribute differencing of the attribute of the selected frames using a frame to frame comparison to determine an attribute difference at one or more frame locations in the frames and determining if the attribute difference from one frame to the other frame exceeds the attribute sensitivity;

d. means for storing the attribute difference and the motion statistics of the attribute difference in the motion history record;

e. means for determining a relevant motion if the attribute difference exceeds the attribute sensitivity when the segmentation mask has not yet been updated;

f. means for determining a relevant motion, if the segmentation mask has been updated and the attribute difference exceeds one or more of the motion statistics within a statistical confidence level;

g. means for updating the segmentation mask at one or more mask locations corresponding to the frame locations if a relevant motion is determined; and

h. means for updating a strobe photo with the relevant motion at one or more photo locations corresponding to the mask and frame locations if the segmentation mask is updated, so that a history of relevant motions is recorded on the strobe photo.

9. A video processing system method comprising the steps of:

a. initializing a motion history record with one or more motion statistics, a segmentation mask, an attribute, and an attribute sensitivity of a digital video stream having a plurality of still frames in an ordered sequence;

b. iteratively selecting two of the still frames in a sampled order;

c. attribute differencing of the attribute in the selected frames using a frame to frame comparison to determine an attribute difference at one or more frame locations in the frames and determining if the attribute difference from one frame to the other frame exceeds the attribute sensitivity;

d. storing the attribute difference and the motion statistics of the attribute difference in the motion history record;

e. determining a relevant motion if the attribute difference exceeds the attribute sensitivity when the segmentation mask has not yet been updated;

f. determining a relevant motion if the segmentation mask has been updated and the attribute difference exceeds one or more of the motion statistics within a statistical confidence level;

g. updating the segmentation mask at one or more mask locations corresponding to the frame locations if a relevant motion is determined; and

h. updating a strobe photo with the relevant motion at one or more photo locations corresponding to the mask and frame locations if the segmentation mask is updated so that a history of relevant motions is recorded on the strobe photo.

10. A computer program product which performs the steps of:

a. initializing a motion history record with one or more motion statistics, a segmentation mask, an attribute, and an attribute sensitivity of a digital video stream having a plurality of still frames in an ordered sequence;

b. iteratively selecting two of the still frames in a sampled order;

c. attribute differencing of the attribute in the selected frames using a frame to frame comparison to determine an attribute difference at one or more frame locations in the frames and determining if the attribute difference from one frame to the other frame exceeds the attribute sensitivity;

d. storing the attribute difference and the motion statistics of the attribute difference in the motion history record;

e. determining a relevant motion if the attribute difference exceeds the attribute sensitivity when the segmentation mask has not yet been updated;

f. determining a relevant motion if the segmentation mask has been updated and the attribute difference exceeds one or more of the motion statistics within a statistical confidence level;

g. updating the segmentation mask at one or more mask locations corresponding to the frame locations if a relevant motion is determined; and

h. updating a strobe photo with the relevant motion at one or more photo locations corresponding to the mask and frame locations if the segmentation mask is updated so that a history of relevant motions is recorded on the strobe photo.