Estimating motion of an event captured using a digital video camera

• US 9,256,789 B2
• Filed: 03/21/2008
• Issued: 02/09/2016
• Est. Priority Date: 03/21/2008
• Status: Active Grant

• Current Assignee: Intel Corporation
• Original Assignee: Intel Corporation
• Inventors: Adiletta, Matthew J., Yang, Chengda
• Primary Examiner(s): Seth, Manav
• Application Number:
  US12/053,328
• Publication Number:
  US 20090237503A1
• Time in Patent Office: 2,881 Days
• Field of Search: 382/103, 382/104, 382/254, 382/282, 348/155, 348/148, 348/143
• US Class Current: 1/1
• CPC Class Codes: G06T 2207/10016 Video; Image sequence
• CPC Class Codes: G06T 2207/10024 Color image
• CPC Class Codes: G06T 2207/20076 Probabilistic image processing
• CPC Class Codes: G06T 2207/30232 Surveillance
• CPC Class Codes: G06T 5/50 using two or more images, e.g. averaging or subtraction
• CPC Class Codes: G06T 7/254 involving subtraction of images
• CPC Class Codes: G06V 20/44 Event detection
• CPC Class Codes: G06V 20/52 Surveillance or monitoring of activities, e.g. for recognising suspicious objects recognising microscopic objects G06V20/69
• CPC Class Codes: G08B 13/19608 Tracking movement of a target, e.g. by detecting an object predefined as a target, using target direction and or velocity to predict its new position
• CPC Class Codes: H04N 7/188 Capturing isolated or intermittent images triggered by the occurrence of a predetermined event, e.g. an object reaching a predetermined position signal generation from motion picture films H04N5/253

Abstract

An event aware video system (EAVS) is to capture video frames during a first time period and process events in the video frames before transferring the processed data to a central computing system. The EAVS may establish a present no-event frame from the video frames by marking the last frame as the present no-event frame if the difference between adjacent pair of video frames is less than a threshold value. The EAVS may establish an event frame, wherein a present frame captured after establishing the no-event frame is marked as the event frame if the difference between the present frame and a previous frame is greater than the threshold value. The EAVS may provide event information including motion vectors to a central computing system by performing one-dimensional search on a moving object of the event frame, wherein the motion vectors may represent displacement of objects moving within the moving object.

Claims

1. A method in a video capturing device comprising:

capturing a first plurality of video frames during a first time period;

establishing a present no-event frame from the first plurality of video frames, wherein a last frame of the first plurality of video frames is marked as the present no-event frame if the difference between adjacent pair of frames of the first plurality of video frames is less than a threshold value;

establishing an event frame, wherein a present frame captured after establishing the no-event frame is marked as the event frame if the difference between the present frame and a previous frame captured prior to the present frame is greater than the threshold value;

generating event information including motion vectors, wherein the motion vectors represent displacement of a plurality of objects moving within a moving object, wherein the motion vectors are generated by performing one-dimensional search on a moving object of the event frame; and

generating a real motion vector of a moving object in the event frame, wherein generating the real motion vector comprises;

labeling the moving object;

pairing the moving object in the event frame with a moving object in the previous frame;

resizing a boundary of the moving object in the event frame to match a boundary of the moving object in the previous frame; and

computing the co-ordinates of the center of gravity of the boundary of the moving object in the event frame and the previous frame.

2. The method of claim 1, wherein generating the motion vectors comprise rejecting the present frame captured after the first time period if the difference between the present frame and the no-event frame is less than the threshold value.

3. The method of claim 1, wherein generating the motion vectors comprises removing background information of the event frame before sending the event information to the central computing system.

4. The method of claim 3, wherein removing the background comprises:

identifying an area that excludes the event in the event frame; and

assigning a first color value to the pixels of the area, wherein assigning the first color to the pixels of the area removes the background information in the event frame.

5. The method of claim 1 further comprises computing motion error of the moving object including:

generating an estimated motion vector by displacing the moving object in the previous frame using the motion vector; and

computing the difference between the real moving object image and the motion estimated moving object image.

6. The method of claim 5 further comprises:

performing motion refining if the motion error is greater than an allowable error threshold; and

performing motion error coding if the motion error is lesser than the allowable error threshold.

7. A method in a video capturing device comprising:

capturing a first plurality of video frames during a first time period;

establishing a present no-event frame from the first plurality of video frames, wherein a last frame of the first plurality of video frames is marked as the present no-event frame if the difference between adjacent pair of frames of the first plurality of video frames is less than a threshold value;

establishing an event frame, wherein a present frame captured after establishing the no-event frame is marked as the event frame if the difference between the present frame and a previous frame captured prior to the present frame is greater than the threshold value; and

generating event information including motion vectors, wherein the motion vectors represent displacement of a plurality of objects moving within a moving object, wherein the motion vectors are generated by performing one-dimensional search on a moving object of the event frame;

wherein generating each of the motion vectors comprises dividing the boundary of the moving object in the event frame into a plurality of sub-rectangles, and determining motion of the object within the plurality of sub-rectangles.

8. The method of claim 7, wherein generating the motion vectors comprises identifying border sub-rectangles of the plurality of sub-rectangles, wherein a sub-rectangle of the plurality of sub-rectangles is identified as a border sub-rectangle if the sub-rectangle includes at least a first quantity of background pixels and at least a second quantity of object pixels.

9. The method of claim 7, wherein generating the motion vectors comprise designating a pivotal border pixel for the border sub-rectangles, wherein a pixel nearest to the center of the border sub-rectangle is designated as the pivotal border pixel.

10. The method of claim 9, wherein generating the motion vectors comprise:

determining motion vectors for the border sub-rectangles by comparing the border sub-rectangle of the event frame led by the pivotal border pixel with the border of the previous frame; and

determining motion vectors for non-border sub-rectangles using the motion vectors of the border sub-rectangles, wherein the sub-rectangles other than the border sub-rectangles of the plurality of sub-rectangles is identified as non-border sub-rectangles.

11. An apparatus comprising:

an image capturing unit to capture a first plurality of video frames during a first time period;

an event aware video system enabled controller coupled to the image capturing unit, wherein the event aware video system enabled controller is to;

establish a present no-event frame from the first plurality of video frames, wherein a last frame of the first plurality of video frames is marked as the present no-event frame if the difference between adjacent pair of frames of the first plurality of video frames is less than a threshold value; and

establish an event frame, wherein a present frame captured after establishing the no-event frame is marked as the event frame if the difference between the present frame and a previous frame captured prior to the present frame is greater than the threshold value; and

generate event information including motion vectors, wherein the motion vectors represent displacement of a plurality of objects moving within a moving object, wherein each of the motion vectors is generated by performing one-dimensional search on a moving object of the event frame; and

to generate a real motion vector of a moving object in the event frame, the event aware video system enabled controller is to;

label the moving object,

pair the moving object in the event frame with a moving object in the previous frame,

resize a boundary of the moving object in the event frame to match a boundary of the moving object in the previous frame, and

compute the co-ordinates of the center of gravity of the boundary of the moving object in the event frame and the previous frame.

12. The apparatus of claim 11, wherein the event aware video system enabled controller is to reject the present frame captured after the first time period if the difference between the present frame and the no-event frame is less than the threshold value.

13. The apparatus of claim 11, wherein the event aware video system enabled controller is to remove background information of the event frame before sending the event information to the central computing system.

14. The apparatus of claim 13, wherein the event aware video system enabled controller is to:

identify an area that excludes the event in the event frame; and

assign a first color value to the pixels of the area, wherein assigning the first color to the pixels of the area removes the background information in the event frame.

15. The apparatus of claim 11, wherein the event aware video system enabled controller is to:

generate an estimated motion vector by displacing the moving object in the previous frame using the motion vector; and

compute the difference between the real moving object image and the motion estimated moving object image.

16. The apparatus of claim 15, wherein the event aware video system enabled controller is to:

perform motion refining if the motion error is greater than an allowable error threshold; and

perform motion error coding if the motion error is lesser than the allowable error threshold.

17. An apparatus comprising:

an image capturing unit to capture a first plurality of video frames during a first time period;

an event aware video system enabled controller coupled to the image capturing unit, wherein the event aware video system enabled controller is to;

establish a present no-event frame from the first plurality of video frames, wherein a last frame of the first plurality of video frames is marked as the present no-event frame if the difference between adjacent pair of frames of the first plurality of video frames is less than a threshold value;

establish an event frame, wherein a present frame captured after establishing the no-event frame is marked as the event frame if the difference between the present frame and a previous frame captured prior to the present frame is greater than the threshold value;

generate event information including motion vectors, wherein the motion vectors represent displacement of a plurality of objects moving within a moving object, wherein each of the motion vectors is generated by performing one-dimensional search on a moving object of the event frame;

divide the boundary of the moving object in the event frame into a plurality of sub-rectangles; and

determine motion of the object within the plurality of sub-rectangles.

18. The apparatus of claim 17, wherein the event aware video system enabled controller is to identify border sub-rectangles of the plurality of sub-rectangles, wherein a sub-rectangle of the plurality of sub-rectangles is identified as a border sub-rectangle if the sub-rectangle includes at least a first quantity of background pixels and at least a second quantity of object pixels.

19. The apparatus of claim 17, wherein the event aware video system enabled controller is to designate a pivotal border pixel for the border sub-rectangles, wherein a pixel nearest to the center of the border sub-rectangle is designated as the pivotal border pixel.

20. The apparatus of claim 19, wherein the event aware video system enabled controller is to:

determine motion vectors for the border sub-rectangles by comparing the border sub-rectangle of the event frame led by the pivotal border pixel with the border of the previous frame; and

determine motion vectors for non-border sub-rectangles using the motion vectors of the border sub-rectangles, wherein the sub-rectangles other than the border sub-rectangles of the plurality of sub-rectangles is identified as non-border sub-rectangles.

21. A system comprising:

a central computer system, coupled to a network to receive event information; and

an event aware video system coupled to the network, wherein the event aware video system is to transfer the event information over the network;

wherein the event aware video system is to establish a present no-event frame from the first plurality of video frames, wherein a last frame of the first plurality of video frames is marked as the present no-event frame if the difference between adjacent pair of frames of the first plurality of video frames is less than a threshold value; and

wherein the event aware video system is to establish an event frame, wherein a present frame captured after establishing the no-event frame is marked as the event frame if the difference between the present frame and a previous frame captured prior to the present frame is greater than the threshold value; and

wherein the event aware video system is to generate event information including motion vectors, wherein the motion vectors represent displacement of a plurality of objects moving within a moving object, wherein the motion vectors is generated by performing one-dimensional search on a moving object of the event frame;

wherein the event aware video system is to divide the boundary of the moving object in the event frame into a plurality of sub-rectangles;

wherein the event aware video system is to identify border sub-rectangles of the plurality of sub-rectangles, wherein a sub-rectangle of the plurality of sub-rectangles is identified as a border sub-rectangle if the sub-rectangle includes at least a first quantity of background pixels and at least a second quantity of object pixels; and

wherein the event aware video system is to determine motion of the object within the plurality of sub-rectangles.

22. The system of claim 21, wherein the event aware video system is to:

designate a pivotal border pixel for the border sub-rectangles, wherein a pixel nearest to the center of the border sub-rectangle is designated as the pivotal border pixel;

determine motion vectors for the border sub-rectangles by comparing the border sub-rectangle of the event frame led by the pivotal border pixel with the border of the previous frame; and

determine motion vectors for non-border sub-rectangles using the motion vectors of the border sub-rectangles, wherein the sub-rectangles other than the border sub-rectangles of the plurality of sub-rectangles is identified as non-border sub-rectangles.