Estimating motion of an event captured using a digital video camera

• US 9,251,423 B2
• Filed: 03/21/2008
• Issued: 02/02/2016
• Est. Priority Date: 03/21/2008
• Status: Active Grant

• Current Assignee: Intel Corporation
• Original Assignee: Intel Corporation
• Inventors: Adiletta, Matthew J., Yang, Chengda
• Primary Examiner(s): Seth, Manav
• Application Number:
  US12/053,319
• Publication Number:
  US 20090238411A1
• Time in Patent Office: 2,874 Days
• Field of Search: 382/103, 382/104, 382/254, 382/282, 348/155, 348/148, 348/143
• US Class Current: 1/1
• CPC Class Codes: G06T 2207/10016 Video; Image sequence
• CPC Class Codes: G06T 2207/30232 Surveillance
• CPC Class Codes: G06T 5/50 using two or more images, e.g. averaging or subtraction
• CPC Class Codes: G06T 7/246 using feature-based methods, e.g. the tracking of corners or segments
• CPC Class Codes: G06V 20/44 Event detection
• CPC Class Codes: G06V 20/52 Surveillance or monitoring of activities, e.g. for recognising suspicious objects recognising microscopic objects G06V20/69
• CPC Class Codes: G08B 13/19608 Tracking movement of a target, e.g. by detecting an object predefined as a target, using target direction and or velocity to predict its new position
• CPC Class Codes: H04N 7/188 Capturing isolated or intermittent images triggered by the occurrence of a predetermined event, e.g. an object reaching a predetermined position signal generation from motion picture films H04N5/253

Abstract

An event aware video system (EAVS) is to capture video frames during a first time period and process events in the video frames before transferring the processed data to a central computing system. The EAVS may establish a no-event frame by marking a last frame as the no-event frame if the difference between adjacent pair of video frames is less than a threshold value. The EVAS may mark a present frame captured after establishing the no-event frame as the event frame if the difference between the present and a previous frame is greater than the threshold value. The EAVS may provide event information to the central computing system by performing temporal blending, which includes linearly combining the movement of objects within the moving object in adjacent event frames to generate blurred images. The difference between the blurred images may represent displacement of objects moving within the moving object.

Claims

1. A method in a video capturing device comprising:

capturing a first plurality of video frames during a first time period;

establishing a present no-event frame from the first plurality of video frames, wherein a last frame of the first plurality of video frames is marked as the present no-event frame if the difference between adjacent pair of frames of the first plurality of video frames is less than a threshold value;

establishing an event frame, wherein a present frame captured after establishing the present no-event frame is marked as the event frame if the difference between the present frame and a previous frame captured prior to the present frame is greater than the threshold value; and

generating event information including motion vectors, wherein the motion vectors represent displacement of a plurality of objects moving within a moving object, wherein the motion vectors are generated by linearly combining position of an object of the plurality of objects captured in adjacent event frames;

labeling the moving object;

pairing the moving object in the event frame with a moving object in the previous frame;

resizing a boundary of the moving object in the event frame to match a boundary of the moving object in the previous frame; and

computing the co-ordinates of the center of gravity of the boundary of the moving object in the event frame and the previous frame.

2. The method of claim 1, wherein generating the motion vectors comprise rejecting the present frame captured after the first time period if the difference between the present frame and the present no-event frame is less than the threshold value.

3. The method of claim 1, wherein generating the motion vectors comprises removing background information of the event frame before sending the event information to a central computing system.

4. The method of claim 3, wherein removing the background comprises:

identifying an area that excludes an event in the event frame; and

assigning a first color value to the pixels of the area, wherein assigning the first color to the pixels of the area removes the background information in the event frame.

5. The method of claim 1 further comprises computing motion error of the moving object including:

generating an estimated motion vector by displacing the moving object in the previous frame using the motion vector; and

computing the difference between the real moving object image and the motion estimated moving object image.

6. The method of claim 5 further comprises:

performing motion refining if the motion error is greater than an allowable error threshold; and

performing motion error coding if the motion error is lesser than the allowable error threshold.

7. A method in a video capturing device comprising:

capturing a first plurality of video frames during a first time period;

establishing a present no-event frame from the first plurality of video frames, wherein a last frame of the first plurality of video frames is marked as the present no-event frame if the difference between adjacent pair of frames of the first plurality of video frames is less than a threshold value;

establishing an event frame, wherein a present frame captured after establishing the present no-event frame is marked as the event frame if the difference between the present frame and a previous frame captured prior to the present frame is greater than the threshold value; and

generating event information including motion vectors, wherein the motion vectors represent displacement of a plurality of objects moving within a moving object, wherein the motion vectors are generated by linearly combining position of an object of the plurality of objects captured in adjacent event frames;

wherein generating the motion vectors comprises;

selecting a sub-boundary rectangle of a first object of the plurality of objects from adjacent event frames;

generating a first blurred image and a second blurred image of the first object by linearly combining the movement of the first object within the first set and a second set of adjacent event frames; and

determining the difference value by comparing the first blurred image and the second blurred image of the first object.

8. The method of claim 7, wherein generating the first blurred image comprises:

assigning a first, second, and a third weight value to a first, second, and a third event frame of the first set of adjacent event frames; and

combining a first position, a second position, and a third position of the first object, wherein the first position is processed with the first weight value, the second position is processed with the second weight value, and the third position is processed with the third weight value.

9. The method of claim 8, wherein generating the second blurred image comprises:

assigning a fourth weight value to a fourth event frame; and

combining the second position, the third position, and a fourth position of the first object, wherein the second position is processed with the second weight value, the third position is processed with the third weight value, and the fourth position is processed with the fourth weight value.

10. The method of claim 9, wherein generating the motion vector of the first object comprises:

generating a first difference value by computing the difference between the first and the second blurred value;

generating an encoded value by encoding the first difference value; and

transferring the encoded value over a network.

11. An apparatus comprising:

an image capturing unit to capture a first plurality of video frames during a first time period; and

an event aware video system enabled controller coupled to the image capturing unit, wherein the event aware video system enabled controller is to;

establish a present no-event frame from the first plurality of video frames, wherein a last frame of the first plurality of video frames is marked as the present no-event frame if the difference between adjacent pair of frames of the first plurality of video frames is less than a threshold value; and

establish an event frame, wherein a present frame captured after establishing the present no-event frame is marked as the event frame if the difference between the present frame and a previous frame captured prior to the present frame is greater than the threshold value;

generate event information including motion vectors, wherein the motion vectors represent displacement of a plurality of objects moving within a moving object, wherein the motion vectors is generated by linearly combining the movement of objects of the plurality of objects captured in adjacent event frames; and

to generate a real motion vector of a moving object in the event frame, the event aware video system enabled controller is to;

label the moving object;

pair the moving object in the event frame with a moving object in the previous frame,

resize a boundary of the moving object in the event frame to match a boundary of the moving object in the previous frame; and

compute the co-ordinates of the center of gravity of the boundary of the moving object in the event frame and the previous frame.

12. The apparatus of claim 11, wherein the event aware video system enabled controller is to reject the present frame captured after the first time period if the difference between the present frame and the present no-event frame is less than the threshold value.

13. The apparatus of claim 11, wherein the event aware video system enabled controller is to remove background information of the event frame before sending the event information to a central computing system.

14. The apparatus of claim 13, wherein the event aware video system enabled controller is to:

identify an area that excludes an event in the event frame; and

assign a first color value to the pixels of the area, wherein assigning the first color to the pixels of the area removes the background information in the event frame.

15. The apparatus of claim 11, wherein the event aware video system enabled controller is to:

generate an estimated motion vector by displacing the moving object in the previous frame using the motion vector; and

compute the difference between the real moving object image and the motion estimated moving object image.

16. The apparatus of claim 15, wherein the event aware video system enabled controller is to:

perform motion refining if the motion error is greater than an allowable error threshold; and

perform motion error coding if the motion error is lesser than the allowable error threshold.

17. An apparatus comprising:

an image capturing unit to capture a first plurality of video frames during a first time period; and

an event aware video system enabled controller coupled to the image capturing unit, wherein the event aware video system enabled controller is to;

establish a present no-event frame from the first plurality of video frames, wherein a last frame of the first plurality of video frames is marked as the present no-event frame if the difference between adjacent pair of frames of the first plurality of video frames is less than a threshold value; and

establish an event frame, wherein a present frame captured after establishing the present no-event frame is marked as the event frame if the difference between the present frame and a previous frame captured prior to the present frame is greater than the threshold value;

generate event information including motion vectors, wherein the motion vectors represent displacement of a plurality of objects moving within a moving object, wherein the motion vectors is generated by linearly combining the movement of objects of the plurality of objects captured in adjacent event frames;

select a sub-boundary rectangle of a first object of the plurality of objects from adjacent event frames;

generate a first blurred image and a second blurred image of the first object by linearly combining the movement of the first object within the first and second adjacent event frames; and

determine the difference value by comparing the first blurred image and the second blurred image of the first object.

18. The apparatus of claim 17, wherein the event aware video system enabled controller is to:

assign a first, second, and a third weight value to a first, second, and a third event frame of the first set of adjacent event frames; and

combine a first position, a second position, and a third position of the first object, wherein the first position is processed with the first weight value, the second position is processed with the second weight value, and the third position is processed with the third weight value.

19. The apparatus of claim 17, wherein the event aware video system enabled controller is to:

assign a fourth weight value to a fourth event frame; and

combine the second position, the third position, and a fourth position of the first object, wherein the second position is processed with the second weight value, the third position is processed with the third weight value, and the fourth position is processed with the fourth weight value.

20. The apparatus of claim 17, wherein the event aware video system enabled controller is to:

generate a first difference value by computing the difference between the first blurred value and the second blurred value;

generate an encoded value by encoding the first difference value; and

transfer the encoded value over a network.

21. A system comprising:

a central computer system, coupled to a network to receive event information; and

an event aware video system coupled to the network, wherein the event aware video system is to transfer the event information over the network;

wherein the event aware video system is to establish a present no-event frame from the first plurality of video frames, wherein a last frame of the first plurality of video frames is marked as the present no-event frame if the difference between adjacent pair of frames of the first plurality of video frames is less than a threshold value; and

wherein the event aware video system is to establish an event frame, wherein a present frame captured after establishing the present no-event frame is marked as the event frame if the difference between the present frame and a previous frame captured prior to the present frame is greater than the threshold value;

wherein the event aware video system is to generate event information including motion vectors, wherein the motion vectors represent displacement of a plurality of objects moving within a moving object, wherein the motion vectors is generated by linearly combining the movement of objects of the plurality of objects captured in adjacent event frames;

wherein the event aware video system is to select a sub-boundary rectangle of a first object of the plurality of objects from adjacent event frames;

wherein the event aware video system is to generate a first blurred image and a second blurred image of the first object by linearly combining the movement of the first object within the first and second adjacent event frames; and

wherein the event aware video system is to determine the difference value by comparing the first blurred image and the second blurred image of the first object.

22. The system of claim 21, wherein the event aware video system is to:

assign a first, second, and a third weight value to a first, second, and a third event frame of the first set of adjacent event frames; and

combine a first position, a second position, and a third position of the first object, wherein the first position is processed with the first weight value, the second position is processed with the second weight value, and the third position is processed with the third weight value.

23. The system of claim 21, wherein the event aware video system is to,

assign a fourth weight value to a fourth event frame;

and

combine the second position, the third position, and a fourth position of the first object, wherein the second position is processed with the second weight value, the third position is processed with the third weight value, and the fourth position is processed with the fourth weight value.

24. The system of claim 22, wherein the event aware video system is to:

generate a first difference value by computing the difference between the first blurred value and the second blurred value;

generate an encoded value by encoding the first difference value; and

transfer the encoded value over a network.