Motion detection using normal optical flow

• US 6,647,131 B1
• Filed: 08/27/1999
• Issued: 11/11/2003
• Est. Priority Date: 08/27/1999
• Status: Expired due to Term

• Current Assignee: Intel Corporation
• Original Assignee: Intel Corporation
• Inventors: Bradski, Gary R.
• Primary Examiner(s): Patel, Jayanti K.
• Application Number:
  US09/384,462
• Time in Patent Office: 1,537 Days
• Field of Search: 382/100, 382/103, 382/128, 382/133, 382/168-172, 382/218-219, 382/232, 382/236, 382/107, 382/199, 342/64, 345/473, 345/474, 348/239, 348/700, 375/240.16
• US Class Current: 382/107
• CPC Class Codes: G06T 7/215 Motion-based segmentation
• CPC Class Codes: G06T 7/269 using gradient-based methods
• CPC Class Codes: G06V 40/20 Movements or behaviour, e.g. gesture recognition recognition of facial expressions G06V40/16

Abstract

A system and method obtain images of an object and generate a motion region image of the object. The motion region image is processed to obtain normal gradients of the portion of the object that has been moved. The normal gradient data is further processed to remove erroneous data points. The erroneous data can be removed by using either an eroding method or a threshold method. After the erroneous data is removed, the remaining gradient information is used to identify a motion of the object. This can be performed using a histogram recognition operation.

Claims

1. A method of detecting motion comprising:

generating a plurality of silhouettes from a corresponding plurality of images taken of an object over a predetermined period of time, the plurality of images including at least three images and having time-difference information associated with sequential images of the plurality to indicate change in velocity of movement of portions of the object;

generating a motion region image of the object from the plurality of silhouettes;

calculating normal gradients over time for edges within the motion region image, the normal gradients having an orientation substantially perpendicular to motion of the edges of the silhouettes;

removing erroneous normal gradients to provide a normal flow image comprised of at least some of the normal gradients; and

identifying a type of movement of the object based on the normal flow image.

2. The method of claim 1 and having a magnitude proportional to a rate of change of motion between edges of sequential silhouettes, and wherein a greater magnitude of a particular one of the normal gradients indicates a greater change in velocity associated with a portion of the object, and

wherein generating the motion region image comprises:

isolating the object in the plurality of images from a background to generate the silhouettes;

generating a difference image using the plurality of images, wherein the difference image isolates a portion of the object which has changed location during the plurality of images; and

setting pixels of the difference image to an equal value to generate the motion region image, and

wherein identifying includes comparing the magnitudes and the orientations of the normal gradients of the normal flow image with magnitudes and orientations of the normal gradients of the model image flows.

3. The method of claim 2 wherein removing erroneous normal gradients comprises:

removing normal gradients which have a magnitude outside a predetermined range, and wherein identifying including using the change in velocity of movement to at least in part identify the type of movement.

4. The method of claim 3 wherein calculating normal gradients comprises:

applying a Sobel operator to silhouette pixels of the motion region image to generate a response function;

obtaining the orientations of gradients of the silhouette pixels of the motion region image from the response function; and

obtaining the magnitudes of the gradients of the silhouette pixels from the response function.

5. The method of claim 4 wherein identifying movement comprises:

generating a histogram from the normal flow image, the histogram using Chi Square and minimum intersection measures; and

comparing the histogram of the normal flow image to model histograms to identify the type of movement performed by the object, wherein comparing including comparing the magnitudes and orientations of the normal gradients of the histogram with magnitudes and orientations of the normal gradients of the model histograms.

6. The method of claim 1 wherein removing erroneous normal gradients comprises:

removing normal gradients which are located in a boundary region of the motion region image, the boundary region including edges of some of the silhouettes.

7. A method of detecting motion comprising:

obtaining a plurality of at least three images of an object using a camera, the plurality of images being obtained over a predetermined period of time;

generating a motion region image of the object from the plurality of images by isolating the object in the plurality of images from a background to generate a plurality of corresponding silhouettes, generating a difference image using the plurality of images,

wherein the difference image isolates a portion of the object which has changed location during the plurality of images, and setting pixels of the difference image to an equal value;

calculating normal gradients over time of the motion region image, the normal gradients having an orientation substantially perpendicular to motion of edges of the silhouettes and having a magnitude proportional to changing velocity of motion between edges of sequential silhouettes;

removing erroneous normal gradients to provide a normal flow image; and

identifying movement based on the normal flow image.

8. The method of claim 7 wherein a greater magnitude of a particular one of the normal gradients indicates a greater changing velocity associated with a portion of the object, and, wherein removing erroneous normal gradients comprises removing normal gradients which have a magnitude outside a predetermined range, and

wherein identifying including using a change in velocity of movement to at least in part identify the type of movement.

9. The method of claim 7 wherein removing erroneous normal gradients comprises removing normal gradients which are located in a boundary region of the motion region image, the boundary region including edges of some of the silhouettes.

10. The method of claim 7 wherein calculating normal gradients comprises:

applying a Sobel operator to silhouette pixels of the motion region image;

obtaining an orientation of gradients of pixels of the motion region image from a response of the Sobel operator; and

obtaining a magnitude of the gradients of the pixels from the response.

11. The method of claim 7 wherein identifying movement comprises:

analyzing the normal flow image using a histogram of the normal flow image, the histogram using Chi Square and minimum intersection measures; and

comparing the histogram of the normal flow image to model histograms to identify a type of movement performed by the object, wherein comparing including comparing the magnitudes and orientations of the normal gradients of the histogram with magnitudes and orientations of the normal gradients of the model histograms.

12. A computer readable medium comprising instructions stored thereon that when executed by a processor result in:

generating a plurality of silhouettes from a corresponding plurality of images taken of an object over a predetermined period of time, the plurality of image including at least three images and having time-difference information associated with sequential images of the plurality,

generating a motion region image of the object from the plurality of silhouettes;

calculating normal gradients for edges within the motion region image, the normal gradients hang an orientation substantially perpendicular to motion of the edges of the silhouettes and having a magnitude proportional to changing velocity of motion between edges of sequential silhouettes;

removing erroneous normal gradients to provide a normal flow image comprised of at least some of the normal gradients; and

identifying a type of movement of the object based on the normal flow image.

13. The computer readable medium of claim 12 further comprising instructions, when executed by the processor to generate the motion region image further result in:

isolating the object in the plurality of images from a background to generate the silhouettes;

generating a difference image using the plurality of images, wherein the difference image isolates a portion of the object which has changed location during the plurality of images; and

setting pixels of the difference image to an equal value to generate the motion region image, and

wherein identifying includes comparing the magnitudes and the orientations of the normal gradients of the normal flow image with magnitudes and orientations of the normal gradients of the model image flows,

and wherein a greater magnitude of a particular one of the normal gradients indicates a greater changing velocity associated with a portion of the object.

14. The computer readable medium of claim 12 further comprising instructions, that when executed by the processor remove erroneous normal gradients by removing normal gradients which have a magnitude outside a predetermined range, and

wherein identifying including using the change in velocity of movement to at least in part identify the type of movement.

15. The computer readable medium of claim 12 further comprising instructions, that when executed by the processor remove erroneous normal gradients by removing normal gradients which are located in a boundary region of the motion region image, the boundary region including edges of some of the silhouettes.

16. The computer readable medium of claim 12 further comprising instructions, that when executed by the processor to calculate normal gradients further result in:

applying a Sobel operator to silhouette pixels of the motion region image to generate a response function;

obtaining the orientations of gradients of the silhouette pixels of the motion region image from the response function; and

obtaining the magnitudes of the gradients of the silhouette pixels from the response function.

17. The computer readable medium of claim 12 further comprising instructions to instruct the processor to identify movement by performing a method of:

analyzing the normal flow image using a histogram of the normal flow image using Chi Square and minimum intersection measures; and

comparing the histogram of the normal flow image to model histograms to identify a type of movement performed by the object.

18. A system comprising:

a camera to obtain a plurality of at least three images of the object taken over a predetermined period of time, the images having time-difference information associated therewith to indicate change in velocity of movement of portions of the object; and

a processor coupled to the camera, to generate a plurality of silhouettes corresponding with the plurality of at least three images, to generate a motion region image of the object from the plurality of silhouettes, to calculate normal gradients for edges of the motion region image,

to remove erroneous normal gradients to provide a normal flow image, and to identify movement based on the normal flow image using a processor,

wherein the normal gradients calculated by the processor have an orientation substantially perpendicular to motion of the edges of the silhouettes.

19. The system of claim 18 wherein a greater magnitude of a particular one of the normal gradients indicates a greater change in velocity associated with a portion of the object, and wherein the processor identifies movement by comparing a histogram of the normal flow image to model histograms to identify a type of movement performed by the object, the histogram using Chi Square and minimum intersection measures, and

wherein identifying including using the change in velocity of movement to at least in part identify the type of movement.