Method and apparatus for low cost motion detection

• US 8,325,227 B2
• Filed: 07/15/2008
• Issued: 12/04/2012
• Est. Priority Date: 07/15/2008
• Status: Active Grant

• Current Assignee: Aptina Imaging Corporation (ON Semiconductor Corporation)
• Original Assignee: Aptina Imaging Corporation (ON Semiconductor Corporation)
• Inventors: Lin, Peng, Sachs, Todd
• Primary Examiner(s): Chan, Wing
• Assistant Examiner(s): MIAH, RAZU A
• Application Number:
  US12/173,327
• Publication Number:
  US 20100013919A1
• Time in Patent Office: 1,603 Days
• Field of Search: 348143-149, 348/155, 37524002-24024
• US Class Current: 348/143
• CPC Class Codes: G06T 2200/28 involving image processing hardware
• CPC Class Codes: G06T 2207/10016 Video; Image sequence
• CPC Class Codes: G06T 2207/20068 Projection on vertical or horizontal image axis
• CPC Class Codes: G06T 2207/30232 Surveillance
• CPC Class Codes: G06T 7/254 involving subtraction of images
• CPC Class Codes: H04N 23/68 for stable pick-up of the scene, e.g. compensating for camera body vibrations
• CPC Class Codes: H04N 5/144 Movement detection for video coding H04N19/503; analysis of motion in general G06T7/20

Abstract

A non-frame-based motion detection method and apparatus for imagers requires only a few line buffers and little computation. The non-frame-based, low cost motion detection method and apparatus are well suited for “system-a-chip” (SOC) imager implementations.

Claims

1. A method of detecting motion of an object in a first image, said method comprising:

using a motion detection circuit, determining weighted row and column averages of a reference image;

using the motion detection circuit, determining weighted row and column averages of the first image;

using the motion detection circuit, detecting motion of the object based on the determined weighted column and row averages of the reference image and the first image;

using the motion detection circuit, shifting the weighted row average of the reference image to compensate for vertical movement of an imager that captured the first image; and

using the motion detection circuit, shifting the weighted column average of the reference image to compensate for horizontal movement of the imager.

2. The method of claim 1, wherein the detecting motion step comprises:

using the motion detection circuit, creating a vertical motion mask based on the determined weighted row average of the first image and the shifted weighted row average of the reference image;

using the motion detection circuit, creating a horizontal motion mask based on the determined weighted column average of the first image and the shifted weighted column average of the reference image; and

using the motion detection circuit, combining the vertical and horizontal motion masks to create a two-dimensional motion mask comprising at least one portion indicative of motion of the object.

3. The method of claim 1 further comprising the step of, using the motion detection circuit, updating the weighted row and column averages of the reference image on a periodic basis.

4. The method of claim 1 further comprising the step of, using the motion detection circuit, updating the weighted row and column averages of the reference image on an aperiodic basis.

5. The method of claim 4, wherein the weighted row and column averages of reference image are updated when a detected motion area becomes greater than a predefined portion of the first image.

6. The method of claim 1, wherein the weighted row and column averages are weighted to detect motion of the object in an area of interest.

7. The method of claim 1, wherein the weighted row and column averages are weighted to detect motion of the object from sub-sampled portions of the first and reference images.

8. A method of detecting motion of an object in a first image, said method comprising:

using a motion detection circuit, determining weighted row and column averages of a reference image;

using the motion detection circuit, determining weighted row and column averages of the first image; and

using the motion detection circuit, detecting motion of the object based on the determined weighted column and row averages of the reference image and the first image, wherein the detecting motion step comprises;

using the motion detection circuit, creating a vertical motion mask based on the determined weighted row averages of the first and reference images;

using the motion detection circuit, creating a horizontal motion mask based on the determined weighted column average of the first and reference images; and

using the motion detection circuit, combining the vertical and horizontal motion masks to create a two-dimensional motion mask comprising at least one portion indicative of motion of the object.

9. The method of claim 8, wherein the combining step comprises, using the motion detection circuit, performing a Cartesian product of the vertical and horizontal motion masks.

10. The method of claim 8, wherein the horizontal motion mask creating step comprises:

using the motion detection circuit, determining an absolute difference of the weighted column averages of the first image and the reference image;

using the motion detection circuit, thresholding the absolute difference based on a predetermined threshold; and

using the motion detection circuit, performing a consolidation process on the thresholded absolute difference to create the horizontal motion mask.

11. The method of claim 8, wherein the vertical motion mask creating step comprises:

using the motion detection circuit, determining an absolute difference of the weighted row averages of the first image and the reference image;

using the motion detection circuit, thresholding the absolute difference based on a predetermined threshold;

using the motion detection circuit, performing a consolidation process on the thresholded absolute difference; and

using the motion detection circuit, connecting non-zero connected components in the consolidated thresholded absolute difference to create the vertical motion mask.

12. A motion detection circuit comprising:

a plurality of line buffers, a first line buffer for storing a weighted row average of a reference image, a second line buffer for storing a weighed row average of a first image, a third line buffer for storing a weighted column average of the reference image and a fourth line buffer for storing a weighted column average of the first image; and

logic for detecting motion of an object in the first image by computing and storing the weighted row and column averages of the reference and first images and detecting the motion of the object based on the determined weighted column and row averages of the reference and first images, wherein said logic shifts the weighted row average of the reference image to compensate for vertical movement of an imager that captured the first image and shifts the weighted column average of the reference image to compensate for horizontal movement of the imager.

13. The motion detection circuit of claim 12, wherein the logic detects motion of the object by:

creating a vertical motion mask based on the determined weighted row average of the first image and the shifted weighted row average of the reference image;

creating a horizontal motion mask based on the determined weighted column average of the first image and the shifted weighted column average of the reference image; and

combining the vertical and horizontal motion masks to create a two-dimensional motion mask comprising at least one portion indicative of motion of the object.

14. The motion detection circuit of claim 12, wherein said logic updates the weighted row and column averages of the reference image on a periodic basis.

15. The motion detection circuit of claim 12, wherein said logic updates the weighted row and column averages of the reference image when a detected motion area becomes greater than a predefined portion of the first image.

16. The motion detection circuit of claim 12, wherein the weighted row and column averages are weighted to detect motion of the object in an area of interest.

17. The motion detection circuit of claim 12, wherein the weighted row and column averages are weighted to detect motion of the object from sub-sampled portions of the first and reference images.

18. A motion detection circuit comprising:

a plurality of line buffers, a first line buffer for storing a weighted row average of a reference image, a second line buffer for storing a weighed row average of a first image, a third line buffer for storing a weighted column average of the reference image and a fourth line buffer for storing a weighted column average of the first image; and

logic for detecting motion of an object in the first image by computing and storing the weighted row and column averages of the reference and first images and detecting the motion of the object based on the determined weighted column and row averages of the reference and first images, wherein the logic detects motion of the object by;

creating a vertical motion mask based on the determined weighted row averages of the first and reference images;

creating a horizontal motion mask based on the determined weighted column average of the first and reference images; and

combining the vertical and horizontal motion masks to create a two-dimensional motion mask comprising at least one portion indicative of motion of the object.

19. The motion detection circuit of claim 18, wherein the logic combines the horizontal and vertical motion masks by performing a Cartesian product of the vertical and horizontal motion masks.

20. The motion detection circuit of claim 18, wherein the logic creates the horizontal motion mask by:

determining an absolute difference of the weighted column averages of the first image and the reference image;

thresholding the absolute difference based on a predetermined threshold; and

performing a consolidation process on the thresholded absolute difference to create the horizontal motion mask.

21. The motion detection circuit of claim 18, wherein the logic creates the vertical motion mask by:

determining an absolute difference of the weighted row averages of the first image and the reference image;

thresholding the absolute difference based on a predetermined threshold;

performing a consolidation process on the thresholded absolute difference; and

connecting non-zero connected components in the consolidated thresholded absolute difference to create the vertical motion mask.

22. An imager comprising:

an image sensor for capturing a reference image and a first image and for outputting digital reference image data and first image data; and

a motion detection circuit connected to input the reference image data and the first image data, said circuit comprising;

a plurality of line buffers, a first line buffer for storing a weighted row average of the reference image, a second line buffer for storing a weighed row average of the first image, a third line buffer for storing a weighted column average of the reference image and a fourth line buffer for storing a weighted column average of the first image; and

a processor programmed to perform motion of an object in the first image, said processor performing a first process to compute and store the weighted row average of the reference image, a second process to compute and store the weighted column average of the reference image, a third process to compute and store the weighted row average of the first image, a fourth process to compute and store the weighted column average of the first image, and a fifth process for detecting the motion of the object based on the determined weighted column and row averages of the reference and first images, wherein the processor performs a sixth process for shifting the weighted row average of the reference image to compensate for vertical movement of the image sensor and a seventh process for shifting the weighted column average of the reference image to compensate for horizontal movement of the image sensor.

23. The imager of claim 22, wherein the weighted row and column averages are weighted to detect motion of the object in an area of interest.

24. The imager of claim 22, wherein the weighted row and column averages are weighted to detect motion of the object from sub-sampled portions of the first and reference images.

25. The imager of claim 22, wherein said processor updates the weighted row and column averages of the reference image on a periodic basis and when a detected motion area becomes greater than a predefined portion of the first image.

26. An imager comprising:

an image sensor for capturing a reference image and a first image and for outputting digital reference image data and first image data; and

a motion detection circuit connected to input the reference image data and the first image data, said circuit comprising;

a plurality of line buffers, a first line buffer for storing a weighted row average of the reference image, a second line buffer for storing a weighed row average of the first image, a third line buffer for storing a weighted column average of the reference image and a fourth line buffer for storing a weighted column average of the first image; and

a processor programmed to perform motion of an object in the first image, said processor performing a first process to compute and store the weighted row average of the reference image, a second process to compute and store the weighted column average of the reference image, a third process to compute and store the weighted row average of the first image, a fourth process to compute and store the weighted column average of the first image, and a fifth process for detecting the motion of the object based on the determined weighted column and row averages of the reference and first images, wherein the fifth process comprises;

a process for creating a vertical motion mask based on the determined weighted row averages of the first and reference images;

a process for creating a horizontal motion mask based on the determined weighted column average of the first and reference images; and

a process for combining the vertical and horizontal motion masks to create a two-dimensional motion mask comprising at least one portion indicative of motion of the object.

27. An imager comprising:

an image sensor for capturing a reference image and a first image and for outputting digital reference image data and first image data; and

a motion detection circuit connected to input the reference image data and the first image data, said circuit comprising;

a plurality of line buffers, a first line buffer for storing a weighted row average of the reference image, a second line buffer for storing a weighed row average of the first image, a third line buffer for storing a weighted column average of the reference image and a fourth line buffer for storing a weighted column average of the first image; and

a processor programmed to perform motion of an object in the first image, said processor performing a first process to compute and store the weighted row average of the reference image, a second process to compute and store the weighted column average of the reference image, a third process to compute and store the weighted row average of the first image, a fourth process to compute and store the weighted column average of the first image, and a fifth process for detecting the motion of the object based on the determined weighted column and row averages of the reference and first images, wherein the weighted row average A_row (f)(y) of the reference image or first image is computed according to

28. An imager comprising:

an image sensor for capturing a reference image and a first image and for outputting digital reference image data and first image data; and

a motion detection circuit connected to input the reference image data and the first image data, said circuit comprising;

a plurality of line buffers, a first line buffer for storing a weighted row average of the reference image, a second line buffer for storing a weighed row average of the first image, a third line buffer for storing a weighted column average of the reference image and a fourth line buffer for storing a weighted column average of the first image; and

a processor programmed to perform motion of an object in the first image, said processor performing a first process to compute and store the weighted row average of the reference image, a second process to compute and store the weighted column average of the reference image, a third process to compute and store the weighted row average of the first image, a fourth process to compute and store the weighted column average of the first image, and a fifth process for detecting the motion of the object based on the determined weighted column and row averages of the reference and first images, wherein the weighted column average A_col (f)(x) of the reference image or first image is computed according to