Motion detection apparatus and motion detection method

• US 7,602,946 B2
• Filed: 06/07/2005
• Issued: 10/13/2009
• Est. Priority Date: 09/24/2004
• Status: Expired due to Fees

• Current Assignee: Nissan Motor Co., Ltd., Shell Oil Company (Shell Plc)
• Original Assignee: Nissan Motor Co., Ltd.
• Inventors: Watanabe, Seigo
• Primary Examiner(s): Bali; Vikkram
• Assistant Examiner(s): Park; Soo Jin
• Application Number:
  US11/146,041
• Publication Number:
  US 20060078165A1
• Time in Patent Office: 1,589 Days
• Field of Search: 382/107
• US Class Current: 382/107
• CPC Class Codes: G06T 2207/10016 Video; Image sequence
• CPC Class Codes: G06T 2207/30241 Trajectory
• CPC Class Codes: G06T 2207/30248 Vehicle exterior or interior
• CPC Class Codes: G06T 7/248 involving reference images or patches
• CPC Class Codes: G06T 7/73 using feature-based methods

Abstract

A motion detection device includes a feature extracting section 2 extracting a feature area of an object from among frame images clipped by an image pickup section 1, a feature standardizing section 3 standardizing a size of a feature area extracted by the feature extracting section 2, a balloting section 4 counting up a ballot value of an image area on a frame image located in a standardized pattern that is standardized by the feature standardizing section 3, and a shift speed detecting section 5 calculating a gradient of a counted-up ballot value and calculating a shift direction, a shift speed and position of a feature area based on the gradient.

Claims

1. A motion detection apparatus for detecting a motion of an object, comprising:

an image pickup section picking up a motion image and successively clipping a time series of frame images;

a feature extracting section extracting a first feature area, representing a feature of the object, from a first frame image among the time series of frame images;

a feature area standardizing section standardizing a size of the first feature area to be set as a first standardized feature area;

a mask generating section setting an area on the first frame image, which is a complement of the first standardized feature area in the first frame image, as a first masked area;

a balloting section resetting a count value associated with each pixel when the relevant pixel is located on the first masked area, and incrementing the count value when the relevant pixel is located on a first on-masked area that is the first standardized feature area; and

shift speed calculating section calculating a shift direction and shift speed of the first feature area based on a spatial gradient of the count values, accumulated for the time series of frame images, associated with the pixels located on the first standardized feature area, wherein the feature extracting section detects a corner edge component of the object as the first feature area.

2. A motion detection apparatus for detecting a motion of an object, comprising:

an image pickup section picking up a motion image and successively clipping a time series of frame images;

a feature extracting section extracting a first feature area, representing a feature of the object, from a first frame image among the time series of frame images;

a feature area standardizing section standardizing a size of the first feature area to be set as a first standardized feature area;

a mask generating section setting an area on the first frame image, which is a complement of the first standardized feature area in the first frame image, as a first masked area;

a balloting section resetting a count value associated with each pixel when the relevant pixel is located on the first masked area, and incrementing the count value when the relevant pixel is located on a first non-masked area that is the first standardized feature area;

a shift speed calculating section calculating a shift direction and shift speed of the first feature area based on a spatial gradient of the count values, accumulated for the time series of frame images, associated with the pixels located on the first standardized feature area; and

a reset timing generating section that sets a reset timing at which the count values of the pixels located on the first non-masked area and a setting of the first masked area are reset, based on a positional relationship between the first standardized feature area and a second standardized feature area in which a size of a second feature area adjacent to the first feature area is standardized.

3. The motion detection apparatus according to claim 2, wherein the reset timing generating section generates the reset timing, when the non-masked area of the first standardized feature area and the non-masked area of the second standardized feature area overlap each other, and resets the count values of the pixels located on the first non-masked area and the setting of the first masked area.

4. A motion detection apparatus for detecting a motion of an object, comprising:

an image pickup section picking up a motion image and successively clipping a time series of frame images;

a feature extracting section extracting a first feature area, representing a feature of the object, from a first frame image among the time series of frame images;

a feature area standardizing section standardizing a size of the first feature area to be set as a first standardized feature area;

a mask generating section setting an area on the first frame image, which is a complement of the first standardized feature area in the first frame image, as a first masked area;

a balloting section resetting a count value associated with each pixel when the relevant pixel is located on the first masked area, and incrementing the count value when the relevant pixel is located on a first non-masked area that is the first standardized feature area; and

a shift speed calculating section calculating a shift direction and shift speed of the first feature area based on a spatial gradient of the count values, accumulated for the time series of frame images, associated with the pixels located on the first standardized feature area, wherein the mask generating section sets an area, generated by a logical product between the first masked area in the first frame image, and an area, which is the complement of the first standardized feature area in the second frame image adjacent to the first frame image in time series, as a masked area of the first standardized feature area in the second frame image.

5. The motion detection apparatus according to claim 4 wherein the feature extracting section detects an edge component of the object as the first feature area.

6. The motion detection apparatus according to claim 4, wherein the feature extracting section detects a spatial feature vector of the object as the first feature area.

7. The motion detection apparatus according to claim 4, wherein the feature area standardizing section specifies a representative point from the first feature area to represent the first standardized feature area.

8. The motion detection apparatus according to claim 4 wherein the shift speed calculating section calculates a position of the first feature area based on the spatial gradient of the accumulated count values associated with the pixels located on the first standardized feature area and a minimal count value in the accumulated count values.

9. The motion detection apparatus according to claim 4, wherein the feature area standardizing section specifies a size of standardization depending on the shift speed of the first feature area.

10. The motion detection apparatus according to claim 4, further comprising:

a vanishing point detecting section detecting a vanishing point on the first frame image; and

a ballot timing control section controlling a timing at which the balloting section increments the count values of the pixels located on the first non-masked area and a timing at which the balloting section resets the count values of the pixels located on the first masked area, based on location of the vanishing point on the first frame image, wherein the ballot timing control section allows a timing at which the count values of the pixels located on an area on the first non-masked area remote from the vanishing point are incremented, to be shorter than that at which the count values of the pixels located on an area on the first non-masked area closer to the vanishing point are incremented, and allows a timing at which the counted values of the pixels located on an area on the first masked area remote from the vanishing point are reset, to be shorter than that at which the counted values of the pixels located on an area on the first masked area closer to the vanishing point are reset.

11. The motion detection apparatus according to claim 10, wherein the timing control section is configured to set a polar coordinate system with an origin at the vanishing point to control the timing at which the balloting section increments the count values of the pixels located on the first non-masked area and the timing at which the balloting section resets the count values of the pixels located on the first masked area, in the polar coordinate system.

12. A motion detection method for detecting a motion of an object, comprising:

an image picking-up step for picking up a motion image and successively clipping a time series of frame images;

a feature extracting step for extracting, by a feature extracting section, a first feature area representing a feature of the object, from a first frame image among the time series of frame images;

a feature area standardizing step for standardizing a size of the first feature area to be set as a first standardized feature area;

a mask generating step for setting an area on the first frame image, which is a complement of the first standardized feature area in the first frame image, as a first masked area;

a balloting step for resetting a count value associated with each pixel when the relevant pixel is located on the first masked area, and incrementing the count value when the relevant pixel is located on a first non-masked area that is the first standardized feature area; and

a shift speed calculating step for calculating a shift direction and shift speed of the first feature area based on a spatial gradient of the count values, accumulated for the time series of frame images, associated with the pixels located on the first standardized feature area, wherein the feature extracting step detects a corner edge component of the object as the first feature area.

13. A motion detection method for detecting a motion of an object, comprising:

an image picking-up step for picking up a motion image and successively clipping a time series of frame images;

a feature extracting step for extracting, by a feature extracting section, a first feature area, representing a feature of the object, from a first frame image among the time series of frame images;

a feature area standardizing step for standardizing a size of the first feature area to be set as a first standardized feature area;

a mask generating step for setting an area on the first frame image, which is a complement of the first standardized feature area in the first frame image, as a first masked area;

a balloting step for resetting a count value associated with each pixel when the relevant pixel is located on the first masked area, and incrementing the count value when the relevant pixel is located on a first non-masked area that is the first standardized feature area;

a shift speed calculating step for calculating a shift direction and shift speed the first feature area based on a spatial gradient of the count values, accumulated for the time series of frame images, associated with the pixels located on the first standardized feature area; and

a reset timing generating step for setting a reset timing at which the count values of the pixels located on the first non-masked area and a setting of the first masked area are reset, based on a positional relationship between the first standardized feature area and a second standardized feature area in which a size of a second feature area adjacent to the first feature area is standardized.

14. The motion detection method according to claim 13, wherein the reset timing generating steps generates the reset timing, when the non-masked area of the first standardized feature area and the non-masked area of the second standardized feature area overlap each other, and resets the count values of the pixels located on the first non-masked area and the setting of the first masked area to be executed by the balloting step.

15. A motion detection method for detecting a motion of an object, comprising:

an image picking-up step for picking up a motion image and successively clipping a time series of frame images;

a feature extracting step for extracting, by a feature extracting section, a first feature area, representing a feature of the object, from a first frame image among the time series of frame images;

a feature area standardizing step for standardizing a size of the first feature area to be set as a first standardized feature area;

a mask generating step for setting an area on the first frame image, which is a complement of the first standardized feature area in the first frame image, as a first masked area;

a balloting step for resetting a count value associated with each pixel when the relevant pixel is located on the first masked area, and incrementing the count value when the relevant pixel it located on a first non-masked area that is the first standardized feature area; and

a shift speed calculating step for calculating a shift direction and shift speed of the first feature area based on a spatial gradient of the count values, accumulated for the time series of frame images, associated with the pixels located on the first standardized feature area, wherein the mask generating step sets an area, generated by a logical product between the first masked area in the first frame image, and an area, which is the complement of the first standardized feature area in the second frame image adjacent to the first frame image in time series, as a masked area of the first standardized feature area in the second frame image.

16. The motion detection method according to claim 15 wherein the feature extracting step detects an edge component of the object as the first feature are a.

17. The motion detection method according to claim 15, wherein the feature extracting step detects a spatial feature vector of the object as the first feature area.

18. The motion detection method according to claim 15, wherein the feature area standardizing step specifies a representative point from the first feature area to represent the first standardized feature area.

19. The motion detection method according to claim 15, wherein the shift speed calculating step calculates a position of the first feature area based on the spatial gradient of the accumulated count values associated with the pixels located on the first standardized feature area and a minimal count value in the accumulated count values.

20. The motion detection method according to claim 15, wherein the feature area standardizing step specifies a size of standardization depending on the shift speed of the first feature area.

21. The motion detection method according to claim 15, further comprising:

a vanishing point detecting step detecting a vanishing point on the first frame image; and

a ballot timing control step controlling a timing at which the balloting step increments the count values of the pixels located on the first non-masked area and resets the count values of the pixels located on the first masked area, based on location of the vanishing point on the first frame image, detected in the vanishing point detecting step, wherein the ballot timing control step allows a timing at which the count value of the pixel located on an area remote from the vanishing point detected by the vanishing point detecting step is incremented and reset, to be shorter than that at which the count value of the pixel located on the area closer to the vanishing point is incremented and reset.

22. A motion detection apparatus for detecting a motion of an object, comprising:

image pickup means for picking up a motion image and consecutively clipping a time series of frame images;

feature extracting means for extracting a first feature area, representing a feature of the object, from a first frame image among the time series of frame images;

feature area standardizing means for standardizing a size of the first feature area to be set as a first standardized feature area;

mask generating means for setting an area on the first frame image, which is a complement of the first standardized feature area in the first frame image, as a first masked area;

balloting means for resetting a count value associated with each pixel when the relevant pixel is located on the first masked area, and incrementing the count value when the relevant pixel is located on a first non-masked area that is the first standardized feature area; and

shift speed calculating means for calculating a shift direction and shift speed of the first feature area based on a spatial gradient of the count values, accumulated for the time series of frame images, associated with the pixels located on the first standardized feature area, wherein the mask generating means sets an area, generated by a logical product between the first masked area in the first frame image, and an area, which is the complement of the first standardized feature area in the second frame image adjacent to the first frame image in time series, as a masked area of the first standardized feature area in the second frame image.