Moving object detection device and moving object detection method

• US 7,702,019 B2
• Filed: 03/01/2006
• Issued: 04/20/2010
• Est. Priority Date: 11/20/2003
• Status: Active Grant

• Current Assignee: Panasonic Corporation (Panasonic Holdings Corporation)
• Original Assignee: Panasonic Corporation (Panasonic Holdings Corporation)
• Inventors: Iwasaki, Masahiro, Imagawa, Taro, Kindo, Toshiki
• Primary Examiner(s): Philippe; Gims S
• Application Number:
  US11/364,284
• Publication Number:
  US 20060262857A1
• Time in Patent Office: 1,511 Days
• Field of Search: 382/104, 382/224, 375/240.24
• US Class Current: 375/240.16
• CPC Class Codes: G06T 7/20 Analysis of motion motion estimation for coding, decoding, compressing or decompressing digital video signals H04N19/43, H04N19/51
• CPC Class Codes: G06V 40/25 Recognition of walking or running movements, e.g. gait recognition

Abstract

A moving object detection device including a spatiotemporal data generation unit 120 generating time series data, which arranges data indicating a moving object along a temporal axis, based on an output from a camera 100, an inter-leg information unit 140 extracting based on the generated time series data, inter-leg information, which is information regarding a temporal change in an inter-leg area arising from movement of a moving object that has two or more legs, and a periodicity analysis unit 150 analyzing a periodicity within the extracted inter-leg information. Further, the moving object detection devices includes a moving object detection unit 160 generating, from the analyzed periodicity, movement information that includes the presence or lack thereof of a moving object.

Claims

1. A moving object detection device that detects a moving object based on an output from a target object detection sensor installed one- or two-dimensionally, said device comprising:

a spatiotemporal data generation unit operable to generate, based on the output from the target object detection sensor, time series data which indicates a spatiotemporal image in which two-dimensional images that show the moving object are arranged along a time axis;

a spatiotemporal fragment extraction unit operable to extract a spatiotemporal fragment from the spatiotemporal image indicated by the generated time series data, the spatiotemporal fragment being an image found in a cut plane or a cut fragment when the spatiotemporal image is cut along the time axis;

an inter-leg information extraction unit operable to extract, based on the extracted spatiotemporal fragment, inter-leg information, which is information regarding an inter-leg temporal change arising from a movement of the moving object that has two or more legs;

a periodicity analysis unit operable to analyze a periodicity found in the extracted inter-leg information;

a movement information generation unit operable to generate, from the analyzed periodicity, movement information that indicates whether or not the moving object is present; and

a parameter change unit operable to change, based on the generated movement information, spatiotemporal fragment extraction parameters, the spatiotemporal fragment extraction parameters specifying the cut plane or the cut fragment,

wherein said spatiotemporal fragment extraction unit is operable to extract, as the spatiotemporal fragment, an image found when the spatiotemporal image is cut by the cut plane or the cut fragment specified by the spatiotemporal fragment extraction parameters changed by said parameter change unit.

2. The moving object detection device according to claim 1,

wherein said inter-leg information extraction unit is operable to extract, as the inter-leg information, a temporal interval or a spatial interval indicated by a step length of the moving object.

3. The moving object detection device according to claim 1,

wherein said inter-leg information extraction unit is operable to extract, as the inter-leg information, data that indicates a temporal change in a correlation length, which is a spatial interval in which an autocorrelation value is a minimum value or a peak value, the correlation length being calculated per one-dimensional data, and the one-dimensional data being data per unit time that forms the spatiotemporal fragment.

4. The moving object detection device according to claim 1,

wherein said parameter change unit is operable to change the spatiotemporal fragment extraction parameters until movement information, which indicates that the moving object exists, is generated by said movement information generation unit.

5. The moving object detection device according to claim 4, further comprising a movement direction calculation unit operable to calculate, from time series data generated by said spatiotemporal data generation unit, a movement direction of the moving object indicated by the time series data,

wherein said spatiotemporal fragment extraction unit is operable to cut the spatiotemporal image in accordance with the movement direction.

6. The moving object detection device according to claim 1,

wherein said spatiotemporal data generation unit is further operable to divide each respective image that forms the spatiotemporal image into subregions, transform each respective image that form the spatiotemporal image into parameters, as an image characteristic volume, per each subregion of the respective image, to obtain parameters for N images, and generate an N-dimensional parameter vector which results from collecting the obtained parameters for N images, and

wherein said inter-leg information extraction unit is further operable to extract the inter-leg information based on the N-dimensional parameter vector.

7. The moving object detection device according to claim 6, further comprising a time series combination generation unit operable to generate, from the N-dimensional parameter vector, a time series combination of parameter vectors, by combining parameters that have a predetermined relationship in time series,

wherein said inter-leg information extraction unit is operable to extract the inter-leg information based on the time series combination.

8. The moving object detection device according to claim 7,

wherein said inter-leg information extraction unit is operable to calculate, for the time series combination, a correlation length, which is a spatial interval in which an autocorrelation value is a minimum value or a peak value, and to extract, as the inter-leg information, data that indicates a temporal change in the calculated correlation length.

9. The moving object detection unit according to claim 6,

wherein the N-dimensional parameter vector includes, as the image characteristic volume, one of a wavelet transform coefficient, a Hough transform parameter, and a frequency based on a Fourier transform.

10. The moving object detection device according to claim 6,

wherein the target object detection sensor is a camera that generates a two-dimensional image, and

wherein said spatiotemporal data generation unit is operable to generate the subregions by dividing the two-dimensional image into sizes determined in accordance with setting parameters of the camera, and generate the N-dimensional parameter vector for the generated subregions.

11. The moving object detection device according to claim 1,

wherein said inter-leg information extraction unit is farther operable to generate a time series of parameters specifying a geometric figure, by applying the geometric figure that indicates a temporal change in an inter-leg area to the time series data, and to extract the inter-leg information based on the generated time series of parameters.

12. The moving object detection device according to claim 11,

wherein the geometric figure is two or more straight lines which have modeled legs of an animal including a human, and the parameters specifying the geometric figure are an angle at which the straight lines intersect.

13. The moving object detection device according to claim 12,

wherein said inter-leg information extraction unit is operable to extract, as the inter-leg information, information that indicates a temporal change in the angle.

14. The moving object detection device according to claim 1,

wherein said periodicity analysis unit is operable to transform the inter-leg information into a frequency and analyze the periodicity from a spectrum of the frequency obtained by transforming the inter-leg information.

15. The moving object detection device according to claim 1,

wherein the target object detection sensor generates images, and

wherein said spatiotemporal data generation unit is operable to generate, as the time series data, time series image data in which the images generated by the target object detection sensor are arranged along the time axis.

16. The moving object detection device according to claim 15,

wherein said spatiotemporal data generation unit is operable to generate the time series image data by performing a Hough transform, a Fourier transform, or a wavelet transform on the images generated by the target object detection sensor.

17. The moving object detection device according to claim 1,

wherein said movement information generation unit is operable to specify an autocorrelation peak position in the inter-leg information, judge whether or not the moving object exists by judging whether or not the isolated peak position matches a movement period unique to the moving object, and generate movement information indicating a result of the judgment.

18. The moving object detection device according to claim 17,

wherein said movement information generation unit is, when said movement information generation unit judges that the moving object exists, further operable to isolate, and generate as the movement information, a presence position and a movement direction of the moving object.

19. A moving object detection method for detecting a moving object based on an output from a target object detection sensor installed one- or two-dimensionally, said method comprising:

generating, based on the output from the target object detection sensor, time series data which indicates a spatiotemporal image in which two-dimensional images that show the moving object are arranged along a time axis;

extracting a spatiotemporal fragment from the spatiotemporal image indicated by the generated time series data, the spatiotemporal fragment being an image found in a cut plane or a cut fragment when the spatiotemporal image is cut along the time axis;

extracting, based on the extracted spatiotemporal fragment, inter-leg information, which is information regarding an inter-leg temporal change arising from a movement of the moving object that has two or more legs;

analyzing a periodicity found in the extracted inter-leg information;

generating, from the analyzed periodicity, movement information that indicates whether or not the moving object is present; and

changing, based on the generated movement information, spatiotemporal fragment extraction parameters, the spatiotemporal fragment extraction parameters specifying the cut plane or the cut fragment,

wherein an image found when the spatiotemporal image is cut by the cut plane or the cut fragment specified by the spatiotemporal fragment extraction parameters changed in said changing of the spatiotemporal fragment extraction parameter is extracted as the spatiotemporal fragment in said extracting of the spatiotemporal fragment.

20. A computer-readable recording medium having a program recorded thereon, the program for detecting a moving object based on an output from a target object detection sensor installed one- or two-dimensionally, said program causing a computer to execute a method comprising:

generating, based on the output from the target object detection sensor, time series data which indicates a spatiotemporal image in which two-dimensional images that show the moving object are arranged along a time axis;

extracting a spatiotemporal fragment from the spatiotemporal image indicated by the generated time series data, the spatiotemporal fragment being an image found in a cut plane or a cut fragment when the spatiotemporal image is cut along the time axis;

extracting, based on the extracted spatiotemporal fragment, inter-leg information, which is information regarding an inter-leg temporal change arising from a movement of the moving object that has two or more legs;

analyzing a periodicity found in the extracted inter-leg information;

generating, from the analyzed periodicity, movement information that indicates whether or not the moving object is present; and

changing, based on the generated movement information, spatiotemporal fragment extraction parameters, the spatiotemporal fragment extraction parameters specifying the cut plane or the cut fragment,

wherein an image found when the spatiotemporal image is cut by the cut plane or the cut fragment specified by the spatiotemporal fragment extraction parameters changed in said changing of the spatiotemporal fragment extraction parameter is extracted as the spatiotemporal fragment in said extracting of the spatiotemporal fragment.