Moving object detection device and moving object detection method

• US 20060262857A1
• Filed: 03/01/2006
• Published: 11/23/2006
• Est. Priority Date: 11/20/2003
• Status: Active Grant

• Current Assignee: Panasonic Corporation (Panasonic Holdings Corporation)
• Original Assignee: Panasonic Corporation (Panasonic Holdings Corporation)
• Inventors: Iwasaki, Masahiro, Imagawa, Taro, Kindo, Toshiki
• Granted Patent: US 7,702,019 B2
• Time in Patent Office: Days
• US Class Current: 375/240.190
• CPC Class Codes: G06T 7/20 Analysis of motion motion estimation for coding, decoding, compressing or decompressing digital video signals H04N19/43, H04N19/51
• CPC Class Codes: G06V 40/25 Recognition of walking or running movements, e.g. gait recognition

Abstract

The present invention provides a moving object detection device that includes: a spatiotemporal data generation unit 120 which generates time series data which arranges data indicating a moving object along a temporal axis, based on an output from a camera 100; an inter-leg information unit 140 which extracts, based on the generated time series data, inter-leg information, which is information regarding a temporal change in an inter-leg area arising from movement of a moving object that has two or more legs; a periodicity analysis unit 150 which analyzes a periodicity within the extracted inter-leg information; and a moving object detection unit 160 which generates, from the analyzed periodicity, movement information that includes the presence or lack thereof of a moving object.

Claims

1. A moving object detection device that detects a moving object based on an output from a target object detection sensor installed one- or two-dimensionally, said device comprising:

a spatiotemporal data generation unit operable to generate, based on the output from the target object detection sensor, time series data which indicates a spatiotemporal image in which two-dimensional images that show a moving object are arranged along a time axis;

a spatiotemporal fragment extraction unit operable to extract a spatiotemporal fragment, which is an image found in a cut plane or cut fragment when the spatiotemporal image is cut along the time axis;

an inter-leg information extraction unit operable to extract, based on the extracted spatiotemporal fragment, inter-leg information which is information regarding an inter-leg temporal change arising from movement of a moving object that has two or more legs;

a periodicity analysis unit operable to analyze a periodicity found in the extracted inter-leg information;

a movement information generation unit operable to generate, from the analyzed periodicity, movement information that includes whether or not the moving object is present; and

a parameter change unit operable to change, based on the generated movement information, spatiotemporal fragment extraction parameters, which are parameters that specify the cut plane or cut fragment,

wherein said spatiotemporal fragment extraction unit is operable to extract, as the spatiotemporal fragment, an image found when the spatiotemporal image is cut by the cut plane or cut fragment indicated by the spatiotemporal fragment extraction parameters changed by said parameter change unit.

2. The moving object detection device according to claim 1,

wherein said inter-leg information extraction unit is operable to extract, as the inter-leg information, a temporal interval or a spatial interval indicated by a step length of the moving object.

3. The moving object detection device according to claim 1,

wherein said inter-leg information extraction unit is operable to extract, as the inter-leg information, data that indicates a temporal change in a correlation length, which is a spatial interval in which an autocorrelation value is a minimum value or a peak value, by calculating, per one-dimensional data, the correlation length, the one-dimensional data being data per unit time that forms the spatiotemporal fragment.

4. The moving object detection device according to claim 1,

wherein said parameter change unit is operable to change the spatiotemporal fragment extraction parameters until movement information, which indicates that a moving object exists, is generated by said movement information generation unit.

5. The moving object detection device according to claim 4, further comprising

a movement direction calculation unit operable to calculate, from time series data generated by said spatiotemporal data generation unit, a movement direction of the moving object included in the time series data,

wherein said spatiotemporal fragment extraction unit is operable to cut the spatiotemporal image in accordance with the movement direction.

6. The moving object detection device according to claim 1,

wherein said spatiotemporal data generation unit is further operable to divide each image that forms the spatiotemporal data into subregions, transform each image into parameters as an image characteristic volume per divided subregion, and generate an N-dimensional parameter vector which results from collecting the obtained parameters for N images, and

said inter-leg information extraction unit is further operable to extract the inter-leg information based on the N-dimensional parameter vector.

7. The moving object detection device according to claim 6, further comprising:

a time series combination generation unit operable to generate, from the N-dimensional parameter vector, a time series combination of parameter vectors, by combining parameters that have a predetermined relationship in time series,

wherein said inter-leg information extraction unit is operable to extract the inter-leg information based on the time series combination.

8. The moving object detection device according to claim 7,

wherein said inter-leg information extraction unit is operable to calculate, for the time series combination, a correlation length, which is a spatial interval in which an autocorrelation value is a minimum value or a peak value, and to extract, as the inter-leg information, data that indicates a temporal change in the calculated correlation length.

9. The moving object detection unit according to claim 6,

wherein the N-dimensional parameter vector includes, as the image characteristic volume, one of a wavelet transform coefficient, a Hough transform parameter, and a frequency based on a Fourier transform.

10. The moving object detection device according to claim 6,

wherein the target object detection sensor is a camera that generates a two-dimensional image, and

said spatiotemporal data generation unit is operable to generate the subregions by dividing the two-dimensional image into sizes determined in accordance with setting parameters of the camera, and generate the N-dimensional parameter vector for the generated subregions.

11. The moving object detection device according to claim 1,

wherein said inter-leg information extraction unit is further operable to generate a time series of parameters specifying a geometric figure, by applying the geometric figure that indicates a temporal change in the inter-leg area to the time series data, and to extract the inter-leg information based on the generated time series of parameters.

12. The moving object detection device according to claim 11,

wherein the geometric figure is two or more straight lines which have modeled legs of an animal including a human, and the parameters are an angle at which the straight lines intersect.

13. The moving object detection device according to claim 12,

wherein said inter-leg information extraction unit is operable to extract, as the inter-leg information, information that indicates a temporal change in the angle.

14. The moving object detection device according to claim 1,

wherein said periodicity analysis unit is operable to transform the inter-leg information into a frequency and analyze the periodicity from the obtained frequency spectrum.

15. The moving object detection device according to claim 1,

wherein the target object detection sensor generates images, and

said spatiotemporal data generation unit is operable to generate, as the time series data, time series image data in which the images generated by the target object detection sensor are arranged along the time axis.

16. The moving object detection device according to claim 15,

wherein said spatiotemporal data generation unit is operable to generate the time series image data by performing a Hough transform, a Fourier transform, or a wavelet transform on the images.

17. The moving object detection device according to claim 1,

wherein said movement information generation unit is operable to specify an autocorrelation peak position in the inter-leg information, judge whether or not the moving object exists by judging whether or not the isolated peak position matches with a movement period unique to the moving object, and generate movement information indicating the judgment result.

18. The moving object detection device according to claim 17,

wherein said movement information generation unit is, in the case where it is judged that the moving object exists, further operable to isolate, and generate as the movement information, a presence position and a movement direction of the moving object.

19. A moving object detection method for detecting a moving object based on an output from a target object detection sensor installed one- or two-dimensionally, said method comprising:

generating, based on the output from the target object detection sensor, time series data which indicates a spatiotemporal image in which two-dimensional images that show a moving object are arranged along a time axis;

extracting a spatiotemporal fragment, which is an image found in a cut plane or cut fragment when the spatiotemporal image is cut along the time axis;

extracting, based on the extracted spatiotemporal fragment, inter-leg information which is information regarding an inter-leg temporal change arising from movement of a moving object that has two or more legs;

analyzing a periodicity found in the extracted inter-leg information;

generating, from the analyzed periodicity, movement information that includes whether or not the moving object is present; and

changing, based on the generated movement information, spatiotemporal fragment extraction parameters, which are parameters that specify the cut plane or cut fragment,

wherein an image found when the spatiotemporal image is cut by the cut plane or cut fragment indicated by the spatiotemporal fragment extraction parameters changed in said spatiotemporal fragment extraction parameter changing is extracted as the spatiotemporal fragment in said spatiotemporal fragment extracting.

20. A program for detecting a moving object based on an output from a target object detection sensor installed one- or two-dimensionally, said program causing a computer to execute the following:

generating, based on the output from the target object detection sensor, time series data which indicates a spatiotemporal image in which two-dimensional images that show a moving object are arranged along a time axis;

extracting a spatiotemporal fragment, which is an image found in a cut plane or cut fragment when the spatiotemporal image is cut along the time axis;

extracting, based on the extracted spatiotemporal fragment, inter-leg information which is information regarding an inter-leg temporal change arising from movement of a moving object that has two or more legs;

analyzing a periodicity found in the extracted inter-leg information;

generating, from the analyzed periodicity, movement information that includes whether or not the moving object is present; and

changing, based on the generated movement information, spatiotemporal fragment extraction parameters, which are parameters that specify the cut plane or cut fragment,

wherein an image found when the spatiotemporal image is cut by the cut plane or cut fragment indicated by the spatiotemporal fragment extraction parameters changed in said spatiotemporal fragment extraction parameter changing is extracted as the spatiotemporal fragment in said spatiotemporal fragment extracting.