Moving body detecting apparatus

• US 7,433,494 B2
• Filed: 09/16/2003
• Issued: 10/07/2008
• Est. Priority Date: 09/19/2002
• Status: Expired due to Fees

• Current Assignee: DENSO Corporation
• Original Assignee: DENSO Corporation
• Inventors: Niwa, Akimasa
• Primary Examiner(s): Mariam; Daniel G
• Application Number:
  US10/662,401
• Publication Number:
  US 20040057600A1
• Time in Patent Office: 1,848 Days
• Field of Search: 382/103, 382/107, 382/190, 382/203, 382/218, 382/236, 382/224, 348/169
• US Class Current: 382/103
• CPC Class Codes: B60R 25/104 characterised by the type of theft warning signal, e.g. visual or audible signals with special characteristics
• CPC Class Codes: B60R 25/305 using a camera
• CPC Class Codes: G06T 7/215 Motion-based segmentation
• CPC Class Codes: G06T 7/254 involving subtraction of images

Abstract

Provided is a moving body detecting apparatus capable of detecting only a body to be notified without detecting a body similar in feature to the body to be notified. To confirm whether a break-in body appears in static images from a camera, the apparatus sets a monitoring region for monitoring a window portion in the static images and a monitoring region for monitoring a door body portion therein, and produces difference images from the static images. A pixel set existing in common to the difference images continuing in time sequence is extracted from an image composed so that pixel values reach a maximum. On the basis of the motion vector, a decision is made as to whether the movement of the pixel set from the window portion to the door body portion occurs or not.

Claims

1. A moving body detecting apparatus comprising:

image acquiring means for acquiring a plurality of static images of a predetermined region of a real space sequentially in time; and

extracting means for extracting a figure indicating a candidate moving body from said plurality of static images;

setting means for setting a watching region in said predetermined region of said real space, the watching region defining a high probability of existence of said candidate moving body;

first determining means for determining whether or not said candidate moving body is a moving body which moves in said watching region by using information about said figure indicating said candidate moving body extracted from said plurality of static images by said extracting means;

classifying means for classifying said watching region of two or more of the images into either an allowable area or an unallowable area based on an extracted result of the extracting means, said allowable area allowing said moving body to exist therein and said unallowable area prohibiting said moving body from existing therein; and

second determining means for determining that said moving body moves only when said figure indicating said candidate moving body moves from said allowable area to said unallowable area.

2. The apparatus according to claim 1, wherein each of said allowable area and said unallowable area is prescribed by a boundary line of its outer circumference.

3. The apparatus according to claim 1, further comprising:

difference calculating means for calculating a difference of luminance level of images and producing a difference image based on said calculated difference of luminance images,

wherein said image acquiring means acquires a background image of said predetermined region of said real space in addition to said plurality of static images and measures said luminance level of each of said point of said each image of both said plurality of static images and said background image of a predetermined region of a real space,

wherein said difference calculating means calculates a difference in luminance level at each point between each of said plurality of static images and said background image, and produces a plurality of differential images based on said difference in luminance level at each point between said each of said plurality of static images and said background image, and

third determining means for determining if said candidate moving body is a moving body which moves in said watching region based upon a comparison between said plurality of differential images.

4. The apparatus according to claim 3, further comprising preliminary determining means for calculating a difference with respect to said allowable area between said background image and each of said plurality of static images received from said image acquiring means and making a preliminary decision as to whether or not said difference with respect to said allowable area therebetween exceeds a predetermined value based on said calculated result obtained by said difference calculating means,

wherein said third determining means carries out the determination only when a preliminary decision result in said preliminary determining means shows said difference in luminance level at each point with respect to said allowable area between said background image and each of said plurality of static images exceeds said predetermined value.

5. The apparatus according to claim 3, further comprising:

conversion means for converting each of said differential images into binary data and performing image labeling upon said differential images for obtaining information describing at least one of an area and position of said figure of said candidate moving body,

wherein said first determining means determines whether or not said candidate moving body is a moving body which moves in said watching region based on said obtained information describing at least one of an area and position of said figure of said candidate moving body.

6. The apparatus according to claim 1, further comprising:

motion vector calculating means for calculating a motion vector indicative of a difference in position of said figure of said candidate body based upon a luminance level at each point of said each image of both said plurality of static images and said background image; and

third determining means for determining whether or not said candidate moving body is a moving body which moves in said watching region based upon said motion vector of said figure of said candidate body.

7. The apparatus according to claim 6, further comprising:

image composing means for producing a composite image from all or a part of said plurality of static images and said background image of said predetermined region of said real space such that said each point of said composite image have highest luminance level among luminance levels of a corresponding point of said plurality of static images,

wherein said motion vector calculating means for calculating a plurality of motion vectors, each moving vector being indicative of a difference in position of said figure of said candidate body between a first position of said figure of said candidate body in a first image of said plurality of static images and a second one in a second image of said plurality of static images which acquired after said first image by said image acquiring means, and

fourth determining means for determining whether or not said candidate moving body is a moving body which moves in said watching region based on said motion vector of said figure of said candidate body.

8. The apparatus according to claim 7, wherein moving length of said moving body is obtained on the basis of said motion vector extracted by said motion vector calculating means.

9. The apparatus according to claim 1, further comprising notifying means for notifying the existence of a moving body in said predetermined region of said real space to a user when said first determining means concludes that said moving body exists in said watching region in said predetermined region of said real space.

10. The apparatus according to claim 1, further comprising warning means for warning of a moving body in said predetermined region of said real space when said first determining means concludes that said moving body continuously exists in said predetermined region of said real space for a predetermined period of time.

11. The apparatus according to claim 1, wherein said moving body detecting apparatus is used as an antitheft apparatus.

12. The apparatus according to claim 11, wherein said antitheft apparatus is provided with a vehicle.

13. The apparatus according to claim 1, further comprising:

image composing means for producing a composite image from all or a part of said plurality of static images and said background image of said predetermined region of said real space such that each point of said composite image have a highest luminance level among luminance levels of corresponding point of said plurality of static images, and

third determining means for determining whether or not said candidate moving body is a moving body which moves in said watching region based on said composite image.

14. The apparatus according to claim 13, further comprising:

difference calculating means for calculating a difference of luminance level of images and producing a difference image based on said calculated difference of luminance level of images,

wherein said image acquiring means further acquires a background image of said predetermined region of said real space in addition to said plurality of static images and measures a luminance level of each of said point of said each image of both said plurality of static images and said background image of a predetermined region of said real space,

wherein said difference calculating means calculates a difference in luminance level at each point between each of said plurality of static images and said background image, and produces a plurality of differential images based on said difference in luminance level at each point between said each of said plurality of static images and said background image,

image composing means produces a composite image from all or a part of said plurality of static image and said background image of said predetermined region of said real space such that said each point of said composite image have highest luminance level among luminance levels of a corresponding point of said plurality of static images,

wherein said third determining means determines whether or not said candidate moving body is a moving body which moves in said watching region by using the comparison results compared by the comparison performing means based on said composite image.

15. The apparatus according to claim 13, wherein said third determining means conducts the determining process by which whether or not said candidate moving body is a moving body is judged, repeatedly.

16. The apparatus according to claim 13, further comprising preliminary determining means for calculating a difference with respect to said allowable area between said background image and each of said plurality of static images to make a preliminary determination as to whether or not said difference between said background image and each of said plurality of static images exceeds a predetermined values,

wherein said third determining means carries out the determining only when said preliminary determining means concludes that said difference between said allowable area in said background image and each of said plurality of static images is larger than a predetermined level.

17. The apparatus according to claim 13, wherein, when receiving one of said static images from said image acquiring means, said third determining means makes a decision as to whether to select said one of said static images, and continues the decision processing only when selecting said one of said static images.

18. The apparatus according to claim 13, further comprising:

conversion means for converting each of said difference images into binary data and performing image labeling upon said difference images for obtaining an information about at least one of an area and shape of said figure of said candidate moving body,

wherein said third determining means determines whether or not said candidate moving body is a moving body which moves in said watching region based on said obtained information about at least one of said area and said shape of said figure of said candidate moving body.

19. The apparatus according to claim 13, further comprising notifying means for notifying the existence of a moving body in said predetermined region of said real space to a user when said third determining means determines that said moving body exists in said watching region.

20. The apparatus according to claim 1, further comprising:

motion vector calculating means for calculating difference in position of said figure of said candidate body estimated from one of said static images captured at a first time and another static images captured at a subsequent time of said first time,

wherein said classifying means classifies said watching region into an allowable area and an unallowable area based on said difference vector of positions of said figure of said candidate body obtained by said motion vector calculating means.

21. A method for detecting a moving body, the method comprising:

acquiring a plurality of static images of a predetermined region of a real space sequentially in time;

extracting a figure of a candidate moving body from said plurality of static images of said predetermined region of said real space based on a result of comparing said plurality of static images;

setting a watching region in said predetermined region of said real space, the watching region defining a high probability of existence of said candidate moving body;

classifying said predetermined region of said real space into an allowable area and an unallowable area based on a result of extracting said figure of said candidate moving body from said plurality of static images of said predetermined region of said real space, said allowable area being where existing of said moving body is allowed and said unallowable area being where existing of said moving body is prohibited; and

determining whether or not said moving body moves in said watching region based on whether or not said figure of said candidate moving body moves in said watching region from said allowable area to said unallowable area.

22. The method according to claim 21,

wherein said classifying step for classifying said predetermined region of said real space into said allowable area and said unallowable area is configured to divide only said watching region of said predetermined region of said real space into said allowable area and said unallowable area.

23. The method according to claim 22, further comprising:

acquiring a background image of said predetermined region of said real space which are captured in advance; and

producing a plurality of difference images, each of said difference image is obtained by subtracting said background image of said predetermined region of said real space from each of said plurality of static images of said predetermined region of said real space,

wherein said extracting step in which said figure of said candidate moving body is extracted is configured to extract said figure of said candidate moving body based on a result of performing a comparison among said plurality of static images of said predetermined region of said real space.

24. The method according to claim 23, further comprising:

calculating a motion vector being indicative of a difference in position of said figure of said candidate body within a couple of difference images paired up from a plurality of selected images selected from said plurality of difference images of said watching region of said predetermined region of said real space,

wherein said determining step is configured to determine whether or not said moving body moves in said watching region based on whether or not said motion vector of said figure of said candidate moving body moves in said watching region from said allowable area to said unallowable area.

25. The method according to claim 24, wherein said plurality of selected images are selected from said plurality of static images of said predetermined region of said real space based on a maximum contrast in brightness of an individual image of said plurality of selected images.

26. The method according to claim 23, further comprising a composite image producing step for producing a composite image of said watching region of said predetermined region of said real space which is made by superposing a plurality of selected images from said plurality of difference images of said predetermined region of said real space obtained in said difference images producing steps, wherein said extracting step is configured to extract said figure of a candidate moving body from said composite image.

27. The method according to claim 26, wherein said composite image producing step is configured to select said plurality of selected images from said plurality of static images of said predetermined region of said real space based on a maximum contrast in brightness within an individual image of said plurality of static images.

28. The method according to claim 27, further comprising a step of calculating area of said figure of said candidate moving body, wherein said determining step is configured to determine whether or not said moving body moves in said watching region is carried out only respect to said candidate body having area larger than a predetermined value.

29. The method according to claim 28, further comprising a recognizing step for issuing a warning of the existence of a moving body in said predetermined region of said real space.

30. The method according to claim 24, further comprising calculating an area of said figure of said candidate moving body, wherein said determining step is configured to determine whether or not said moving body moves in said watching region is carried out only when area of said candidate body is larger than a predetermined value.

31. The method according to claim 30, wherein said producing step uses said plurality of difference images consisting of a plurality of binary data in which each binary data has an information of position thereof on each of said plurality of difference images and data pertaining said figure of a candidate moving body.

32. The method according to claim 31, wherein said information includes in binary data pertaining said figure of said candidate moving body is brightness of a corresponding position of said difference image.

33. The method according to claim 32, further comprising a recognizing step for issuing a warning of the existence of said moving body in said predetermined region of said real space when said determining step concludes that said moving body moves in said watching region.

34. The method according to claim 24, further comprising a composite image producing step for producing a composite image of said watching region of said predetermined region of said real space which is made by superposing said couple of difference images paired up from said plurality of difference images of said watching region of predetermined region of said real space, wherein said determining step is configured to determine whether or not said moving body moves in said watching region is determined based on whether or not said motion vector of said figure of said candidate moving body moves in said watching region from said allowable area to said unallowable area.

35. The method according to claim 34, wherein said composite image producing step is configured to select said plurality of selected images from said plurality of static images of said predetermined region of said real space based on a maximum contrast in brightness of an individual image of said plurality of selected images.

36. The method according to claim 21, further comprising a composite image producing step for producing a composite image of said watching region of said predetermined region of said real space which is made by superposing a plurality of selected images from said plurality of static images of said predetermined region of said real space, wherein said extracting step is configured to extract said figure of said candidate moving body from said composite image.

37. An apparatus for detecting a body moving within a predetermined region of a real space, comprising:

an imaging device for acquiring a plurality of static images of said predetermined region of said real space sequentially in time;

a feature calculating unit for extracting a figure of a candidate moving body from said plurality of static images of said predetermined region of said real space based on a result of comparing said plurality of static images of said predetermined region of said real space;

setting device for setting a watching region in said predetermined region of said real space, the watching region defining a high probability of existence of said candidate moving body;

an area classifying unit for classifying said predetermined region of said real space into an allowable area and an unallowable area based on a result of extracting said figure of said candidate moving body from said plurality of static images of said predetermined region of said real space, said allowable area being where existing of said moving body is allowed and said unallowable area being where existing of said moving body is prohibited; and

a motion detecting unit for determining whether or not said moving body moves in said watching region based on whether or not said figure of said candidate moving body moves in said watching region from said allowable area to said unallowable area.

38. The apparatus according to claim 37, wherein said setting device classifies said watching area of said predetermined region of said real space into said allowable area and said unallowable area using information about said figure of said candidate moving body, and divides only said watching region of said predetermined region of said real space into said allowable area and said unallowable area.

39. The apparatus according to claim 38, further comprising:

a difference calculating unit; and

an imaging device for further detecting a background image of said predetermined region of said real space, wherein

said difference calculating unit produces a plurality of difference images of said watching region, each of said difference images of said watching region is obtained by subtracting a corresponding part to said watching region of said background image of said predetermined region of said real space from a further corresponding part to said watching region of each of said plurality of static images, and

said feature calculating unit extracts said figure of a candidate moving body based on a result of performing a comparison among said plurality of difference images of said watching region.

40. The apparatus according to claim 37, further comprising:

a difference calculating unit for differentiating a pair of images,

wherein said difference calculating unit produces a plurality of difference images, each of said difference image is obtained by subtracting said background image of said predetermined region of said real space from each of said plurality of static images of said predetermined region of said real space, and

said feature calculating unit extracts said figure of said candidate moving body based on a result of performing a comparison among said plurality of difference images.

41. The apparatus according to claim 40, wherein said motion detecting unit further calculates a motion vector being indicative of a difference in position of said figure of said candidate body within a couple of difference images paired up from a plurality of selected images selected from said plurality of difference images of said predetermined region of said real space, and determines whether or not said moving body moves in said watching region is determined based on whether or not said motion vector of said figure of said candidate moving body moves in said predetermined region of said real space from said allowable area to said unallowable area.

42. The apparatus according to claim 41, further comprising:

an image composing unit for producing a composite image of said predetermined region of said real space which is made by superposing a plurality of selected images from said plurality of static images of said predetermined region of said real space,

wherein said feature calculating unit extracts said figure of said candidate moving body from said composite image.

43. The apparatus according to claim 42, wherein an image composing unit selects said plurality of selected images from said plurality of static images of said predetermined region of said real space based on a maximum contrast in brightness within an individual image of said plurality of static images.

44. The apparatus according to claim 43, further comprising:

a recognizing unit for issuing a warning of the existence of a moving body in said predetermined region of said real space,

wherein said recognizing unit issues a warning of the existence of said moving body in said predetermined region of said real space when said determining unit concludes that said moving body moves in said predetermined region of said real space.

45. The apparatus according to claim 41, wherein

said feature calculating unit further calculates area of said figure of said candidate moving body,

said motion detecting unit determines whether or not said moving body moves in said predetermined region of said real space is carried out only when area of said candidate body is larger than a predetermined value.

46. The apparatus according to claim 45, further comprising a recognizing unit for issuing a warning of the existence of a moving body in said predetermined region of said real space, wherein said recognizing unit issues said warning based on a result of whether or not said moving body moves in said predetermined region of said real space.

47. The apparatus according to claim 41, further comprising:

an image composing unit for producing a composite image of said predetermined region of said real space which is made by superposing said a couple of difference images paired up from a plurality of selected images selected from said plurality of difference images of said predetermined region of said real space,

wherein said motion detecting unit determines whether or not said moving body moves in said watching region based on whether or not said motion vector of said figure of said candidate moving body moves in said predetermined region from said allowable area to said unallowable area.

48. The apparatus according to claim 47, wherein

said image composing unit selects said plurality of selected images from said plurality of static images of said predetermined region of said real space based on a maximum contrast in brightness within an individual image of said plurality of static images.

49. The apparatus according to claim 40, wherein said motion detecting unit further calculates a motion vector being indicative of a difference in position of said figure of said candidate body within a couple of difference images paired up from a plurality of selected images selected from said plurality of difference images of said watching region of predetermined region of said real space, and determines whether or not said moving body moves in said watching region is determined based on whether or not said motion vector of said figure of said candidate moving body moves in said watching region from said allowable area to said unallowable area.

50. The apparatus according to claim 49, wherein

said feature calculating unit further calculates area of said figure of said candidate moving body,

said motion detecting unit determines whether or not said moving body moves in said watching region is carried out only when area of said candidate body is larger than a predetermined value.

51. The apparatus according to claim 50, further comprising a recognizing unit for issuing a warning of the existence of a moving body in said predetermined region of said real space, wherein said recognizing unit issues said warning based on a result of whether or not said moving body moves in said watching region.

52. The apparatus according to claim 49, further comprising an image composing unit for producing a composite image of said watching region of said predetermined region of said real space which is made by superposing said couple of difference images paired up from a plurality of selected images selected from said plurality of difference images of said watching region, wherein said motion detecting unit determines whether or not said moving body moves in said watching region based on whether or not said motion vector of said figure of said candidate moving body moves in said watching region from said allowable area to said unallowable area.

53. The apparatus according to claim 52, wherein

said image composing unit selects said plurality of selected images from said plurality of static images of said predetermined region of said real space based on a maximum contrast in brightness within said watching region of an individual image of said plurality of static images.

54. The apparatus according to claim 49, further comprising:

an image composing unit for producing a composite image of said watching region of said predetermined region of said real space which is made by superposing a plurality of selected images from said plurality of static images of said predetermined region of said real space with only respect to said watching region,

wherein said feature calculating unit extracts said figure of said candidate moving body from said composite image.

55. The apparatus according to claim 54, wherein an image composing unit selects said plurality of selected images from said plurality of static images of said predetermined region of said real space based on a maximum contrast in brightness within said watching region of an individual image of said plurality of static images.

56. The apparatus according to claim 55, further comprising:

a recognizing unit for issuing a warning of the existence of a moving body in said watching region,

wherein said recognizing unit issues a warning of the existence of said moving body in said predetermined region of said real space when said determining unit concludes that said moving body moves in said watching region.

57. A moving body detecting method comprising steps of:

acquiring a plurality of static images of a predetermined region in a real space sequentially in time;

comparing said plurality of static images of said predetermined region of said real space;

extracting a figure of a candidate moving body from said plurality of static images based on a result of the comparing step;

setting a watching region in said predetermined region in said real space, the watching region being defined as an area to which a smooth traveling path of said candidate moving body is extended from a position where said figure of said candidate moving body is extracted;

classifying said predetermined region of a real space into an allowable area and an unallowable area based on a result of the extracting step, said allowable area allowing said moving body to exist therein and said unallowable area not allowing said moving body to exist therein;

judging whether or not said figure of said candidate moving body moves in said watching region from said allowable area to said unallowable area; and

determining whether or not said moving body moves in said watching region based on a result of the judgment whether or not said figure of said candidate moving body moves in said watching region from said allowable area to said unallowable area.

58. The moving body detecting method according to claim 57, further comprising steps of:

acquiring a background image of said predetermined region of a real space which are captured in advance; and

producing a plurality of difference images, each of said difference image is obtained by subtracting said background image of said predetermined region of a real space from each of said plurality of static images of a predetermined region of a real space, wherein said extracting step in which said figure of said candidate moving body is extracted is configured to extract said figure of said candidate moving body based on a result of performing a comparison among said plurality of static images of said predetermined region of a real space.

59. The moving body detecting method according to claim 58, further comprising a step of:

calculating a motion vector being indicative of a difference in position of said figure of said candidate body within a couple of difference images paired up from a plurality of selected images selected from said plurality of difference images of said watching region of predetermined region of a real space, wherein determining step is configured to determine whether or not said moving body moves in said watching region based on whether or not said motion vector of said figure of said candidate moving body moves in said watching region from said allowable area to said unallowable area.

60. The moving body detecting method according to claim 59, further comprising a step of:

producing step for producing a composite image of said watching region of said predetermined region of a real space which is made by superposing a plurality of selected images from said plurality of static images of said predetermined region of a real space, wherein said extracting step is configured to extract said figure of said candidate moving body from said composite image.

61. An apparatus that detects a body moving within a predetermined region of a real space, comprising:

an imaging device that acquires a plurality of static images of a predetermined region in a real space sequentially in time;

a comparing device that compares said plurality of static images acquired by said imaging device;

a feature calculating device that extracts a figure of a candidate moving body from said plurality of static images based on a comparison result obtained by the comparing device;

a setting device that sets a watching region in said predetermined region in said real space, the watching region being, defined as an area to which a smooth traveling path of said candidate moving body is extended from a position where said figure of said candidate moving body is extracted by said feature calculating unit;

a classifying device that classifies said predetermined region of a real space into an allowable area and an unallowable area based on a result of the extracting step, said allowable area allowing said moving body to exist therein and said unallowable area not allowing said moving body to exist therein;

a judging device that judges whether or not said figure of said candidate moving body moves in said watching region from said allowable area to said unallowable area; and

a motion detecting device that determines whether or not said moving body moves in said watching region based on a judgment preformed by the judging unit.

62. The apparatus according to claim 61, wherein said imaging device further acquires a background image of said predetermined region of said real space which are captured in advance, further comprising:

a producing device that produces a plurality of difference images, each of said difference image is obtained by subtracting said background image from each of said plurality of static images acquired by said imaging device, wherein said feature calculating device extracts said figure of said candidate moving body based on the comparison result performed by the comparing device.

63. The apparatus according to claim 62, further comprising:

a calculating device that calculates a motion vector being indicative of a difference in position of said figure of said candidate body within said couple of difference images paired up from said plurality of selected images selected from said plurality of difference images acquired by said imaging device, and a motion detecting device that determines whether or not said motion vector of said figure of said candidate moving body moves in said watching region from said allowable area to said unallowable area, wherein said motion detecting device determines whether or not said moving body moves in said watching region based on the result obtained by the motion detecting device.

64. The apparatus according to claim 63, further comprising;

a producing device that produces a composite image of said watching region of said predetermined region of said real space which is made by superposing a plurality of selected images from said plurality of static images of said predetermined region of a real space, wherein said feature calculating device extracts said figure of said candidate moving body from said composite image.