Method of distinguishing a moving object and apparatus of tracking and monitoring a moving object

• US 6,445,409 B1
• Filed: 07/28/1999
• Issued: 09/03/2002
• Est. Priority Date: 05/14/1997
• Status: Expired due to Term

• Current Assignee: Hitachi Denshi Kabushiki Kaisha
• Original Assignee: Hitachi Denshi Kabushiki Kaisha
• Inventors: Ito, Wataru, Ueda, Hirotada, Yamada, Hiromasa
• Primary Examiner(s): BRITTON, HOWARD W
• Application Number:
  US09/362,212
• Time in Patent Office: 1,133 Days
• Field of Search: H04/N.718, 348/152, 348/154, 348/155, 348/169, 348/172
• US Class Current: 348/155
• CPC Class Codes: G01S 3/783 using amplitude comparison of signals derived from static detectors or detector systems
• CPC Class Codes: G01S 3/7865 using correlation of the live video image with a stored image
• CPC Class Codes: G08B 13/19602 Image analysis to detect motion of the intruder, e.g. by frame subtraction
• CPC Class Codes: G08B 13/19606 Discriminating between target movement or movement in an area of interest and other non-signicative movements, e.g. target movements induced by camera shake or movements of pets, falling leaves, rotating fan
• CPC Class Codes: G08B 13/19608 Tracking movement of a target, e.g. by detecting an object predefined as a target, using target direction and or velocity to predict its new position
• CPC Class Codes: G08B 13/1961 Movement detection not involving frame subtraction, e.g. motion detection on the basis of luminance changes in the image
• CPC Class Codes: H04N 7/181 for receiving images from a plurality of remote sources

Abstract

A moving object recognition method for recognizing a specified moving object in an input image signal in distinction from other objects, and a moving object tracking and monitoring apparatus using the method. One or more moving objects in the input image signal are sequentially detected by the subtraction method, the physical quantities of the detected moving objects are sequentially stored in a memory. Based on this physical quantities, one or more loci of the detected moving objects are determined. A group of physical quantities associated with each locus are stored in a memory as correlated information series of a moving object. Based on the information series thus stored, at least one of a plurality of classification methods is applied to each moving object for determining whether the moving object belongs to the specified moving object or other than the specified moving object. Thus, a detected moving object is recognized as to whether it is a specified moving object or not. A plurality of the classification methods include a method based on the continuity of the locus of a detected moving object, a method based on the smoothness of position change of the detected moving object and a method based on the size of the detected moving object.

Claims

1. A moving object recognition method for recognizing a specified moving object in sequentially input image signal in distinction from other objects, comprising the steps of:

sequentially detecting at least one moving object in the sequentially input image signals by a subtraction method;

sequentially storing, for each input image signal, in a memory at least one characteristic physical quantity which said moving object has;

detecting at least one locus of said moving object based on characteristic physical quantities in said input image signals;

storing in a memory characteristic physical quantities of said detected locus in such a manner that said characteristic physical quantities are associated with each other as a series of characteristic physical quantities of said moving object;

previously setting a test or criterion of determining whether said moving object belongs to said specified moving object or belongs to objects other than said specified moving object; and

determining from said series of characteristic physical quantities, whether or not said moving object is said specified moving object, based on said determining criterion.

2. A moving object recognition method according to claim 1, wherein said step of sequentially storing at least one physical quantity includes the step of storing a plurality of characteristic physical quantities of moving objects in said memory,

said step of detecting at least one locus includes the step of detecting a plurality of loci of said moving objects based on said plurality of characteristic physical quantities,

said step of storing characteristic physical quantities includes the step of storing a plurality of physical quantities of said detected loci in said memory in such a manner that said plurality of physical quantities of said detected loci are associated with each other as series of a plurality of characteristic physical quantities of said moving objects,

said step of previously setting a determining test includes the step of setting a determining test for each of said series of a plurality of characteristic physical quantities, and

said step of determining includes the step of determining whether or not said moving objects are said specified moving object from at least two of said series of characteristic physical quantities, based on respective associated ones of said determining tests.

3. A moving object recognition method according to claim 2, wherein a plurality of said determining tests include the steps of:

calculating a time during which each of the loci of said moving object exists continuously from a series of detection time and detection positions of said detected moving object included in said series of characteristic physical quantities stored in memory; and

determining said moving object having said calculated time longer than a predetermined time as said specified moving object and a moving object having said calculated time not longer than said predetermined time as an object other than said specified moving object.

4. A moving object recognition method according to claim 2, wherein a plurality of said determining tests include the steps of:

calculating the variations in the position change of each of said moving objects from a series of detection time and detection positions of said detected moving object included in said series of characteristic physical quantities stored in said memory; and

determining a moving object with the calculated position change variation smaller than a predetermined value as said specified moving object and a moving object with the calculated position change variation not smaller than said predetermined value as an object other than said specified moving object.

5. A moving object recognition method according to claim 2, wherein a plurality of said determining tests include the steps of:

calculating a maximum size of said moving object from an area of said detected moving object included in said series of characteristic physical quantities stored in memory; and

determining said moving object with the calculated maximum size larger than a predetermined value as said specified moving object and said moving object with the calculated maximum size not larger than said predetermined value as an object other than said specified moving object.

6. A moving object recognition method according to claim 2, wherein a plurality of said determining tests include the steps of:

calculating a minimum area of said moving object from an area of said detected moving object included in said series of characteristic physical quantities stored in memory; and

classifying a moving object with the calculated minimum area smaller than a predetermined value as said specified moving object and said moving object with the calculated maximum area not smaller than said predetermined value as an object other than said specified moving object.

7. A moving object recognition method according to claim 2, wherein a plurality of said determining tests include the steps of:

calculating a maximum area of each of said moving objects from an area of said detected moving objects included in said series of characteristic physical quantities stored in memory; and

calculating a minimum area of each of said moving objects from the area of said detected moving object; and

determining said moving object with said maximum area larger than a first predetermined value and said minimum area smaller than a second predetermined value as said specified moving object, and classifying each of the remaining moving objects as objects other than said specified moving object.

8. A moving object recognition method according to claim 7, wherein said first predetermined value is an estimated minimum area of said specified moving object and said second predetermined value is an estimated maximum area of said specified moving object.

16. A moving object recognition method according to claim 3, wherein the locus of said detected moving object is detected using more than two input image signal frames, and said locus continuous existence time is calculated using the number of the continuous image signal frames upon detection of said locus.

29. A moving object recognition method according to claim 16, wherein the locus of said detected moving object is determined using more than two input image signal frames, said step of determining said moving object to be said specified moving object or to be an object other than said specified moving object includes the substep of determining said moving object according to whether a time during which the locus of said moving object exists continuously is longer than a predetermined time, and said locus continuous existence time is calculated using the number of continuous image signal frames upon detection of said locus.

30. A moving object recognition method according to claim 5, wherein the maximum size of said moving object is the areal size of the circumscribed polygon about the area of said detected moving object.

31. A moving object recognition method according to claim 5, wherein the maximum size of said moving object is the areal size of the area of said detected moving object.

55. A moving object recognition method according to claim 1, wherein said determining criterion or test includes the steps of:

classifying each of said detected moving objects into one of a plurality of types based on at least one of the shape and the size of the area of said detected moving object;

selecting at least one of objects to be tracked from said classified detected objects; and

tracking said selected object as an object to be tracked.

56. A moving object recognition method according to claim 55, wherein said step of classifying based on the shape includes the substep of classifying said detected moving objects according to the ratio between the horizontal length and the vertical length of the circumscribed rectangle about each detected area.

57. A moving object recognition method according to claim 55, wherein said step of classifying based on the shape includes the substep of classifying said detected object by the vertical length of each detected area.

58. A moving object recognition method according to claim 57, further comprising the step of calculating the histogram of the number of pixels in vertical direction of said detected area, said histogram having a maximum value equal to the vertical length of said detected area.

59. A moving object recognition method according to claim 58, further comprising the step of classifying the type of said detected area by the number of horizontally continuous pixels with the histogram having a value not smaller than a predetermined first threshold.

60. A moving object recognition method according to claim 55, wherein said step of classifying said detected moving object based on the areal size of said detected area includes the substep of classifying said detected moving object by the ratio between the area of the circumscribed rectangle about said detected area and the areal size of said detected area.

61. A moving object recognition method according to claim 55, wherein one of a plurality of said types is a specified type, said moving object tracking method further comprising the step of removing the detected object classified as said specified type from the objects to be tracked.

62. A moving object recognition method according to claim 61, wherein said specified type is a wave.

9. A moving object recognition method for recognizing a specified moving object in serially input image signals in distinction from other objects, comprising the steps of:

sequentially detecting a plurality of moving objects in serially input image signal;

sequentially storing characteristic physical quantities of said detected moving objects;

detecting at least a locus of at least one of said detected moving objects based on said characteristic physical quantities;

storing a group of characteristic physical quantities included in each of said loci in association with each other as series of characteristic physical quantities of moving objects, said characteristic physical quantities including a series of detection time and detection position and area of each of said moving objects; and

determining whether each moving object is said specified moving object or not based on said characteristic physical quantities;

said step of determining including at least one of the substeps of;

calculating a time during which the locus of each moving object exists continuously from said detection time and said detection position included in said series of characteristic physical quantities of said moving object, and determining whether said moving object is said specified moving object or not from said continuous locus existence time;

calculating a variation in position change of said moving object from a detection position included series of characteristic physical quantities of each moving object, and determining whether said moving object is said specified moving object or not from said calculated variation;

calculating a maximum area of each moving object from said area included in said series of characteristic physical quantities of said moving object, and determining whether said moving object is said specified moving object or not from said calculated maximum area; and

calculating a minimum area of each moving object from said area included in said series of characteristic physical quantities of said moving object, and determining whether said moving object is said specified moving object or not from said calculated minimum area.

10. A moving object recognition method according to claim 9, further comprising the step of generating noticeable information that a moving object determined to be said specified moving object is an intruding object.

11. A moving object recognition method according to claim 9, wherein the locus of said detected moving object is detected using more than two input image signal frames, and said locus continuous existence time is calculated using a continuous image signal frame upon detection of said locus.

12. A moving object recognition method according to claim 9, wherein said area included in the series of characteristic physical quantities of each moving object is calculated from the circumscribed polygon about the area occupied by said moving object.

13. A moving object recognition method according to claim 9, wherein the size of said area included in the series of characteristic physical quantities of each moving object is calculated from the size of the area occupied by said moving object.

14. A moving object tracking and monitoring method for sequentially detecting moving objects intruding into an imaging view field by a subtraction method and sequentially tracking and monitoring said moving objects with an object tracking and monitoring apparatus, comprising the steps of:

sequentially storing a characteristic physical quantity of a moving object detected by said object tracking and monitoring apparatus;

detecting a locus of said detected moving object based on said characteristic physical quantities;

storing characteristic physical quantities included in each locus as a series of characteristic of a moving object;

previously setting a test for determining whether said moving object belongs to an object to be tracked and monitored;

classifying said moving object into objects to be tracked and monitored and objects not to be tracked and monitored from said stored series of characteristic physical quantities, based on said determining test; and

tracking and monitoring a moving object classified as objects to be tracked and monitored as an intruding object.

15. A moving object tracking and monitoring method according to claim 14, wherein said step of sequentially storing includes the step of sequentially storing a plurality of characteristic physical quantities of a plurality of moving objects detected by said object tracking and monitoring apparatus;

said step of detecting includes the step of detecting a plurality of loci of said moving objects based on said plurality of characteristic physical quantities;

said step of storing characteristic physical quantities includes the step of storing a plurality of characteristic physical quantities included in each of said detected loci as a series of a plurality of characteristic physical quantities in association with each other,

said step of previously setting a test includes setting a determining test for each of said series of characteristic physical quantities,

said step of classifying includes the step of classifying said moving objects into objects to be tracked and monitored and objects not to be monitored and tracked from at least two series of characteristic physical quantities, based on respective ones of associated determining tests.

17. A moving object tracking and monitoring apparatus comprising:

an image input device;

an image memory for storing an image sequentially picked up by said image input device;

a program memory for storing a program for operating said object tracking and monitoring apparatus;

a central processing unit for activating said object tracking and monitoring apparatus in accordance with said program;

a work memory for analyzing said image stored in said image memory;

an alarm unit activated by said central processing unit in dependence on the result of analysis stored in said work memory; and

a monitor unit for displaying a monitor image from said central processing unit generated in accordance with the analysis in said work memory;

said program including;

code means for sequentially detecting the moving objects stored in said image memory;

code means for sequentially storing characteristic physical quantities of said detected moving objects in said work memory;

code means for determining one or more loci of a moving object based on said characteristic physical quantities of said detected moving object;

code means for storing a group of characteristic physical quantities belonging to each of said loci in association with each other as a series of characteristic physical quantities of one moving object in said work memory;

code means for classifying said one moving object into an object to be tracked and monitored and an object not to be tracked and monitored based on said stored series of characteristic physical quantities; and

code means for activating said object tracking and monitoring apparatus in such a manner that said one moving object classified as an object to be tracked and monitored is tracked as an intruding object.

18. A moving object tracking and monitoring apparatus according to claim 17, wherein said classifying code means includes:

code means for determining the continuity of said one moving object;

code means for determining the variation of the position change of said one moving object; and

code means for determining the size of said one moving object.

19. A computer readable medium having computer readable program code means embodied therein for recognizing a specified moving object in an input image signal in distinction from other objects, comprising:

code means for sequentially detecting at least one moving object in sequentially input image signals;

code means for sequentially storing characteristic physical quantities of said detected moving object;

code means for determining at least one locus of said detected moving object based on said characteristic physical quantities;

code means for storing a group of characteristic physical quantities of each locus in association with each other in a memory as a series of characteristic physical quantities the moving object; and

code means for sequentially applying a plurality of determining tests to each moving object for determining whether said moving object belongs to said specified moving object or to objects other than said specified object and thus determining whether said detected moving object is said specified moving object or not based on said stored series of characteristic physical quantities.

20. A computer readable medium according to claim 19, wherein said code means for applying a plurality of said determining tests includes:

code means for calculating a time during which each of the loci of said moving object continuously exists from said stored series of characteristic physical quantities; and

code means for determining said moving object with said calculated time longer than a predetermined time to be said specified moving object and said moving object with said calculated time not longer than said predetermined time to be an object other than said specified moving object.

21. A computer readable medium according to claim 19, wherein said code means for applying a plurality of said classifying methods includes:

code means for calculating the variation of position change of each of said moving objects from said stored correlated information series; and

code means for classifying a moving object with said calculated position change variation smaller than a predetermined value as an object to be recognized and a moving object with said calculated position change variation not smaller than said predetermined value as an object not to be recognized.

22. A computer readable medium according to claim 19, wherein said code means for applying a plurality of said determining tests includes:

code means for calculating a maximum size of said moving object from said stored series of characteristic physical quantities;

code means for determining a moving object with said calculated maximum size larger than a predetermined value to be said specified moving object and a moving object with said calculated maximum size not larger than said predetermined value to be an object other than said specified moving object.

23. A computer readable medium according to claim 19, wherein said code means for applying a plurality of said determining tests includes:

code means for calculating a minimum area of each of said moving objects from said stored series of characteristic physical quantities;

code means for determining a moving object with said calculated minimum area smaller than a predetermined value to be said specified moving object and a moving object with said calculated minimum area not smaller than said predetermined value to be an object other than said specified moving object.

24. A computer readable medium according to claim 19, wherein said code means for applying a plurality of said determining tests includes:

code means for calculating a maximum area of each of said moving objects from said stored series of characteristic physical quantities;

code means for calculating a minimum area of each of said moving objects from said stored series of characteristic physical quantities; and

code means for determining a moving object with said maximum area larger than a first predetermined value and said minimum area smaller than a second predetermined value to be said specified moving object and other moving objects to be moving objects other than said specified moving objects.

25. A computer readable medium according to claim 24, wherein

said first predetermined first value is an estimated minimum area of said specified moving object and said second predetermined value is an estimated maximum area of said specified moving object.

26. A computer readable medium having computer readable program code means embodied therein for recognizing a specified moving object in an input image signal in distinction from other objects, comprising:

code means for sequentially detecting a plurality of moving objects in sequentially input image signals;

code means for sequentially storing characteristic physical quantities of said detected moving objects;

code means for determining at least one locus of said detected moving objects based on said characteristic physical quantities;

code means for storing a group of characteristic physical quantities included in each of said loci in association with each other as a series of characteristic physical quantities of one moving object, said physical quantities including detection time, position and area at each moment of each of said moving objects;

code means for determining each of said moving objects to be said specified moving object or an object other than said specified moving object, based on said characteristic physical quantities;

said determining code means including;

code means for calculating a time during which the locus of each of said moving objects continuously exists based on the detection time and the detection position included in said series of characteristic physical quantities of said moving object;

code means for determining said moving object to be said specified moving object or an object other than said specified moving object, based on said continuous locus existence time;

code means for calculating the variation of the position change of each of said moving objects based on the detection position included in said series of characteristic physical quantities of each moving object;

code means for determining said moving object to be said specified moving object or an object other than said specified moving object, based on said calculated variation;

code means for calculating a maximum area of each of said moving objects based on the area included in the series of characteristic physical quantities of said moving object;

code means for determining said moving object to be said specified moving object or an object other than said specified moving object, based on said calculated maximum area;

code means for calculating a minimum area of each of said moving objects based on the area included in the series of characteristic physical quantities of said moving object; and

code means for determining said moving object to be said specified moving object or an object other than said specified moving object, based on said calculated minimum area.

27. A computer readable medium according to claim 26, further comprising code means for tracking as an intruding object a moving object determined to be said specified moving object.

28. A computer readable medium having computer readable program code means embodied therein for sequentially tracking and monitoring a moving object intruding into an imaging view field, comprising:

code means for sequentially storing characteristic physical quantities of at least one moving object detected by said object tracking and monitoring apparatus;

code means for determining the locus of said detected moving object based on said characteristic physical quantities;

code means for storing a group of characteristic physical quantities included in each of said loci in association with each other as a series of characteristic physical quantities of one moving object;

code means for determining said moving object to be said specified moving object and an object other than said specified moving object, based on said stored series of characteristic physical quantities; and

code means for tracking as an intruding object said moving object classified as an object determined to be said specified moving object.

32. A moving object tracking method comprising the steps of:

detecting as an object each area where the difference between an input image and a reference background image is not less than a predetermined threshold;

classifying each of said detected objects into one of a plurality of types based on at least one of the shape and the size of the area of said detected object;

selecting one or more objects to be tracked from said classified detected objects, and tracking said selected object as an object to be tracked.

33. A moving object tracking method according to claim 32, wherein said step of classifying an object based on the shape of said detected area includes the substep of classifying said detected object by the ratio between the horizontal length and the vertical length of the circumscribed rectangle about said detected area.

34. A moving object tracking method according to claim 32, wherein said step of classifying an object based on the shape of said detected area includes the substep of classifying said detected object by the vertical length of said detected area.

35. A moving object tracking method according to claim 34, further comprising the substep for calculating the histogram of the number of pixels in vertical direction of said detected area, and the maximum value of said histogram is regarded as the vertical length of said detected area.

36. A moving object tracking method according to claim 35, further comprising the step of subdividing the type of said detected area by the number of continuous horizontal pixels of a histogram value not less than a predetermined first threshold.

37. A moving object tracking method according to claim 32, wherein said step of classifying an object based on the areal size of said detected area includes the substep of classifying said detected object according to the ratio between the areal size of the circumscribed rectangle about each detected area and the areal size of said detected area.

38. A moving object tracking method according to claim 32, wherein one of said types is a specified type, said method further comprising the step of removing the detected object classified as said specified type from said objects to be tracked.

39. A moving object tracking method according to claim 38, wherein said specified type is a wave.

40. A moving object tracking method according to claim 38, further comprising the steps of:

storing the information of said detected object classified into a plurality of detected objects over a plurality of frames for each detected object; and

reclassifying the type of each of said detected objects based on said information of said frames.

41. An object tracking apparatus comprising:

an imaging device;

an interface unit for converting the input signal from said imaging device into an image signal;

a detector for detecting as an object each area in which the intensity difference for each pixel between said image signal and the reference background image is larger than a predetermined value;

a central processing unit;

a memory; and

means for classifying said detected area into one of a plurality of types based on at least selected one of the shape and the areal size detected by said detector in order to select an object to be tracked from said objects;

wherein said object selected as an object to be tracked is tracked.

42. An object tracking apparatus according to claim 41, wherein the detected area classified as a specified one of a plurality of said types is removed from the category of objects to be tracked.

43. An object tracking apparatus according to claim 41, wherein said specified one type is a wave.

44. An object tracking and monitoring apparatus comprising:

an image input device;

an image memory for storing an image picked up by said image input device;

a program memory for storing the program for the operation of said object tracking and monitoring apparatus;

a central processing unit for activating said object tracking and monitoring apparatus in accordance with said program;

an image memory for analyzing said image stored in said image memory;

an alarm unit activated by said central processing unit in dependence on the result of analysis by said work memory; and

a monitor unit for displaying a monitor image from said central processing unit generated in accordance with the analysis in said work memory; and

said program includes;

code means for calculating the difference of the intensity value for each pixel between the image from said image input device and the reference background image;

code means for detecting a pixel area where an object exists as a plurality of pixels with said intensity difference not less than a predetermined threshold value regarded as pixels where an object exists;

first classification code means for classifying said detected object based on the ratio which the areal size of the area of said detected object represents of the areal size of said circumscribed rectangle area on the assumption that the shape of the circumscribed rectangle about said pixel area where said detected object exists is horizontally long;

second classification code means for classifying said object based on the vertical size of said pixel area where said detected object exists;

code means for calculating the histogram of the number of pixels in vertical direction of said pixel area where said detected object exists;

code means for determining the number of continuous pixels in horizontal direction having a histogram value not less than a predetermined threshold; and

third classification code means for classifying said detected object based on the number of continuous pixels in horizontal direction;

wherein detected objects other the objects to be tracked are removed by said first, second and third classification code means.

45. An object tracking and monitoring apparatus according to claim 44, wherein said first, second and third classification code means classify said detected object as a wave or other objects.

46. A computer readable medium having computer readable program code means embodied therein for tracking an moving object, comprising:

code means for detecting as an object each area having the difference between an input image and a reference background image not less than a predetermined threshold;

code means for classifying each of said detected objects into one of a plurality of types based on at least selected one of the shape and the size of the area of said detected object;

code means for selecting one or more objects to be tracked from said classified detected objects; and

code means for tracking said selected object as an object to be tracked.

47. A computer readable medium according to claim 46, wherein said classification based on the shape of said detected area includes the classification of said detected object based on the ratio between the horizontal length and the vertical length of the circumscribed rectangle about said detected area.

48. A computer readable medium according to claim 46, wherein said classification of said detected area based on the shape is the classification of said detected object based on the vertical length of said detected area.

49. A computer readable medium according to claim 48, further comprising code means for calculating the histogram of the number of pixels in vertical direction of said detected area, and the maximum value of said histogram is the vertical length of said detected area.

50. A computer readable medium according to claim 49, further comprising code means for classifying the type of said detected area according to the number of horizontally continuous pixels with the histogram value thereof not smaller than said first threshold.

51. A computer readable medium according to claim 46, further comprising code means in which said classification based on the size of said detected area is the classification of said detected object based on the ratio of the size between the circumscribed area about each detected area and said detected area.

52. A computer readable medium according to claim 46, wherein one of said types is a specified type, and said moving object tracking program further includes code means for removing the detected object classified as said specified type from said category of objects to be tracked.

53. A computer readable medium according to claim 52, wherein said specified type is a wave.

54. A computer readable medium according to claim 52, further comprising;

code means for storing the information of said detected object classified into a plurality of said detected objects over a plurality of frames for each detected object; and

code means for reclassifying the type of said detected object based on the information of said plurality of said frames.