Method for tracking entering object and apparatus for tracking and monitoring entering object

• US 6,404,455 B1
• Filed: 05/14/1998
• Issued: 06/11/2002
• Est. Priority Date: 05/14/1997
• Status: Expired due to Term

• Current Assignee: Hitachi Denshi Kabushiki Kaisha
• Original Assignee: Hitachi Denshi Kabushiki Kaisha
• Inventors: Ito, Wataru, Endo, Miyuki, Ueda, Hirotada, Okada, Toshimichi
• Primary Examiner(s): Kelley, Chris
• Assistant Examiner(s): An, Shawn S.
• Application Number:
  US09/078,521
• Time in Patent Office: 1,489 Days
• Field of Search: 348/169, 348/214, 348/152, 348/143, 348/155, 348/15, 348/36, 348/44, 348/172, 348/135, 382/219, 382/218, 382/103
• US Class Current: 348/169
• CPC Class Codes: G08B 13/19602 Image analysis to detect motion of the intruder, e.g. by frame subtraction
• CPC Class Codes: G08B 13/19604 involving reference image or background adaptation with time to compensate for changing conditions, e.g. reference image update on detection of light level change
• CPC Class Codes: G08B 13/19608 Tracking movement of a target, e.g. by detecting an object predefined as a target, using target direction and or velocity to predict its new position
• CPC Class Codes: G08B 13/19628 of wide angled cameras and camera groups, e.g. omni-directional cameras, fish eye, single units having multiple cameras achieving a wide angle view
• CPC Class Codes: G08B 13/19643 wherein the cameras play different roles, e.g. different resolution, different camera type, master-slave camera
• CPC Class Codes: G08B 13/19689 Remote control of cameras, e.g. remote orientation or image zooming control for a PTZ camera
• CPC Class Codes: H04N 7/181 for receiving images from a plurality of remote sources

Abstract

A system for processing a tracking picture of an intruding object includes a first TV camera for outputting a video signal used for detecting the intruding object, a second TV camera for outputting a video signal used for monitoring a magnified tracking picture of the intruding object, and a picture processing device for processing and operating on the video signal inputted from the first TV camera and outputting a rotation control signal and a zooming control signal. The system detects a difference between the video signal inputted from the first TV camera and a reference background video signal, and a change of a detecting state from the change of the numbers of objects detected in the continuous input video signal.

Claims

1. A method of tracking and detecting objects entering into an imaging field of view of a camera in an object tracking and monitoring apparatus provided with said camera, comprising the steps of:

sequentially outputting picture signals from said camera;

detecting objects in said picture signals;

detecting changes in the detected objects from said picture signals; and

classifying the detected changes in the detected objects into at least one of a plurality of predetermined state changes to be classified include appearance, single connection, coupling, disappearance and separation.

2. The method as claimed in claim 1, further comprising the step of:

detecting said objects to be detected based on classification of said detected changes.

3. The method as claimed in claim 1, further comprising the steps of:

creating connecting information for representing a location and a change of an area of a region of said objects detected at each time from said detected changes; and

inferring a trace of said objects by a moving distance of said detected objects from said created connecting information.

4. The method as claimed in claim 3, further comprising the steps of:

determining if one object is split into plural objects when said objects are detected, from said connecting information; and

correcting the plural objects determined as a split state into one object.

5. The system as claimed in claim 4, further comprising the step of:

recursively determining if the split state is corrected at said step of correcting the split state,

wherein said step of determining the split state and said step of correcting the split state are repeated until no more corrections for the split state are possible.

6. The method as claimed in claim 3, further comprising the steps of:

determining if one object is detected as split objects when said objects are detected, from said connecting information;

correcting plural objects determined as the split state into one object;

determining if plural objects are detected as a connected one object when said object is detected, from said connecting information;

correcting one object determined as the connected state into plural objects from the connecting information before and after said detection; and

wherein plural objects determined as the split state are corrected into one object and one object determined as the connected state is corrected into plural objects based on the connecting information before and after said detection.

7. The method as claimed in claim 6, wherein the operation is executed to hold a location of said object, area of a region occupied by said object, and a pixel pattern of said region, detected as the connecting information at said step of creating the connecting information, and the operation is executed to determine the connected state by matching the pixel pattern at said step of determining the connected state.

8. The method as claimed in claim 3, further comprising the steps of:

determining if one object is detected as split plural objects when said plural objects are detected, from said connecting information;

correcting said plural objects determined as the split state into one object;

recursively determining if the correction for the split state is executed at said step of correcting the split state;

determining if plural objects are detected as a connected one object when said one object is detected, from said connecting information;

correcting said one object determined as the connected state into plural objects, based on the connecting information before and after said detection;

recursively determining if the correction for the connected state is executed at said step of correcting the connected state; and

wherein said step of determining the split state and said step of correcting the split state are repeated until no more correction for the split state is possible and said step of determining the connected state and said step of correcting the connected state are repeated until no more correction for the connected state is possible.

9. The method as claimed in claim 8, wherein the operation is executed to hold a location of object, area of a region occupied by said object, and a pixel pattern of said region, detected as the connecting information at said step of creating the connecting information and the operation is executed to determine the connected state by matching the pixel pattern at said step of determining the connected state.

10. The method as claimed in claim 1, further comprising the steps of:

creating the connecting information for representing change of a location and area of a region occupied by said objects detected at each time from said detecting state change; and

inferring a trace of said objects by a moving distance of said detected objects from said created connected information.

11. The method as claimed in claim 10, further comprising the steps of:

determining if one object is detected as a split state when plural objects are detected, from said connecting information; and

correcting said plural objects determined as the split state into one object.

12. The method as claimed in claim 11, further comprising the steps of recursively determining if the correction for the split state is executed at said step of correcting the split state, and wherein said step of determining the split state and said step of correcting the split state are repeated until no more correction for the split state is possible.

13. The method as claimed in claim 10, further comprising the steps of:

determining if plural objects are detected as a connected one object when said one object is detected, from said connecting information; and

correcting said one object determined as the connected one object into plural objects, based on said connecting information before and after said detection.

14. The method as claimed in claim 13, further comprising the step of recursively determining if the correction for the connected state is executed at said step of correcting the connected state, and wherein said step of determining the connected state and said step of correcting the connected state are repeated until no more correction for the connected state is possible.

15. The method as claimed in claim 10, further comprising the steps of:

determining if one object is detected as split plural objects when said, plural objects are detected, from said connecting information;

correcting said plural objects determined as the split state into one object;

determining if plural objects are detected as a connected one object when said one object is detected, from said connecting information;

correcting said one object determined as the connected state into plural objects, based on the connecting information before and after said detection; and

wherein plural objects determined as the split state is corrected into one object and one object determined as the connected state is corrected into plural objects, based on the connecting information before and after said detection.

16. The method as claimed in claim 15, further comprising the steps of holding a location of said object, area of a region occupied by said object and a pixel pattern of said region, detected as the connecting information at said step of creating the connecting information, and determining the connected state by matching the pixel pattern at said step of determining the connected state.

17. The method as claimed in claim 10, further comprising the steps of:

determining if one object is detected as split plural objects when said plural objects are detected, from said connecting information;

correcting said plural objects determined as the split state into one object;

recursively determining if the correction for the split state is executed at said step of correcting the split state;

determining if plural objects are detected as a connected one object when said one object is detected, from said connecting information;

correcting said one object determined as the connected state into plural objects, based on the connecting information before and after said detection;

recursively determining if the correction for the connected state is executed at said step of correcting the connected state; and

wherein said step of determining the split state and said step of correcting the split state are repeated until no more correction for the split state is possible and said step of determining the connected state and said step of correcting the connected step are repeated until no more correction for the connected state is possible.

18. The method as claimed in claim 17, further comprising the steps of holding a location of said object, area of a region occupied by said object and a pixel pattern of said region, detected as the connecting information at said step of creating the connecting information, and determining the connected state by matching the pixel pattern at said step of determining the connected state.

19. The method as claimed in claim 17, further comprising the steps of holding a location of said object, area of a region occupied by said object and a pixel pattern of said region, detected as the connecting information at said step of creating the connecting information, and determining the connected state by matching the pixel pattern at said step of determining the connected state.

20. A system for processing a tracking and monitoring a picture of objects, comprising:

a first camera for imaging an overall monitoring area and outputting a first picture for detecting said objects entering a given monitoring area;

a second camera for outputting a picture signal for monitoring a picture to track said objects; and

a picture processing device for generating a control signal for controlling said second camera based on said first picture signal input from the first camera,

wherein said picture processing device comprises;

a detecting unit for detecting changes in the detected objects from said second picture signal; and

a classifing unit for classifying the detected changes in the detected objects into at least one of a plurality of predetermined state changes including appearance, single connection, coupling, disappearance and separation, and

wherein said second camera operates to automatically track and image said objects entering into said monitoring area in response to said control signal.

21. The system as claimed in claim 20, wherein said first camera is provided with an imaging lens of a wide field of view, said second camera is provided with a zoom lens, and said second camera is mounted on an electric pan and tilt head, and

wherein said picture processing device generates a rotation control signal for controlling said electric pan and tilt head signal inputted from said first camera.

22. The system as claimed in claim 21, wherein said picture processing device is served to generate a zooming control signal for zooming said zoom lens by a given amount of from said video signal inputted from said first camera, and said second camera is served to automatically track said object entering said monitoring area and image said object as automatically zooming said lens under the control of said picture processing device.

23. The system as claimed in claim 22, wherein said picture processing device includes a data bus, a central processing unit (CPU), a program memory for saving a program for operating said CPU, picture memories, a working memory, and a pan and tilt head control unit, the latter five of which are connected with said data bus, and

wherein said CPU operates to generate a picture of said entering object from the video signal inputted from said first camera, save said picture in said picture memory, detect said entering object from said picture, operate the data about said entering object, store the operated data in said working memory, and enable said pan and tilt head control unit to generate said rotation control signal and said zooming control signal based on said operated data.

24. The system as claimed in claim 23, wherein the data operated through the effect of the CPU and stored in said working memory contains a locational coordinate of said entering object.

25. The system as claimed in claim 23, wherein said picture memory contains:

an input picture memory for storing the video signal inputted from said first camera;

a background picture memory for storing a background video signal used for detecting said entering object;

two working memories for storing the input video signal and the background video signal used for doing an operation for detecting a difference between said background picture and said input picture;

an inter-picture operating result picture memory for storing the detected and operated difference between said background picture and the input picture; and

a digital picture memory for storing as a picture of said entering object a binary-coded video signal of said detected and operated difference between said background picture and said input picture.

26. The system as claimed in claim 21, wherein said picture processing device includes a data bus, a CPU, a program memory for saving a program for operating said CPU, picture memories, a working memory, and a pan and tilt head control unit, the latter five of which are connected with said data bus, and

wherein said CPU operates to generate a picture of said entering object from the video signal inputted from said first camera, store the video signal in said picture memory, detect said entering object from said picture, operate data about said intruded object, store said operated data in said working memory, and enable said pan and tilt head control unit to generate said rotation control signal based on said operated data.

27. The system as claimed in claim 26, wherein the data operated through the effect of the CPU and stored in said working memory contains a locational coordinate of said entering object.

28. The system as claimed in claim 26, wherein said picture memory contains:

an input picture memory for storing the video signal inputted from said first camera;

a background picture memory for storing a background video signal used for detecting said entering object;

two working memories for storing the input video signal and the background video signal for carrying out an operation for detecting a difference between the background picture and the in put picture;

an inter-picture operation result picture memory for storing the detected and operated difference between said background picture and said input picture; and

a binary-coded picture memory for storing as a picture of said entering object a binary-coded image signal of said detected and operated difference between said background picture and said input picture.

29. The method as claimed in claim 28, further comprising the steps of:

determining if plural objects are connected as one object when said objects is detected, from said connecting information; and

correcting one object determined as a connected state into plural objects based on the connecting information before and after the detection.

30. The method as claimed in claim 29, further comprising the step of:

recursively determining if the correction for the connected state is executed at said step of correcting the connected state,

wherein said step of determining the connected state and said step of correcting the connected state are repeated until no more corrections for the connected state are possible.

31. The method as claimed in claim 30, wherein the operation is executed to hold a location of said object, area of a region occupied by said object, and a pixel pattern of said region, detected as the connecting information at said step of creating the connecting information, and the operation is executed to determine the connected state by matching the pixel pattern at said step of determining the connected state.