Object detecting method and object detecting apparatus and intruding object monitoring apparatus employing the object detecting method

• US 7,082,209 B2
• Filed: 08/21/2001
• Issued: 07/25/2006
• Est. Priority Date: 08/31/2000
• Status: Active Grant

• Current Assignee: Hitachi Kokusai Electric Incorporated (Hitachi, Ltd.)
• Original Assignee: Hitachi Kokusai Electric Incorporated (Hitachi, Ltd.)
• Inventors: Ito, Wataru, Ueda, Hirotada, Okada, Toshimichi
• Primary Examiner(s): Choobin, Barry
• Application Number:
  US09/933,164
• Publication Number:
  US 20020041698A1
• Time in Patent Office: 1,799 Days
• Field of Search: 387/103, 387/312, 382/218, 382/294, 382/103, 382/107, 382/173, 382/171, 382/104, 348/169, 348/152, 348/352, 348/699, 342/75, 342/76, 235/411, 378/98.11
• US Class Current: 382/103
• CPC Class Codes: G06T 7/254 involving subtraction of images

Abstract

An object detecting method, object detecting apparatus and intruding object monitoring apparatus employing the object detecting method. A predetermined monitor area having no object to be detected is imaged by an imaging device and reference background sequential images are registered in advance in a storage unit. When monitoring, the predetermined monitor area is imaged by the imaging device while at the same time sequentially outputting the images frame by frame from the imaging device, a reference background image corresponding to the image from the imaging device is selected from the reference background sequential images, subtraction processing is carried out between the image from the imaging device and a corresponding reference background image, and the object detecting processing is carried out based on the result of subtraction processing.

Claims

1. An object detecting method for detecting an object in a predetermined monitor area, comprising the steps of:

imaging a plurality of different areas in said predetermined monitor area having no object at a predetermined time by an imaging device and registering respective images corresponding to said different areas having no image of said object to be detected;

at a different time from said predetermined time, imaging said predetermined monitor area by said imaging device;

comparing an image from said imaging device with a corresponding one of said plurality of registered images; and

detecting said object to be detected based on a result of the comparison,

wherein said plurality of registered images are reference background sequential images and said step of comparing is subtraction processing between said image from said imaging device and said corresponding one of said reference background sequential images.

2. A method according to claim 1, further comprising the step of detecting a displacement between the image from the imaging device and a corresponding reference background image, then correcting the image from said imaging device in accordance with the detected displacement, wherein said subtraction processing is carried out between the corrected image and the corresponding reference background image.

3. A method according to claim 2, wherein said step of detecting the displacement includes the step of applying a template matching between said image from said imaging device and the corresponding reference background image to detect said displacement.

4. A method according to claim 3,

wherein said template matching step is such that the reference background image is divided into a plurality of sections,

wherein said image from said imaging device is subjected to said template matching using the image of each section as a template, and

wherein an average of displacements detected is used as said displacement.

5. A method according to claim 1, further comprising the step of detecting a frame displacement between the frame of the image from the imaging device and the frame of said corresponding reference background image, wherein when there is any frame displacement, a different reference background image is selected.

6. A method according to claim 5, wherein said frame displacement detecting step includes the step of applying a template matching between said image from said imaging device and said corresponding reference background image and detecting a frame displacement.

7. A method according to claim 6,

wherein said template matching step is such that said corresponding reference background image is divided into a plurality of sections, and using the image of each section as a template, the template matching is carried out with said image from said imaging device, and

wherein in accordance with frame displacement information detected, a reference background image of the frame temporally ahead of or lagging behind the frame of said reference background image is selected.

8. A method according to claim 5, wherein said frame displacement detecting step corrects the frame displacement based on at least one of a position and imaging visual field information of said imaging device.

9. A method according to claim 8, wherein a specific position of the imaging device and a specific frame of the reference background image corresponding to said specific position are set in association with each other in advance, and when the imaging device arrives at said specific position, the frame displacement is corrected using the reference background image of said specific frame.

10. A method according to claim 8,

wherein said imaging visual field information contains a specific object within the predetermined monitor visual field as a mark, and said mark and the reference background image of a specific frame corresponding to said mark are set in association with each other in advance, and

wherein when the imaging device picks up the image of said mark, the frame displacement is corrected using the reference background image of the specific frame.

11. A method according to claim 1, further comprising the step of updating the reference background sequential images to update at least one of the reference background sequential images.

12. A method according to claim 11, wherein said update step functions in such a manner that when an object is not detected in said image from said imaging device in said object detection processing step, said corresponding reference background image is updated with said image.

13. An object detecting method for detecting an object in a predetermined monitor area, comprising the steps of:

imaging a plurality of different areas in the predetermined monitor area at a predetermined time by the imaging device in accordance with a predetermined scanning pattern, and registering respective reference background sequential images corresponding to said different areas in a storage device;

at a different time from said predetermined time, imaging said predetermined monitor area by said imaging device substantially in accordance with said predetermined scanning pattern in synchronism with the operation of reading said registered reference background sequential images from the storage device;

carrying out the subtraction processing between the images from the imaging device and the reference background sequential images read out; and

carrying out object detection processing based on a result of the subtraction processing.

14. A method according to claim 13, wherein said predetermined scanning pattern is a predetermined chronological change of at least one or a combination of two or more of the zoom ratio of a zoom lens of said imaging device, an imaging direction and a track along which said imaging device moves.

15. A method according to claim 13, wherein said predetermined scanning pattern is such that said imaging device moves on a predetermined speed profile along a predetermined moving track.

16. A method according to claim 13, wherein said predetermined scanning pattern is such that the position of the imaging device is fixed while the zoom ratio and the imaging direction of the imaging device periodically change.

17. A method according to claim 13,

wherein said reference background sequential images are a set of images obtained by scanning said predetermined monitor area having no object with the imaging device according to the predetermined scanning pattern and sampling the images from said imaging device at predetermined sampling intervals,

wherein said registration step is such that each image of said reference background sequential images is registered by being assigned a frame number in order of sampling, and

wherein said imaging step is such that the imaging operation of the imaging device and the operation of reading the reference background image from the storage device are carried out in synchronism with each other using the frame number, so that a reference background image corresponding to said image from the imaging device is selected from the reference background sequential images.

18. A method according to claim 17,

wherein the frame number of the corresponding reference background image is calculated from a relation between the time elapsed from start of monitoring to the present point in time and the predetermined sampling interval, and

wherein the imaging operation of the imaging device and the operation of reading the reference background image from the storage device are synchronized with each other using said calculated frame number.

19. A method according to claim 18, further comprising the step of detecting a frame displacement between the image from the imaging device and said selected reference background image, wherein in the presence of a frame displacement, another reference background image is selected.

20. A method according to claim 19, wherein said frame displacement detection step includes the template matching step for detecting the frame displacement by template matching between said image from said imaging device and said selected reference background image.

21. A method according to claim 20, wherein said template matching step includes the substeps of:

dividing said selected reference background image into a plurality of sections;

carrying out the template matching on said image from said imaging device using the image of each of said section as a template; and

selecting a reference background image temporally ahead of or lagging behind said selected reference background image in accordance with the detected displacement information.

22. A method according to claim 19, wherein said frame displacement detection step is such that said frame displacement is corrected based on at least one of a position of said imaging device and imaging visual field information.

23. A method according to claim 22, wherein a specific position of said imaging device and a specific frame number of the reference background image corresponding to said specific position are set in association with each other in advance, and when said imaging device reaches said specific position, said frame displacement is corrected using said specific frame number.

24. A method according to claim 22,

wherein said imaging visual field information contains a specific object located within said predetermined monitor area as a mark,

wherein said mark and a specific frame number of the reference background image corresponding to said mark are set in association with each other in advance, and

wherein when said imaging device picks up the image of said mark, said frame displacement is corrected using said specific frame number.

25. A method according to claim 13, further comprising the step of detecting a spatial displacement between said image from said imaging device and said reference background image that has been read, and correcting said read image in accordance with said detected spatial displacement, said subtraction processing being carried out using said corrected image.

26. An object detecting apparatus for detecting an object located within a predetermined monitor area, comprising:

an imaging device;

an image input interface connected to said imaging device for converting a video signal of the imaging device to image data;

a processing unit including a central processing unit and a memory for processing said image data; and

a bus for interconnecting said image input interface and said processing unit;

wherein in order to detect the object, said processing unit controls said object detecting apparatus such that;

images from said imaging device imaging said predetermined monitor area having no object to be detected are stored sequentially in said memory,

an image from said imaging device imaging said predetermined monitor area in accordance with a predetermined scanning pattern is sequentially inputted to said processing unit,

an image having no object to be detected corresponding to said inputted image is read out of said memory, and

said input image and said read image having no object to be detected are compared and based on a result of comparison, object detection processing is carried out,

wherein said frame images having no image of said object to be detected are reference background sequential images and said comparing including subjecting said image from said imaging device and said corresponding image having no image of said object to be detected to subtraction processing.

27. An image detecting apparatus for detecting an object located within a predetermined monitor area, comprising:

an imaging device;

an image input interface connected to said imaging device for converting a video signal of the imaging device to image data;

a processing unit including a central processing unit and a memory for processing said image data; and

a bus for interconnecting said image input interface and said processing unit,

wherein in order to detect the object, said processing unit controls said object detecting apparatus to;

store in said memory a plurality of images having different visual fields from said imaging device which images said predetermined monitor area at different visual fields,

read out of said memory a stored image which corresponds to an image output from said imaging device, and

compare said input image with said image read out of said memory and perform object detection processing based on said comparison.

28. An apparatus according to claim 27, further comprising a zoom lens control unit connected to said bus for changing a zoom ratio of a zoom lens of said imaging device, and a pan and tilt head control unit for changing an imaging direction of said imaging device.

29. An apparatus according to claim 27, wherein said imaging device is mounted on a moving unit.

30. An apparatus according to claim 29, wherein said moving unit includes a mobile unit.

31. An apparatus according to claim 29, wherein said moving device includes a pan and tilt head.

35. An apparatus according to claim 27, wherein said plurality of images having different visual fields stored in said memory are reference background images, respectively, and said comparison of said inputted image with said image read out of said memory includes subtraction processing between said inputted image and a reference background image corresponding to said inputted image.

32. An object detecting apparatus for detecting an object within a predetermined monitor area, comprising:

an imaging device;

an image input interface connected to the imaging device for converting a video signal from the imaging device to image data;

a processing unit including a central processing unit and a memory for processing the image data; and

a bus for interconnecting the image input interface and the processing unit;

wherein in order to detect the object, said processing unit controls said detecting apparatus such that;

an image from said imaging device imaging said predetermined monitor area having no object to be detected is recorded in said memory beforehand as a reference background image of reference background sequential images;

an image from the imaging device imaging the predetermined monitor area in accordance with a predetermined scanning pattern is sequentially inputted to the processing unit;

a reference background image is read out of said reference background sequential images from the image memory in synchronism with picking up the input frame images; and

a difference in a pixel value between the input image and the read reference background image is calculated for each pixel, and an area associated with a large difference value is detected as an object.

33. A monitoring apparatus for monitoring an object intruding into a predetermined monitor area, comprising:

an imaging device;

an image input interface connected to the imaging device for converting a video signal from the imaging device to image data;

a processing unit including a central processing unit and a memory for processing said image data;

a monitor; and

a bus for interconnecting said image input interface and said processing unit;

wherein said processing unit controls said monitoring apparatus such that;

each n-th image from the imaging device imaging the predetermined monitor area having no object to be detected is recorded in the memory beforehand as reference background sequential images, where n is an integer not less than unity;

an image from the imaging device imaging the predetermined monitor area is sequentially input to the processing unit;

a reference background image corresponding to the input image is read from the memory; and

a difference in a pixel value between the input image and the reference background image corresponding to the input image read is calculated for each pixel, and an area associated with a large difference value is detected as an object and displayed on said monitor.

34. A monitoring apparatus for monitoring an object intruding into a predetermined monitor area, comprising:

an imaging device;

an image input interface connected to said imaging device for converting a video signal from the imaging device to image data;

a processing unit including a central processing unit and a memory for processing said image data;

a monitor; and

a bus for interconnecting the image input interface, the processing unit and the monitor;

wherein said processing unit controls said monitoring apparatus such that;

each n-th image from the imaging device imaging the predetermined monitor area having no object to be detected is recorded in the memory beforehand as reference background sequential images, where n is an integer not less than unity;

images from the imaging device imaging the predetermined monitor area in accordance with a predetermined scanning pattern are sequentially input to the processing unit;

reference background sequential images are read from the memory in synchronism with the operation of picking up the input images; and

a difference in a pixel value between an input image and a reference background image read is calculated for each pixel, and an area associated with a large difference value is detected as an object and displayed on the monitor.