Method and apparatus for sensing object located within visual field of imaging device

• US 6,088,468 A
• Filed: 05/07/1996
• Issued: 07/11/2000
• Est. Priority Date: 05/17/1995
• Status: Expired due to Term

• Current Assignee: Hitachi Denshi Kabushiki Kaisha
• Original Assignee: Hitachi Denshi Kabushiki Kaisha
• Inventors: Ito, Wataru, Ueda, Hirotada
• Primary Examiner(s): Chang, Jon
• Assistant Examiner(s): Patel, Jayanti K.
• Application Number:
  US08/646,018
• Time in Patent Office: 1,526 Days
• Field of Search: 348/143, 348/149, 348/152, 348/155, 348/169-172, 382/103, 382/104, 382/130, 382/218, 382/219, 342/454-457
• US Class Current: 382/103
• CPC Class Codes: G06T 7/254 involving subtraction of images
• CPC Class Codes: G06V 20/54 of traffic, e.g. cars on the road, trains or boats

Abstract

An apparatus for sensing an object within an imaging field of an image pickup device includes a memory for storing data of a reference background picture of the imaging field, a device for inputting picture data from the image pickup device, a device for comparing the inputted picture data with the stored reference background picture data to detect a difference value between the inputted picture data and the stored reference background picture data, and to detect a polarity of the difference value, a converter for converting the difference value into binarization data based on a difference threshold value which is changed in response to the polarity of the difference value so that an object within the imaging field can be distinguished from an area of increased brightness within the imaging field and an area of increased darkness within the imaging field, a calculator for calculating a similarity between the inputted picture data and the stored reference background picture data, and a detector for sensing whether an object is within the imaging field based on the polarity of the difference value and the similarity.

Claims

1. An apparatus for sensing an object within an imaging field of an image pickup device, the apparatus comprising:

a memory for storing data of a reference background picture of the imaging field;

means for inputting picture data from the image pickup device;

means for comparing the inputted picture data with the stored reference background picture data to detect a difference value between the inputted picture data and the stored reference background picture data, and to detect a polarity of the difference value;

a converter for converting the difference value into binarization data based on a difference threshold value which is changed in response to the polarity of the difference value so that an object within the imaging field can be distinguished from an area of increased brightness within the imaging field and an area of increased darkness within the imaging field;

a calculator for calculating a similarity between the inputted picture data and the stored reference background picture data; and

a detector for sensing whether an object is within the imaging field based on the polarity of the difference value and the similarity.

2. An apparatus as claimed in claim 1, wherein the converter includes

a first converter which converts the difference value into first binarization data based on a first difference threshold value if the polarity of the difference value is positive so that an object within the imagine field can be distinguished from an area of increased brightness within the imaging field, and

a second converter which converts the difference value into second binarization data based on a second difference threshold value if the polarity of the difference value is negative so that an object within the imaging field can be distinguished from an area of increased darkness within the imaging field;

and

wherein the detector senses whether an object is within the imaging field based on the first binarization data, the second binarization data, and the similarity.

3. An apparatus as claimed in claim 1, wherein the detector senses whether an object is within the imaging field by comparing the similarity with a similarity threshold value which is changed in response to the polarity of the difference value so that an object within the imaging field is distinguished from an area of increased brightness within the imaging field and an area of increased darkness within the imaging field.

4. A method of sensing an object within an imaging field of an image pickup device, the method comprising the steps of:

storing data of a reference background picture of the imaging field;

reading out picture data from the image pickup device;

comparing the read-out picture data with the stored reference background picture data to detect a difference value between the read-out picture data and the stored reference background picture data, and to detect a polarity of the difference value;

converting the difference value into binarization data based on a difference threshold value which is changed in response to the polarity of the difference value so that an object within the imaging field can be distinguished from an area of increased brightness within the imaging field and an area of increased darkness within the imaging field;

calculating a similarity between the read-out picture data and the stored reference background picture data; and

sensing whether an object is within the imaging field based on the polarity of the difference value and the similarity.

5. A method as claimed in claim 4, wherein the converting step includes the steps of

converting the difference value into first binarization data based on a first difference threshold value if the polarity of the difference value is positive so that an object within the imaging field can be distinguished from an area of increased brightness within the imaging field, and

converting the difference value into second binarization data based on a second difference threshold value if the polarity of the difference value is negative so that an object within the imaging field can be distinguished from an area of increased darkness within the imaging field;

and

wherein the sensing step includes the step of sensing whether an object is within the imaging field based on the first binarization data, the second binarization data, and the similarity.

6. A method as claimed in claim 4, wherein the step of storing the reference background picture data includes the steps of:

reading out a predetermined number of frames of picture data from the image pickup device during a predetermined period;

calculating median picture data of the predetermined number of frames of picture data read out from the image pickup device; and

updating the stored reference background picture data with the median picture data.

7. A method as claimed in claim 6, wherein the predetermined number of frames of picture data read out from the image pickup device during the predetermined period is smaller than a total number of frames produced by the image pickup device during the predetermined period.

8. A method as claimed in claim 6, wherein a period at which each of the predetermined number of frames of picture data is read out from the image pickup device is longer than a period at which the image pickup device produces one frame of picture data.

9. A method as claimed in claim 6, wherein the sensing step includes the step of issuing an alarm if an object is sensed;

and

wherein the step of updating the stored reference background picture data includes the step of updating the stored reference background picture data if an object is not sensed in the sensing step.

10. A method as claimed in claim 6, wherein the step of reading out the predetermined number of frames of picture data includes the steps of

reading out N1 frames of picture data from the image pickup device, N1 being a positive integer of at least two, and

reading out N2 frames of picture data from the image pickup device if an object is not sensed in the sensing step, N2 being a positive integer of at least two, N2 being less than N1;

wherein the step of calculating median picture data includes the steps of

calculating first median picture data of the N1 frames of picture data; and

calculating second median picture data of the N2 frames of picture data; and

wherein the step of updating the stored reference background picture data includes the step of replacing the stored reference background picture data with any one of the first median picture data and the second median picture data.

11. A method as claimed in claim 6, wherein the step of updating the stored reference background picture data includes the step of updating the stored reference background picture data with the median picture data when an object is not sensed in the sensing step.

12. A method as claimed in claim 4, further comprising the step of updating the stored reference background picture data when the object is not sensed.

13. A method as claimed in claim 4, wherein the sensing step includes the step of sensing whether an object is within the imaging field by comparing the similarity with a similarity threshold value which is changed in response to the polarity of the difference value so that an object within the imaging field is distinguished from an area of increased brightness within the imaging field and an area of increased darkness within the imaging field.

14. A method of sensing an object within an imaging field of an image pickup device, the method comprising the steps of:

storing data of a plurality of pixels of a reference background picture of the imaging field;

reading out picture data of a plurality of pixels from the image pickup device;

comparing the data of the pixels of the read-out picture data with the data of corresponding ones of the pixels of the stored reference background picture data to detect respective difference values between the data of the pixels of the read-out picture data and the data of the corresponding one of the pixels of the stored reference background picture data, and to detect respective polarities of the difference values;

converting the difference values into respective binarization data based on a difference threshold value which is changed in response to the respective polarities of the difference values so that an object within the imaging field can be distinguished from an area of increased brightness within the imaging field and an area of increased darkness within the imaging field;

calculating respective similarities between the pixels of the read-out picture data and the corresponding one of the pixels of the stored reference background picture data; and

sensing whether an object is within the imaging field based on the polarities of the difference values and the similarities.

15. A method as claimed in claim 14, wherein the calculating step is performed after the converting step.

16. A method as claimed in claim 15, wherein the sensing step includes the steps of:

comparing the similarities with a similarity threshold value; and

sensing that an object is within an area of the imaging field where pixels have respective similarities which are not greater than the similarity threshold value.

17. A method as claimed in claim 15, wherein the sensing step includes the steps of:

comparing the similarities with a similarity threshold value; and

sensing that an object is not within an area of the imaging field where pixels have respective difference values having a positive polarity, and have respective similarities which are greater than the similarity threshold value.

18. A method as claimed in claim 15, wherein the sensing step includes the steps of:

comparing the similarities with a similarity threshold value; and

sensing that an object is not within an area of the imaging field where pixels have respective difference values having a negative polarity, and have respective similarities which are greater than the similarity threshold value.

19. A method as claimed in claim 15, wherein the sensing step includes the steps of:

comparing the similarities with a similarity threshold value; and

distinguishing an area of the imaging field where pixels have respective similarities which are greater than the similarity threshold value from an area of the imaging field where pixels have respective similarities which are not greater than the similarity threshold value.

20. A method as claimed in claim 19, wherein the image pickup device is an infrared image pickup device;

and

wherein the distinguishing step includes the step of determining an area of the imaging field where pixels have respective difference values having a positive polarity and have respective similarities which are not greater than the similarity threshold value to be an area of the imaging field where a temperature has increased.

21. A method as claimed in claim 19, wherein the image pickup device is an infrared image pickup device;

and

wherein the distinguishing step includes the step of determining an area of the imaging field where pixels have respective difference values having a negative polarity and have respective similarities which are not greater than the similarity threshold value to be an area of the imaging field where a temperature has decreased.

22. A method as claimed in claim 14, wherein the calculating step is performed after the converting step, and is performed only for pixels having respective difference values which are higher than the difference threshold value.

23. A method as claimed in claim 14, wherein the sensing step includes the step of sensing whether an object is within the imaging field by comparing the similarities with a similarity threshold value which is changed in response to the polarities of the difference values so that an object within the imaging field is distinguished from an area of increased brightness within the imaging field and an area of increased darkness within the imaging field.

24. An apparatus for sensing an object within an imaging field of an image pickup device, the apparatus comprising:

a memory for storing data of a reference background picture of the imaging field;

means for inputting picture data from the image pickup device;

means for comparing the inputted picture data with the stored reference background picture data to detect a difference value between the inputted picture data and the stored reference background picture data, and to detect a polarity of the difference value;

a converter for converting the difference value into binarization data based on a difference threshold value which depends on the polarity of the difference value;

a calculator for calculating a similarity between the inputted picture data and the stored reference background picture data; and

a detector for sensing whether an object is within the imaging field based on the polarity of the difference value, the similarity, and a similarity threshold value which is changed in response to the polarity of the difference value so that an object within the imaging field is distinguished from an area of increased brightness within the imaging field and an area of increased darkness within the imaging field.

25. An apparatus as claimed in claim 24, wherein the converter includes

a first converter which converts the difference value into first binarization data based on a first difference threshold value if the polarity of the difference value is positive, and

a second converter which converts the difference value into second binarization data based on a second difference threshold value if the polarity of the difference value is negative;

and

wherein the detector senses whether an object is within the imaging field based on the first binarization data, the second binarization data, the similarity, and the similarity threshold value.

26. A method of sensing an object within an imaging field of an image pickup device, the method comprising the steps of:

storing data of a reference background picture of the imaging field;

reading out picture data from the image pickup device;

comparing the read-out picture data with the stored reference background picture data to detect a difference value between the read-out picture data and the stored reference background picture data, and to detect a polarity of the difference value;

converting the difference value into binarization data based on a difference threshold value which depends on the polarity of the difference value;

calculating a similarity between the read-out picture data and the stored reference background picture data; and

sensing whether an object is within the imaging field based on the polarity of the difference value, the similarity, and a similarity threshold value which is changed in response to the polarity of the difference value so that an object within the imaging field is distinguished from an area of increased brightness within the imaging field and an area of increased darkness within the imaging field.

27. A method as claimed in claim 26, wherein the converting step includes the steps of

converting the difference value into first binarization data based on a first difference threshold value if the polarity of the difference value is positive, and

converting the difference value into second binarization data based on a second difference threshold value if the polarity of the difference value is negative;

and

wherein the sensing step includes the step of sensing whether an object is within the imaging field based on the first binarization data, the second binarization data, the similarity, and the similarity threshold value.

28. A method as claimed in claim 26, wherein the step of storing the reference background picture data includes the steps of:

reading out a predetermined number of frames of picture data from the image pickup device during a predetermined period;

calculating median picture data of the predetermined number of frames of picture data read out from the image pickup device; and

updating the stored reference background picture data with the median picture data.

29. A method as claimed in claim 28, wherein the predetermined number of frames of picture data read out from the image pickup device during the predetermined period is smaller than a total number of frames produced by the image pickup device during the predetermined period.

30. A method as claimed in claim 28, wherein a period at which each of the predetermined number of frames of picture data is read out from the image pickup device is longer than a period at which the image pickup device produces one frame of picture data.

31. A method as claimed in claim 28, wherein the sensing step includes the step of issuing an alarm if an object is sensed;

and

wherein the step of updating the stored reference background picture data includes the step of updating the stored reference background picture data if an object is not sensed in the sensing step.

32. A method as claimed in claim 28, wherein the step of reading out the predetermined number of frames of picture data includes the steps of

reading out N1 frames of picture data from the image pickup device, N1 being a positive integer of at least two, and

reading out N2 frames of picture data from the image pickup device if an object is not sensed in the sensing step, N2 being a positive integer of at least two, N2 being less than N1;

wherein the step of calculating median picture data includes the steps of

calculating first median picture data of the N1 frames of picture data; and

calculating second median picture data of the N2 frames of picture data; and

wherein the step of updating the stored reference background picture data includes the step of replacing the stored reference background picture data with any one of the first median picture data and the second median picture data.

33. A method as claimed in claim 28, wherein the step of updating the stored reference background picture data includes the step of updating the stored reference background picture data with the median picture data when an object is not sensed in the sensing step.

34. A method as claimed in claim 26, further comprising the step of updating the stored reference background picture data when the object is not sensed.

35. A method of sensing an object within an imaging field of an image pickup device, the method comprising the steps of:

storing data of a plurality of pixels of a reference background picture of the imaging field;

reading out picture data of a plurality of pixels from the image pickup device;

comparing the data of the pixels of the read-out picture data with the data of corresponding ones of the pixels of the stored reference background picture data to detect respective difference values between the data of the pixels of the read-out picture data and the data of the corresponding one of the pixels of the stored reference background picture data, and to detect respective polarities of the difference values;

converting the difference values into respective binarization data based on a threshold value which depends on the respective polarities of the difference values;

calculating respective similarities between the pixels of the read-out picture data and the corresponding one of the pixels of the stored reference background picture data; and

sensing whether an object is within the imaging field based on the polarities of the difference values, the similarities, and a similarity threshold value which is changed in response to the polarities of the difference values so that an object within the imaging field is distinguished from an area of increased brightness within the imaging field and an area of increased darkness within the imaging field.

36. A method as claimed in claim 35, wherein the calculating step is performed after the converting step.

37. A method as claimed in claim 36, wherein the sensing step includes the steps of:

comparing the similarities with the similarity threshold value; and

sensing that an object is within an area of the imaging field where pixels have respective similarities which are not greater than the similarity threshold value.

38. A method as claimed in claim 36, wherein the sensing step includes the steps of:

comparing the similarities with the similarity threshold value; and

sensing that an object is not within an area of the imaging field where pixels have respective difference values having a positive polarity, and have respective similarities which are greater than the similarity threshold value.

39. A method as claimed in claim 36, wherein the sensing step includes the steps of:

comparing the similarities with the similarity threshold value; and

sensing that an object is not within an area of the imaging field where pixels have respective difference values having a negative polarity, and have respective similarities which are greater than the similarity threshold value.

40. A method as claimed in claim 36, wherein the sensing step includes the steps of:

comparing the similarities with the similarity threshold value; and

distinguishing an area of the imaging field where pixels have respective similarities which are greater than the similarity threshold value from an area of the imaging field where pixels have respective similarities which are not greater than the similarity threshold value.

41. A method as claimed in claim 40, wherein the image pickup device is an infrared image pickup device;

and

wherein the distinguishing step includes the step of determining an area of the imaging field where pixels have respective difference values having a positive polarity and have respective similarities which are not greater than the similarity threshold value to be an area of the imaging field where a temperature has increased.

42. A method as claimed in claim 40, wherein the image pickup device is an infrared image pickup device;

and

wherein the distinguishing step includes the step of determining an area of the imaging field where pixels have respective difference values having a negative polarity and have respective similarities which are not greater than the similarity threshold value to be an area of the imaging field where a temperature has decreased.

43. A method as claimed in claim 35, wherein the calculating step is performed after the converting step, and is performed only for pixels having respective difference values which are higher than the difference threshold value.