Object detection for sudden illumination changes using order consistency

• US 7,006,128 B2
• Filed: 05/30/2002
• Issued: 02/28/2006
• Est. Priority Date: 05/30/2002
• Status: Active Grant

• Current Assignee: Siemens Schweiz AG (Siemens AG)
• Original Assignee: Siemens Corporate Research Incorporated (Siemens AG)
• Inventors: Xie, Binglong, Ramesh, Visvanathan, Boult, Terrance
• Primary Examiner(s): PHILIPPE, GIMS S
• Application Number:
  US10/158,632
• Publication Number:
  US 20030228058A1
• Time in Patent Office: 1,370 Days
• Field of Search: 348/700, 348/144, 348/126, 348/143, 348/131, 348/129, 375/240.29, 375/240.16, 382/236, 382/151, 382/218, 356/394
• US Class Current: 348/143
• CPC Class Codes: G06T 7/254 involving subtraction of images
• CPC Class Codes: G08B 13/19604 involving reference image or background adaptation with time to compensate for changing conditions, e.g. reference image update on detection of light level change

Abstract

A system and method for object detection with sudden illumination changes provides a system including a camera model unit for computing a difference map between a first input image and a second input image, an illumination model unit in signal communication with the camera model unit for comparing the sign of the difference between two pixels in a first input image against the sign of the difference between two pixels in a second input image, and a CPU in signal communication with the camera model unit and the illumination model unit for detecting a changed object in accordance with said comparison when said sign of the difference is changed; and a corresponding method including providing a first image having a number of pixels, receiving a second image subsequent to the first image, calculating a difference map between the first and second images, selecting a pair of pixel locations from the first or second images in accordance with the difference map, statistically modeling a camera noise level for the first and second images, computing the likelihood of order inconsistency between a pair of pixel values at the selected locations in the first image and a pair of pixel values at the selected locations in the second image in accordance with the camera noise level; and detecting a changed object in accordance with the computed likelihood.

Claims

1. A method for object detection with sudden illumination changes, the method comprising:

providing a first image having a plurality of pixels;

receiving a second image corresponding to a time other than that of said first image;

calculating a difference map between said first and second images;

selecting a pair of pixel locations from at least one of said first and second images in accordance with said difference map;

statistically modeling a camera noise level for at least one of said first and second images;

computing the likelihood of order inconsistency between a pair of pixel values at said selected locations in said first image and a pair of pixel values at said selected locations in said second image in accordance with said camera noise level; and

detecting a changed object in accordance with said computed likelihood,

wherein the sign of the difference between the incident radiances of a pair of pixels in said first image is the same as the sign of the difference between the sensed pixel values.

2. A method as defined in claim 1 wherein said second image is temporally consecutive with said first image.

3. A method as defined in claim 1, further comprising maintaining said first image as a reference for calculating a plurality of difference maps corresponding to a plurality of second images.

4. A method as defined in claim 1, further comprising mixing said first image with at least one past image.

5. A method as defined in claim 4 wherein mixing comprises pixel

averaging.

6. A method as defined in claim 1 wherein said first image is a separately maintained background image.

7. A method as defined in claim 1, further comprising photometrically calibrating said statistical model for camera noise.

8. A method as defined in claim 1, further comprising applying a monotone but non-linear camera response function to at least one of said first and second images.

9. A method as defined in claim 8, further comprising photometrically calibrating said camera response function.

10. A method as defined in claim 1 wherein a change in at least one of camera gain and illumination level does not alter the difference between the incident radiances of said pair of pixels for an unchanged object, which remains the same as the sign of the difference between the sensed pixel values.

11. A method as defined in claim 1 wherein said comparing the sign comprises applying a Phong shading model for the surface material.

12. A method as defined in claim 1 wherein said detecting a changed object is responsive to local pixel information.

13. A method as defined in claim 1 wherein said detecting a changed object is not compromised by a sudden illumination change.

14. A system for object detection with sudden illumination changes, the system comprising:

providing means for providing a first image having a plurality of pixels;

receiving means for receiving a second image corresponding to a time other than that of said first image;

calculating means for calculating a difference map between said first and second images;

selecting means for selecting a pair of pixel locations from at least one of said first and second images in accordance with said difference map;

modeling means for statistically modeling a camera noise level for at least one of said first and second images;

computing means for computing the likelihood of order inconsistency between a pair of pixel values at said selected locations in said first image and a pair of pixel values at said selected locations in said second image in accordance with said camera noise level; and

detecting means for detecting a changed object in accordance with said computed likelihood,

wherein the sign of the difference between the incident radiances of a pair of pixels in said first image is the same as the sign of the difference between the sensed pixel values.

15. A system as defined in claim 14, further comprising:

a camera model unit for computing a difference map between a first input image and a second input image;

an illumination model unit in signal communication with the camera model unit for comparing the sign of the difference between two pixels in a first input image against the sign of the difference between two pixels in a second input image; and

a CPU in signal communication with said camera model unit and said illumination model unit for detecting a changed object in accordance with said comparison when said sign of the difference is changed.

16. A system as defined in claim 15 wherein at least one of said first and second input images comprises a surveillance image.

17. A system as defined in claim 15, further comprising:

a display adapter in signal communication with the CPU for displaying at least one of said map and said first and second input images; and

an I/O adapter in signal communication with the CPU for recalling the locations of the pixels from the displayed image to provide an indication of the location of a changed object within an input image.

18. A system as defined in claim 15, further comprising:

a user interface adapter in signal communication with the CPU for at least receiving a selection decision for at least one image from a user.

19. A system as defined in claim 14 wherein said second image is temporally consecutive with said first image.

20. A system as defined in claim 14, further comprising maintaining means for maintaining said first image as a reference for calculating a plurality of difference maps corresponding to a plurality of second images.

21. A system as defined in claim 14, further comprising mixing means for mixing said first image with at least one past image.

22. A system as defined in claim 21 wherein said mixing means comprises pixel averaging means.

23. A system as defined in claim 14 wherein said first image is a separately maintained background image.

24. A system as defined in claim 14, further comprising first calibrating means for photometrically calibrating said statistical model for camera noise.

25. A system as defined in claim 14, further comprising second modeling means for applying a monotone but non-linear camera response function to at least one of said first and second images.

26. A system as defined in claim 25, further comprising second calibration means for photometrically calibrating said camera response function.

27. A system as defined in claim 14 wherein a change in at least one of camera gain and illumination level does not alter the difference between the incident radiances of said pair of pixels for an unchanged object, which remains the same as the sign of the difference between the sensed pixel values.

28. A system as defined in claim 14 wherein said comparing the sign comprises applying a Phong shading model for the surface material.

29. A system as defined in claim 14 wherein said detecting a changed object is responsive to local pixel information.

30. A system as defined in claim 14 wherein said detecting a changed object is not compromised by a sudden illumination change.

31. A program storage device readable by machine, tangibly embodying a program of instructions executable by the machine to perform method steps for object detection with sudden illumination changes, the method steps comprising:

providing a first image having a plurality of pixels;

receiving a second image corresponding to a time other than that of said first image;

calculating a difference map between said first and second images;

selecting a pair of pixel locations from at least one of said first and second images in accordance with said difference map;

statistically modeling a camera noise level for at least one of said first and second images;

computing the likelihood of order inconsistency between a pair of pixel values at said selected locations in said first image and a pair of pixel values at said selected locations in said second image in accordance with said camera noise level; and

detecting a changed object in accordance with said computed likelihood,

wherein the sign of the difference between the incident radiances of a pair of pixels in said first image is the same as the sign of the difference between the sensed pixel values.

32. A program storage device as defined in claim 31 wherein said second image is temporally consecutive with said first image.

33. A program storage device as defined in claim 31, the method steps further comprising maintaining said first image as a reference for calculating a plurality of difference maps corresponding to a plurality of second images.

34. A program storage device as defined in claim 31, the method steps further comprising mixing said first image with at least one past image.

35. A program storage device as defined in claim 34 wherein the method step of mixing comprises pixel averaging.

36. A program storage device as defined in claim 31 wherein said first image is a separately maintained background image.

37. A program storage device as defined in claim 31, the method steps further comprising photometrically calibrating said statistical model for camera noise.

38. A program storage device as defined in claim 31, the method steps further comprising applying a monotone but non-linear camera response function to at least one of said first and second images.

39. A program storage device as defined in claim 38, the method steps further comprising photometrically calibrating said camera response function.

40. A program storage device as defined in claim 31 wherein a change in at least one of camera gain and illumination level does not alter the difference between the incident radiances of said pair of pixels for an unchanged object, which remains the same as the sign of the difference between the sensed pixel values.

41. A program storage device as defined in claim 31 wherein said comparing the sign comprises applying a Phong shading model for the surface material.

42. A program storage device as defined in claim 31 wherein said detecting a changed object is responsive to local pixel information.

43. A program storage device as defined in claim 31 wherein said detecting a changed object is not compromised by a sudden illumination change.