Method and apparatus for moving object extraction based on background subtraction

• US 5,748,775 A
• Filed: 03/09/1995
• Issued: 05/05/1998
• Est. Priority Date: 03/09/1994
• Status: Expired due to Fees

• Current Assignee: Nippon Telegraph and Telephone Corporation
• Original Assignee: Nippon Telegraph and Telephone Corporation
• Inventors: Sato, Atsushi, Ishii, Kenichiro, Tsuchikawa, Megumu, Tomono, Akira
• Primary Examiner(s): Boudreau, Leo
• Assistant Examiner(s): Chen, Wenpeng
• Application Number:
  US08/401,972
• Time in Patent Office: 1,153 Days
• Field of Search: 382/168, 382/169, 382/170, 382/171, 382/172, 382/190, 382/192, 382/309
• US Class Current: 382/190
• CPC Class Codes: G06T 7/277 involving stochastic approaches, e.g. using Kalman filters
• CPC Class Codes: G06V 20/52 Surveillance or monitoring of activities, e.g. for recognising suspicious objects recognising microscopic objects G06V20/69
• CPC Class Codes: G06V 40/103 Static body considered as a whole, e.g. static pedestrian or occupant recognition
• CPC Class Codes: H04N 19/20 using video object coding

Abstract

A moving object extraction based on background subtraction capable of stably extracting the moving object under various environments. Temporal changes of image feature parameter values for sub-regions subdividing a frame of each input image are stored, and the background image is reconstructed by statistically processing a temporal change of the image feature parameter values for each sub-region within a prescribed target region of the frame over a prescribed period of time to obtain the statistical quantity characterizing that temporal change, judging whether that temporal change is due to an illumination change or not according to the obtained statistical quantity and a prescribed illumination change judging condition, and updating a background image value for each sub-region by a new background image value according to the image feature parameter values for each sub-region during the prescribed period of time. Then, a subtraction processing is applied to one of the input images and the reconstructed background image, and a binarization processing is applied to the obtained subtraction image so as to extract the moving object region from the input images.

Claims

1. A method of moving object extraction based on background subtraction, comprising the steps of:

(a) sequentially entering input images containing a moving object region to be extracted;

(b) storing temporal changes of image feature parameter values for sub-regions subdividing a frame of each input image entered at the step (a);

(c) statistically processing a temporal change of the image feature parameter values for each sub-region within a prescribed target region of the frame stored at the step (b) over a prescribed period of time t₀ to obtain at least one statistical quantity characterizing said temporal change, judging whether said temporal change is due to an illumination change or not according to said statistical quantity and a prescribed illumination change judging condition, and updating a background image value for said each sub-region by a new background image value according to the image feature parameter values for said each sub-region during the prescribed period of time t₀, so as to obtain a reconstructed background image;

(d) applying a subtraction processing to one of the input images entered at the step (a) and the reconstructed background image obtained at the step (c) to obtain a subtraction image; and

(e) applying a binarization processing to the subtraction image obtained at the step (d) to extract the moving object region from the input images entered at the step (a).

2. The method of claim 1, wherein at the step (c), whether said temporal change is due to the illumination change or not is judged by obtaining a histogram of the image feature parameter values for said each sub-region over the prescribed period of time t₀, obtaining said statistical quantity from said histogram, and comparing said statistical quantity with a predetermined reference statistical quantity representing a case of the illumination change.

3. The method of claim 1, wherein at the step (c), the new background image value is either one of an extreme value or a mean value of the image feature parameter values for said each sub-region over the prescribed period of time t₀.

4. The method of claim 1, wherein at the step (c), said each sub-region is a pixel in the frame, and said prescribed target region is a slit shaped region in the frame.

5. The method of claim 1, wherein at the step (c), whether said temporal change is due to the illumination change or not is judged by obtaining a histogram of the image feature parameter values for said each sub-region over the prescribed period of time t₀, obtaining said statistical quantity from said histogram, and relatively comparing said statistical quantity for said each sub-region with statistical quantities for other sub-regions in a vicinity of said each sub-region.

6. The method of claim 1, wherein at the step (b), the image feature parameter values of n types of image feature parameters for the sub-regions are stored with respect to the input images sequentially entered at the step (a), where n is an integer greater than one, and at the step (c), whether said temporal change is due to the illumination change or not is judged by forming n-dimensional feature vectors from the image feature parameter values of said n types of the image feature parameters for said each sub-region over the prescribed period of time t₀, obtaining said statistical quantity by statistically processing a set of said n-dimensional feature vectors, and judging whether said statistical quantity satisfies the predetermined illumination change judging condition.

7. The method of claim 6, wherein at the step (c), whether said temporal change is due to the illumination change or not is judged by obtaining a histogram of distances between frames of each of said n-dimensional feature vectors over the prescribed period of time t₀, obtaining said statistical quantity from said histogram, and comparing said statistical quantity with a predetermined reference statistical quantity representing a case of the illumination change.

8. The method of claim 6, wherein at the step (c), whether said temporal change is due to the illumination change or not is judged by obtaining in advance a characteristic curve of said n-dimensional vectors for said each sub-region indicating a manner by which said n-dimensional vectors change in a case of the illumination change, obtaining a histogram of distances between said characteristic curve and said n-dimensional feature vectors, obtaining said statistical quantity from said histogram, and comparing said statistical quantity with a predetermined reference statistical quantity representing a case of the illumination change.

9. The method of claim 8, wherein at the step (c), said statistical quantity is at least one of a mean value of the distances in said histogram and a variance of said histogram.

10. The method of claim 8, wherein the step (c) also judges whether the illumination change is gradual or abrupt according to a motion vector of a point projecting each of said n-dimensional vectors for said each sub-region onto said characteristic curve.

11. The method of claim 1, wherein at the step (c), the prescribed illumination change judging condition includes a plurality of conditions for specifying a plurality of different manners to update the background image value for said each sub-region.

12. The method of claim 1, wherein at the step (d), the subtraction processing is applied to a latest one of the input images entered at the step (a) and the reconstructed background image obtained at the step (c).

13. The method of claim 1, wherein at the step (d), the subtraction processing is applied to one of the input images entered at a prescribed period of time t₁ earlier than a current timing by the step (a) and the reconstructed background image obtained at the current timing by the step (c).

14. The method of claim 13, wherein at the step (c), the new background image value is a mean value of the image feature parameter values for said each sub-region over a prescribed period of time t₂ centered around a timing which is the prescribed period of time t₁ earlier than the current timing.

15. The method of claim 1, wherein at the step (e), a threshold used in the binarization processing is set to be different for different sub-regions.

16. The method of claim 15, wherein at the step (e), the threshold used in the binarization processing for said each sub-region is sequentially updated according to a value obtained by statistically processing the temporal change of the image feature parameter values for said each sub-region over the prescribed period of time t₀.

17. The method of claim 1, further comprising the steps of judging whether there is any change in the background image according to the temporal change of the image feature parameter values stored at the step (b), and omitting an operation to update the background image value at the step (c) when it is judged that there is no change in the background image.

18. The method of claim 1, wherein at the step (c), said statistical quantity is at least one of a variance and a distribution shape of a histogram of the image feature parameter values for said each sub-region over the prescribed period of time t₀.

19. The method of claim 18, wherein at the step (c), said statistical quantity further includes a maximum value of a histogram of temporal differential of the image feature parameter values for said each sub-region over the prescribed period of time t₀.

20. The method of claim 1, wherein each of the input images entered at the step (a) is a color image including a plurality of color components, and the steps (b), (d), and (e) are executed for each of said plurality of color components of each input image separately.

21. An apparatus for moving object extraction based on background subtraction, comprising:

input means for sequentially entering input images containing a moving object region to be extracted;

storage means for storing temporal changes of image feature parameter values for sub-regions subdividing a frame of each input image entered by the input means;

background update means for statistically processing a temporal change of the image feature parameter values for each sub-region within a prescribed target region of the frame stored by the storage means over a prescribed period of time t₀ to obtain at least one statistical quantity characterizing said temporal change, judging whether said temporal change is due to an illumination change or not according to said statistical quantity and a prescribed illumination change judging condition, and updating a background image value for said each sub-region by a new background image value according to the image feature parameter values for said each sub-region during the prescribed period of time t₀, so as to obtain a reconstructed background image;

subtraction means for applying a subtraction processing to one of the input images entered by the input means and the reconstructed background image obtained by the background update means to obtain a subtraction image; and

binarization means for applying a binarization processing to the subtraction image obtained by the subtraction means to extract the moving object region from the input images entered by the input means.

22. The apparatus of claim 21, wherein the background update means judges whether said temporal change is due to the illumination change or not by obtaining a histogram of the image feature parameter values for said each sub-region over the prescribed period of time t₀, obtaining said statistical quantity from said histogram, and comparing said statistical quantity with a predetermined reference statistical quantity representing a case of the illumination change.

23. The apparatus of claim 21, wherein the background update means uses the new background image value which is either one of an extreme value or a mean value of the image feature parameter values for said each sub-region over the prescribed period of time t₀.

24. The apparatus of claim 21, wherein said each sub-region is a pixel in the frame, and the background update means uses said prescribed target region which is a slit shaped region in the frame.

25. The apparatus of claim 21, wherein the background update means judges whether said temporal change is due to the illumination change or not by obtaining a histogram of the image feature parameter values for said each sub-region over the prescribed period of time t₀, obtaining said statistical quantity from said histogram, and relatively comparing said statistical quantity for said each sub-region with statistical quantities for other sub-regions in a vicinity of said each sub-region.

26. The apparatus of claim 21, wherein the storage means stores the image feature parameter values of n types of image feature parameters for said each sub-region with respect to the input images sequentially entered by the input means, where n is an integer greater than one, and the background update means judges whether said temporal change is due to the illumination change or not by forming n-dimensional feature vectors from the image feature parameter values of said n types of the image feature parameters for said each sub-region over the prescribed period of time t₀, obtaining said statistical quantity by statistically processing a set of said n-dimensional feature vectors, and judging whether said statistical quantity satisfies the predetermined illumination change judging condition.

27. The apparatus of claim 26, wherein the background update means judges whether said temporal change is due to the illumination change or not by obtaining a histogram of distances between frames of each of said n-dimensional feature vectors over the prescribed period of time t₀, obtaining said statistical quantity from said histogram, and comparing said statistical quantity with a predetermined reference statistical quantity representing a case of the illumination change.

28. The apparatus of claim 26, wherein the background update means judges whether said temporal change is due to the illumination change or not by obtaining in advance a characteristic curve of said n-dimensional vectors for said each sub-region indicating a manner by which said n-dimensional vectors change in a case of the illumination change, obtaining a histogram of distances between said characteristic curve and said n-dimensional feature vectors, obtaining said statistical quantity from said histogram, and comparing said statistical quantity with a predetermined reference statistical quantity representing a case of the illumination change.

29. The apparatus of claim 28, wherein the background update means obtains said statistical quantity which is at least one of a mean value of the distances in said histogram and a variance of said histogram.

30. The apparatus of claim 28, wherein the background update means also judges whether the illumination change is gradual or abrupt according to a motion vector of a point projecting each of said n-dimensional vectors for said each sub-region onto said characteristic curve.

31. The apparatus of claim 21, wherein the background update means uses the prescribed illumination change judging condition which includes a plurality of conditions for specifying a plurality of different manners to update the background image value for said each sub-region.

32. The apparatus of claim 21, wherein the subtraction means applies the subtraction processing to a latest one of the input images entered by the input means and the reconstructed background image obtained by the background update means.

33. The apparatus of claim 21, wherein the subtraction means applies the subtraction processing to one of the input images entered at a prescribed period of time t₁ earlier than a current timing by the input means and the reconstructed background image obtained at the current timing by the background update means.

34. The apparatus of claim 33, wherein the background update means uses the new background image value which is a mean value of the image feature parameter values for said each sub-region over a prescribed period of time t₂ centered around a timing which is the prescribed period of time t₁ earlier than the current timing.

35. The apparatus of claim 21, further comprising setting means for setting a threshold used in the binarization processing by the binarization means to be different for different sub-regions.

36. The apparatus of claim 35, wherein the setting means sequentially updates the threshold used in the binarization processing for said each sub-region according to a value obtained by statistically processing the temporal change of the image feature parameter values for said each sub-region over the prescribed period of time t₀.

37. The apparatus of claim 21, wherein the background update means also judges whether there is any change in the background image according to the temporal change of the image feature parameter values stored by the storage means, and omits an operation to update the background image value when it is judged that there is no change in the background image.

38. The apparatus of claim 21, wherein the background update means uses said statistical quantity which is at least one of a variance and a distribution shape of a histogram of the image feature parameter values for said each sub-region over the prescribed period of time t₀.

39. The apparatus of claim 38, wherein the background update means uses said statistical quantity which further includes a maximum value of a histogram of temporal differential of the image feature parameter values for said each sub-region over the prescribed period of time t₀.

40. The apparatus of claim 21, wherein each of the input images entered by the input means is a color image including a plurality of color components, and each of the storage means, the subtraction means, and the binarization means function for each of said plurality of color components of each input image separately.