Device and method for detecting shadow in image

• US 8,295,606 B2
• Filed: 02/04/2010
• Issued: 10/23/2012
• Est. Priority Date: 03/12/2009
• Status: Active Grant

• Current Assignee: Ricoh Company Limited
• Original Assignee: Ricoh Company Limited
• Inventors: Li, Tao, Yuan, Xun, Wang, Gang, Shi, Zhongchao
• Primary Examiner(s): Bella, Matthew
• Assistant Examiner(s): Li, Ruiping
• Application Number:
  US12/656,586
• Publication Number:
  US 20100232705A1
• Time in Patent Office: 992 Days
• Field of Search: 392/195, 392/224, 392/168, 392/190, 392/103, 392/199, 348/169
• US Class Current: 382/195
• CPC Class Codes: G06F 18/2411 based on the proximity to a decision surface, e.g. support vector machines
• CPC Class Codes: G06T 2207/10024 Color image
• CPC Class Codes: G06T 2207/20021 Dividing image into blocks, subimages or windows
• CPC Class Codes: G06T 7/11 Region-based segmentation
• CPC Class Codes: G06T 7/174 involving the use of two or more images
• CPC Class Codes: G06T 7/90 Determination of colour characteristics
• CPC Class Codes: G06V 10/764 using classification, e.g. of video objects
• CPC Class Codes: G06V 20/52 Surveillance or monitoring of activities, e.g. for recognising suspicious objects recognising microscopic objects G06V20/69

Abstract

A device for detecting a shadow region in an image includes an imaging module generating a multi-channel image including brightness, red, green, and blue channels, a brightness correcting module correcting values of the brightness channel based on imaging parameters and outputting a corrected multi-channel image, a scene classifying module determining to carry out a shadow detection on the corrected multi-channel image, a shadow detecting module classifying pixels of the corrected multi-channel image into a shadow or non-shadow pixel, and generating a shadow classification mark matrix having pixels having a shadow classification mark value corresponding to the classification, a region segmentation module segmenting the multi-channel image into regions having pixels having similar color values, and generating a region mark matrix having pixels having a region mark value, and a post-processing module updating the shadow classification mark matrix based on the shadow classification mark matrix and region mark matrix.

Claims

1. A device for detecting a shadow region in an image comprising:

an imaging module configured to generate a first multi-channel image including a brightness channel, a red channel, a green channel, and a blue channel;

a brightness correcting module configured to correct values of the brightness channel of the first multi-channel image based on imaging parameters utilized in the imaging module, and to output the corrected first multi-channel image as a second multi-channel image;

a scene classifying module configured to determine whether a shadow detection is to be carried out on the second multi-channel image based on a characteristic value computed based on the second multi-channel image;

a shadow detecting module configured to classify each pixel of the second multi-channel image on which it is determined that the shadow detection is to be carried out by the scene classifying module into a shadow pixel or a non-shadow pixel, and to generate a first shadow classification mark matrix in which each pixel has a shadow classification mark value corresponding to the classification of the shadow and non-shadow pixel;

a region segmentation module configured to segment the first multi-channel image generated by the imaging module into a plurality of regions each having a set of pixels having similar color values, and to generate a region mark matrix in which each pixel has a region mark value corresponding to the region and the pixels belonging to the same region have the same region mark value; and

a post-processing module configured to update the first shadow classification mark matrix based on the first shadow classification mark matrix generated from the shadow detecting module and the region mark matrix output from the region segmentation module, and to output the updated first shadow classification mark matrix as a second shadow classification mark matrix.

2. The device as claimed in claim 1, wherein the second multi-channel image is obtained by multiplying the values of the brightness channel of the first multi-channel image by a predetermined coefficient which is determined based on exposure time, numerical aperture, and sensitivity employed by the imaging module when the imaging module has taken the first multi-channel image.

3. The device as claimed in claim 1, wherein the scene classifying module classifies the second multi-channel image into a first type image including a sunny day image and a second type image including a cloudy day image based on the characteristic value computed based on the second multi-channel image, and determines that the shadow detection is to be carried out on the second multi-channel image when the second multi-channel image is classified into the first type image, and that the shadow detection is not to be carried out on the second multi-channel image when the second multi-channel image is classified into the second type image.

4. The device as claimed in claim 1, wherein in the scene classifying module, the characteristic value computed based on the second multi-channel image includes a mean and a standard deviation of values of the brightness channel of the second multi-channel image, and a standard deviation of ratios of values of the blue channel and values of the red channel of the second multi-channel image.

5. The device as claimed in claim 1, wherein the scene classifying module includes:

a first input module configured to input the second multi-channel image;

a first characteristic computation module configured to compute a mean and a standard deviation of values of the brightness channel of the second multi-channel image, and to output the mean and the standard deviation of the values of the brightness channel of the second multi-channel image as a first characteristic and a second characteristic, respectively;

a BR matrix computation module configured to compute a ratio of a value of the blue channel and a value of the red channel for the pixels of the second multi-channel image to obtain a BR matrix;

a second characteristic computation module configured to compute a standard deviation of the BR matrix, and to output the standard deviation of the BR matrix as a third characteristic; and

a confidence level computation module configured to input the first, the second, and the third characteristics to a scene classifier to obtain the confidence level, and, if the computed confidence level exceeds a predetermined threshold, to transmit the second multi-channel image to the shadow detecting module so as to perform the classification on the second multi-channel image, and, if the computed confidence level is equal to or lower than a predetermined threshold, not to transmit the second multi-channel image to the shadow detecting module so as not to perform the classification.

6. The device as claimed in claim 1, wherein the shadow detecting module includes:

a second input module configured to input the second multi-channel image;

a third characteristic computation module configured to compute a unique vector of each of the pixels of the second multi-channel image; and

a shadow classification mark matrix computation module configured to compute whether each pixel of the second multi-channel image is a shadow class or a non-shadow class based on the unique vector computed by the third characteristic computation module and a shadow classifier, and to generate the first shadow classification mark matrix in which the pixel classified as the shadow class has a first mark value and the pixel classified as the non-shadow class has a second mark value.

7. The device as claimed in claim 6, wherein the unique vector includes values of the brightness channel of the second multi-channel image, a ratio of a value of the blue channel and a value of the red channel, and a value of saturation of the second multi-channel image.

8. The device as claimed in claim 1, wherein the region segmentation module segments the first multi-channel image into the plurality of regions based on the values of the brightness channel of the first multi-channel image.

9. The device as claimed in claim 1, wherein the post-processing module includes:

a third input module configured to input the first shadow classification mark matrix generated by the shadow detecting module;

a fourth input module configured to input the region mark matrix generated by the region segmentation module; and

a shadow classification mark update module configured to compute a number of elements having a first mark value and a number of elements having a second mark value in a corresponding region of the first shadow classification mark matrix with respect to the respective regions of the region mark matrix, to update the all elements in the corresponding region of the first shadow classification mark matrix with either one of the first mark value and the second mark value based on the obtained number of the elements having the first mark value and the obtained number of the elements having the second mark value, and to output the updated first shadow classification mark matrix as the second shadow classification mark matrix.

10. A method for detecting a shadow region in an image comprising the steps of:

generating a first multi-channel image including a brightness channel, a red channel, a green channel, and a blue channel;

correcting values of the brightness channel of the first multi-channel image based on imaging parameters utilized in the generating step, and outputting the corrected first multi-channel image as a second multi-channel image;

determining whether a shadow detection is to be carried out on the second multi-channel image based on a characteristic value computed based on the second multi-channel image;

classifying each pixel of the second multi-channel image on which it is determined that the shadow detection is to be carried out in the determining step into a shadow pixel or a non-shadow pixel, and generating a first shadow classification mark matrix in which each pixel has a shadow classification mark value corresponding to the classification of the shadow and non-shadow pixel;

segmenting the first multi-channel image generated in the generating step into a plurality of regions each having a set of pixels having similar color values, and generating a region mark matrix in which each pixel has a region mark value corresponding to the region and the pixels belonging to the same region have the same region mark value; and

updating the first shadow classification mark matrix based on the first shadow classification mark matrix generated in the segmenting step, and outputting the updated shadow classification mark as a second shadow classification mark matrix.

11. The method as claimed in claim 10, wherein the second multi-channel image is obtained by multiplying the values of the brightness channel of the first multi-channel image by a predetermined coefficient which is determined based on exposure time, numerical aperture, and sensitivity employed in the generating step when the first multi-channel image is imaged.

12. The method as claimed in claim 10, wherein in the determining step, the second multi-channel image is classified into a first type image including a sunny day image and a second type image including a cloudy day image based on the characteristic value computed based on the second multi-channel image, and the shadow detection is determined to be carried out on the second multi-channel image when the second multi-channel image is classified into the first type image, and the shadow detection is determined not to be carried out on the second multi-channel image when the second multi-channel image is classified into the second type image.

13. The method as claimed in claim 10, wherein in the determining step, the characteristic value computed based on the second multi-channel image includes a mean and a standard deviation of values of the brightness channel of the second multi-channel image, and a standard deviation of ratios of values of the blue channel and values of the red channel of the second multi-channel image.

14. The method as claimed in claim 10, wherein the determining step includes:

inputting the second multi-channel image;

computing a mean and a standard deviation of values of the brightness channel of the second multi-channel image, and outputting the mean and standard deviation of the values of the brightness channel of the second multi-channel image as a first characteristic and a second characteristic, respectively;

computing a ratio of a value of the blue channel and a value of the red channel for the pixels of the second multi-channel image to obtain a BR matrix;

computing a standard deviation of the BR matrix, and outputting the standard deviation of the BR matrix as a third characteristic; and

inputting the first, the second, and the third characteristics to a scene classifier to obtain the confidence level, and, if the computed confidence level exceeds a predetermined threshold, transmitting the second multi-channel image to a shadow detecting module so as to perform the classification on the second multi-channel image, and, if the computed confidence level is equal to or lower than a predetermined threshold, not transmitting the second multi-channel image so as not to perform the classification.

15. The method as claimed in claim 10, wherein the classifying step includes:

inputting the second multi-channel image;

computing a unique vector of each of the pixels of the second multi-channel image; and

computing whether each of the pixels of the second multi-channel image is a shadow class or a non-shadow class based on the unique vector computed in the computing the unique vector step and a shadow classifier, and generating the first shadow classification mark matrix in which the pixel classified as the shadow class has a first mark value and the pixel classified as the non-shadow class has a second mark value.

16. The method as claimed in claim 15, wherein the unique vector includes values of the brightness channel of the second multi-channel image, a ratio of a value of the blue channel and a value of the red channel, and a value of saturation of the second multi-channel image.

17. The method as claimed in claim 10, wherein in the segmenting step, the first multi-channel image is segmented into the plurality of regions based on the values of the brightness channel of the first multi-channel image.

18. The method as claimed in claim 10, wherein the updating step includes:

inputting the first shadow classification mark matrix generated in the classifying step;

inputting the region mark matrix generated in the segmenting step; and

computing a number of elements having a first mark value and a number of elements having a second mark value in a corresponding region of the first shadow classification mark matrix with respect to the respective regions of the region mark matrix, updating the all elements in the corresponding region of the first shadow classification mark matrix with either one of the first mark value and the second mark value based on the obtained number of the elements having the first mark value and the number of the elements having the second mark value, and outputting the updated first shadow classification mark matrix as a second shadow classification mark matrix.