Method and apparatus for single channel color image segmentation using local context based adaptive weighting

• US 6,449,389 B1
• Filed: 09/24/1999
• Issued: 09/10/2002
• Est. Priority Date: 09/24/1999
• Status: Expired due to Fees

• Current Assignee: Xerox Corporation (Xerox Holdings Corp.)
• Original Assignee: Xerox Corporation (Xerox Holdings Corp.)
• Inventors: Schweid, Stuart A.
• Primary Examiner(s): Au, Amelia M.
• Assistant Examiner(s): LAROSE, COLIN M
• Application Number:
  US09/405,927
• Time in Patent Office: 1,082 Days
• Field of Search: 382/164, 382/167, 382/173, 382/174, 382/162, 382/199, 345/593, 345/603, 345/600, 345/604, 358/515, 358/518, 348/598, 348/655, 348/676, 348/711, 348/600
• US Class Current: 382/164
• CPC Class Codes: G06T 2207/10016 Video; Image sequence
• CPC Class Codes: G06T 2207/10024 Color image
• CPC Class Codes: G06T 2207/20012 Locally adaptive
• CPC Class Codes: G06T 5/20 using local operators
• CPC Class Codes: G06T 5/94 based on local image properties, e.g. for local contrast enhancement
• CPC Class Codes: G06T 7/11 Region-based segmentation

Abstract

A method and apparatus for single channel color image segmentation using local context based adaptive weighting is provided. The varying weightings of the projection vector are determined as a function of local input image activity context. A Sobel operator is used to calculate the input image activity. A binary map is created for each color channel and is adapted to store binary markers indicative of local activity levels on a per pixel basis. The binary maps are low pass filtered and then normalized to generate a context based adaptive weighting vector for use in single color segmentation of a multi-channel color image signal.

Claims

1. A method of generating a context based adaptive weighting vector for use in single color segmentation of a color image formed of first color channel pixels R(i,k), second color channel pixels G(i,k), and third color channel pixels B(i,k), the method comprising the steps of:

obtaining a first color channel activity estimate R_s(i,k) representation of a measure of local first color channel video signal variation at each first color channel pixel R(i,k) of the color image;

obtaining a second color channel activity estimate G_s(i,k) representation of a measure of local second color channel video signal variation at each second color channel pixel G(i,k) of the color image;

obtaining a third color channel activity estimate B_s(i,k) representation of a measure of local third color channel video signal variation at each third color channel pixel B(i,k) of the color image;

for each pixel (i,k) of the color image, comparing the first color channel activity estimate R_s(i,k) with the second color channel activity estimate G_s(i,k) and with the third color channel activity estimate B_s(i,k) to identify a one of the first, second, and third color channels having greatest activity;

generating a first color channel binary map R_sb(i,k) by storing, for each pixel (i,k) of the color image, a first binary value for pixel locations where the first color channel had said greatest activity and a second binary value for pixel locations where the first color channel did not have said greatest activity;

generating a second color channel binary map G_sb(i,k) by storing, for each pixel (i,k) of the color image, the first binary value for pixel locations where the second color channel had said greatest activity and the second binary value for pixel locations where the second color channel did not have said greatest activity;

generating a third color channel binary map B_sb(i,k) by storing, for each pixel (i,k) of the color image, the first binary value for pixel locations where the third color channel had said greatest activity and the second binary value for pixel locations where the third color channel did not have said greatest activity;

low pass filtering the first color channel binary map R_sb(i,k) to generate a first color low pass filtered binary map R_sb1(i,k);

low pass filtering the second color channel binary map G_sb(i,k) to generate a second color low pass filtered binary map G_sb1(i,k);

low pass filtering the third color channel binary map B_sb(i,k) to generate a third color low pass filtered binary map B_sb1(i,k); and

,

generating an adaptive weighting vector w(i,k) by combining said first, second, and third color low pass filtered binary maps as w(i,k)=[R_sb1(i,k) G_sb1(i,k) B_sb1(i,k)]′

.

2. The method of generating a context based adaptive weighting vector according to claim 1 wherein:

the step of obtaining said first color channel activity estimate R_s(i,k) includes obtaining a red channel activity estimate R_s(i,k) representation of a measure of local red channel video signal variation at each red channel pixel R(i,k) of the color image;

the step of obtaining said second color channel activity estimate G_s(i,k) includes obtaining a green channel activity estimate G_s(i,k) representation of a measure of local green channel video signal variation at each green channel pixel G(i,k) of the color image;

the step of obtaining said third color channel activity estimate B_s(i,k) includes to obtaining a blue channel activity estimate B_s(i,k) representation of a measure of local blue channel video signal variation at each blue channel pixel B(i,k) of the color image;

the step of comparing the first color channel activity estimate with the second color channel activity estimate and with the third color channel activity estimate includes comparing said red channel activity estimate R_s(i,k) with said green channel activity estimate G_s(i,k) and with said blue channel activity estimate B_s(i,k) to identify a one of the red, green, and blue channels having greatest activity;

the step of generating said first color channel binary map R_sb(i,k) includes generating a red channel binary map R_sb(i,k) for storing, for each pixel (i,k) of the color image, said first binary value for pixel locations where said red channel had said greatest activity and said second binary value for pixel locations where said red channel did not have said greatest activity;

the step of generating said second color channel binary map G_sb(i,k) includes generating a green channel binary map G_sb(i,k) for storing, for each pixel (i,k) of the color image, said first binary value for pixel locations where said green channel had said greatest activity and said second binary value for pixel locations where said green channel did not have said greatest activity;

the step of generating said third color channel binary map B_sb(i,k) includes generating a blue channel binary map B_sb(i,k) for storing, for each pixel (i,k) of the color image, said first binary value for pixel locations where said blue channel had said greatest activity and said second binary value for pixel locations where said blue channel did not have said greatest activity;

the step of low pass filtering said first color channel binary map includes low pass filtering said red channel binary map R_sb(i,k) to generate a red low pass filtered binary map R_sb1(i,k);

the step of low pass filtering said second color channel binary map includes low pass filtering said green channel binary map G_sb(i,k) to generate a green low pass filtered binary map G_sb1(i,k);

the step of low pass filtering said third color channel binary map includes low pass filtering said blue channel binary map B_sb(i,k) to generate a blue low pass filtered binary map B_sb1(i,k); and

,

the step of generating said adaptive weighting vector w(i,k) includes generating said adaptive weighting vector w(i,k) by combining said red, green, and blue low pass filtered binary maps as $w<br><br>\left(i,k\right)=\frac{{\left[R_{\mathrm{sb1}}<br><br>\left(i,k\right)<br><br>\text{\hspace{1em}<br><br>}<br><br>G_{\mathrm{sb1}}<br><br>\left(i,k\right)<br><br>\text{\hspace{1em}<br><br>}<br><br>B_{\mathrm{sb1}}<br><br>\left(i,k\right)\right]}^{\mathrm{\prime <br><br>}}}{k}.$

3. The method of generating a context based adaptive weighting vector according to claim 2 wherein:

the step of obtaining said red channel activity estimate R_s(i,k) includes the step of applying a Sobel filter to each of said red channel pixels R(i,k);

the step of obtaining said green channel activity estimate G_s(i,k) includes the step of applying a Sobel filter to each of said green channel pixels G(i,k); and

,

the step of obtaining said blue channel activity estimate B_s(i,k) includes the step of applying a Sobel filter to each of said blue channel pixels B(i,k).

4. The method of generating a context based adaptive weighting vector according to claim 3 wherein:

the step of applying said Sobel filter to each of said red channel pixels R(i,k) includes applying said Sobel filter in both vertical and horizontal red channel image directions;

the step of applying said Sobel filter to each of said green channel pixels G(i,k) includes applying said Sobel filter in both vertical and horizontal green channel image directions; and

,

the step of applying said Sobel filter to each of said blue channel pixels B(i,k) includes applying said Sobel filter in both vertical and horizontal blue channel image directions.

5. The method of generating a context based adaptive weighting vector according to claim 4 wherein:

the step of applying the Sobel filter in both said vertical and horizontal red image directions includes calculating a norm {square root over ((S_x²+L +S_y²+L ))} of vertical and horizontal image direction components as said red channel activity estimate R_s(i,k);

the step of applying the Sobel filter in both said vertical and horizontal green channel image directions includes calculating a norm {square root over ((S_x²+L +S_y²+L ))} of vertical and horizontal image direction components as said green channel activity estimate G_s(i,k); and

, the step of applying the Sobel filter in both said vertical and horizontal blue channel image directions includes calculating a norm {square root over ((S_x²+L +S_y²+L ))} of vertical and horizontal image direction components as said blue channel activity estimate B_s(i,k).

6. The method of generating a context based adaptive weighting vector according to claim 2 wherein:

the step of generating said red channel binary map R_sb(i,k) includes storing, for each pixel (i,k) of the RGB color image, a logical “

1”

as said first binary value where the red channel had said greatest activity and storing a logical “

0”

as said second binary value for pixel locations where the red channel did not have said greatest activity;

the step of generating said green channel binary map G_sb(i,k) includes storing, for each pixel (i,k) of the RGB color image, a logical “

1”

as said first binary value where the green channel had said greatest activity and storing a logical “

0”

as said second binary value for pixel locations where the green channel did not have said greatest activity; and

,

the step of generating said blue channel binary map B_sb(i,k) includes storing, for each pixel (i,k) of the RGB color image, a logical “

1”

as said first binary value where the blue channel had said greatest activity and storing a logical “

0”

as said second binary value for pixel locations where the blue channel did not have said greatest activity.

7. The method of generating a context base adaptive weighting vector according to claim 2 wherein the step of comparing the red channel activity estimate R_s(i,k) with the green channel activity estimate G_s(i,k) and with the blue channel activity estimate B_s(i,k) includes comparing the red channel activity estimate R_s(i,k) with the green channel activity estimate G_s(i,k) and with the blue channel activity estimate B_s(i,k) according to:

if (G_s≧

R_s&

&

G_s≧

B_s)

G_sb=1

else if (R_s≧

B_s)

R_sb=1

else

B_sb=1.

8. The method of generating a context based adaptive weighting vector according to claim 2 wherein:

the step of low pass filtering the red channel binary map R_sb(i,k) includes low pass filtering the red channel binary map R_sb(i,k) using a first M×

N low pass pyramidal filter;

the step of low pass filtering the green channel binary map G_sb(i,k) includes low pass filtering the green channel binary map G_sb(i,k) using said first M×

N low pass pyramidal filter; and

,

the step of low pass filtering the blue channel binary map B_sb(i,k) includes low pass filtering the blue channel binary map B_sb(i,k) using said first M×

N low pass pyramidal filter.

9. The method of generating a context based adaptive weighting vector according to claim 8 wherein the step of low pass filtering the red, green, and blue channel binary maps includes normalizing the red, green, and blue channel binary maps to a unitary sum at each pixel according to:

10. The method of generating a context based adaptive weighting vector according to claim 9 wherein the step of generating said adaptive weighting vector w(i,k) includes the step of combining said red, green, and blue low pass filter binary maps as:

11. A method of generating a context based adapted weighting vector for use in single channel segmentation of an image formed of first channel pixels A(i,k) and second channel pixels B(i,k) the method comprising the steps of:

obtaining activity estimate representations of a measure of local channel signal variation at each first channel pixel as A_s(i,k) and at each second channel pixel as B_s(i,k) of the image;

for each pixel (i,k) of the image, comparing said activity estimate representations to identify a one of a first and second channels having greatest activity;

generating a first channel binary map A_sb(i,k) and a second channel binary map B_sb(i,k) by storing in each first and second binary maps, for each pixel (i,k) of the color image, a first binary value for pixel locations where the first and second channels had said greatest activity and a second binary value for pixel locations where the channels did not have said greatest activity;

filtering the first channel binary map A_sb(i,k) and the second channel binary map B_sb(i,k) to generate a first filtered binary map A_sb1(i,k) and a second filtered binary map B_b1(i,k); and

,

generating an adaptive weighting vector w(i,k) by combining said first and second filtered binary maps as $w<br><br>\left(i,k\right)=\frac{{\left[A_{\mathrm{sb1}}<br><br>\left(i,k\right)<br><br>\text{\hspace{1em}<br><br>}<br><br>B_{\mathrm{sb1}}<br><br>\left(i,k\right)\right]}^{\mathrm{\prime <br><br>}}}{k}.$

12. The method of generating a context based adaptive weighting vector according to claim 11 wherein the step of obtaining said first channel activity estimate A_s(i,k) and said second channel activity estimate B_s(i,k) includes the step of applying a Sobel filter to each of said first channel pixels A(i,k) and said second channel pixels B(i,k) in both vertical and horizontal channel directions to calculate a norm at each pixel in each said first and second channels.

13. The method of generating a context based adaptive weighting vector according to claim 11 wherein the step of filtering said first channel binary map A_sb(i,k) and said second channel binary map B_sb(i,k) includes the step of low pass filtering the first and second channel binary maps using a low pass M×

N pyramidal filter.

14. The method of generating a context based adaptive weighting vector according to claim 13 wherein the step of low pass filtering the first and second channel binary maps includes normalizing the first and second channel binary maps to a unity sum at each pixel according to:

15. The method generating a context based adaptive weighting vector according to claim 14 wherein the step of generating said adaptive weighting vector w(i,k) includes the step of combining said first and second low pass filter binary maps as:

16. An apparatus of generating a context based adapted weighting vector for use in single channel segmentation of an image formed of first channel pixels A(i,k) and second channel pixels B(i,k), the apparatus comprising:

an activity estimator circuit for obtaining activity estimate representations of a measure of local channel signal variation at each first channel pixel as A_s(i,k) and at each second channel pixel as B_s(i,k) of the image;

a comparator circuit for comparing, for each pixel (i,k) of the image, comparing said activity estimate representations to identify a one of a first and second channels having greatest activity and generating a first channel binary map A_sb(i,k) and a second channel binary map B_sb(i,k) by storing in each first and second binary maps, for each pixel (i,k) of the color image, a first binary value for pixel locations where the first and second channels had said greatest activity and a second binary value for pixel locations where the channels did not have said greatest activity;

a filter circuit for filtering the first channel binary map A_sb(i,k) and the second channel binary map B_sb(i,k) to generate a first filtered binary map A_sb1(i,k) and a second filtered binary map B_b1(i,k); and

,

a normalizer circuit for generating an adaptive weighting vector w(i,k) by combining said first and second filtered binary maps as $w<br><br>\left(i,k\right)=\frac{{\left[A_{\mathrm{sb1}}<br><br>\left(i,k\right)<br><br>\text{\hspace{1em}<br><br>}<br><br>B_{\mathrm{sb1}}<br><br>\left(i,k\right)\right]}^{\mathrm{\prime <br><br>}}}{k}.$

17. The apparatus according to claim 16 wherein the activity estimator circuit includes a circuit for applying a Sobel filter to each of said first channel pixel A(i,k) and said second channel pixels B(i,k) in both vertical and horizontal first and second channel directions to calculate a norm at each pixel in each said first and second channels.

18. The apparatus according to claim 16 wherein the filter circuit includes a circuit for low pass filtering the first and second channel binary maps using a first low pass M×

N pyramidal filter.

19. The apparatus according to claim 18 wherein the normalizer circuit includes a circuit for normalizing the first and second channel binary maps to a unity sum at each pixel according to:

20. The apparatus according to claim 19 wherein the normalizer circuit includes a circuit for combining said first and second low pass filter binary maps as: