Digital image processing system and method for emphasizing a main subject of an image

• US 7,212,668 B1
• Filed: 08/18/2000
• Issued: 05/01/2007
• Est. Priority Date: 08/18/2000
• Status: Expired due to Term

• Current Assignee: Monument Peak Ventures, LLC (Dominion Harbor Enterprises, LLC)
• Original Assignee: Eastman Kodak Company
• Inventors: Luo, Jiebo, Gray, Robert T.
• Primary Examiner(s): SHERALI, ISHRAT I
• Application Number:
  US09/642,533
• Time in Patent Office: 2,447 Days
• Field of Search: 382162-167, 382173-180, 382224-228, 382/203, 382282-283, 382/190, 358453-465, 358515-527, 345589-604, 348699-703
• US Class Current: 382/165
• CPC Class Codes: G06T 11/001 Texturing; Colouring; Generation of texture or colour inpainting G06T5/77
• CPC Class Codes: G06T 11/60 Editing figures and text; Combining figures or text
• CPC Class Codes: G06T 2207/30201 Face
• CPC Class Codes: G06T 7/11 Region-based segmentation

Abstract

A system and method for automatically altering an image having pixels is disclosed. An image is segmented into regions and various features of the regions are extracted and used to calculate a confidence factor, or belief value, that a particular region is a main subject of the image. The belief values are assembled into a belief map that is used to define how pixel values are to be altered. Various color and appearance values of the pixel are altered in a predetermined way to emphasize the main subject of the image. The emphasis is applied in a binary fashion or as in a gradient of values. Either the main subject can be emphasized, or the secondary or background subjects can be de-emphasized. The system can be implement on a variety of computer systems.

Claims

1. A computer method for modifying an image having a main subject and a background pixels, comprising the steps of:

automatically identifying the main subject of the image, and

automatically altering pixel values of said image to emphasize said main subject, said altering following said identifying;

said altering follows any and all identifying of said main subject and wherein said identifying further comprises;

segmenting said image into a plurality of regions; and

generating a plurality of belief values, each said belief value being associated with one of a plurality of regions of the image, said belief values each being related to the probability that the associated region is a main subject of the image, to provide a main subject belief map.

2. The method of claim 1 wherein said altering pixel values further comprises altering pixel color saturation.

3. The method of claim 1 wherein said altering pixel values further comprises altering pixel hue.

4. The method of claim 1 wherein said altering pixel values further comprises altering pixel luminescence.

5. The method of claim 1 wherein said altering pixel values further comprises altering pixel blur.

6. The method of claim 1 wherein said altering emphasizes said main subject by altering pixel values that are a part of said main subject.

7. The method of claim 1 wherein said altering emphasizes said main subject by altering pixel values that are not a part of said main subject.

8. The method of claim 7 wherein said altering further comprises:

calculating the luminance values for the pixels that are not a part of said main subject, and

replacing the color values of the pixels that are not a part of said main subject with respective luminance values.

9. The method of claim 1 wherein said altering further comprises enhancing saturation of the pixels that are a part of said main subject.

10. The method of claim 1 wherein said altering further comprises inverting the pixel hue values of the pixels that are a part of said main subject.

11. The method of claim 1 wherein said altering further comprises inverting the pixel hue values of the pixels that are not a part of said main subject.

12. The method of claim 1 wherein said identifying further comprises the steps of:

segmenting the image into a plurality of regions based on uniform image characteristics;

calculating a level of saliency for said plurality of regions, and

assigning a believe value to the pixels corresponding to said level of saliency.

19. The method of claim 1 wherein said main subject includes skin tone pixels.

20. The method of claim 1 wherein said segmenting further comprises extracting features of said input digital image and said generating further comprises using a reasoning engine to compute said belief map from said features.

21. The method of claim 20 wherein said reasoning engine is Bayes net-based.

22. The method of claim 20 wherein said features include semantic features and structural features, said semantic features including skin, said structural features including centrality.

13. A system for modifying an image having a main subject and a background pixels, comprising the steps of:

means for automatically identifying the main subject of the image, and

means for automatically altering pixel values of said image to emphasize said main subject, said altering following said identifying;

said altering following any and all identifying of said main subject and wherein said means for identifying further comprises;

segmenting said image into a plurality of regions; and

generating a plurality of belief values, each said belief value being associated with one of a plurality of regions of the image, said belief values each being related to the probability that the associated region is a main subject of the image, to provide a main subject belief map.

14. A computer method of modifying an image having a main subject and a background pixels, comprising the steps of:

automatically generating one or more belief values, each said belief value being associated with one of a plurality of regions of the image, said belief values each being related to the probability that the associated region is the main subject of the image;

following said generating, automatically altering pixel values in said plurality of regions of said image, in accordance with said associated belief values;

and said altering follows all said generating.

15. The method of claim 14 wherein said altering further comprises:

determining a saturation value for each pixel; and

altering the pixel saturation values according to said associated belief values.

16. The method of claim 14 wherein said altering further comprises:

determining a luminance value for each pixel; and

altering the pixel luminance values according to said associated belief values.

17. The method of claim 14 wherein said altering further comprises:

determining a hue value for each pixel, and

altering the pixel hue values according to said associated belief values.

18. The method of claim 14 wherein said altering further comprises:

determining a blur value for each pixel, and

altering the pixel blur values according to said associated belief values.

23. A digital image modification method for use on an input digital image having a main subject and a background pixels to produce an output image, the method comprising the steps of:

receiving a user selection of one of a plurality of different types of emphasizing to provide a selected emphasis;

identifying the main subject and the background in the input digital image;

altering pixel values of said input digital image, in accordance with said selected emphasis, to provide the output image wherein said main subject is emphasized;

wherein said identifying and altering steps are without manual intervention and wherein said identifying further comprises;

segmenting said image into a plurality of regions; and

generating a plurality of belief values, each said belief value being associated with one of a plurality of regions of the image, said belief values each being related to the probability that the associated region is a main subject of the image, to provide a main subject belief map.

24. The method of claim 23 further comprising delivering said output image to an output device.

25. The method of claim 24 wherein said output device is a printer or a display.

26. The method of claim 23 wherein said main subject includes skin tone pixels.

27. The method of claim 23 wherein said types of emphasizing include one or more of:

altering pixel color saturation, altering pixel hue, altering pixel luminescence, and altering pixel blur of one of said main subject and said background.

28. The method of claim 27 wherein said emphasizing alters said main subject.

29. The method of claim 27 wherein said emphasizing alters said background.

30. A digital image modification method for use on an input digital image having a main subject and a background pixels in a computer system to produce an output image, the method comprising the steps of:

segmenting the input digital image into a plurality of regions, said regions each being homogeneous;

computing the main subject belief map of said regions, said map defining a main subject in said input digital image;

then, altering pixel values in said input digital image, in accordance with said map, to provide the output image wherein said main subject is emphasized; and

next, delivering said output image to an output device;

wherein said steps are without manual intervention and wherein said computing further comprises;

generating a plurality of belief values, each said belief value being associated with one of a plurality of regions of the image, said belief values each being related to the probability that the associated region is a main subject of the image, to provide a main subject belief map.

31. The method of claim 30 wherein said output device is a printer or a display.

32. The method of claim 30 wherein said computing of said map is based upon degree of centrality, borderness, and chrominance space distribution.

33. The method of claim 30 wherein said altering further comprises modifying pixel values of said input digital image exclusive of said main subject.

34. The method of claim 30 wherein said main subject includes skin tone pixels.

35. The method of claim 30 wherein said computing further comprises:

calculating a level of saliency of each of said regions; and

assigning a belief value to each of said pixels corresponding to the level of saliency of the respective said region.

36. The method of claim 35 wherein said calculating further comprises considering a plurality of semantic features including skin regions and a plurality of structural features including centrality.

37. The method of claim 36 wherein said main subject has a high degree of said centrality.

38. The method of claim 36 wherein said main subject includes skin tone pixels.