Method for automatic text placement in digital images

• US 6,711,291 B1
• Filed: 09/17/1999
• Issued: 03/23/2004
• Est. Priority Date: 09/17/1999
• Status: Expired due to Term

• Current Assignee: Monument Peak Ventures, LLC (Dominion Harbor Enterprises, LLC)
• Original Assignee: Eastman Kodak Company
• Inventors: Stubler, Peter O., Mehrotra, Rajiv
• Primary Examiner(s): Dastouri, Mehrdad
• Application Number:
  US09/398,221
• Time in Patent Office: 1,649 Days
• Field of Search: 382/195, 382/199, 382/203, 382/205, 382/180, 382/176, 382/274, 345/581, 345/582, 345/619, 345/626, 345/629, 345/630, 345/632, 345/641, 358/450, 358/452, 358/453, 358/1.18, 358/464, 348/584, 348/586
• US Class Current: 382/195
• CPC Class Codes: G06T 11/60 Editing figures and text; Combining figures or text
• CPC Class Codes: G06T 7/70 Determining position or orientation of objects or cameras camera calibration G06T7/80
• CPC Class Codes: G06V 10/25 Determination of region of interest [ROI] or a volume of interest [VOI]

Abstract

A method for placing a human understandable item, such as a caption, text or a figurative element, in a digital image includes the steps of (a) digitally processing the digital image to recognize and identify an optimal location in the digital image for placing the item and (b) modifying the placement of the item in relation to human understandable image content so as to minimally obscure such human understandable image content in the digital image. The recognition and identification of an optimal location further includes the identification of an optimal open space region and then finding an optimal location for the item within the open space region. The placement of the item may be done so as to minimally obscure human understandable image content either in other locations of the optimal open space region, or in regions of the digital image adjoining the optimal open space region. Placement of the image may further be based on evaluating the proportion of image pixels in the item that occlude pixels outside of either the optimal location or the optimal region. Moreover, one or more attributes of the item, such as the color, size or shape of the item, may be modified in relation to the human understandable image content in the digital image to enhance the prominence of the item in the digital image.

Claims

1. A method for placing a human understandable item in a digital image, said method comprising the steps of:

(a) automatically analyzing the pixels of the digital image to recognize and identify pixel content indicating a plurality of candidate regions for placement of the human understandable item;

(b) processing the plurality of candidate regions to exclude one or more regions based upon characteristics including at least one of size, shape and color which are indicative of image content that makes the region unsuitable as a candidate region for item placement, the remaining of said candidate regions comprising contiguous regions in the digital image that do not contain any significant or valuable human understandable image content and are sufficiently large to contain a substantial portion of the human understandable item;

(c) automatically selecting one of the candidate regions as an optimal location in the digital image for placing the human understandable item;

(d) modifying the placement of the human understandable item in relation to human understandable image content in the digital image so as to minimally obscure the human understandable image content in the digital image; and

(e) rendering the digital image with the human understandable item overlaid substantially into the optimal location such that the human understandable image content is spatially unaffected by the overlaid human understandable item.

2. A method as claimed in claim 1 in which step (c) comprises identifying an optimal region comprising open space in the digital image and then finding an optimal location within the optimal region.

3. A method as claimed in claim 2 in which step (d) comprises modifying the placement of the human understandable item in relation to human understandable image content so as to minimally obscure such human understandable image content in other locations of the optimal region.

4. A method as claimed in claim 1 in which step (d) comprises modifying the placement of the human understandable item in relation to human understandable image content so as to minimally obscure such human understandable image content in a region of the digital image adjoining the optimal region.

5. A method as claimed in claim 1 in which step (d) comprises evaluating the proportion of image pixels in the human understandable item that occlude pixels outside of either the optimal location or the optimal region.

6. A method as claimed in claim 1 in which the human understandable items include at least one of a caption, text and a figurative element.

7. A method as claimed in claim 1 further comprising a step (f) of modifying one or more attributes of the human understandable item in relation to the human understandable image content in the digital image to enhance the prominence of the human understandable item in the digital image.

8. A method as claimed in claim 7 in which step (f) comprises modifying one or more attributes of the human understandable item in relation to the human understandable image content in the digital image outside of the optimal location.

9. A method as claimed in claim 7 in which the step (f) of altering one or more attributes of the human understandable item comprises altering at least one of the color, size and shape of the human understandable item.

10. A method for placing a human understandable item in a digital image so as enhance the prominence of the human understandable item in relation to human understandable image content in the digital image, said method comprising the steps of:

(a) automatically analyzing the pixels of the digital image to recognize and identify pixel content indicating a plurality of candidate regions for placement of the human understandable item;

(b) processing the plurality of candidate regions to exclude one or more regions based upon characteristics including at least one of size, shape and color which are indicative of image content that makes the region unsuitable as a candidate region for item placement, the remaining of said candidate regions comprising contiguous regions in the digital image that do not contain any significant or valuable human understandable image content and are sufficiently large to contain a substantial portion of the human understandable item;

(c) automatically selecting one of the candidate regions as an optimal location in the digital image for placing the human understandable item;

(d) modifying one or more attributes of the human understandable item in relation to the human understandable image content in the digital image to enhance the prominence of the human understandable item in the digital image; and

(e) rendering the digital image with the human understandable item overlaid substantially into the optimal location such that the human understandable image content is spatially unaffected by the overlaid human understandable item.

11. A method as claimed in claim 10 in which the step (c) comprises identifying an optimal region comprising open space in the digital image and then finding an optimal location within the optimal region.

12. A method as claimed in claim 10 in which step (d) comprises modifying one or more attributes of the human understandable item in relation to the human understandable image content in the digital image outside of the optimal location.

13. A method as claimed in claim 10 in which step (d) comprises modifying at least one of the color, size and shape of the human understandable item to contrast with the human understandable image content.

14. A computer program product for placing a human understandable item in a digital image comprising:

a computer readable storage medium having a computer program stored thereon for performing the steps of;

(a) automatically analyzing the pixels of the digital image to recognize and identify pixel content indicating a plurality of candidate regions for placement of the human understandable item;

(b) processing the plurality of candidate regions to exclude one or more regions based upon a color characteristic which is indicative of a flesh colored image content that makes the region unsuitable as a candidate region for item placement, the remaining of said candidate regions comprising contiguous regions in the digital image that do not contain any significant or valuable human understandable image content and are sufficiently large to contain a substantial portion of the human understandable item;

(c) automatically selecting one of the candidate regions as an optimal location in the digital image for placing the human understandable item;

(d) modifying the placement of the human understandable item in relation to the human understandable image content in the digital image so as to minimally obscure the human understandable image content in the digital image; and

(e) rendering the digital image with the human understandable item overlaid substantially into the optimal location such that the human understandable image content is spatially unaffected by the overlaid human understandable item.

15. A computer program product as claimed in claim 14 in which step (c) comprises identifying an optimal region comprising open space in the digital image and then finding an optimal location within the optimal region.

16. A computer program product as claimed in claim 15 in which step (d) comprises modifying the placement of the human understandable item in relation to human understandable image content so as to minimally obscure such human understandable image content in other locations of the optimal region.

17. A computer program product as claimed in claim 14 in which step (d) comprises modifying the placement of the human understandable item in relation to human understandable image content so as to minimally obscure such human understandable image content in a region of the digital image adjoining the optimal region.

18. A computer program product as claimed in claim 14 in which step (d) comprises evaluating the proportion of image pixels in the human understandable item that occlude pixels outside of either the optimal location or the optimal region.

19. A computer program product as claimed in claim 14 further comprising a step (f) of modifying one or more attributes of the human understandable item in relation to the human understandable image content in the digital image to enhance the prominence of the human understandable item in the digital image.

20. A computer program product as claimed in claim 19 in which step (f) comprises modifying one or more attributes of the human understandable item in relation to the human understandable image content in the digital image outside of the optimal location.

21. A computer program product as claimed in claim 19 in which the step (f) of altering one or more attributes of the human understandable item comprises altering at least one of the color, size and shape of the human understandable item.

22. A method for automatically placing text or a figurative element within an underlying image area of a digital image so as to minimize the amount of visual image information obscured, the method comprising the steps of.

(a) processing the pixels of the image to recognize and identify pixel content indicating a contiguous region in the digital image that does not contain any substantial presence of a characteristic indicative of significant or valuable human understandable image content, including a color characteristic indicative of a flesh color, said human understandable image content being part of the underlying image area, thereby identifying a suitable area for placement of the text or figurative element, wherein said text or figurative element is separate and distinct from said human understandable image content;

(b) determining an optimal location in the digital image within the suitable area for the text or figurative element to be placed;

(c) determining a goodness of fit within the optimal location for the text or figurative element;

(d) transforming the text or figurative element based upon the results of step (c) to improve the goodness of fit for the text or figurative element without affecting the spatial location and integrity of said human understandable content;

(e) determining an optimal color for the text or figurative element based upon the location of the text or figurative element in the image; and

(f) rendering the digital image with the human understandable item overlaid substantially into the optimal location such that the human understandable image content is spatially unaffected by the overlaid item.

23. A method as claimed in claim 22 wherein step (a) further includes finding open spaces within the image.

24. A method as claimed in claim 22 wherein step (a) further includes finding regions of the image not containing the main subject.

25. A method as claimed in claim 22 wherein step (b) further includes searching the image for the location where the text or figurative element occludes a minimum of significant visual image content.

26. A method as claimed in claim 22 wherein step (d) further includes scaling the text or figurative element.

27. A method as claimed in claim 22 wherein step (d) further includes splitting the text into multiple lines with various justification.

28. A method as claimed in claim 22 wherein step (e) further includes choosing white or black as the color of the text so as to maximize the contrast between the text and the surrounding image area.

29. A method as claimed in claim 22 wherein step (e) further includes choosing a dominant color appearing within the image as the color of the text so as to maximize the contrast between the text and the surrounding image area.

30. A method as in claim 22 wherein step (e) further includes choosing a color of the text so as to maximize the contrast between the text and the surrounding image area.

31. A method for placing a human understandable item in a digital image, said method comprising the steps of:

(a) digitally processing the digital image to recognize open space regions in the digital image;

(b) digitally processing the open space regions to exclude one or more open space regions based upon a color characteristic which is indicative of a flesh colored image content that makes the open space region unsuitable as a candidate region for item placement and produce a suitability map that indicates significant sized open space regions suitable for item placement and a user preference for placement of the human understandable item within the digital image;

(c) digitally processing the suitability map to identify an optimal location in the digital image for placing the human understandable item;

(d) modifying the placement of the human understandable item in relation to human understandable image content in the digital image so as to minimally obscure the human understandable image content in the digital image; and

(e) rendering the digital image with the human understandable item overlaid substantially into the optimal location such that the human understandable image content is spatially unaffected by the overlaid human understandable item.

32. A method for placing a human understandable item in a digital image, said method comprising the steps of:

(a) digitally processing the digital image to recognize a main subject in the digital image by (a1) extracting regions of arbitrary shape and size defined by actual objects in the image, (a2) extracting for each of the regions at least one structural saliency feature and at least one semantic saliency feature, and (a3) integrating the structural saliency feature and the semantic saliency feature using a probabilistic reasoning engine into an estimate of a belief that each region is the main subject;

(b) producing a suitability map from the belief estimate that indicates significant sized contiguous regions not containing the main subject and a user preference for placement of the human understandable item within the digital image;

(c) digitally processing the suitability map to identify an optimal location in the digital image for placing the human understandable item;

(d) modifying the placement of the human understandable item in relation to human understandable image content in the digital image so as to minimally obscure the human understandable image content in the digital image; and

(e) rendering the digital image with the human understandable item overlaid substantially into the optimal location such that the human understandable image content is spatially unaffected by the overlaid human understandable item.