Overlaying text in images for display to a user of a digital magazine

• US 9,483,855 B2
• Filed: 01/14/2014
• Issued: 11/01/2016
• Est. Priority Date: 01/15/2013
• Status: Active Grant

• Current Assignee: Flipboard Incorporated
• Original Assignee: Flipboard Incorporated
• Inventors: Ying, Charles H
• Primary Examiner(s): Wang, Jin-Cheng
• Application Number:
  US14/155,053
• Publication Number:
  US 20140198127A1
• Time in Patent Office: 1,022 Days
• Field of Search: 382/217, 382/218, 382/283, 382/284, 715/246, 715/247
• US Class Current: 1/1
• CPC Class Codes: G06F 16/24578 using ranking
• CPC Class Codes: G06F 16/93 Document management systems
• CPC Class Codes: G06F 16/9535 Search customisation based on user profiles and personalisation
• CPC Class Codes: G06F 16/958 Organisation or management of web site content, e.g. publishing, maintaining pages or automatic linking
• CPC Class Codes: G06F 40/10 Text processing natural language analysis G06F40/20; semantic analysis G06F40/30; processing or translation of natural language G06F40/40
• CPC Class Codes: G06F 40/106 Display of layout of documents; Previewing
• CPC Class Codes: G06F 40/114 Pagination
• CPC Class Codes: G06F 40/131 Fragmentation of text files, e.g. creating reusable text-blocks; Linking to fragments, e.g. using XInclude; Namespaces
• CPC Class Codes: G06F 40/186 Templates
• CPC Class Codes: G06T 11/60 Editing figures and text; Combining figures or text
• CPC Class Codes: G06T 2207/10024 Color image
• CPC Class Codes: G06T 7/11 Region-based segmentation
• CPC Class Codes: G06T 7/136 involving thresholding

Abstract

A digital magazine server presents content retrieved from various sources to a user. The content may include various images, with images selected by the digital magazine server for presentation to a user along with additional content. Text associated with an image may also be retrieved and presented to the user along with the image. To improve presentation of images with text, the digital magazine server identifies features in an image and identifies regions of the image where text is capable of being presented without obscuring or being obscured by features of the image. The image is modified to present text associated with the image in one or more of the identified images, allowing the text to be presented without detracting from features of the image.

Claims

1. A method comprising:

maintaining image templates in a template store, each image template including one or more regions for presenting text in one or more locations of the image template, each region having a size and a position relative to one or more regions;

obtaining an image for presentation to a user of a digital magazine server;

identifying one or more features of the image, the features identifying regions containing one or more objects included in the image;

identifying one or more regions in the image capable of including text based at least in part on the one or more identified features, a region in the image capable of including text having at least a threshold distance from an identified feature and having at least a threshold size;

selecting an image template including regions having at least a threshold number of attributes matching attributes of the identified one or more regions in the image;

selecting text for presentation along with the image;

selecting one or more regions of the selected image template having a maximum degree of similarity to one or more regions in the image capable of including text; and

modifying the image to present the selected text in the one or more regions in the image capable of including text corresponding to the selected regions in the selected image template.

2. The method of claim 1, wherein identifying one or more features of the image further comprises:

identifying edges associated with one or more of the objects included in the image; and

identifying regions containing one or more objects included in the image, based at least in part on the identified edges associated with one or more of the objects included in the image.

3. The method of claim 1, wherein identifying one or more features of the image comprises:

determining a location of a feature relative to one or more boundaries of the image;

associating information describing the location with the feature; and

storing the information describing the location of the feature in association with an identifier associated with the feature.

4. The method of claim 3, wherein the information describing the location of the feature includes one or more selected from a group consisting of:

size information describing one or more dimensions of the feature, information describing a shape of the feature, information describing the location of the feature relative to one or more boundaries of the image, and any combination thereof.

5. The method of claim 1, wherein identifying one or more features of the image comprises:

generating a feature identifier associated with a feature, the feature identifier associating the feature with the image.

6. The method of claim 1, wherein identifying one or more regions in the image capable of including text based at least in part on the one or more identified features further comprises:

determining information describing one or more attributes of a region based at least in part on a location of a region relative to one or more boundaries of the image; and

storing the information describing one or more attributes of the region in association with an identifier corresponding to the region.

7. The method of claim 6, wherein the information describing the one or more attributes of the region include information selected from a group consisting of:

the location of the region relative to one or more boundaries of the image, a description of a shape of the region, a description of one or more dimensions of the region, and any combination thereof.

8. The method of claim 6 wherein the identifier corresponding to the region associates the region with the image.

9. The method of claim 1, wherein selecting text for presentation along with the image further comprises:

identifying text associated with the image; and

retrieving the identified text for presentation along with the image.

10. The method of claim 1, wherein selecting one or more of the one or more regions in the image capable of including text comprises:

identifying text associated with the image;

identifying one or more portions of the identified text, each portion of the identified text associated with one or more attributes; and

selecting a region in the image capable of including text in which a portion of the identified text is presented based at least in part on one or more of the attributes associated with the portion of the identified text.

11. A method comprising:

maintaining image templates in a template store, each image template including one or more regions for presenting text in one or more locations of the image template, each region having a size and a position relative to one or more regions;

obtaining an image for presentation to a user of a digital magazine server;

identifying one or more features of the image, the features identifying regions containing one or more objects included in the image;

identifying one or more regions in the image capable of including text based at least in part on the one or more identified features, a region in the image capable of including text having at least a threshold distance from an identified feature and having at least a threshold size;

selecting an image template including regions having at least a threshold number of attributes matching attributes of the identified one or more regions in the image;

selecting text for presentation along with the image;

selecting one or more of the one or more regions in the image capable of including text and corresponding to regions in the selected image template;

modifying the image to present the selected text in the one or more selected regions in the image capable of including text; and

displaying the modified image to the user of the digital magazine sever.

12. The method of claim 11, wherein selecting one or more of the one or more regions in the image capable of including text further comprises:

identifying one or more portions of the selected text for presentation in a region in the image capable of including text based at least in part on characteristics of the region in the image capable of including text; and

associating the one or more portions of the selected text with the region in the image capable of including text.

13. The method of claim 11, wherein modifying the image to present the selected text in the one or more selected regions in the image capable of including text further comprises:

identifying one or more locations of the image corresponding to each of the one or more selected regions in the image; and

overlaying the selected text in the identified one or more locations of the image.

14. A computer program product comprising a non-transitory computer-readable medium having instructions encoded thereon that, when executed by a processor, cause the processor to:

maintain image templates in a template store, each image template including one or more regions for presenting text in one or more locations of the image template, each region having a size and a position relative to one or more regions;

obtain an image for presentation to a user of a digital magazine server;

identify one or more features of the image, the features identifying regions containing one or more objects included in the image;

identify one or more regions in the image capable of including text based at least in part on the one or more identified features, a region in the image capable of including text having at least a threshold distance from an identified feature and having at least a threshold size;

select an image template including regions having at least a threshold number of attributes matching attributes of the identified one or more regions in the image;

select text for presentation along with the image;

select one or more of the one or more regions in the image capable of including text;

modify the image to present the selected text in the one or more selected regions in the image capable of including text; and

display the modified image to the user of the digital magazine server.

15. The computer program product of claim 14, wherein select one or more of the one or more regions in the image capable of including text further comprises:

identify an image template including region for presenting text having at least a threshold similarity to at least a threshold number of the one or more identified regions capable of presenting text;

select one or more regions for presenting text in the identified image template for presenting text; and

identify regions in the image capable of including text based on locations associated with the selected one or more regions for presenting text in the identified image template.

16. The computer program product of claim 14, wherein select one or more of the one or more regions in the image capable of including text further comprises:

identify an image template including region for presenting text having at least a threshold similarity to at least a threshold number of the one or more identified regions capable of presenting text;

select one or more regions for presenting text in the identified image template for presenting text based at least in part on location information associated with the regions for presenting text in the identified image template identifying positions of the regions for presenting text relative to one or more boundaries of the identified image template; and

identify regions in the image capable of including text based on locations associated with the selected one or more regions for presenting text in the identified image template.

17. The computer program product of claim 14, wherein select one or more of the one or more regions in the image capable of including text comprises:

identify text associated with the image;

identify one or more portion of the identified text, each portion of the identified text associated with one or more attributes; and

select a region in the image capable of including text in which a portion of the identified text is presented based at least in part on one or more of the attributes associated with the portion of the identified text.

18. The computer program product of claim 14, wherein identify one or more regions in the image capable of including text based at least in part on the one or more identified features further comprises:

determine information describing one or more attributes of a region based at least in part on a location of a region relative to one or more boundaries of the image; and

store the information describing one or more attributes of the region in association with an identifier corresponding to the region.

19. The computer program product of claim 18, wherein the information describing the one or more attributes of the region include information selected from a group consisting of:

the location of the region relative to one or more boundaries of the image, a description of a shape of the region, a description of one or more dimensions of the region, and any combination thereof.

20. The computer program product of claim 14, wherein identify one or more regions in the image capable of including text based at least in part on the one or more identified features further comprises:

determine one or more colors associated with a region.

21. The computer program product of claim 14, wherein select text for presentation along with the image further comprises:

identify text associated with the image;

determine one or more colors associated with one or more portions of the identified text; and

retrieving one or more portions of the identified text for presentation along with the image based at least in part on the one or more colors associated with the one or more portions of the identified text.

22. The computer program product of claim 14, wherein modify the image to present the selected text in the one or more selected regions in the image capable of including text further comprises:

modifying the selected text based on one or more colors associated with the one or more selected regions.

23. The computer program product of claim 22, wherein modifying the selected text further comprises:

presenting an outline around one or more borders of the selected text, modifying a color of the selected text, modifying a font of the selected text, and any combination thereof.