Systems and methods for automatically generating content layout based on selected highest scored image and selected text snippet

• US 9,946,695 B2
• Filed: 05/09/2013
• Issued: 04/17/2018
• Est. Priority Date: 02/22/2013
• Status: Active Grant

• Current Assignee: Google LLC (Alphabet Inc.)
• Original Assignee: Google LLC (Alphabet Inc.)
• Inventors: Zhang, Guannan, Ye, Kai, Zhu, Yongtai, Yan, Zuo, Foster, Andrew, Bayliss-McCulloch, Jordan, Chang, Daly, Nonaka, Ayaka
• Primary Examiner(s): Paula, Cesar B
• Assistant Examiner(s): Nguyen, Luu-Phuong T
• Application Number:
  US13/890,915
• Publication Number:
  US 20140245115A1
• Time in Patent Office: 1,804 Days
• Field of Search: 715201
• CPC Class Codes: G06F 16/345 Summarisation for human users
• CPC Class Codes: G06F 16/434 using image data, e.g. images, photos, pictures taken by a user
• CPC Class Codes: G06F 16/58 Retrieval characterised by using metadata, e.g. metadata not derived from the content or metadata generated manually
• CPC Class Codes: G06F 16/583 using metadata automatically derived from the content
• CPC Class Codes: G06F 16/5838 using colour
• CPC Class Codes: G06F 16/5846 using extracted text
• CPC Class Codes: G06F 16/5866 using information manually generated, e.g. tags, keywords, comments, manually generated location and time information
• CPC Class Codes: G06F 3/04842 Selection of displayed objects or displayed text elements G06F3/0482 takes precedence
• CPC Class Codes: G06F 40/103 Formatting, i.e. changing of presentation of documents automatic justification G06F40/189; automatic line break hyphenation G06F40/191
• CPC Class Codes: G06F 40/106 Display of layout of documents; Previewing
• CPC Class Codes: G06F 40/131 Fragmentation of text files, e.g. creating reusable text-blocks; Linking to fragments, e.g. using XInclude; Namespaces
• CPC Class Codes: G06F 40/166 Editing, e.g. inserting or deleting
• CPC Class Codes: G06F 40/30 Semantic analysis
• CPC Class Codes: G06T 1/00 General purpose image data processing
• CPC Class Codes: G06T 7/00 Image analysis

Abstract

A computerized method for automatically generating display content includes receiving a uniform resource locator, wherein the uniform resource locator specifies a landing resource and extracting visual information from the landing resource, wherein the visual information defines one or more images, texts, and colors displayed on the landing resource. The method further includes selecting one or more images, one or more text snippets, and one or more colors based on the visual information extracted from the landing resource, generating a layout for a content item based on one or more of the selected images or selected text snippets, and assembling the content item by applying the selected images, the selected text snippets, and the selected colors to the generated layout.

Claims

1. A computerized method for automatically generating display content, the method comprising:

receiving a uniform resource locator specifying a landing resource;

extracting, from the landing resource, visual information defining a plurality of images, a plurality of text snippets, and a plurality of colors displayed on the landing resource, each of the plurality of images having respective associated metadata;

calculating automatically, by one or more processors, a respective score for each of the plurality of images defined by the landing resource based on a specified keyword in the metadata associated with the respective image, a display size of the respective image, and an alignment scale proportional to a distance between a center of the respective image and a horizontal center of the landing resource;

selecting, by the one or more processors, a highest scored image of the plurality of images based on the respective score calculated for each of the plurality of images;

selecting, by the one or more processors, one of the plurality of text snippets and one of the plurality of colors extracted from the landing resource;

generating, by the one or more processors, a layout for a content item based on the selected highest scored image and the selected text snippet; and

assembling, by the one or more processors, the content item by applying the selected highest scored image, the selected text snippet, and the selected color to the generated layout.

2. The method of claim 1, wherein the visual information further defines a plurality of fonts displayed on the landing resource, wherein a font is selected based on the visual information, and wherein the assembled content item includes the selected font.

3. The method of claim 1, wherein extracting visual information from the landing resource comprises rendering the landing resource and generating a document object model tree based on the rendered resource.

4. The method of claim 1, wherein the metadata includes at least one of:

a URL, a display position, a display size, and alt text.

5. The method of claim 4, wherein the specified keyword selects for logo images.

6. The method of claim 1, wherein selecting one of the plurality of text snippets comprises selecting a portion of the text displayed on the landing resource.

7. The method of claim 1, wherein selecting one of the plurality of text snippets comprises:

obtaining a plurality of user reviews comprising user-provided comments relating to a business, product, or service, featured on the landing resource;

identifying a first phrase in the user reviews, wherein the first phrase includes a keyword indicative of a first type of sentiment; and

extracting a portion of the user reviews, wherein the extracted portion includes the identified first phrase.

8. The method of claim 1, wherein selecting one of the plurality of colors comprises:

receiving a snapshot of the landing resource, wherein the snapshot is a rendered image of the landing resource; and

extracting the plurality of colors from the snapshot image.

9. The method of claim 8, wherein the plurality of colors are extracted from the snapshot image using a k-means clustering technique involving:

measuring a color value for each pixel in the snapshot image, wherein the color value for a pixel is a numerical representation of a color displayed by the pixel;

identifying a plurality of initial color clusters, wherein each color cluster has a numerical color value;

assigning each pixel in the snapshot image to a color cluster, wherein a pixel is assigned to the color cluster having a color value closest to the color value of the pixel;

determining a mean color value for each of the color clusters, wherein the mean color value is an average of the color values of all pixels assigned to the color cluster; and

extracting the mean color values from the color clusters.

10. The method of claim 9, further comprising:

reassigning each pixel in the snapshot image to a color cluster, wherein each pixel is reassigned to the color cluster having a mean color value closest to the color value of the pixel; and

re-determining the mean color value for each of the color clusters, wherein the mean color value is an average of the color values of all pixels reassigned to the color cluster.

11. The method of claim 9, further comprising:

assigning a weight to each of the mean color values extracted from the color clusters, wherein the assigned weight for a color value is based on a number of pixels in the color cluster from which the color value was extracted relative to a number of total pixels in the snapshot image.

12. The method of claim 9, wherein the color values are red-green-blue (RGB) color values and wherein the numerical representations of the color values include a red value, a green value, and a blue value.

13. The method of claim 1, wherein generating a layout for the content item comprises:

receiving dimensions for the content item, wherein the dimensions specify a display height and a display width for the content item;

creating a frame for the content item, wherein the frame has a height and a width corresponding to the dimensions of the content item;

placing the selected highest scored image in a starting location within the frame;

dividing any remaining unused space in the frame into a plurality of rectangles;

combining, among the plurality of rectangles, a first rectangle that is either horizontally adjacent or vertically adjacent to the location of the placed highest scored image with a second rectangle that is diagonally adjacent to the location of the placed highest scored image in the remaining unused space to create a combined rectangle according to a display size or an aspect ratio of an unplaced text snippet or image; and

placing the unplaced text snippet or image into the combined rectangle.

14. The method of claim 13, wherein one or more of the plurality of rectangles have a display size or aspect ratio based on a display size or aspect ratio of an unused text snippet or an unused image.

15. The method of claim 1, wherein the uniform resource locator is received from a content requestor, the method further comprising:

scoring the assembled content item and, wherein the score is based on at least one of;

how efficiently space is used in the content item, an amount of cropping, stretching, or shrinking applied to one or more images in the content item, relative locations of images, text snippets, and action buttons in the content item, a contrast between text color and background color in the content item, and individual scores, rankings, or weights for each of the images, text snippets, and colors used in the content item; and

presenting the assembled content item to the content requestor.

16. A system for automatically generating display content, the system comprising:

one or more processors, at least one memory device, and at least one network interface;

wherein the one or more processors are configured to;

extract visual information from a landing resource defining a plurality of images, a plurality of text snippets, and a plurality of colors displayed on the landing resource, each of the plurality of images having respective associated metadata, the extracted visual information stored in the at least one memory device,

calculate automatically a respective score for each of the plurality of images defined by the landing resource based on a specified keyword in the metadata associated with the respective image, a display size of the respective image, and an alignment scale proportional to a distance between a center of the respective image and a horizontal center of the landing resource,

select a highest scored image of the plurality of images based on the respective score calculated for each of the plurality of images,

select one of the plurality of text snippets and one of the plurality of colors extracted from the landing resource,

generate a layout for a content item based on the selected highest scored image and the selected text snippet, and

assemble the content item by applying the selected highest scored image, the selected text snippet, and the selected color to the generated layout; and

wherein the at least one network interface is configured to provide the assembled content item to a content requestor.