Transforming data to create layouts

• US 9,595,298 B2
• Filed: 07/18/2012
• Issued: 03/14/2017
• Est. Priority Date: 07/18/2012
• Status: Active Grant

• Current Assignee: Microsoft Technology Licensing LLC (Microsoft Corporation)
• Original Assignee: Microsoft Technology Licensing LLC (Microsoft Corporation)
• Inventors: Lee, David Benjamin, Freier, Nathan George, Tumanov, Ilya
• Primary Examiner(s): Bloomquist, Keith
• Application Number:
  US13/552,416
• Publication Number:
  US 20140026038A1
• Time in Patent Office: 1,700 Days
• US Class Current: 1/1
• CPC Class Codes: G06F 3/04845 for image manipulation, e.g. dragging, rotation, expansion or change of colour
• CPC Class Codes: G06F 40/106 Display of layout of documents; Previewing
• CPC Class Codes: G06T 11/60 Editing figures and text; Combining figures or text
• CPC Class Codes: G06T 2200/24 involving graphical user interfaces [GUIs]
• CPC Class Codes: G11B 27/28 by using information signals recorded by the same method as the main recording G11B27/22 takes precedence
• CPC Class Codes: G11B 27/34 Indicating arrangements indicating means incorporated in magazine or cassette G11B23/046 and G11B23/0875; indicating measured values in general G01D

Abstract

Concepts and technologies are described herein for transforming data to create layouts. In accordance with the concepts and technologies disclosed herein, a computing device can execute a layout generator. The computing device can be configured to receive or retrieve data that includes visual content such as an image and other content such as text. The computing device can analyze the visual content to identify a salient region and/or an invariant region within the visual content. The computing device can access designs for laying out visual content and other content based upon an identified salient region and/or invariant region. The computing device can evaluate the designs to determine if a design matches the data. The computing device can load the data into the selected design to generate the layout.

Claims

1. A computer-implemented method for transforming data to generate a layout, the computer-implemented method comprising performing computer-implemented operations for:

executing a layout generator at a computing device to obtain the data, the data comprising an image and at least one of text or another element of content;

analyzing the image, using the layout generator, to identify;

a salient region of the image, the salient region comprising a subject of the image; and

an invariant region of the image, at least one of color variation, luminance variation, sharpness variation, blur variation, or pattern variation of the invariant region being below a threshold to achieve a contrast between the text and the image;

comparing the data to a plurality of designs;

selecting a design of the plurality of designs to generate the layout, the design comprising a viewport frame, a salient region placement frame, and a text placement frame, and the selecting being based at least in part on;

determining that the salient region corresponds to the salient region placement frame; and

determining that the invariant region corresponds to the text placement frame; and

generating the layout based at least in part on;

placing the image such that at least a first portion of the salient region is located in a first position corresponding to the salient region placement frame; and

placing at least one of the text or other element of content such that at least a second portion of the at least one of the text or other element of content is located in a second position corresponding to the text placement frame.

2. The method of claim 1, wherein determining that the salient region corresponds to the salient region placement frame comprises determining that the image can be moved or cropped such that at least the first portion of the salient region fits within the salient region placement frame.

3. The method of claim 1, wherein selecting the design further comprises:

determining a score representative of a degree to which each design of the plurality of designs relates to the data;

ranking, based at least in part on the score, the plurality of designs; and

determining that the design is a highest ranked design of the plurality of designs.

4. The method of claim 1, wherein the viewport frame defines viewable limits of the layout.

5. The method of claim 1, wherein analyzing the image to identify the salient region is based at least in part on analyzing color differences at or within the salient region.

6. The method of claim 1, wherein analyzing the image to identify of the salient region is based at least in part on performing facial recognition processing on the image.

7. The method of claim 1, wherein analyzing the image to identify the salient region is based at least in part on analyzing blurring or sharpness at or within the salient region.

8. The method of claim 1, further comprising, prior to determining that the salient region corresponds to the salient region placement frame, performing at least one manipulation to the image.

9. The method of claim 8, wherein the at least one manipulation comprises zooming, panning, scrolling, cropping, extending, infilling, or seam carving of the image.

10. A computer storage medium having computer readable instructions stored thereupon that, when executed by a computer, cause the computer to:

execute a layout generator to obtain data comprising visual content and text;

analyze the image to identify;

a salient region of the image, the salient region comprising a subject of the image; and

an invariant region of the image, at least one of color variation, luminance variation, sharpness variation, blur variation, or pattern variation of the invariant region being below a threshold to achieve a contrast between the text and the image;

compare the data to a plurality of designs to provide a comparison of the data;

select, based at least partially upon the comparison of the data, a design of the plurality of designs, the design defining a layout and including a viewport frame, a salient region placement frame, and a text placement frame;

adjust a position of the image relative to the viewport frame to locate the salient region at a first location within the layout corresponding to the salient region placement frame;

overlay the text at a location within the layout that corresponds to a second location of the text placement frame within the design; and

generate the layout, the layout comprising at least a first portion of the image and at least a second portion of the text.

11. The computer storage medium of claim 10, wherein the computer executable instructions further cause the computer to:

compare the salient region to first locations respectively corresponding to salient region placement frames in individual designs of the plurality of designs;

compare the invariant region to second locations respectively corresponding to text placement frames in the individual designs of the plurality of designs;

determine that the first location matches the salient region;

determine that the second location overlaps the invariant region; and

select the design based at least in part on determining that the first location matches the salient region and that the second location overlaps the invariant region.

12. The computer storage medium of claim 10, wherein the computer executable instructions further cause the computer to:

prior to selecting the design, resize the image to generate a scaled version of the image; and

select the design based at least in part on the scaled version of the image causing the salient region to correspond to the salient region placement frame and the invariant region to correspond to the text placement frame.

13. The computer storage medium of claim 10, wherein analyzing the image to identify the salient region is based at least in part on analyzing color distributions at or within the salient region.

14. The computer storage medium of claim 10, wherein analyzing the image to identify the salient region is based at least in part on performing object recognition.

15. The method of claim 10, wherein the invariant region excludes the salient region.

16. An apparatus comprising:

a processor;

a memory communicatively coupled to the processor; and

an application program which executes in the processor from the memory and which, when executed by the processor, causes the processor to;

execute a layout generator to obtain data comprising visual content and text;

analyze the image to identify a salient region of the image and an invariant region of the image, the salient region comprising a subject of the image and the invariant portion comprising a first portion of the image at which at least one of color variation, luminance variation, sharpness variation, blur variation, or pattern variation is below a threshold;

compare the data to a plurality of designs to provide a comparison of the data;

select, based at least partially upon the comparison of the data, a design defining a layout, the design comprising a viewport frame defining visible limits of the layout, a salient region placement frame corresponding to a first location at which the salient region of the image is to be placed, and a text placement frame corresponding to a second location at which the text is to be placed within the layout;

load the data into the design based, at least partially, upon the salient region, the invariant region, the salient region placement frame, and the text placement frame; and

generate the layout, the layout comprising at least the subject of the image and the text.

17. The apparatus of claim 16, wherein the application program further causes the processor to:

based at least in part on comparing the data to the plurality of designs, determine a score for each design of the plurality of designs, the score representative of a degree to which each design of relates to the data;

rank, based at least in part on the score, the plurality of designs; and

determine that the design is a highest ranked design of the plurality of designs.

18. The apparatus of claim 16, wherein analyzing the image to identify the salient region of the image is based at least in part on performing facial recognition.

19. The apparatus of claim 16, wherein analyzing the image to identify the salient region of the image is based at least in part on analyzing brightness data at or within the salient region.

20. The apparatus of claim 16, wherein the application program further causes the processor to determine, based at least in part the comparison of the data, that the design matches the data based at least in part on:

determining that at least a second portion of the salient region fits within the salient region placement frame; and

determining that at least a third portion of the invariant portion fits within the text placement frame.