Transforming data to create layouts

• US 10,031,893 B2
• Filed: 01/26/2017
• Issued: 07/24/2018
• Est. Priority Date: 07/18/2012
• Status: Active Grant

• Current Assignee: Microsoft Technology Licensing LLC (Microsoft Corporation)
• Original Assignee: Microsoft Technology Licensing LLC (Microsoft Corporation)
• Inventors: Lee, David Benjamin, Freier, Nathan George, Tumanov, Ilya
• Primary Examiner(s): Bloomquist, Keith D
• Application Number:
  US15/416,953
• Publication Number:
  US 20170139877A1
• Time in Patent Office: 544 Days
• CPC Class Codes: G06F 3/04845 for image manipulation, e.g. dragging, rotation, expansion or change of colour
• CPC Class Codes: G06F 40/106 Display of layout of documents; Previewing
• CPC Class Codes: G06T 11/60 Editing figures and text; Combining figures or text
• CPC Class Codes: G06T 2200/24 involving graphical user interfaces [GUIs]
• CPC Class Codes: G11B 27/28 by using information signals recorded by the same method as the main recording G11B27/22 takes precedence
• CPC Class Codes: G11B 27/34 Indicating arrangements indicating means incorporated in magazine or cassette G11B23/046 and G11B23/0875; indicating measured values in general G01D

Abstract

Concepts and technologies are described herein for transforming data to create layouts. In accordance with the concepts and technologies disclosed herein, a computing device can execute a layout generator. The computing device can be configured to receive or retrieve data that includes visual content such as an image and other content such as text. The computing device can analyze the visual content to identify a salient region and/or an invariant region within the visual content. The computing device can access designs for laying out visual content and other content based upon an identified salient region and/or invariant region. The computing device can evaluate the designs to determine if a design matches the data. The computing device can load the data into the selected design to generate the layout.

Claims

1. A computer-implemented method for transforming data to generate a layout, the computer-implemented method comprising:

obtaining data comprising an image and a content item;

analyzing the image to identify;

a salient region of the image, the salient region comprising a subject of the image; and

an invariant region of the image, the invariant region comprising a region of the image wherein a variation of at least one feature is below a threshold to achieve a contrast between the content item and the image;

comparing the data to a plurality of designs;

selecting a design of the plurality of designs to generate the layout, the design comprising a viewport frame, a salient region placement frame, and a content placement frame, and the selecting being based at least in part on;

determining that the salient region corresponds to the salient region placement frame; and

determining that the invariant region corresponds to the content placement frame; and

generating the layout based at least in part on the design.

2. The method of claim 1, further comprising, prior to determining that the salient region corresponds to the salient region placement frame, performing at least one manipulation to the image.

3. The method of claim 2, wherein the at least one manipulation comprises zooming, panning, scrolling, cropping, extending, infilling, or seam carving of the image.

4. The method of claim 1, wherein selecting the design comprises:

determining a score representative of a degree to which each design of the plurality of designs relates to the data;

ranking, based at least in part on the score, the plurality of designs; and

determining that the design is a highest ranked design of the plurality of designs.

5. The method of claim 1, wherein the viewport frame defines viewable limits of the layout.

6. The method of claim 1, wherein the at least one feature comprises color, luminance, sharpness, blur, or pattern.

7. The method of claim 1, wherein generating the layout comprises:

placing the image such that at least a first portion of the salient region is located in a first position corresponding to the salient region placement frame; and

placing the content item such that at least a second portion of the content item is located in a second position corresponding to the content placement frame.

8. The method of claim 1, wherein analyzing the image to identify the salient region is based at least in part on one of:

analyzing color differences at or within the salient region;

performing facial recognition processing on the image;

or

analyzing blurring or sharpness at or within the salient region.

9. A computer storage medium having computer readable instructions stored thereupon that, when executed by a computer, cause the computer to:

obtain data comprising visual content and a content item;

analyze the image to identify;

a salient region of the image, the salient region comprising a subject of the image; and

an invariant region of the image, the invariant region comprising a region of the image wherein a variation of a feature is below a threshold to achieve a contrast between the content item and the image;

compare the data to a plurality of designs to provide a comparison of the data;

select, based at least partially upon the comparison of the data, a design of the plurality of designs, the design defining a layout and including a viewport frame, a salient region placement frame, and a content placement frame, and the selecting being based at least in part on;

determining that the salient region corresponds to the salient region placement frame; and

determining that the invariant region corresponds to the content placement frame; and

generate a layout based at least in part on the design.

10. The computer storage medium of claim 9, wherein the feature includes at least one of color, luminance, sharpness, blur, or pattern.

11. The computer storage medium of claim 9, wherein the computer readable instructions further cause the computer to:

adjust a position of the image relative to the viewport frame to locate the salient region at a first location within the layout corresponding to the salient region placement frame; and

overlay the content item at a location within the layout that corresponds to a second location of the content placement frame within the design.

12. The computer storage medium of claim 11, wherein the computer readable instructions further cause the computer to:

compare the salient region to first locations respectively corresponding to salient region placement frames in individual designs of the plurality of designs;

compare the invariant region to second locations respectively corresponding to content placement frames in the individual designs of the plurality of designs;

determine that the first location matches the salient region;

determine that the second location overlaps the invariant region; and

select the design based at least in part on determining that the first location matches the salient region and that the second location overlaps the invariant region.

13. The computer storage medium of claim 9, wherein the computer readable instructions further cause the computer to:

prior to selecting the design, resize the image to generate a scaled version of the image; and

select the design based at least in part on the scaled version of the image causing the salient region to correspond to the salient region placement frame and the invariant region to correspond to the content placement frame.

14. The computer storage medium of claim 9, wherein analyzing the image to identify the salient region is based on at least one of:

analyzing color distributions at or within the salient region;

performing object recognition;

or

performing facial recognition.

15. The method of claim 9, wherein the invariant region excludes the salient region.

16. An apparatus comprising:

a processor;

a memory communicatively coupled to the processor; and

an application program which executes in the processor from the memory and which, when executed by the processor, causes the processor to;

obtain data comprising visual content and a content item;

analyze the visual content to identify a salient region of the visual content and an invariant region of the visual content, the salient region comprising a subject of the visual content and the invariant portion comprising a first portion of the visual content wherein variation within the visual content is below a threshold;

compare the data to a plurality of designs to provide a comparison of the data;

select, based at least partially upon the comparison of the data, a design defining a layout, the design comprising a viewport frame defining visible limits of the layout, a salient region placement frame corresponding to a first location at which the salient region of the visual content is to be placed, and a content placement frame corresponding to a second location at which the content item is to be placed within the layout;

load the data into the design based, at least partially, upon the salient region, the invariant region, the salient region placement frame, and the content placement frame; and

generate the layout, the layout comprising at least the subject of the visual content and the content item.

17. The apparatus of claim 16, wherein the application program further causes the processor to:

based at least in part on comparing the data to the plurality of designs, determine a score for each design of the plurality of designs, the score representative of a degree to which each design of the plurality of designs relates to the data;

rank, based at least in part on the score, the plurality of designs; and

determine that the design is a highest ranked design of the plurality of designs.

18. The apparatus of claim 16, wherein analyzing the visual content to identify the salient region of the visual content is based at least in part on analyzing brightness data at or within the salient region.

19. The apparatus of claim 16, wherein the application program further causes the processor to determine, based at least in part the comparison of the data, that the design matches the data based at least in part on:

determining that at least a second portion of the salient region fits within the salient region placement frame; and

determining that at least a third portion of the invariant portion fits within the content placement frame.

20. The apparatus of claim 16, wherein the content item comprises text or another element of content.