Automated image scale adjustment based upon document and image context

• US 10,699,171 B2
• Filed: 04/12/2018
• Issued: 06/30/2020
• Est. Priority Date: 04/12/2018
• Status: Active Grant

• Current Assignee: Microsoft Technology Licensing LLC (Microsoft Corporation)
• Original Assignee: Microsoft Technology Licensing LLC (Microsoft Corporation)
• Inventors: Govrin, Amnon Itamar, Kikin Gil, Erez, Talbot, Keri, Bashaw, Aleta
• Primary Examiner(s): Tsai, Tsung Yin
• Application Number:
  US15/952,013
• Publication Number:
  US 20190318212A1
• Time in Patent Office: 810 Days
• CPC Class Codes: G06F 40/103 Formatting, i.e. changing of presentation of documents automatic justification G06F40/189; automatic line break hyphenation G06F40/191
• CPC Class Codes: G06T 11/60 Editing figures and text; Combining figures or text
• CPC Class Codes: G06T 3/40 Scaling of whole images or parts thereof, e.g. expanding or contracting
• CPC Class Codes: G06V 30/274 Syntactic or semantic context, e.g. balancing
• CPC Class Codes: G06V 30/414 Extracting the geometrical structure, e.g. layout tree; Block segmentation, e.g. bounding boxes for graphics or text
• CPC Class Codes: G06V 40/172 Classification, e.g. identification

Abstract

Disclosed are methods, systems, and machine readable mediums which provide for an automatic image scaling that may be utilized in a variety of contexts, including for scaling images inserted into documents. The image may be scaled based upon one or more scaling features that may be utilized alone or in combination. Scaling features may be described by scaling feature categories, although as will be appreciated, the scaling features may be placed in one or more categories. Example feature categories may include image characteristics; image content; content of the document and context of the image within the document; past user behavior and preferences; and the like.

Claims

1. A method for automated digital image scaling, the method comprising:

receiving a selection of a digital image and a desired placement position in a digital document, the digital image having a first size;

automatically determining a second size of the digital image based upon a determined relevance of one or more recognized objects depicted in the digital image to a semantic meaning of textual content of the digital document, the textual content within a defined proximity to the desired placement position, the second size different than the first size;

automatically scaling the digital image from the first size to the second size; and

automatically placing the scaled digital image in the digital document at the desired placement position.

2. The method of claim 1, wherein automatically determining the second size of the digital image comprises:

determining the second size based upon the determined relevance of the one or more recognized objects depicted in the digital image to the semantic meaning of the textual content of the digital document and a frequency with which one of the one or more recognized objects are mentioned in the textual content of the digital document.

3. The method of claim 1, wherein automatically determining the second size of the digital image comprises:

determining the second size based upon the determined relevance of the one or more recognized objects depicted in the digital image to the semantic meaning of the textual content of the digital document and a relevance of the one or more recognized objects depicted in the digital image to one or more recognized objects in a second digital image that is already in the digital document and that is within a defined proximity to the desired placement position.

4. The method of claim 1, further comprising:

calculating a semantic meaning of the digital image based upon the one or more recognized objects depicted in the digital image; and

wherein automatically determining the second size of the digital image comprises;

determining the second size by submitting the semantic meaning of the textual content, the semantic meaning of the digital image, and the one or more recognized objects to a machine-learned model.

5. The method of claim 1, wherein automatically determining the second size of the digital image comprises:

determining the second size based upon the determined relevance of the one or more recognized objects depicted in the digital image to the semantic meaning of the textual content of the digital document and one or more of;

a resolution, an aspect ratio, or a color profile.

6. The method of claim 1, wherein automatically determining the second size of the digital image comprises determining the second size based upon the determined relevance of the one or more recognized objects depicted in the digital image to the semantic meaning of textual content of the digital document and based upon past behaviors of a user that selected the desired placement.

7. A machine-readable medium for automated digital image scaling, the machine-readable medium comprising instructions, which when performed by a machine, causes the machine to perform operations comprising:

receiving a selection of a digital image and a desired placement position in a digital document, the digital image having a first size;

automatically determining a second size of the digital image based upon a determined relevance of one or more recognized objects depicted in the digital image to a semantic meaning of textual content of the digital document, the textual content within a defined proximity to the desired placement position, the second size different than the first size;

automatically scaling the digital image from the first size to the second size; and

automatically placing the scaled digital image in the digital document at the desired placement position.

8. The machine-readable medium of claim 7, wherein the one or more recognized objects depicted in the digital image comprise textual content in the digital image.

9. The machine-readable medium of claim 7, wherein the one or more recognized objects depicted in the digital image comprise recognized faces in the digital image.

10. The machine-readable medium of claim 7, wherein the one or more recognized objects depicted in the digital image comprise one of:

logos, icons, decorations, or clip art.

11. The machine-readable medium of claim 7, wherein the operations of automatically determining the second size of the digital image comprises determining the second size based upon the determined relevance of the one or more recognized objects depicted in the digital image to the semantic meaning of textual content of the digital document and based upon past behaviors of a user that selected the desired placement.

12. A computing device for automated digital image scaling, the computing device comprising:

a processor;

a memory, the memory comprising instructions, which when executed by the processor, cause the computing device to perform operations comprising;

receiving a selection of a digital image and a desired placement position in a digital document, the digital image having a first size;

automatically determining a second size of the digital image based upon a determined relevance of one or more recognized objects depicted in the digital image to a semantic meaning of textual content of the digital document, the textual content within a defined proximity to the desired placement position, the second size different than the first size;

automatically scaling the digital image from the first size to the second size; and

automatically placing the scaled digital image in the digital document at the desired placement position.

13. The computing device of claim 12, wherein the one or more recognized objects depicted in the digital image comprise textual content in the digital image.

14. The computing device of claim 12, wherein the one or more recognized objects depicted in the digital image comprise recognized faces in the digital image.

15. The computing device of claim 12, wherein the one or more recognized objects depicted in the digital image comprise one of:

logos, icons, decorations, or clip art.

16. The computing device of claim 12, wherein the operations of automatically determining the second size of the digital image comprises:

determining the second size based upon the determined relevance of the one or more recognized objects depicted in the digital image to the semantic meaning of the textual content of the digital document and a frequency with which one of the one or more recognized objects are mentioned in the textual content of the digital document.

17. The computing device of claim 12, wherein the operations of automatically determining the second size of the digital image comprises:

determining the second size based upon the determined relevance of the one or more recognized objects depicted in the digital image to the semantic meaning of the textual content of the digital document and a relevance of one or more recognized objects depicted in the digital image to one or more recognized objects in a second digital image that is already in the digital document and that is within a defined proximity to the desired placement position.

18. The computing device of claim 12, wherein the operations further comprise:

calculating a semantic meaning of the digital image based upon the one or more recognized objects depicted in the digital image; and

wherein the operations of automatically determining the second size of the digital image comprises;

determining the second size by submitting the semantic meaning of the textual content, the semantic meaning of the digital image, and the one or more recognized objects to a machine-learned model.

19. The computing device of claim 12, wherein the operations of automatically determining the second size of the digital image comprises:

determining the second size based upon the determined relevance of the one or more recognized objects depicted in the digital image to the semantic meaning of the textual content of the digital document and one or more of;

a resolution, an aspect ratio, or a color profile.

20. The computing device of claim 12, wherein the operations of automatically determining the second size of the digital image comprises determining the second size based upon the determined relevance of the one or more recognized objects depicted in the digital image to the semantic meaning of textual content of the digital document and based upon past behaviors of a user that selected the desired placement.