Method for layout of speech bubbles associated with characters in an image

• US 9,342,487 B2
• Filed: 09/04/2013
• Issued: 05/17/2016
• Est. Priority Date: 09/04/2013
• Status: Active Grant

• Current Assignee: Adobe Inc.
• Original Assignee: Adobe Systems Incorporated (Adobe Inc.)
• Inventors: Georgiev, Evtim Ivanov
• Primary Examiner(s): RIES, LAURIE ANNE
• Application Number:
  US14/017,633
• Publication Number:
  US 20150067482A1
• Time in Patent Office: 986 Days
• US Class Current: 1/1
• CPC Class Codes: G06F 3/167 Audio in a user interface, e.g. using voice commands for navigating, audio feedback
• CPC Class Codes: G06F 40/103 Formatting, i.e. changing of presentation of documents automatic justification G06F40/189; automatic line break hyphenation G06F40/191
• CPC Class Codes: G06F 40/106 Display of layout of documents; Previewing
• CPC Class Codes: G06T 11/60 Editing figures and text; Combining figures or text

Abstract

A system and method of speech bubbles layout are described. A context module determines geometric constraints of speech bubbles for characters in an image and features of the characters in the image, receives a speech content for one or more characters, and identifies a conversation order of the characters. A layout module generates a layout of the speech bubbles based on the features of the characters, the speech content, and the conversation order. The layout of the speech bubbles is within the geometric constraints of the speech bubbles in the image.

Claims

1. A method comprising:

determining geometric constraints for speech bubbles within an image, the image including characters, wherein the geometric constraints are based on boundaries of the characters within the image and boundaries of the image;

determining features of the characters in the image;

receiving speech content for the characters;

identifying a conversation order of the speech content with respect to the characters; and

generating a layout of the speech bubbles that defines a location and size of each speech bubble, with respect to the image, based on the geometric constraints, the features of the characters, the speech content, and the conversation order.

2. The method of claim 1,

wherein determining features of the characters comprises:

identifying each of the characters in the image;

identifying a position of a mouth of each identified character;

identifying an orientation of a face of each identified character; and

determining a speech source of each identified character based on the respective position of the mouth and the respective orientation of the face for each identified character, wherein generating the layout of the speech bubbles is further based on the respective speech source for each identified character.

3. The method of claim 1, wherein the layout of the speech bubbles further defines a shape of the speech bubbles, and an identification of a position, size, and shape of text of the speech content within the speech bubbles.

4. The method of claim 1, further comprising:

receiving user-defined constraints for one or more of the speech bubbles, the user-defined constraints defining a specific location, shape, or size of the one or more of the speech bubbles

wherein generating the layout of the speech bubbles is further based on the user-defined constraints.

5. The method of claim 1, further comprising:

receiving input modifying the speech content of the characters; and

adjusting the layout of the speech bubbles based on the modified speech content of the characters.

6. The method of claim 1, further comprising:

based on the geometric constraints, identifying regions of importance in the image not to be obscured by the speech bubbles;

for each speech bubble, generating a plurality of frames, each frame of a corresponding speech bubble identifying different possible positions and sizes for the corresponding speech bubble; and

generating a ranking value for each frame of a corresponding speech bubble, the ranking value determined based on whether a frame overlaps the identified regions of importance.

7. The method of claim 6, further comprising:

identifying one or more frames for each speech bubble such that the speech bubbles do not overlap each other, and a relative position of each speech bubble with respect to an adjacent speech bubble is consistent with the order of conversation of the characters.

8. The method of claim 7, wherein generating the layout of the speech bubbles is based further on the ranking value for each identified frame.

9. The method of claim 1, wherein the speech content identifies a font type and a font size for text of the speech content wherein generating the layout of the speech bubbles is further based on the identified font type and font size.

10. A speech bubble layout system comprising:

a context module, executed with at least one processor, to;

determine geometric constraints for speech bubbles within an image, the image including characters;

determine features of the characters in the image;

receive a list of speech content for the characters; and

identify a conversation order of the characters based on an order of the list of speech content; and

a layout module to;

generate a layout of the speech bubbles that defines a location and size of each speech bubble, with respect to the image, based on the geometric constraints, the features of the characters, the speech content, and the conversation order.

11. The speech bubble layout system of claim 10,

wherein the geometric constraints are based on boundaries of the image and boundaries of the characters,

wherein to determine the features of the characters comprises:

identification of each character in the image;

identification of a speech source for each identified character based on a position of a mouth of each identified character, and an orientation of a face of each identified character, wherein to generate the layout of the speech bubbles is further based on the respective speech source for each identified character.

12. The speech bubble layout system of claim 10, wherein the layout of the speech bubbles further defines a shape of the speech bubbles, and an identification of a position, size, and shape of text of the speech content within the speech bubbles.

13. The speech bubble layout system of claim 10, wherein the context module is configured to:

receive user-defined constraints for one or more of the speech bubbles, the user-defined constraints defining a specific location, shape, or size of the one or more of the speech bubbles, wherein the layout module is to generate the layout of the speech bubbles based further on the user-defined constraints.

14. The speech bubble layout system of claim 10, wherein the layout module is configured to:

receive input that modifies the speech content of the characters; and

adjust the layout of the speech bubbles based on the modified speech content of the characters.

15. The speech bubble layout system of claim 10, wherein the layout module is configured to:

based on the geometric constraints, identify regions of importance in the image not to be obscured by the speech bubbles;

for each speech bubble, generate a plurality of frames, each frame of a corresponding speech bubble identifying different possible positions and sizes for the corresponding speech bubble; and

generate a ranking value for each frame of a corresponding speech bubble, the ranking value determined based on whether a frame overlaps the identified regions of importance.

16. The speech bubble layout system of claim 15, wherein the layout module is configured to:

identify one or more frames for each speech bubble such that the speech bubbles do not overlap each other, and a relative position of each speech bubble with respect to an adjacent speech bubble is consistent with the order of conversation of the characters.

17. The speech bubble layout system of claim 16, wherein to

generate the layout of the speech bubbles is based further on the ranking value for each identified frame.

18. The speech bubble layout system of claim 10, wherein the speech content identifies a font type and a font size for text of the speech content, wherein the layout module is to generate the layout of the speech bubbles based further on the identified font type and font size.

19. A non-transitory machine-readable storage medium storing instructions which, when executed by at least one processor, cause the at least one processor to:

determine geometric constraints for speech bubbles within an image, the image including characters, wherein the geometric constraints are based on boundaries of the characters within the image and boundaries of the image;

determine features of the characters in the image;

receive a list of speech content for the characters;

identify a conversation order of the characters based on an order of the list of speech content; and

generate a layout of the speech bubbles that defines a location and size of each speech bubble, with respect to the image, based on the geometric constraints, the features of the characters, the speech content, and the conversation order.