METHOD FOR LAYOUT OF SPEECH BUBBLES ASSOCIATED WITH CHARACTERS IN AN IMAGE

• US 20150067482A1
• Filed: 09/04/2013
• Published: 03/05/2015
• Est. Priority Date: 09/04/2013
• Status: Active Grant

• Current Assignee: Adobe Inc.
• Original Assignee: Adobe Systems Incorporated (Adobe Inc.)
• Inventors: Georgiev, Evtim Ivanov
• Granted Patent: US 9,342,487 B2
• Time in Patent Office: Days
• US Class Current: 715/244
• CPC Class Codes: G06F 3/167 Audio in a user interface, e.g. using voice commands for navigating, audio feedback
• CPC Class Codes: G06F 40/103 Formatting, i.e. changing of presentation of documents automatic justification G06F40/189; automatic line break hyphenation G06F40/191
• CPC Class Codes: G06F 40/106 Display of layout of documents; Previewing
• CPC Class Codes: G06T 11/60 Editing figures and text; Combining figures or text

Abstract

A system and method of speech bubbles layout are described. A context module determines geometric constraints of speech bubbles for characters in an image and features of the characters in the image, receives a speech content for one or more characters, and identifies a conversation order of the characters. A layout module generates a layout of the speech bubbles based on the features of the characters, the speech content, and the conversation order. The layout of the speech bubbles is within the geometric constraints of the speech bubbles in the image.

Claims

1. A method comprising:

determining geometric constraints of speech bubbles for characters in an image;

determining features of the characters in the image;

receiving a speech content for one or more characters;

identifying a conversation order of the characters; and

generating a layout of the speech bubbles based on the features of the characters, the speech content, and the conversation order, the layout of the speech bubbles being within the geometric constraints of the speech bubbles in the image.

2. The method of claim 1,

wherein the geometric constraints comprise visible areas for speech bubbles in the image, the visible areas defined by boundaries of the image and the characters,

wherein the features of the characters comprise an identification of each character in the image, an identification of a speech source for each identified character based on a position of a mouth of each identified character, and an orientation of a face of each identified character.

3. The method of claim 1, wherein the layout comprises an identification of a position, size, and shape of the speech bubbles, and an identification of a position, size, and shape of text corresponding to the speech content within the speech bubbles.

4. The method of claim 1, further comprising:

receiving user-defined constraints of the speech bubbles; and

generating the layout of the speech bubbles based on the user-defined constraints of the speech bubbles.

5. The method of claim 1, further comprising:

modifying the speech content of the characters; and

adjusting the layout of the speech bubbles based on the modified speech content of the characters.

6. The method of claim 1, further comprising:

identifying regions of importance in the image not to be obscured by the speech bubbles;

for each speech bubble, generating a text layout and a speech bubble size based on the identified regions of importance;

for each speech bubble, generating a plurality of frames, each frame comprising the text layout, the speech bubble size, the identified character, the speech source, and a speech bubble type; and

generating a ranking value for each frame of a corresponding speech bubble, the ranking value determined based on whether a frame overlaps the identified regions of importance.

7. The method of claim 6, further comprising:

determining a relative rank from the relative positions of the frames based on an order of conversation of the characters; and

identifying one frame for each speech bubble such that the speech bubbles do not overlap each other, and the relative rank of each speech bubble is consistent with the order of conversation of the characters.

8. The method of claim 7, further comprising:

generating a ranked plurality of layouts of the speech bubbles based on the ranking value for each identified frame.

9. The method of claim 1, wherein the speech content includes a font type of the content and a size of the content.

10. The method of claim 1, wherein the geometric constraints of the speech bubbles are based on speech bubble geometric dimensions.

11. A speech bubble layout system comprising:

a context module, executed with at least one processor, configured to determine geometric constraints of speech bubbles for characters in an image, to determine features of the characters in the image, to receive a speech content for one or more characters, and to identify a conversation order of the characters; and

a layout module configured to generate a layout of the speech bubbles based on the features of the characters, the speech content, and the conversation order, the layout of the speech bubbles within the geometric constraints of the speech bubbles in the image.

12. The speech bubble layout system of claim 11,

wherein the geometric constraints comprise visible areas for speech bubbles in the image, the visible areas defined by boundaries of the image and the characters,

wherein the features of the characters comprise an identification of each character in the image, an identification of a speech source for each identified character based on a position of a mouth of each identified character, and an orientation of a face of each identified character.

13. The speech bubble layout system of claim 11, wherein the layout comprises an identification of a position, size, and shape of the speech bubbles, and an identification of a position, size, and shape of text corresponding to the speech content within the speech bubbles.

14. The speech bubble layout system of claim 11, wherein the context module is configured to:

receive user-defined constraints of the speech bubbles, wherein the layout module is configured to generate the layout of the speech bubbles based on the user-defined constraints of the speech bubbles.

15. The speech bubble layout system of claim 11, wherein the layout module is configured to:

modify the speech content of the characters; and

adjust the layout of the speech bubbles based on the modified speech content of the characters.

16. The speech bubble layout system of claim 11, wherein the layout module is configured to:

identify regions of importance in the image not to be obscured by the speech bubbles;

for each speech bubble, generate a text layout and a speech bubble size based on the identified regions of importance;

for each speech bubble, generate a plurality of frames, each frame comprising the text layout, the speech bubble size, the identified character, the speech source, and a speech bubble type; and

generate a ranking value for each frame of a corresponding speech bubble, the ranking value determined based on whether a frame overlaps the identified regions of importance.

17. The speech bubble layout system of claim 16, wherein the layout module is configured to:

determine a relative rank from relative positions of the frames based on an order of conversation of the characters; and

identify one frame for each speech bubble such that the speech bubbles do not overlap each other, and the relative rank of each speech bubble is consistent with the order of conversation of the characters.

18. The speech bubble layout system of claim 17, wherein the layout module is configured to:

generate a ranked plurality of layouts of the speech bubbles based on the ranking value for each identified frame.

19. The speech bubble layout system of claim 11, wherein the speech content includes a font type of the content and a size of the content, wherein the geometric constraints of the speech bubbles are based on speech bubble geometric dimensions.

20. A non-transitory machine-readable storage medium storing instructions which, when executed by at least one processor, performs operations comprising:

determining geometric constraints of speech bubbles for characters in an image;

determining features of the characters in the image;

receiving a speech content for one or more characters;

identifying a conversation order of the characters; and

generating a layout of the speech bubbles based on the features of the characters, the speech content, and the conversation order, the layout of the speech bubbles being within the geometric constraints of the speech bubbles in the image.