Method and system for generating comic panels

• US 6,069,622 A
• Filed: 03/08/1996
• Issued: 05/30/2000
• Est. Priority Date: 03/08/1996
• Status: Expired due to Term

• Current Assignee: Microsoft Technology Licensing LLC (Microsoft Corporation)
• Original Assignee: Microsoft Corporation
• Inventors: Kurlander, David J.
• Primary Examiner(s): SAX, STEVEN PAUL
• Application Number:
  US08/612,560
• Time in Patent Office: 1,544 Days
• Field of Search: 345/326, 345/327, 345/328, 345/329, 345/330, 345/340, 345/341, 345/342, 345/351, 345/348, 345/331-332, 345/333, 345/334, 345/335, 345/336, 345/337, 345/338, 345/339 , 345/343, 345/349, 345/345, 345/346, 345/347, 345/350, 345/352, 345/117
• US Class Current: 715/753
• CPC Class Codes: G06T 11/60 Editing figures and text; Combining figures or text
• CPC Class Codes: H04L 12/1827 Network arrangements for conference optimisation or adaptation

Abstract

A comic generation system controls the generation of comic panels. A comic panel is a graphical representation of an instance in time of a sequential course of events. Each graphical representation typically includes a graphical representation of at least one character. The character has gestures that reflect a body pose and expressions that reflect a facial pose. Moreover, the character may have text associated with it (i.e., the character "speaks"), which is displayed in the comic panel within a bounded area called a "balloon." The comic generation system receives an interaction event and generates a comic panel. Examples of interaction events include text input by a user and selection of gestures and expressions by a user. When the interaction event provides text, the comic generation system automatically searches the text to identify any words or phrases that give rise to gestures and expressions for the character representing the user who provided the interaction event. Moreover, the comic generation system searches the text for trigger words that indicate alterations to the comic panel. The comic generation system generates the comic panel based on the results of its searches. The comic generation system also determines which characters to place in a comic panel, the positions of the characters, and the orientations of the characters. The comic generation system additionally determines the placement of balloons containing text. Also, the comic generation system determines when to generate a new comic panel and when to modify a comic panel that has already been generated. Furthermore, the comic generation system maintains a graphical history of the comic panels.

Claims

1. In a network including a plurality of data processing systems each having an associated display device, a method comprising:

(a) receiving at each of the data processing systems an interaction event generated by any of the data processing systems;

(b) automatically generating a comic panel based on the received interaction event, the comic panel providing a graphical representation of an instance in time of a sequential course of events;

(c) displaying the generated comic panel on each of the display devices associated with the data processing systems;

(d) when an input associated with a graphical representation of a character is received, automatically generating a balloon that includes text that corresponds to the received input associated with the character and automatically generating a tail that is positioned between a position of the balloon and another position for the graphical representation of the character in a current comic panel, the balloon, tail and graphical representation of the character being automatically disposed at positions that are non-overlapping of any other positions for balloons, tails and graphical representations of characters that were previously positioned for display in the current comic panel; and

(e) when the input associated with the character is received and non-overlapping positions for displaying the balloon, tail and graphical representation of the character in the current comic panels are unavailable, automatically displaying a new comic panel that includes the balloon, tail and graphical representation of the character, wherein the balloon, tail and graphical representation of the character are disposed at separate positions that are non-overlapping in the display of the new comic panel.

2. The method of claim 1, wherein automatically generating each comic panel further comprises automatically generating each comic panel based upon a last generated comic panel.

3. The method of claim 1, wherein each of the data processing systems includes a plurality of comic panels and a scroll bar displayed on its associated display device for scrolling through the plurality of comic panels, the scroll bar including a scroll box, the scroll box having a position in the scroll bar, the position corresponding to one of the plurality of comic panels, further comprising displaying the plurality of comic panels based on the position of the scroll box in the scroll bar.

4. The method of claim 1, wherein each character has a body portion with at least one body feature and wherein the interaction event provides a gesture indicating a modification of a body feature of one of the characters.

5. The method of claim 1, wherein each character has a head portion with at least one facial feature and wherein the interaction event provides an expression indicating a modification of a facial feature of one of the characters.

6. The method of claim 1, wherein the interaction event provides a modification of a semantic element.

8. The method of claim 1, wherein providing a plurality of comic panels further comprises displaying the plurality of comic panels on each display device.

9. The method of claim 2, wherein each data process system is enabled to provide a scroll bar displayed on the display device for scrolling through the plurality of comic panels, the scroll bar including a scroll box, the scroll box having a position in the scroll bar, the position corresponding to one of the plurality of comic panels, and wherein displaying the plurality of comic panels displays a plurality of comic panels based on the position of the scroll box in the scroll bar.

10. The method of claim 1, wherein generating a comic panel further comprises:

(a) determining whether to generate a new comic panel that reflects the interaction event;

(b) when the determination to generate the new comic panel is affirmative, generating the new comic panel incorporating the modification indicated by the interaction event; and

(c) when the determination to generate the new comic panel is negative, incorporating the modification indicated by the interaction event into a last generated comic panel in the plurality of comic panels.

11. The method of claim 1, wherein each character has a body portion with at least one body feature and wherein the interaction event provides a gesture indicating a modification of a body feature of at least one of the characters.

12. The method of claim 1, wherein each character has a head portion with at least one facial feature and wherein the interaction event provides an expression indicating a modification of a facial feature of at least one of the characters.

13. The method of claim 1, wherein the comic panel includes at least one semantic element and wherein the interaction event provides a modification of a semantic element.

14. The method of claim 1, wherein automatically generating at the data processing systems the comic panel reflecting the interaction event further comprises automatically generating a plurality of comic panels reflecting the interaction event.

16. The method of claim 9, wherein displaying the current comic panel further comprises displaying a predetermined number of a plurality of comic panels including the current comic panel.

17. The method of claim 10, wherein each of the data processing systems includes a scroll bar displayed on its associated display device for scrolling through the plurality of comic panels, the scroll bar including a scroll box, the scroll box having a position in the scroll bar, the position corresponding to one of the plurality of comic panels, and wherein displaying the predetermined number of the plurality of comic panels displays the predetermined number of the plurality of comic panels based on the position of the scroll box in the scroll bar.

19. The method of claim 12, wherein each character has a body portion with at least one body feature, further comprising, after determining which of the characters to display in the new comic panel, performing actions, including:

(a) determining when a gesture indicating a modification of a body feature of a character is to be displayed in the new comic panel; and

(b) when the determination is affirmative, generating the modification of the body feature based on the gesture for display in the new comic panel.

20. The method of claim 12, wherein each character has a head portion with at least one facial feature, further comprising, after determining each character to display in the new comic panel, performing actions, including:

(a) determining when an expression indicating a modification of a facial feature of the character is to be displayed in the new comic panel; and

(b) when the determination is affirmative, generating the modification of the body feature based on the facial feature for display in the new comic panel.

21. The method of claim 12, wherein determining the placement for each character in the new comic panel further comprises determining the placement for each character in the new comic panel based upon the disposition of each character that is communicating in the new comic panel.

22. The method of claim 12, including a prior generated comic panel and wherein determining the placement for each character further comprises determining the placement for each character based upon that character'"'"'s position in the prior generated comic panel.

23. The method of claim 12, wherein determining the orientation for each character further comprises determining the orientation for each character based upon the disposition of each character that is communicating in the new comic panel.

24. The method of claim 12, further comprising, determining a background of the comic panel based upon the interaction event.

25. The method of claim 12, further comprising selecting a camera zoom for the comic panel, the camera zoom indicating the size of the characters in the comic panel.

26. The method of claim 19, further comprising selecting a panel frame for the comic panel.

28. The method of claim 21, wherein providing the comic panel provides a plurality of comic panels.

29. The method of claim 21, wherein the balloon outline indicates a type of textual input received, the type of textual input indicating a manner in which the textual input is to be displayed.

30. The method of claim 21, including providing the comic panel having a plurality of balloon bodies and wherein determining the position of the balloon further comprises determining a position of a balloon body of the balloon so that each balloon body in the comic panel is displayed in an order that the textual input for each character is received.

32. The method of claim 25, wherein displaying the current comic panel, further comprises displaying a plurality of comic panels, including the current comic panel, and wherein displaying the comic panel different from the previous current comic panel, further includes displaying the plurality of comic panels different from a predetermined number of the previously displayed plurality of comic panels.

7. In a first data processing system having a display device, the first data processing system being coupled to at least one additional data processing system having an additional display device, a method, comprising:

(a) providing a plurality of comic panels, the plurality of comic panels representing a sequential course of events, each of the comic panels providing a graphical representation of an instance in time of the sequential course of events, each graphical representation including a graphical representation of at least one character;

(b) receiving an interaction event from any one of the first and additional data processing systems, the interaction event indicating a modification of a comic panel;

(c) automatically generating at each of the first and additional data processing systems the comic panel reflecting the interaction event;

(d) displaying the generated comic panel on each display device of each data processing system;

(e) when an input associated with a character is received, automatically generating a balloon that includes text that corresponds to the received input associated with the character and generating a tail that is positioned between a position of the balloon and another position for the graphical representation of the character in a current comic panel, the balloon, tail and graphical representation of the character being automatically disposed at positions that are non-overlapping of any other positions for balloons, tails and graphical representations of characters that were previously positioned for display in the current comic panel; and

(f) when the input associated with the character is received and non-overlapping positions for displaying the balloon, tail and graphical representation of the character in the current comic panel are unavailable, automatically displaying a new comic panel that includes the balloon, tail and graphical representation of the character, wherein the balloon, tail and graphical representation of the character are disposed at separate positions that are non-overlapping in the display of the new comic panel.

15. In a data processing system that is coupled to at least one other data processing system, wherein each of the data processing systems includes an associated display, a method comprising:

(a) providing at least one comic panel, each comic panel providing a graphical representation of an instance in time of a sequential course of events, each graphical representation including a graphical representation of at least one character, each character being adapted to represent a user of one of the data processing systems;

(b) displaying a current comic panel on each of the display devices, the current comic panel representing a most recent event relative to each other event in the sequential course of events;

(c) receiving an interaction event from any of the data processing systems, the interaction event indicating a modification of at least one comic panel;

(d) after receiving the interaction event from any one of the data processing systems, performing actions including;

(i) determining when a new comic panel incorporating the modification indicated by the interaction event is to be generated;

(ii) when the determination to generate the new comic panel is affirmative, generating the new comic panel incorporating the modification indicated by the interaction event; and

(iii) when the determination to generate the new comic panel is negative, modifying the current comic panel to incorporate the modification indicated by the interaction event; and

(e) displaying on each display associated with the data processing systems the comic panel incorporating the modification indicated by the interaction event;

(f) when an input associated with a character is received, automatically generating a balloon that includes text that corresponds to the received input associated with the character and automatically generating a tail that is positioned between a position of the balloon and another position for the graphical representation of the character in the current comic panel, the balloon, tail and graphical representation of the character being automatically disposed at positions that are non-overlapping of any other positions for balloons, tails and graphical representations of characters that were previously positioned for display in the current comic panel; and

(g) when the input associated with the character is received and non-overlapping positions for displaying the balloon, tail and graphical representation of the character in the current comic panel are unavailable, automatically generating the new comic panel that includes the balloon, tail and graphical representation of the character, wherein the balloon, tail and graphical representation of the character are disposed at separate positions that are non-overlapping in the display of the new comic panel.

18. In a data processing system having a display, a method comprising:

(a) determining that a new comic panel is to be generated, the comic panel providing a graphical representation of an instance in time of a sequential course of events, each graphical representation including a graphical representation of at least one character, the comic panel thereby being capable of containing a plurality of characters; and

(b) upon determining that the new comic panel is to be generated, performing actions, including;

(i) determining each character to display in the new comic panel;

(ii) determining a placement for each character in the new comic panel, the placement indicating a sequential position of a character relative to each other character in the comic panel;

(iii) determining an orientation for each character relative to each other character in the new comic panel;

(iv) generating the new comic panel;

(v) displaying the generated new comic panel on the display device at the data processing system;

(vi) when an input associated with a character is received, automatically generating a balloon that includes text that corresponds to the received input associated with the character and automatically generating a tail that is positioned between a position of the balloon and another position for the graphical representation of the character in a current comic panel, the balloon, tail and graphical representation of the character being automatically disposed at positions that are non-overlapping of any other positions for balloons, tails and graphical representations of characters that were previously positioned for display in the current comic panel; and

(vii) when the input associated with the character is received and non-overlapping positions for displaying the balloon, tail and graphical representation of the character in the current comic panel are unavailable, automatically displaying the new comic panel that includes the balloon, tail and graphical representation of the character, wherein the balloon, tail and graphical representation of the character are disposed at separate positions that are non-overlapping in the display of the new comic panel.

27. In a data processing system having a display, a method comprising:

(a) providing a comic panel that provides a graphical representation of an instance in time of a sequential course of events, the graphical representation including a graphical representation of at least one character, the comic panel thereby being capable of containing more than one character; and

(b) when textual input for a character is received, performing actions, including;

(i) placing the received textual input within a balloon having a balloon outline, the balloon outline including a balloon body containing the textual input and a balloon tail indicating the character to which the textual input corresponds;

(ii) automatically determining a position of the balloon containing the textual input and another position of the balloon tail in the comic panel, the position of the balloon and the other position of the balloon tail being non-overlapping of the display of any other balloons, balloon tails and graphical representations of characters already positioned in the comic panel, and

automatically displaying the balloon containing the textual input and the balloon tail in the comic panel at non-overlapping positions; and

(iii) when non-overlapping positions for displaying the balloon and the balloon tail in the comic panel are unavailable, automatically displaying a new comic panel that includes the balloon, tail and graphical representation of the character, wherein the balloon, tail and graphical representation of the character are disposed at separate positions that are non-overlapping in the display of the new comic panel.

31. In a network including a plurality of data processing systems each having an associated display device, a method comprising:

(a) providing a plurality of comic panels, the plurality of comic panels representing a sequential course of events, each of the comic panels providing a graphical representation of an instance in time of the sequential course of events, each graphical representation including a graphical representation of at least one character;

(b) providing a scroll bar displayed on the display device for scrolling through the plurality of comic panels, the scroll bar including a scroll box, the scroll box having a position in the scroll bar, the position corresponding to one of the plurality of comic panels;

(c) displaying a current comic panel on each of the display devices, the current comic panel representing a most recent event relative to each other event in the sequential course of events, the scroll box on the scroll bar having a current position corresponding to the current comic panel;

(d) receiving an indication to reposition the scroll box on the scroll bar;

(e) displaying the comic panel different from the previous current comic panel on each of the display devices after receiving the indication to reposition the scroll box in the scroll bar;

(f) when an input associated with a character is received, automatically generating a balloon that includes text that corresponds to the received input associated with the character and automatically generating a tail that is positioned between a position of the balloon and another position for the graphical representation of the character in the current comic panel, the balloon, tail and graphical representation of the character being automatically disposed at positions that are non-overlapping of any other positions for balloons, tails and graphical representations of characters that were previously positioned for display in the current comic panel; and

(g) when the input associated with the character is received and non-overlapping positions for displaying the balloon, tail and graphical representation of the character in the current comic panel are unavailable, automatically displaying a new comic panel that includes the balloon, tail and graphical representation of the character, wherein the balloon, tail and graphical representation of the character are disposed at separate positions that are non-overlapping in the display of the new comic panel.

33. A first data processing system connected to a second data processing system, comprising:

(a) first display device associated with the first data processing system;

(b) a second display device associated with the second data processing system;

(c) a comic generation system coupled to the first and second data processing systems for generating comic panels, each comic panel providing a graphical representation of an instance in time of a sequential course of events, each graphical representation including a graphical representation of at least one character, each character being adapted to represent a user of one of the data processing systems, the comic generation system comprising;

(i) a receiver for receiving an interaction event from any one of the data processing systems indicating a modification of a comic panel;

(ii) a generator for generating a new comic panel incorporating the interaction event received by the receiver;

(iii) a displayer for displaying the new comic panel generated by the generator on the first and second displays of the first and second data processing systems; and

(iv) an engine for performing logical actions, comprising;

(A) when the engine determines that an input associated with a character is received, causing the generator to automatically generate a balloon that includes text that corresponds to the received input associated with the character and to automatically generate a tail that is positioned between a position of the balloon and another position for the graphical representation of the character in a current comic panel, the balloon, tail and graphical representation of the character being automatically displayed by the displayer at positions that are non-overlapping of any other positions for balloons, tails and graphical representations of characters that were previously positioned for display in the current comic panel; and

(B) when the engine determines that the input associated with the character is received and non-overlapping positions for displaying the balloon, tail and graphical representation of the character in the current comic panel are unavailable, employing the generator to automatically generate the new comic panel that includes the balloon, tail and graphical representation of the character, wherein the balloon, tail and graphical representation of the character are disposed at separate positions that are non-overlapping in the display of the new comic panel.

34. In a data processing system having a display device, a server application program, and a client application program, a method comprising:

(a) receiving at the server application program an interaction event from the client application program;

(b) automatically generating with the server application program a comic panel based on the received interaction event, the comic panel providing a graphical representation an instance in time of a sequential course of events, the graphical representation including at least one graphical representation of a character;

(c) forwarding from the server application program to the client application program the generated comic panel;

(d) when an input associated with a graphical representation of a character is received by the client program, employing the server program to automatically generate a balloon that includes text that corresponds to the received input associated with the character and to automatically generate a tail that is positioned between a position of the balloon and another position for the graphical representation of the character in a current comic panel, the balloon, tail and graphical representation of the character being automatically disposed by the client program at positions that are non-overlapping of any other positions for balloons, tails and graphical representations of characters that were previously positioned for display in the current comic panel; and

(e) when the input associated with the character is received by the client program and non-overlapping positions for displaying the balloon, tail and graphical representation of the character in the current comic panel are unavailable, employing the server program to automatically generate the new comic panel that includes the balloon, tail and graphical representation of the character, wherein the balloon, tail and graphical representation of the character are disposed by the client program at separate positions that are non-overlapping in the display of the new comic panel.

35. The method of claim 34, wherein the data processing system includes a memory for storing a plurality of comic panels and a scroll bar displayed on the display device for scrolling through the plurality of comic panels, the scroll bar including a scroll box, the scroll box having a position in the scroll bar, the position corresponding to one of the plurality of comic panels, and wherein the server application program displays a plurality of comic panels based on the position of the scroll box in the scroll bar upon request by the client application program.

36. In a data processing system having a display device, a server application program, and a client application program, the data processing system being coupled to least one other data processing system, each coupled data processing system having a client application program, a method comprising:

(a) under control of the client application program, performing actions, including;

(i) receiving an interaction event;

(ii) generating a comic panel, the comic panel providing a graphical representation of an instance in time of a sequential course of events, each graphical representation including a graphical representation of at least one character;

(iii) displaying the generated comic panel at the data processing system; and

(iv) forwarding the received interaction event to the server application program; and

(b) under control of the server application program, performing actions, including;

(i) receiving the forwarded interaction event from the client application program;

(ii) forwarding the received interaction event to each other client application program at each connected data processing system;

(iii) when an input associated with a character is received by the client application program, automatically generating a balloon that includes text that corresponds to the received input associated with the character and automatically generating a tail that is positioned between a position of the balloon and another position for the graphical representation of the character displayed by the client application program in a current comic panel, the balloon, tail and graphical representation of the character being automatically disposed at positions that are non-overlapping of any other positions for balloons, tails and graphical representations of characters that were previously positioned for display by the client application program in the current comic panel; and

(vi) when the input associated with the character is received by the client application program and non-overlapping positions for displaying the balloon, tail and graphical representation of the character in the current comic panel are unavailable, having the client application program automatically display a new comic panel that includes the balloon, tail and graphical representation of the character, wherein the balloon, tail and graphical representation of the character are disposed at separate positions that are non-overlapping in the display of the new comic panel.

37. The method of claim 36, wherein under control of each client application program at each coupled data processing system, further comprising:

(a) receiving the forwarded interaction event from the server application program;

(b) generating the comic panel, the comic panel providing a graphical representation of an instance in time of a sequential course of events, each graphical representation of the instance in time including the graphical representation of at least one character; and

(c) displaying the generated comic panel at each data processing system.

38. A computer-readable storage medium, upon which is stored a comic generation program for automatically generating comic panels among a plurality of data processing systems, the comic generation program performing actions, comprising:

(a) displaying at least one comic panel, each comic panel providing a graphical representation of an instance in time of a sequential course of events;

(b) receiving an interaction event from any of the data processing systems indicating a modification of one of the comic panels;

(c) generating a comic panel based on the received interaction event;

(d) displaying the generated comic panel on a display device;

(e) when an input associated with a graphical representation of a character is received, automatically generating a balloon that includes text that corresponds to the received input associated with the character and automatically generating a tail that is positioned between a position of the balloon and another position for the graphical representation of the character in a current comic panel, the balloon, tail and graphical representation of the character being automatically disposed at positions that are non-overlapping of any other positions for balloons, tails and graphical representation of characters that were previously positioned for display in the current comic panel; and

(f) when the input associated with the character is received and non-overlapping positions for displaying the balloon, tail and graphical representation of the character in the current comic panel are unavailable, automatically displaying a new comic panel that includes the balloon, tail and graphical representation of the character, wherein the balloon, tail and graphical representation of the character are disposed at separate positions that are non-overlapping in the display of the new comic panel.