TECHNIQUES FOR IMPROVED USER INTERFACE HELPING SUPER GUIDES

• US 20120021840A1
• Filed: 06/14/2011
• Published: 01/26/2012
• Est. Priority Date: 06/14/2010
• Status: Active Grant

• Current Assignee: Nintendo Company Limited
• Original Assignee: Nintendo Company Limited
• Inventors: SHIMURA, Yukimi, JOHNSON, Jonathan
• Granted Patent: US 10,010,793 B2
• Time in Patent Office: Days
• US Class Current: 463/43
• CPC Class Codes: A63F 13/2145 the surface being also a display device, e.g. touch screens
• CPC Class Codes: A63F 13/26 having at least one additional display device, e.g. on the game controller or outside a game booth
• CPC Class Codes: A63F 13/49 Saving the game status; Pausing or ending the game
• CPC Class Codes: A63F 13/497 Partially or entirely replaying previous game actions
• CPC Class Codes: A63F 13/5375 for graphically or textually suggesting an action, e.g. by displaying an arrow indicating a turn in a driving game
• CPC Class Codes: A63F 13/60 Generating or modifying game content before or while executing the game program, e.g. authoring tools specially adapted for game development or game-integrated level editor
• CPC Class Codes: A63F 13/63 by the player, e.g. authoring using a level editor
• CPC Class Codes: A63F 13/69 by enabling or updating specific game elements, e.g. unlocking hidden features, items, levels or versions
• CPC Class Codes: A63F 13/822 Strategy games; Role-playing games A63F13/825, A63F13/828 take precedence
• CPC Class Codes: A63F 13/92 Video game devices specially adapted to be hand-held while playing
• CPC Class Codes: A63F 2300/1075 using a touch screen
• CPC Class Codes: A63F 2300/305 for providing a graphical or textual hint to the player
• CPC Class Codes: A63F 2300/634 for replaying partially or entirely the game actions since the beginning of the game

Abstract

User interface approaches to providing help to players by playing back previously recorded inputs. Certain exemplary embodiments involve displaying, with the recorded playback, an input device icon to show where the inputs are occurring and/or where the inputs should occur if the player were to make them. Modifications are made to the previously recorded input to improve the user'"'"'s ability to see what is happening, as well as making the playback seem more consistent, mechanical, and automatic. The help makes the computer application seem like the user interface is operating itself in a way that is easy to follow, rather than giving the impression that a previously recorded user input stream simply is being played back in accordance with the more organic and potentially idiosyncratic behaviors of the recorded person. The modifications may in certain exemplary embodiments include, for example, path smoothing/correction, automatic snapping, delay periods, speed normalization, etc.

Claims

1. A non-transitory computer readable storage medium tangibly storing instructions that are executable by a processor, at least to:

enable a user to play a game by running a game program;

receive a request for help from the user from within the game program;

determine a current state of the game program;

determine a next appropriate action for the user based on the current state and the request for help; and

cause an annotated display of modified pre-recorded input or series of inputs, selected from a catalog of pre-recorded inputs and/or series of inputs, based on the next appropriate action for the user, to guide the user in accordance with the request for help.

2. The non-transitory computer readable storage medium of claim 1, wherein the modified pre-recorded input or series of inputs involve one or more of:

a prolongation of at least a portion of the pre-recorded input or series of inputs so that the corresponding annotated display is visible for at least a predetermined amount of time;

a smoothing and/or filtering of at least a portion of the pre-recorded input or series of inputs so that the corresponding annotated display lacks jitters that otherwise would be present;

a removal of at least a portion of the pre-recorded input or series of inputs so that the corresponding annotated display proceeds more directly from a start point to an end point; and

a normalization of at least a portion of the pre-recorded input or series of inputs so that the corresponding annotated display proceeds at either a predetermined pace and/or within a predetermined amount of time.

3. The non-transitory computer readable storage medium of claim 2, wherein modifications to the pre-recorded input or series of inputs are processed and stored ahead of time so that annotated displays are retrievable from a storage location.

4. The non-transitory computer readable storage medium of claim 2, wherein modifications to the pre-recorded input or series of inputs are processed upon requests for help.

5. The non-transitory computer readable storage medium of claim 1, wherein the game program is at least temporarily paused during the annotated display of modified pre-recorded input or series of inputs.

6. The non-transitory computer readable storage medium of claim 1, wherein the game program is output to a first display device and the annotated display is output to a second display device.

7. The non-transitory computer readable storage medium of claim 1, wherein the modified pre-recorded input or series of inputs involves a prolongation of at least a portion of the pre-recorded input or series of inputs so that the corresponding annotated display is visible for at least a predetermined amount of time.

8. The non-transitory computer readable storage medium of claim 1, wherein the modified pre-recorded input or series of inputs involves a smoothing and/or filtering of at least a portion of the pre-recorded input or series of inputs so that the corresponding annotated display lacks jitters that otherwise would be present.

9. The non-transitory computer readable storage medium of claim 1, wherein the modified pre-recorded input or series of inputs involves a removal of at least a portion of the pre-recorded input or series of inputs so that the corresponding annotated display proceeds more directly from a start point to an end point.

10. The non-transitory computer readable storage medium of claim 1, wherein the modified pre-recorded input or series of inputs involves a normalization of at least a portion of the pre-recorded input or series of inputs so that the corresponding annotated display proceeds at either a predetermined pace or within a predetermined amount of time.

11. The non-transitory computer readable storage medium of claim 1, wherein the modified pre-recorded input or series of inputs involves adjusting a line so as to snap the line to a grid having cells that are one or more pixels in each dimension such that a straighter line and/or segment of line is/are provided.

12. A method of guiding a user of an application program running on a computer system, the method comprising:

enabling the user to use the application program on the computer system;

receiving a request for help from the user from within the application program;

determining, via a processor of the computer system, an appropriate next action or next series of actions based on the request for help; and

visually demonstrating to the user how to take the appropriate next action or next series of actions by playing back a modified suggestion of input or series of inputs to simulate the appropriate next action or next series of actions to be taken.

13. The method of claim 12, further comprising determining a current state of the application program, the current state being considered in the determining of the appropriate next action or next series of actions.

14. The method of claim 12, wherein the appropriate next action or next series of actions to be taken is simulated using one or more visual cues demonstrating how the user is to use a physical input device in connection with a user interface of the application program.

15. The method of claim 14, wherein the a modified suggestion of input or series of inputs involve one or more of:

a prolongation of at least a portion of the suggestion of input or series of inputs so that the corresponding simulation is visible for at least a predetermined amount of time;

a smoothing and/or filtering of at least a portion of the suggestion of input or series of inputs so that the corresponding simulation lacks jitters that otherwise would be present therein if the pre-recorded input or series of inputs were reproduced;

a removal of at least a portion of the suggestion of input or series of inputs so that the corresponding simulation proceeds more directly from a start point to an end point that otherwise would be present if the suggestion of input or series of inputs were reproduced; and

a normalization of at least a portion of the suggestion of input or series of inputs so that the corresponding simulation proceeds at either a predetermined pace and/or within a predetermined amount of time.

16. The method of claim 12, further comprising modifying suggestion of input or series of inputs as the application program is being executed by the user.

17. The method of claim 12, wherein suggestion of input or series of inputs are modified and stored to a storage location prior to execution of the application program by the user.

18. A computerized system, comprising:

a memory; and

a processor configured to execute a game program on the computerized system, the game program being programmed to;

receive a request for help from the user,

determine, via the processor, an appropriate next action or next series of actions based on the request for help, and

cause a visual suggestion to be displayed to the user regarding how to use one or more physical input mediums to supply to the game program the appropriate next action or next series of actions, the visual suggestion including the playing back of annotated versions of modified pre-provided input or series of inputs corresponding to the appropriate next action or next series of actions.

19. The system of claim 18, further comprising a non-transitory storage location tangibly storing the annotated version of the modified pre-provided input or series of inputs.

20. The system of claim 19, wherein annotations and/or modifications to the pre-provided input or series of inputs are processed and stored in the storage location ahead of time so that annotated displays are retrievable during game program execution by the user.

21. The system of claim 18, wherein the game program is further programmed to modify the pre-provided input or series of inputs are processed at runtime.

22. The system of claim 18, wherein the game program is further programmed to at least temporarily pause the game program during the visual suggestion.

23. The system of claim 18, further comprising a first display device and at least one second display device,

wherein the game program is output to the first display device and the visual suggestion is output to the at least one second display device.

24. The system of claim 18, further comprising:

a console gaming system;

a first display device connected to the console gaming system on which a common screen of the game program is displayable; and

a plurality of input devices, each said input device being usable by a different user and including a second display device on which an individualized user screen of the game program is displayable.

25. The system of claim 24, wherein a visual suggestion is displayable only on a second display device of the user requesting help.

26. The system of claim 18, wherein the game program is further programmed to overlay the visual suggestion on a main game screen.

27. The system of claim 18, wherein the pre-provided input or series of inputs is/are pre-recorded.

28. The system of claim 18, wherein the pre-provided input or series of inputs is/are simulated.

29. A method of developing a guide for an application program, the method comprising:

capturing user inputs at a predetermined rate;

associating captured user inputs with actions or events in the application program; and

generating, via at least one processor, modification data corresponding to how the captured user inputs are to be modified so that, when reproduced, the user inputs appear more mechanical than the captured user inputs;

wherein the user inputs are reproducible on a display in modified form based on the modification data.

30. The method of claim 29, further comprising storing the modification data together with an indication of the associated action or event.

31. The method of claim 29, further comprising storing to a non-transitory storage computer readable storage medium modified versions of the user inputs, the modified versions of the captured user inputs being based on the modification data and the captured user inputs.

32. The method of claim 31, wherein the modified versions of the captured user inputs are stored together with indications of the associated actions or events.

33. The method of claim 29, wherein the modification data corresponds to an adjustment in a duration of a display of the captured user inputs.

34. The method of claim 29, wherein the modification data corresponds to a smoothing and/or filtering of at least a portion of the captured user inputs, the smoothing and/or filtering reducing an amount of jitters occurring in the captured user inputs.

35. The method of claim 29, wherein the modification data corresponds to a removing at least a portion of the captured user inputs so that a path of the user input appears to proceed more directly from a start point to an end point than it otherwise would.

36. The method of claim 29, wherein the modification data corresponds to a normalization of a rate at which at least a portion of the captured user inputs are to be reproduced.

37. A non-transitory computer readable storage medium tangibly storing instructions that are executable by a processor, at least to:

capture user inputs to an application program at a predetermined rate;

associate captured user inputs with actions or events in the application program; and

generate modification data corresponding to how the captured user inputs are to be modified so that, when reproduced, the user inputs appear more mechanical than the captured user inputs,

wherein the user inputs are reproducible on a display in modified form based on the modification data.

38. A system for developing a guide for an application program, comprising:

a processor and a memory;

a recording module configured to capture user inputs at a predetermined rate;

a recording sequencer configured to associate captured user inputs with actions or events in the application program; and

an annotator configured to generate modification data corresponding to how the captured user inputs are to be modified so that, when reproduced, the user inputs appear more mechanical than the captured user inputs,

wherein the recording module, the recording sequencer, and the annotator are all under control of the processor, and

wherein the user inputs are reproducible on a display in modified form based on the modification data.