Touch screen with virtual joystick and methods for use therewith

• US 9,262,073 B2
• Filed: 05/20/2010
• Issued: 02/16/2016
• Est. Priority Date: 05/20/2010
• Status: Active Grant

• Current Assignee: Brian G. Howard, Jeremy R. Howa, John W. Howard
• Original Assignee: Brian G. Howard, Jeremy R. Howa, John W. Howard
• Inventors: Howard, John W., Howard, Brian G., Howa, Jeremy R.
• Primary Examiner(s): Awad, Amr
• Assistant Examiner(s): BRAY, STEPHEN A
• Application Number:
  US12/784,293
• Publication Number:
  US 20110285636A1
• Time in Patent Office: 2,098 Days
• Field of Search: 345/161, 345173-178, 463 30- 34, 463 36- 39, 715/711, 715/773
• US Class Current: 1/1
• CPC Class Codes: A63F 13/2145 the surface being also a display device, e.g. touch screens
• CPC Class Codes: A63F 13/42 by mapping the input signals into game commands, e.g. mapping the displacement of a stylus on a touch screen to the steering angle of a virtual vehicle
• CPC Class Codes: A63F 13/426 involving on-screen location information, e.g. screen coordinates of an area at which the player is aiming with a light gun
• CPC Class Codes: G06F 3/0416 Control or interface arrangements specially adapted for digitisers
• CPC Class Codes: G06F 3/04815 Interaction with a metaphor-based environment or interaction object displayed as three-dimensional, e.g. changing the user viewpoint with respect to the environment or object
• CPC Class Codes: G06F 3/04842 Selection of displayed objects or displayed text elements G06F3/0482 takes precedence
• CPC Class Codes: G06F 3/04883 for inputting data by handwriting, e.g. gesture or text
• CPC Class Codes: G06F 3/04886 by partitioning the display area of the touch-screen or the surface of the digitising tablet into independently controllable areas, e.g. virtual keyboards or menus

Abstract

Joystick display data is generated for display on a touch screen of an electronic device. The joystick display data creates a visual representation of a virtual joystick when displayed on the touch screen. Touch data from the touch screen is processed to determine when the touch data indicates user interaction with the virtual joystick. When the touch data indicates user interaction with the virtual joystick, the joystick display data is adjusted to reflect the user interaction with the virtual joystick, based on the touch data, and joystick data is generated based on the user interaction with the virtual joystick. The display of at least one other element on the touch screen can be adjusted based on the joystick data.

Claims

1. A method for use in conjunction with a handheld device that includes a touch screen, the method comprising:

generating joystick display data for display on the touch screen, wherein the joystick display data creates a visual representation of a virtual joystick when displayed on the touch screen, wherein the visual representation of the virtual joystick includes a boundary and an actuator that moves within the boundary;

processing touch data from the touch screen to determine when the touch data indicates user interaction with the virtual joystick, by;

determining the touch data indicates user interaction with the virtual joystick when the touch data corresponds to a touch of the touch screen inside the boundary; and

determining the touch data indicates user interaction with the virtual joystick when the touch data corresponds to dragging the touch from inside the boundary to outside the boundary, wherein the boundary maintains a fixed position on the display screen;

when the touch data indicates user interaction with the virtual joystick;

adjusting the joystick display data to reflect the user interaction with the virtual joystick, based on the touch data by modifying a position of the actuator to correspond to a position of the touch of the touch screen, wherein the joystick display data includes an indication of the boundary at the fixed position and the joystick display data maintains an actuator position within the boundary regardless of when the touch data corresponds to dragging the touch from inside the boundary to outside the boundary;

generating joystick data based on the user interaction with the virtual joystick, wherein display of a game object of a game on the touch screen is controlled based on the joystick data wherein generating the joystick data based on the user interaction with the virtual joystick includes;

generating orientation data that indicates an orientation of the virtual joystick;

comparing the orientation indicated by the orientation data to a nonlinear capture function that includes a range of orientations about a plurality of advantageous orientations that are advantageous to operation of the game;

generating the joystick data to indicate a selected one of the plurality of advantageous orientations when the orientation data compares favorably to the range of orientations corresponding to the selected one of the plurality of advantageous orientations;

generating the joystick data to indicate the orientation indicated by the orientation data when the orientation data compares favorably to another range of orientations, distinct from the range of orientations corresponding to the plurality of advantageous orientations;

generating the joystick data to indicate a null orientation when the orientation data compares favorably to at least one null range of orientations, distinct from the range of orientations corresponding to the plurality of advantageous orientations and the another range of orientations, wherein the range of orientations about the plurality of advantageous orientations together with the another range of orientations and the at least one null range of orientations correspond to a full 360 degree range of possible orientations of the joystick data; and

displaying visual orientation feedback on the touch screen, based on the joystick data, wherein the visual orientation feedback includes an orientation object that visually indicates an orientation selected by the virtual joystick, and wherein the object is separate from the visual representation of the virtual joystick and is separate from the game object, but displayed in conjunction with an axis centered on the game object.

2. The method of claim 1 wherein display of the game object on the touch screen is controlled to indicate a first mode of motion when the orientation data compares favorably to the range of orientations corresponding to the selected one of the plurality of advantageous orientations;

and

wherein display of the game object on the touch screen is controlled to indicate a second mode of motion when the orientation data compares favorably to the another range of orientations.

3. The method of claim 1 wherein the game object is displayed at the selected one of the plurality of advantageous orientations and the visual orientation feedback is displayed at the orientation indicated by the orientation data.

4. The method of claim 1 further comprising:

generating instruction data for display on the touch screen, wherein the instruction data generates a display of a visual prompt on the touch screen that instructs the user on use of the virtual joystick.

5. The method of claim 4 further comprising:

terminating the display of the visual prompt when the touch data indicates user interaction with the virtual joystick.

6. The method of claim 1 wherein generating joystick display data for display on the touch screen includes modifying a position of the virtual joystick on the touch screen in response to a user selection of the position of the virtual joystick.

7. The method of claim 1 further comprising:

processing the touch data to determine a bias corresponding to the user interaction with the virtual joystick;

wherein generating the joystick data based on the user interaction with the virtual joystick includes bias compensation.

8. The method of claim 1 wherein the virtual joystick includes at least one adjunct button that is selectable in response to user interaction;

wherein processing the touch data further includes determining when the at least one adjunct button is selected; and

wherein generating the joystick data includes generating button selection data when the at least one adjunct button is selected.

9. A method for use in conjunction with an electronic device that includes a touch screen, the method comprising:

generating joystick display data for display on the touch screen, wherein the joystick display data creates a visual representation of a virtual joystick when displayed on the touch screen;

processing touch data from the touch screen to determine when the touch data indicates user interaction with the virtual joystick, and when the touch data indicates user interaction with the virtual joystick;

adjusting the joystick display data to reflect the user interaction with the virtual joystick, based on the touch data, wherein the joystick display data includes an indication of a boundary at a fixed position and the joystick display data maintains an actuator position within the boundary regardless of when the touch data corresponds to dragging the touch from inside the boundary to outside the boundary, and wherein the boundary maintains a fixed position on the display screen; and

generating joystick data based on the user interaction with the virtual joystick, wherein display of a game object of a game on the touch screen is adjusted based on the joystick data, wherein generating the joystick data based on the user interaction with the virtual joystick includes;

generating orientation data that indicates an orientation of the virtual joystick;

comparing the orientation indicated by the orientation data to a nonlinear capture function that includes a range of orientations about a plurality of advantageous orientations that are advantageous to operation of the game;

generating the joystick data to indicate a selected one of the plurality of advantageous orientations when the orientation data compares favorably to the range of orientations corresponding to the selected one of the plurality of advantageous orientations;

generating the joystick data to indicate the orientation indicated by the orientation data when the orientation data compares favorably to another range of orientations, distinct from the range of orientations corresponding to the plurality of advantageous orientations;

generating the joystick data to indicate a null orientation when the orientation data compares favorably to at least one null range of orientations, distinct from the range of orientations corresponding to the plurality of advantageous orientations and the another range of orientations, wherein the range of orientations about the plurality of advantageous orientations together with the another range of orientations and the at least one null range of orientations correspond to a full 360 degree range of possible orientations of joystick data; and

displaying visual orientation feedback on the touch screen, based on the joystick data, wherein the visual orientation feedback includes an orientation object that visually indicates an orientation selected by the virtual joystick, and wherein the object is separate from the visual representation of the virtual joystick and is separate from the game object, but displayed in conjunction with an axis centered on the game object.

10. The method of claim 9 wherein the visual representation of the virtual joystick includes a boundary and wherein processing touch data from the touch screen to determine when the touch data indicates user interaction with the virtual joystick includes determining when the touch data corresponds to a touch of the touch screen inside the boundary.

11. The method of claim 9 wherein processing touch data from the touch screen to determine when the touch data indicates user interaction with the virtual joystick includes determining when the touch data corresponds to a touch of the touch screen inside the boundary and further corresponds to dragging the touch outside the boundary.

12. The method of claim 9 wherein the visual representation of the virtual joystick includes an actuator and wherein adjusting the joystick display data to reflect the user interaction with the virtual joystick includes modifying a position of the actuator to correspond to a position of a touch of the touch screen.

13. The method of claim 9 wherein display of the game object on the touch screen is controlled to indicate a first mode of motion when the orientation data compares favorably to the range of orientations corresponding to the selected one of the plurality of advantageous orientations;

and

wherein display of the game object on the touch screen is controlled to indicate a second mode of motion when the orientation data compares favorably to the another range of orientations.

14. The method of claim 13 wherein the game object is displayed at the selected one of the plurality of advantageous orientations and the visual orientation feedback is displayed at the orientation indicated by the orientation data.

15. The method of claim 9 further comprising:

generating instruction data for display on the touch screen, wherein the instruction data generates a display of a visual prompt on the touch screen that instructs the user on use of the virtual joystick.

16. The method of claim 15 further comprising:

terminating the display of the visual prompt when the touch data indicates user interaction with the virtual joystick.

17. The method of claim 9 wherein generating joystick display data for display on the touch screen includes modifying a position of the virtual joystick on the touch screen in response to a user selection of the position of the virtual joystick.

18. The method of claim 9 further comprising:

processing the touch data to determine a bias corresponding to the user interaction with the virtual joystick;

wherein generating the joystick data based on the user interaction with the virtual joystick includes bias compensation.

19. The method of claim 18 wherein the processing the touch data to determine the bias includes determining a difference between at least two touch positions indicated by the touch data.

20. The method of claim 9 wherein the virtual joystick includes at least one adjunct button that selectable in response to user interaction;

wherein processing the touch data further includes determining when the at least one adjunct button is selected; and

wherein generating the joystick data includes generating button selection data when the at least one adjunct button is selected.

21. The method of claim 9 wherein the joystick display data creates a visual representation of a plurality of virtual joysticks.

22. The method of claim 9 wherein the joystick data indicates one of n discrete directions, where n is an integer.

23. The method of claim 9 wherein adjusting the display of the game object includes moving a background.

24. The method of claim 9 further comprising:

adjusting the joystick display data, in response to user interaction with the touch screen, to reposition the virtual joystick to a new location on the touch screen.