Virtual controller for touch display

• US 9,817,569 B2
• Filed: 07/21/2016
• Issued: 11/14/2017
• Est. Priority Date: 12/29/2010
• Status: Active Grant

• Current Assignee: Microsoft Technology Licensing LLC (Microsoft Corporation)
• Original Assignee: Microsoft Technology Licensing LLC (Microsoft Corporation)
• Inventors: Berkes, Otto, Matthews, III, Joseph H., Geiger, Avi
• Primary Examiner(s): Tung, David
• Application Number:
  US15/216,625
• Publication Number:
  US 20160328069A1
• Time in Patent Office: 481 Days
• CPC Class Codes: A63F 13/2145 the surface being also a display device, e.g. touch screens
• CPC Class Codes: A63F 13/218 using pressure sensors, e.g. generating a signal proportional to the pressure applied by the player
• CPC Class Codes: A63F 13/25 Output arrangements for video game devices
• CPC Class Codes: A63F 2300/1075 using a touch screen
• CPC Class Codes: A63F 2300/204 the platform being a handheld device
• CPC Class Codes: A63F 2300/6045 for mapping control signals received from the input arrangement into game commands
• CPC Class Codes: A63F 2300/609 for unlocking hidden game elements, e.g. features, items, levels
• CPC Class Codes: G06F 2203/04104 Multi-touch detection in digitiser, i.e. details about the simultaneous detection of a plurality of touching locations, e.g. multiple fingers or pen and finger
• CPC Class Codes: G06F 2203/04105 Pressure sensors for measuring the pressure or force exerted on the touch surface without providing the touch position
• CPC Class Codes: G06F 2203/04808 Several contacts: gestures triggering a specific function, e.g. scrolling, zooming, right-click, when the user establishes several contacts with the surface simultaneously; e.g. using several fingers or a combination of fingers and pen
• CPC Class Codes: G06F 3/01 Input arrangements or combined input and output arrangements for interaction between user and computer G06F3/16 takes precedence
• CPC Class Codes: G06F 3/03547 Touch pads, in which fingers can move on a surface
• CPC Class Codes: G06F 3/0412 Digitisers structurally integrated in a display
• CPC Class Codes: G06F 3/0414 using force sensing means to determine a position
• CPC Class Codes: G06F 3/04166 Details of scanning methods, e.g. sampling time, grouping of sub areas or time sharing with display driving Synchronisation with the driving of the display or the backlighting unit to avoid interferences generated internally G06F3/04184
• CPC Class Codes: G06F 3/04883 for inputting data by handwriting, e.g. gesture or text
• CPC Class Codes: G06F 3/04886 by partitioning the display area of the touch-screen or the surface of the digitising tablet into independently controllable areas, e.g. virtual keyboards or menus

Abstract

Systems and methods are provided for use with a computing device having a touch sensitive display including a touch sensor configured to detect touches of a digit of a user. The method may include detecting an initial digit down position on the display via the touch sensor, and establishing a neutral position for a virtual controller at the digit down position. The method may further include detecting a subsequent movement of the digit relative to the initial digit down position, and determining a controller input parameter based on the subsequent movement of the digit relative to the initial digit down position. The method may further include generating a controller input message indicating the determined controller input parameter.

Claims

1. An input method for use with a computing device having a touch sensitive display including a touch sensor configured to detect touches of a digit of a user, the method comprising:

establishing a neutral position for a virtual controller at a detected initial digit down position on the touch sensitive display;

establishing an ancillary control zone spaced away from the virtual controller and configured as a virtual button;

detecting a first digit up event subsequent to establishing the neutral position based on at least detecting a lack of contact of the digit of the user for a predetermined threshold period of time;

detecting a first subsequent digit down position in the ancillary control zone;

upon detecting the first subsequent digit down position in the ancillary control zone, inhibiting repositioning of the neutral position;

determining an ancillary control input parameter based on the first subsequent digit down position in the ancillary control zone;

detecting a second digit up event based on at least detecting a lack of contact of the digit of the user for the predetermined threshold period of time;

detecting creep between a second subsequent digit down position not in the ancillary control zone and the initial digit down position at which the neutral position was established; and

repositioning the neutral position for the virtual controller at the second subsequent digit down position not in the ancillary control zone upon detecting the creep.

2. The input method of claim 1, further comprising:

determining a nearest acceptable virtual controller position if the initial digit down position and/or the second subsequent digit down position occurs in a region adjacent a border of the touch sensitive display that would result in an actuable area of the virtual controller being impeded by the border.

3. The input method of claim 1, wherein the digit is a thumb, and the virtual controller is a virtual joystick controller.

4. The input method of claim 1, further comprising determining a controller input parameter based on subsequent movement of the digit relative to the initial digit down position, wherein determining the first controller input parameter includes determining a direction, velocity, and/or relative magnitude of the subsequent movement of the digit relative to the initial digit down position.

5. The input method of claim 1, wherein establishing the neutral position, establishing the ancillary control zone, detecting the first digit up event, detecting the first subsequent digit down position, inhibiting repositioning of the neutral position, determining the ancillary control input parameter, detecting the second digit up event, detecting creep between the second subsequent digit down position not in the ancillary zone and the initial digit down position, and repositioning the neutral position for the virtual controller, are performed by an application programming interface executed on the computing device, the method further comprising:

generating a controller input message including an indication of the ancillary control input parameter; and

outputting the controller input message from the application programming interface to a program executed on the computing device.

6. The input method of claim 1, further comprising:

detecting pressure of the digit against the touch sensitive display during detection of initial digit down position; and

generating a controller input message including an indication of the detected pressure subsequent to establishing the neutral position.

7. The input method of claim 6, wherein detecting pressure includes determining pressure from changes in area of the touch associated with the digit over time and/or determining pressure from a pressure sensor coupled to the touch sensitive display.

8. The input method of claim 1, wherein the ancillary control zone is configured as a virtual button or directional pad.

9. A computing device, comprising a touch sensitive display including a touch sensor configured to detect touches of a digit of a user;

a virtual controller input module executed by a processor of the computing device, the virtual controller input module being configured to;

establish a neutral position for a virtual controller at a detected initial digit down position on the touch sensitive display;

establish an ancillary control zone spaced away from the virtual controller and configured as a virtual button;

detect a first digit up event subsequent to establishing the neutral position based on at least detecting a lack of contact of the digit of the user for a predetermined threshold period of time;

detect a first subsequent digit down position in the ancillary control zone;

upon detecting the first subsequent digit down position in the ancillary control zone, inhibit repositioning of the neutral position;

determine an ancillary control input parameter based on the first subsequent digit down position in the ancillary control zone;

detect a second digit up event based on at least detecting a lack of contact of the digit of the user for the predetermined threshold period of time;

detect creep between a second subsequent digit down position not in the ancillary control zone and the initial digit down position at which the neutral position was established; and

reposition the neutral position for the virtual controller at the second subsequent digit down position not in the ancillary control zone upon detecting the creep.

10. The computing device of claim 9, wherein the digit is a thumb, and the virtual controller is a virtual joystick controller.

11. The computing device of claim 9, wherein the virtual controller input module is configured to display a control icon anchored at the neutral position, the control icon providing a visual indication to the user of a positioning of the control.

12. The computing device of claim 9, wherein the computing device is configured as a mobile telephone or tablet computer.

13. The computing device of claim 9, wherein the virtual controller input module is configured to generate a controller input message including an indication of a pressure of the digit against the display subsequent to establishing the neutral position, wherein the pressure is detected based on a signal received from a pressure sensor coupled to the touch sensitive display or based on changes in area of the touch associated with the digit over time.

14. The computing device of claim 9, wherein the ancillary control zone is configured as a virtual button or directional pad, and wherein the virtual controller input module is configured to display on the touch sensitive display an ancillary control icon in the ancillary control zone to provide an indication of a position of the ancillary control zone.

15. An input method for use with a computing device having a multi-touch display including a touch sensor configured to detect a plurality of touches of a digit of a user, the method comprising:

detecting an initial digit down position on the multi-touch display via the touch sensor;

establishing a neutral position for a virtual controller at the digit down position;

detecting a subsequent movement of the digit relative to the initial digit down position;

determining a controller input parameter based on a direction, velocity, and/or relative magnitude of the subsequent movement of the digit relative to the initial digit down position;

generating a controller input message indicating the determined controller input parameter;

establishing an ancillary control zone spaced away from the virtual controller and configured as a virtual button;

detecting a first digit up event subsequent to generating the controller input message based on at least detecting a lack of contact of the digit of the user for a predetermined threshold period of time;

detecting a first subsequent digit down position in the ancillary control zone or a first subsequent digit down position not in the ancillary control zone;

upon detecting the first subsequent digit down position not in the ancillary control zone, repositioning the neutral position for the virtual controller at the first subsequent digit down position not in the ancillary control zone;

upon detecting the first subsequent digit down position in the ancillary control zone, inhibiting repositioning of the neutral position;

determining an ancillary control input parameter based on the first subsequent digit down position in the ancillary control zone;

generating a second controller input message that includes an indication of the ancillary control input parameter;

detecting a second digit up event based on at least detecting a lack of contact of the digit of the user for the predetermined threshold period of time;

detecting creep between a second subsequent digit down position not in the ancillary control zone and the initial digit down position at which the neutral position was established; and

repositioning the neutral position for the virtual controller at the second subsequent digit down position not in the ancillary control zone upon detecting the creep.