PORTABLE ELECTRONIC DEVICE WITH RELATIVE GESTURE RECOGNITION MODE

• US 20100064261A1
• Filed: 09/09/2008
• Published: 03/11/2010
• Est. Priority Date: 09/09/2008
• Status: Abandoned Application

• Current Assignee: Microsoft Technology Licensing LLC (Microsoft Corporation)
• Original Assignee: Microsoft Corporation
• Inventors: Fong, Jeffrey, Abanami, Thamer, Andrews, Anton
• Application Number:
  US12/206,747
• Publication Number:
  US 20100064261A1
• Time in Patent Office: Days
• US Class Current: 715/863
• CPC Class Codes: G06F 3/04883 for inputting data by handwriting, e.g. gesture or text

Abstract

A computer program executable on a portable electronic device having a touch screen sensor is provided. The computer program may include an input mode switching module configured to receive a mode switch user input to switch between a direct input mode and a relative gesture recognition mode. The computer program may further include a gesture-based control module configured, in the relative gesture recognition mode, to recognize a contact point on the touch screen sensor between a digit of a user and a surface of the touch screen sensor in a defined region in which the graphical user interface elements are unselectable, and to identify a detected gesture based on user touch input originating from the contact point, and to send a message to an application program to adjust an operation of the portable electronic device based on the detected gesture.

Claims

1. A computer program executable on a portable electronic device having a touch screen sensor, the computer program comprising:

an input mode switching module configured to receive a mode switch user input to switch between a direct input mode and a relative gesture recognition mode in response to a user input, wherein in the direct input mode, one or more graphical user interface elements of a graphical user interface of the portable electronic device are selectable via touch input of the user, and wherein in the relative gesture recognition mode, the graphical user interface elements in at least a defined region of the graphical user interface are made to be unselectable; and

a gesture-based control module configured, in the relative gesture recognition mode, to recognize a contact point on the touch screen sensor between a digit of a user and a surface of the touch screen sensor in the defined region in which the graphical user interface elements are unselectable, and to present in the defined region a gesture control proximate to the contact point, the gesture-based control module being further configured to identify a detected gesture based on user touch input originating from the contact point, and to send a message to an application program to adjust an operation of the portable electronic device based on the detected gesture.

2. The computer program of claim 1, wherein the mode switch user input is selected from the group consisting of a user input via a clutch key associated with the portable electronic device and a user input via a contact between the digit of a user and the surface of the touch screen sensor.

3. The computer program of claim 1, wherein the gesture control includes a transport control configured to snap a frame of reference for the transport control to the contact point within the defined region.

4. The computer program of claim 3,

wherein the gesture-based control module is configured to send a message to a media playback application program to adjust the operation of the portable electronic device according to the detected gesture identified by the gesture-based control module;

wherein in a playback control mode of the transport control, the detected gesture is identified based on detection of a substantially horizontal direction of movement of the digit of a user relative to the frame of reference, and in response adjusts a temporal position of a media playback;

wherein in a volume control mode of the transport control, the detected gesture is identified based on detection of a substantially vertical direction of the digit of the user relative to the frame of reference, and in response adjusts a volume of the media playback; and

wherein in a pause control mode of the transport control, the detected gesture is identified based detection of a tapping movement of the digit of the user relative to the frame of reference, and in response changes a pause status of the media playback.

5. The computer program of claim 1,

wherein the gesture control includes a virtual game control configured to spawn a virtual control stick control for the virtual game control at the contact point;

wherein the gesture-based control module is further configured to define a control perimeter surrounding the virtual game control stick;

wherein the gesture-based control module is further configured to send a message to a computer game application program to adjust the operation of the portable electronic device based on a relative distance of the virtual control stick control from the control perimeter or the contact point.

6. The computer program of claim 5,

wherein the gesture-based control module is further configured to define a full-scale deflection of the virtual control stick control at the control perimeter;

wherein the message sent to the computer game application program when the detected gesture based on the user touch input received by the virtual game control via the virtual control stick control is within the control perimeter is in proportion to a measured deflection of the virtual control stick control with respect to the full-scale deflection of the virtual control stick control; and

wherein the message sent to the computer game application program when the detected gesture based on the user touch input received by the virtual game control via the virtual control stick control is received outside of the control perimeter is substantially the same as a full-scale deflection of the virtual control stick control.

7. The computer program of claim 1, the gesture-based control module further configured to position the defined region independent of the graphical user interface of the portable electronic device.

8. The computer program of claim 1, further configured to enable the gesture-based control module to access developer specified control parameters by which the gesture control is configured to operate.

9. The computer program of claim 8, wherein the developer specified control parameters are selected from the group consisting of a volume parameter, a playback speed parameter, a playback direction parameter, a control perimeter definition parameter, and a defined region definition parameter.

10. A method of controlling a portable electronic device having a touch screen sensor, comprising:

initiating a relative gesture recognition mode responsive to a mode-switch user input, wherein in the relative gesture recognition mode one or more graphical user interface elements in a defined region of a graphical user interface are made to be unselectable;

recognizing a contact point on the touch screen sensor between a digit of a user and a surface of the touch screen sensor in the defined region in which the graphical user interface elements are unselectable in the relative gesture recognition mode;

presenting a gesture control having a defined control perimeter proximate to the contact point in the defined region in which the graphical user interface elements are unselectable in the relative gesture recognition mode;

identifying a detected gesture based on a user touch input originating from the contact point received by the gesture control in the defined region in which the graphical user interface elements are unselectable within the touch screen sensor via the touch screen sensor; and

adjusting an operation of the portable electronic device based on a relative distance from a pre-determined location on the defined control perimeter to the detected gesture or based on a relative distance from the contact point to the detected gesture.

11. The method of claim 10, wherein the mode switch user input is selected from the group consisting of a user input via a clutch key associated with the portable electronic device and a user input via a contact between the digit of a user and the surface of the touch screen sensor.

12. The method of claim 10,

wherein presenting a gesture control includes presenting a transport control;

wherein presenting a gesture control further includes snapping a frame of reference for the transport control to the contact point within the defined region.

13. The method of claim 12, wherein adjusting an operation of the portable electronic device includes:

adjusting a temporal position of a media playback responsive to the detected gesture identified by a substantially horizontal direction of the digit of a user relative to the frame of reference;

adjusting a volume of a media playback responsive to responsive to the detected gesture identified by a substantially vertical direction of the digit of a user relative to the frame of reference; and

adjusting a pause status of a media playback responsive to the detected gesture identified by a tapping movement of the digit of a user relative to the frame of reference.

14. The method of claim 10, wherein presenting a gesture control having a defined control perimeter proximate to the contact point includes spawning a virtual control stick control for a virtual game control at the contact point, and wherein the defined control perimeter has a full-scale deflection of a virtual control stick control at the defined control perimeter.

15. The method of claim 14, wherein adjusting an operation of the portable electronic device includes:

outputting a response from the virtual game control that is in proportion to a measured deflection of the virtual control stick control with respect to the full-scale deflection of the virtual control stick control, when the gesture received by the touch screen sensor is received within the defined control perimeter; and

outputting a response from the virtual game control that is substantially the same as a full-scale deflection of the virtual control stick control, when the relative gesture is received outside of the defined control perimeter.

16. The method of claim 10, further comprising positioning the defined region of the graphical user interface in which the graphical user interface elements are unselectable independent of the graphical user interface.

17. The method of claim 10, further comprising enabling the gesture-based control module to access developer specified control parameters by which the gesture control is configured to operate.

18. The method of claim 17, wherein the developer specified control parameters are selected from the group consisting of a volume parameter, a playback speed parameter, a playback direction parameter, a control perimeter definition parameter, and a defined region definition parameter.

19. The method of claim 10, identifying a detected gesture including interpreting the detected gesture based at least in part on a comparison of the detected gesture received by the gesture control in the defined region in which the graphical user interface elements are unselectable via the touch screen sensor to a definition corresponding to one of a set of one or more pre-defined gestures within a library of pre-defined gestures.

20. A computer-readable storage medium comprising instructions executable by a computing device to perform a method comprising:

initiating a relative gesture recognition mode responsive to a mode-switch user input, wherein in the relative gesture recognition mode one or more graphical user interface elements in a defined region of a graphical user interface are made to be unselectable;

recognizing a contact point on the touch screen sensor between a digit of a user and a surface of the touch screen sensor in the defined region in which the graphical user interface elements are unselectable in the relative gesture recognition mode;

presenting a gesture control having a defined control perimeter proximate to the contact point in the defined region in which the graphical user interface elements are unselectable in the relative gesture recognition mode;

identifying a detected gesture based on a user touch input originating from the contact point received by the gesture control in the defined region in which the graphical user interface elements are unselectable within the touch screen sensor via the touch screen sensor; and

adjusting an operation of the portable electronic device based on a relative distance from a pre-determined location on the defined control perimeter to the detected gesture or based on a relative distance from the contact point to the detected gesture.