Motion-based gestures for a computing device

• US 10,031,586 B2
• Filed: 06/12/2013
• Issued: 07/24/2018
• Est. Priority Date: 06/12/2013
• Status: Expired due to Fees

• Current Assignee: Amazon Technologies, Inc. (Amazon.com, Inc.)
• Original Assignee: Amazon Technologies, Inc. (Amazon.com, Inc.)
• Inventors: Touloumtzis, Michael P.
• Primary Examiner(s): Jansen, II, Michael J
• Application Number:
  US13/916,464
• Publication Number:
  US 20140368441A1
• Time in Patent Office: 1,868 Days
• Field of Search: 345173
• CPC Class Codes: G06F 1/1626 with a single-body enclosure integrating a flat display, e.g. Personal Digital Assistants [PDAs]
• CPC Class Codes: G06F 2200/1637 Sensing arrangement for detection of housing movement or orientation, e.g. for controlling scrolling or cursor movement on the display of an handheld computer
• CPC Class Codes: G06F 3/017 Gesture based interaction, e.g. based on a set of recognized hand gestures interaction based on gestures traced on a digitiser G06F3/04883
• CPC Class Codes: G06F 3/0487 using specific features provided by the input device, e.g. functions controlled by the rotation of a mouse with dual sensing arrangements, or of the nature of the input device, e.g. tap gestures based on pressure sensed by a digitiser
• CPC Class Codes: G06F 3/04883 for inputting data by handwriting, e.g. gesture or text

Abstract

Approaches to enable a computing device, such as a phone or tablet computer, to use sensor information obtained from sensors of the computing device to interpret one or more gestures and/or other input provided by the user. In particular, the computing device may use information about the movement of the device gathered from an accelerometer or gyroscope in combination with input data detected by the touch screen or cameras of the device in order to disambiguate between several different types of gestures for the device.

Claims

1. A computing device, comprising:

one or more sensors configured to capture sensor data, the sensor data indicating a physical movement of the computing device performed by at least a first hand of a user;

a touch screen display, generating touch location data representing touch locations on the touch screen display by at least a first finger and a second finger of a second hand of the user;

a processor for executing instructions stored in a memory which, when executed by the processor, cause the computing device to;

determine, during a first period, using the touch location data, first touch location data associated with the first finger contacting the touch screen display;

determine, during the first period, using the touch location data, second touch location data associated with the second finger contacting the touch screen display;

determine, during a second period after the first period, using the touch location data, third touch location data associated with the first finger contacting the touch screen display;

determine, during the second period, using the touch location data, fourth touch location data associated with the second finger contacting the touch screen display;

determine, using the sensor data, information about the physical movement of the computing device between the first period and the second period;

determine, based at least on the first touch location data and the third touch location data being different, that a first position of the first finger on the touch screen display during the first period is different than a third position of the first finger on the touch screen display during the second period;

determine, based at least on the second touch location data and the fourth touch location data being different, that a second position of the second finger on the touch screen display during the first period is different than a fourth position of the second finger on the touch screen display during the second period;

determine a first gesture during a third period based at least on the physical movement of the computing device between the first period and the second period, the first position of the first finger on the touch screen display during the first period being different than the third position of the first finger on the touch screen display during the second period, and the second position of the second finger on the touch screen display during the first period being different than the fourth position of the second finger on the touch screen display during the second period, wherein the first gesture corresponds to the second hand remaining steady while the computing device is rotated between the first period and the second period;

select the first gesture;

invoke a first function of the computing device, the first function corresponding to the first gesture, the first function including switching the touch screen display from a landscape display mode to a portrait display mode and then maintaining the touch screen display in the portrait display mode, or switching the touch screen display from a portrait display mode to a landscape display mode and then maintaining the touch screen display in the landscape display mode;

determine a second gesture during a fourth period, the second gesture different than the first gesture; and

invoke a second function of the computing device, the second function corresponding to the second gesture, the second function including maintaining the touch screen display in either the portrait display mode or the landscape display mode while the computing device is rotated.

2. The computing device of claim 1, wherein the one or more sensors further comprise at least one of:

an accelerometer configured to measure an acceleration of the computing device;

or

a gyroscope configured to measure an orientation of the computing device.

3. The computing device of claim 1, wherein the memory further includes instructions that, when executed by the processor, further cause the computing device to:

determine the second gesture, wherein the second gesture corresponds to at least the computing device remaining stationary between a fifth period and a sixth period; and

select the second gesture.

4. The computing device of claim 3, wherein the memory further includes instructions that, when executed by the processor, further cause the computing device to determine that the touch location data between the fifth period and the sixth period represents the first and second fingers physically moving while the computing device remains stationary.

5. A computer implemented method, comprising:

under the control of a computing device configured with executable instructions,

receiving user input data generated by a touch screen, the user input data representing at least;

first touch location data of a first finger of a second hand of a user on the touch screen during a first period;

second touch location data of a second finger of the second hand on the touch screen during the first period;

third touch location data of the first finger of the second hand on the touch screen during a second period, the second period after the first period; and

fourth touch location data of the second finger of the second hand on the touch screen during the second period;

processing data captured by one or more sensors of the computing device to determine a rotation of the computing device performed by at least a first hand of the user between the first period and the second period;

determining, based at least in part on the first touch location data and the third touch location data being different, that a first position of the first finger on the touch screen during the first period is different than a third position of the first finger on the touch screen during the second period;

determining, based at least in part on the second touch location data and the fourth touch location data being different, that a second position of the second finger on the touch screen during the first period is different than a fourth position of the second finger on the touch screen during the second period;

determining a first gesture during a third period based at least on the rotation of the computing device between the first period and the second period, the first position of the first finger on the touch screen during the first period being different than the third position of the first finger on the touch screen during the second period, and the second position of the second finger on the touch screen during the first period being different than the fourth position of the second finger on the touch screen during the second period, wherein the first gesture corresponds to the second hand remaining steady while the computing device is rotated between the first period and the second period;

invoking a first function of the computing device, the first function corresponding to the first gesture, the first function comprising at least switching the touch screen from a portrait display mode to a landscape display mode and then maintaining the touch screen in the landscape display mode;

determining a second gesture during a fourth period, the second gesture different than the first gesture; and

invoking a second function of the computing device, the second function corresponding to the second gesture, the second function including maintaining the touch screen in either the portrait display mode or the landscape display mode while the computing device is rotated.

6. The computer implemented method of claim 5, wherein the first gesture corresponds to the first finger and the second finger rotating relative to the computing device while making physical contact with the touch screen of the computing device.

7. The computer implemented method of claim 5, wherein the first function causes rotation of an orientation of a graphical image displayed on the touch screen.

8. The computer implemented method of claim 5, wherein the one or more sensors of the computing device further comprise at least one of:

an accelerometer configured to measure an acceleration of the computing device;

or

a gyroscope configured to measure an orientation of the computing device.

9. A non-transitory computer readable storage medium storing one or more sequences of instructions that, when executed by one or more processors, cause a computing device to:

receive user input data generated by a touch screen, the user input data representing at least;

first touch location data of a first finger of a second hand of a user relative to the touch screen during a first period;

second touch location data of a second finger of the second hand relative to the touch screen during the first period;

third touch location data of the first finger of the second hand relative to the touch screen during a second period; and

fourth touch location data of the second finger of the second hand relative to the touch screen during the second period;

process data captured by one or more sensors of the computing device to determine a rotation of the computing device performed by at least a first hand of the user between the first period and the second period;

determine, based at least in part on the first touch location data and the third touch location data being different, that a first position of the first finger on the touch screen during the first period is different than a third location of the first finger on the touch screen during the second period;

determine, based at least in part on the second touch location data and the fourth touch location data being different, that a second position of the second finger on the touch screen during the first period is different than a fourth location of the second finger on the touch screen during the second period;

determine a first gesture during a third period based at least on the rotation of the computing device between the first period and the second period, the first position of the first finger on the touch screen during the first period being different than the third location of the first finger on the touch screen during the second period, and the second position of the second finger on the touch screen during the first period being different than the fourth location of the second finger on the touch screen during the second period, wherein the first gesture corresponds to the second hand remaining steady while the computing device is rotated between the first period and the second period;

invoke a first function of the computing device, the first function corresponding to the first gesture, the first function comprising at least switching the touch screen from a portrait display mode to a landscape display mode and then maintaining the touch screen in the landscape display mode;

determine a second gesture during a fourth period, the second gesture different than the first gesture; and

invoke a second function of the computing device, the second function corresponding to the second gesture, the second function including maintaining the touch screen in either the portrait display mode or the landscape display mode while the computing device is rotated.

10. The non-transitory computer readable storage medium of claim 9, wherein the first gesture corresponds to the first finger and the second finger rotating relative to the computing device while making physical contact with the touch screen of the computing device.

11. The non-transitory computer readable storage medium of claim 9, wherein the first function causes rotation of an orientation of a graphical image displayed on the touch screen.

12. The non-transitory computer readable storage medium of claim 9, wherein the one or more sensors of the computing device further comprise at least one of:

an accelerometer configured to measure an acceleration of the computing device;

or

a gyroscope configured to measure an orientation of the computing device.