Technologies for graphical user interface manipulations using multi-finger touch interactions

• US 10,817,172 B2
• Filed: 03/27/2015
• Issued: 10/27/2020
• Est. Priority Date: 03/27/2015
• Status: Active Grant

• Current Assignee: Intel Corporation
• Original Assignee: Intel Corporation
• Inventors: Maloo, Raghvendra, Valavi, John J., Hsin, Chih-Fan, Subramaniam, Gokul V.
• Primary Examiner(s): Parcher, Daniel W
• Application Number:
  US14/671,726
• Publication Number:
  US 20160283105A1
• Time in Patent Office: 2,041 Days
• CPC Class Codes: G06F 2203/04808 Several contacts: gestures triggering a specific function, e.g. scrolling, zooming, right-click, when the user establishes several contacts with the surface simultaneously; e.g. using several fingers or a combination of fingers and pen
• CPC Class Codes: G06F 3/04883 for inputting data by handwriting, e.g. gesture or text
• CPC Class Codes: G06F 3/04886 by partitioning the display area of the touch-screen or the surface of the digitising tablet into independently controllable areas, e.g. virtual keyboards or menus

Abstract

Technologies for manipulating a graphical user interface (GUI) of a display of a computing device include a touch screen of the display capable of receiving user input via physical contact of the user'"'"'s fingers. The computing device determines whether the user has initiated multi-finger input gesture from which and a hand rotation angle may be determined based on touch points corresponding to the physical contact between the user'"'"'s fingers and the touch screen. Based on the hand rotation angle, the computing device may take an action on the GUI, such as rotate an orientation of the GUI and/or display a virtual keyboard on the GUI relative to the hand rotation angle and/or the touch points. Other embodiments are described herein and claimed.

Claims

1. A computing device to manipulate a graphical user interface (GUI) of a display of the computing device, the computing device comprising:

the display that includes a touch screen to receive input from a user based on physical contact of one or more fingers of the user with the touch screen, wherein each physical contact between a finger of the user and the touch screen establishes a corresponding touch point;

an input gesture determination circuitry configured to detect a presence of five touch points on the touch screen and determine a five-finger-press-and-hold input gesture received from the user in response to a determination that each of the five touch points is established for a duration that exceeds a predetermined duration;

a hand orientation determination circuitry configured to determine a hand rotation angle based on the detected five touch points and a weight independently applied to a farthest touch point of the five touch points;

a GUI orientation management circuitry configured to (i) determine a present orientation of a GUI element, (ii) determine a proposed orientation of the GUI element based on the hand rotation angle, and (iii) cause a rotation of the GUI element on the display in response to a determination that the proposed orientation and the present orientation of the GUI element do not match; and

a virtual keyboard display management circuitry configured to (i) determine, from a number of software applications presently running on the computing device, a software application that corresponds to the five-finger-press-and-hold input gesture, wherein to determine the software application that corresponds to the five-finger-press-and-hold input gesture comprises to determine the software application based on which of the software applications presently running has a software application GUI in which a majority of the five touch points are at least one of in contact with or hovering over the software application GUI, and (ii) display a context-specific virtual keyboard, in addition to the GUI element, that includes a plurality of soft keys mapped to functions related to the determined software application at a location based on the five touch points.

2. The computing device of claim 1, wherein to determine the hand rotation angle comprises to (i) determine a centroid based on the five touch points, (ii) determine a touch point vector for each of the five touch points based on the determined centroid and each corresponding touch point, (iii) calculate a rotation vector based on the five touch point vectors, and (iv) determine the hand rotation angle based on the rotation vector relative to a positive x-axis through the centroid.

3. The computing device of claim 2, wherein to calculate the rotation vector comprises to (i) normalize each of the five touch point vectors, (ii) calculate a sum of the five normalized touch point vectors, (iii) determine the farthest touch point based on which touch point of the five touch point vectors is farthest from the centroid, and (iv) determine the rotation vector further based on the weight applied to the farthest touch point.

4. The computing device of claim 1, wherein the hand orientation determination circuitry is further configured to validate the hand rotation angle, wherein to validate the hand rotation angle comprises to (i) determine a centroid based on the five touch points, (ii) determine a touch point vector for each of the five touch points based on the determined centroid and each corresponding touch point, (iii) determine three touch point vectors of the five touch point vectors that are closest to the centroid, (iv) determine an angle between each of the three touch point vectors and an axis of a centroidal axes through the centroid, and (v) determine a confidence level of the hand rotation angle based on each angle being less than a maximum angle.

5. The computing device of claim 1, wherein to determine the proposed orientation of the GUI element based on the hand rotation angle comprises to determine a proposed GUI rotation angle relative to a normal orientation of the display.

6. The computing device of claim 5, wherein to cause the rotation of the orientation of the GUI element on the display comprises to cause the rotation of the orientation of the GUI element to rotate the GUI element to the proposed GUI rotation angle in response to a determination that the hand rotation angle is greater than or equal to 45°

and less than 135°

,

wherein to cause the rotation of the orientation of the GUI element on the display comprises to cause the rotation of the orientation of the GUI element to rotate the GUI element to the proposed GUI rotation angle in response to a determination that the hand rotation angle is greater than or equal to 135° and

less than 225°

,

wherein to cause the rotation of the orientation of the GUI element on the display comprises to cause the rotation of the orientation of the GUI element to rotate the GUI element to the proposed GUI rotation angle in response to a determination that the hand rotation angle is greater than or equal to 225° and

less than 315°

, and

wherein to cause the rotation of the orientation of the GUI element on the display comprises to cause the rotation of the orientation of the GUI element to rotate the GUI element to the proposed GUI rotation angle in response to a determination that the hand rotation angle is greater than or equal to 315°

.

7. One or more non-transitory, computer-readable storage media comprising a plurality of instructions that in response to being executed cause a computing device to:

detect a presence of five touch points on a touch screen of a display of the computing device, wherein each of the five touch points is indicative of a physical contact of a finger of a user of the computing with the touch screen;

determine a five-finger-press-and-hold input gesture received from the user in response to a determination that the presence of each of the five touch points is established for a duration that exceeds a predetermined duration;

determine a hand rotation angle based on the detected five touch points and a weight independently applied to a farthest touch point of the five touch points;

determine a present orientation of a GUI element;

determine a proposed orientation of the GUI element based on the hand rotation angle;

cause a rotation of the GUI element on the display in response to a determination that that the proposed orientation and the present orientation of the GUI element do not match;

determine, from a number of software applications presently running on the computing device, a software application that corresponds to the five-finger-press-and-hold input gesture, wherein to determine the software application that corresponds to the five-finger-press-and-hold input gesture comprises to determine the software application based on which of the software applications presently running has a software application GUI in which a majority of the five touch points are at least one of in contact with or hovering over the software application GUI; and

display a context-specific virtual keyboard, in addition to the GUI element, that includes a plurality of soft keys mapped to functions related to the determined software application at a location based on the five touch points.

8. The one or more non-transitory, computer-readable storage media of claim 7, wherein determining the hand rotation angle comprises (i) determining a centroid based on the five touch points, (ii) determining a touch point vector for each of the five touch points based on the determined centroid and each corresponding touch point, (iii) calculating a rotation vector based on the five touch point vectors, and (iv) determining the hand rotation angle based on the rotation vector relative to a positive x-axis through the centroid.

9. The one or more non-transitory, computer-readable storage media of claim 8, wherein calculating the rotation vector comprises (i) normalizing each of the five touch point vectors, (ii) calculating a sum of the five normalized touch point vectors, (iii) determining the farthest touch point based on which touch point of the five touch point vectors is farthest from the centroid and (iv) determining the rotation vector further based on the weight applied to the farthest touch point.

10. The one or more non-transitory, computer-readable storage media of claim 7, further comprising a plurality of instructions that in response to being executed cause the computing device to:

validate the hand rotation angle, wherein validating the hand rotation angle comprises (i) determining a centroid based on the five touch points;

determine a touch point vector for each of the five touch points based on the determined centroid and each corresponding touch point

determine three touch point vectors of the five touch point vectors that are closest to the centroid

determine an angle between each of the three touch point vectors and an axis of a centroidal axes through the centroid, and

determine a confidence level of the hand rotation angle based on each angle being less than a maximum angle.

11. The one or more non-transitory, computer-readable storage media of claim 7, wherein to cause the rotation of the orientation of the GUI element on the display comprises to cause the rotation of the orientation of the GUI element to rotate the GUI element to the proposed GUI rotation angle in response to a determination that the hand rotation angle is greater than or equal to 45°

and less than 135°

,

wherein to cause the rotation of the orientation of the GUI element on the display comprises to cause the rotation of the orientation of the GUI element to rotate the GUI element to the proposed GUI rotation angle in response to a determination that the hand rotation angle is greater than or equal to 135° and

less than 225°

,

wherein to cause the rotation of the orientation of the GUI element on the display comprises causing the rotation of the orientation of the GUI element to rotate the GUI element to the proposed GUI rotation angle in response to a determination that the hand rotation angle is greater than or equal to 225° and

less than 315°

, and

wherein to cause the rotation of the orientation of the GUI element on the display comprises to cause the rotation of the orientation of the GUI element to rotate the GUI to the proposed GUI rotation angle in response to a determination that the hand rotation angle is greater than or equal to 315°

.

12. A method for manipulating a graphical user interface (GUI) of a display of the computing device, the method comprising:

detecting a presence of five touch points on a touch screen of the display of the computing device, wherein each of the five touch points is indicative of a physical contact of a finger of a user of the computing device with the touch screen;

determining a five-finger-press-and-hold input gesture received from the user in response to a determination that the presence of each of the five touch points is established for a duration that exceeds a predetermined duration;

determining a hand rotation angle based on the detected five touch points and a weight applied independently to a farthest touch point of the five touch points;

determining a present orientation of a GUI element;

determining a proposed orientation of the GUI element based on the hand rotation angle;

causing a rotation of the GUI element on the display in response to a determination that that the proposed orientation and the present orientation of the GUI element do not match;

determining, from a number of software applications presently running on the computing device, a software application that corresponds to the five-finger-press-and-hold input gesture, wherein determining the software application that corresponds to the five-finger-press-and-hold input gesture comprises to determine the software application based on which of the software applications presently running has a software application GUI in which a majority of the five touch points are at least one of in contact with or hovering over the software application GUI; and

displaying a context-specific virtual keyboard, in addition to the GUI element, that includes a plurality of soft keys mapped to functions related to the determined software application at a location based on the five touch points.

13. The method of claim 12, wherein determining the hand rotation angle comprises (i) determining a centroid based on the five touch points, (ii) determining a touch point vector for each of the five touch points based on the determined centroid and each corresponding touch point, (iii) calculating a rotation vector based on the five touch point vectors, and (iv) determining the hand rotation angle based on the rotation vector relative to a positive x-axis through the centroid.

14. The method of claim 13, wherein calculating the rotation vector comprises (i) normalizing each of the five touch point vectors, (ii) calculating a sum of the five normalized touch point vectors, (iii) determining the farthest touch point based on which touch point of the five touch point vectors is farthest from the centroid, and (iv) determining the rotation vector further based on the weight applied to the farthest touch point.

15. The method of claim 12, further comprising validating the hand rotation angle, wherein validating the hand rotation angle comprises (i) determining a centroid based on the five touch points, (ii) determining a touch point vector for each of the five touch points based on the determined centroid and each corresponding touch point, (iii) determining three touch point vectors of the five touch point vectors that are closest to the centroid, (iv) determining an angle between each of the three touch point vectors and an axis of a centroidal axes through the centroid, and (v) determining a confidence level of the hand rotation angle based on each angle being less than a maximum angle.

16. The method of claim 12, wherein causing the rotation of the orientation of the GUI element on the display comprises causing the rotation of the orientation of the GUI element to rotate the GUI element to the proposed GUI rotation angle in response to a determination that the hand rotation angle is greater than or equal to 45°

and less than 135°

,

wherein causing the rotation of the orientation of the GUI element on the display comprises causing the rotation of the orientation of the GUI element to rotate the GUI element to the proposed GUI rotation angle in response to a determination that the hand rotation angle is greater than or equal to 135° and

less than 225°

,

wherein causing the rotation of the orientation of the GUI element on the display comprises causing the rotation of the orientation of the GUI element to rotate the GUI element to the proposed GUI rotation angle in response to a determination that the hand rotation angle is greater than or equal to 225° and

less than 315°

, and

wherein causing the rotation of the orientation of the GUI element on the display comprises causing the rotation of the orientation of the GUI element to rotate the GUI element to the proposed GUI rotation angle in response to a determination that the hand rotation angle is greater than or equal to 315°

.