GESTURE CONTROLS FOR MULTI-SCREEN USER INTERFACE

• US 20120084735A1
• Filed: 11/17/2010
• Published: 04/05/2012
• Est. Priority Date: 10/01/2010
• Status: Active Grant

• Current Assignee: Z124 LLC (Flexis)
• Original Assignee: Flextronics ID LLC
• Inventors: Sirpal, Sanjiv
• Granted Patent: US 9,046,992 B2
• Time in Patent Office: Days
• US Class Current: 715/863
• CPC Class Codes: G06F 1/1616 with folding flat displays, e.g. laptop computers or notebooks having a clamshell configuration, with body parts pivoting to an open position around an axis parallel to the plane they define in closed position
• CPC Class Codes: G06F 1/1641 the display being formed by a plurality of foldable display components G06F1/1647 takes precedence
• CPC Class Codes: G06F 1/1647 including at least an additional display G06F1/1692 takes precedence
• CPC Class Codes: G06F 3/017 Gesture based interaction, e.g. based on a set of recognized hand gestures interaction based on gestures traced on a digitiser G06F3/04883
• CPC Class Codes: G06F 3/0412 Digitisers structurally integrated in a display
• CPC Class Codes: G06F 3/0416 Control or interface arrangements specially adapted for digitisers
• CPC Class Codes: G06F 3/0481 based on specific properties of the displayed interaction object or a metaphor-based environment, e.g. interaction with desktop elements like windows or icons, or assisted by a cursor's changing behaviour or appearance
• CPC Class Codes: G06F 3/04817 using icons graphical or visual programming using iconic symbols G06F8/34
• CPC Class Codes: G06F 3/0482 Interaction with lists of selectable items, e.g. menus
• CPC Class Codes: G06F 3/0483 Interaction with page-structured environments, e.g. book metaphor
• CPC Class Codes: G06F 3/04842 Selection of displayed objects or displayed text elements G06F3/0482 takes precedence
• CPC Class Codes: G06F 3/04845 for image manipulation, e.g. dragging, rotation, expansion or change of colour
• CPC Class Codes: G06F 3/04847 Interaction techniques to control parameter settings, e.g. interaction with sliders or dials
• CPC Class Codes: G06F 3/0486 Drag-and-drop
• CPC Class Codes: G06F 3/0488 using a touch-screen or digitiser, e.g. input of commands through traced gestures
• CPC Class Codes: G06F 3/04883 for inputting data by handwriting, e.g. gesture or text
• CPC Class Codes: G06F 3/04886 by partitioning the display area of the touch-screen or the surface of the digitising tablet into independently controllable areas, e.g. virtual keyboards or menus
• CPC Class Codes: G06F 3/1423 controlling a plurality of local displays, e.g. CRT and flat panel display

Abstract

Method and apparatus for controlling a computing device using gesture inputs. The computing device may be a handheld computing device with multiple displays. The displays may be capable of displaying a graphical user interface (GUI). The GUI may be a multi screen GUI or a single screen GUI such that receipt of gesture inputs may result in the movement of a GUI from one display to another display or may result in maximization of a multi screen GUI across multiple displays.

Claims

1. A method for controlling a plurality of displays of a handheld computing device, comprising:

executing an application on said handheld computing device such that a screen of said application is displayed in a first display of the plurality of displays;

receiving a first gesture input at a gesture sensor of said handheld computing device;

modifying the plurality of displays in response to the first gesture input such that the screen is displayed in at least a second display of the plurality of displays.

2. The method according to claim 1, wherein the first gesture input is directional.

3. The method according to claim 2, wherein the second display is located in a direction with respect to the first display corresponding to the direction of the first gesture.

4. The method according to claim 3, wherein the screen occupies a single display and said modifying includes moving the screen from the first display to the second display.

5. The method according to claim 4, wherein the modifying includes revealing an underlying screen in the first display and obscuring another screen in the second display.

6. The method according to claim 3, wherein the application is expandable to occupy multiple displays and the modifying includes expanding the application such that one or more screens corresponding to the application occupy both the first display and the second display.

7. The method according to claim 6, wherein the expanding includes moving a node screen of the application to the second display while maintaining a parent screen of the application in the first display.

8. The method according to claim 6, wherein the expanding includes scaling the application to occupy the first and second display.

9. The method according to claim 6, wherein the expanding includes enlarging the viewable area of the application into the second display.

10. The method according to claim 6, wherein the method further includes receiving a second gesture input different than the first gesture input at the gesture sensor and moving the screen from the first display to the second display in response to the second gesture input.

11. The method according to claim 1, wherein the modifying includes constrained movement of the screen linearly between the first display and the second display.

12. The method according to claim 11, wherein the handheld computing device is in operative communication with an external display, and the method further comprises relocating the screen from the first display to the external display.

13. The method according to claim 11, wherein once the screen has been relocated to the external display, the screen is unconstrainably moveable about the external display.

14. The method according to claim 1, wherein the gesture sensor is a touch sensitive device.

15. The method according to claim 14, wherein the touch sensitive device is separate from and non overlapping with respect to the first display or the second display.

16. The method according to claim 14, wherein the touch sensitive device is a touch screen display including at least one of the first or second displays.

17. A method for controlling a plurality of displays of a handheld computing device, comprising:

executing an application on said handheld computing device such that one or more screens of said application is displayed in a first display and a second display of said handheld computing device;

receiving a gesture input at a gesture sensor of said handheld computing device;

modifying the plurality of displays in response to the gesture input such that the application is displayed in the first display of the handheld computing device and not in the second display.

18. The method of claim 17, wherein the gesture display is directional and the first display corresponds to the direction of the gesture.

19. The method according to claim 18, wherein said modifying includes hiding a node screen displayed in the second display prior to the modifying.

20. The method according to claim 18, wherein said modifying includes scaling the one or more screens to the first display.

21. The method according to claim 18, wherein said modifying includes reducing the visible area of the application.