GESTURE CAPTURE FOR MANIPULATION OF PRESENTATIONS ON ONE OR MORE DEVICE DISPLAYS

• US 20120084680A1
• Filed: 09/28/2011
• Published: 04/05/2012
• Est. Priority Date: 10/01/2010
• Status: Abandoned Application

• Current Assignee: Z124 LLC (Flexis)
• Original Assignee: IMERJ LLC (Flex, Ltd.)
• Inventors: Gimpl, Martin, Cassar, Ron, Marintchenko, Maxim, Swaminathan, Nikhil
• Application Number:
  US13/247,977
• Publication Number:
  US 20120084680A1
• Time in Patent Office: Days
• US Class Current: 715/761
• CPC Class Codes: G06F 1/16 Constructional details or arrangements
• CPC Class Codes: G06F 1/1616 with folding flat displays, e.g. laptop computers or notebooks having a clamshell configuration, with body parts pivoting to an open position around an axis parallel to the plane they define in closed position
• CPC Class Codes: G06F 1/1632 External expansion units, e.g. docking stations
• CPC Class Codes: G06F 1/1641 the display being formed by a plurality of foldable display components G06F1/1647 takes precedence
• CPC Class Codes: G06F 1/1647 including at least an additional display G06F1/1692 takes precedence
• CPC Class Codes: G06F 1/1677 for detecting open or closed state or particular intermediate positions assumed by movable parts of the enclosure, e.g. detection of display lid position with respect to main body in a laptop, detection of opening of the cover of battery compartment
• CPC Class Codes: G06F 1/1694 the I/O peripheral being a single or a set of motion sensors for pointer control or gesture input obtained by sensing movements of the portable computer
• CPC Class Codes: G06F 2200/1614 Image rotation following screen orientation, e.g. switching from landscape to portrait mode
• CPC Class Codes: G06F 2200/1637 Sensing arrangement for detection of housing movement or orientation, e.g. for controlling scrolling or cursor movement on the display of an handheld computer
• CPC Class Codes: G06F 2203/04803 Split screen, i.e. subdividing the display area or the window area into separate subareas
• CPC Class Codes: G06F 3/017 Gesture based interaction, e.g. based on a set of recognized hand gestures interaction based on gestures traced on a digitiser G06F3/04883
• CPC Class Codes: G06F 3/0346 with detection of the device orientation or free movement in a 3D space, e.g. 3D mice, 6-DOF [six degrees of freedom] pointers using gyroscopes, accelerometers or tilt-sensors
• CPC Class Codes: G06F 3/041 Digitisers, e.g. for touch screens or touch pads, characterised by the transducing means
• CPC Class Codes: G06F 3/0412 Digitisers structurally integrated in a display
• CPC Class Codes: G06F 3/0416 Control or interface arrangements specially adapted for digitisers
• CPC Class Codes: G06F 3/044 by capacitive means
• CPC Class Codes: G06F 3/048 Interaction techniques based on graphical user interfaces [GUI]
• CPC Class Codes: G06F 3/0481 based on specific properties of the displayed interaction object or a metaphor-based environment, e.g. interaction with desktop elements like windows or icons, or assisted by a cursor's changing behaviour or appearance
• CPC Class Codes: G06F 3/04812 Interaction techniques based on cursor appearance or behaviour, e.g. being affected by the presence of displayed objects
• CPC Class Codes: G06F 3/04817 using icons graphical or visual programming using iconic symbols G06F8/34
• CPC Class Codes: G06F 3/0482 : Interaction with lists of selectable items, e.g. menus
• CPC Class Codes: G06F 3/0483 : Interaction with page-structured environments, e.g. book metaphor
• CPC Class Codes: G06F 3/0484 : for the control of specific functions or operations, e.g. selecting or manipulating an object, an image or a displayed text element, setting a parameter value or selecting a range
• CPC Class Codes: G06F 3/04842 : Selection of displayed objects or displayed text elements G06F3/0482 takes precedence
• CPC Class Codes: G06F 3/04845 : for image manipulation, e.g. dragging, rotation, expansion or change of colour
• CPC Class Codes: G06F 3/04847 : Interaction techniques to control parameter settings, e.g. interaction with sliders or dials
• CPC Class Codes: G06F 3/0486 : Drag-and-drop
• CPC Class Codes: G06F 3/0488 : using a touch-screen or digitiser, e.g. input of commands through traced gestures
• CPC Class Codes: G06F 3/04883 : for inputting data by handwriting, e.g. gesture or text
• CPC Class Codes: G06F 3/04886 : by partitioning the display area of the touch-screen or the surface of the digitising tablet into independently controllable areas, e.g. virtual keyboards or menus
• CPC Class Codes: G06F 3/1423 : controlling a plurality of local displays, e.g. CRT and flat panel display
• CPC Class Codes: G06F 3/1446 : display composed of modules, e.g. video walls
• CPC Class Codes: G06T 3/02 : Affine transformations for image registration G06T3/147; for image mosaicing G06T3/4038
• CPC Class Codes: G06T 3/40 : Scaling of whole images or parts thereof, e.g. expanding or contracting
• CPC Class Codes: G09G 2356/00 : Detection of the display position w.r.t. other display screens
• CPC Class Codes: G09G 5/14 : Display of multiple viewports
• CPC Class Codes: H04L 12/1827 : Network arrangements for conference optimisation or adaptation

Abstract

An intuitive technique for inputting user gestures into a handheld computing device is disclosed allowing a user to better manipulate different types of screen display presentations, such as desktops and application windows, when performing tasks thereon, e.g., minimization, maximization, moving between display screens, and increasing/decreasing a display thereof across multiple display screens. For manipulating an application window on a device display screen for performing tasks as identified above, user gestures are input to a corresponding gesture capture area for this display screen, wherein this capture area is separate from this display screen.

Claims

1. A method for configuring a device having at least one display screen for displaying one or more of screen display presentations and for displaying one or more operably different screen display presentations, wherein the screen display presentations are able to be manipulated via user gesture input differently from the operably different screen display presentations, comprising:

receiving a first user gesture input to a gesture capture area associated with the at least one screen, the gesture capture area being separate from the at least one display screen;

determining whether a first of the one or more of screen display presentations is displayed on the at least one screen;

when the first screen display presentation is determined to be displayed, perform a step of interpreting the user gesture input for determining a predetermined operation to apply to the first screen display presentation;

performing the predetermined operation;

when none of the one or more screen display presentations are determined to be displayed in the step of determining, perform a step of preventing the device from using the user gesture input to change the at least one display screen;

receiving a second user gesture input in response to a user gesture input that includes user contact directly to the at least one screen;

second determining whether one of the operably different screen display presentations is displayed on the at least one screen;

when the one operably different screen display presentation is determined to be displayed on the at least one screen, performing a step of second interpreting the second user gesture input for determining a predetermined second operation to apply to the one operably different screen display presentation; and

when none of the operably different screen display presentations are determined in the step of second determining, perform a step of second preventing the device from using the second user gesture input to change the at least one display screen.

2. The method of claim 1, wherein the gesture capture area includes a capacitive touch sensitive area for receiving the user gesture input.

3. The method of claim 2, wherein the gesture capture area is not capable of displaying any of the screen display presentations, and is not capable of displaying any of the operably different screen display presentations.

4. The method of claim 1, wherein the first screen display presentation includes an application window.

5. The method of claim 1, wherein the one operably different screen display presentation includes a desktop.

6. The method of claim 1, wherein the user gesture input includes one of:

a drag, a flick, a pinch, and a spread.

7. The method of claim 1, wherein the second user gesture input includes one of:

a drag, a flick, a pinch, and a spread.

8. The method of claim 1, wherein each of the first screen display presentation and the one operably different screen display presentation has corresponding data included in a corresponding stack, wherein a user inputs user gestures to the device rearranging entries in each stack.

9. The method of claim 1, wherein each of the user gesture input and the second user gesture input is for a same user gesture.

10. The method of claim 1, wherein the at least one display screen is one of a plurality of display screens of the device, and the predetermined operation moves the first screen display presentation from the at least screen to another of the plurality of screens.

11. The method of claim 8, wherein the predetermined second operation moves the one operably different screen display presentation from the at least screen to another of the plurality of screens.

12. The method of claim 1, wherein there is a predetermined collection of a plurality of user gesture input descriptions for recognizing user gesture inputs that are interpretable in the step of interpreting,

wherein the one or more screen display presentations includes a plurality of application windows,

wherein each of a plurality of software applications provide a corresponding one of the plurality of application windows,

wherein each of the user gesture inputs (UGI) of the predetermined collection is operable for recognizing user gesture inputs for each application window (AW) of the plurality of application windows when:

(1) the user gesture input UGI is an instance of the first user gesture input, and (2) the application window AW is an instance of the first screen display presentation for which the user gesture input UGI is an input.

13. The method of claim 12, wherein each of the software applications is operable on the device independently of each of the other of the software applications.

14. A computer readable medium for configuring a device having at least one display screen for displaying one or more of screen display presentations and for displaying one or more operably different screen display presentations, wherein the screen display presentations are able to be manipulated via user gesture input differently from the operably different screen display presentations, comprising:

machine instructions for performing the following steps;

receiving a first user gesture input to a gesture capture area associated with the at least one screen, the gesture capture area being separate from the at least one display screen;

determining whether a first of the one or more of screen display presentations is displayed on the at least one screen;

when the first screen display presentation is determined to be displayed, perform a step of interpreting the user gesture input for determining a predetermined operation to apply to the first screen display presentation;

performing the predetermined operation;

when none of the one or more screen display presentations are determined to be displayed in the step of determining, perform a step of preventing the device from using the user gesture input to change the at least one display screen;

receiving a second user gesture input in response to a user gesture input that includes user contact directly to the at least one screen;

second determining whether one of the operably different screen display presentations is displayed on the at least one screen;

when the one operably different screen display presentation is determined to be displayed on the at least one screen, performing a step of second interpreting the second user gesture input for determining a predetermined second operation to apply to the one operably different screen display presentation; and

when none of the operably different screen display presentations are determined in the step of second determining, perform a step of second preventing the device from using the second user gesture input to change the at least one display screen.

15. The computer readable medium of claim 14, wherein the step of interpreting is for rearranging of a stack for the screen display presentations so that an order of presentation of the screen display presentations is rearranged.

16. The computer readable medium of claim 15, wherein each of the screen display presentations includes an application window.

17. The computer readable medium of claim 14, wherein the step of second interpreting is for rearranging of a stack for the operably different screen display presentations so that an order of presentation of the operably different screen display presentations is rearranged.

18. The computer readable medium of claim 17, wherein each of the operably different screen display presentations includes a desktop.

19. A hand-held device, comprising:

at least one display screen for displaying one or more of screen display presentations and for displaying one or more operably different screen display presentations;

a gesture capture area associated with the at least one screen for receiving first user gesture input, the gesture capture area being separate from the at least one display screen;

wherein the screen display presentations are operably manipulated via the first user gesture input received by the gesture capture area;

wherein the at least one display screen is responsive to user touch for receiving second user gestures input for manipulating the operably different screen presentations;

a display manager for determining;

(a) whether one of the screen display presentations is displayed on the at least one screen, and (b) whether one of the operably different screen display presentations is displayed on the at least one screen;

a gesture interpreter for interpreting each of the first and second user gesture inputs for determining an action to perform on a corresponding displayed one of the screen display presentations or a corresponding displayed one of the operably different screen display presentations, wherein the gesture interpreter compares each of the first and second user gesture inputs with a predetermined collection of a plurality of user gesture input descriptions for recognizing user gesture inputs that are interpretable;

wherein for the first user gesture input, if the one screen display presentation is determined to be displayed on the at least one screen, the interpreter interprets the first user gesture input for determining a predetermined first task to apply to the one screen display presentation;

wherein when none of the one or more screen display presentations are determined to be displayed in the display manger prevents the device from using the first user gesture input to change the at least one display screen;

wherein for the second user gestures input, if the one operably different screen display presentation is determined to be displayed on the at least one screen, the interpreter interprets the second user gesture input for determining a predetermined second task to apply to the one operably different screen display presentation; and

wherein when none of the operably different screen display presentations are determined to be displayed in the display manger prevents the device from using the second user gesture input to change the at least one display screen.

a task manager for performing the predetermined first and second tasks.

20. The device of claim 19, wherein the gesture capture area is not capable of displaying any of the screen display presentations, and is not capable of displaying any of the operably different screen display presentations.

21. The device of claim 19, wherein the one screen display presentation includes an application window.

22. The method of claim 19, wherein the one operably different screen display presentation includes a desktop.