Mobile device off-screen gesture area

• US 10,168,973 B2
• Filed: 11/20/2015
• Issued: 01/01/2019
• Est. Priority Date: 09/27/2011
• Status: Active Grant

• Current Assignee: Z124 LLC (Flexis)
• Original Assignee: Z124 LLC (Flexis)
• Inventors: Reeves, Paul E., Sirpal, Sanjiv
• Primary Examiner(s): Edwards, Carolyn R
• Application Number:
  US14/947,383
• Publication Number:
  US 20160179208A1
• Time in Patent Office: 1,138 Days
• CPC Class Codes: B29D 11/00673 Supports for light guides
• CPC Class Codes: E05D 3/12 with two parallel pins and one arm
• CPC Class Codes: E05Y 2900/606 for electronic devices
• CPC Class Codes: G02B 6/0001 specially adapted for lighting devices or systems lighting or signalling on vehicles using light guides B60Q1/00; lighting devices for vehicle dashboards B60Q3/10; lighting devices for vehicle interior using light guides B60Q3/62; lighting devices mounted on the vehicle rear part using light guides F21S43/235; measuring arrangements having light conducting pointers G01D13/265; illumination of liquid crystal displays G02F1/1336; illuminated signs G09F13/00
• CPC Class Codes: G06F 1/16 Constructional details or arrangements
• CPC Class Codes: G06F 1/1601 Constructional details related to the housing of computer displays, e.g. of CRT monitors, of flat displays constructional details related to flat displays integrated in a portable computer, e.g. laptop, handheld computer G06F1/1637; constructional details related to television receivers H04N5/64
• CPC Class Codes: G06F 1/1605 Multimedia displays, e.g. with integrated or attached speakers, cameras, microphones
• CPC Class Codes: G06F 1/1616 with folding flat displays, e.g. laptop computers or notebooks having a clamshell configuration, with body parts pivoting to an open position around an axis parallel to the plane they define in closed position
• CPC Class Codes: G06F 1/1618 the display being foldable up to the back of the other housing with a single degree of freedom, e.g. by 360° rotation over the axis defined by the rear edge of the base enclosure
• CPC Class Codes: G06F 1/1637 Details related to the display arrangement, including those related to the mounting of the display in the housing
• CPC Class Codes: G06F 1/1641 the display being formed by a plurality of foldable display components G06F1/1647 takes precedence
• CPC Class Codes: G06F 1/1643 the display being associated to a digitizer, e.g. laptops that can be used as penpads details related to the relative motion of the display enclosure with respect to the body enclosure, e.g. to move between laptop and tablet PC configuration G06F1/1615
• CPC Class Codes: G06F 1/1647 including at least an additional display G06F1/1692 takes precedence
• CPC Class Codes: G06F 1/1649 the additional display being independently orientable, e.g. for presenting information to a second user
• CPC Class Codes: G06F 1/1677 for detecting open or closed state or particular intermediate positions assumed by movable parts of the enclosure, e.g. detection of display lid position with respect to main body in a laptop, detection of opening of the cover of battery compartment
• CPC Class Codes: G06F 1/1681 Details related solely to hinges hinge details related to the transmission of signals or power are classified in G06F1/1683
• CPC Class Codes: G06F 1/1683 for the transmission of signal or power between the different housings, e.g. details of wired or wireless communication, passage of cabling
• CPC Class Codes: G06F 1/1692 the I/O peripheral being a secondary touch screen used as control interface, e.g. virtual buttons or sliders
• CPC Class Codes: G06F 16/51 Indexing; Data structures therefor; Storage structures
• CPC Class Codes: G06F 16/54 Browsing; Visualisation therefor
• CPC Class Codes: G06F 2203/04803 : Split screen, i.e. subdividing the display area or the window area into separate subareas
• CPC Class Codes: G06F 3/00 : Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
• CPC Class Codes: G06F 3/01 : Input arrangements or combined input and output arrangements for interaction between user and computer G06F3/16 takes precedence
• CPC Class Codes: G06F 3/016 : Input arrangements with force or tactile feedback as computer generated output to the user
• CPC Class Codes: G06F 3/017 : Gesture based interaction, e.g. based on a set of recognized hand gestures interaction based on gestures traced on a digitiser G06F3/04883
• CPC Class Codes: G06F 3/0346 : with detection of the device orientation or free movement in a 3D space, e.g. 3D mice, 6-DOF [six degrees of freedom] pointers using gyroscopes, accelerometers or tilt-sensors
• CPC Class Codes: G06F 3/041 : Digitisers, e.g. for touch screens or touch pads, characterised by the transducing means
• CPC Class Codes: G06F 3/0412 : Digitisers structurally integrated in a display
• CPC Class Codes: G06F 3/0416 : Control or interface arrangements specially adapted for digitisers
• CPC Class Codes: G06F 3/044 : by capacitive means
• CPC Class Codes: G06F 3/048 : Interaction techniques based on graphical user interfaces [GUI]
• CPC Class Codes: G06F 3/0481 : based on specific properties of the displayed interaction object or a metaphor-based environment, e.g. interaction with desktop elements like windows or icons, or assisted by a cursor's changing behaviour or appearance
• CPC Class Codes: G06F 3/04817 : using icons graphical or visual programming using iconic symbols G06F8/34
• CPC Class Codes: G06F 3/0482 : Interaction with lists of selectable items, e.g. menus
• CPC Class Codes: G06F 3/0483 : Interaction with page-structured environments, e.g. book metaphor
• CPC Class Codes: G06F 3/0484 : for the control of specific functions or operations, e.g. selecting or manipulating an object, an image or a displayed text element, setting a parameter value or selecting a range
• CPC Class Codes: G06F 3/04842 : Selection of displayed objects or displayed text elements G06F3/0482 takes precedence
• CPC Class Codes: G06F 3/04845 : for image manipulation, e.g. dragging, rotation, expansion or change of colour
• CPC Class Codes: G06F 3/04847 : Interaction techniques to control parameter settings, e.g. interaction with sliders or dials
• CPC Class Codes: G06F 3/0485 : Scrolling or panning
• CPC Class Codes: G06F 3/0486 : Drag-and-drop
• CPC Class Codes: G06F 3/0487 : using specific features provided by the input device, e.g. functions controlled by the rotation of a mouse with dual sensing arrangements, or of the nature of the input device, e.g. tap gestures based on pressure sensed by a digitiser
• CPC Class Codes: G06F 3/0488 : using a touch-screen or digitiser, e.g. input of commands through traced gestures
• CPC Class Codes: G06F 3/04883 : for inputting data by handwriting, e.g. gesture or text
• CPC Class Codes: G06F 3/04886 : by partitioning the display area of the touch-screen or the surface of the digitising tablet into independently controllable areas, e.g. virtual keyboards or menus
• CPC Class Codes: G06F 3/04897 : Special input arrangements or commands for improving display capability
• CPC Class Codes: G06F 3/1423 : controlling a plurality of local displays, e.g. CRT and flat panel display
• CPC Class Codes: G06F 3/1438 : using more than one graphics controller
• CPC Class Codes: G06F 3/1446 : display composed of modules, e.g. video walls
• CPC Class Codes: G06F 3/1454 : involving copying of the display data of a local workstation or window to a remote workstation or window so that an actual copy of the data is displayed simultaneously on two or more displays, e.g. teledisplay
• CPC Class Codes: G06F 3/167 : Audio in a user interface, e.g. using voice commands for navigating, audio feedback
• CPC Class Codes: G06F 9/00 : Arrangements for program control, e.g. control units program control for peripheral devices G06F13/10
• CPC Class Codes: G06F 9/44 : Arrangements for executing specific programs
• CPC Class Codes: G06F 9/451 : Execution arrangements for user interfaces
• CPC Class Codes: G06G 1/00 : Hand manipulated computing devices planimeters G01B5/26
• CPC Class Codes: G06T 3/00 : Geometric image transformations in the plane of the image
• CPC Class Codes: G06T 3/20 : Linear translation of whole images or parts thereof, e.g. panning
• CPC Class Codes: G06T 3/40 : Scaling of whole images or parts thereof, e.g. expanding or contracting
• CPC Class Codes: G09G 1/00 : Control arrangements or circuits, of interest only in connection with cathode-ray tube indicators; General aspects or details, e.g. selection emphasis on particular characters, dashed line or dotted line generation; Preprocessing of datacathode-ray oscilloscopes G01R13/20; radar display arrangements G01S7/04; display of digital non-picture data in television systems H04N7/0255
• CPC Class Codes: G09G 2300/023 : Display panel composed of stacked panels
• CPC Class Codes: G09G 2330/021 : Power management, e.g. power saving
• CPC Class Codes: G09G 2354/00 : Aspects of interface with display user
• CPC Class Codes: G09G 5/00 : Control arrangements or circuits for visual indicators common to cathode-ray tube indicators and other visual indicators image data processing or generation, in general G06T
• CPC Class Codes: G09G 5/12 : Synchronisation between the display unit and other units, e.g. other display units, video-disc players
• CPC Class Codes: G09G 5/14 : Display of multiple viewports
• CPC Class Codes: G09G 5/34 : for rolling or scrolling
• CPC Class Codes: G09G 5/373 : for modifying the size of the graphic pattern
• CPC Class Codes: G09G 5/377 : for mixing or overlaying two or more graphic patterns G09G5/02, G09G5/397 take precedence
• CPC Class Codes: H04B 1/3833 : Hand-held transceivers
• CPC Class Codes: H04M 1/0206 : Portable telephones comprising a plurality of mechanically joined movable body parts, e.g. hinged housings
• CPC Class Codes: H04M 1/0214 : Foldable telephones, i.e. with body parts pivoting to an open position around an axis parallel to the plane they define in closed position combined with translation movement H04M1/0235
• CPC Class Codes: H04M 1/0216 : Foldable in one direction, i.e. using a one degree of freedom hinge
• CPC Class Codes: H04M 1/0266 : for a display module assembly
• CPC Class Codes: H04N 21/4316 : for displaying supplemental content in a region of the screen, e.g. an advertisement in a separate window
• CPC Class Codes: H04N 21/47 : End-user applications
• CPC Class Codes: H04N 23/63 : by using electronic viewfinders
• CPC Class Codes: H04N 23/631 : Graphical user interfaces [GUI] specially adapted for controlling image capture or setting capture parameters
• CPC Class Codes: H04N 5/222 : Studio circuitry; Studio devices; Studio equipment cameras or camera modules comprising electronic image sensors, or control thereof H04N23/00
• CPC Class Codes: H04N 5/2628 : Alteration of picture size, shape, position or orientation, e.g. zooming, rotation, rolling, perspective, translation
• CPC Class Codes: H04W 24/02 : Arrangements for optimising operational condition
• CPC Class Codes: H04W 4/02 : Services making use of location information
• CPC Class Codes: H04W 48/18 : Selecting a network or a communication service
• CPC Class Codes: H04W 68/00 : User notification, e.g. alerting and paging, for incoming communication, change of service or the like
• CPC Class Codes: H04W 72/563 : of the wireless resources
• CPC Class Codes: H04W 88/02 : Terminal devices
• CPC Class Codes: H04W 88/06 : adapted for operation in multiple networks or having at least two operational modes, e.g. multi-mode terminals
• CPC Class Codes: H05K 13/00 : Apparatus or processes specially adapted for manufacturing or adjusting assemblages of electric components
• CPC Class Codes: H05K 13/046 : Surface mounting surface mounted components H05K3/341
• CPC Class Codes: H05K 5/0017 : with operator interface units
• CPC Class Codes: H05K 5/0226 : Hinges
• CPC Class Codes: H05K 5/04 : Metal casings
• CPC Class Codes: H05K 7/02 : Arrangements of circuit components or wiring on supporting structure
• CPC Class Codes: H05K 7/1452 : Mounting of connectors; Switching; Reinforcing of back panels
• CPC Class Codes: Y10T 29/49826 : Assembling or joining
• CPC Class Codes: Y10T 29/4984 : Retaining clearance for motion between assembled parts

Abstract

Methods and devices for receiving input and presenting a user interface with two screens and an off screen gesture area. The device may have an off screen gesture area that accepts user input outside the display area. The interface inputs received in the off screen gesture are may have special handling and cause different display changes. Further, the device, having two screens, may receive user interface inputs that cross the seam between the two displays. To provide a display that acts like a single display area, the device can predict motions may cross the seam and then interrelate separate inputs on separate screens. The interrelated inputs can cause display changes as if the inputs were received as a single user interaction.

Claims

1. A method, comprising:

providing a device having at least first and second screens with a seam located between the first and the second screens, wherein the device includes an off-screen gesture area;

receiving a first user interface input in the off-screen gesture area that triggers a mode change in the device to select an object on a display area of one of the first and second screens, the display area being separate from the off-screen gesture area;

receiving a second user interface input across the seam in a second off-screen gesture area on the second screen;

determining whether the first and second user interface inputs should be considered as a single user interaction or two separate user interactions based at least on a prediction that the first user interface input will continue across the seam onto the second screen;

determining an application mode of the device, and

in response to the first and second user interface inputs, and the determined application mode, changing a display of the object in one or more of the first and second screens.

2. The method of claim 1, wherein the first user interface input is a gesture.

3. The method of claim 2, wherein the gesture is a flick or a drag.

4. The method of claim 2, further comprising determine whether the first user input and second user input are within temporal proximity.

5. The method of claim 2, wherein the off-screen gesture area includes a gesture capture region.

6. The method of claim 5, wherein the gesture capture region is an area on a screen of the device that accepts user input but is not part of the display area.

7. The method of claim 6, wherein the change of the display is a movement of a window from a first screen to a second screen.

8. The method of claim 2, wherein the change of the display is an expansion of a window over the first and second screens.

9. The method of claim 7, wherein the device determines if the movement is completed.

10. The method of claim 9, if the movement is not completed, the window is returned to the first screen.

11. A device, comprising:

a first screen;

a second screen;

a seam located between the first and the second screens;

an off screen gesture area associated with the first screen;

a second off screen gesture area associated with second screen;

a memory;

a processor in communication with the memory, the first screen, and the second screen, the processor operable to;

receive a first user interface input in the off-screen gesture area that triggers a mode change in the device to select an object on a display area of one of the first and second screens, the display area being separate from the off-screen gesture area;

receive a second user interface input across the seam in the second off-screen gesture area;

determine whether the first and second user interface inputs should be considered to be a single user interaction or two separate user interactions based at least on a prediction that the first user interface input will continue across the seam onto the second screen;

determine an application mode of the device, and

in response to the first and second user interface inputs, and the determined application mode, change a display of the object in one or more of the first screen or second screen.

12. The device of claim 11, wherein the first user interface input is a gesture.

13. The device of claim 12, wherein the gesture is a flick or a drag and the processor is further adapted to determine whether the first user input and second user input are within temporal proximity.

14. The device of claim 13, wherein the change of the display is a movement of a window from a first screen to a second screen.

15. The device of claim 14, wherein the device determines if the movement is completed, and, if the movement is not completed, the window is returned to the first screen.

16. A non-transitory computer readable medium having stored thereon computer-executable instructions, the computer executable instructions causing a processor of a multi-screen device with a seam between a first screen and a second screen to execute a method for providing a user interface, the computer-executable instructions comprising:

instructions to receive a first user interface input in the off-screen gesture area of the first screen that triggers a mode change in the device to select an object on a display area of one of the first and second screens, the display area being separate from the off-screen gesture area;

instructions to receive a second user interface input across the seam in a second off-screen gesture area on the second screen;

instructions to determine whether the first user interface input and the second user interface input should be considered as a single user interaction or two separate user interactions based at least on a prediction that the first user interface input will continue across the seam onto the second screen;

instructions to determine an application mode of the device, and

in response to the first and second user interface inputs, and the determined application mode, instructions to change a display of the object in one or more of the first screen and second screen.

17. The non-transitory computer readable medium of claim 16, wherein the first user interface input is a gesture.

18. The non-transitory computer readable medium of claim 17, wherein the gesture is a flick or a drag and further comprising instructions to determine whether the first user input and second user input are within temporal proximity.

19. The non-transitory computer readable medium of claim 18, wherein the change of the display is a movement of a window from a first screen to a second screen or an expansion of the window.

20. The non-transitory computer readable medium of claim 16, wherein the device determines if the movement is completed, and, if the movement is not completed, the window is returned to the first screen.