Drag motion across seam of displays

• US 9,075,558 B2
• Filed: 09/27/2012
• Issued: 07/07/2015
• Est. Priority Date: 09/27/2011
• Status: Active Grant

• Current Assignee: Z124 LLC (Flexis)
• Original Assignee: Z124 LLC (Flexis)
• Inventors: Reeves, Paul E., Visosky, John S., Chen, Tong
• Primary Examiner(s): NABI, REZA U
• Application Number:
  US13/629,085
• Publication Number:
  US 20130086493A1
• Time in Patent Office: 1,013 Days
• Field of Search: 715/761
• US Class Current: 1/1
• CPC Class Codes: B29D 11/00673 Supports for light guides
• CPC Class Codes: E05D 3/12 with two parallel pins and one arm
• CPC Class Codes: E05Y 2900/606 for electronic devices
• CPC Class Codes: G02B 6/0001 specially adapted for lighting devices or systems lighting or signalling on vehicles using light guides B60Q1/00; lighting devices for vehicle dashboards B60Q3/10; lighting devices for vehicle interior using light guides B60Q3/62; lighting devices mounted on the vehicle rear part using light guides F21S43/235; measuring arrangements having light conducting pointers G01D13/265; illumination of liquid crystal displays G02F1/1336; illuminated signs G09F13/00
• CPC Class Codes: G06F 1/16 Constructional details or arrangements
• CPC Class Codes: G06F 1/1601 Constructional details related to the housing of computer displays, e.g. of CRT monitors, of flat displays constructional details related to flat displays integrated in a portable computer, e.g. laptop, handheld computer G06F1/1637; constructional details related to television receivers H04N5/64
• CPC Class Codes: G06F 1/1605 Multimedia displays, e.g. with integrated or attached speakers, cameras, microphones
• CPC Class Codes: G06F 1/1616 with folding flat displays, e.g. laptop computers or notebooks having a clamshell configuration, with body parts pivoting to an open position around an axis parallel to the plane they define in closed position
• CPC Class Codes: G06F 1/1618 the display being foldable up to the back of the other housing with a single degree of freedom, e.g. by 360° rotation over the axis defined by the rear edge of the base enclosure
• CPC Class Codes: G06F 1/1637 Details related to the display arrangement, including those related to the mounting of the display in the housing
• CPC Class Codes: G06F 1/1641 the display being formed by a plurality of foldable display components G06F1/1647 takes precedence
• CPC Class Codes: G06F 1/1643 the display being associated to a digitizer, e.g. laptops that can be used as penpads details related to the relative motion of the display enclosure with respect to the body enclosure, e.g. to move between laptop and tablet PC configuration G06F1/1615
• CPC Class Codes: G06F 1/1647 including at least an additional display G06F1/1692 takes precedence
• CPC Class Codes: G06F 1/1649 the additional display being independently orientable, e.g. for presenting information to a second user
• CPC Class Codes: G06F 1/1677 for detecting open or closed state or particular intermediate positions assumed by movable parts of the enclosure, e.g. detection of display lid position with respect to main body in a laptop, detection of opening of the cover of battery compartment
• CPC Class Codes: G06F 1/1681 Details related solely to hinges hinge details related to the transmission of signals or power are classified in G06F1/1683
• CPC Class Codes: G06F 1/1683 for the transmission of signal or power between the different housings, e.g. details of wired or wireless communication, passage of cabling
• CPC Class Codes: G06F 1/1692 the I/O peripheral being a secondary touch screen used as control interface, e.g. virtual buttons or sliders
• CPC Class Codes: G06F 16/51 Indexing; Data structures therefor; Storage structures
• CPC Class Codes: G06F 16/54 Browsing; Visualisation therefor
• CPC Class Codes: G06F 2203/04803 : Split screen, i.e. subdividing the display area or the window area into separate subareas
• CPC Class Codes: G06F 3/00 : Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
• CPC Class Codes: G06F 3/01 : Input arrangements or combined input and output arrangements for interaction between user and computer G06F3/16 takes precedence
• CPC Class Codes: G06F 3/016 : Input arrangements with force or tactile feedback as computer generated output to the user
• CPC Class Codes: G06F 3/017 : Gesture based interaction, e.g. based on a set of recognized hand gestures interaction based on gestures traced on a digitiser G06F3/04883
• CPC Class Codes: G06F 3/0346 : with detection of the device orientation or free movement in a 3D space, e.g. 3D mice, 6-DOF [six degrees of freedom] pointers using gyroscopes, accelerometers or tilt-sensors
• CPC Class Codes: G06F 3/041 : Digitisers, e.g. for touch screens or touch pads, characterised by the transducing means
• CPC Class Codes: G06F 3/0412 : Digitisers structurally integrated in a display
• CPC Class Codes: G06F 3/0416 : Control or interface arrangements specially adapted for digitisers
• CPC Class Codes: G06F 3/044 : by capacitive means
• CPC Class Codes: G06F 3/048 : Interaction techniques based on graphical user interfaces [GUI]
• CPC Class Codes: G06F 3/0481 : based on specific properties of the displayed interaction object or a metaphor-based environment, e.g. interaction with desktop elements like windows or icons, or assisted by a cursor's changing behaviour or appearance
• CPC Class Codes: G06F 3/04817 : using icons graphical or visual programming using iconic symbols G06F8/34
• CPC Class Codes: G06F 3/0482 : Interaction with lists of selectable items, e.g. menus
• CPC Class Codes: G06F 3/0483 : Interaction with page-structured environments, e.g. book metaphor
• CPC Class Codes: G06F 3/0484 : for the control of specific functions or operations, e.g. selecting or manipulating an object, an image or a displayed text element, setting a parameter value or selecting a range
• CPC Class Codes: G06F 3/04842 : Selection of displayed objects or displayed text elements G06F3/0482 takes precedence
• CPC Class Codes: G06F 3/04845 : for image manipulation, e.g. dragging, rotation, expansion or change of colour
• CPC Class Codes: G06F 3/04847 : Interaction techniques to control parameter settings, e.g. interaction with sliders or dials
• CPC Class Codes: G06F 3/0485 : Scrolling or panning
• CPC Class Codes: G06F 3/0486 : Drag-and-drop
• CPC Class Codes: G06F 3/0487 : using specific features provided by the input device, e.g. functions controlled by the rotation of a mouse with dual sensing arrangements, or of the nature of the input device, e.g. tap gestures based on pressure sensed by a digitiser
• CPC Class Codes: G06F 3/0488 : using a touch-screen or digitiser, e.g. input of commands through traced gestures
• CPC Class Codes: G06F 3/04883 : for inputting data by handwriting, e.g. gesture or text
• CPC Class Codes: G06F 3/04886 : by partitioning the display area of the touch-screen or the surface of the digitising tablet into independently controllable areas, e.g. virtual keyboards or menus
• CPC Class Codes: G06F 3/04897 : Special input arrangements or commands for improving display capability
• CPC Class Codes: G06F 3/1423 : controlling a plurality of local displays, e.g. CRT and flat panel display
• CPC Class Codes: G06F 3/1438 : using more than one graphics controller
• CPC Class Codes: G06F 3/1446 : display composed of modules, e.g. video walls
• CPC Class Codes: G06F 3/1454 : involving copying of the display data of a local workstation or window to a remote workstation or window so that an actual copy of the data is displayed simultaneously on two or more displays, e.g. teledisplay
• CPC Class Codes: G06F 3/167 : Audio in a user interface, e.g. using voice commands for navigating, audio feedback
• CPC Class Codes: G06F 9/00 : Arrangements for program control, e.g. control units program control for peripheral devices G06F13/10
• CPC Class Codes: G06F 9/44 : Arrangements for executing specific programs
• CPC Class Codes: G06F 9/451 : Execution arrangements for user interfaces
• CPC Class Codes: G06G 1/00 : Hand manipulated computing devices planimeters G01B5/26
• CPC Class Codes: G06T 3/00 : Geometric image transformations in the plane of the image
• CPC Class Codes: G06T 3/20 : Linear translation of whole images or parts thereof, e.g. panning
• CPC Class Codes: G06T 3/40 : Scaling of whole images or parts thereof, e.g. expanding or contracting
• CPC Class Codes: G09G 1/00 : Control arrangements or circuits, of interest only in connection with cathode-ray tube indicators; General aspects or details, e.g. selection emphasis on particular characters, dashed line or dotted line generation; Preprocessing of datacathode-ray oscilloscopes G01R13/20; radar display arrangements G01S7/04; display of digital non-picture data in television systems H04N7/0255
• CPC Class Codes: G09G 2300/023 : Display panel composed of stacked panels
• CPC Class Codes: G09G 2330/021 : Power management, e.g. power saving
• CPC Class Codes: G09G 2354/00 : Aspects of interface with display user
• CPC Class Codes: G09G 5/00 : Control arrangements or circuits for visual indicators common to cathode-ray tube indicators and other visual indicators image data processing or generation, in general G06T
• CPC Class Codes: G09G 5/12 : Synchronisation between the display unit and other units, e.g. other display units, video-disc players
• CPC Class Codes: G09G 5/14 : Display of multiple viewports
• CPC Class Codes: G09G 5/34 : for rolling or scrolling
• CPC Class Codes: G09G 5/373 : for modifying the size of the graphic pattern
• CPC Class Codes: G09G 5/377 : for mixing or overlaying two or more graphic patterns G09G5/02, G09G5/397 take precedence
• CPC Class Codes: H04B 1/3833 : Hand-held transceivers
• CPC Class Codes: H04M 1/0206 : Portable telephones comprising a plurality of mechanically joined movable body parts, e.g. hinged housings
• CPC Class Codes: H04M 1/0214 : Foldable telephones, i.e. with body parts pivoting to an open position around an axis parallel to the plane they define in closed position combined with translation movement H04M1/0235
• CPC Class Codes: H04M 1/0216 : Foldable in one direction, i.e. using a one degree of freedom hinge
• CPC Class Codes: H04M 1/0266 : for a display module assembly
• CPC Class Codes: H04N 21/4316 : for displaying supplemental content in a region of the screen, e.g. an advertisement in a separate window
• CPC Class Codes: H04N 21/47 : End-user applications
• CPC Class Codes: H04N 23/63 : by using electronic viewfinders
• CPC Class Codes: H04N 23/631 : Graphical user interfaces [GUI] specially adapted for controlling image capture or setting capture parameters
• CPC Class Codes: H04N 5/222 : Studio circuitry; Studio devices; Studio equipment cameras or camera modules comprising electronic image sensors, or control thereof H04N23/00
• CPC Class Codes: H04N 5/2628 : Alteration of picture size, shape, position or orientation, e.g. zooming, rotation, rolling, perspective, translation
• CPC Class Codes: H04W 24/02 : Arrangements for optimising operational condition
• CPC Class Codes: H04W 4/02 : Services making use of location information
• CPC Class Codes: H04W 48/18 : Selecting a network or a communication service
• CPC Class Codes: H04W 68/00 : User notification, e.g. alerting and paging, for incoming communication, change of service or the like
• CPC Class Codes: H04W 72/563 : of the wireless resources
• CPC Class Codes: H04W 88/02 : Terminal devices
• CPC Class Codes: H04W 88/06 : adapted for operation in multiple networks or having at least two operational modes, e.g. multi-mode terminals
• CPC Class Codes: H05K 13/00 : Apparatus or processes specially adapted for manufacturing or adjusting assemblages of electric components
• CPC Class Codes: H05K 13/046 : Surface mounting surface mounted components H05K3/341
• CPC Class Codes: H05K 5/0017 : with operator interface units
• CPC Class Codes: H05K 5/0226 : Hinges
• CPC Class Codes: H05K 5/04 : Metal casings
• CPC Class Codes: H05K 7/02 : Arrangements of circuit components or wiring on supporting structure
• CPC Class Codes: H05K 7/1452 : Mounting of connectors; Switching; Reinforcing of back panels
• CPC Class Codes: Y10T 29/49826 : Assembling or joining
• CPC Class Codes: Y10T 29/4984 : Retaining clearance for motion between assembled parts

Abstract

Methods and devices for receiving input and presenting a user interface with two screens and an off screen gesture area. The device may have an off screen gesture area that accepts user input outside the display area. The interface inputs received in the off screen gesture are may have special handling and cause different display changes. Further, the device, having two screens, may receive user interface inputs that cross the seam between the two displays. To provide a display that acts like a single display area, the device can predict motions may cross the seam and then interrelate separate inputs on separate screens. The interrelated inputs can cause display changes as if the inputs were received as a single user interaction.

Claims

1. A method, comprising:

providing a device having at least a first screen with a first display, a second screen with a second display, and a seam between the first and second screens;

displaying a first desktop on the first display;

opening a first application;

ceasing display of the first desktop on the first display;

displaying a first window of the first application on the first display, wherein the first window is active;

opening a second application;

displaying a second window of the second application on the second display, wherein the second window is active;

receiving a first user interface input in the first screen;

determining a vector for the first user interface input;

predicting, using the vector, that the first user interface input will continue onto the second screen;

waiting an amount of time for a second user interface input to be received in the second screen;

receiving the second user interface input in the second screen,

wherein the first user interface is in temporal proximity to the second user interface input;

determining, when the second user interface input is received within the amount of time, that the first user interface input and the second user interface input are part of a single user interaction with the device, wherein the single user interaction is across the seam of the device; and

interrelating the first and second user inputs,

wherein determining the vector for the first user interface input further comprises;

determining a speed of the first user interface input; and

determining a direction of the first user interface input, wherein the amount of time is based on the determined vector.

2. The method claim 1, screen further comprising:

moving the first window to the second display;

ceasing display of the second window on the second display, wherein the second window is inactive;

displaying the first desktop on the first display; and

displaying the first window on the second display.

3. The method of claim 2, wherein the single user interaction is a drag of the first window.

4. The method of claim 2, wherein moving the first window to the second display further comprises:

moving the first window in a direction of the vector toward the second display, wherein

during the moving the first window is displayed at least partially on each of the first display and the second display;

during the moving, uncovering the first desktop on the first display as the first window moves onto the second display; and

during the moving, covering the second window with the first window as the first window moves onto the second display.

5. The method of claim 1, further comprising:

determining, when the second user interface input is not received within the amount of time, that the first user interface input and the second user interface input are not part of the single user interaction with the device; and

dropping the first window on the first display.

6. The method of claim 5, first screen further comprising:

before determining that the first user interface input and the second user interface input are not part of the single user interaction with the device, moving the first window in a direction of the vector toward the second display, wherein during the moving the first window is displayed at least partially on each of the first display and the second display, wherein the first desktop is at least partially displayed on the first display, and wherein the second window is at least partially covered by the first window on the second display; and

after determining that the first user interface input and the second user interface input are not part of the single user interaction with the device, moving the first window back to the first display, wherein the first desktop is not displayed, and wherein the second window is displayed.

7. The method of claim 1 , wherein, for at least a portion of the single user interaction, a finger of the user crosses the seam and is not in physical contact with either a first display or the second display.

8. A device, comprising:

a first screen with a first display;

a second screen with a second display;

an off screen gesture area;

a memory;

a processor in communication with the memory, the first screen, and the second screen, the processor operable to;

display a first desktop on the first display;

open a first application;

cease display of the first desktop on the first display;

display a first window of the first application on the first display, wherein the first window is active;

open a second application;

display a second window of the second application on the second display, wherein the second window is active;

receive a first user interface input in the first screen;

determine a vector for the first user interface input;

predict, using the vector, that the first user interface input will continue onto the second screen;

wait an amount of time for the second user interface input to be received in the second screen;

receive the second user interface input in the second screen, wherein the first user interface input is in temporal proximity to the second user interface input;

determining, when the second user interface input is received within the amount of time, that the first user interface input and the second user interface input are part of a single user interaction with the device, wherein the single user interaction is across the seam of the device; and

interrelate the first and second user inputs;

wherein determining the vector for the first user interface input further comprises the operations;

determining a speed of the first user interface input; and

determining a direction of the first user interface input, wherein the amount of time is based on the determined vector.

9. The device of claim 8, wherein the first user interface input is in temporal proximity to the second user interface input, wherein if the second user interface input is not within temporal proximity of the first user interface input, the first window is dropped on the display.

10. The device of claim 8, wherein the processor is further operable to determine, when the second user interface input is received within the amount of time, that the first user interface input and the second user interface input are part of a single user interaction with the device.

11. The device of claim 10, wherein the single user interaction is a drag of the first window, wherein the first window is selected on the first screen, and wherein the processor is further operable to:

before determining that the first user interface input and the second user interface input are part of the single user interaction with the device, move the first window in a direction of the vector toward the second display, wherein during the moving the first window is displayed at least partially on each of the first display and the second display, wherein the first desktop is at least partially displayed on the first display, and wherein the second window is at least partially covered by the first window on the second display;

after determining that the first user interface input and the second user interface input are part of a single user interaction with the device, move the first window to the second display, wherein the processor ceases display of the second window on the second display, wherein the second window is inactive, wherein the first desktop is displayed on the first display, and wherein first window is displayed on the second display; and

after determining, when the second user interface input is not received within the amount of time, that the first user interface input and the second user interface input are not part of a single user interaction with the device, move the first window back to the first display, wherein the first desktop is not displayed on the first display, and wherein the second window is displayed on the second display.

12. The device of claim 11, wherein, for at least a portion of the single user interaction, a finger of the user crosses a seam between the first and second screens and is not in physical contact with either the first or second display and wherein determining the vector for the first user interface input further comprises the processor:

determining a speed of the first user interface input; and

determining a direction of the first user interface input.

13. A non-transitory computer readable medium having stored thereon computer-executable instructions, the computer executable instructions causing a processor of a device to execute a method for providing a user interface, the computer-executable instructions comprising:

instructions to display a first desktop on a first display of a first screen of the device;

instructions to open a first application;

instructions to cease display of the first desktop on the first display;

instructions to display a first window of the first application on the first display, wherein the first window is active;

instructions to open a second application;

instructions to display a second window of the second application on a second display of a second screen of the device, wherein the second window is active;

instructions to receive a first user interface input in the first screen;

instructions to determine a vector for the first user interface input;

instructions to predict, using the vector, that the first user interface input will continue onto the second screen;

instructions to wait an amount of time for the second user interface input to be received in the second screen;

instructions to receive the second user interface input in the second screen, wherein the first user interface input is in temporal proximity to the second user interface input;

instructions to determine, when the second user interface input is received within the amount of time, that the first user interface input and the second user interface input are part of a single user interaction with the device, wherein the single user interaction is across the seam of the device; and

instructions to interrelate the first and second user inputs;

wherein the instructions to determine the vector for the first user interface input further comprises instructions to;

determine a speed of the first user interface input; and

determine a direction of the first user interface input, wherein the amount of time is based on the determined vector.

14. The non-transitory computer readable medium of claim 13, wherein the first user interface input is in temporal proximity to the second user interface input, wherein if the second user interface input is not within temporal proximity of the first user interface input, the first window is dropped on the display, and wherein the instructions to determine the vector for the first user interface input further comprise instructions to determine a speed of the first user interface input and instructions to determine a direction of the first user interface input.

15. The non-transitory computer readable medium of claim 13, further comprising instructions to determine, when the second user interface input is received within the amount of time, that the first user interface input and the second user interface input are part of a single user interaction with the device.

16. The non-transitory computer readable medium of claim 15, wherein the single user interaction is a drag of the first window, and wherein the instructions further comprise:

instructions to move, before determining that the first user interface input and the second user interface input are part of the single user interaction with the device, the first window in a direction of the vector toward the second display, wherein during the moving the first window is displayed at least partially on each of the first display and the second display, wherein the first desktop is at least partially displayed on the first display, and wherein the second window is at least partially covered by the first window on the second display;

instructions to move, after determining that the first user interface input and the second user interface input are part of a single user interaction with the device, the first window to the second display, wherein the processor ceases display of the second window on the second display, wherein the second window is inactive, wherein the first desktop is displayed on the first display, and wherein first window is displayed on the second display; and

instructions to move, after determining, when the second user interface input is not received within the amount of time, that the first user interface input and the second user interface input are not part of a single user interaction with the device, the first window back to the first display, wherein the first desktop is not displayed on the first display, and wherein the second window is displayed on the second display.

17. The non-transitory computer readable medium of claim 15, wherein, for at least a portion of the single user interaction, a finger of the user crosses the seam and is not in physical contact with either a first or second touch sensitive display.