Methods of interfacing with multi-input devices and multi-input display systems employing interfacing techniques

• US 9,256,342 B2
• Filed: 04/08/2009
• Issued: 02/09/2016
• Est. Priority Date: 04/10/2008
• Status: Active Grant

• Current Assignee: Microsoft Technology Licensing LLC (Microsoft Corporation)
• Original Assignee: Perceptive Pixel Incorporated (Microsoft Corporation)
• Inventors: Davidson, Philip L., Han, Jefferson Y.
• Primary Examiner(s): Pesin, Boris
• Assistant Examiner(s): HEFFINGTON, JOHN M
• Application Number:
  US12/420,501
• Publication Number:
  US 20090259965A1
• Time in Patent Office: 2,498 Days
• Field of Search: 715/788, 715/863, 715/761, 715/767, 715/768, 715/770, 715/778, 715/781, 715/790, 715/792, 715/798, 715/803, 715/815
• US Class Current: 1/1
• CPC Class Codes: G06F 2203/04806 Zoom, i.e. interaction techniques or interactors for controlling the zooming operation
• CPC Class Codes: G06F 2203/04808 Several contacts: gestures triggering a specific function, e.g. scrolling, zooming, right-click, when the user establishes several contacts with the surface simultaneously; e.g. using several fingers or a combination of fingers and pen
• CPC Class Codes: G06F 3/0481 based on specific properties of the displayed interaction object or a metaphor-based environment, e.g. interaction with desktop elements like windows or icons, or assisted by a cursor's changing behaviour or appearance
• CPC Class Codes: G06F 3/0485 Scrolling or panning
• CPC Class Codes: G06F 3/0488 using a touch-screen or digitiser, e.g. input of commands through traced gestures
• CPC Class Codes: G06F 3/04883 for inputting data by handwriting, e.g. gesture or text
• CPC Class Codes: G09G 5/14 Display of multiple viewports

Abstract

Methods and systems for interfacing with multi-input devices employ various techniques for controlling the window framing of images. Such techniques provide control, including moving, sizing, and orientating, of one or more displayed window frames in which one or more images are displayed.

Claims

1. A method of interfacing with a multi-input display device, the method comprising:

displaying on a multi-input display device at least first and second graphical windows having borders, the display device displaying a first portion and a second portion of an object, wherein the first portion is displayed within the first graphical window and the second portion is displayed within the second graphical window;

identifying a number, position and movement of one or more first elements contacting the displayed first graphical window;

identifying a number, position and movement of one or more second elements contacting the displayed second graphical window;

manipulating the displayed first graphical window as a function of the identified number, position and movement of the one or more first elements contacting the first graphical window;

manipulating the displayed second graphical window as a function of the identified number, position and movement of the one or more second elements contacting the second graphical window;

displaying on the display device, as a consequence of manipulating the first graphical window, at least a third portion of the object within the manipulated first graphical window such that positions on the display device of common portions of the first and the third portions of the object are substantially the same; and

displaying on the display device, as a consequence of manipulating the second graphical window, at least a fourth portion of the object within the manipulated second graphical window such that positions on the display device of common portions of the second and fourth portions of the object are substantially the same,

wherein manipulation of the first and second graphical windows results in an overlapping region between the windows, and further comprising;

removing the borders of the first and second graphical windows in the overlapping region to provide an unobstructed view of a portion of the object corresponding to the overlapping region.

2. The method of claim 1, wherein displaying the first portion and the second portion of the object includes displaying within the overlapping region content having a predefined characteristic and displaying within a non-overlapping region or regions or the first and second graphical windows content not having the predefined characteristic.

3. The method of claim 1,

wherein the object is a map, and

wherein displaying the first portion and the second portion of the object comprises displaying within the overlapping region a first portion of the map at a first resolution and displaying within a non-overlapping region or regions of the first and second graphical windows a second portion or portions of the map at a second resolution, the first and second resolutions being substantially different.

4. The method of claim 1,

wherein the object is a map, and

wherein displaying the first portion and the second portion of the object comprises displaying within the overlapping region a first portion of the map having real time traffic data, and displaying within a non-overlapping region or regions of the first and second graphical windows a second portion or portions of the map not having real time traffic data.

5. The method of claim 1,

wherein the object is a photograph, and

wherein displaying the first portion and the second portion of the object comprises displaying within the overlapping region a first portion of the photograph having an image enhancement characteristic, and displaying within a non-overlapping region or regions of the first and second graphical windows a second portion or portions of the photograph not having the image enhancement characteristic.

6. The method of claim 1,

wherein the object represents information containing hidden data, and

wherein displaying the first portion and the second portion of the object comprises displaying within the overlapping region at least the part of the first portion of the object and at least the part of the second portion of the object and revealing within the overlapping region at least a portion of the hidden data, and displaying within a non-overlapping region or regions of the first and second graphical windows an other part or other parts of the object and not revealing any of the hidden data within the non-overlapping region or regions.

7. The method of claim 1,

wherein displaying at least the first and the second graphical windows includes displaying the first and the second graphical windows with the overlapping region and respective non-overlapping regions, and

wherein displaying the first portion and the second portion of the object includes:

displaying within the overlapping region a portion of the object with a first characteristic,

displaying within the non-overlapping region of the displayed first graphical window a portion of the object with a second characteristic, and

displaying within the non-overlapping region of the displayed second graphical window a portion of the object with a third characteristic, the first, second and third characteristics being different from one another.

8. The method of claim 1,

wherein at least the first graphical window has a non-overlapping region, and

wherein displaying the first portion and the second portion of the object comprises:

displaying a part of the object within the entire second graphical window in accordance with a predefined characteristic, and

displaying a part of the object within the non-overlapping region of the first graphical window not in accordance with the predefined characteristic.

9. The method of claim 1, further comprising uniformly changing at least one of a shape and a size of both the first and the second graphical windows as a function of the position and movement of elements contacting both the first and the second graphical windows.

10. The method of claim 9, further comprising manipulating portions of the object displayed within the manipulated first and second graphical windows in a same manner the first and the second graphical windows are changed.

21. The method of claim 1 further comprising joining the first and second graphical windows to form a single graphical window.

11. A multi-input display system, comprising:

a display device configured to display on a display surface at least first and second graphical windows having borders, the display device displaying a first portion and a second portion of an object, wherein the first portion is displayed within the first graphical window and the second portion is displayed within the second graphical window, the display device adapted to detect one or more elements contacting the display surface; and

a controller adapted to perform operations including;

identify a number, a position and a movement of one or more first elements contacting the displayed first graphical window,

identify a number, a position and a movement of one or more second elements contacting the displayed second graphical window,

manipulate the displayed first graphical window as a function of the identified number, position and movement of the one or more first elements contacting the first graphical window,

manipulate the displayed second graphical window as a function of the identified number, position and movement of the one or more second elements contacting the second graphical window,

control the display device to display on the display surface, as a consequence of manipulating the first graphical window, at least a third portion of the object within the manipulated first graphical window such that positions on the display surface of common portions of the first and the third portions of the object are substantially the same; and

control the display device to display on the display surface, as a consequence of manipulating the second graphical window, at least a fourth portion of the object within the manipulated second graphical window such that positions on the display surface of common portions of the second and fourth positions of the object are substantially the same,

wherein manipulation of the first and second graphical windows results in an overlapping region between the windows, and further comprising;

removing the borders of the first and second graphical windows in the overlapping region to provide an unobstructed view of a portion of the object corresponding to the overlapping region.

12. The system of claim 11, wherein the controller being adapted to control the display device to display the first and the second graphical windows comprises the controller being adapted to control the display device to display within the overlapping region content having a predefined characteristic and control the display device to display within a non-overlapping region or regions of the first and second graphical windows content not having the predefined characteristic.

13. The system of claim 11,

wherein the object is a map, and

wherein the controller being adapted to control the display device to display the first and the second graphical windows comprises the controller being adapted to control the display device to display within the overlapping region a first portion of the map at a first resolution and control the display device to display within a non-overlapping region or regions of the first and second graphical windows a second portion or portions of the map at a second resolution, the first and second resolutions being substantially different.

14. The system of claim 11,

wherein the object is a map, and

wherein the controller being adapted to control the display device to display the first and the second graphical windows comprises the controller being adapted to control the display device to display within the overlapping region a first portion of the map having real time traffic data, and control the display device to display within a non-overlapping region or regions of the first and second graphical windows a second portion or portions of the map not having real time traffic data.

15. The system of claim 11,

wherein the object is a photograph, and

wherein the controller being adapted to control the display device to display the first and the second graphical windows comprises the controller being adapted to control the display device to display within the overlapping region a first portion of the photograph having an image enhancement characteristic, and control the display device to display within a non-overlapping region or regions of the first and second graphical windows a second portion or portions of the photograph not having the image enhancement characteristic.

16. The system of claim 11,

wherein the object represents information containing hidden data, and

wherein the controller being adapted to control the display device to display the first and second graphical windows comprises the controller being adapted to control the display device to display within the overlapping region at least the part of the first portion of the object and at least the part of the second portion of the object and to reveal within the overlapping region at least a portion of the hidden data, and control the display device to display within a non-overlapping region or regions of the first and second graphical windows an other part or other parts of the object and to not reveal any of the hidden data within the non-overlapping region or regions.

17. The system of claim 11,

wherein the controller is adapted to control the display device to display the first and second graphical windows with the overlapping region and respective non-overlapping regions, and

wherein the controller being adapted to control the display device to display the first and the second graphical windows comprises the controller being adapted to control the display device to display within the overlapping region a portion of the object with a first characteristic, to display within the non-overlapping region of the displayed first graphical window a portion of the object with a second characteristic, and to display within the non-overlapping region of the displayed second graphical window a portion of the object with a third characteristic, the first, the second and the third characteristics being different from one another.

18. The system of claim 11,

wherein the controller is adapted to control the display device to display at least a non-overlapping region within the first graphical window, and

wherein the controller being adapted to control the display device to display the first and the second graphical windows comprises the controller being adapted to control the display device to display within the overlapping region a portion of the object in accordance with a predefined characteristic, and to display within the non-overlapping region of the first graphical window a portion of the object not in accordance with the predefined characteristic.

19. The system of claim 11, wherein the controller is adapted to uniformly change at least one of a shape and a size of both the displayed first and the second graphical windows as a function of the position and movement of elements contacting both the first and the second graphical windows.

20. The system of claim 19, wherein the controller is adapted to manipulate portions of the object displayed within the manipulated first and the second graphical windows in a same manner the first and the second graphical windows are changed.

22. The system of claim 19 wherein the controller is adapted to join the first and second graphical windows to form a single graphical window.