Methods of interfacing with multi-input devices and multi-input display systems employing interfacing techniques

• US 9,372,591 B2
• Filed: 04/08/2009
• Issued: 06/21/2016
• Est. Priority Date: 04/10/2008
• Status: Active Grant

• Current Assignee: Microsoft Technology Licensing LLC (Microsoft Corporation)
• Original Assignee: Perceptive Pixel Incorporated (Microsoft Corporation)
• Inventors: Davidson, Philip L., Han, Jefferson Y.
• Primary Examiner(s): Johnson, M Good
• Application Number:
  US12/420,480
• Publication Number:
  US 20090256857A1
• Time in Patent Office: 2,631 Days
• Field of Search: 345/648, 345/173
• US Class Current: 1/1
• CPC Class Codes: G06F 2203/04806 Zoom, i.e. interaction techniques or interactors for controlling the zooming operation
• CPC Class Codes: G06F 2203/04808 Several contacts: gestures triggering a specific function, e.g. scrolling, zooming, right-click, when the user establishes several contacts with the surface simultaneously; e.g. using several fingers or a combination of fingers and pen
• CPC Class Codes: G06F 3/0481 based on specific properties of the displayed interaction object or a metaphor-based environment, e.g. interaction with desktop elements like windows or icons, or assisted by a cursor's changing behaviour or appearance
• CPC Class Codes: G06F 3/0485 Scrolling or panning
• CPC Class Codes: G06F 3/0488 using a touch-screen or digitiser, e.g. input of commands through traced gestures
• CPC Class Codes: G06F 3/04883 for inputting data by handwriting, e.g. gesture or text
• CPC Class Codes: G09G 5/14 Display of multiple viewports

Abstract

Methods and systems for interfacing with multi-input devices employ various techniques for controlling the window framing of images. Such techniques provide control, including moving, sizing, and orientating, of one or more displayed window frames in which one or more images are displayed.

Claims

1. A method of interfacing with a multi-input display device, comprising the steps of:

displaying on a multi-input display device a graphical window and at least a first portion of an image not displayed in its entirety within the graphical window;

accessing data that identifies a position and movement of three or more input elements that are simultaneously in contact with three or more different edges of the displayed graphical window;

selecting, based on the accessed data,

a modification implemented when the three or more input elements are simultaneously in contact with the three or more different edges of the displayed graphical window, the modification simultaneously moving each of the three or more edges of the displayed graphical window contacted by the three or more input elements in a respective direction of identified movement of the respective input element normal to an axis of the respective edge while the position of the displayed first portion of the image remains unchanged relative to the display device;

manipulating the displayed graphical window in accordance with the selected modification to reveal new content of the image not displayed in its entirety; and

based on the manipulation of the displayed graphical window, displaying on the display device at least a second portion of the image not displayed in its entirety within the manipulated graphical window that is different from the first portion, but that includes the newly revealed content, wherein positions on the display device of common portions of the first and second portions of the image not displayed in its entirety are substantially the same.

2. The method of claim 1, wherein:

the modification corresponds to simultaneously moving each of the edges of the displayed graphical window contacted by a respective one of the input elements in a direction of identified movement of the respective input element normal to an axis parallel to the respective edge.

3. The method of claim 1,

wherein displaying the graphical window comprises displaying the graphical window over a background display,

wherein the modification simultaneously moving each of the edges of the displayed graphical window comprises the modification simultaneously moving each of the edges of the displayed graphical window relative to the background display.

4. The method of claim 1, wherein displaying on the display device at least a second portion of the image not displayed in its entirety within the manipulated graphical window comprises displaying on the display device at least a second portion of the image not displayed in its entirety within the manipulated graphical window that is different from the first portion, but that includes the newly revealed content, wherein positions on the display device of common portions of the first and second portions of the image not displayed in its entirety are the same.

5. A multi-input display system, comprising:

a display device configured to display on a display surface a graphical window and at least a first portion of an image not displayed in its entirety within the graphical window and detect one or more input elements contacting the display surface; and

a controller configured to access data that identifies a position and movement of three or more input elements that are simultaneously in contact with three or more different edges of the displayed graphical window, the controller being adapted to select, based on the accessed data, a modification from among multiple modifications and manipulate the displayed graphical window in accordance with the selected modification,

wherein the selected modification is implemented when the three or more input elements are simultaneously in contact with the three or more different edges of the graphical window, the modification simultaneously moving each of the three or more edges of the graphical window contacted by the three or more input elements in a respective direction of identified movement of the respective input element normal to an axis of the respective edge while the position of the displayed first portion of the image remains unchanged relative to the display surface,

wherein the controller is configured to manipulate the displayed graphical window to reveal new content of the image not displayed in its entirety based on the selected modification to display on the display surface at least a second portion of the image not displayed in its entirety within the manipulated graphical window that is different from the first portion, but that includes the newly revealed content, wherein positions on the display surface of common portions of the first and the second portions of the image not displayed in its entirety are substantially the same.

6. The system of claim 5, wherein:

the modification corresponds to simultaneously moving each of the edges of the displayed graphical window contacted by a respective one of the input elements in a direction of identified movement of the respective input element normal to an axis parallel to the respective edge.

7. The system of claim 5,

wherein the display device being configured to display on the display surface the graphical window comprises the display device being configured to display on the display surface the graphical window over a background display,

wherein the selected modification simultaneously moving each of the edges of the displayed graphical window comprises the selected modification simultaneously moving each of the edges of the displayed graphical window relative to the background display.

8. The system of claim 5, wherein the controller being configured to control the display device comprises the controller being configured to control the display device to display on the display surface at least a second portion of the image not displayed in its entirety within the manipulated graphical window that is different from the first portion, but that includes the newly revealed content, wherein positions on the display surface of common portions of the first and the second portions of the image not displayed in its entirety are the same.

9. A method of interfacing with a multi-input display device, comprising the steps of:

displaying on a multi-input display device a graphical window and at least a first portion of an image not displayed in its entirety within the graphical window;

accessing data that identifies positions and movement of a plurality of input elements that are simultaneously in contact with one or more edges of the displayed graphical window;

selecting, based on the accessed data, either;

a first modification implemented when two of the plurality of input elements are simultaneously in contact with adjacent edges of the displayed graphical window, the first modification changing at least one of a shape and a size of the displayed graphical window in accordance with the identified positions and movement of the two input elements contacting the adjacent edges while the position of the displayed first portion of the image remains unchanged relative to the display device, or

a second modification implemented when two of the plurality of input elements are simultaneously in contact with a same edge of the displayed graphical window, the second modification changing at least one of a position and a shape of the displayed graphical window in accordance with the identified positions and movement of the two input elements contacting the same edge while the position of the displayed first portion of the image remains unchanged relative to the display device;

manipulating the displayed graphical window in accordance with the selected modification to reveal new content of the image not displayed in its entirety; and

based on the manipulation of the displayed graphical window, displaying on the display device at least a second portion of the image not displayed in its entirety within the manipulated graphical window that is different from the first portion, but that includes the newly revealed content, wherein positions on the display device of common portions of the first and the second portions of the image not displayed in its entirety are substantially the same.

10. The method of claim 9, wherein the first modification comprises uniformly scaling the size of the displayed graphical window in accordance with the identified movement of the two input elements contacting the adjacent edges.

11. The method of claim 9, wherein the second modification comprises changing the shape of the displayed graphical window by changing a length of said same edge in accordance with a change in distance between the positions of the two input elements contacting said same edge.

12. The method of claim 9,

wherein displaying the graphical window comprises displaying the graphical window over a background display,

wherein the first modification changing at least one of a shape and a size of the displayed graphical window comprises the first modification changing at least one of the shape and the size of the displayed graphical window relative to the background display, and

wherein the second modification changing at least one of a position and a shape of the displayed graphical window comprises the second modification changing at least one of the position and the shape of the displayed graphical window relative to the background display.

13. The method of claim 9, wherein selecting either the first modification or the second modification comprises selecting the first modification for manipulating the displayed graphical window, and

wherein manipulating the displayed graphical window in accordance with the selected modification comprises manipulating the displayed graphical window in accordance with the selected first modification.

14. The method of claim 9, wherein selecting either the first modification or the second modification comprises selecting the second modification for manipulating the displayed graphical window, and

wherein manipulating the displayed graphical window in accordance with the selected modification comprises manipulating the displayed graphical window in accordance with the selected second modification.

15. The method of claim 9, wherein displaying on the display device at least a second portion of the image not displayed in its entirety within the manipulated graphical window comprises displaying on the display device at least a second portion of the image not displayed in its entirety within the manipulated graphical window that is different from the first portion, but that includes the newly revealed content, wherein positions on the display device of common portions of the first and the second portions of the image not displayed in its entirety are the same.

16. A multi-input display system, comprising:

a display device configured to display on a display surface a graphical window and at least a first portion of an image not displayed in its entirety within the graphical window and detect input elements simultaneously in contact with the display surface; and

a controller configured to access data that identifies positions and movement of a plurality of input elements that are simultaneously in contact with one or more edges of the displayed graphical window, select, based on the accessed data, a modification from among multiple modifications, and manipulate the displayed graphical window in accordance with the selected modification, the multiple modifications either;

a first modification implemented when two of the plurality of input elements are simultaneously in contact with adjacent edges of the displayed graphical window, the first modification changing at least one of a shape and a size of the displayed graphical window in accordance with the identified positions and movement of the two input elements contacting the adjacent edges while the position of the displayed first portion of the image remains unchanged relative to the display device, or

a second modification implemented when two of the plurality of input elements are simultaneously in contact with a same edge of the displayed graphical window, the second modification changing at least one of a position and a shape of the displayed graphical window in accordance with the identified positions and movement of the two input elements contacting the same edge while the position of the displayed first portion of the image remains unchanged relative to the display device,

wherein the controller is configured to manipulate the displayed graphical window to reveal new content of the image not displayed in its entirety based on the selected modification to display on the display surface at least a second portion of the image not displayed in its entirety within the manipulated graphical window that is different from the first portion, but that includes the newly revealed content, wherein positions on the display surface of common portions of the first and the second portions of the image not displayed in its entirety are substantially the same.

17. The system of claim 16, wherein the first modification comprises uniformly scaling the size of the displayed graphical window in accordance with the identified movement of the two input elements contacting the adjacent edges.

18. The system of claim 16, wherein the second modification comprises changing the shape of the displayed graphical window by changing a length of said same edge in accordance with a change in distance between the positions of the two input elements contacting said same edge.

19. The system of claim 16,

wherein the display device being configured to display on the display surface the graphical window comprises the display device being configured to display on the display surface the graphical window over a background display,

wherein the first modification changing at least one of a shape and a size of the displayed graphical window comprises the first modification changing at least one of the shape and the size of the displayed graphical window relative to the background display, and

wherein the second modification changing at least one of a position and a shape of the displayed graphical window comprises the second modification changing at least one of the position and the shape of the displayed graphical window relative to the background display.

20. The system of claim 16, wherein the controller being configured to select either the first modification or the second modification comprises the controller being configured to select the first modification for manipulating the displayed graphical window, and

wherein the controller being configured to manipulate the displayed graphical window in accordance with the selected modification comprises the controller being configured to manipulate the displayed graphical window in accordance with the selected first modification.

21. The system of claim 16, wherein the controller being configured to select either the first modification or the second modification comprises the controller being configured to select the second modification for manipulating the displayed graphical window, and

wherein the controller being configured to manipulate the displayed graphical window in accordance with the selected modification comprises the controller being configured to manipulate the displayed graphical window in accordance with the selected second modification.

22. The system of claim 16, wherein the controller being configured to control the display device comprises the controller being configured to control the display device to display on the display surface at least a second portion of the image not displayed in its entirety within the manipulated graphical window that is different from the first portion, but that includes the newly revealed content, wherein positions on the display surface of common portions of the first and the second portions of the image not displayed in its entirety are the same.

23. A method of interfacing with a multi-input display device, comprising the steps of:

displaying on a multi-input display device a graphical window and at least a first portion of an image not displayed in its entirety within the graphical window, the graphical window including a frame;

identifying a position and movement of three or more input elements that have been determined to have simultaneously contacted respectively three or more different edges of the displayed frame of the graphical window;

manipulating the displayed graphical window in accordance with

a modification implemented when the three or more input elements have been determined to have simultaneously contacted respectively the three or more different edges of the displayed frame of the graphical window, the modification simultaneously moving each of the three or more edges of the displayed frame contacted by a respective one of the three or more input elements in a direction of identified movement of the respective input element normal to an axis of the respective edge while the position of the displayed first portion of the image remains unchanged relative to the display device; and

based on the manipulation of the graphical window, displaying on the display device at least a second portion of the image not displayed in its entirety within the manipulated graphical window that is different from the first portion, but that includes the newly revealed content, wherein positions on the display device of common portions of the first and second portions of the image not displayed in its entirety are substantially the same.

24. The method of claim 23, wherein the multi-input display device comprises a multi-touch display device.

25. The method of claim 23, wherein displaying on the display device at least a second portion of the image not displayed in its entirety within the manipulated graphical window comprises displaying on the display device at least a second portion of the image not displayed in its entirety within the manipulated graphical window that is different from the first portion, but that includes the newly revealed content, wherein positions on the display device of common portions of the first and second portions of the image not displayed in its entirety are the same.