Method and system for providing multiple input connecting user interface

• US 20060112335A1
• Filed: 11/18/2004
• Published: 05/25/2006
• Est. Priority Date: 11/18/2004
• Status: Active Grant

• Current Assignee: Microsoft Technology Licensing LLC (Microsoft Corporation)
• Original Assignee: Microsoft Corporation
• Inventors: Hofmeister, Nicholas M., Valencia, Ann, Storrusten, Dane T.
• Granted Patent: US 7,925,996 B2
• Time in Patent Office: Days
• US Class Current: 715/701
• CPC Class Codes: G06F 3/0488 using a touch-screen or digitiser, e.g. input of commands through traced gestures

Abstract

A plurality of elements is identified by a user for input to the computing system by at least one of sequential and simultaneous selection of elements presented on a display of a computing system. Based on the elements thus identified, which may include at least one operator and at least one operand, the computing system recognizes a function to be performed by the computing system and causes a visual indication to be presented on the display. The computing system thus may present a visual indication showing a connection between, for example, the operator and an operand, or an operand and a functional area of the display associated with the function to be performed as a result of the elements identified. The visual indication thus confirms to the user the elements that the user has identified, and the resulting function that will be initiated.

Claims

1. A method for controlling functions of a plurality of elements presented on a display associated with a computing system, comprising the steps of:

(a) responding to identifications of the plurality of elements presented on the display;

(b) recognizing a function performable by the computing system as a result of a combination of the plurality of elements that were identified; and

(c) performing the function involving at one of;

(i) each of the plurality elements identified; and

(ii) at least one of the plurality of elements that were identified and a portion of the display associated with the function that was recognized.

2. The method of claim 1, further comprising presenting a visual indication on the display indicating a connection between one of:

(a) each of the plurality of elements identified; and

(b) at least one of the plurality of elements that were identified and a portion of the display associated with the function that was recognized.

3. The method of claim 2, wherein the visual indication presented varies according to the function that was recognized.

4. The method of claim 1, further comprising presenting a visual option indicator on the display upon detecting a first identification indicating an option of a next identification, the visual option indicator including one of:

(a) visually highlighting a choice of available operands when an operator is selected with the first identification; and

(b) visually highlighting a choice of available operators when at least one operand is selected with the first identification.

5. The method of claim 1, wherein the identifications of the plurality of elements presented on the display are received substantially simultaneously.

6. The method of claim 1, wherein at least one of the identifications is performed by detecting a physical object disposed adjacent to the display, approximately at a location where one of the plurality of elements is displayed, and wherein the physical object includes at least one of:

(a) an inanimate object; and

(b) a part of a user'"'"'s body.

7. The method of claim 6, wherein a plurality of identifications are made by a combination of physical objects including a plurality of:

(a) a part of a first user'"'"'s body;

(b) a part of a second user'"'"'s body;

(c) an inanimate object manipulated by a first user; and

(d) an inanimate object manipulated by a second user.

8. The method of claim 6, wherein the function recognized varies based on a time interval during which the physical object is disposed adjacent to the display at the location where said one of the plurality of elements is displayed.

9. The method of claim 1, wherein an order in which the plurality elements are identified changes at least one of:

(a) the function recognized; and

(b) the visual indication presented.

10. The method of claim 1, wherein the combination of the plurality of elements includes at least one operator and at least one operand, such that identification of the operator and the at least one operand causes a predefined function represented by the operator to be performed on the at least one operand.

11. The method of claim 10, further comprising the step of enabling a plurality of operands to be identified when a first operand is identified, and while an action causing the first operand to be identified continues, enabling an additional operand to be identified.

12. The method of claim 11, wherein the action causing the first operand to be identified includes the step of enabling pointing to the first operand;

and wherein the action continues while the pointing is directed by dragging, to identify the additional operand.

13. The method of claim 10, where in a media player application, the operator represents a player function, and the at least one operand includes a representation of a media recording, and wherein the player function includes at least one of:

(a) an add function that adds a media selection to a playlist;

(b) a delete function that removes a media selection from playlist; and

(c) a play function that plays one or more media selections in the playlist.

14. The method of claim 13, wherein the portion of the display associated with the function recognized includes one of:

(a) a displayed playlist; and

(b) the operator representing the player function.

15. The method of claim 10, where in a document processing application, the operator represents a document processing function, and the at least one operand represents a document, and wherein the document processing function includes at least one of:

(a) a function to attach the document to an e-mail;

(b) a function to delete the document;

(c) an open document function;

(d) a save document function;

(e) a copy document function; and

(f) a move document function.

16. The method of claim 10, where in a gaming application, the at least one operand represents a game element, and the operator represents a game function.

17. The method of claim 1, where in a help facility, the function recognized is providing instruction to a user regarding operation of the computing system, and the visual indication illustrates consequences of a particular user action.

18. The method of claim 1, wherein the visual indication presented on the display indicating the connection includes at least one of:

(a) a linear connection; and

(b) an animation.

19. A memory medium on which are stored machine instructions for carrying out the steps of claim 1.

20. A method for providing a visual feedback of a selection of an operator and at least one operand presented on a display that is coupled to a computing system, comprising the steps of:

(a) receiving identifications of the operator and the at least one operand;

(b) recognizing a function performable by the computing system by a pattern of the identifications of the operator and the at least one operand;

(c) presenting a visual indication on the display that is indicative of the function recognized, indicating a connection between one of;

(i) the operator and the at least one operand; and

(ii) the at least one operand and a portion of the display associated with the function recognized; and

(d) causing the function recognized to be initiated on the at least one operand.

21. The method of claim 20, further comprising presenting a visual option indicator on the display upon detecting a first identification indicating an option of a next identification, the visual option indicator including one of:

(a) visually highlighting a choice of available operands when an operator is selected with the first identification; and

(b) visually highlighting a choice of available operators when at least one operand is selected with the first identification.

22. The method of claim 20, wherein the step of receiving the identifications of the operator and the at least one operand occurs one of:

(a) substantially simultaneously;

(b) sequentially, without regard to order; and

(c) sequentially, in a predefined order.

23. The method of claim 20, wherein at least one of the identifications is performed by detecting a physical object disposed adjacent to the display, approximately at a location where one of the plurality of elements is displayed, and the physical object includes at least one of:

(a) a portion of a user'"'"'s body; and

(b) an inanimate object.

24. The method of claim 23, wherein a plurality of identifications are made by a combination of physical objects including a plurality of:

(a) a part of a first user'"'"'s body;

(b) a part of a second user'"'"'s body;

(c) an inanimate object manipulated by a first user; and

(d) an inanimate object manipulated by a second user.

25. The method of claim 23, wherein the function recognized varies based on a time interval during which the physical object is disposed adjacent to the display at the location where one of the plurality of elements is displayed.

26. The method of claim 20, further comprising the steps of enabling a plurality of operands to be identified when a first operand is identified;

and while the first operand is being identified, identifying an additional operand.

27. The method of claim 26, wherein the first operand is identified by a pointing directed to the first operand, and wherein the additional operand is identified by dragging the pointing to be directed to the additional operand.

28. The method of claim 20, where in a media player application, the operator represents a player function, and the at least one operand includes a representation of a media recording, and wherein the player function includes at least one of:

(a) a function to add a selection to a playlist;

(b) a delete function to delete a selection from the playlist; and

(c) a play function to play any selection in the playlist.

29. The method of claim 28, wherein the portion of the display associated with the function recognized includes one of:

(a) the playlist; and

(b) the operator representing the player function.

30. The method of claim 20, where in a data processing application, the operator represents a data processing function, and the at least one operand represents a data file, and wherein the data file processing function includes at least one of:

(a) a function to attach the data file to an e-mail;

(b) a delete data file function;

(c) an open data file function;

(d) a save data file function;

(e) an edit data file function;

(f) a copy data file function; and

(g) a move data file function.

31. The method of claim 20, where in a gaming application, the at least one operand represents a game element, and the operator represents a game function.

32. The method of claim 20, where in a help facility, the function recognized is providing instruction to a user regarding operation of the computing system;

and wherein the visual indication illustrates consequences of a particular user action.

33. The method of claim 20, wherein the visual indication presented on the display indicating the connection includes at least one of:

(a) a linear connection; and

(b) an animation.

34. A memory medium on which are stored machine instructions for carrying out the steps of claim 20.

35. A system for providing an input to an application that is being executed, comprising:

(a) an interactive display surface adjacent to which a plurality of physical objects are manipulated, said interactive display surface presenting visual entities;

(b) a sensor that detects a location of the physical objects that are adjacent to the interactive display surface;

(c) a processor in communication with the light sensor and the display device; and

(d) a memory in communication with the processor, the memory storing data and machine instructions that cause the processor to carry out a plurality of functions, including;

(i) receiving identifications of a plurality of the visual entities, the identifications being made by the processor as a function of the locations of the physical objects adjacent to the plurality of visual entities presented on the interactive display surface;

(ii) recognizing a function that is to be performed by the system based upon a combination of the visual entities that were identified; and

(iii) performing the function between one of;

(A) each of the visual entities that were identified; and

(B) at least one of the visual entities that was identified and a portion of the interactive display surface that is associated with the function recognized.

36. The system of claim 35, wherein the machine language instructions further cause the processor to present a visual indication on the interactive display surface indicating a connection between one of:

(a) each of the visual identifies that were identified; and

(b) at least one of the visual identifies that was identified and the portion of the interactive display surface that is associated with the function recognized.

37. The system of claim 36, wherein the machine language instructions further cause the processor to vary the presentation of the visual indication according to the function that was recognized.

38. The system of claim 35, wherein the machine language instructions further cause the processor to present a visual option indicator on the display upon detecting a first identification indicating an option of a next identification, the visual option indicator including one of:

(a) visually highlighting a choice of available operands when an operator is selected with the first identification; and

(b) visually highlighting a choice of available operators when at least one operand is selected with the first identification

39. The system of claim 35, wherein the identifications of the plurality of visual entities presented on the display are received substantially simultaneously.

40. The system of claim 35, wherein an identification of one of the visual entities comprising the combination is made by the sensor and is determined to be maintained while an identification of another of the visual entities comprising the combination is being made by the sensor.