Numbering scheme for selection by voice

• US 20060074687A1
• Filed: 09/24/2004
• Published: 04/06/2006
• Est. Priority Date: 09/24/2004
• Status: Active Grant

• Current Assignee: Microsoft Technology Licensing LLC (Microsoft Corporation)
• Original Assignee: Microsoft Corporation
• Inventors: Mowatt, David, Murillo, Oscar E., Bickel, Ryan Terry
• Granted Patent: US 7,742,923 B2
• Time in Patent Office: Days
• US Class Current: 704/275
• CPC Class Codes: G06F 3/0481 based on specific properties of the displayed interaction object or a metaphor-based environment, e.g. interaction with desktop elements like windows or icons, or assisted by a cursor's changing behaviour or appearance

Abstract

A numbering scheme is disclosed for implementation in the context of an application display. A user is able to select an item on the display by speaking a number corresponding to a desired control item. In some cases, the screen can include so many numbers that the user loses context and is unable to identify which number they want to select. For this reason, in one embodiment, a temporal switching mechanism is implemented wherein periodic switches (e.g., second-long intervals) occur between showing numbered items and showing a non-numbered screen. In one embodiment, an optional secondary confirmation step is implemented wherein the user sees only the item they just selected and has the chance to (a) learn the programmatic name of the item they selected and/or (b) either confirm and proceed with their selection, or cancel. In one embodiment, the optional secondary confirmation step is omitted if the user speaks a number followed by a predetermined command word.

Claims

1. A computer-implemented method of facilitating interaction with a user, the method comprising:

receiving a request from the user for interaction assistance; and

responding to the request by layering a control indication scheme over an application display.

2. The method of claim 1, wherein receiving a request comprises receiving a spoken request.

3. The method of claim 1, wherein receiving a request comprises receiving a request communicated through interaction with a physical input device.

4. The method of claim 1, wherein receiving a request comprises recognizing an event sequence.

5. The method of claim 1, wherein layering comprises applying an identifier to each exposed control item.

6. The method of claim 1, wherein layering comprises drawing a box around each of a plurality of exposed control items.

7. The method of claim 1, wherein layering comprises determining and rendering a bounding box for each of a plurality of exposed control items, and further comprises applying an identifier to the boundary boxes.

8. The method of claim 1, wherein layering comprises numbering exposed controls from a top left portion of the application display to a bottom right portion.

9. The method of claim 1, wherein layering comprises numbering exposed controls with a consistent font.

10. The method of claim 1, wherein layering comprises numbering exposed controls consecutively from left to right.

11. The method of claim 1, wherein layering comprises placing a visible indicator over each exposed control.

12. The method of claim 1, wherein layering comprises placing multiple instances of a same number proximate a single exposed control having a relatively large size.

13. The method of claim 1, wherein layering comprises placing attention-drawing lines to attract visual attention to a number that is placed proximate an exposed control.

14. The method of claim 1, further comprising:

receiving a spoken indication from the user; and

identifying an exposed control that corresponds to the spoken indication.

15. The method of claim 14, further comprising:

receiving a spoken predetermined command word immediately after the spoken indication; and

performing a function that is consistent with the exposed control that corresponds to the spoken indication.

16. The method of claim 14, further comprising altering the appearance of the control indication scheme to emphasize the exposed control that corresponds to the spoken indication.

17. The method of claim 14, further comprising switching between a display that does and does not include the control indication scheme to emphasize correlation to a set of underlying graphical representations.

18. The method of claim 17, further comprising altering the appearance of the control indication scheme to emphasize the exposed control that corresponds to the spoken indication.

19. The method of claim 14, further comprising displaying a name associated with the exposed control that corresponds to the spoken indication.

20. The method of claim 14, further comprising:

receiving a spoken cancel command; and

restoring the control indication scheme.

21. The method of claim 14, further comprising receiving and responding to a spoken cancel command.

22. The method of claim 14, further comprising:

receiving a confirmation command; and

performing a function that is consistent with the exposed control that corresponds to the spoken indication.

23. The method of claim 1, further comprising:

waiting a predetermined amount of time;

periodically switching between a display that includes the control indication scheme and one that does not.

24. A computer-implemented method of facilitating interaction with a user, the method comprising:

receiving a spoken indicator from the user, wherein the spoken indicator corresponds to a visual representation of a control item;

receiving a spoken predetermined command word after the spoken number; and

performing a function that is consistent with the control item that corresponds to the visual representation.

25. A computer-implemented method of facilitating interaction with a user, the method comprising:

receiving a spoken indicator from the user, wherein the spoken indicator corresponds to a representation of a control item that is rendered on a display; and

altering a control indication scheme to emphasize the representation of the control item that is rendered on the display.

26. The method of claim 25, further comprising displaying a name associated with the control item that is rendered on the display.

27. The method of claim 25, further comprising:

receiving a spoken cancel command from the user; and

altering a control indication scheme to emphasize a plurality of representations of control items rendered on the display.

28. The method of claim 25, further comprising:

receiving a confirmation command; and

performing a function that is consistent with the representation of the control item.

29. A computer-implemented method of facilitating interaction with a user, the method comprising automatically switching between display screens that do and do not include a control indication scheme layered thereon.

30. The method of claim 29, wherein automatically switching comprises automatically switching following a period of inaction having a predetermined length.

31. The method of claim 29, wherein automatically switching comprises switching based on expirations of a predetermined periodic time interval.

32. The method of claim 29, wherein automatically switching comprises switching in response to a control indication spoken by the user.

33. The method of claim 29, wherein automatically switching comprises switching in response to a command spoken by the user.