Methods and apparatus for voice-activated control of an interactive display

• US 10,198,246 B2
• Filed: 08/19/2016
• Issued: 02/05/2019
• Est. Priority Date: 08/19/2016
• Status: Active Grant

• Current Assignee: HONEYWELL/LK GLOBAL
• Original Assignee: Honeywell International Inc.
• Inventors: Paulraj, Vasantha Selvi, Krishna, Kiran Gopala, Agarwal, Jitender Kumar, Pasupathi, Sivaprakash
• Primary Examiner(s): Nguyen, Cao H
• Application Number:
  US15/242,104
• Publication Number:
  US 20180052657A1
• Time in Patent Office: 900 Days
• Field of Search: 715728, 715771, 715761-765
• CPC Class Codes: G06F 3/0482 Interaction with lists of selectable items, e.g. menus
• CPC Class Codes: G06F 3/04842 Selection of displayed objects or displayed text elements G06F3/0482 takes precedence
• CPC Class Codes: G06F 3/14 Digital output to display device ; Cooperation and interconnection of the display device with other functional units
• CPC Class Codes: G06F 3/167 Audio in a user interface, e.g. using voice commands for navigating, audio feedback
• CPC Class Codes: G06F 40/174 Form filling; Merging
• CPC Class Codes: G10L 15/22 Procedures used during a speech recognition process, e.g. man-machine dialogue
• CPC Class Codes: G10L 2015/221 Announcement of recognition results

Abstract

A method for controlling an interactive display is provided. The method receives a set of voice input data, via a voice input device communicatively coupled to the interactive display; interprets, by at least one processor, the set of voice input data to produce an interpreted result, wherein the at least one processor is communicatively coupled to the voice input device and the interactive display; presents, by the interactive display, a text representation of the interpreted result coupled to a user-controlled cursor; receives, by a user interface, a user input selection of a textual or graphical element presented by the interactive display, wherein the user interface is communicatively coupled to the at least one processor and the interactive display; and performs, by the at least one processor, an operation associated with the interpreted result and the user input selection.

Claims

1. A method for controlling an interactive display, the method comprising:

receiving a set of voice input data, via a voice input device communicatively coupled to the interactive display;

interpreting, by at least one processor, the set of voice input data to produce an interpreted result, wherein the at least one processor is communicatively coupled to the voice input device and the interactive display;

presenting, by the interactive display, a combination element comprising a text representation of the interpreted result coupled to a user-controlled cursor, wherein presentation of the combination element provides visual feedback indicating content of the interpreted result, and wherein the combination element is configured for user manipulation including positioning the combination element and making selections using the combination element;

receiving, by a user interface, a user input selection of a textual or graphical element presented by the interactive display, wherein the user input selection is performed via the user manipulation of the combination element over the textual or graphical element and performance of a click to select the textual or graphical element, and wherein the user interface is communicatively coupled to the at least one processor and the interactive display;

performing, by the at least one processor, an operation associated with the interpreted result and the user input selection, wherein the interpreted result comprises a command, and wherein the command comprises the operation applicable to manipulate the textual or graphical element presented by the interactive display; and

after performing the operation associated with the interpreted result, presenting the user-controlled cursor without the text representation of the interpreted result, to indicate performance of the operation associated with the interpreted result.

2. The method of claim 1, wherein receiving the user input selection further comprises receiving a user input selection of a data field;

and

wherein performing the operation associated with the interpreted result and the user input selection further comprises;

positioning the text representation coupled to the user-controlled cursor in the data field; and

pasting the text representation in the data field, based on the positioning.

3. The method of claim 1, wherein receiving the user input selection further comprises receiving a user input selection of a data field;

and

wherein performing the operation associated with the interpreted result and the user input selection further comprises;

identifying a data type associated with the text representation;

positioning the text representation coupled to the user-controlled cursor in the data field; and

when the data type associated with the text representation corresponds to a type associated with the data field, pasting the text representation in the data field, based on the positioning.

4. The method of claim 1, wherein receiving the user input selection further comprises receiving a user input selection of a graphical list comprising a plurality of list items;

and

wherein performing the operation associated with the interpreted result and the user input selection further comprises;

performing an automatic selection of one of the list items, based on the interpreted result; and

presenting the one of the list items.

5. The method of claim 1, wherein receiving the user input selection further comprises receiving a user input selection of one of a plurality of displayed features;

wherein the method further comprises identifying a command, based on the interpreted result; and

wherein performing the operation associated with the interpreted result and the user input selection further comprises;

executing the command, by the at least one processor;

altering the one of the plurality of displayed features, based on executing the command, to create an altered plurality of displayed features; and

presenting the altered plurality of displayed features, by the interactive display.

6. The method of claim 1, wherein the voice input device comprises a push-to-talk (PTT) voice input device communicatively coupled to the interactive display, the PTT voice input device comprising a work-center selection switch;

and

wherein the method further comprises;

receiving, via the work-center selection switch, a user input selection of a second work-center; and

routing the interpreted result to a second interactive display associated with the second work-center, wherein the interactive display is associated with a first work-center;

wherein the text representation is presented by the second interactive display; and

wherein the user input selection of the textual or graphical element is presented by the second interactive display.

7. The method of claim 1, further comprising:

wherein the voice input device comprises a voice-activated microphone communicatively coupled to the interactive display; and

wherein the method further comprises;

receiving, via the voice-activated microphone, a user input selection of a second work-center; and

routing the interpreted result to a second interactive display associated with the second work-center, wherein the interactive display is associated with a first work-center;

wherein the text representation is presented by the second interactive display; and

wherein the user input selection of the textual or graphical element is presented by the second interactive display.

8. The method of claim 1, further comprising:

receiving, by a second voice input device, a user input voice command selecting the interactive display, and a second set of voice input data, wherein the second voice input device is communicatively coupled to a second interactive display and the at least one processor, and wherein the at least one processor is communicatively coupled to the interactive display and the second interactive display;

interpreting, by the at least one processor, the second set of voice input data to produce a second interpreted result;

presenting, by the interactive display, a second text representation of the second interpreted result coupled to the user-controlled cursor;

receiving, by the user interface, a second user input selection of a second graphical element presented by the interactive display; and

performing, by the at least one processor, a second operation associated with the second interpreted result and the second user input selection.

9. A system for user interaction with an interactive display, the system comprising:

a system memory element;

a voice input device, configured to receive a set of voice input data;

an interactive display, configured to present a plurality of graphical elements and a user-controlled cursor;

a user interface, configured to receive a user input manipulation of the user-controlled cursor;

at least one processor, communicatively coupled to the system memory element, the voice input device, the interactive display, and the user interface, the at least one processor configured to;

interpret the set of voice input data to produce an interpreted result;

present, via the interactive display, a combination element comprising a text representation of the interpreted result coupled to the user-controlled cursor, wherein presentation of the combination element provides visual feedback indicating content of the interpreted result, and wherein the combination element is configured for user manipulation including positioning the combination element and making selections using the combination element;

receive, via the user interface, a user input selection of one of the plurality of graphical elements, wherein the user input selection is performed via the user manipulation of the combination element over the textual or graphical element and performance of a click to select the textual or graphical element;

perform an operation associated with the interpreted result and the user input selection, wherein the interpreted result comprises a command, and wherein the command comprises the operation applicable to manipulate the textual or graphical element presented by the interactive display; and

after performing the operation associated with the interpreted result, present the user-controlled cursor without the text representation of the interpreted result, to indicate performance of the operation associated with the interpreted result.

10. The system of claim 9, wherein the at least one processor is further configured to:

receive a user input selection of a data field, wherein the one of the plurality of graphical elements comprises the data field;

wherein performing the operation associated with the interpreted result and the user input selection further comprises;

positioning the text representation coupled to the user-controlled cursor in the data field; and

pasting the text representation in the data field, based on the positioning.

11. The system of claim 9, wherein the at least one processor is further configured to:

receive a user input selection of a data field, wherein the one of the plurality of graphical elements comprises a data field; and

wherein performing the operation associated with the interpreted result and the user input selection further comprises;

identifying a data type associated with the text representation;

positioning the text representation coupled to the user-controlled cursor in the data field; and

when the data type associated with the text representation corresponds to a type associated with the data field, pasting the text representation in the data field, based on the positioning.

12. The system of claim 9, wherein the at least one processor is further configured to:

receive a user input selection of a graphical list comprising a plurality of list items, wherein the one of the plurality of graphical elements comprises the graphical list;

wherein performing the operation associated with the interpreted result and the user input selection further comprises;

performing an automatic selection of one of the list items, based on the interpreted result; and

presenting the one of the list items.

13. The system of claim 9, wherein the at least one processor is further configured to:

receive a user input selection of one of a plurality of displayed features, wherein the one of the plurality of graphical elements comprises the one of the plurality of displayed features; and

identify a command, based on the interpreted result; and

wherein performing the operation associated with the interpreted result and the user input selection further comprises;

executing the command;

altering the one of the plurality of displayed features, based on executing the command, to create an altered plurality of displayed features;

presenting the altered plurality of displayed features, via the interactive display.

14. The system of claim 9, wherein the voice input device comprises a push-to-talk (PTT) voice input device communicatively coupled to the interactive display;

and

wherein the at least one processor is further configured to receive a first PTT indication, the set of voice input data, and a second PTT indication, and wherein the first PTT indication and the second PTT indication bound the set of voice input data.

15. The system of claim 9, wherein the voice input device comprises a microphone communicatively coupled to the interactive display;

and

wherein the at least one processor is further configured to;

identify a starting point in time and an endpoint in time; and

obtain voice data between the starting point and the endpoint, wherein the set of voice input data comprises the voice data.

16. A non-transitory, computer-readable medium containing instructions thereon, which, when executed by a processor, perform a method comprising:

performing, by the processor, an operation of an interactive display using a set of voice input data and a user-selected graphical element, by;

creating a combination element comprising a text representation of an interpreted result of the set of voice data coupled to a user-controlled cursor;

presenting the combination element to provide visual feedback indicating content of the interpreted result, and wherein the combination element is configured for user manipulation including positioning the combination element and making selections using the combination element;

receiving, by a user interface, a user input selection of the user-selected graphical element presented by the interactive display, wherein the user input selection is performed via the user manipulation of the combination element over the user-selected graphical element and performance of a click to select the user-selected graphical element, and wherein the user interface is communicatively coupled to the at least one processor and the interactive display;

performing, by the processor, the operation associated with the interpreted result and the user input selection, wherein the interpreted result comprises a command, and wherein the command comprises the operation applicable to manipulate the textual or graphical element presented by the interactive display; and

after performing the operation associated with the interpreted result, presenting the user-controlled cursor without the text representation of the interpreted result, to indicate performance of the operation associated with the interpreted result.

17. The non-transitory, computer-readable medium of claim 16, wherein the method further comprises:

receiving the set of voice input data, via a voice input device communicatively coupled to the interactive display and the processor; and

interpreting, by the processor, the set of voice input data to produce an interpreted result;

wherein the operation is performed based on the interpreted result.

18. The non-transitory, computer-readable medium of claim 17, wherein the voice input device comprises at least one of a push-to-talk (PTT) device and a microphone.

19. The non-transitory, computer-readable medium of claim 17, wherein the method further comprises:

presenting, by the interactive display, a text representation of the interpreted result coupled to a user-controlled cursor;

receiving, via a user interface, a user input selection of a graphical element presented by the interactive display, wherein the user interface is communicatively coupled to the processor and the interactive display;

wherein the operation is associated with the interpreted result and the user input selection.

20. The non-transitory, computer-readable medium of claim 19, wherein receiving the user input selection further comprises receiving a user input selection of a data field;

and

wherein performing the operation associated with the interpreted result and the user input selection further comprises;

identifying a data type associated with the text representation;

positioning the text representation coupled to the user-controlled cursor in the data field; and

when the data type associated with the text representation corresponds to a type associated with the data field, pasting the text representation in the data field, based on the positioning.