Interactive manual, system and method for vehicles and other complex equipment
First Claim
Patent Images
1. An interactive manual system of a device, comprising:
- a model package database storing;
a set of at least one of phrases and sentences;
an object file that includes data objects that include information regarding an appearance of, and spatial characteristics of, a plurality of structures of the device and that include data representing visual displays that are associated with the structures of the device; and
a grammar table that identifies associations between (a) the information and data and (b) respective ones of the set of the at least one of phrases and sentences; and
a processor configured to execute;
a speech engine module for converting an utterance into a word sequence, determining the word sequence'"'"'s meaning, which is associated with one or more of the set of the at least one of phrases and sentences, and identifying a grammar category to which the word sequence as a whole conforms; and
a dialog manager module for extracting a subset of the information regarding the appearance of, and spatial characteristics of, the one or more structures of the device; and
an output arrangement configured to output what has been extracted;
wherein the extraction is based on the identified grammar category, the one or more of the set of the at least one of phrases and sentences to which the determined meaning of the word sequence corresponds, and the associations of the grammar table, such that the extraction includes collecting from the data objects a plurality of data objects that concern a particular one of the structures of the device associated by the grammar table with the one or more of the set of the at least one of phrases and sentences that are associated with the determined meaning of the word sequence, different ones of the data objects that concern the particular structure being included as part of the collected plurality of data objects depending on the category to which the processor has identified the word sequence as a whole conforms.
0 Assignments
0 Petitions
Accused Products
Abstract
A method and system of providing an interactive manual, including a speech engine to receive and process speech from a user, convert the speech into a word sequence, and identify meaning structures from the word sequence, a structured manual including information related to an operation of a device, a visual model to relate visual representation of the information, a dialog management arrangement to interpret the meaning structures in a context and to extract pertinent information and the visual representation from the structured manual and the visual model, and an output arrangement to output the information and visual representation.
21 Citations
24 Claims
-
1. An interactive manual system of a device, comprising:
-
a model package database storing; a set of at least one of phrases and sentences; an object file that includes data objects that include information regarding an appearance of, and spatial characteristics of, a plurality of structures of the device and that include data representing visual displays that are associated with the structures of the device; and a grammar table that identifies associations between (a) the information and data and (b) respective ones of the set of the at least one of phrases and sentences; and a processor configured to execute; a speech engine module for converting an utterance into a word sequence, determining the word sequence'"'"'s meaning, which is associated with one or more of the set of the at least one of phrases and sentences, and identifying a grammar category to which the word sequence as a whole conforms; and a dialog manager module for extracting a subset of the information regarding the appearance of, and spatial characteristics of, the one or more structures of the device; and an output arrangement configured to output what has been extracted; wherein the extraction is based on the identified grammar category, the one or more of the set of the at least one of phrases and sentences to which the determined meaning of the word sequence corresponds, and the associations of the grammar table, such that the extraction includes collecting from the data objects a plurality of data objects that concern a particular one of the structures of the device associated by the grammar table with the one or more of the set of the at least one of phrases and sentences that are associated with the determined meaning of the word sequence, different ones of the data objects that concern the particular structure being included as part of the collected plurality of data objects depending on the category to which the processor has identified the word sequence as a whole conforms. - View Dependent Claims (2, 3, 4, 5, 6, 7, 8, 9, 10, 11)
-
-
12. A computer-implemented method of accessing an interactive manual system of a device, the interactive manual system including a model package database storing (i) a set of at least one of phrases and sentences, (ii) an object file that includes data objects that include information regarding an appearance of, and spatial characteristics of, a plurality of structures of the device and that include data representing visual displays that are associated with the structures of the device, and (iii) a grammar table that identifies associations between (a) the information and data and (b) the respective ones of the set of the at least one of phrases and sentences, the method comprising:
-
receiving, by a computer processor, a speech input from a user; converting, by the computer processor executing a speech engine application, the speech input into a word sequence and determining the word sequence'"'"'s meaning, which is associated with one or more of the set of the at least one of phrases and sentences; identifying, by the computer processor, a grammar category to which the word sequence as a whole conforms; extracting, by the computer processor a subset of the information regarding the appearance of, and spatial characteristics of, the one or more structures of the device; and outputting, by the computer processor, via an output arrangement what has been extracted; wherein the extraction is based on the identified grammar category, the one or more of the set of the at least one of phrases and sentences to which the determined meaning of the word sequence corresponds, and the associations of the grammar table, such that the extraction includes collecting from the data objects a plurality of data objects that concern a particular one of the structures of the device associated by the grammar table with the one or more of the set of the at least one of phrases and sentences that are associated with the determined meaning of the word sequence, different ones of the data objects that concern the particular structure being included as part of the collected plurality of data objects depending on the category to which the processor has identified the word sequence as a whole conforms. - View Dependent Claims (13, 14, 15, 16, 17, 18, 19, 20, 21)
-
-
22. A non-transitory hardware computer-readable storage medium having stored thereon a set of instructions executable by a processor to perform a method of accessing an interactive manual system of a device, the interactive manual system including a model package database storing (i) a set of at least one of phrases and sentences, (ii) an object file that includes data objects that include information regarding an appearance of, and spatial characteristics of, a plurality of structures of the device and that include data representing visual displays that are associated with the structures of the device, and (iii) a grammar table that identifies associations between (a) the information and data and (b) the respective ones of the set of the at least one of phrases and sentences, the method comprising:
-
receiving a speech input from a user; converting the speech input into a word sequence and determining the word sequence'"'"'s meaning, which is associated with one or more of the set of the at least one of phrases and sentences; identifying a grammar category to which the word sequence as a whole conforms; extracting a subset of the information regarding the appearance of, and spatial characteristics of, the one or more structures of the device; and outputting via an output arrangement what has been extracted; wherein the extraction is based on the identified grammar category, the one or more of the set of the at least one of phrases and sentences to which the determined meaning of the word sequence corresponds, and the associations of the grammar table, such that the extraction includes collecting from the data objects a plurality of data objects that concern a particular one of the structures of the device associated by the grammar table with the one or more of the set of the at least one of phrases and sentences that are associated with the determined meaning of the word sequence, different ones of the data objects that concern the particular structure being included as part of the collected plurality of data objects depending on the category to which the processor has identified the word sequence as a whole conforms.
-
-
23. A device comprising:
-
a processor, coupled to a model package database that stores; a set of at least one of phrases and sentences; an object file that includes data objects that include information regarding an appearance of, and spatial characteristics of, a plurality of structures of the device and that include data representing visual displays that are associated with the structures of the device; and a grammar table that identifies associations between (a) the information and data and (b) respective ones of the set of the at least one of phrases and sentences; wherein; the processor is configured to; receive a speech input from a user; convert the speech input into a word sequence; determine the word sequence'"'"'s meaning, which is associated with one or more of the set of the at least one of phrases and sentences; identify a grammar category to which the word sequence as a whole conforms; extract a subset of the information regarding the appearance of, and spatial characteristics of, the one or more structures of the device; and output via an output arrangement what has been extracted; and the extraction is based on the identified grammar category, the one or more of the set of the at least one of phrases and sentences to which the determined meaning of the word sequence corresponds, and the associations of the grammar table, such that the extraction includes collecting from the data objects a plurality of data objects that concern a particular one of the structures of the device associated by the grammar table with the one or more of the set of the at least one of phrases and sentences that are associated with the determined meaning of the word sequence, different ones of the data objects that concern the particular structure being included as part of the collected plurality of data objects depending on the category to which the processor has identified the word sequence as a whole conforms.
-
-
24. An interactive manual system of a device, comprising:
-
a model package database including a set of at least one of phrases and sentences, and data objects that include information regarding components of the device, the information being associated with respective ones of the set of the at least one of phrases and sentences; and a processor configured to execute; a speech engine module for converting an utterance into a word sequence, based on determining the word sequence'"'"'s meaning, which is associated with one or more of the set of the at least one of phrases and sentences, and identifying a grammar category to which the word sequence as a whole conforms; and a dialog manager module for extracting a subset of the information; and an output arrangement configured to output the extracted information; wherein the extraction is based on the identified grammar category, the one or more of the set of the at least one of phrases and sentences to which the determined meaning of the word sequence corresponds, and the associations of the information with the respective ones of the set of the at least one of phrases and sentences, such that the extraction includes collecting from the data objects a plurality of data objects that concern a particular one of the components of the device associated by the grammar table with the one or more of the set of the at least one of phrases and sentences that are associated with the determined meaning of the word sequence, different ones of the data objects that concern the particular structure being included as part of the collected plurality of data objects depending on the category to which the processor has identified the word sequence as a whole conforms.
-
Specification