Information processing apparatus using pointing input and speech input

• US 5,677,993 A
• Filed: 08/31/1993
• Issued: 10/14/1997
• Est. Priority Date: 08/31/1992
• Status: Expired due to Fees

• Current Assignee: Hitachi, Ltd.
• Original Assignee: Hitachi, Ltd.
• Inventors: Seki, Hiroshi, Kawaguchi, Koichi, Arita, Setsuo, Ohga, Yukiharu
• Primary Examiner(s): MacDonald, Allen R.
• Assistant Examiner(s): EDOUARD, PATRICK NESTOR
• Application Number:
  US08/113,816
• Time in Patent Office: 1,505 Days
• Field of Search: 395/2, 395/2.4, 395/2.67, 395/2.79, 395/2.83-2.85, 381/41, 381/48, 381/51, 364/492-495, 376/217, 376/245, 376/259
• US Class Current: 704/257
• CPC Class Codes: G05B 2219/23157 Display process, synoptic, legend, pictogram, mimic
• CPC Class Codes: G05B 2219/23386 Voice, vocal command or message
• CPC Class Codes: G06F 3/023 Arrangements for converting discrete items of information into a coded form, e.g. arrangements for interpreting keyboard generated codes as alphanumeric codes, operand codes or instruction codes
• CPC Class Codes: G06F 3/038 Control and interface arrangements therefor, e.g. drivers or device-embedded control circuitry
• CPC Class Codes: G06F 3/16 Sound input; Sound output speech processing G10L
• CPC Class Codes: G10L 15/26 Speech to text systems G10L15/08 takes precedence

Abstract

Each character string information obtained based on each information inputted by the CRT touch input device and the speech input device respectively is integrated by the input control unit and the memory. The input information analyzing unit analyses the integrated character string information. The output information generating unit generates presentation information corresponding to the demand information obtained by the analysis and outputs this presentation information to the display device. Since the input information from each input device is converted into character string information and it is integrated together and is then analyzed, input information is not restricted by each input device and each input device can be used in a free combination when information is inputted.

Claims

1. An information processing apparatus using a pointing input and a speech input, comprising:

a display device;

pointing input means for inputting information by indicating said information among information displayed on said display device;

speech input means;

first information converting means for converting first information inputted by said pointing input means into first character string information;

second information converting means for converting second information inputted by said speech input means into second character string information;

information integrating means for receiving said first and second character string information from said first and second information converting means, and integrating into a character string said first and second character string information respectively corresponding to said first and second information which has been inputted until input termination information is inputted from either said pointing input means or said speech input means;

information analyzing means for syntax-analyzing said first and second character string information which has been integrated, and generating demand information based on said first and second character string information which has been syntax-analyzed; and

means for preparing presentation information corresponding to said demand information, and outputting said presentation information on said display device.

2. An information processing apparatus according to claim 1, further comprising:

a memory for storing knowledge used for converting said first information into said first character string information,

wherein, by using said knowledge, said first information converting means converts said first information into said first character string information.

3. An information processing apparatus using a pointing input and a speech input, comprising:

a display device;

pointing input means for inputting information by indicating said information among information displayed on said display device;

speech input means;

first information converting means for converting first information inputted by said pointing input means into first character string information;

second information converting means for converting second information inputted by said speech input means into second character string information;

a memory for receiving said first and second character string information from said first and second information converting means, and storing said first and second character string information so as to integrate into a character string said first and second character string information, said first and second character string information respectively corresponding to said first and second information which has been inputted until said input termination information is inputted;

information analyzing means for syntax-analyzing said first and second character string information stored in said memory in relation to said input termination information, and generating demand information based on said first and second input character string information which has been syntax-analyzed; and

presentation preparing means for preparing presentation information corresponding to said demand information, and outputting said presentation information on said display device.

4. An information processing apparatus according to claim 3, further comprising:

means for fetching character string information including said first and second character string information stored in said memory, when said character string information including said first and second character string information has changed; and

means for outputting said changed character string information on said display device.

5. An information processing apparatus according to claim 3, wherein said presentation information generating means includes means for generating answer information in response to said first and second character string information which have been inputted and outputting said answer information to said display device based on said demand information.

6. An information processing apparatus using a pointing input and a speech input, comprising:

a display device;

speech output means;

pointing input means for inputting information by indicating said information among information displayed on said display device;

speech input means;

first information converting means for converting first information inputted by said pointing input means into first character string information;

second information converting means for converting second information inputted by said speech input means into second character string information;

information integrating means for receiving said first and second character string information from said first and second information converting means, and integrating into a character string said first and second character string information respectively corresponding to said first and second information which has been inputted until input termination information is inputted from either said pointing input means or said speech input means;

information analyzing means for syntax-analyzing said first and second character string information which has been integrated, and generating demand information based on said first and second character string information which has been syntax-analyzed;

means for preparing presentation information corresponding to said demand information; and

means for determining said display device or said speech output means in order to output said presentation information, on the basis of said demand information.

7. An information processing apparatus using a pointing input and a speech input, comprising:

a display device;

pointing input means for inputting information by indicating said information among information displayed on said display device;

speech input means;

first information converting means for converting first information inputted by said pointing input means into first character string information;

second information converting means for converting second information inputted by said speech input means into second character string information;

information integrating means for receiving said first and second character string information from said first and second information converting means, and integrating into a character string said first and second character string information respectively corresponding to said first and second information which have been inputted until input termination information is inputted from either said pointing input means or said speech input means;

information analyzing means for syntax-analyzing said first and second character string information which have been integrated and generating demand information based on said first and second character string information which has been syntax-analyzed;

means for preparing display information to be displayed on said display device when said demand information is an information display demand; and

means for preparing control data when said demand information is an operation demand.

8. A plant monitoring apparatus using a pointing input and a speech input, comprising:

a display device;

pointing input means for inputting information by indicating said information among information displayed on said display device;

speech input means;

means for storing measurement data measured at a plant;

first information converting means for converting first information inputted by said pointing input means into first character string information;

second information converting means for converting second information inputted by said speech input means into second character string information;

information integrating means for receiving said first and second character string information from said first and second information converting means, and integrating into a character string said first and second character string information respectively corresponding to said first and second information which have been inputted until input termination information is inputted from either said pointing input means or said speech input means;

information analyzing means for syntax-analyzing said first and second character string information which have been integrated, and generating demand information based on said first and second character string information which has been syntax-analyzed; and

means for preparing presentation information corresponding to said demand information by using said measurement data, and outputting said presentation information on said display device.

9. A plant monitoring apparatus according to claim 8, wherein said presentation information preparing means includes means for generating answer information to inputted character string information and outputting said answer information to said display device based on said demand information.

10. A plant monitoring apparatus according to claim 8, wherein

said display device has a system diagram display area, a trend display area, an input character string display area and an answer display area;

and

said preparation means includes;

means for outputting a system diagram including graphic elements of a plurality of components included in said plant to said display device to display said system diagram in said system diagram display area;

means for generating trend graph display information corresponding to said demand information by using said measured data and outputting said trend graph display information to said display device to display said trend graph display information in said trend display area;

means for outputting character string information stored in said information integrating means to said display device to display said character string information in said input character string display area when said character string information has changed; and

means for generating answer information in response to inputted string information and outputting said answer information to said display device to display said answer information in said answer display area.

11. An information processing apparatus according to claim 8, further comprising:

means for fetching character string information including said first and second character string information from said information integrating means, when said character string information including said first and second character string information has changed; and

means for outputting said changed character string information on said display device.

12. A plant control apparatus using a pointing input and a speech input, comprising:

a display device;

pointing input means for inputting information by indicating said information among information displayed on said display device;

speech input means;

means for storing measurement data measured at a plant;

first information converting means for converting first information inputted by said pointing input means into first character string information;

second information converting means for converting second information inputted by said speech input means into second character string information;

information integrating means for receiving said first and second character string information from said first and second information converting means, and integrating into a character string said first and second character string information respectively corresponding to said first and second information which have been inputted until input termination information is inputted from either said pointing input means or said speech input means;

information analyzing means for syntax-analyzing said first and second character string information which have been integrated, and generating demand information based on said first and second character string information which has been syntax-analyzed;

means for preparing display information to be displayed on said display device when said demand information is an information display demand; and

means for preparing a control demand for a corresponding component in said plant when said demand information is an operation demand.