Multimedia interface and method for computer system

• US 5,208,745 A
• Filed: 06/04/1990
• Issued: 05/04/1993
• Est. Priority Date: 07/25/1988
• Status: Expired due to Fees

• Current Assignee: Electric Power Research Institute Incorporated
• Original Assignee: Electric Power Research Institute
• Inventors: Butler, Arch W., Wunderlin, David J., Bloom, Charles P., Sweet, Stephen R., Quentin, George H., Campos, Francis M., Spoor, David, Bedros, Renee, Isle, Brian A.
• Primary Examiner(s): RUGGIERO, JOSEPH F
• Application Number:
  US07/532,596
• Time in Patent Office: 1,065 Days
• Field of Search: 364/184-187, 364/138, 364/139, 364/188, 364/189, 364/550, 364/551.01, 364/579, 364/580, 364/131-136, 371/29.1, 371/16.4, 371/18, 358/107, 395/153, 395/154, 395/157, 395/158, 395/161, 395/163, 395/50, 395/911, 395/912, 395/913, 395/914, 395/11, 395/12
• US Class Current: 700/83
• CPC Class Codes: G06F 1/1628 Carrying enclosures containing additional elements, e.g. case for a laptop and a printer
• CPC Class Codes: G06F 16/40 of multimedia data, e.g. slideshows comprising image and additional audio data retrieval of still image data G06F16/50; retrieval of audio data G06F16/60; retrieval of video data G06F16/70
• CPC Class Codes: G06F 16/48 Retrieval characterised by using metadata, e.g. metadata not derived from the content or metadata generated manually
• CPC Class Codes: G06F 3/0489 using dedicated keyboard keys or combinations thereof
• CPC Class Codes: G06F 3/16 Sound input; Sound output speech processing G10L
• CPC Class Codes: G06Q 10/10 Office automation; Time management
• CPC Class Codes: H04N 5/765 Interface circuits between an apparatus for recording and another apparatus
• CPC Class Codes: Y10S 706/911 Nonmedical diagnostics
• CPC Class Codes: Y10S 715/978 Audio interaction as part of an operator interface

Abstract

A multimedia interface presents information and receives user commands and data for a computer system. The multimedia interface operates in parallel with another application software module, such as an expert system. To add multimedia features to the application software module, the module is modified so as to generate multimedia commands at the same time as it displays text on a text monitor. The multimedia commands, which are held in a queue, provide additional information in the form of video images and generated speech corresponding to the displayed text. In addition, the multimedia commands are split into at least two sets: one set which is dispatched to the user substantially immediately after displaying the corresponding text, and one set which is dispatched upon request by the user. In the preferred embodiment, the multimedia interface presents information to the user through text, graphics, video, sound, speech production, and printed output. User inputs are made through a keyboard and voice recognition. In one preferred embodiment two data processing units are used: one for an expert system module and one for a video output processing module. Each module includes its own flat VGA display. The video module includes a digital video controller which enables it to display drawings, photographs, still and animated graphics, video stills, full-motion video and motion video with graphic overlays. An optional satellite station facilitates use of the system in environments where it is inconvenient or impossible to bring the complete system.

Claims

1. In a computer system, the combination comprising:

a first data processing unit coupled to a first display apparatus;

first memory means, coupled to said first data processing unit, for storing application software, wherein said application software displays messages on said first display apparatus and also transmits video display instructions;

a second data processing unit coupled to a second display apparatus, and coupled to said first data processing unit so as to receive said transmitted video display instructions;

random access video memory means, coupled to said second data processing unit, for storing a multiplicity of digitized video images; and

a digital video controller, coupled to said second data processing unit and to said video memory means, that displays on said second display apparatus selected ones and selected sequences of said digitized video images stored in said video memory means in accordance with said video display instructions transmitted by said application software;

said application software including means for coordinating said video images displayed on said second display apparatus with said messages displayed on said first display apparatus.

2. In a computer system, the combination comprising:

a first data processing unit coupled to a first display;

first memory means, coupled to said first data processing unit, for storing application software, wherein said application software displays messages on said first display apparatus and also transmits video display instructions;

wherein said application software includes means for defining a predefined set of pieces of equipment and a predefined set of maintenance tasks in conjunction with corresponding ones of said pieces of equipment;

a second data processing unit coupled to a second display, and coupled to said first data processing unit so as to receive said transmitted video display instructions;

random access video memory means, coupled to said second data processing unit, for storing a multiplicity of digitized video images, a multiplicity of said digitized video images stored in said video memory means corresponding to one or more of said pieces of equipment;

a digital video controller, coupled to said second data processing unit and to said video memory means, that displays on said second display apparatus selected ones and selected sequences of said digitized video images stored in said video memory means in accordance with said video display instructions transmitted by said application software; and

user input means for specifying one of said equipment maintenance tasks to be performed in conjunction with corresponding ones of said pieces of equipment;

said application software including software that responds to specification of an equipment maintenance task, entered via said user input means, by generating video display instructions that instruct said digital video controller to display a single stored digitized video image for certain predefined equipment maintenance tasks and a sequence of said stored digitized video images for other predefined equipment maintenance tasks;

said application software simultaneously displaying on said first display messages corresponding to said video images;

whereby said computer system can be used to provide assistance in the form of video sequences and message displays to a person performing a specified equipment maintenance task.

3. The combination set forth in claim 2, wherein said user input means includes voice input and recognition means for receiving spoken user commands.

4. The combination set forth in claim 2,

further including voice output means, coupled to said first data processing unit, for converting specified text strings into synthesized speech;

said application software generating and transmitting to said voice output means specified text strings.

5. The combination set forth in claim 2, including

portable housing that houses said computer system;

said portable housing enabling said computer system to be taken to and used at the place where a selected piece of equipment is located.

6. In a computer system, the combination comprising:

a first data processing unit coupled to a first display;

first memory means, coupled to said first data processing unit, for storing application software, wherein said application software displays messages on said first display and also transmits video display instructions;

a second data processing unit coupled to a second display, and coupled to said first data processing unit so as to receive said transmitted video display instructions;

random access video memory means, coupled to said second data processing unit, for storing a multiplicity of digitized video images;

a digital video controller, coupled to said second data processing unit and to said video memory means, that displays on said second display apparatus selected ones and selected sequences of said digitized video images stored in said video memory means in accordance with said video display instructions transmitted by said application software; and

an audio speaker coupled to said digital video controller;

said random access video memory means storing both digitized video images and a multiplicity of distinct digitized audio sequences;

said application software transmitting audio play instructions to said second data processing unit;

said second data processing unit forwarding corresponding audio play instructions to said digital video controller; and

said digital video controller sending selected digitized audio sequences stored in said video memory means in accordance with said audio play instructions transmitted by said application software;

whereby said computer system can play audio sequences in conjunction with video sequences and message displays.

7. The combination set forth in claim 6, wherein said application software includes means for defining a predefined set of pieces of equipment and a predefined set of maintenance tasks in conjunction with corresponding ones of said pieces of equipment;

a multiplicity of said digitized video images stored in said video memory means corresponding to one or more of said pieces of equipment,

said combination further including user input means for specifying one of said equipment maintenance tasks to be performed in conjunction with corresponding ones of said pieces of equipment;

said application software including software that responds to specification of an equipment maintenance task, entered via said user input means, by generating video display instructions that instruct said digital video controller to display a single stored digitized video image for certain predefined equipment maintenance tasks and a sequence of said stored digitized video images for other predefined equipment maintenance tasks;

said application software simultaneously displaying on said first display messages corresponding to said video images;

whereby said computer system can be used to provide assistance in the form of video sequences and messages to a person performing a specified equipment maintenance task.

8. The combination set forth in claim 7, wherein said user input means includes voice input and recognition means for receiving spoken user commands.

9. The combination set forth in claim 7,

further including voice output means, coupled to said first data processing unit, for converting specified text strings into synthesized speech;

said application software generating and transmitting to said voice output means specified text strings.

10. In a computer system, the combination comprising:

a first data processing unit coupled to a first display;

first memory means, coupled to said first data processing unit, for storing application software, wherein said application software displays messages on said first display and also transmits video display instructions;

a second data processing unit coupled to a second display, and coupled to said first data processing unit so as to receive said transmitted video display instructions;

random access video memory means, coupled to said second data processing unit, for storing a multiplicity of digitized video images;

a digital video controller, coupled to said second data processing unit and to said video memory means, that displays on said second display selected ones and selected sequences of said digitized video images stored in said video memory means in accordance with said video display instructions transmitted by said application software; and

a portable, handheld satellite station, having

a third data processing unit;

a third display;

communication means for communicating with said first data processing unit; and

user input means for interacting with said application software;

said first and second data processing units including means for transmitting messages and video images to said satellite station;

whereby said satellite station allows use of said computer system by a peripatetic/ambulatory user without having to move said entire computer system.

11. The combination set forth in claim 10, wherein said application software includes means for defining a predefined set of pieces of equipment and a predefined set of maintenance tasks in conjunction with corresponding ones of said pieces of equipment;

a multiplicity of said digitized video images stored in said video memory means corresponding to one or more of said pieces of equipment,

said combination further including user input means for specifying one of said equipment maintenance tasks to be performed in conjunction with corresponding ones of said pieces of equipment;

said application software including software that responds to specification of an equipment maintenance task, entered via said user input means, by generating video display instructions that instruct said digital video controller to display a single stored digitized video image for certain predefined equipment maintenance tasks and a sequence of said stored digitized video images for other predefined equipment maintenance tasks;

said application software simultaneously displaying on said first display messages corresponding to said video images;

whereby said computer system can be used to provide assistance in the form of video sequences and message to a person performing a specified equipment maintenance task.

12. The combination set forth in claim 11, wherein said user input means includes voice input and recognition means for receiving spoken user commands.

13. The combination set forth in claim 11,

further including voice output means, coupled to said first data processing unit, for converting specified text strings into synthesized speech;

said application software generating and transmitting to said voice output means specified text strings.

14. The combination set forth in claim 11, including

portable housing that houses said computer system;

said portable housing enabling said computer system to be taken to and used at the place where a selected piece of equipment is located.