System and method for ordering data in a computer system in accordance with an input data sequence

• US 5,974,385 A
• Filed: 05/23/1997
• Issued: 10/26/1999
• Est. Priority Date: 12/19/1994
• Status: Expired due to Term

• Current Assignee: Aurix Ltd. (Avaya Incorporated)
• Original Assignee: The secretary of state for defence in her Britannic Majesty's Government of the United Kingdom of Great Britain And Northern Ireland
• Inventors: Ponting, Keith Michael, Series, Robert William
• Primary Examiner(s): Hudspeth, David R.
• Assistant Examiner(s): SOFOCLEOUS, MICHAEL D
• Application Number:
  US08/849,034
• Time in Patent Office: 886 Days
• Field of Search: 704/251-257, 704/200, 704/270, 704/275, 395/821, 395/878, 395/893
• US Class Current: 704/275
• CPC Class Codes: G10L 15/26 Speech to text systems G10L15/08 takes precedence

Abstract

An information processing system (10) receives input data from a keyboard (26), a mouse (28) and spoken data input device (30) in an input data sequence. A microphone (32) converts received speech into electrical signals which are digitized by an analogue to digital converter (100). A digital signal processor (102) converts the digital signals into multi-dimensional vectors which are stored in a temporary input buffer (104). A recognition processor (106) performs a recognition program in order to match the multi-dimensional vectors to speech models. In order that the system (10) may output data in an output data sequence which corresponds to the input data sequence, each data input receives a time stamp. A timing controller (24) ensures that instructions received from either the keyboard or the mouse are output only when those instructions have a time stamp which is earlier than the time stamp of the data most recently processed by the recognition processor.

Claims

1. An information processing system comprising:

(a) a plurality of data input devices for receiving data in a data input sequence and for generating data elements in response thereto, at least one of the data input devices being an acoustic data input device; and

(b) a processor for receiving the data elements from the data input devices and for outputting processed data in a data output sequence;

wherein

(i) the processor is arranged to provide each data element with at least one respective time sequence indicator indicating the data element'"'"'s position in the data input sequence; and

(ii) the processor arranged to determine and reorder the output sequence in accordance with each data element'"'"'s at least one sequence indicator such that the data output sequence is in correspondence with the data input sequence irrespective of differing delays for data elements generated by differing data input devices.

2. A system according to claim 1, wherein the processor is arranged to receive acoustic data elements from the acoustic data input device and to generate tokens representing the acoustic data elements.

3. A system according to claim 2, wherein the acoustic data elements are feature vectors generated from spoken input by the acoustic data input device and the processor is arranged to compare the feature vectors with stored models to generate tokens representing the acoustic data elements.

4. A system according to claim 3, wherein the processor is arranged to associate each data element with two input sequence indicators comprising a start indicator and an end indicator indicating respectively a time for the start of the acoustic data input and a time for the end of the acoustic data input.

5. A system according to claim 4, wherein the processor incorporates a storage device for storing the data elements and their respective sequence indicators and is arranged to generate a most recent time (MRT) sequence indicator corresponding to the end indicator of the most recent feature vector processed and to receive the MRT sequence indicator and to compare the MRT sequence indicator with the sequence indicators of data elements stored in the storage device to determine the output sequence.

6. A system according to claim 5, wherein the processor incorporates a plurality of programmed processing devices for determining said output sequence and receiving acoustic data elements.

7. A system according to claim 3, wherein the processor is arranged to determine said output sequence and receive acoustic data elements in a single programmed processor.

8. A system according to claim 7, wherein the system is arranged to perform the steps of:

a) providing the processor with non-acoustic data elements from at least one input device,

b) labelling the non-acoustic data elements with a sequence indicator and storing the data together with its sequence indicator in an event list,

c) providing the processor with feature vectors corresponding to spoken acoustic data elements,

d) generating tokens representing the feature vectors,

e) determining a most recent time (MRT) sequence indicator corresponding to the end indicator of the token corresponding to the most recent feature vector processed by the processor,

f) comparing the MRT sequence indicator with the sequence indicators of the non-spoken data stored in the event list,

g) determining whether or not the sequence indicators of stored data predate the MRT sequence indicator and if so passing them to a central processing device, and

h) passing a token to the central processing device when no stored sequence indicator predates the MRT sequence indicator.

9. A method of processing data including the steps of:

a) receiving data elements in a data input sequence from a plurality of input sources, at least one of which provides data elements derived from spoken origin;

b) processing the received data elements by associating a sequence indicator with each respective received data element to indicate its position in the data input sequence and generating tokens to represent data elements of spoken origin by matching each spoken origin data element to a model; and

c) establishing a data output sequence for processed data elements and where necessary reordering them on the basis of their associated sequence indicator such that the data output sequence corresponds to the data input sequence irrespective of differences in processing delays for data elements received from different input sources.

10. An information processing system including

(a) a plurality of data input devices arranged to provide data elements in a data input sequence, and

(b) a processor arranged to receive and process the data elements and to output processed data elements in an output sequence in correspondence with the data input sequence,

said processor arranged to process data elements in processing time intervals which differ dependent upon the data input device;

said processor arranged to associate a time indicator with each data element and use the time indicators of differing processed input data elements associated with respective processed data elements to determine the output sequence.

11. An information processing system comprising:

(a) a plurality of data input means for receiving input data and for generating input data signals in response thereto,

(b) a processor;

(i) for attaching a marker signal to each input data signal showing the time of receipt by the data input means and the relative position in an input data sequence;

(ii) for generating output signals corresponding to a respective input data signal; and

(iii) arranging the output signals in sequence on the basis of the marker signals of the associated input data signals so that the output signals have a sequence which is equivalent to said input data sequence.

12. An information processing system comprising:

(a) a plurality of data input devices for reception of input data and for generation of input data signals in response thereto; and

(b) computer processing apparatus for

(i) generating respective processed signals from the input data signals, each processed signal representing a respective input data signal together with a respective marker signal indicating reception time of the input data to which that input data signal corresponds, said apparatus processing input data signals from one of the input devices at a slower rate than input data signals from other input devices; and

(ii) ordering the processed signals in a sequence which is the same as that in which the data input signals were generated.

13. An information processing system comprising:

(a) a plurality of data input means for receiving data in a data input sequence and for generating data elements in response thereto, at least one of the data input means providing means for receiving acoustic data; and

(b) processing means for receiving the data elements from the data input means and for outputting processed data in a data output sequence;

wherein

(i) the system includes means for associating each data element with at least one respective time sequence indicator associated with receipt of said data for each corresponding data element; and

(ii) the processing means includes means for determining the output sequence from each data element'"'"'s at least one sequence indicator such that the data output sequence is in correspondence with the data input sequence irrespective of differing processing means delays for data elements generated by differing input means,

wherein the processing means includes recognizing means for receiving acoustic data elements from the acoustic data input means and for generating tokens representing the acoustic data in response thereto,

wherein the acoustic data elements are feature vectors generated from spoken input by the acoustic data input means and the recognizing means is arranged to compare the feature vectors with stored models to generate tokens representing the acoustic data elements,

wherein the means for associating each data element with at least one respective sequence indicator is arranged to associate each acoustic data element with two sequence indicators comprising a start indicator and an end indicator indicating respectively a time for the start of the acoustic data input and a time for the end of the acoustic data input.

14. A system according to claim 13, wherein the means for determining said data output sequence incorporates storage means for storing the data elements and their respective sequence indicators and wherein the recognizing means is arranged to generate a most recent time (MRT) sequence indicator corresponding to the end indicator of the most recent feature vector processed by the recognizing means and the means for determining said output sequence is arranged to receive the MRT sequence indicator and to compare the MRT sequence indicator with the sequence indicators of data elements stored in the storage means to determine the output sequence.

15. A system according to claim 14, wherein the means for determining said output sequence and the recognizing means are programmed processors.

16. An information processing system comprising:

(a) a plurality of data input means for receiving data in a data input sequence and for generating data elements in response thereto, at least one of the data input means providing means for receiving acoustic data; and

(b) processing means for receiving the data elements from the data input means and for outputting processed data in a data output sequence;

wherein

(i) the system includes means for associating each data element with at least one respective time sequence indicator associated with receipt of said data for each corresponding data element; and

(ii) the processing means includes means for determining the output sequence from each data element'"'"'s at least one sequence indicator such that the data output sequence is in correspondence with the data input sequence irrespective of differing processing means delays for data elements generated by differing input means, wherein the processing means includes recognizing means for receiving acoustic data elements from the acoustic data input means and for generating tokens representing the acoustic data in response thereto,

wherein the acoustic data elements are feature vectors generated from spoken input by the acoustic data input means and the recognizing means is arranged to compare the feature vectors with stored models to generate tokens representing the acoustic data elements, wherein the means for determining said output sequence and the recognizing means are embodied in a single programmed processor, wherein the system is arranged to perform the steps of;

a) providing the processing means with non-acoustic data elements from at least one input device,

b) labelling the non-acoustic data elements with a sequence indicator and storing the data together with its sequence indicator in an event list,

c) providing the processing means with feature vectors corresponding to spoken acoustic data elements,

d) generating tokens representing the feature vectors,

e) determining a most recent time (MRT) sequence indicator corresponding to the end indicator of the token corresponding to the most recent feature vector processed by the recognizing means,

f) comparing the MRT sequence indicator with the sequence indicators of the non-spoken data stored in the event list,

g) determining whether or not the sequence indicators of stored data predate the MRT sequence indicator and if so passing them to a central processor, and

h) passing a token to the central processor when no stored sequence indicator predates the MRT sequence indicator.