Automatic data routing via voice command annotation

• US 6,295,391 B1
• Filed: 12/23/1998
• Issued: 09/25/2001
• Est. Priority Date: 02/19/1998
• Status: Expired due to Term

• Current Assignee: Hewlett-Packard Development Company, L.P. (HP Inc.)
• Original Assignee: Hewlett-Packard Company (HP Inc.)
• Inventors: Rudd, Michael L, Oliver, Thomas C
• Primary Examiner(s): Couso, Jose L.
• Assistant Examiner(s): Do, Anh Hong
• Application Number:
  US09/219,972
• Time in Patent Office: 1,007 Days
• Field of Search: 382/313, 382/312, 382/321, 707/503, 709/203, 709/217, 725/86, 704/426, 704/275, 381/110, 381/122
• US Class Current: 382/313
• CPC Class Codes: G06F 3/16 Sound input; Sound output speech processing G10L
• CPC Class Codes: G10L 19/00 Speech or audio signals analysis-synthesis techniques for redundancy reduction, e.g. in vocoders; Coding or decoding of speech or audio signals, using source filter models or psychoacoustic analysis in musical instruments G10H
• CPC Class Codes: H04M 3/42204 Arrangements at the exchange for service or number selection by voice at the terminal H04M1/27
• CPC Class Codes: H04N 1/00127 Connection or combination of a still picture apparatus with another apparatus, e.g. for storage, processing or transmission of still picture signals or of information associated with a still picture
• CPC Class Codes: H04N 1/00204 with a digital computer or a digital computer system, e.g. an internet server programmed control between transmitter and receiver or between image input and image output device H04N1/32561
• CPC Class Codes: H04N 1/00326 with a data reading, recognizing or recording apparatus, e.g. with a bar-code apparatus arrangements for the associated working of recording or reproducing apparatus with related apparatus G11B31/00
• CPC Class Codes: H04N 1/00352 Input means H04N1/00411 takes precedence; input arrangements for computers G06F3/00
• CPC Class Codes: H04N 1/32122 in a separate device, e.g. in a memory or on a display separate from image data
• CPC Class Codes: H04N 1/32128 attached to the image data, e.g. file header, transmitted message header, information on the same page or in the same computer file as the image for information embedded in the image H04N1/32144
• CPC Class Codes: H04N 2201/0081 Image reader H04N2201/0091 - H04N2201/0094 take precedence
• CPC Class Codes: H04N 2201/0084 Digital still camera
• CPC Class Codes: H04N 2201/3222 of processing required or performed, e.g. forwarding, urgent or confidential handling
• CPC Class Codes: H04N 2201/3261 of multimedia information, e.g. a sound signal
• CPC Class Codes: H04N 2201/3264 of sound signals
• CPC Class Codes: H04N 2201/3274 Storage or retrieval of prestored additional information
• CPC Class Codes: H04N 2201/3278 Transmission

Abstract

A system for automatically routing data captured by devices, such as digital cameras, personal digital assistants, laptop computers, or portable document scanners, through voice command annotation is disclosed. After capturing an image or document with the device, a voice disposition command is made by a user speaking into a voice pickup component in the device to control the disposition of the captured image or document. Commands such as “fax”, “e-mail”, “print and save” are typical. The voice input is stored in the device in a voice command annotation file and is associated with the image or document data file. Upon copying the image or document data file to a host computer system, the voice command annotation file is automatically copied with it. The host computer system processes the voice command annotation file and executes one or more instructions associated with the command or commands contained in the file.

Claims

1. A voice command annotation method for a device, said method comprising the steps of:

(a) capturing a voice disposition command with a voice pickup component in said device;

(b) storing said voice disposition command in a voice command annotation file;

(c) associating said voice command annotation file with a data file in said device;

(d) copying said data file and said voice command annotation file from said device to a host computer system;

(e) processing said voice command annotation file in said host computer system to identify at least one function associated with said voice disposition command, comprising the steps of

(e1) generating a first recognition pattern representative of said voice disposition command,

(e2) comparing said first recognition pattern to at least one recognition pattern for said at least one function stored in a function recognition table in said memory in said host computer system,

(e3) when said first recognition pattern matches said at least one recognition pattern for said at least one function stored in said function recognition table, retrieving a command number associated with said at least one recognition pattern, and

(e4) accessing, through said command number, said at least one instruction, wherein said at least one instruction is linked to said command number; and

(f) executing at least one instruction associated with said at least one function, wherein said at least one instruction acts upon said data file.

2. The voice command annotation method for a device according to claim 1 wherein step (a) further comprises the step (a1), step (b) further comprises the step (b1), step (e) replaces step (e), and step (f) further comprises the step (f1):

(a1) capturing a plurality of additional voice disposition commands with said voice pickup component in said device;

(b1) storing said plurality of additional voice disposition commands in said voice command annotation file;

(e) processing said voice command annotation file in said host computer system to identify a plurality of functions, wherein one of each of said plurality of functions is associated with one of each of said plurality of additional voice disposition commands, said processing comprising the steps of; and

(f1) executing a plurality of additional at least one instructions, wherein one of each of said plurality of additional at least one instruction is associated with said one of each of said plurality of additional functions, and further wherein each of said plurality of additional at least one instruction acts upon said data file.

3. The voice command annotation method for a device according to claim 1 wherein step (a) further comprises the steps (a1) through (a3), step (b) further comprises the step (b1), and step (c) further comprises the step (c1):

(a1) outputting, from said voice pickup component, said voice disposition command as an analog signal to an analog-to-digital converter in said device;

(a2) converting said analog signal in said analog-to-digital converter to a digital signal;

(a3) outputting said digital signal from said analog-to-digital converter to said controller;

(b1) storing, by said controller, said digital signal in said voice command annotation file in a static memory in said device, wherein said voice command annotation file is an audio file format; and

(c1) associating said data file and said voice command annotation file by giving said data file and said voice command annotation file a same file name, and giving said data file a first file extension name, and giving said voice command annotation file a second file extension name.

4. The voice command annotation method for a device according to claim 1 wherein step (e2) further comprises the steps (e2a) through (e2c):

(e2a) when said first recognition pattern does not match said at least one recognition pattern for said at least one function stored in said function recognition table, outputting a first message in said host computer system indicating that no match was found, and outputting a second message in said host computer system prompting for input to listen to said voice command annotation file;

(e2b) when said input to listen from step (e2a) is received, calling audio file conversion software to process said voice command annotation file for audio output, and outputting a third message in said host computer system prompting for input for manual operations; and

(e2c) when said input for manual operations from step (e2b) is received, suspending said voice command annotation method until manual operations are performed.

5. The voice command annotation method for a device according to claim 1 wherein said command number is a command text.

6. The voice command annotation method for a device according to claim 1 wherein said device is a portable scanner device.

7. The voice command annotation method for a device according to claim 1 wherein said device is a digital camera.

8. The voice command annotation method for a device according to claim 1 wherein step (a) further comprises the step (a0) performed before step (a):

(a0) receiving a voice control command in said device to enable said device to capture said voice disposition command with said voice pickup component.

9. The voice command annotation method for a device according to claim 1 wherein step (a) further comprises the step (a0) performed before step (a), and step (b) further comprises the step (b1):

(a0) receiving button down input in said controller from an annotation control button on said device, wherein said device is enabled to capture said voice disposition command with said voice pickup component;

(b1) receiving button up input in said controller from said annotation control button on said device, wherein said device is disabled from capturing said voice disposition command with said voice pickup component.

10. A voice command annotation method for a device, said method comprising the steps of:

(a) capturing a voice disposition command with a voice pickup component in said device;

(b) processing said voice disposition command with a controller in said device comprising the steps of

(b1) calling, by said controller, voice analysis software from a memory in said device,

(b2) generating a first recognition pattern, wherein said first recognition pattern is representative of said voice disposition command,

(b3) comparing said first recognition pattern to at least one recognition pattern for said at least one function stored in a function recognition table in said memory in said device,

(b4) when said first recognition pattern matches said at least one recognition pattern for said at least one function stored in said function recognition table, retrieving a command number stored in said memory associated with said at least one recognition pattern;

(c) storing said voice disposition command processed with said controller in a voice command annotation file, comprising the step of

(c1) storing, by said controller, said command number in said voice command annotation file in said memory in said device;

(d) associating said voice command annotation file with a data file in said device;

(e) copying said data file and said voice command annotation file from said device to a host computer system;

(f) processing said voice command annotation file in said host computer system to identify at least one function associated with said command number; and

(g) executing at least one instruction associated with said at least one function, wherein said at least one instruction acts upon said data file.

11. The voice command annotation method for a device according to claim 10 wherein step (b3) further comprises the steps (b3a) and (b3b):

(b3a) when said first recognition pattern does not match said at least one recognition pattern for said at least one function stored in said function recognition table, outputting a first message in said device indicating that no match was found; and

(b3b) outputting a second message in said device prompting for input to repeat said voice disposition command or to cancel.

12. The voice command annotation method for a device according to claim 10 wherein said command number is a command text.

13. The voice command annotation method for a device according to claim 10 wherein step (e) further comprises the steps (e0) performed before step (e) and step (e1) performed after step (e), and step (f) further comprises the steps (f1) through (f2):

(e0) loading file transfer software in a memory in said host computer system;

(e1) receiving, through said file transfer software, said data file and said voice command annotation file into said memory in said host computer system;

(f1) retrieving said command number from said voice command annotation file with said file transfer software; and

(f2) accessing said at least one instruction, stored in said function recognition table in said memory, through said command number, wherein said at least one instruction is linked to said command number.

14. The voice command annotation method for a device according to claim 10 wherein step (a) further comprises the steps (a0a) through (a0j) performed before step (a):

(a0a) loading voice analysis software into a memory in a host computer system;

(a0b) connecting said device to said host computer system;

(a0c) selecting a predetermined function, with said voice analysis software, for training and voice analysis of at least one word for invoking said predetermined function;

(a0d) capturing a plurality of voice inputs of said at least one word in said voice pickup component of said device;

(a0e) processing said plurality of voice inputs into a plurality of digital signals in said device;

(a0f) sending said plurality of digital signals from said device to said host computer system;

(a0g) analyzing said plurality of digital signals with said voice analysis software;

(a0h) developing said at least one recognition pattern from said analysis of said plurality of digital signals with said voice analysis software;

(a0i) storing said at least one recognition pattern in said function recognition table in said memory in said host computer system; and

(a0j) transferring said at least one recognition pattern in said function recognition table in said memory in said host computer system for storage in said function recognition table in said static memory in said device, wherein said command number is associated with said at least one recognition pattern.

15. The voice command annotation method for a device according to claim 14 further comprising the steps of:

(a0k) repeating steps (a0c) through (a0j) for a plurality of predetermined functions, wherein a plurality of recognition patterns are developed from a plurality of said plurality of voice inputs for a plurality of said at least one words, and further wherein said plurality of recognition patterns are stored in said function recognition table in said static memory in said device, wherein each of said plurality of recognition patterns are associated with one of a plurality of command numbers stored in said function recognition table in said static memory.

16. The voice command annotation method for a device according to claim 14 wherein step (a0a) is replaced by the new step (a0a), steps (a0b), (a0f), and (a0i) are eliminated, and step (a0j) is replaced by the new step (a0j):

(a0a) accessing voice analysis software in said static memory in said device; and

(a0j) storing said at least one recognition pattern in said function recognition table in said static memory in said device, wherein said at least one recognition pattern is associated with said command number stored in said function recognition table in said static memory.

17. The voice command annotation method for a device according to claim 16 further comprising the steps of:

(a0k) repeating steps (a0c) through (a0j) for a plurality of predetermined functions, wherein a plurality of recognition patterns are developed from a plurality of said plurality of voice inputs for a plurality of said at least one words, and further wherein said plurality of recognition patterns are stored in said function recognition table in said static memory in said device, wherein each of said plurality of recognition patterns are associated with one of a plurality of command numbers stored in said function recognition table in said static memory.

18. A device with voice command annotation comprising:

a voice pickup component for capturing a voice disposition command, and for outputting said voice disposition command as an analog signal;

an analog-to-digital converter, connected to said voice pickup component, for converting said analog signal into a digital signal;

a controller, connected to said analog-to-digital converter, for receiving said digital signal from said analog-to-digital converter and storing said digital signal in a voice command annotation file; and

a host connection, connected to said controller, for establishing a connection with a host computer system, wherein said host computer system comprises

a memory,

voice analysis software loaded into said memory, and

file transfer software loaded into said memory, wherein said file transfer software processes said voice command annotation file by calling said voice analysis software to perform a plurality of frequency domain transforms on said digital signal in said voice command annotation file, generating a first recognition pattern, wherein said first recognition pattern is a spectral transform of said digital signal, and further wherein said file transfer software compares said first recognition pattern to at least one recognition pattern for said at least one function stored in a function recognition table in said memory, wherein when said first recognition pattern matches said at least one recognition pattern for said at least one function stored in said function recognition table, said file transfer software retrieves a command number associated with said at least one recognition pattern and accesses through said command number said at least one instruction, wherein said at least one instruction is linked to said command number, and further wherein said file transfer software executes said at least one instruction.

19. The device with voice command annotation according to claim 18 further comprising:

an annotation control button connected to said controller, wherein said device is enabled to capture said voice disposition command with said voice pickup component when button down input from said annotation control button is received by said controller, and until button up input from said annotation control button is received by said controller, said voice disposition command is captured by said voice pickup component between said button down input and said button up input, and is converted to said analog signal, and then converted to said digital signal by said analog-to-digital converter, and stored as said voice annotation file in said static memory in said device, and associated with said data file stored in said static memory.

20. The device with voice command annotation according to claim 18 wherein said controller associates said voice command annotation file to said data file by giving said data file and said voice command annotation file a same file name, and giving said data file a first file extension name, and giving said voice command annotation file a second file extension name.

21. The device with voice command annotation according to claim 18 wherein said device is a portable scanner device.

22. The device with voice command annotation according to claim 18 wherein said device is a digital camera.

23. A device with voice command annotation comprising:

a voice pickup component for capturing a voice disposition command;

a controller, connected to said voice pickup component, for deriving a first recognition pattern from said voice disposition command; and

a static memory, connected to said controller, for storing at least one recognition pattern in a function recognition table, and for storing a command number in said function recognition table associated with said at least one recognition pattern;

wherein, said controller compares said first recognition pattern with said at least one recognition pattern in said function recognition table stored in said static memory, and further wherein, when said first recognition pattern and said at least one recognition pattern match, said controller retrieves said command number from said function recognition table and stores said command number in a voice command annotation file in said static memory.

24. The device with voice command annotation according to claim 23 further comprising:

an analog-to-digital converter, connected to said voice pickup component;

wherein said voice disposition command is output to said analog-to-digital converter as an analog signal, and said analog-to-digital converter converts said analog signal into a digital signal, and further wherein said analog-to-digital converter outputs said digital signal to said controller, and said controller converts said digital signal into said first recognition pattern.

25. The device with voice command annotation according to claim 24 wherein said controller converts said digital signal into said first recognition pattern by performing a plurality of frequency domain transforms on said digital signal, generating said first recognition pattern, wherein said first recognition pattern is a spectral transform of said digital signal.

26. The device with voice command annotation according to claim 23 further comprising.

a host connection connected to said controller;

and

a host computer system having a communications interface, wherein said host connection of said device is connected to said communications interface of said host computer system, allowing communication to occur between said device and said host computer system;

wherein a plurality of voice inputs of at least one word to be used for invoking a predetermined function are captured by said voice pickup component, processed in said device into a plurality of digital signals, and said plurality of digital signals are transferred from said device to said host computer system, and further wherein said plurality of digital signals are analyzed, with voice analysis software loaded into a memory in said host computer system, generating a recognition pattern for said at least one word from said plurality of digital signals, wherein said recognition pattern is associated with said command number for said predetermined function, and further wherein said recognition pattern and said command number are transferred from said host computer system to said device and stored in said function recognition table in said static memory in said device.

27. The device with voice command annotation according to claim 23 wherein said device is a portable scanner device.

28. The device with voice command annotation according to claim 23 wherein said device is a digital camera.