System, a server for a system and a machine for use in a system

• US 6,975,993 B1
• Filed: 05/02/2000
• Issued: 12/13/2005
• Est. Priority Date: 05/21/1999
• Status: Expired due to Fees

• Current Assignee: Canon Kabushiki Kaisha (Canon Inc.)
• Original Assignee: Canon Kabushiki Kaisha (Canon Inc.)
• Inventors: Keiller, Robert Alexander
• Primary Examiner(s): Azad, Abul K.
• Application Number:
  US09/562,932
• Time in Patent Office: 2,051 Days
• Field of Search: 704/270.1, 704/270, 704/275, 704/273, 707/104.1, 709/203, 709/218, 709/219
• US Class Current: 704/275
• CPC Class Codes: G10L 15/26 Speech to text systems G10L15/08 takes precedence
• CPC Class Codes: G10L 2015/223 Execution procedure of a spoken command
• CPC Class Codes: G10L 2015/228 of application context
• CPC Class Codes: Y10S 707/99945 Object-oriented database structure processing
• CPC Class Codes: Y10S 707/99948 Application of database or data structure, e.g. distributed, multimedia, or image

Abstract

A system has a first machine (9 to 11,12,13,15) couplable to a network (N) and capable of carrying out at least one function, a speech data receiver (26,27) for receiving speech data representing instructions spoken by a user and specifying a function to be carried out by the first machine and a speech data transmitter (27,28) for transmitting the speech data to a speech server (2) couplable to the network (N). The speech server (2) has a speech manager (6) for accessing a speech recognition engine (5) for performing speech recognition on speech data received over the network to produce recognised speech data, an interpreter (7) for processing recognised speech data to derive from the speech data commands for causing the first machine to carry out the function specified in the spoken instructions and a command transmitter (6) for transmitting said commands over the network to the first machine, the first machine having a control command receiver (27,28) for receiving control commands over the network (N) from the speech server (2) and a controller (20,27) responsive to the control commands for causing the function specified by the spoken instructions to be carried out.

Claims

1. A system for enabling a user to control operation of a networked machine by issuing a spoken instruction, the system comprising:

a plurality of processor-controlled machines couplable to the network, each processor-controlled machine being capable of carrying out at least one function and each processor-controlled machine having control command receiving means for receiving control commands over a network, and function actioning means for causing a function specified by received control command data to be carried out;

speech data receiving means for receiving speech data representing speech spoken by a user;

a spoken instruction interpreting apparatus couplable to the network, the spoken instruction interpreting apparatus having processor-controlled machine identifying means for identifying the one of the plurality of processor-controlled machines that the user wishes to control, accessing means for accessing speech recognition means to cause the speech recognition means to perform speech recognition processing on speech data received by the speech data receiving means to produce recognised speech data, processing means for processing recognised speech data received from the speech recognition means to derive from the recognised speech data control command data for controlling the identified processor-controlled machine, and control command data transmitting means for transmitting the derived control command data over the network to the identified processor-controlled machine to enable the function actioning means of the identified processor-controlled machine to cause the function specified by the user'"'"'s spoken instruction to be carried out;

look-up means for storing information relating to the functions capable of being carried out by processor-controlled machines coupled to the network;

determining means for determining whether the identified processor-controlled machine can carry out the function specified in a spoken instruction; and

advising means for advising the user, on the basis of the information stored in the look-up means, if there is another processor-controlled machine coupled to the network that can carry out the function specified in a spoken instruction if the determining means determines that the identified processor-controlled machine cannot carry out the function.

2. A system according to claim 1, wherein the spoken instruction interpreting apparatus has a grammar store which defines at least one of words and phrases that can be used in spoken instructions.

3. A system according to claim 2, wherein at least one of the plurality of processor-controlled machines is capable of carrying out different functions and wherein the grammar store stores a respective grammar for each different function.

4. A system according to claim 3, wherein the grammar store stores a shared grammar containing words common to instructions relating to the different functions.

5. A system according to claim 1, wherein the spoken instruction interpreting apparatus has a grammar store storing rules determining the structure of phrases that can be used in spoken input instructions.

6. A system according to claim 1, wherein the spoken instruction interpreting apparatus has a grammar store storing rules determining the structure of phrases that can be used in spoken instructions and a vocabulary of words that can be used in those phrases.

7. A system according to claim 1, wherein at least one of the plurality of processor-controlled machines is capable of carrying out different functions.

8. A system according to claim 7, wherein the different functions comprise photocopying, facsimile transmission and printing.

9. A system according to claim 1, wherein the plurality of processor-controlled machines comprises a plurality of different types of processor-controlled machine, each different type being capable of carrying out a different function.

10. A system according to claim 1, wherein speech data receiving means and speech data transmitting means are associated with each of the plurality of processor-controlled machines.

11. A system according to claim 10, wherein the speech data receiving means and speech data transmitting means form part of the associated processor-controlled machine.

12. A system according to claim 1, wherein the speech data receiving means and speech data transmitting means are associated with an additional processor-controlled machine coupled to the network.

13. A system according to claim 12, wherein the additional processor-controlled machine is a personal computer or a digital camera and at least one of the plurality of processor-controlled machines comprises a printer.

14. A system according to claim 1, wherein the plurality of processor-controlled machines comprise:

an instruction originating processor-controlled machines coupled to the network, having means for receiving speech data representing instructions spoken by a user of the machine and means for transmitting the speech data over the network to the spoken instruction interpreting apparatus; and

an instruction receiving machine coupled to the network, having means for receiving control commands over the network from the spoken instruction interpreting apparatus and means for causing the instruction receiving machine to act in accordance with the control commands whereby, in use, the instruction receiving machine is caused to carry out the function specified in the instructions spoken by the user of the instruction originating machine.

15. A system according to claim 1, wherein the speech data receiving means comprises a telephone system coupled to one of one of the plurality of processor-controlled machines and the spoken instruction interpreting apparatus.

16. A system according to claim 15, wherein the telephone system comprises a cordless telephone system.

17. A system according to claim 16, wherein the telephone system comprises a DECT telephone system.

18. A system according to claim 17, further comprising means for identifying the processor-controlled machine at which a user is located in accordance with the identity of the user'"'"'s DECT telephone.

19. A system according to claim 17, further comprising means for selecting a speech recognition means to be used by the speech data processing means in accordance with the identity of the DECT telephone.

20. A system according to claim 1, wherein the accessing means is arranged to access a plurality of different speech recognition means and to use the results of speech recognition performed by each of the different speech recognition means to derive the recognised speech data.

21. A system according to claim 20, wherein at least some of the speech recognition means are provided by computers couplable to the network.

22. A system according to claim 1, wherein at least one of the spoken instruction interpreting apparatus and at least one of the processor-controlled machines is provided with the speech recognition means.

23. A system according to claim 1, wherein at least one of the spoken instruction interpreting apparatus and at least one of the processor-controlled machines comprises for identifying the speaker of the instructions and means for accessing speech recognition means trained to the voice of the speaker.

24. A system according to claim 1, wherein;

the accessing means is arranged to access a grammar store associating each of a number of voice macros or spoken words or phrases with a series of functions to be carried out at a first machine.

25. A system according to claim 1, wherein the plurality of processor-controlled machines comprises a plurality of first machines, a second machine,

and means for transmitting the speech data to the second machine, the second machine having the spoken instruction interpreting apparatus for interpreting spoken instructions received by the receiving means and for transmitting commands over the network to one of the plurality of first machines, at least that first machine having means for receiving control commands over the network from the second machine, and at least that first machine having the determining means for determining whether that first machine can process the received commands and the advising means for advising the user of another machine on the network that can process the received commands if the determining means determines that that first machine cannot.

26. A system according to claim 1, comprising a shared grammar which defines words that can be used in spoken instructions in relation to any machine of a particular type.

27. A system according to claim 26, wherein the determining means is arranged to use the shared grammar to determine if a processor-controlled machine can carry out an identified function.

28. A system according to claim 1, comprising a start up grammar, means for identifying a processor-controlled machine type on the basis of speech recognized using the start up grammar, and means for selecting a grammar which defines words that can be used in spoken instructions in relation to the identified processor-controlled machine type.

29. A system according claim 1, wherein a processor-controlled machine comprises a manually operable user interface for receiving manually input commands for causing the machine to carry out a function, the processor-controlled machine being operable to carry out a function in accordance with manually input commands, spoken commands or a combination of manually input and spoken commands.

30. A system according to claim 1, further comprising means for compressing speech data for transmission by the transmitting means.

31. A system comprising:

a speech server couplable to a network;

an instruction originating processor-controlled machine couplable to the network;

means for receiving speech data representing instructions spoken by a user and specifying a function to be carried out;

means for transmitting speech data to the speech server; and

an instruction receiving processor-controlled machine couplable to the network having means for receiving control commands over the network and means for causing the instruction receiving machine to act in accordance with the control commands,

the speech server comprising a spoken instruction interpreting apparatus having processor-controlled machine identifying means for identifying the one of the plurality of processor-controlled machines that the user wishes to control, accessing means for accessing speech recognition means to cause the speech recognition means to perform speech recognition processing on speech data received by the speech data receiving means to produce recognised speech data, processing means for processing recognised speech data received from the speech recognition means to derive from the recognised speech data control command data for controlling the identified processor-controlled machine, and control command data transmitting means for transmitting the derived control command data over the network to the identified processor-controlled machine to enable the function actioning means of the identified processor-controlled machine to cause the function specified by the user'"'"'s spoken instruction to be carried out,

the system further comprising;

look-up means for storing information relating to the functions capable of being carried out by processor-controlled machines coupled to the network;

determining means for determining whether the identified processor-controlled machine can carry out the function specified in a spoken instruction; and

advising means for advising the user, on the basis of the information stored in the look-up means, if there is another processor-controlled machine coupled to the network that can carry out the function specified in a spoken instruction if the determining means determines that the identified processor-controlled machine cannot carry out the function.

32. A system according to claim 31, wherein the instruction originating processor-controlled machine is a digital camera or computer and the instruction receiving processor-controlled machine is a printer.

33. A system according to claim 32, wherein the look-up means comprises a JINI look-up service containing a directory of printers comprising Java virtual machines connected to the network and identifying the functions that can be carried out by those printers, and the spoken instruction interpreting apparatus comprises a Java virtual machine and is operable to download a JINU service object associated with the printer to the instruction originating processor-controlled machine in response to receipt of such a request and to send to the JINI service object commands for causing the printer to carry out a printing function in response to speech data received from the user, the JINI service object providing the determining means for determining whether the printer can carry out the requested printing function and, if so, for communicating with the printer over the network to cause the printer to carry out the requested printing function.

36. Apparatus according to claim 33, wherein the grammar store stores a respective grammar for each of a plurality of different functions that can be carried out by machines coupled to the network.

37. Apparatus according to claim 36, wherein the grammar store stores a shared grammar containing words common to instructions relating to the different functions.

38. Apparatus according to claims 36, wherein the grammar store stores copy, fax and print grammars.

34. An apparatus for interpreting spoken instructions for use in a network system to which processor-controlled machines are coupled, the apparatus comprising:

accessing means for accessing speech recognition means for performing speech recognition to produce recognised speech data on received speech data representing spoken instructions;

processing means for processing recognised speech data to derive from the recognised speech data commands for causing a machine coupled to the network to carry out a function specified by the spoken instructions;

processor-controlled machine identifying means for identifying the one of the plurality of processor-controlled machines that the user wishes to control;

control command data transmitting means for transmitting the derived control command data over the network to the identified processor-controlled machine to enable the identified processor-controlled machine to carry out the function specified by the user'"'"'s spoken instruction;

look-up means for storing information relating to the functions that processor-controlled machines coupled to the network are capable of carrying out;

determining means for determining whether the identified processor-controlled machine can carry out the function specified in a spoken instruction; and

advising means for advising the user, on the basis of the information stored in the look-up means, if there is another processor-controlled machine coupled to the network that can carry out the function specified in a spoken instruction if the determining means determines that the identified processor-controlled machine cannot carry out the function.

35. Apparatus according to claim 34, comprising a grammar store which defines at least one of words and phrases that can be used in spoken instructions.

39. Apparatus according to claim 34, comprising a grammar store storing rules determining the structure of phrases that can be used in spoken instructions.

40. Apparatus according to claim 34, having a grammar store storing rules determining the structure of phrases that can be used in spoken instructions and a vocabulary of words that can be used in those phrases.

41. Apparatus according to claim 34, having means for communicating with a telephone system to receive speech data.

42. Apparatus according to claim 34, wherein the accessing means is arranged to access a plurality of different speech recognition means and to use the results of speech recognition performed by each of the different speech recognition means to derive the recognised speech data.

43. Apparatus according to claim 34, provided with the speech recognition means.

44. Apparatus according to claim 34, further comprising means for identifying the speaker of the instructions and means for accessing speech recognition means trained to the voice of the speaker.

45. A signal carrying processor implementable instructions for causing processing means to be configured to provide an apparatus as claimed in claim 34.

46. Apparatus according to claim 34, wherein the look-up means is a JINI look-up service.

47. A machine for carrying out at least one function, comprising:

means for coupling the machine to a network;

means for receiving speech data representing spoken instructions specifying a function to be carried out by the machine;

means for supplying the speech data to speech processing means coupled to the network;

means for receiving from the network control commands derived from the speech data supplied to the speech processing means;

means responsive to the control commands for causing the function specified by the spoken instructions to be carried out

means for determining whether the function specified by the spoken instructions can be carried out; and

means for advising the user if the function cannot be carried out, wherein the advising means comprises means for accessing a look-up store containing information relating to the functions that can be carried out by machines coupled to the network and means for advising the user of any other machine that can carry out the requested function.

48. A machine according to claim 47, wherein the machine has a photocopying function.

49. A machine according to claim 47, wherein the machine has a facsimile function.

50. A machine according to claim 47, wherein the advising means comprise means for causing a message to be displayed on a display of the machine.

51. A digital camera comprising:

means for coupling the camera to a network;

means for receiving speech data representing spoken instructions specifying a print function to be carried out by a printer coupled to the network;

means for supplying the speech data to speech processing means coupled to the network;

means for receiving from the network control commands derived from the speech data supplied to the speech processing means;

means responsive to control commands received from the network for supplying the control commands together with the data to be printed over the network to the printer for causing the print function specified by the spoken instructions to be carried out by the printer;

means for determining whether the function specified by the spoken instructions can be carried out by the printer; and

means for advising the user if the function cannot be carried out by the printer, wherein the advising means comprises means for accessing a look-up store containing information relating to the functions that can be carried out by the printers coupled to the network and means for advising the user of any other printers that can carry out the specified function.

52. A device for controlling a machine for carrying out at least one function, comprising:

means for coupling the machine to a network;

means for receiving speech data representing spoken instructions specifying a function to be carried out by the machine;

means for supplying the speech data to speech processing means coupled to the network;

means for receiving from the network control commands derived from the speech data supplied to the speech processing means;

means responsive to the control commands for supplying to the machine commands for causing the function specified by the spoken instructions to be carried out;

means for determining whether the function specified by the spoken instructions can be carried out; and

means for advising the user if the function cannot be carried out, wherein the advising means comprises means for accessing a look-up store containing information relating to the functions that can be carried out by machines coupled to the network and means for advising the user of any other machine that can carry out the requested function.

53. A device according to claim 52, operable to control a photocopying machine.

54. A device according to claim 52, operable to control a facsimile machine.

55. A device according to claim 52, operable to control a digital camera,

wherein the means for receiving speech data is operable to receive spoken instructions specifying a print function to be carried out by a printer coupled to the network, and,

wherein the means responsive to control commands received from the network is operable to supply the control commands together with the data to be printed over the network to the printer for causing the print function specified by the spoken instructions to be carried out by the printer.

56. A device according to claim 52, wherein the advising means comprise means for causing a message to be displayed on a display of the machine.

57. A method of controlling operation of a machine coupled to a network, comprising:

receiving speech data representing spoken instructions for controlling the machine;

performing speech recognition on the received speech data to produce recognized speech data;

processing the recognized speech data to derive from the recognized speech data commands for causing the machine to carry out a function specified by the spoken instructions;

determining whether the function specified by the spoken instructions can be carried out; and

in the event the function cannot be carried out, accessing a look-up store containing information relating to the functions that can be carried out by machines coupled to the network and then advising the user of any other machine that can carry out the requested function.

58. A method according to claim 57, which comprises using a grammar store which defines at least one of words and phrases that can be used in spoken instructions to perform said speech recognition.

59. A method according to claim 57, which comprises using a grammar store storing rules determining the structure of phrases that can be used in spoken instructions to perform said speech recognition.

60. A method according to claim 57, which comprises using a grammar store storing rules determining the structure of phrases that can be used in spoken instructions and a vocabulary of words that can be used in those phrases to perform said speech recognition.

61. A method according to claim 57, which comprises using a grammar store storing a respective grammar for each of a plurality of different functions that can be carried out by processor-controlled machines coupled to the network.

62. A method according to claim 61, which comprises using as said grammar store a grammar store storing a shared grammar containing words common to instructions relating to the different functions.

63. A method according to claim 61, which comprises using a grammar store storing copy, fax and print grammars.

64. A method according to claim 57, which comprises receiving speech data over a telephone system.

65. A method according to claim 57, which comprises performing speech recognition using a plurality of different speech recognition means and deriving the recognised speech data using the results of speech recognition performed by each of the different speech recognition means.

66. A method according to claim 57, further comprising identifying the speaker of the instructions and accessing speech recognition means trained to the voice of the speaker to perform the speech recognition.

67. A signal carrying processor implementable instructions for causing processing means to carry out a method as claimed in claim 57.

68. A method of operating a machine, comprising:

receiving speech data representing spoken instructions specifying a function to be carried out by the machine;

supplying the speech data to a network;

receiving from the network control commands derived from the speech data supplied to the network for causing the function specified by the spoken instruction to be carried out;

determining whether the function specified by the spoken instructions can be carried out; and

in the event the function cannot be carried out, accessing a look-up store containing information relating to the functions that can be carried out by machines coupled to the network and then advising the user of any other machine that can carry out the requested function.

69. A method according to claim 68, which comprises advising the user by causing a message to be displayed on a display of the machine.

70. A storage medium carrying processor implementable instructions for causing processing means to carry out a method as claimed in claim 68.

71. An apparatus for interpreting spoken instructions for use in a network system to which processor-controlled machines are coupled, the apparatus comprising:

an accessor operable to access a speech recognizer that performs speech recognition to produce recognised speech data based on received speech data representing spoken instructions;

a processor operable to process recognised speech data to derive from the recognised speech data commands for causing a machine coupled to the network to carry out a function specified by the spoken instructions;

a processor-controlled machine identifier operable to identify the one of the plurality of processor-controlled machines that the user wishes to control;

a control command data transmitter operable to transmit the derived control command data over the network to the identified processor-controlled machine to enable the identified processor-controlled machine to carry out a function;

a look-up store storing information relating to the functions that processor-controlled machines coupled to the network are capable of carrying out;

a determiner operable to determine whether the identified processor-controlled machine can carry out the function specified in a spoken instruction; and

an advisor operable to advise the user, on the basis of the information stored in the look-up store, if there is another processor-controlled machine coupled to the network that can carry out the function specified in a spoken instruction if the determiner determines that the identified processor-controlled machine cannot carry out the function.