Natural language machine interface

• US 7,062,428 B2
• Filed: 03/13/2001
• Issued: 06/13/2006
• Est. Priority Date: 03/22/2000
• Status: Expired due to Fees

• Current Assignee: Canon Kabushiki Kaisha (Canon Inc.)
• Original Assignee: Canon Kabushiki Kaisha (Canon Inc.)
• Inventors: Noble, Robert John, Hogenhout, Wide Roeland
• Primary Examiner(s): Dorvil, Richemond
• Assistant Examiner(s): Han, Qi
• Application Number:
  US09/803,974
• Publication Number:
  US 20010053969A1
• Time in Patent Office: 1,918 Days
• Field of Search: 704/9, 704/275, 704/235, 700/83
• US Class Current: 704/9
• CPC Class Codes: G06F 40/279 Recognition of textual entities
• CPC Class Codes: G06F 40/289 Phrasal analysis, e.g. finite state techniques or chunking
• CPC Class Codes: G06F 40/56 Natural language generation
• CPC Class Codes: G10L 13/00 Speech synthesis; Text to speech systems
• CPC Class Codes: G10L 15/183 using context dependencies, e.g. language models

Abstract

A user interface is provided for use with a machine which can use a number of natural language instructions to reach one of a plurality of possible machine states. In order to provide information to a user to enable them to more efficiently achieve the current machine state using natural language instructions, the current state of the machine is determined and used to generate information to inform the user of a natural language instruction which can be input to a machine to achieve the current state of the machine.

Claims

1. A user interface apparatus for use with a machine which can execute a number of user instructions to reach one of a plurality of possible machine states, wherein said user instructions can be input to said machine using at least natural language, the user interface apparatus comprising:

status means for obtaining a current state of said machine achieved by executing an instruction recieved from a user;

generating means responsive to the obtained current state of said machine, for generating a natural language instruction which could have been excuted by said machine to achieve the current state of said machine, irrespective of the instruction which has actually been executed to achieve the current state; and

output means for outputting information to inform the user of the generated natural language instruction.

2. The user interface apparatus according to claim 1, including means for receiving a request from a user of said machine for said information, wherein said generating means is adapted to be responsive to a received request to generate said information.

3. The user interface apparatus according to claim 1, including means for receiving, from a user who has input a natural language instruction to arrive at the current state of said machine, a request for an appropriate natural language instruction to reach the current state, wherein said generating means is adapted to be responsive to a received request to generate said information as the requested appropriate natural language instruction.

4. The user interface apparatus according to claim 1, wherein said status means is adapted to maintain a data structure containing attributes defining the current state of the machine, each attribute having a number of possible values.

5. The user interface apparatus according to claim 4, wherein each attribute has at least one natural language fragment associated therewith, and said generating means is adapted to generate said information by building up a natural language instruction from said natural language fragments for said attributes defining the current state of said machine.

6. The user interface apparatus according to claim 5, wherein said generating means is adapted to build said natural language instruction in accordance with natural language rules.

7. The user interface apparatus according to claim 5, wherein said generating means is adapted to order said natural language fragments in accordance with order rules.

8. The user interface apparatus according to claim 5, wherein said generating means is adapted to replace elements in the natural language instruction with other elements in dependence upon at least one of previous user interactions, preferred synonyms, user preferences, and natural language input recognition problems.

9. The user interface apparatus according to claim 5, wherein said generating means is adapted to add natural language elements to said natural language instruction as at least one of a surrounding phrase and a politeness term.

10. The user interface apparatus according to claim 1, wherein said generating means is adapted to generate said information as text.

11. The user interface apparatus according to claim 10, wherein said output means is adapted to display said text.

12. The user interface apparatus according to claim 10, wherein said output means includes speech synthesis means for synthesising speech from said text and audio output means for audibly outputting said speech.

13. The user interface apparatus according to claim 1, wherein said generating means is adapted to generate said information as speech data.

14. A machine which can receive a number of user instructions to reach one of a plurality of possible machine states, said machine including:

instructions input means to input said user instructions to said machine using at least natural language as a mode of input; and

the user interface apparatus according to claim 1.

15. A user interface method for use with a machine which can execute a number of user instructions to reach one of a plurality of possible machine states, wherein said user instructions can be input to said machine using at least natural language, the method comprising:

obtaining a current state of said machine achieved by executing an instruction received from a user;

in response to the obtained current state of said machine, generating a natural language instruction which could have been executed by said machine to achieve the current state of said machine, irrespective of the instruction which has actually been executed to achieve the current state; and

outputting information to inform the user of the generated natural language instruction.

16. The user interface method according to claim 15, including receiving a request from a user of said machine for said information, wherein said information is generated in response to said request.

17. The user interface method according to claim 15, including receiving, from a user who has input a natural language instruction to arrive at the current state of said machine, a request for an appropriate natural language instruction to reach the current state, wherein said information is generated as the requested appropriate natural language instruction in response to said request.

18. The user interface method according to claim 15, wherein the current state of said machine is obtained as a data structure containing attributes defining the current state of the machine, each attribute having a number of possible values.

19. The user interface method according to claim 18, wherein each attribute has at least one natural language fragment associated therewith, and said information is generated by building up a natural language instruction from said natural language fragments for said attributes defining the current state of said machine.

20. The user interface method according to claim 19, wherein said natural language instruction is built in accordance with natural language rules.

21. The user interface method according to claim 19, wherein said natural language fragments are ordered in said natural language instruction in accordance with order rules.

22. The user interface method according to claim 19, including replacing elements in the natural language instruction with other elements in dependence upon at least one of previous user interactions, preferred synonyms, user preferences, and natural language input recognition problems.

23. The user interface method according to claim 19, including adding natural language elements to said natural language instruction as at least one of a surrounding phrase and a politeness term.

24. The user interface method according to claim 15, wherein said information is generated as text.

25. The user interface method according to claim 24, wherein said text is displayed so as to be output to the user.

26. The user interface method according to claim 24, including speech data synthesis from said text, wherein said speech data is used to generate audible speech output to the user.

27. The user interface method according to claim 15, wherein said information is generated as speech data.

28. Program code for programming a processor to provide a user interface for a machine which can execute a number of user instructions to reach one of a plurality of possible machine states, wherein said user instructions can be input to said machine using at least natural language, the program code being operable to program the processor to:

obtain a current state of said machine achieved by executing an instruction received from a user;

in response to the obtained current state of said machine, generate a natural language instruction which could have been executed by said machine to achieve the current state of said machine, irrespective of the instruction which has actually been executed to achieve the current state; and

output information to inform the user of the generated natural language instruction.

29. A carrier medium carrying the program code according to claim 28.

30. A user interface apparatus for use with a machine which can execute a number of user instructions to reach one of a plurality of possible machine states, wherein said user instructions can be input to said machine using at least natural language, the user interface apparatus comprising:

a status obtainer operable to obtain a current state of said machine achieved by executing an instruction received from a user;

a generator responsive to the obtained current state of said machine to generate a natural language instruction which could have been executed by said machine to achieve the current state of said machine, irrespective of the instruction which has actually been executed to achieve the current state; and

an outputter operable to output information to inform the user of the generated natural language instruction.