Front-end device independence for natural interaction platform

• US 7,302,394 B1
• Filed: 12/20/2002
• Issued: 11/27/2007
• Est. Priority Date: 12/20/2001
• Status: Active Grant

• Current Assignee: iAnywhere Solutions Incorporated (SAP SE)
• Original Assignee: iAnywhere Solutions Incorporated (SAP SE)
• Inventors: Baray, Cristobal H., Brooks, David A., Dolan, Damian M.
• Primary Examiner(s): Storm; Donald L.
• Application Number:
  US10/326,452
• Time in Patent Office: 1,803 Days
• Field of Search: None
• US Class Current: 704/257
• CPC Class Codes: G10L 15/18 using natural language modelling
• CPC Class Codes: G10L 15/24 Speech recognition using non-acoustical features

Abstract

Roughly described, a natural language interpretation system that provides commands to a back-end application in response to user input is modified to separate out user interaction functions into a user interaction subsystem. The user interaction subsystem can include an interaction block that is specific to each particular I/O agency, and which converts user input received from that agency into an agency-independent form for providing to the natural language interpretation system. The user interaction subsystem also can take results from the back-end application and clarification requests and other dialoguing from the natural language interpretation system, both in device-independent form, and convert them for forwarding to the particular I/O agency.

Claims

1. User interactive apparatus comprising:

a natural language interpreter having an input for receiving token sequences and an output for outputting commands toward a back-end application subsystem in response to said token sequences; and

an interaction subsystem having interfaces to each of first and second I/O agencies, said first and second I/O agencies communicating user input to said interfaces via respective first and second different I/O modalities, said interaction subsystem providing token sequences to said natural language interpreter in response to user input received by said interaction subsystem via each of said interfaces.

2. Apparatus according to claim 1, wherein said natural language interpreter comprises a distributed parser of natural language input.

3. Apparatus according to claim 1, wherein said natural language interpreter comprises an agent network.

4. Apparatus according to claim 3, wherein said agent network prepares said commands in dependence upon satisfaction, in said token sequences, of one or more policy conditions which make reference to one or more keywords in a predetermined set of keywords.

5. Apparatus according to claim 1, wherein said natural language interpreter comprises:

a plurality of agents arranged in a network, each of said agents having at least one interpretation policy having a policy condition and a policy action associated with the condition, each of said interpretation policies making a claim on at least part of a token sequence provided to the agent if the condition of the interpretation policy is satisfied, and executing the associated policy action in response to determination that the claim is at least part of a winning claim.

6. Apparatus according to claim 5, wherein a particular one of said agents has a particular interpretation policy having a policy condition that refers to a downchain one of said agents, any claim made by said particular policy being dependent upon any claims made by said downchain agent.

7. Apparatus according to claim 5, wherein a particular one of said agents has a particular interpretation policy having a particular policy condition that refers to a downchain one of said agents, said particular interpretation policy further having a particular policy action associated with said particular policy condition,

wherein said particular policy action includes at least partial delegation to said downchain agent.

8. Apparatus according to claim 1, wherein said interaction subsystem includes first and second interaction components, and wherein user input is received via said first and second I/O modalities according to first and second different input formats,

said first interaction component converting user input from said first I/O agency according to said first input format to token sequences according to a common format for providing to said natural language interpreter,

and said second interaction component converting user input from said second I/O agency according to said second input format to token sequences according to said common format for providing to said natural language interpreter.

9. Apparatus according to claim 1, wherein said interfaces further communicate output from said natural language interpreter to said first and second I/O agencies via respectively said first and second I/O modalities.

10. Apparatus according to claim 9, wherein said interaction subsystem includes first and second interaction components,

said first interaction component forwarding output from said natural language interpreter toward said first I/O agency via said first I/O modality,

and said second interaction component forwarding output from said natural language interpreter toward said second I/O agency via said second I/O modality.

11. Apparatus according to claim 1, wherein said interfaces further communicate output from said back-end application subsystem to said first and second I/O agencies via respectively said first and second I/O modalities.

12. Apparatus according to claim 11, wherein said interaction subsystem includes first and second interaction components,

said first interaction component forwarding output from said back-end application subsystem toward said first I/O agency via said first I/O modality,

and said second interaction component forwarding output from said back-end application subsystem toward said second I/O agency via said second I/O modality.

13. Apparatus according to claim 1, wherein said first agency has first layout characteristics and said second agency has second layout characteristics different from said first layout characteristics,

and wherein said interaction subsystem includes first and second interaction components,

said first interaction component converting output from said back-end application subsystem for presentation according to said first layout characteristics,

and said second interaction component converting output from said back-end application subsystem for presentation according to said second layout characteristics.

14. Apparatus according to claim 13, wherein said interfaces further communicate output from said back-end application subsystem to said first and second I/O agencies via respectively said first and second I/O modalities,

wherein said first interaction component forwards said output from said back-end application subsystem according to said first layout characteristics toward said first I/O agency via said first I/O modality,

and wherein said second interaction component forwards output from said back-end application subsystem according to said second layout characteristics toward said second I/O agency via said second I/O modality.

15. Apparatus according to claim 1, wherein said interfaces further communicate output from said natural language interpreter and from said back-end application subsystem to said first and second I/O agencies via respectively said first and second I/O modalities.

16. Apparatus according to claim 15, wherein said interaction subsystem includes first and second interaction components,

said first interaction component forwarding output from said natural language interpreter and from said back-end application subsystem toward said first I/O agency via said first I/O modality,

and said second interaction component forwarding output from said natural language interpreter and from said back-end application subsystem toward said second I/O agency via said second I/O modality.

17. Apparatus according to claim 1, wherein said first agency has first layout characteristics and said second agency has second layout characteristics different from said first layout characteristics.

18. Apparatus according to claim 17, wherein said interaction subsystem includes first and second interaction components,

said first interaction component converting output from said natural language interpreter for presentation according to said first layout characteristics,

and said second interaction component converting output from said natural language interpreter for presentation according to said second layout characteristics.

19. Apparatus according to claim 18, wherein said interfaces further communicate output from said natural language interpreter to said first and second I/O agencies via respectively said first and second I/O modalities,

wherein said first interaction component forwards said output from said natural language interpreter according to said first layout characteristics toward said first I/O agency via said first I/O modality,

and wherein said second interaction component forwards output from said natural language interpreter according to said second layout characteristics toward said second I/O agency via said second I/O modality.

20. Apparatus according to claim 17, wherein said interaction subsystem includes first and second interaction components,

said first interaction component converting output from said natural language interpreter and from said back-end application subsystem for presentation according to said first layout characteristics,

and said second interaction component converting output from said natural language interpreter and from said back-end application subsystem for presentation according to said second layout characteristics.

21. Apparatus according to claim 20, wherein said interfaces further communicate output from said natural language interpreter to said first and second I/O agencies via respectively said first and second I/O modalities,

wherein said first interaction component forwards said output from said natural language interpreter and from said back-end application subsystem according to said first layout characteristics toward said first I/O agency via said first I/O modality,

and wherein said second interaction component forwards output from said natural language interpreter and from said back-end application subsystem according to said second layout characteristics toward said second I/O agency via said second I/O modality.

22. User interactive apparatus comprising:

a natural language interpreter having an output for providing clarification requests toward a user in a common output format; and

an interaction subsystem having interfaces to each of first and second I/O agencies, said first and second I/O agencies being arranged to receive output from said apparatus according to respective first and second different output formats,

said interaction subsystem receiving said clarification requests in said common output format, converting a first subset of said clarification requests to said first output format and communicating them toward said first I/O agency, said interaction subsystem further converting a second subset of said clarification requests to said second output format and communicating them toward said second I/O agency.

23. Apparatus according to claim 22, wherein said natural language interpreter comprises a distributed parser of natural language input.

24. Apparatus according to claim 22, wherein said natural language interpreter comprises an agent network.

25. Apparatus according to claim 22, wherein said interaction subsystem includes first and second interaction components,

said first interaction component converting said first subset of said clarification requests to said first output format and communicating them toward said first I/O agency, and said second interaction component converting said second subset of said clarification requests to said second output format and communicating them toward said second I/O agency.

26. Apparatus according to claim 22, wherein said natural language interpreter further has an input for receiving token sequences and an output for outputting commands toward a back-end application subsystem in response to said token sequences,

and wherein said interfaces further communicate output from said back-end application subsystem to said first and second I/O agencies according to said respective first and second different output formats.

27. A user interaction method, comprising the steps of:

instantiating a first user interaction module having a user-side interface arranged to communicate with a user via a first I/O modality;

instantiating a second user interaction module having a user-side interface arranged to communicate with a user via a second I/O modality different from said first I/O modality; and

instantiating a natural language interpreter having an input for receiving token sequences and an output for outputting commands toward a back-end application subsystem in response to said token sequences;

said first user interaction module receiving first user input via said first I/O modality, translating said first user input into first token sequences, and forwarding said first token sequences toward said natural language interpreter for interpretation;

and said second user interaction module receiving second user input via said second I/O modality, translating said second user input into second token sequences and forwarding said second token sequences toward said natural language interpreter for interpretation.

28. A method according to claim 27, further comprising the steps of:

said natural language interpreter outputting a command toward said back-end application subsystem in response to one of said first token sequences;

said back-end application subsystem providing a response to said command; and

a particular one of said first and second user interaction modules receiving said response, translating said response into a form usable by the I/O modality of said particular user interaction module, and forwarding said response toward said user via the I/O modality of said particular user interaction module.

29. A method according to claim 28, further comprising the steps of:

said natural language interpreter outputting a second command toward said back-end application subsystem in response to one of said second token sequences;

said back-end application subsystem providing a second response to said second command; and

said second user interaction module receiving said second response, translating said second response into a form usable by said second I/O modality, and forwarding said second response toward said user via said second modality.

30. A method according to claim 27, further comprising the steps of:

said natural language interpreter issuing a clarification request in response to one of said first or second token sequences; and

said first user interaction module receiving said clarification request, translating said clarification request into a form usable by said first I/O modality, and forwarding said clarification request toward said user via said first modality.

31. A method according to claim 30, further comprising the steps of:

said natural language interpreter outputting a command toward said back-end application subsystem in response to one of said first token sequences;

said back-end application subsystem providing a response to said command; and

said first user interaction module receiving said response, translating said response into a form usable by said first I/O modality, and forwarding said response toward said user via said first modality.

32. A method according to claim 30, further comprising the steps of:

said natural language interpreter issuing a second clarification request in response to one of said second token sequences; and

said second user interaction module receiving said second clarification request, translating said second clarification request into a form usable by said second I/O modality, and forwarding said second clarification request toward said user via said second modality.

33. A method according to claim 32, further comprising the steps of:

said natural language interpreter outputting a first command toward said back-end application subsystem in response to one of said first token sequences;

said back-end application subsystem providing a first response to said first command;

said first user interaction module receiving said first response, translating said first response into a form usable by said first I/O modality, and forwarding said first response toward said user via said first modality;

said natural language interpreter outputting a second command toward said back-end application subsystem in response to one of said second token sequences;

said back-end application subsystem providing a second response to said second command; and

said second user interaction module receiving said second response, translating said second response into a form usable by said second I/O modality, and forwarding said second response toward said user via said second modality.

34. A method according to claim 27, further comprising the step of beginning an interpretation session in response to said step of said first user interaction module receiving first user input, said interpretation session identifying said natural language interpreter,

and wherein said step of said first user interaction module forwarding said first token sequences toward said natural language interpreter comprises the step of said first user interaction module routing said first token sequences toward the natural language interpreter identified in said interpretation session.

35. A method according to claim 27, further comprising the step of registering said first user interaction module with an interaction server.

36. A method according to claim 27, further comprising the step of registering said natural language interpreter with an interaction server.

37. A method according to claim 27, wherein said natural language interpreter comprises a distributed parser of natural language input.

38. A method according to claim 27, wherein said natural language interpreter comprises an agent network.

39. A method according to claim 38, wherein said agent network prepares said commands in dependence upon satisfaction, in said token sequences, of one or more policy conditions which make reference to one or more keywords in a predetermined set of keywords.

40. A method according to claim 27, wherein said natural language interpreter comprises:

a plurality of agents arranged in a network, each of said agents having at least one interpretation policy having a policy condition and a policy action associated with the condition, each of said interpretation policies making a claim on at least part of a token sequence provided to the agent if the condition of the interpretation policy is satisfied, and executing the associated policy action in response to determination that the claim is at least part of a winning claim.

41. A method according to claim 40, wherein a particular one of said agents has a particular interpretation policy having a policy condition that refers to a downchain one of said agents, any claim made by said particular policy being dependent upon any claims made by said downchain agent.

42. A method according to claim 40, wherein a particular one of said agents has a particular interpretation policy having a particular policy condition that refers to a downchain one of said agents, said particular interpretation policy further having a particular policy action associated with said particular policy condition,

wherein said particular policy action includes at least partial delegation to said downchain agent.

43. A user interaction method, comprising the steps of:

instantiating a first user interaction module having a user-side interface arranged to communicate with a user via a first I/O modality;

instantiating a second user interaction module having a second user-side interface arranged to communicate with a user via a second I/O modality different from said first I/O modality;

instantiating a natural language interpreter having an input for receiving token sequences from user interaction modules and an output for outputting commands toward a back-end application subsystem in response to said token sequences;

said natural language interpreter issuing a first clarification request in response to a token sequence received from said first user interaction module;

said first user interaction module receiving said first clarification request, translating said first clarification request into a form usable by said first I/O modality, and forwarding said first clarification request toward said user via said first modality;

said natural language interpreter issuing a second clarification request in response to a token sequence received from said second user interaction module; and

said second user interaction module receiving said second clarification request, translating said second clarification request into a form usable by said second I/O modality, and forwarding said second clarification request toward said user via said second modality.

44. A method according to claim 43, further comprising the steps of:

said natural language interpreter outputting a command toward said back-end application subsystem in response to a token sequence received from said first user interaction module;

said back-end application subsystem providing a response to said command; and

said first user interaction module receiving said response, translating said response into a form usable by said first I/O modality, and forwarding said response toward said user via said first modality.

45. A method according to claim 43, further comprising the steps of:

said natural language interpreter outputting a second command toward said back-end application subsystem in response to a token sequence received from said second user interaction module;

said back-end application subsystem providing a second response to said second command; and

said second user interaction module receiving said second response, translating said second response into a form usable by said second I/O modality, and forwarding said second response toward said user via said second modality.

46. A method according to claim 45, further comprising the steps of:

said natural language interpreter outputting a first command toward said back-end application subsystem in response to a token sequence received from said first user interaction module;

said back-end application subsystem providing a first response to said first command;

said first user interaction module receiving said first response, translating said first response into a form usable by said first I/O modality, and forwarding said first response toward said user via said first modality.

47. A method according to claim 43, wherein said natural language interpreter comprises a distributed parser of natural language input.

48. A method according to claim 43, wherein said natural language interpreter comprises an agent network.

49. A method according to claim 48, wherein said agent network prepares said commands in dependence upon satisfaction, in said token sequences, of one or more policy conditions which make reference to one or more keywords in a predetermined set of keywords.

50. A method according to claim 43, wherein said natural language interpreter comprises:

a plurality of agents arranged in a network, each of said agents having at least one interpretation policy having a policy condition and a policy action associated with the condition, each of said interpretation policies making a claim on at least part of a token sequence provided to the agent if the condition of the interpretation policy is satisfied, and executing the associated policy action in response to determination that the claim is at least part of a winning claim.

51. A method according to claim 50, wherein a particular one of said agents has a particular interpretation policy having a policy condition that refers to a downchain one of said agents, any claim made by said particular policy being dependent upon any claims made by said downchain agent.

52. A method according to claim 50, wherein a particular one of said agents has a particular interpretation policy having a particular policy condition that refers to a downchain one of said agents, said particular interpretation policy further having a particular policy action associated with said particular policy condition,

wherein said particular policy action includes at least partial delegation to said downchain agent.

53. A user interaction method, comprising the steps of:

instantiating a first user interaction module having a user-side interface arranged to communicate with a user via a first I/O modality;

instantiating a second user interaction module having a user-side interface arranged to communicate with a user via a second I/O modality different from said first I/O modality; and

instantiating a natural language interpreter having an input for receiving token sequences from user interaction modules and an output for outputting commands toward a back-end application subsystem in response to said token sequences;

said natural language interpreter outputting a command toward said back-end application subsystem in response to one of said token sequences;

said back-end application subsystem providing a response to said command;

said first user interaction module receiving said response, translating said response into a form usable by said first I/O modality and forwarding said response toward said user via said first modality;

said natural language interpreter outputting a second command toward said back-end application subsystem in response to a token sequence received from said second user interaction module;

said back-end application subsystem providing a second response to said second command; and

said second user interaction module receiving said second response, translating said second response into a form usable by said second I/O modality, and forwarding said second response toward said user via said second modality.

54. A method according to claim 53, wherein said natural language interpreter comprises a distributed parser of natural language input.

55. A method according to claim 53, wherein said natural language interpreter comprises an agent network.

56. A method according to claim 55, wherein said agent network prepares said commands in dependence upon satisfaction, in said token sequences, of one or more policy conditions which make reference to one or more keywords in a predetermined set of keywords.

57. A method according to claim 53, wherein said natural language interpreter comprises:

a plurality of agents arranged in a network, each of said agents having at least one interpretation policy having a policy condition and a policy action associated with the condition, each of said interpretation policies making a claim on at least part of a token sequence provided to the agent if the condition of the interpretation policy is satisfied, and executing the associated policy action in response to determination that the claim is at least part of a winning claim.

58. A method according to claim 57, wherein a particular one of said agents has a particular interpretation policy having a policy condition that refers to a downchain one of said agents, any claim made by said particular policy being dependent upon any claims made by said downchain agent.

59. A method according to claim 57, wherein a particular one of said agents has a particular interpretation policy having a particular policy condition that refers to a downchain one of said agents, said particular interpretation policy further having a particular policy action associated with said particular policy condition,

wherein said particular policy action includes at least partial delegation to said downchain agent.