Using a community of distributed electronic agents to support a highly mobile, ambient computing environment

• US 7,036,128 B1
• Filed: 08/09/2000
• Issued: 04/25/2006
• Est. Priority Date: 01/05/1999
• Status: Expired due to Term

• Current Assignee: IPA Technologies Inc. (Quarterhill Inc. (f/k/a Wi-LAN Inc.))
• Original Assignee: SRI International, Inc.
• Inventors: Julia, Luc, Cheyer, Adam
• Primary Examiner(s): BULLOCK JR, LEWIS ALEXANDER
• Application Number:
  US09/475,092
• Time in Patent Office: 2,085 Days
• Field of Search: 709/202, 719/317
• US Class Current: 719/317
• CPC Class Codes: G06F 16/3334 Selection or weighting of terms from queries, including natural language queries
• CPC Class Codes: G06F 16/90332 Natural language query formulation or dialogue systems
• CPC Class Codes: G06F 9/465 Distributed object oriented systems remote method invocation [RMI] G06F9/548
• CPC Class Codes: G06N 5/043 Distributed expert systems; Blackboards

Abstract

A highly mobile, ambient computing environment is disclosed for serving a knowledge worker away from the their desk. The present invention allows a knowledge worker to get increased leverage from personal, networked, and interactive computing devices while in their car, airplane seat, or in a conference room with others. An Open Agent Architecture is used to incorporate elements such as GPS agents, speech recognition, and opportunistic connectivity among meeting participants. Communication and cooperation between agents are brokered by one or more facilitators, which are responsible for matching requests, from users and agents, with descriptions of the capabilities of other agents. It is not generally required that a user or agent know the identities, locations, or number of other agents involved in satisfying a request, and relatively minimal effort is involved in incorporating new agents and “wrapping” legacy applications. Extreme flexibility is achieved through an architecture organized around the declaration of capabilities by service-providing agents, the construction of arbitrarily complex goals by users and service-requesting agents, and the role of facilitators in delegating and coordinating the satisfaction of these goals, subject to advice and constraints that may accompany them.

Claims

1. A collaborative computer-implemented community of distributed electronic agents, organized to provide a mobile computing environment, the computer-implemented community of distributed electronic agents comprising:

an agent registry wherein one or more capabilities of each of the electronic agents are registered in the form of an interagent communication language (ICL), wherein the interagent language includes a layer of conversational protocol defined by event types and parameter lists associated with one or more events, and wherein the parameter lists further refine the one or more events;

a facilitator agent arranged to coordinate cooperative task completion among the electronic agents by delegating one or more received ICL goals to a selected one or more of the electronic agents based upon the registered capabilities of the selected agents;

one or more service-providing electronic agents, being in bi-directional communication with the facilitator agent, including at least one location agent operable to ascertain a current physical location of a user; and

one or more computer interface agents being in bi-directional communication with the facilitator agent, the mobile computer interface agents being operable to process at least one mobile user input type and to responsively generate and present to the facilitator agent one or more ICL goals corresponding to the user'"'"'s desired request.

2. The computer-implemented community of agents as recited in claim 1, wherein the mobile computer interface agents and the location agent are collaboratively operable to provide map information relative to the user'"'"'s current location.

3. The computer-implemented community of agents as recited in claim 2, wherein the map information includes a Multimodal Map displaying site geographic location information based on site type.

4. The computer-implemented community of agents as recited in claim 2, wherein the map information includes spoken directions by means of a text-to-speech output agent.

5. The computer-implemented community of agents as recited in claim 1, further including one or more conditional triggers associated with one or more of the electronic service agents, and wherein the community is further collaboratively operable to proactively interact with the user in response to said triggers.

6. The computer-implemented community of agents as recited in claim 5, wherein said proactive interaction is based in part on the user'"'"'s physical location derived by the location agent.

9. The computer-implemented community of agents as recited in claim 1, wherein the mobile computer interface agents comprise a communication center panel including one or more communication facilities from the following group:

an email interface, a telephony interface, and

a voice mail interface.

10. The computer-implemented community of agents as recited in claim 9, wherein the email interface is responsive to voice command input, and wherein the email interface delivers email utilizing cross-media conversion.

11. The computer-implemented community of agents as recited in claim 9, wherein the telephony interface is responsive to dialing by voice command input.

12. The computer-implemented community of agents as recited in claim 9, wherein the voicemail interface is responsive to voice commands, and wherein the voice commands include decision branching commands.

13. The computer-implemented community of agents as recited in claim 1, wherein the mobile computer interface agents comprise a recreation center interface including:

a game interface; and

an Internet interface.

14. The computer-implemented community of agents as recited in claim 13, wherein the game interface provides access to 3D games, and wherein the game interface is responsive to voice commands.

15. The computer-implemented community of agents as recited in claim 13, wherein the Internet interface is responsive to voice command input.

16. The computer-implemented community of agents as recited in claim 13, wherein the recreation interface further comprises a television interface, wherein the television interface is responsive to voice commands.

17. The computer-implemented community of agents as recited in claim 1, wherein the mobile computer interface agents comprise a technical information center including:

a car manual interface; and

a diagnostic interface.

18. The computer-implemented community of agents as recited in claim 17, wherein the car manual interface provides access to car information utilizing the Internet.

19. The computer-implemented community of agents as recited in claim 17, wherein the diagnostic interface includes a virtual automobile expert responsive to voice commands.

20. The computer-implemented community of agents as recited in claim 1, wherein the at least one mobile user input type is selected from the following group:

speech, gestures, pen/stylus input, and touch-screen input.

21. The computer-implemented community of agents as recited in claim 1, wherein the mobile computing environment is provided in a handheld computing device.

7. The computer-implemented community of agents as recited in 1, wherein the mobile computer interface agents comprise an electronic device control panel to control one or more automotive electronic devices.

8. The computer-implemented community of agents as recited in claim 7, wherein the electronic devices include one or more devices from the following group:

a sound system, a cruise control, and lights.

22. A method for providing a mobile, ambient computing environment utilizing a community of distributed electronic agents, the community of agents including one or more user interface agents and at least one location agent providing information as to a current physical location of a user, the method comprising the acts of:

(a) registering one or more capabilities for each of the electronic agents in an interagent communication language (ICL), wherein the interagent language includes a layer of conversational protocol defined by event types and parameter lists associated with one or more events, and wherein the parameter lists further refine the one or more events;

(b) receiving one or more user input requests presented in one or more mobile input types;

(c) using the one or more user interface agents to interpret said input request and generate a corresponding goal formulated in ICL;

(d) using a facilitator agent to delegate the ICL goal, in the form of one or more ICL sub-goals, to a selected one or more of the electronic agents based upon the registered capabilities of the agents;

(e) using the selected electronic agents to perform the delegated ICL sub-goals;

(f) in course of said performance by the selected electronic agents, generating one or more new ICL goals for processing by the facilitator agent in accordance with step (d); and

(g) iteratively performing the process of steps (d)–

(f) until the original ICL goal is satisfied, wherein one or more of the ICL sub-goals or the new ICL goals requires user location information provided by the location agent.

23. A method as recited in claim 22, wherein the method further includes the steps of collaboratively using the location agent an the user interface agents to provide navigational information to the user.

24. A method as recited in claim 23, wherein the navigational information includes a Multimodal Map displaying site geographic location information based on site type.

25. A method as recited in claim 23, wherein the navigational information includes route information recited by a text-to-speech agent.

26. A method as recited in claim 22, further including the steps of setting one or more conditional triggers associated with one or more of the electronic agents, and proactively interacting with a user based in part on the conditional triggers and on the user'"'"'s physical location.

29. A method as recited in claim 22, wherein the user interface agents provide a communication center panel including one or more communication facilities from the following group:

an email interface;

a telephony interface; and

a voice mail interface.

30. A method as recited in claim 29, wherein the email interface is responsive to voice command input, and wherein the email interface delivers email utilizing cross-media conversion.

31. A method as recited in claim 29, wherein the telephony interface is responsive to dialing by voice command input.

32. A method as recited in claim 29, wherein the voicemail interface is responsive to voice commands, and wherein the voice commands include decision branching commands.

33. A method as recited in claim 22, wherein the user interface agents provide a recreation center interface including:

a game interface; and

an Internet interface.

34. A method as recited in claim 33, wherein the game interface provides access to 3D games, and wherein the game interface is responsive to voice commands.

35. A method as recited in claim 33, wherein the Internet interface is responsive to voice command input.

36. A method as recited in claim 33, wherein the recreation interface further comprises a television interface, wherein the television interface is responsive to voice commands.

37. A method as recited in claim 22, wherein the user interface agents provide a technical information center including:

a car manual interface; and

a diagnostic interface.

38. A method as recited in claim 37, wherein the car manual interface provides access to car information utilizing the Internet.

39. A method as recited in claim 37, wherein the diagnostic interface includes a virtual automobile expert responsive to voice commands.

40. A method as recited in claim 22, wherein the mobile user input types include speech, gestures, pen/stylus input, and touch-screen input.

27. A method as recited in 22, wherein the user interface agents provide an electronic device control panel to control one or more electronic devices.

28. A method as recited in claim 27, wherein the one or more electronic devices are selected from the following group:

a sound system, a cruise control, and lights.

41. A method for proactively providing a mobile user with location-sensitive navigational information utilizing a community of distributed electronic agents, the method comprising the acts of:

registering one or more capabilities of the electronic agents in an interagent communication language wherein the interagent language includes a layer of conversational protocol defined by event types and parameter lists associated with one or more events, and wherein the parameter lists further refine the one or more events;

setting an automatic software trigger, the trigger being conditioned upon detecting a specified condition;

detecting the specified condition;

using an electronic location agent to ascertain the mobile user'"'"'s current location; and

in response to the software trigger, using one or more interface agents to present navigational information to the user relative to the user'"'"'s current location.

42. The method of claim 41, wherein the navigational information includes one or more of the following:

visual display of a map and spoken directions.

43. The method of claim 41, wherein the specified trigger condition includes the status of a vehicle sensor.

44. The method of claim 41, wherein the specified trigger condition includes the contents of an electronic calendar.

45. A computer-implemented highly mobile, ambient computing environment utilizing a community of distributed electronic agents, the computer environment comprising:

a plurality of autonomous service-providing electronic agents associated with available resources, wherein one or more capabilities of the service-providing electronic agents are registered in the form of an interagent communication language and wherein the interagent language includes a layer of conversational protocol defined by event types and parameter lists associated with one or more events, and wherein the parameter lists further refine the one or more events;

a facilitator agent arranged to coordinate cooperative task completion utilizing the plurality of autonomous service-providing electronic agents; and

a mobile computer interface responsive to a plurality of user input types, the mobile computer interface being in bi-directional communication with the facilitator agent, the mobile computer interface operable to forward a user request for resource access to the facilitator agent for processing, the mobile computer interface further operable to provide the user the requested resource access as provided by the facilitator agent,

whereby the mobile user is capable of accessing both local and remote resources.