Adaptive mobile robot system with knowledge-driven architecture

• US 7,966,093 B2
• Filed: 12/27/2007
• Issued: 06/21/2011
• Est. Priority Date: 04/17/2007
• Status: Expired due to Fees

• Current Assignee: Yefim Zhuk
• Original Assignee: Yefim Zhuk
• Inventors: Zhuk, Yefim
• Primary Examiner(s): Tran, Khoi
• Assistant Examiner(s): Sample, Jonathan L
• Application Number:
  US11/965,063
• Publication Number:
  US 20090187278A1
• Time in Patent Office: 1,272 Days
• Field of Search: None
• US Class Current: 700/246
• CPC Class Codes: G05D 1/0088 characterized by the autonomous decision making process, e.g. artificial intelligence, predefined behaviours using knowledge based models G06N5/00
• CPC Class Codes: G05D 1/0246 using a video camera in combination with image processing means
• CPC Class Codes: G05D 1/0278 using satellite positioning signals, e.g. GPS
• CPC Class Codes: G05D 1/0291 Fleet control monitoring fleets in traffic control systems for road vehicles G08G1/127, G08G1/127
• CPC Class Codes: G06N 3/008 based on physical entities controlled by simulated intelligence so as to replicate intelligent life forms, e.g. based on robots replicating pets or humans in their appearance or behaviour

Abstract

The invention integrates software and knowledge engineering with robotics technology to improve robot-to-robot and robot-to-human conversational interface and provide on-the-fly translations of situational requirements into adaptive behavior models and further down to service scenarios for a collaborative robot teams.

Claims

1. An adaptive mobile robot system with knowledge-driven architecture in communications with subject matter experts that describe the situation and assign robot tasks with situational requirements and comprising:

Distributed specialized knowledge nodes that store adaptive behavior models related to specific knowledge domains and provide components of these models as situational service scenarios with rule-based service invocations, while participating in transformation of new situational requirements into situational service scenarios;

at least one robot team with at least one robot that includes;

a) A plurality of robot sensors that produce sensor data;

b) A plurality of robot controllers that via service invocations define robot'"'"'s behavior;

c) Service Orchestration Engine connected to robot controllers and transforming service orchestration scenarios into rule-based service invocations, which can be executed by robot controllers;

d) Specialized knowledge node (SKN) that interacts internally with Service Orchestration Engine within the robot and externally with distributed specialized knowledge nodes and subject matter experts to transform incoming situational requirements, information from other specialized knowledge nodes and robot sensor data into service orchestration scenarios that define robot'"'"'s behavior models, wherein SKN includes;

(i) Service Dictionary describes and stores adaptive behavior models for each robot as a dynamically changeable set of services, service orchestrations that consist of composite services with a plurality of rule-based service invocations, and situational service scenarios that consist of plurality of composite services assembled into groups connected by conditional rules, which include sensor data describing situational components, where each group represents a component of robot'"'"'s behavior in a specific situation;

(ii) Natural Language Interpreter interprets natural language based information coming from subject matter experts into a map of knowledge topics, subjects related to a described situation called in the future conversational subjects;

(iii) Situational Scenario Interpreter working in collaboration with the Natural Language Interpreter to transform-natural language based situational description and changeable situational task requirements into situational service scenarios that represent adaptive behavior models executed by robots;

(iv) Specialized Data Interpreter interprets plurality of special data formats including XML-based service descriptions, mechanical and video sensor data and transforms them into components of service orchestration scenarios;

(v) Conversation Manager initiates and maintains collaborative conversations between robots, specialized knowledge nodes and subject matter experts, while requesting existing components of situational service scenarios from distributed specialized knowledge nodes and additional information, corrections and clarifications of situational requirements from subject matter experts while transforming situational requirements into behavior models defined with situational service scenarios;

(vi) Knowledge Bus Manager distinguishes different types of data incoming to the common bus from internal and external sources, uses publish-subscribe mechanism to publish this information in message queues providing sequence and type identification to each data source, and makes this data available to distributed knowledge nodes and subject matter experts as well as internally to components-subscribers within SKN including Situational Scenario Interpreter, Natural Language Interpreter, Special Data Interpreter, Service Orchestration Engine, Service Dictionary and Conversational Manager.

2. The adaptive mobile robot system with knowledge-driven architecture of claim 1, wherein

The Specialized Knowledge Node translates multiple types of incoming data into xml-based service scenarios and the Scenario Orchestration Engine transforms service scenarios into rule-based service invocations thus dynamically defining adaptive behavior models on a plurality, of robot controllers.

3. The adaptive mobile robot system with knowledge-driven architecture of claim 1, wherein

Conversation Manager via the Knowledge Manager Bus initiates and maintains multi-directional data exchange-conversations between robots, distributed knowledge nodes, and subject matter experts, while Situational Scenario Interpreter working in collaboration with Natural Language Interpreter, Special Data Interpreter and Service Orchestration Engine are resolving new situational requirements into a well formatted situational service scenario with service orchestrations filled with rule-based service invocations.

4. The adaptive mobile robot system with knowledge-driven architecture of claim 1, wherein Specialized Data Interpreter, Natural Language Interpreter, and Situational Scenario interpreter, parse incoming data to transform them into situational service scenario formats and report their results to the Conversation Manager, which, upon receiving the failure of the attempt to transform situational requirements into service scenarios, starts another conversational cycle by generating a proper question sent to a proper channel, including internal and distributed knowledge nodes and subject matter experts, otherwise, upon receiving the success of transformation, the Conversation Manager ends the conversation and updates Service Dictionary by assembling successful results into service scenarios and knowledge rules.

5. The adaptive mobile robot system with knowledge-driven architecture of claim 2, where Service Dictionary, stores service orchestrations and situational scenarios that represent dynamically changeable behavior models as a set of skills for a robot and a specialized distributed knowledge node.

6. The adaptive mobile robot system with knowledge-driven architecture of claim 2, further comprising Knowledgebase, which includes pre-defined fundamental concepts, generic facts and rules and enables knowledge update.

7. The adaptive mobile robot system with knowledge-driven architecture of claim 3, wherein Knowledge Bus Manager

supports publish and subscribe mechanisms for multiple knowledge topics and types of data for external and internal communications;

is connected to messaging clients that are subscribed to this information including;

Natural Language Interpreter is subscribed to situational requirements coming from subject matter experts in natural language,

Situational Scenario Interpreter is subscribed to situational requirements and to the related conversational subjects provided by the Natural Language Interpreter,

Specialized Data Interpreter is subscribed to data in special formats including sensor data and XML-based information,

Service Orchestration Engine is subscribed to situational service scenarios and their components provided by internal Service Dictionary and coming from other robots and specialized knowledge nodes,

Internal Service Dictionary and Distributed Specialized Knowledge nodes are subscribed to the requests for information from the Conversation Manager including requests for existing behavior models defined as situational service scenarios,

Subject Matter Experts are subscribed to the requests from the Conversation Manager for clarification of situational requirements,

Conversational Manager is subscribed to reports on success or failure of the interpretation cycle from Situational Scenario Interpreter and Service Orchestration Engine and to conversational subjects provided by the Natural Language Interpreter and upon receiving the conversational subjects the Conversation Manager initiates an iterative conversation cycle requesting information from internal components including Service Dictionary and robot'"'"'s sensors and externally from other robots, distributed knowledge nodes and subject matter experts, where response data are coming via the Knowledge Bus Manager for interpretation, and if the information is not sufficient to complete a situational scenario in the current iteration, the Conversation Manager uses the response from the Situational Scenario Interpreter to form and send to Knowledge Bus Manager a new request that will start another conversational cycle with robots, distributed specialized knowledge nodes, and subject matter experts, requesting for more information from the following sources including specialized knowledge nodes with more precise and better formatted information, subject matter experts with more formal requirements, specific sensor of a specific robot in a specific team with additional information obtained at run-time until situational requirements can be completely resolved into an executable scenario or subject matter experts cancel the iteration.

8. The adaptive mobile robot system with knowledge-driven architecture of claim 2, wherein built-in robot knowledge node participates in a collaborative action by multiple robots and distributed specialized knowledge nodes and is assigned at run-time to play a specific role in this collaborative action.

9. The adaptive mobile robot system with knowledge-driven architecture of claim 1, wherein Distributed Specialized Knowledge Nodes respond to the situational requirements when requirements match knowledge topics registered in their Knowledge Bus Managers.

10. The adaptive mobile robot system with knowledge-driven architecture of claim 7, wherein Manager evaluate situational requirements and publish its criteria indicators and matching service scenarios to the network.

11. The adaptive mobile robot system with knowledge-driven architecture of claim 10, wherein

The Knowledge Bus Manager with the highest score of service scenarios matching to incoming situational requirements, registers itself as the team coordinator for the request, and, when task requires more skills, services and efforts that can be provided by the selected coordinator, the coordinator will select and register more participants for the task based on their published criteria indicators and service scenarios.

12. The adaptive mobile robot system with knowledge-driven architecture of claim 6, wherein distributed knowledge node, further comprising:

the Inference Engine, which checks for Knowledgebase integrity with every information update;

the Inference Engine looks for possible conflicts between an existing set of rules and definitions and an incoming new set, while resolving situational requirements and situational data;

Knowledgebase and Inference Engine reports to the Conversation Manager success or failure such an update, and in the case of failure the Conversation Manager starts another conversation cycle by requesting for additional information.

13. The adaptive mobile robot system with knowledge-driven architecture of claim 3, wherein Conversational Manager registers initial information of situational requirements and converse with multiple parties, including robots, knowledge nodes and subject matter experts to resolve this information into instructional scenario executable by the system.

14. The adaptive mobile robot system with knowledge-driven architecture of claim 3, wherein Conversational Manager arranges data exchange between subject matter experts, Natural Language Interpreter, Service Dictionary and Knowledgebase, to transform situational requirements into situational service scenarios with specific rules added to the Knowledgebase and built-in and referenced by the resulting scenarios.

15. The adaptive mobile robot system with knowledge-driven architecture of claim 3, wherein Conversational Manager stores the resulting situational scenario and service orchestrations in the Service Dictionary dynamically changing the set of skills and behavior models of including but not limited to a particular robot and a specialized knowledge node.

16. The adaptive mobile robot system with knowledge-driven architecture of claim 4, wherein Situational Scenario Interpreter contacts the Service Dictionary and distributed knowledge nodes to check if any parts of the scenario can be resolved via existing scenarios stored in the internal Service Dictionary, as well as a Service Dictionary of one of distributed knowledge nodes.

17. The adaptive mobile robot system with knowledge-driven architecture of claim 4, wherein Specialized Data Interpreter interprets special data types, including robot'"'"'s mechanical and video sensors.

18. The adaptive mobile robot system with knowledge-driven architecture of claim 7, wherein Situational Scenario Interpreter receives incoming information to create formatted service scenarios, which consist of service invocations and business logics provided by references to existing Knowledgebase rules and built-in scenario rules based on existing knowledgebase rules expressed in close to natural language format, including associated noun-constants and verb-predicates.

19. The adaptive mobile robot system with knowledge-driven architecture of claim 7, wherein Situational Scenario Interpreter reports completed scenario to the Conversational Manager and further to the Knowledge Bus Manager which stores a new scenario in the Service Dictionary and sends it for execution to the Service Orchestration Interpreter of the current distributed node and in the case when the team responding to the task consists from more than one member, the scenario is sent to node selected by the team coordinator'"'"'s Knowledge Bus Manager.

20. The adaptive mobile robot system with knowledge-driven architecture of claim 2, wherein Service Orchestration Interpreter makes the final transformation of the service scenarios into service invocations of robot controllers using the Service Dictionary where operations on robot sensors, drivers, and controlled elements are stored as services and service scenarios.

21. The adaptive mobile robot system with knowledge-driven architecture of claim 7, wherein Knowledge Bus Manager can receive new data from robot sensors and other sources during the execution of a scenario and arrange a new cycle of transformation of situational information into another executable scenario.