System and method for using context in navigation dialog

• US 7,831,433 B1
• Filed: 02/09/2007
• Issued: 11/09/2010
• Est. Priority Date: 02/03/2005
• Status: Expired due to Fees

• Current Assignee: HRL Laboratories LLC (The Boeing Co.)
• Original Assignee: HRL Laboratories LLC (The Boeing Co.)
• Inventors: Hein, Cheryl, Daily, Michael, Belvin, Robert, Martin, Kevin R., Lee, Craig A., Srinivasa, Narayan
• Primary Examiner(s): MCFADDEN, SUSAN IRIS
• Application Number:
  US11/704,790
• Time in Patent Office: 1,369 Days
• Field of Search: 704/275
• US Class Current: 704/275
• CPC Class Codes: G10L 13/00 Speech synthesis; Text to speech systems
• CPC Class Codes: G10L 15/18 using natural language modelling

Abstract

Described is a navigation system. The navigation system comprises a route planning module and a route guidance module. The route planning module is configured to receive a request from a user for guidance to a particular destination. Based on a starting point, the route planning module determines a route from the starting point to the particular destination. The route guidance module is configured to receive the route, and based on the route and current location of the user, provide location-specific instructions to the user. The location-specific instructions include reference to specific visible objects within the vicinity of the user.

Claims

1. A navigation system, comprising:

a route planning module configured to receive a request from a user for guidance to a particular destination, and based on a starting point, determining a route from the starting point to the particular destination; and

a route guidance module configured to receive the route from the route planning module, and based on the route and current location of the user, provide location-specific instructions to the user, where the location-specific instructions include reference to specific visible objects within the vicinity of the user; and

wherein the route guidance module further comprises a vision module, the vision module being configured to receive a plurality of image frames and detect and track objects proximate to the navigation system that are represented in the image frames, such that each object is assigned a track identifier and is then tracked in subsequent image frames.

2. A navigation system as set forth in claim 1, wherein the route planning module further comprises a route planning dialog manager, where the route planning dialog manager is configured to receive an interpreted input utterance from a speech understanding system requesting guidance to a particular destination, and based on the starting point, the input utterance, and the particular destination, the route planning dialog manager generates a starting and stopping point on the user'"'"'s route, and once generated, confirms with the user that the generated starting and stopping points are correct starting and stopping points.

3. A navigation system as set forth in claim 2, wherein the route planning module further comprises a navigation route server, wherein the navigation route server is configured to receive the starting and stopping points, and based on the starting and stopping points, generate a route with directional instructions from the starting point to the stopping point.

4. A navigation system as set forth in claim 3, where the route guidance module further comprises a vehicle position and heading module, wherein the vehicle position and heading module is configured to determine a specific location of the navigation system.

5. A navigation system as set forth in claim 4, wherein the route guidance module further comprises a context monitor, the context monitor being configured to receive and combine information pertaining to the route, the specific location of the navigation system, and objects being detected and tracked that are proximate to the navigation system, and based on the combination of the information, the context monitor is configured to generate situational data, the situational data being a continuously updated status of the navigation system and its current local situation.

6. A navigation system as set forth in claim 4, wherein the route guidance module further comprises a local object blackboard, where the local object blackboard is configured to receive the track identifier and maintain a record of track identifiers that are currently proximate to the navigation system, otherwise deleting the track identifiers.

7. A navigation system as set forth in claim 4, wherein the route guidance module further comprises a local reference matcher, the local reference matcher being configured to receive the track identifier, the specific location of the navigation system, and user specific instructions regarding objects proximate to the navigation system, and further configured to match the user specific instructions regarding objects proximate to the navigation system with a specific track identifier.

8. A navigation system as set forth in claim 5, wherein the route guidance module further comprises a route guidance dialog manager, where the route guidance dialog manager is configured to receive a combination of information consisting of an interpreted input utterance from the speech understanding system, the route, and the situational data, and based on the combination of information, generating a phrase selected from a group consisting of the phrase being generated in response to a reference to a specific visible object within the vicinity of the user, and the phrase including a reference to a specific visible object within the vicinity of the user.

9. A navigation system as set forth in claim 8, further comprising a speech generator, the speech generator being configured to receive the phrase from the route guidance dialog manager and convert the phrase into an audible, speech signal.

10. A navigation system as set forth in claim 1, further comprising an automobile with the route planning module and route guidance module affixed thereto, thereby creating a driver navigation system.

11. A navigation system as set forth in claim 1, further comprising a speech understanding system for receiving an input speech utterance from a user and converting the input speech utterance into an interpreted input utterance, the interpreted input utterance consisting of a semantically labeled phrase.

12. A computer implemented method for forming a navigation system, comprising:

an act of encoding a memory of a processor with instruction means to include a route planning module and a route guidance module,

wherein the route planning module is configured to receive a request from a user for guidance to a particular destination, and based on a starting point, determining a route from the starting point to the particular destination; and

wherein the route guidance module configured to receive the route from the route planning module, and based on the route and current location of the user, provide location-specific instructions to the user, where the location-specific instructions include reference to specific visible objects within the vicinity of the user; and

wherein the route guidance module is further configured to include a vision module and to receive a plurality of image frames and detect and track objects proximate to the navigation system that are represented in the image frames, such that each object is assigned a track identifier and is then tracked in subsequent image frames.

13. A method for forming a navigation system as set forth in claim 12, wherein the act of configuring the route planning module further comprises an act of configuring a route planning dialog manager to receive an interpreted input utterance from a speech understanding system requesting guidance to a particular destination, and based on the starting point, the input utterance, and the particular destination, the route planning dialog manager generates a starting and stopping point on the user'"'"'s route, and once generated, confirms with the user that the generated starting and stopping points are correct starting and stopping points.

14. A method for forming a navigation system as set forth in claim 13, wherein the act of configuring the route planning module further comprises an act of configuring a navigation route server to receive the starting and stopping points, and based on the starting and stopping points, generate a route with directional instructions from the starting point to the stopping point.

15. A method for forming a navigation system as set forth in claim 14, where the act of configuring the route guidance module further comprises an act of configuring a vehicle position and heading module to determine a specific location of the navigation system.

16. A method for forming a navigation system as set forth in claim 15, wherein the act of configuring the route guidance module further comprises an act of configuring a context monitor to receive and combine information pertaining to the route, the specific location of the navigation system, and objects being detected and tracked that are proximate to the navigation system, and based on the combination of the information, the context monitor is configured to generate situational data, the situational data being a continuously updated status of the navigation system and its current local situation.

17. A method for forming a navigation system as set forth in claim 15, wherein the act of configuring the route guidance module further comprises an act of configuring a local object blackboard to receive the track identifier and maintain a record of track identifiers that are currently proximate to the navigation system, otherwise deleting the track identifiers.

18. A method for forming a navigation system as set forth in claim 15, wherein the act of configuring the route guidance module further comprises an act of configuring a local reference matcher to receive the track identifier, the specific location of the navigation system, and user specific instructions regarding objects proximate to the navigation system, and further configured to match the user specific instructions regarding objects proximate to the navigation system with a specific track identifier.

19. A method for forming a navigation system as set forth in claim 16, wherein the act of configuring the route guidance module further comprises an act of configuring a route guidance dialog manager to receive a combination of information consisting of an interpreted input utterance from the speech understanding system, the route, and the situational data, and based on the combination of information, generating a phrase selected from a group consisting of the phrase being generated in response to a reference to a specific visible object within the vicinity of the user, and the phrase including a reference to a specific visible object within the vicinity of the user.

20. A method for forming a navigation system as set forth in claim 19, further comprising an act of configuring a speech generator to receive the phrase from the route guidance dialog manager and convert the phrase into an audible, speech signal.

21. A method for forming a navigation system as set forth in claim 12, further comprising an act of configuring a speech understanding system to receive an input speech utterance from a user and converting the input speech utterance into an interpreted input utterance, the interpreted input utterance consisting of a semantically labeled phrase.

22. A method for forming a navigation system as set forth in claim 12, further comprising an act of affixing the route planning module and route guidance module to an automobile, thereby creating a driver navigation system.

23. A computer program product for navigating, the computer program product comprising computer-readable instruction means stored on a non-transitory computer-readable medium that are executable by a computer for causing the computer to:

receive a request from a user for guidance to a particular destination, and based on a starting point, determine a route from the starting point to the particular destination; and

based on the route and current location of the user, provide location-specific instructions to the user, where the location-specific instructions include reference to specific visible objects within the vicinity of the user; and

receive a plurality of image frames and detect and track objects proximate to the computer that are represented in the image frames, such that each object is assigned a track identifier and is then tracked in subsequent image frames.

24. A computer program product as set forth in claim 23, further comprising instruction means for causing the computer to receive an interpreted input utterance requesting guidance to a particular destination, and based on the starting point, the input utterance, and the particular destination, generate a starting and stopping point on the user'"'"'s route, and once generated, confirms with the user that the generated starting and stopping points are correct starting and stopping points.

25. A computer program product as set forth in claim 24, further comprising instruction means for causing the computer to receive the starting and stopping points, and based on the starting and stopping points, generate a route with directional instructions from the starting point to the stopping point.

26. A computer program product as set forth in claim 25, further comprising instruction means for causing the computer to determine a specific location of the navigation system.

27. A computer program product as set forth in claim 26, further comprising instruction means for causing the computer to receive and combine information pertaining to the route, the specific location of the navigation system, and objects being detected and tracked that are proximate to the computer, and based on the combination of the information, the generating situational data, the situational data being a continuously updated status of the computer and its current local situation.

28. A computer program product as set forth in claim 26, further comprising instruction means for causing the computer to receive the track identifier and maintain a record of track identifiers that are currently proximate to the computer, otherwise deleting the track identifiers.

29. A computer program product as set forth in claim 26, further comprising instruction means for causing the computer to receive the track identifier, the specific location of the computer, and user specific instructions regarding objects proximate to the computer, and further comprising instruction means for causing the computer to match the user specific instructions regarding objects proximate to the computer with a specific track identifier.

30. A computer program product as set forth in claim 27, further comprising instruction means for causing the computer to receive a combination of information consisting of an interpreted input utterance, the route, and the situational data, and based on the combination of information, generating a phrase selected from a group consisting of the phrase being generated in response to a reference to a specific visible object within the vicinity of the user, and the phrase including a reference to a specific visible object within the vicinity of the user.

31. A computer program product as set forth in claim 30, further comprising instruction means for causing the computer to convert the phrase into an audible, speech signal.

32. A computer program product as set forth in claim 23, further comprising instruction means for causing the computer to receive an input speech utterance from a user and convert the input speech utterance into an interpreted input utterance, the interpreted input utterance consisting of a semantically labeled phrase.

33. A navigation system, comprising:

a route planning module configured to receive a request from a user for guidance to a particular destination, and based on a starting point, determining a route from the starting point to the particular destination;

a route guidance module configured to receive the route from the route planning module, and based on the route and current location of the user, provide location-specific instructions to the user, where the location-specific instructions include reference to specific visible objects within the vicinity of the user;

a speech understanding system for receiving an input speech utterance from a user and converting the input speech utterance into an interpreted input utterance, the interpreted input utterance consisting of a semantically labeled phrase;

wherein the route planning module further comprises a route planning dialog manager, where the route planning dialog manager is configured to receive an interpreted input utterance from a speech understanding system requesting guidance to a particular destination, and based on the starting point, the input utterance, and the particular destination, the route planning dialog manager generates a starting and stopping point on the user'"'"'s route, and once generated, confirms with the user that the generated starting and stopping points are correct starting and stopping points;

wherein the route planning module further comprises a navigation route server, wherein the navigation route server is configured to receive the starting and stopping points, and based on the starting and stopping points, generate a route with directional instructions from the starting point to the stopping point;

wherein the route guidance module further comprises a vehicle position and heading module, wherein the vehicle position and heading module is configured to determine a specific location of the navigation system;

wherein the route guidance module further comprises a vision module, the vision module being configured to receive a plurality of image frames and detect and track objects proximate to the navigation system that are represented in the image frames, such that each object is assigned a track identifier and is then tracked in subsequent image frames;

wherein the route guidance module further comprises a context monitor, the context monitor being configured to receive and combine information pertaining to the route, the specific location of the navigation system, and objects being detected and tracked that are proximate to the navigation system, and based on the combination of the information, the context monitor is configured to generate situational data, the situational data being a continuously updated status of the navigation system and its current local situation;

wherein the route guidance module further comprises a local object blackboard, where the local object blackboard is configured to receive the track identifier and maintain a record of track identifiers that are currently proximate to the navigation system, otherwise deleting the track identifiers;

wherein the route guidance module further comprises a local reference matcher, the local reference matcher being configured to receive the track identifier, the specific location of the navigation system, and user specific instructions regarding objects proximate to the navigation system, and further configured to match the user specific instructions regarding objects proximate to the navigation system with a specific track identifier;

wherein the route guidance module further comprises a route guidance dialog manager, where the route guidance dialog manager is configured to receive a combination of information consisting of an interpreted input utterance from the speech understanding system, the route, and the situational data, and based on the combination of information, generating a phrase selected from a group consisting of the phrase being generated in response to a reference to a specific visible object within the vicinity of the user, and the phrase including a reference to a specific visible object within the vicinity of the user;

a speech generator, the speech generator being configured to receive the phrase from the route guidance dialog manager and convert the phrase into an audible, speech signal; and

an automobile with the route planning module, route guidance module, speech understanding system, and speech generator affixed thereto, thereby creating a driver navigation system.