MULTIMODAL NATURAL LANGUAGE QUERY SYSTEM FOR PROCESSING AND ANALYZING VOICE AND PROXIMITY-BASED QUERIES

• US 20080162471A1
• Filed: 03/14/2008
• Published: 07/03/2008
• Est. Priority Date: 01/24/2005
• Status: Active Grant

• Current Assignee: Portal Communications, LLC (Dominion Harbor Enterprises, LLC)
• Original Assignee: The Intellection Group, Inc.
• Inventors: Bernard, David E.
• Granted Patent: US 7,873,654 B2
• Time in Patent Office: Days
• US Class Current: 707/5
• CPC Class Codes: G06F 16/3329 Natural language query formulation or dialogue systems
• CPC Class Codes: G06F 16/40 of multimedia data, e.g. slideshows comprising image and additional audio data retrieval of still image data G06F16/50; retrieval of audio data G06F16/60; retrieval of video data G06F16/70
• CPC Class Codes: G06F 40/30 Semantic analysis
• CPC Class Codes: G10L 15/1815 Semantic context, e.g. disambiguation of the recognition hypotheses based on word meaning
• CPC Class Codes: G10L 15/26 Speech to text systems G10L15/08 takes precedence

Abstract

The present invention provides a natural language query system and method for processing and analyzing multimodally-originated queries, including voice and proximity-based queries. The natural language query system includes a Web-enabled device including a speech input module for receiving a voice-based query in natural language form from a user and a location/proximity module for receiving location/proximity information from a location/proximity device. The query system also includes a speech conversion module for converting the voice-based query in natural language form to text in natural language form and a natural language processing module for converting the text in natural language form to text in searchable form. The query system further includes a semantic engine module for converting the text in searchable form to a formal database query and a database-look-up module for using the formal database query to obtain a result related to the voice-based query in natural language form from a database.

Claims

1. A natural language query system for processing and analyzing voice and proximity-based queries, comprising:

a device, comprising;

a speech input module for receiving a voice-based query in natural language form from a user; and

a location/proximity module for receiving location/proximity information from a location/proximity device;

one or more servers connected to the device, wherein the one or more servers comprise;

a speech conversion module for converting the voice-based query in natural language form to text in natural language form;

a natural language processing module for converting the text in natural language form to text in searchable form using lexicons and grammar rules to parse sentences and determine underlying meanings of the voice-base query;

a semantic engine module for converting the text in searchable form to a formal database query; and

a database-look-up module for using the formal database query to obtain a result related to the voice-based query in natural language form from a database, wherein the location/proximity module is configured to provide a context and environment to the database-look-up module for narrowing and streamlining the formal database query associated with the voice-based query, and wherein the narrowing and streamlining is responsive to the location/proximity information.

2. The natural language query system of claim 1, wherein the device comprises a device selected from the group consisting of a Web-enabled portable personal computer, a Web-enabled laptop computer, a Web-enabled personal digital assistant, and a Web-enabled phone;

and

wherein the connection between the device and the one or more servers comprises one of a wireless connection and a wired connection.

3. The natural language query system of claim 1, wherein the speech input module comprises a speech plug-in and a microphone.

4. The natural language query system of claim 1, wherein the location/proximity module comprises a location/proximity module selected from the group consisting of a radio frequency identification reader and a global positioning system;

and

wherein the location/proximity device comprises a location/proximity device selected from the group consisting of a radio frequency identification tag and a satellite.

5. The natural language query system of claim 1, further comprising a speech output module for delivering the result related to the voice-based query in natural language form to the user.

6. The natural language query system of claim 1, further comprising a user database configured to receive and store data from the speech conversion module, the natural language processing module, the semantic engine module, and the database-look-up module.

7. The natural language query system of claim 6, wherein the user database is configured to analyze utilization of the natural language query system by the user.

8. A Web-enabled natural language query device for processing and analyzing voice and proximity-based queries, comprising:

a speech plug-in for receiving a voice-based query in natural language form from a user;

a location/proximity plug-in for receiving location/proximity information from a location/proximity device; and

a connection mechanism to a server, wherein the server comprises;

a speech conversion module for converting the voice-based query in natural language form to text in natural language form;

a natural language processing module for converting the text in natural language form to text in searchable form using lexicons and grammar rules to parse sentences and determine underlying meanings of the voice-base query;

a semantic engine module for converting the text in searchable form to a formal database query; and

a database-look-up module for using the formal database query to obtain a result related to the voice-based query in natural language form from a database, wherein the location/proximity plug-in is configured to provide a context and environment to the database-look-up module for narrowing and streamlining the formal database query associated with the voice-based query, and wherein the narrowing and streamlining is responsive to the location/proximity information.

9. The Web-enabled natural language query device of claim 8, wherein the Web-enabled device comprises a Web-enabled device selected from the group consisting of a Web-enabled portable personal computer, a Web-enabled laptop computer, a Web-enabled personal digital assistant, and a Web-enabled phone;

and

wherein the connection mechanism comprises one of a wireless connection and a wired connection.

10. The Web-enabled natural language query device of claim 8, wherein the speech input plug-in comprises a speech input plug-in and a microphone.

11. The Web-enabled natural language query device of claim 8, wherein the location/proximity plug-in comprises a location/proximity plug-in selected from the group consisting of a radio frequency identification reader and a global positioning system;

and

wherein the location/proximity device comprises a location/proximity device selected from the group consisting of a radio frequency identification tag and a satellite.

12. The Web-enabled natural language query device of claim 8, further comprising a speech output plug-in for delivering the result related to the voice-based query in natural language form to the user.

13. The Web-enabled natural language query device of claim 8, wherein the server further comprises a user database configured to receive and store data from the speech conversion module, the natural language processing module, the semantic engine module, and the database-look-up module.

14. The Web-enabled natural language query device of claim 13, wherein the user database is configured to analyze utilization of the Web-enabled natural language query device by the user.

15. A natural language query method for processing and analyzing voice and proximity-based queries, comprising:

providing a device, comprising;

a speech input module for receiving a voice-based query in natural language form from a user; and

a location/proximity module for receiving location/proximity information from a location/proximity device;

converting the voice-based query in natural language form to text in natural language form using a speech conversion module;

converting the text in natural language form to text in searchable form using a natural language processing module configured to use lexicons and grammar rules to parse sentences and determine underlying meanings of the voice-based query;

converting the text in searchable form to a formal database query using a semantic engine module;

narrowing the formal database query responsive to the location/proximity information from the location/proximity module, wherein the location/proximity module is configured to provide a context and environment for narrowing and streamlining the formal database query associated with the voice-based query; and

obtaining a result related to the voice-based query in natural language form from a database using the formal database query and a database-look-up module.

16. The natural language query method of claim 15, wherein the device comprises a device selected from the group consisting of a Web-enabled portable personal computer, a Web-enabled laptop computer, a Web-enabled personal digital assistant, and a Web-enabled phone;

wherein the speech input module comprises a speech plug-in and a microphone;

wherein the location/proximity module comprises a location/proximity module selected from the group consisting of a radio frequency identification reader and a global positioning system; and

wherein the location/proximity device comprises a location/proximity device selected from the group consisting of a radio frequency identification tag and a satellite.

17. The natural language query method of claim 15, wherein the speech conversion module, the natural language processing module, the semantic engine module, and the database-look-up module reside in one or more servers located remotely from the device;

and

further comprising transmitting the voice-based query from the device to the one or more servers.

18. The natural language query method of claim 15, further comprising delivering the result related to the voice-based query in natural language form to the user using a speech output module.

19. The natural language query method of claim 15, further comprising storing data from the speech conversion module, the natural language processing module, the semantic engine module, and the database-look-up module in a user database, wherein the data comprises information related to the voice-based query.

20. The natural language query method of claim 19, further comprising analyzing the data in the user database to provide analysis of a plurality of voice-based queries.