MULTIMODAL NATURAL LANGUAGE INTERFACE FOR FACETED SEARCH

• US 20130226892A1
• Filed: 02/28/2013
• Published: 08/29/2013
• Est. Priority Date: 02/29/2012
• Status: Abandoned Application

• Current Assignee: Nant Holdings IP, LLC (NantWorks, LLC)
• Original Assignee: Fluential, LLC
• Inventors: Ehsani, Farzad, Witt-Ehsani, Silke Maren
• Application Number:
  US13/781,524
• Publication Number:
  US 20130226892A1
• Time in Patent Office: Days
• US Class Current: 707/706
• CPC Class Codes: G06F 16/90332 Natural language query formulation or dialogue systems
• CPC Class Codes: G06F 16/951 Indexing; Web crawling techniques
• CPC Class Codes: G06F 16/9532 Query formulation
• CPC Class Codes: G06F 16/9535 Search customisation based on user profiles and personalisation
• CPC Class Codes: G06F 16/9538 Presentation of query results

Abstract

Search interfaces, systems, and methods are presented. Contemplated search interfaces allow electronic devices to capture multi-modal interaction data, including audio signals. A dialog interface capable of interacting with a user processes the interaction data and communicates with a user to establish a desirable query interpretation. Further, the dialog interface can identify a target search engine for a corresponding query based on modalities of the interaction data beyond the data represented by the audio signal.

Claims

1. A search interface comprising:

an electronic device comprising a plurality of interfaces, the interfaces collectively capable of receiving multimodal input including audio signals; and

a dialog interface module disposed within the electronic device and configured to;

obtain an audio signal representing audio data of a spoken utterance from the plurality of interfaces;

obtain a second signal representing a modality other than audio data from the plurality of interfaces;

map the audio signal and second signal to a query interpretation by correlating audio signal and second signal to a plurality of data facets;

construct proposed search criteria representing the query interpretation and including alternative values for each data facet of the plurality of data facets;

present the proposed search criteria, including the alternative values, to a user via the electronic device;

receive at least one selected value from the presented alternative values;

identify a targeted search engine having an indexing system based on the at least one selected value, the second signal, and the query interpretation;

translate the query interpretation to a targeted query as a function the audio signal, second signal, and the at least one selected value, the targeted query constructed according to an indexing system of the target search engine;

cause the targeted query to be submitted to the target search engine; and

enable the electronic device to present search results from the target search engine response to submission of the targeted query.

2. The interface of claim 1, wherein the second signal comprises a representation of a haptic interaction.

3. The interface of claim 1, wherein the second signal comprises a representation of an image.

4. The interface of claim 1, wherein the second signal comprises a representation of at least one of the following:

motion, location, time, biometrics, intonation, inflection, accent, gestures, text, taste, hardware status and proximity.

5. The interface of claim 1, wherein the electronic device comprises at least one of the following:

a phone, a tablet, a set top box, an appliance, a gaming device, a computer, a medical device, a search engine server, a vehicle, a home control interface and a kiosk.

6. The interface of claim 1, wherein the dialog interface is further configured to present the targeted query to the user via the electronic device.

7. The interface of claim 6, wherein the electronic device is configured to present the proposed search criteria according to the modality of the second signal.

8. The interface of claim 6, wherein the electronic device is configured to present the proposed search criteria as audio data.

9. The interface of claim 6, wherein the electronic device is configured to present the proposed search criteria using different visual characteristics according to the data facets.

10. The interface of claim 9, wherein the visual characteristics include at least one of the following:

a color, a font size, a font, a font modality such as underlined, an intensity, a timed pattern, a shape, a pattern, an image, an animation and an icon.

11. The interface of claim 1, wherein the dialog interface is further configured to rank the alternative values of the proposed search criteria.

12. The interface of claim 11, wherein the alternative values are ranked according to a type of modality of the second signal.

13. The interface of claim 11, wherein the alternative values are ranked according to frequency of previous searches.

14. The interface of claim 11, wherein the alternative values are ranked according to an ontology representing the data facets.

15. The interface of claim 11, wherein the alternative values are ranked according to a structured product listing.

16. The interface of claim 11, wherein the alternative values are ranked according to a catalog.

17. The interface of claim 11, wherein the alternative values are ranked according to a diagnostic hierarchy.

18. The interface of claim 1, wherein the dialog interface module is further configured to identify the target search engine based on a modality of the second signal.

19. The interface of claim 18, wherein the target search engine is identified based on the indexing system being compatible with the modality of the second signal.