METHODS AND SYSTEMS FOR SEARCHING UTILIZING ACOUSTICAL CONTEXT

• US 20140372401A1
• Filed: 03/28/2012
• Published: 12/18/2014
• Est. Priority Date: 03/28/2011
• Status: Active Grant

• Current Assignee: Diablo Canyon Collective LLC
• Original Assignee: Ambientz
• Inventors: Goldstein, Steven W., Cohen, Jordan
• Granted Patent: US 10,409,860 B2
• Time in Patent Office: Days
• US Class Current: 707/706
• CPC Class Codes: G06F 16/635 Filtering based on additional data, e.g. user or group profiles
• CPC Class Codes: G06F 16/638 Presentation of query results
• CPC Class Codes: G06F 16/68 Retrieval characterised by using metadata, e.g. metadata not derived from the content or metadata generated manually
• CPC Class Codes: G06F 16/9032 Query formulation
• CPC Class Codes: G06F 16/953 Querying, e.g. by the use of web search engines
• CPC Class Codes: G10L 17/00 Speaker identification or verification techniques
• CPC Class Codes: G10L 2015/088 Word spotting
• CPC Class Codes: G10L 25/54 for retrieval
• CPC Class Codes: G10L 25/63 for estimating an emotional state
• CPC Class Codes: G10L 25/78 Detection of presence or absence of voice signals switching of direction of transmission by voice frequency in two-way loud-speaking telephone systems H04M9/10

Abstract

Systems, controllers and methods for contextual-based searching are provided. A system includes one or more devices configured to collect at least one audio stream and a contextual search system. The contextual search system includes a data analyzer and a search engine. The data analyzer is configured to receive the at least one audio stream from among the one or more devices and to determine contextual information from the received at least one audio stream. The search engine is configured to perform a search of at least one search term using the contextual information, to produce a search result.

Claims

1. A system for contextual-based searching comprising:

one or more devices configured to collect at least one audio stream; and

a contextual search system including;

a data analyzer configured to receive the at least one audio stream from among the one or more devices and to determine contextual information from the received at least one audio stream, and

a search engine configured to perform a search of at least one search term using the contextual information, to produce a search result.

2. The system according to claim 1, wherein the system provides the search result to at least one device among the one or more devices.

3. The system according to claim 1, wherein the contextual information indicates a search intent of the search term.

4. The system according to claim 1, wherein the contextual search system includes at least one classifier to classify the audio stream into classified information, the data analyzer using the classified information and the at least one search term to determine the contextual information.

5. The system according to claim 4, wherein the classifier classifies each audio stream into at least one of a key word, a speech event, a non-speech event, a topic, a transcription, a speaker identity or an emotional state associated with the audio stream.

6. The system according to claim 4, wherein the data analyzer is configured to receive non-acoustic information associated with the respective device, the data analyzer using at least one of the non-acoustic information or further classified information from the non-acoustic information to determine the contextual information.

7. The system according to claim 6, wherein the non-acoustic information includes at least one of a geographical location, a time, a speaker identity, an image, textual information, biometric information or environmental information.

8. The system according to claim 6, further including a storage device configured to store at least one of the at least one audio stream, the classified information, the non-acoustic information, the further classified information, the contextual information, the at least one search term, the search result or a log of the classified information over a period of time.

9. The system according to claim 1, wherein each device has at least a communication capability for interchanging information with at least one of others of the devices or a server.

10. The system according to claim 9, wherein at least one of the server or the one or more of the devices includes the contextual search system.

11. The system according to claim 1, wherein each of at least some of the devices include a user interface, the search being performed responsive to an indication received from the user interface.

12. The system according to claim 1, wherein the contextual search system continuously monitors the at least one audio stream, and a further search is performed automatically responsive to an indication in the monitored audio stream.

13. The system according to claim 1, wherein each of at least some of the devices have one or more sensors for acquiring sensor data related to an environment proximate to the device.

14. The system according to claim 13, wherein the one or more sensors includes at least one of a microphone, a motion-related sensor, a temperature sensor, a biometric sensor, a compass, an image sensor, a light detection sensor, a proximity sensor, a gravity detection sensor, a magnetic field detection sensor, an electrical field detection sensor, a vibration sensor, a pressure sensor, a humidity sensor, a moisture sensor, a toxin detection sensor, a nutrient detection sensor or a pheromone detection sensor.

15. The system according to claim 1, wherein at least one of the devices includes at least one of a user interface, a display, a warning indicator, a speaker, a position module or a privacy module.

16. The system according to claim 1, wherein at least one of the devices includes at least one of a mobile telephone device, an earpiece, a hearing aid, a navigation device, a computer or a sensor module having at least one sensor.

17. The system according to claim 1, wherein at least one of the devices includes at least one of a wireless device or a wired device.

18. The system according to claim 1, wherein at least one of the devices includes at least one of a fixed device or a mobile device.

19. A controller for contextual-based searching, the controller configured to interact with a plurality of devices, each device having at least a communication capability for interchanging information with at least one of others of the devices, the controller or a communication system, at least one of one or more of the devices or the controller configured to collect at least one audio stream, the controller comprising:

a contextual search system including;

a data analyzer configured to receive the at least one audio stream from among at least one of the one or more devices or the controller and to determine contextual information from the received at least one audio stream, and

a search engine configured to perform a search of at least one search term using the contextual information, to produce a search result.

20. The controller according to claim 19, further including a communication module configured to receive the at least one audio stream from among the one or more devices.

21. The controller according to claim 19, further including a sensor module for collecting the at least one audio stream.

22. The controller according to claim 21, wherein the sensor module includes a microphone and at least one of a motion-related sensor, a temperature sensor, a biometric sensor, a compass, an image sensor, a light detection sensor, a proximity sensor, a gravity detection sensor, a magnetic field detection sensor, an electrical field detection sensor, a vibration sensor, a pressure sensor, a humidity sensor, a moisture sensor, a toxin detection sensor, a nutrient detection sensor or a pheromone detection sensor.

23. The controller according to claim 19, wherein the controller provides the search result to at least one device among the one or more devices.

24. The controller according to claim 19, wherein the contextual search system includes at least one classifier to classify the audio stream into classified information, the data analyzer using the classified information and the at least one search term to determine the contextual information.

25. The controller according to claim 24, wherein the data analyzer is configured to receive non-acoustic information associated with the respective device, the data analyzer using at least one of the non-acoustic information or further classified information from the non-acoustic information to determine the contextual information.

26. The controller according to claim 25, further including a storage device configured to store at least one of the at least one audio stream, the classified information, the non-acoustic information, the further classified information, the contextual information, the at least one search term, the search result or a log of the classified information over a period of time.

27. The controller according to claim 19, wherein the contextual search system continuously monitors the at least one audio stream, and a further search is performed automatically responsive to an indication in the monitored audio stream.

28. The controller according to claim 19, further including a user interface, the search being performed responsive to an indication received from the user interface.

29. A method for contextual-based searching comprising:

collecting at least one audio stream from among one or more devices;

determining, by a data analyzer, contextual information from the at least one audio stream received from among the one or more devices; and

performing, by a search engine, a search of at least one search term using the contextual information, to produce a search result.

30. The method according to claim 29, the method further including providing the search result to at least one device among the one or more devices, the search result being presented at least one of visually, aurally or by a haptic interface.

31. The method according to claim 30, the method further comprising:

collecting at least one further audio stream from among the one or more devices responsive to the presented search result;

determining further contextual information from the at least one further audio stream; and

determining an effectiveness of the search result from the further contextual information.

32. The method according to claim 29, wherein the contextual information is used to disambiguate a search intent of the search term.

33. The method according to claim 29, the method further including:

continuously monitoring the at least one audio stream; and

automatically performing a further search responsive to an indication in the monitored audio stream.

34. The method according to claim 29, wherein the search is performed responsive to an indication received from at least one of the one or more devices.

35. The method according to claim 34, wherein a first portion of the at least one audio stream prior to the indication and a second portion of the at least one audio stream after the indication are used to determine the contextual information.

36. The method according to claim 29, the method further comprising:

performing a further search of the at least one search term using the search result, to produce a further search result.

37. The method according to claim 29, wherein the determining of the contextual information includes classifying the audio stream into classified information, the classified information and the at least one search term being used to determine the contextual information.

38. The method according to claim 37, wherein the classified information includes at least one of a key word, a speech event, a non-speech event, a topic, a transcription, a speaker identity or an emotional state associated with the audio stream.

39. The method according to claim 37, wherein the determining of the contextual information further includes using at least one of non-acoustic information or further classified information from the non-acoustic information.

40. The method according to claim 39, wherein the non-acoustic information includes at least one of a geographical location, a time, a speaker identity, an image, textual information, biometric information or environmental information.