CONTEXTUAL SEARCH USING NATURAL LANGUAGE

• US 20160210363A1
• Filed: 10/05/2015
• Published: 07/21/2016
• Est. Priority Date: 01/21/2015
• Status: Active Grant

• Current Assignee: Microsoft Technology Licensing LLC (Microsoft Corporation)
• Original Assignee: Microsoft Technology Licensing LLC (Microsoft Corporation)
• Inventors: Doran, Ed, Rambhia, Amy, Syed, Harris
• Granted Patent: US 10,509,829 B2
• Time in Patent Office: Days
• US Class Current: 1/1
• CPC Class Codes: G06F 16/248 Presentation of query results
• CPC Class Codes: G06F 16/951 Indexing; Web crawling techniques
• CPC Class Codes: G06F 16/9537 Spatial or temporal dependent retrieval, e.g. spatiotemporal queries
• CPC Class Codes: G06F 3/0482 Interaction with lists of selectable items, e.g. menus
• CPC Class Codes: G06F 3/167 Audio in a user interface, e.g. using voice commands for navigating, audio feedback
• CPC Class Codes: G06F 40/205 Parsing
• CPC Class Codes: G06F 9/453 Help systems
• CPC Class Codes: G10L 15/1822 Parsing for meaning understanding
• CPC Class Codes: G10L 17/00 Speaker identification or verification techniques
• CPC Class Codes: G10L 17/22 Interactive procedures; Man-machine interfaces
• CPC Class Codes: H04L 67/10 in which an application is distributed across nodes in the network software deployment G06F8/60; multiprogramming arrangements G06F9/46

Abstract

A context aware digital assistant supported on devices such as smartphones, tablet computers, wearable computing devices, personal computers (PCs), game consoles, and the like is configured with a natural language interface that enables a user to launch searches for content using contextual references such as time, date, event, location, schedule, activity, contacts, or device. The user can thus use natural language to express the context that is applicable to the sought-after content rather than having to formulate a query that uses a specific syntax. The digital assistant can comprehensively search for the content across applications (i.e., both first and third party applications), devices, and services.

Claims

1. A method performed on a device for facilitating a search for content using contextual references, comprising:

exposing a user interface supporting natural language inputs for the search from a device user;

receiving the natural language inputs from the user through the user interface;

parsing the received natural language inputs for contextual references, the contextual references including one of time, date, event, location, schedule, activity, contact, or device; and

providing results of the search, the search results identifying content matching one or more of the parsed contextual references.

2. The method of claim 1 further including providing the search results using contextual reference tags.

3. The method of claim 1 further including determining a contextual reference tag for a given piece of content and applying the contextual reference tag to the content.

4. The method of claim 1 in which the user interface is supported at least in part by a digital assistant operating on the device.

5. The method of claim 4 in which the digital assistant interacts with the user using a spoken voice over an audio interface supported on the device.

6. The method of claim 4 in which the digital assistant interacts with the user over a graphical user interface supported on the device and further including displaying the search results on the graphical user interface.

7. The method of claim 4 further including configuring the digital assistant, responsively to voice input, gesture input, or manual input for performing at least one of sharing contact information, sharing a stored contact, scheduling a meeting, looking at a user'"'"'s calendar, scheduling a reminder, making a call, operating a device, playing games, making purchases, taking notes, scheduling an alarm or wake-up reminder, sending a message, checking social media for updates, scraping a website, interacting with a search service, sharing or showing files, sending a link to a website, or sending a link to a resource.

8. The method of claim 1 in which the content is stored in one or more remote cloud-based stores or cloud-based services.

9. The method of claim 1 further including rank ordering the search results according to the contextual references.

10. The method of claim 1 further including enabling content identified in the search results to be available for consumption on the device.

11. A device, comprising:

one or more processors;

a user interface (UI) configured to interact with a user of the device using graphics or audio; and

a memory device storing computer-readable instructions which, when executed by the one or more processors, perform a method to;

expose a digital assistant on the device in which the digital assistant is configured for voice interactions with a device user using the UI,

receive voice inputs from the device user,

trigger a search for content using contextual references that are included in the voice inputs, and

handle content that is identified in results from the search using the digital assistant.

12. The device of claim 11 further comprising displaying search results on the UI as an aggregated list.

13. The device of claim 11 further comprising using the digital assistant to provide search results using audio.

14. The device of claim 11 in which the digital assistant is configured to use a natural language interface to interact with the user using spoken words.

15. The device of claim 11 in which at least a portion of content includes one or more tags to identify contextual references associated with the portion.

16. The device of claim 15 in which the tags identify one or more of time, date, event, location, schedule, activity, contact, or device.

17. One or more computer-readable memory devices storing instructions which, when executed by one or more processors disposed in a cloud-based computer server, perform a method comprising the steps of:

receiving registrations from applications and services running on one or more local devices;

monitoring user interactions at the one or more local devices with registered applications and services; and

tagging content using contextual reference tags responsively to the monitoring, the contextual reference tags including one or more of time, date, event, location, schedule, activity, contact, or device.

18. The one or more computer-readable memory devices of claim 17 further including receiving a search request from a user for content that matches one or more contextual references.

19. The one or more computer-readable memory devices of claim 18 further including performing a search responsively to the request.

20. The one or more computer-readable memory devices of claim 19 further including transmitting the search results to one or more of the local devices.