Voice-Based Media Searching

• US 20140081633A1
• Filed: 11/19/2012
• Published: 03/20/2014
• Est. Priority Date: 09/19/2012
• Status: Active Grant

• Current Assignee: Apple Inc.
• Original Assignee: Apple Inc.
• Inventors: Badaskar, Sameer
• Granted Patent: US 9,547,647 B2
• Time in Patent Office: Days
• US Class Current: 704/235
• CPC Class Codes: G06F 16/3344 using natural language analysis
• CPC Class Codes: G06F 16/43 Querying
• CPC Class Codes: G06F 16/432 Query formulation
• CPC Class Codes: G06F 16/433 using audio data
• CPC Class Codes: G06F 16/438 Presentation of query results
• CPC Class Codes: G06F 16/48 Retrieval characterised by using metadata, e.g. metadata not derived from the content or metadata generated manually
• CPC Class Codes: G06F 16/487 using geographical or spatial information, e.g. location
• CPC Class Codes: G06F 16/489 using time information
• CPC Class Codes: G06F 16/58 Retrieval characterised by using metadata, e.g. metadata not derived from the content or metadata generated manually
• CPC Class Codes: G06F 16/587 using geographical or spatial information, e.g. location
• CPC Class Codes: G06F 3/167 Audio in a user interface, e.g. using voice commands for navigating, audio feedback
• CPC Class Codes: G10L 15/22 Procedures used during a speech recognition process, e.g. man-machine dialogue
• CPC Class Codes: G10L 15/26 Speech to text systems G10L15/08 takes precedence

Abstract

Methods and systems for searching for media items using a voice-based digital assistant are described. Natural language text strings corresponding to search queries are provided. The search queries include query terms. The text strings may correspond to speech inputs input by a user into an electronic device. At least one information source is searched to identify at least one parameter associated with at least one of the query terms. The parameters include at least one of a time parameter, a date parameter, or a geo-code parameter. The parameters are compared to tags of media items to identify matches. In some implementations, media items whose tags match the parameter are presented to the user.

Claims

1. A method for searching for media items using a voice-based digital assistant, comprising:

at an electronic device with a processor and memory storing instructions for execution by the processor;

providing multiple media items wherein at least some of the media items are each associated with a respective tag comprising at least one of a time tag, a date tag, or a geo-code tag;

providing a natural language text string corresponding to a search query for one or more media items, wherein the search query includes one or more query terms;

searching at least one information source to identify at least one parameter associated with at least one of the one or more query terms, wherein the at least one parameter comprises at least one of a time parameter, a date parameter, or a geo-code parameter, and wherein the at least one parameter is separate from the tags associated with the multiple media items;

comparing the respective tags to the at least one parameter to identify at least one media item whose tag matches the identified parameter; and

facilitating the presentation of the at least one media item to a user.

2. The method of claim 1, wherein the multiple media items are photographs.

3. The method of claim 1, wherein the multiple media items are videos.

4. The method of claim 1, wherein the tags are metadata that is stored with the media items.

5. The method of claim 1, wherein at least a subset of the multiple media items is stored on the electronic device.

6. The method of claim 1, wherein at least a subset of the multiple media items is stored remotely from the electronic device.

7. The method of claim 6, wherein the remotely stored media items are associated with a social networking account of a user of the electronic device.

8. The method of claim 1, further comprising receiving a speech input from a user, wherein the natural language text string corresponds to the speech input.

9. The method of claim 8, further comprising performing speech-to-text processing on the speech input to generate the natural language text string.

10. The method of claim 1, wherein the at least one information source is selected from the group consisting of:

a calendar;

email messages;

text messages;

social network postings;

a contact book;

travel itineraries; and

an event schedule.

11. The method of claim 1, wherein the one or more query terms do not include any of a date, a time, or a geo-code.

12. The method of claim 1, wherein the one or more query terms include an event name, and the at least one information source includes information that associates event names with dates.

13. The method of claim 12, wherein the event name is a religious holiday.

14. The method of claim 12, wherein the event name is a national holiday.

15. The method of claim 12, wherein the event name is selected from the group consisting of:

a birthday;

a season; and

a sporting event.

16. The method of claim 1, wherein the one or more query terms include a holiday name, the method further comprising:

searching a calendar associated with a user to identify an event spanning a range of dates including and/or near the holiday, and

searching the date tags associated with the multiple media items to identify at least one media item whose associated date tag falls within the range of dates.

17. The method of claim 1, wherein the time parameter comprises a time range surrounding a time specified in the one or more query terms.

18. The method of claim 1, wherein the date parameter comprises a date range surrounding a date specified in the one or more query terms.

19. The method of claim 1, wherein the geo-code parameter comprises a range of geo-codes associated with a location specified in the one or more query terms.

20. The method of claim 19, wherein the range of geo-codes corresponds to geographical boundaries of the location.

21. The method of claim 19, wherein the range of geo-codes corresponds to a principal geo-code and one or more distances from the principal geo-code in one or more directions.

22. The method of claim 1, wherein one of the at least one parameter corresponds to a location, and wherein comparing the respective tags to the at least one parameter further comprises determining whether the geo-code tags correspond to the location.

23. The method of claim 1, further comprising assigning a confidence value to each identified media item, wherein the confidence value corresponds to the strength of the match between the time tag, the date tag, or the geo-code tag of the media item and the time, the date, or the geo-code associated with the at least one of the one or more query terms.

24. The method of claim 23, wherein the at least one media item comprises a plurality of media items, the method further comprising:

displaying, on a display associated with the electronic device, at least a subset of the plurality of media items.

25. The method of claim 24, further comprising, prior to the displaying, ranking the plurality of media items based on the confidence values, wherein the plurality of media items are displayed in an order based on the ranking.

26. The method of claim 1, wherein the media item is audio.

27. The method of claim 1, wherein the electronic device is a handheld electronic device.

28. The method of claim 1, wherein the electronic device is a server, and the text string is received at a second electronic device separate from the electronic device.

29. An electronic device including one or more processors, memory, and one or more programs stored in the memory and configured to be executed by the one or more processors, the one or more programs comprising instructions for:

providing multiple media items wherein at least some of the media items are each associated with a respective tag comprising at least one of a time tag, a date tag, or a geo-code tag;

providing a natural language text string corresponding to a search query for one or more media items, wherein the search query includes one or more query terms;

searching at least one information source to identify at least one parameter associated with at least one of the one or more query terms, wherein the at least one parameter comprises at least one of a time parameter, a date parameter, or a geo-code parameter, and wherein the at least one parameter is separate from the tags associated with the multiple media items;

comparing the respective tags to the at least one parameter to identify at least one media item whose tag matches the identified parameter; and

facilitating the presentation of the at least one media item to a user.

30. A non-transitory computer readable storage medium storing one or more programs configured for execution by a server system, the one or more programs comprising instructions for:

providing multiple media items wherein at least some of the media items are each associated with a respective tag comprising at least one of a time tag, a date tag, or a geo-code tag;

providing a natural language text string corresponding to a search query for one or more media items, wherein the search query includes one or more query terms;

searching at least one information source to identify at least one parameter associated with at least one of the one or more query terms, wherein the at least one parameter comprises at least one of a time parameter, a date parameter, or a geo-code parameter, and wherein the at least one parameter is separate from the tags associated with the multiple media items;

comparing the respective tags to the at least one parameter to identify at least one media item whose tag matches the identified parameter; and

facilitating the presentation of the at least one media item to a user.