VOICE-BASED MEDIA SEARCHING

• US 20170161268A1
• Filed: 12/19/2016
• Published: 06/08/2017
• Est. Priority Date: 09/19/2012
• Status: Active Grant

• Current Assignee: Apple Inc.
• Original Assignee: Apple Inc.
• Inventors: BADASKAR, Sameer
• Granted Patent: US 9,971,774 B2
• Time in Patent Office: Days
• CPC Class Codes: G06F 16/3344 using natural language analysis
• CPC Class Codes: G06F 16/43 Querying
• CPC Class Codes: G06F 16/432 Query formulation
• CPC Class Codes: G06F 16/433 using audio data
• CPC Class Codes: G06F 16/438 Presentation of query results
• CPC Class Codes: G06F 16/48 Retrieval characterised by using metadata, e.g. metadata not derived from the content or metadata generated manually
• CPC Class Codes: G06F 16/487 using geographical or spatial information, e.g. location
• CPC Class Codes: G06F 16/489 using time information
• CPC Class Codes: G06F 16/58 Retrieval characterised by using metadata, e.g. metadata not derived from the content or metadata generated manually
• CPC Class Codes: G06F 16/587 using geographical or spatial information, e.g. location
• CPC Class Codes: G06F 3/167 Audio in a user interface, e.g. using voice commands for navigating, audio feedback
• CPC Class Codes: G10L 15/22 Procedures used during a speech recognition process, e.g. man-machine dialogue
• CPC Class Codes: G10L 15/26 Speech to text systems G10L15/08 takes precedence

Abstract

Methods and systems for searching for media items using a voice-based digital assistant are described. Natural language text strings corresponding to search queries are provided. The search queries include query terms. The text strings may correspond to speech inputs input by a user into an electronic device. At least one information source is searched to identify at least one parameter associated with at least one of the query terms. The parameters include at least one of a time parameter, a date parameter, or a geo-code parameter. The parameters are compared to tags of media items to identify matches. In some implementations, media items whose tags match the parameter are presented to the user.

Claims

1-30. -30. (canceled)

31. An electronic device, comprising:

a memory;

one or more programs, wherein the one or more programs are stored in the memory and configured to be executed by the one or more processors, the one or more programs including instructions for;

providing a plurality of media items wherein at least some of the media items are each associated with one or more respective tags;

providing a natural language text string corresponding to a search query for one or more media items, wherein the search query includes one or more query terms;

identifying, based on the search query, a first media item of the plurality of media items, wherein the first media item is associated with user-generated information and wherein the user-generated information matches a query term of the one or more query terms;

comparing the respective tags of the identified first media item to the respective tags of one or more other media items of the plurality of media items to identify a second media item; and

facilitating the presentation of the first media item and the second media item to a user.

32. The electronic device of claim 31, wherein the one or more respective tags comprise at least one of a time tag, a date tag, or a geo-code tag.

33. The electronic device of claim 31, wherein the one or more programs further include instructions for:

accessing the user-generated information from an information source.

34. The electronic device of claim 33, wherein the information source is selected from the group consisting of:

a calendar;

email messages;

text messages;

social network postings;

a contact book;

travel itineraries; and

an event schedule.

35. The electronic device of claim 33, wherein the one or more programs further include instructions for:

accessing the user-generated information from the information source in response to receiving the search query.

36. The electronic device of claim 33, wherein the one or more query terms include an event name, and wherein the information source associates event names with dates.

37. The electronic device of claim 31, wherein the user-generated information is part of a caption or a comment of a social network posting.

38. The electronic device of claim 31, wherein the one or more query terms do not include any of a date, a time, or a geo-code.

39. The electronic device of claim 31, wherein the second media item is not associated with the user-generated information.

40. The electronic device of claim 31, wherein at least one of the plurality of media items is a photograph.

41. The electronic device of claim 31, wherein at least one of the plurality of media items is a video.

42. The electronic device of claim 31, wherein at least one of the plurality of media items is an audio.

43. The electronic device of claim 31, wherein the tags are metadata stored with the plurality of media items.

44. The electronic device of claim 31, wherein at least a subset of the plurality of media items is stored on the electronic device.

45. The electronic device of claim 31, wherein at least a subset of the multiple media items is stored remotely from the electronic device.

46. The electronic device of claim 45, wherein the remotely stored media items are associated with a social networking account of a user of the electronic device.

47. The electronic device of claim 31, wherein the one or more programs further include instructions for:

receiving a speech input from a user, wherein the natural language text string corresponds to the speech input.

48. The electronic device of claim 47, wherein the one or more programs further include instructions for:

performing speech-to-text processing on the speech input to generate the natural language text string.

49. The electronic device of claim 31, wherein the one or more programs further include instructions for:

displaying, on a display of the electronic device, the first media item and the second media item.

50. The electronic device of claim 49, wherein the one or more programs further include instructions for:

assigning a confidence value to each of the first media item and the second media item;

prior to the displaying, ranking the first media item and the second media item based on the respective confidence values, wherein the first media item and the second media item are displayed in an order based on the ranking.

51. The electronic device of claim 31, wherein the electronic device is a handheld electronic device.

52. A method for searching for media items using a voice-based digital assistant, comprising:

at an electronic device with a processor and memory storing instructions for execution by the processor;

providing a plurality of media items wherein at least some of the media items are each associated with a respective tag comprising at least one of a time tag, a date tag, or a geo-code tag;

providing a natural language text string corresponding to a search query for one or more media items, wherein the search query includes one or more query terms;

identifying, based on the search query, a first media item of the plurality of media items, wherein the media item is associated with user-generated information and wherein the user-generated information matches a query term of the one or more query terms;

comparing the respective tags of the identified first media item to the respective tags of one or more other media items of the plurality of media items to identify a second media item; and

facilitating the presentation of the first media item and the second media item to a user.

53. A non-transitory computer readable storage medium storing one or more programs, the one or more programs comprising instructions, which when executed by one or more processors of an electronic device, cause the device to:

provide a plurality of media items wherein at least some of the media items are each associated with a respective tag comprising at least one of a time tag, a date tag, or a geo-code tag;

provide a natural language text string corresponding to a search query for one or more media items, wherein the search query includes one or more query terms;

identify, based on the search query, a first media item of the plurality of media items, wherein the media item is associated with user-generated information and wherein the user-generated information matches a query term of the one or more query terms;

compare the respective tags of the identified first media item to the respective tags of one or more other media items of the plurality of media items to identify a second media item; and

facilitate the presentation of the first media item and the second media item to a user.