Virtual assistant for media playback

• US 11,010,127 B2
• Filed: 03/21/2019
• Issued: 05/18/2021
• Est. Priority Date: 06/29/2015
• Status: Active Grant

• Current Assignee: Apple Inc.
• Original Assignee: Apple Inc.
• Inventors: Orr, Ryan M., Mandel, Daniel J., Sinesio, Andrew J., Barnett, Connor J.
• Primary Examiner(s): Jackson, Jakieda R
• Application Number:
  US16/360,695
• Publication Number:
  US 20190220246A1
• Time in Patent Office: 789 Days
• Field of Search: 704 9, 704270, 7042701, 704275
• CPC Class Codes: G06F 16/48 Retrieval characterised by using metadata, e.g. metadata not derived from the content or metadata generated manually
• CPC Class Codes: G06F 3/167 Audio in a user interface, e.g. using voice commands for navigating, audio feedback
• CPC Class Codes: G06F 40/169 Annotation, e.g. comment data or footnotes
• CPC Class Codes: G06F 40/279 Recognition of textual entities
• CPC Class Codes: G06F 40/30 Semantic analysis
• CPC Class Codes: G10L 15/26 Speech to text systems G10L15/08 takes precedence

Abstract

An exemplary method for identifying media may include receiving user input associated with a request for media, where that user input includes unstructured natural language speech including one or more words; identifying at least one context associated with the user input; causing a search for the media based on the at least one context and the user input; determining, based on the at least one context and the user input, at least one media item that satisfies the request; and in accordance with a determination that the at least one media item satisfies the request, obtaining the at least one media item.

Claims

1. A non-transitory computer-readable storage medium storing one or more programs, the one or more programs comprising instructions, which when executed by one or more processors of an electronic device, cause the electronic device to:

receive a first natural language speech input indicative of a media request;

determine a first search criteria included in the first natural language speech input;

obtain a first media item satisfying the media request, wherein obtaining the first media item includes;

causing a first search for a first plurality of media items to obtain the first media item, the first search including the first search criteria and a first context;

while causing the first search, causing a second search including the first search criteria and a second context to obtain a candidate media item; and

determining that the first media item from the first plurality of media items matches the candidate media item;

provide playback of a first media item from the first plurality of media items;

while providing playback of the first media item, receive a second natural language speech input;

determine, based on the second natural language speech input, a user intent of obtaining a media item different from the first media item; and

in accordance with determining the user intent of obtaining a media item different from the first media item;

obtain a second media item different from the first media item based on the first search criteria and the second natural language speech input, wherein obtaining the second media item includes;

initiating a third search for a second plurality of media items, the third search including the previously determined first search criteria included in the first natural language speech input; and

provide the second media item from the second plurality of media items.

2. The non-transitory computer-readable storage medium of claim 1, wherein the first context is based on a context of the first natural language speech input.

3. The non-transitory computer-readable storage medium of claim 1, wherein determining the user intent of obtaining a media item different from the first media item includes determining the user intent of obtaining a media item different from the first media item based on a context of the second natural language speech input.

4. The non-transitory computer-readable storage medium of claim 1, wherein initiating the third search for the second plurality of media items further includes:

searching for the second media item based on a context of the second natural language speech input.

5. The non-transitory computer-readable storage medium of claim 4, wherein the context of the second natural language speech input includes device context.

6. The non-transitory computer-readable storage medium of claim 5, wherein the device context includes a content of the first media item.

7. The non-transitory computer-readable storage medium of claim 5, wherein the device context includes data associated with media stored on the electronic device.

8. The non-transitory computer-readable storage medium of claim 4, wherein the context of the second natural language speech input includes a user context.

9. The non-transitory computer-readable storage medium of claim 8, wherein the user context includes a content of the second natural language speech input.

10. The non-transitory computer-readable storage medium of claim 8, wherein the user context includes media associated with a user.

11. The non-transitory computer-readable storage medium of claim 8, wherein the user context includes information relating to a musical preference of a user.

12. The non-transitory computer-readable storage medium of claim 4, wherein the context of the second natural language input includes social context.

13. The non-transitory computer-readable storage medium of claim 12, wherein the social context includes an access frequency of a particular media item across a plurality of users.

14. The non-transitory computer-readable storage medium of claim 12, wherein the social context includes a number of references to a media item in a social media database.

15. The non-transitory computer-readable storage medium of claim 1, wherein initiating the third search for the second plurality of media items includes:

searching for a set of media items based on the first search criteria; and

discarding, from the set of media items, one or more media items matching the first media item.

16. The non-transitory computer-readable storage medium of claim 1, wherein providing the second media item includes:

terminating playback of the first media item; and

providing playback of the second media item.

17. The non-transitory computer-readable storage medium of claim 1, wherein the one or more programs further comprise instructions, which when executed by the one or more processors, cause the electronic device to:

present, to a user, an option to add the second media item to a library associated with the user;

receive from the user, a user selection of the option; and

in response to receiving the user selection of the option, add the second media item to the library associated with the user.

18. The non-transitory computer-readable storage medium of claim 1, wherein the one or more programs further comprise instructions, which when executed by the one or more processors, cause the electronic device to:

while providing the second media item, receive a third natural speech input including a request for alternate media; and

in accordance with receiving the third natural language speech input, provide a third media item different from the second media item.

19. The non-transitory computer-readable storage medium of claim 18, wherein the third media item includes a different version of the second media item.

20. A method for operating a digital assistant comprising:

at an electronic device with one or more processors and memory;

receiving a first natural language speech input indicative of a media request;

determining a first search criteria included in the first natural language speech input;

obtaining a first media item satisfying the media request, wherein obtaining the first media item includes;

causing a first search for a plurality of media items to obtain the first media item, the first search including the first search criteria and a first context;

while causing the first search, causing a second search including the first search criteria and a second context to obtain a candidate media item; and

determining that the first media item from the first plurality of media items matches the candidate media item;

providing playback of a first media item from the first plurality of media items;

while providing playback of the first media item, receiving a second natural language speech input;

determining, based on the second natural language speech input, a user intent of obtaining a media item different from the first media item; and

in accordance with determining the user intent of obtaining a media item different from the first media item;

obtaining a second media item different from the first media item based on the first search criteria and the second natural language speech input, wherein obtaining the second media item includes;

initiating a third search for a second plurality of media items, the third search including the previously determined first search criteria included in the first natural language speech input; and

providing the second media item from the second plurality of media items.

21. The method of claim 20, wherein the first context is based on a context of the first natural language speech input.

22. The method of claim 20, wherein determining the user intent of obtaining a media item different from the first media item includes determining the user intent of obtaining a media item different from the first media item based on a context of the second natural language speech input.

23. The method of claim 20, wherein initiating the third search for the second plurality of media items further includes:

searching for the second media item based on a context of the second natural language speech input.

24. The method of claim 23, wherein the context of the second natural language speech input includes device context.

25. The method of claim 24, wherein the device context includes a content of the first media item.

26. The method of claim 24, wherein the device context includes data associated with media stored on the electronic device.

27. The method of claim 23, wherein the context of the second natural language speech input includes a user context.

28. The method of claim 27, wherein the user context includes a content of the second natural language speech input.

29. The method of claim 27, wherein the user context includes media associated with a user.

30. The method of claim 27, wherein the user context includes information relating to a musical preference of a user.

31. The method of claim 23, wherein the context of the second natural language input includes social context.

32. The method of claim 31, wherein the social context includes an access frequency of a particular media item across a plurality of users.

33. The method of claim 31, wherein the social context includes a number of references to a media item in a social media database.

34. The method of claim 20, wherein initiating the third search for the second plurality of media items includes:

searching for a set of media items based on the first search criteria; and

discarding, from the set of media items, one or more media items matching the first media item.

35. The method of claim 20, wherein providing the second media item includes:

terminating playback of the first media item; and

providing playback of the second media item.

36. The method of claim 20, further comprising:

presenting, to a user, an option to add the second media item to a library associated with the user;

receiving from the user, a user selection of the option; and

in response to receiving the user selection of the option, adding the second media item to the library associated with the user.

37. The method of claim 20, further comprising:

while providing the second media item, receiving a third natural speech input including a request for alternate media; and

in accordance with receiving the third natural language speech input, providing a third media item different from the second media item.

38. The method of claim 37, wherein the third media item includes a different version of the second media item.

39. An electronic device, comprising:

one or more processors;

a memory; and

one or more programs, wherein the one or more programs are stored in the memory and configured to be executed by the one or more processors, the one or more programs including instructions for;

receiving a first natural language speech input indicative of a media request;

determining a first search criteria included in the first natural language speech input;

obtaining a first media item satisfying the media request, wherein obtaining the first media item includes;

causing a first search for a first plurality of media items to obtain the first media item, the first search including the first search criteria and a first context;

while causing the first search, causing a second search including the first search criteria and a second context to obtain a candidate media item; and

determining that the first media item from the first plurality of media items matches the candidate media item;

providing playback of a first media item from the first plurality of media items;

while providing playback of the first media item, receiving a second natural language speech input;

determining, based on the second natural language speech input, a user intent of obtaining a media item different from the first media item; and

in accordance with determining the user intent of obtaining a media item different from the first media item;

obtaining a second media item different from the first media item based on the first search criteria and the second natural language speech input, wherein obtaining the second media item includes;

initiating a third search for a second plurality of media items, the third search including the previously determined first search criteria included in the first natural language speech input; and

providing the second media item from the second plurality of media items.

40. The device of claim 39, wherein the first context is based on a context of the first natural language speech input.

41. The device of claim 39, wherein determining the user intent of obtaining a media item different from the first media item includes determining the user intent of obtaining a media item different from the first media item based on a context of the second natural language speech input.

42. The device of claim 39, wherein initiating the third search for the second plurality of media items further includes:

searching for the second media item based on a context of the second natural language speech input.

43. The device of claim 42, wherein the context of the second natural language speech input includes device context.

44. The device of claim 43, wherein the device context includes a content of the first media item.

45. The device of claim 43, wherein the device context includes data associated with media stored on the electronic device.

46. The device of claim 42, wherein the context of the second natural language speech input includes a user context.

47. The device of claim 46, wherein the user context includes a content of the second natural language speech input.

48. The device of claim 46, wherein the user context includes media associated with a user.

49. The device of claim 46, wherein the user context includes information relating to a musical preference of a user.

50. The device of claim 42, wherein the context of the second natural language input includes social context.

51. The device of claim 50, wherein the social context includes an access frequency of a particular media item across a plurality of users.

52. The device of claim 50, wherein the social context includes a number of references to a media item in a social media database.

53. The device of claim 39, wherein initiating the third search for the second plurality of media items includes:

searching for a set of media items based on the first search criteria; and

discarding, from the set of media items, one or more media items matching the first media item.

54. The device of claim 39, wherein providing the second media item includes:

terminating playback of the first media item; and

providing playback of the second media item.

55. The device of claim 39, wherein the one or more programs further include instructions for:

presenting, to a user, an option to add the second media item to a library associated with the user;

receiving from the user, a user selection of the option; and

in response to receiving the user selection of the option, adding the second media item to the library associated with the user.

56. The device of claim 39, wherein the one or more programs further include instructions for:

while providing the second media item, receiving a third natural speech input including a request for alternate media; and

in accordance with receiving the third natural language speech input, providing a third media item different from the second media item.

57. The device of claim 56, wherein the third media item includes a different version of the second media item.