Electronic devices with voice command and contextual data processing capabilities

• US 9,959,867 B2
• Filed: 07/11/2016
• Issued: 05/01/2018
• Est. Priority Date: 10/02/2008
• Status: Active Grant

• Current Assignee: Apple Inc.
• Original Assignee: Apple Inc.
• Inventors: Lindahl, Aram M.
• Primary Examiner(s): Chawan, Vijay B
• Application Number:
  US15/207,248
• Publication Number:
  US 20160336010A1
• Time in Patent Office: 659 Days
• Field of Search: 704270, 7042701, 704275, 704250, 704231, 704257, 704201, 704243, 709206, 709204, 709217, 709227, 379 9324, 37910008, 381 77, 381311, 700 94
• CPC Class Codes: G06F 16/43 Querying
• CPC Class Codes: G06F 3/167 Audio in a user interface, e.g. using voice commands for navigating, audio feedback
• CPC Class Codes: G10L 15/1822 Parsing for meaning understanding
• CPC Class Codes: G10L 15/22 Procedures used during a speech recognition process, e.g. man-machine dialogue
• CPC Class Codes: G10L 15/30 Distributed recognition, e.g. in client-server systems, for mobile phones or network applications
• CPC Class Codes: G10L 2015/223 Execution procedure of a spoken command
• CPC Class Codes: G10L 2015/228 of application context
• CPC Class Codes: G10L 21/06 Transformation of speech into a non-audible representation, e.g. speech visualisation or speech processing for tactile aids G10L15/26 takes precedence

Abstract

An electronic device may capture a voice command from a user. The electronic device may store contextual information about the state of the electronic device when the voice command is received. The electronic device may transmit the voice command and the contextual information to computing equipment such as a desktop computer or a remote server. The computing equipment may perform a speech recognition operation on the voice command and may process the contextual information. The computing equipment may respond to the voice command. The computing equipment may also transmit information to the electronic device that allows the electronic device to respond to the voice command.

Claims

1. A method for processing voice commands, comprising:

at an electronic device with one or more processors and memory;

receiving a voice command containing a request for media;

storing contextual information of the electronic device, the contextual information related to a media playback operation occurring at the electronic device while receiving the voice command;

after receiving the voice command and storing the contextual information, transmitting at least a portion of the voice command and the stored contextual information from the electronic device to remote computing equipment; and

receiving, from the remote computing equipment, media results obtained based on the voice command and the contextual information.

2. The method of claim 1, wherein the media playback operation comprises a media player application of the electronic device playing a media object while receiving the voice command.

3. The method of claim 2, wherein the media results include a second media object having a same attribute as the media object being played by the media player application.

4. The method of claim 2, wherein the contextual information includes one or more attributes related to the media object.

5. The method of claim 1, wherein the contextual information comprises a media attribute related to the media playback operation.

6. The method of claim 5, wherein the media attribute comprises a track name.

7. The method of claim 5, wherein the media attribute comprises a title.

8. The method of claim 5, wherein the media attribute comprises an artist name.

9. The method of claim 5, wherein the media attribute comprises an album name.

10. The method of claim 5, wherein the media attribute comprises a year.

11. The method of claim 5, wherein the media attribute comprises a genre.

12. The method of claim 1, wherein the voice command does not explicitly describe an attribute required to satisfy the request for media.

13. The method of claim 1, wherein the request for media is a request to search for media objects.

14. The method of claim 1, wherein the voice command includes an ambiguous term that refers to the contextual information.

15. The method of claim 1, wherein the contextual information comprises media files on the electronic device.

16. The method of claim 1, wherein the contextual information comprises information associated with a web-browsing operation of an Internet browsing application of the electronic device occurring while receiving the voice command.

17. The method of claim 1, wherein the contextual information comprises a location of the electronic device when receiving the voice command.

18. The method of claim 1, wherein the contextual information comprises information from an application of the electronic device available while receiving the voice command.

19. The method of claim 1, wherein the contextual information comprises information associated with an operation occurring in a background of the electronic device while receiving the voice command.

20. The method of claim 1, further comprising:

prior to receiving the voice command;

receiving a second voice command containing a second request for media;

storing second contextual information of the electronic device, the second contextual information related to the second voice command;

after receiving the second voice command and storing the second contextual information, transmitting at least a portion of the second voice command and the stored second contextual information from the electronic device to the remote computing equipment; and

receiving, from the remote computing equipment, second media results obtained based on the second voice command and the second contextual information.

21. The method of claim 20, wherein the contextual information comprises the second media results.

22. The method of claim 20, wherein the media playback operation comprises a media player application of the electronic device playing a media object while receiving the voice command, and wherein the second media results include information associated with the media object.

23. The method of claim 20, wherein the contextual information comprises information associated with the second request for media.

24. A electronic device comprising:

a microphone;

one or more processors; and

memory storing one or more programs configured to be executed by the one or more processors, the one or more programs including instructions for;

receiving a voice command containing a request for media;

storing contextual information of the electronic device, the contextual information related to a media playback operation occurring at the electronic device while receiving the voice command;

after receiving the voice command and storing the contextual information, transmitting at least a portion of the voice command and the stored contextual information from the electronic device to remote computing equipment; and

receiving, from the remote computing equipment, media results obtained based on the voice command and the contextual information.

25. The device of claim 24, wherein the media playback operation comprises a media player application of the electronic device playing a media object while receiving the voice command.

26. The device of claim 25, wherein the media results include a second media object having a same attribute as the media object being played by the media player application.

27. The device of claim 25, wherein the contextual information includes one or more attributes related to the media object.

28. The device of claim 24, wherein the contextual information comprises a media attribute related to the media playback operation.

29. The device of claim 28, wherein the media attribute comprises a track name.

30. The device of claim 28, wherein the media attribute comprises a title.

31. The device of claim 28, wherein the media attribute comprises an artist name.

32. The device of claim 28, wherein the media attribute comprises an album name.

33. The device of claim 28, wherein the media attribute comprises a year.

34. The device of claim 28, wherein the media attribute comprises a genre.

35. The device of claim 24, wherein the voice command does not explicitly describe an attribute required to satisfy the request for media.

36. The device of claim 24, wherein the request for media is a request to search for media objects.

37. The device of claim 24, wherein the voice command includes an ambiguous term that refers to the contextual information.

38. The device of claim 24, wherein the contextual information comprises media files on the electronic device.

39. The device of claim 24, wherein the contextual information comprises information associated with a web-browsing operation of an Internet browsing application of the electronic device occurring while receiving the voice command.

40. The device of claim 24, wherein the contextual information comprises a location of the electronic device when receiving the voice command.

41. The device of claim 24, wherein the contextual information comprises information from an application of the electronic device available while receiving the voice command.

42. The device of claim 24, wherein the contextual information comprises information associated with an operation occurring in a background of the electronic device while receiving the voice command.

43. The device of claim 24, wherein the one or more programs include instructions for:

prior to receiving the voice command;

receiving a second voice command containing a second request for media;

storing second contextual information of the electronic device, the second contextual information related to the second voice command;

after receiving the second voice command and storing the second contextual information, transmitting at least a portion of the second voice command and the stored second contextual information from the electronic device to the remote computing equipment; and

receiving, from the remote computing equipment, second media results obtained based on the second voice command and the second contextual information.

44. The device of claim 43, wherein the contextual information comprises the second media results.

45. The device of claim 43, wherein the media playback operation comprises a media player application of the electronic device playing a media object while receiving the voice command, and wherein the second media results include information associated with the media object.

46. The device of claim 43, wherein the contextual information comprises information associated with the second request for media.

47. A non-transitory computer-readable storage medium storing one or more programs, the one or more programs comprising instructions, which when executed by an electronic device in communication with remote computing equipment over a communications path, cause the device to:

receive a voice command containing a request for media;

store contextual information of the electronic device, the contextual information related to a media playback operation occurring at the electronic device while receiving the voice command;

after receiving the voice command and storing the contextual information, transmit at least a portion of the voice command and the stored contextual information from the electronic device to the remote computing equipment; and

receive, from the remote computing equipment, media results obtained based on the voice command and the contextual information.

48. The computer-readable storage medium of claim 47, wherein the instructions further cause the device to:

prior to receiving the voice command;

receive a second voice command containing a second request for media;

store second contextual information of the electronic device, the second contextual information related to the second voice command;

after receiving the second voice command and storing the second contextual information, transmit at least a portion of the second voice command and the stored second contextual information from the electronic device to the remote computing equipment; and

receive, from the remote computing equipment, second media results obtained based on the second voice command and the second contextual information.