INTELLIGENT AUTOMATED ASSISTANT FOR TV USER INTERACTIONS

• US 20170230709A1
• Filed: 04/24/2017
• Published: 08/10/2017
• Est. Priority Date: 06/30/2014
• Status: Active Grant

• Current Assignee: Apple Inc.
• Original Assignee: Apple Inc.
• Inventors: RUSSELL, Jonathan H., VAN OS, Marcel, SADDLER, Harry J., NAPOLITANO, Lia T., LISTER, Patrick M., DASARI, Rohit
• Granted Patent: US 10,904,611 B2
• Time in Patent Office: Days
• CPC Class Codes: G06F 16/73 Querying
• CPC Class Codes: G06F 16/738 Presentation of query results
• CPC Class Codes: G06F 16/7867 using information manually generated, e.g. tags, keywords, comments, title and artist information, manually generated time, location and usage information, user ratings
• CPC Class Codes: G06F 3/167 Audio in a user interface, e.g. using voice commands for navigating, audio feedback
• CPC Class Codes: G09C 1/00 Apparatus or methods whereby a given sequence of signs, e.g. an intelligible text, is transformed into an unintelligible sequence of signs by transposing the signs or groups of signs or by replacing them by others according to a predetermined system cryptographic typewriters G09C3/00
• CPC Class Codes: G10L 15/1815 Semantic context, e.g. disambiguation of the recognition hypotheses based on word meaning
• CPC Class Codes: G10L 15/22 Procedures used during a speech recognition process, e.g. man-machine dialogue
• CPC Class Codes: G10L 15/26 Speech to text systems G10L15/08 takes precedence
• CPC Class Codes: G10L 17/00 Speaker identification or verification techniques
• CPC Class Codes: G10L 17/22 Interactive procedures; Man-machine interfaces
• CPC Class Codes: G10L 2015/223 Execution procedure of a spoken command
• CPC Class Codes: H04N 21/2541 Rights Management protecting software against unauthorised usage in a vending or licensing environment G06F21/10; security in data switching network management H04L41/28; security management or policies for network security H04L63/20; access security in wireless networks H04W12/08
• CPC Class Codes: H04N 21/25891 being end-user preferences retrieval of video data in a video database based on user preferences G06F16/739; arrangements for recognizing users' preferences H04H60/46; user profiles in network data switching protocols H04L67/306; processing of user preferences or user profiles in wireless networks H04W8/18
• CPC Class Codes: H04N 21/4122 additional display device, e.g. video projector digital output for controlling a plurality of local displays G06F3/1423
• CPC Class Codes: H04N 21/41265 having a remote control device for bidirectional communication between the remote control device and client device unidirectional remote control device H04N21/42204
• CPC Class Codes: H04N 21/42203 sound input device, e.g. microphone
• CPC Class Codes: H04N 21/42222 Additional components integrated in the remote control device, e.g. timer, speaker, sensors for detecting position, direction or movement of the remote control, microphone or battery charging device
• CPC Class Codes: H04N 21/4312 involving specific graphical features, e.g. screen layout, special fonts or colors, blinking icons, highlights or animations
• CPC Class Codes: H04N 21/4316 for displaying supplemental content in a region of the screen, e.g. an advertisement in a separate window
• CPC Class Codes: H04N 21/4394 involving operations for analysing the audio stream, e.g. detecting features or characteristics in audio streams arrangements characterised by components specially adapted for monitoring, identification or recognition of audio in broadcast systems H04H60/58
• CPC Class Codes: H04N 21/4438 : Window management, e.g. event handling following interaction with the user interface
• CPC Class Codes: H04N 21/4532 : involving end-user characteristics, e.g. viewer profile, preferences monitoring of user activities for profile generation for accessing a video database G06F16/739; user profiles in network data switching protocols H04L67/306; processing of user preferences or user profiles in wireless networks H04W8/18
• CPC Class Codes: H04N 21/4627 : Rights management associated to the content security in data switching network management H04L41/28; security management or policies for network security H04L63/20; access security in wireless networks H04W12/08
• CPC Class Codes: H04N 21/47214 : for content reservation or setting reminders; for requesting event notification, e.g. of sport results or stock market stock exchange data over packet-switching network H04L12/1804; push services over packet-switching network H04L12/1859; notification of incoming messages in packet switching networks H04L51/224
• CPC Class Codes: H04N 21/4722 : for requesting additional data associated with the content
• CPC Class Codes: H04N 21/482 : End-user interface for program selection broadcast systems using EPGs H04H60/72
• CPC Class Codes: H04N 21/4826 : using recommendation lists, e.g. of programs or channels sorted out according to their score
• CPC Class Codes: H04N 21/4828 : for searching program descriptors retrieval of video data G06F16/739
• CPC Class Codes: H04N 21/4882 : for displaying messages, e.g. warnings, reminders arrangements for providing short real-time information to substation in data switching networks H04L12/1895
• CPC Class Codes: H04N 21/84 : Generation or processing of descriptive data, e.g. content descriptors systems specially adapted for using meta-information in broadcast systems H04H60/73
• CPC Class Codes: H04N 21/858 : Linking data to content, e.g. by linking an URL to a video object, by creating a hotspot

Abstract

Systems and processes are disclosed for controlling television user interactions using a virtual assistant. In an example process, a virtual assistant can interact with a television set-top box to control content shown on a television display. Speech input for the virtual assistant can be received from a device with a microphone. The speech input can comprise a query associated with content shown on the television display. A user intent of the query can be determined based on one or more of the content shown on the television display and a viewing history of media content. A result of the query can be caused to be displayed based on the determined user intent.

Claims

1. An electronic device, comprising:

one or more processors; and

memory storing one or more programs configured to be executed by the one or more processors, the one or more programs including instructions for;

receiving speech input from a user, wherein the speech input comprises a query associated with content shown on a television display;

determining a user intent of the query based on one or more of the content shown on the television display and a viewing history of media content; and

causing display of a result of the query based on the determined user intent.

2. The device of claim 1, wherein the speech input is received at a remote control.

3. The device of claim 1, wherein the speech input is received at a mobile telephone.

4. The device of claim 1, wherein the result of the query is displayed on the television display.

5. The device of claim 1, wherein the content shown on the television display comprises a movie.

6. The device of claim 1, wherein the content shown on the television display comprises a television show.

7. The device of claim 1, wherein the content shown on the television display comprises a sporting event.

8. The device of claim 1, wherein the query comprises a request for information about a person associated with the content shown on the television display, and the result of the query comprises information about the person.

9. The device of claim 8, wherein the result of the query comprises media content associated with the person.

10. The device of claim 9, wherein the media content comprises one or more of a movie, a television show, or a sporting event associated with the person.

11. The device of claim 1, wherein the query comprises a request for information about a character in the content shown on the television display, and the result of the query comprises information about the character or information about the actor who plays the character.

12. The device of claim 11, wherein the result of the query comprises media content associated with the actor who plays the character.

13. The device of claim 12, wherein the media content comprises one or more of a movie, a television show, or a sporting event associated with the actor who plays the character.

14. The device of claim 1, wherein the one or more programs further include instructions for:

determining the result of the query based on metadata associated with the content shown on the television display or the viewing history of media content.

15. The device of claim 14, wherein the metadata comprises one or more of a title, a description, a list of characters, a list of actors, a list of players, a genre, or a display schedule associated with the content shown on the television display or the viewing history of media content.

16. The device of claim 1, wherein the content shown on the television display comprises a list of media content, and the query comprises a request to display one of the items in the list.

17. The device of claim 16, wherein the content shown on the television display further comprises an item in the list of media content having focus;

and

wherein determining the user intent of the query comprises identifying the item having focus.

18. The device of claim 1, wherein the one or more programs further include instructions for:

determining the user intent of the query based on menu or search content recently displayed on the television display.

19. The device of claim 18, wherein the content shown on the television display comprises a page of listed media, and the recently displayed menu or search content comprises a previous page of listed media.

20. The device of claim 1, wherein the content shown on the television display comprises one or more categories of media, and one of the one or more categories of media has focus.

21. The device of claim 20, wherein the one or more programs further include instructions for:

determining the user intent of the query based on the one of the one or more categories of media having focus.

22. The device of claim 20, wherein the categories of media comprise movies, television programs, and music.

23. The device of claim 1, wherein the one or more programs further include instructions for:

determining alternative media content for display based on one or more of a user preference, a show popularity, and a status of a live sporting event; and

causing display of a notification comprising the determined alternative media content.

24. A method for controlling television interaction using a virtual assistant, the method comprising:

at an electronic device;

receiving speech input from a user, wherein the speech input comprises a query associated with content shown on a television display;

determining a user intent of the query based on one or more of the content shown on the television display and a viewing history of media content; and

causing display of a result of the query based on the determined user intent.

25. A non-transitory computer-readable storage medium storing one or more programs configured to be executed by one or more processors of an electronic device with a display and a location sensor, the one or more programs including instructions for:

receiving speech input from a user, wherein the speech input comprises a query associated with content shown on a television display;

determining a user intent of the query based on one or more of the content shown on the television display and a viewing history of media content; and

causing display of a result of the query based on the determined user intent.