VOICE CONTROL OF MULTIMEDIA CONTENT

• US 20100318357A1
• Filed: 10/22/2009
• Published: 12/16/2010
• Est. Priority Date: 04/30/2004
• Status: Abandoned Application

• Current Assignee: Vulcan SA (Ampco-Pittsburgh Corporation)
• Original Assignee: Vulcan SA (Ampco-Pittsburgh Corporation)
• Inventors: Stark, Korina J.B., Istvan, Anthony F., Budd, Robin
• Application Number:
  US12/603,633
• Publication Number:
  US 20100318357A1
• Time in Patent Office: Days
• US Class Current: 704/251
• CPC Class Codes: H04N 21/4147 PVR [Personal Video Recorder] H04N5/76 takes precedence ; arrangements for broadcast with accumulation-type receivers H04H20/40
• CPC Class Codes: H04N 21/42204 User interfaces specially adapted for controlling a client device through a remote control device; Remote control devices therefor constructive details of casings for the remote control device H01H9/0235; user interfaces for controlling a tuning device of a television receiver through a remote control H03J9/00; remote control of peripheral devices connected to a television receiver through the remote control device of the television receiver H04B1/205
• CPC Class Codes: H04N 21/42206 characterized by hardware details
• CPC Class Codes: H04N 21/42222 Additional components integrated in the remote control device, e.g. timer, speaker, sensors for detecting position, direction or movement of the remote control, microphone or battery charging device
• CPC Class Codes: H04N 21/4223 Cameras H04N23/00 takes precedence
• CPC Class Codes: H04N 21/4325 by playing back content from the storage medium reproduction of recorded television signals H04N5/76; reproduction of recorded television signals H04N9/79
• CPC Class Codes: H04N 21/4334 Recording operations recording of a television signal H04N5/76; arrangements for recording or accumulating broadcast information or broadcast-related information H04H60/27
• CPC Class Codes: H04N 21/43615 Interfacing a Home Network, e.g. for connecting the client to a plurality of peripherals home Audio Video Interoperability [HAVI] data switching networks H04L12/2805
• CPC Class Codes: H04N 21/4394 involving operations for analysing the audio stream, e.g. detecting features or characteristics in audio streams arrangements characterised by components specially adapted for monitoring, identification or recognition of audio in broadcast systems H04H60/58
• CPC Class Codes: H04N 21/4622 Retrieving content or additional data from different sources, e.g. from a broadcast channel and the Internet web site content organization and management for information retrieval from the Internet G06F16/958; transmission by internet of broadcast information H04H60/82; stock exchange data over packet-switching network H04L12/1804; push services including data channel over packet-switching network H04L12/1859
• CPC Class Codes: H04N 21/47 End-user applications
• CPC Class Codes: H04N 21/47214 for content reservation or setting reminders; for requesting event notification, e.g. of sport results or stock market stock exchange data over packet-switching network H04L12/1804; push services over packet-switching network H04L12/1859; notification of incoming messages in packet switching networks H04L51/224
• CPC Class Codes: H04N 21/47217 for controlling playback functions for recorded or on-demand content, e.g. using progress bars, mode or play-point indicators or bookmarks specific graphical features in visual interfaces H04N21/4312
• CPC Class Codes: H04N 21/4722 for requesting additional data associated with the content
• CPC Class Codes: H04N 21/4751 for defining user accounts, e.g. accounts for children
• CPC Class Codes: H04N 21/482 End-user interface for program selection broadcast systems using EPGs H04H60/72
• CPC Class Codes: H04N 21/4823 using a channel name
• CPC Class Codes: H04N 21/4828 for searching program descriptors retrieval of video data G06F16/739

Abstract

Techniques are described for managing various types of content in various ways, such as based on voice commands or other voice-based control instructions provided by a user. In some situations, at least some of the content being managed includes content of a variety of types, such as music and other audio information, photos, images, non-television video information, videogames, Internet Web pages and other data, etc., which may be managed via the voice controls in a variety of ways, such as to allow a user to locate and identify content of potential interest, to schedule recordings of selected content, to manage previously recorded content (e.g., to play or delete the content), to control live television, etc. This abstract is provided to comply with rules requiring it, and is submitted with the intention that it will not be used to interpret or limit the scope or meaning of the claims.

Claims

1-20. -20. (canceled)

21. A method for concurrently controlling presentation of multiple types of content on multiple presentation devices using voice commands, the method comprising:

at a computing device in a home environment that controls presentation of content, receiving multiple pieces of content of multiple types from at least one content server system and receiving metadata information about the received pieces of content, the multiple types of content including at least one of audio content, image content, and video content; and

under control of the computing device,

receiving multiple voice commands from a user of the computing device, wherein each voice command contains one or more criteria for selecting one or more pieces of content to be controlled, an instruction related to a type of control, and an indication of a type of content;

for each of the multiple voice commands,

analyzing the voice command to identify the one or more criteria, the instruction, and the indicated type of content;

selecting from multiple presentation devices a presentation device at which to perform the identified instruction of the voice command, wherein the presentation device is selected based at least in part on the identified type of content;

determining a set of allowable instructions based on a current state of the selected presentation device, wherein the set of allowable instructions is a subset of instructions that are allowed based on the current state of the selected presentation device;

determining whether the identified instruction of the voice command corresponds to one of the determined set of allowable instructions;

when the identified instruction of the voice command corresponds to one of the determined set of allowable instructions,

using the metadata information to identify one or more of the received pieces of content that correspond to the identified one or more criteria, and

performing at the selected presentation device the identified instruction of the voice command on at least one of the identified pieces of content; and

when the identified instruction of the voice command does not correspond to one of the determined set of allowable instructions, notifying the user that the identified instruction of the voice command is not allowed based on the current state of the selected presentation device; and

displaying on a display device associated with the computing device a first user interface, wherein the first user interface is a voice command user interface that includes a control selectable by the user to display a second user interface, and wherein the second user interface is a user interface of the computing device,

wherein the identified instructions of each of the multiple voice commands are performed concurrently at the selected presentation devices, and wherein the performing of the identified instructions includes sending the identified instructions to the selected presentation devices for use in controlling presentation of the at least one identified piece of content.

22. The method of claim 21 wherein the computing device is one of a digital video recorder (“

DVR”

) device, a set-top box device and a media center device,

wherein the user is a current one of multiple users of the computing device,

wherein the current user is at a first location in the home environment and wherein the computing device is located at a second distinct location in the home environment,

wherein the current user provides the voice command to a remote control device that is located with the current user at the first location,

wherein the receiving of the voice command by the computing device is in response to transmitting of the voice command by the remote control device,

wherein the analyzing of the voice command includes performing speech recognition in a manner specific to the current user and uses current state information for the computing device and is performed so as to identify one or more words for the instruction and one or more words for the criteria, and

wherein the performing of the identified instruction on at least one of the identified pieces of content includes presenting information to the current user that indicates the one or more identified pieces of content.

23. The method of claim 22 wherein the one or more words for the criteria include one or more descriptive words,

wherein the instruction is to search for one or more corresponding pieces of content that satisfy the criteria by matching those descriptive words, and

wherein the identifying of the one or more received pieces of content by using the metadata information includes performing the search.

24. The method of claim 22 wherein the presenting to the current user of the information that indicates the one or more identified pieces of content includes:

transmitting the information to the display device,

receiving an additional voice command from a user that selects one of the identified pieces of content, and

in response to receiving the additional voice command, presenting the one identified piece of content.

25. The method of claim 21 wherein the computing device is a digital video recorder (“

DVR”

) device,

wherein the at least one identified piece of content is streamed or broadcasted content that will be received at a future time,

wherein the identified instruction indicates to perform a recording, and

wherein the performing of the identified instruction by the DVR device includes recording the at least one identified piece of content at the future time.

26. The method of claim 21 wherein the computing device is a media center device, wherein the user is local to the set-top box device in the home environment,

wherein the at least one identified piece of content includes audio information that is currently available for presentation, and

wherein the performing of the identified instruction by the media center device includes initiating current presentation of the at least one identified piece of content to the user on at least one audio presentation device in the home environment.

27. The method of claim 21 further comprising, before the performing of the identified instruction on the at least one identified piece of content, displaying feedback to the user that indicates the instruction and the criteria that are identified from the analyzing of the voice command and modifying at least one of the instruction and the criteria based on additional information received from the user.

28. The method of claim 21 wherein the computing device receives the voice command from a remote control device to which the user had provided the voice command, and

wherein the analyzing of the voice command includes identifying one or more words for the instruction and determining the identified instruction by mapping the identified words to one of multiple predefined instructions that are supported by the computing device and/or by an associated presentation device in such a manner that the remote control device can transmit signals to the computing device and/or the associated presentation device that correspond to the predefined instructions based on manual operation by the user of one or more controls on the remote control device.

29. The method of claim 21 wherein a piece of content is being presented to the user at a time of the receiving of the voice command, the piece of content having multiple portions of content to be presented over a period of time,

wherein the identified one or more criteria are an indication of the piece of content being presented, and

wherein the identified instruction is to change the portion of the piece of content that is currently being presented.

30. The method of claim 29 wherein the voice command includes one or more words that specify an amount of time or that indicate a selected content portion of the piece of content that is distinct from a portion being presented at the time of the receiving of the voice command, and

wherein the performing of the identified instruction includes modifying presentation of the piece of content such that presentation is initiated of the selected content portion or such that presentation is initiated of a portion of the piece of content that differs from the portion currently being presented by the specified amount of time.

31. A computer-readable storage medium whose contents enable a computing device to concurrently manage content on multiple presentation devices based on voice-based control instructions, by performing a method comprising:

receiving metadata information for multiple pieces of content;

receiving multiple voice-based control instructions generated by a user, wherein the multiple voice-based control instructions include;

a first voice-based control instruction that relates to grouping two or more of the multiple pieces of content together; and

a second voice-based control instruction that relates to a type of control of the two or more pieces of content;

in response to receiving the voice-based control instructions,

identifying one or more actions to be performed regarding the two or more pieces of content, the identifying based at least in part on the received voice-based control instructions and based at least in part on the received metadata information;

selecting from multiple presentation devices at least one presentation device at which to perform the identified one or more actions;

determining a set of allowable actions based on a current state of the selected at least one presentation device, wherein the set of allowable actions is a subset of all actions;

determining whether the identified one or more actions correspond to one of the determined set of allowable actions;

for the identified one or more actions that correspond to one of the determined set of allowable actions, performing at the selected at least one presentation device the identified one or more actions regarding the two or more pieces of content; and

for the identified one or more actions that do not correspond to one of the determined set of allowable actions, notifying the user that the identified one or more actions are not allowed based on the current state of the selected at least one presentation device.

32. The computer-readable storage medium of claim 31 wherein the multiple pieces of content are of multiple types, wherein the method further comprises:

identifying at least one type of content to which the received control instructions relate;

identifying the one or more pieces of content based at least in part on the identified at least one type of content; and

determining a presentation device associated with the identified at least one type of content,

wherein the performing of the identified one or more actions regarding the one or more pieces of content includes forwarding information to the determined presentation device to cause performance of the identified one or more actions regarding the identified pieces of content.

33. The computer-readable storage medium of claim 31 wherein the computing device is one or more of a digital video recorder (“

DVR”

) device, a set-top box device, and a media center device, and

wherein the presentation device is one or more digital video recorder (“

DVR”

) devices, set-top box devices, media center devices, speakers, music players, gaming device, image display devices, cameras, videophones, Internet appliance devices, cellular telephones, or general purpose computing devices.

34. The computer-readable storage medium of claim 31 wherein the computer-readable storage medium is a memory of the computing device.

35. The computer-readable storage medium of claim 31 wherein the contents are instructions that when executed cause the computing device to perform the method.

36. A computing device configured to manage multiple types of non-television content on multiple presentation devices based on voice commands, comprising:

at least one input mechanism configured to receive via a cell phone or landline phone connection multiple voice commands generated by a user that relate to a type of control of one or more of multiple types of content; and

a voice command processing system configured to analyze the received voice commands and, for each of the received voice commands, to;

identify one or more actions to be performed regarding one or more pieces of content of at least one of the multiple types based at least in part on metadata information about those pieces of content, wherein the one or more actions are identified based at least in part on user-specific information, and wherein the user-specific information includes at least one of user preferences, custom filters, prior searches, and prior recordings or viewings by the user;

select from multiple presentation devices a presentation device at which to perform the identified one or more actions;

determine a set of allowable actions based on a current state of the selected presentation device, wherein the set of allowable actions is a subset of actions that are allowed based on the current state of the selected presentation device;

determine whether the identified one or more actions correspond to one of the determined set of allowable actions;

for the identified one or more actions that correspond to one of the determined set of allowable actions, initiate performance of the identified one or more actions regarding the one or more items of content at the selected presentation device;

for the identified one or more actions that do not correspond to one of the determined set of allowable actions, notify the user that the identified one or more actions are not allowed based on the current state of the selected presentation device; and

display on a display device coupled to the computing device a voice command processing system user interface, wherein the voice command processing system user interface includes a user-selectable control configured to display a computing device user interface,

wherein the identified one or more actions of each of the multiple voice commands are performed substantially concurrently at the selected presentation devices.

37. The computing device of claim 36 wherein the at least one input mechanism includes one or more of a microphone, a network interface connection, a direct physical connection from one or more other devices, and a connection to allow wireless communication from one or more other devices.

38. The computing device of claim 36 wherein the voice command processing system is further configured to:

receive one or more voice annotations from the user, each of the voice annotations providing descriptive information related to a piece of content; and

initiate storage of each of the voice annotations in a manner associated with the piece of content for the voice annotation.

39. The computing device of claim 36 wherein the one or more pieces of content include at least one of music recordings, non-music audio recordings, images, and video recordings,

wherein the one or more pieces of content include streamed content and non-streamed content, and

wherein the initiating performance of the identified one or more actions regarding the one or more items of content includes sending an identified action or the one or more items of content to the selected presentation device, the selected presentation device comprising at least one of a speaker device, music player device, gaming device, image display device, cellphone device, Internet appliance device, camera, videophone, and general purpose computing device.

40. The computing device of claim 36 wherein the voice command processing system user interface further includes an element that provides textual and audio feedback to the user, wherein the provided feedback adapts as the user:

presses a microphone button of the computing device,

speaks,

releases the microphone button, and

observes results displayed by the computing device.