Multimedia search application for a mobile device

• US 9,536,520 B2
• Filed: 12/19/2013
• Issued: 01/03/2017
• Est. Priority Date: 09/14/2005
• Status: Active Grant

• Current Assignee: Microsoft Technology Licensing LLC (Microsoft Corporation)
• Original Assignee: AT&T Intellectual Property I LP (AT&T, Inc.)
• Inventors: Chang, Harry
• Primary Examiner(s): COLUCCI, MICHAEL C
• Application Number:
  US14/134,929
• Publication Number:
  US 20140108009A1
• Time in Patent Office: 1,111 Days
• Field of Search: 704/231, 704/9, 704/275, 704/270.1, 704/270, 704/255, 725/88, 725/58, 725/48, 725/29, 725/13, 455/575.1, 455/556.1, 455/445, 455/12.1, 386/83, 386/259, 345/169, 345/1.3
• US Class Current: 1/1
• CPC Class Codes: G10L 15/08 Speech classification or search
• CPC Class Codes: G10L 15/26 Speech to text systems G10L15/08 takes precedence
• CPC Class Codes: H04N 21/2543 Billing , e.g. for subscription services payment schemes, architectures or protocols G06Q20/00; e-commerce G06Q30/00; arrangements for billing for the use of broadcast information or broadcast-related information H04H60/21
• CPC Class Codes: H04N 21/41407 embedded in a portable device, e.g. video client on a mobile phone, PDA, laptop constructional details of equipment or arrangements specially adapted for portable computer application G06F1/1626; arrangements specially adapted for mobile receivers in broadcast systems H04H20/57
• CPC Class Codes: H04N 21/42203 sound input device, e.g. microphone
• CPC Class Codes: H04N 21/47202 for requesting content on demand, e.g. video on demand
• CPC Class Codes: H04N 21/47211 for requesting pay-per-view content payment schemes payment architectures or payment protocols G06Q20/00, G07F
• CPC Class Codes: H04N 21/4828 for searching program descriptors retrieval of video data G06F16/739

Abstract

In accordance with one aspect of the present invention, a method selects a program from a library of programs. A user selection is determined based upon a voice command, and the program is presented at a display device in accordance with the voice command. In accordance with another aspect of the present invention, a system selects a program from a library of programs. The system includes a processor that determines a user selection based upon a voice command, and also includes a display device that presents the program in accordance with the voice command. In accordance with yet another embodiment of the present invention, a computer-readable medium contains a set of instructions that when executed by a processor cause the processor to determine a user selection based upon a voice command and to command a display device to present the program, in accordance with the voice command.

Claims

1. A mobile device, comprising:

a display screen configured to display multimedia content;

a microphone to detect user speech of a user and provide an audio signal indicative of the user speech;

an automated speech recognition engine;

a speech enabled application configured to;

provide the audio signal to the automated speech recognition engine;

receive from the automated speech recognition engine, a search term;

provide the search term to a multimedia search engine;

obtain from the multimedia search engine, a search result indicative of a particular program; and

obtain, from a multimedia library, at least a portion of the particular program; and

locally stored data indicative of particular programs indicated in previous search results obtained from the multimedia search engine;

wherein the automated speech recognition engine comprises a local speech recognition engine that is initially inactive and subsequently activated responsive to a number of locally stored speech recognition grammar entries exceeding an activation threshold.

2. The mobile device of claim 1, wherein the automated speech recognition engine, once activated, attempts to recognize the user speech.

3. The mobile device of claim 1, wherein the mobile device includes a voice library and wherein a search result obtained during voice browsing is based, at least in part, on matching the user speech with a voice included in the voice library.

4. The mobile device of claim 1, wherein the speech enabled application supports a rating guard configured to block multimedia content that violates a rating restriction.

5. The mobile device of claim 1, wherein the operations include:

querying the user by generating and displaying menus.

6. The mobile device of claim 5, wherein the operations include:

querying the user based, at least in part, on a keypad entry provided by the user in response to at least one of the menus.

7. The mobile device of claim 1, wherein the mobile device is configured to:

display the portion of the particular program on the display screen.

8. The mobile device of claim 1, wherein the multimedia library includes a first version of the particular program and a second version of the particular program, wherein the first version is suitable for a smaller display screen and the second version is suitable for a larger display screen and wherein the mobile device is configured to receive and display the first version in response to sending the search query.

9. The mobile device of claim 1, wherein the automated speech recognition engine is configured to record the user speech.

10. The mobile device of claim 1, wherein the operations include:

automatically updating a search result obtained from voice browsing the multimedia library.

11. A computer readable medium including processor executable program instructions that, when executed by a processor, cause the processor to perform operations, comprising:

initializing a local speech recognizer of a mobile device with an inactive status;

receiving, from a microphone of the mobile device, an audio signal indicative of user speech, including a search term, associated with a user and detected by the microphone;

providing the audio signal to an automated speech recognizer to identify a search term, wherein providing the audio signal to the automated speech recognizer includes;

determining a status of a local speech recognizer;

responsive to determining the local speech recognizer has an active status, providing the audio signal to the local speech recognizer; and

responsive to determining the local speech recognizer has the inactive status, providing the audio signal to a network speech recognizer via a communication network;

sending the search term to a multimedia search engine configured to search, based on the search term, a multimedia library;

receiving a search result indicating a particular multimedia program associated with the search term; and

displaying the search result on a display screen of the mobile device;

storing vocabulary entries, based on words and phrases in at least one of;

the search term and the search result, to a local vocabulary; and

activating the local speech recognizer when a number of the vocabulary entries exceeds a vocabulary threshold.

12. The computer readable medium of claim 11, wherein the local vocabulary includes data indicative of previously identified multimedia programs.

13. The computer readable medium of claim 11, wherein the operations include:

requesting an initial voice search page from a web server associated with the multimedia search engine; and

receiving the initial voice search page from the web server.

14. The computer readable medium of claim 13, wherein:

a request for the initial voice search page includes identification information identifying at least one of;

the user, the mobile device, a wireless service account associated with the mobile device, and a multimedia service account associated with the user; and

receipt of the initial voice search page is indicative of a verification of the identification information.

15. The computer readable medium of claim 11, wherein:

the search result comprises an intermediate search result and wherein the operations include;

sending a subsequent search query to the multimedia search engine; and

receiving a subsequent search result based on the subsequent search query.

16. The computer readable medium of claim 15, wherein the intermediate search result comprises a menu page.

17. The computer readable medium of claim 15, wherein the intermediate search is indicative of categories of multimedia content.

18. The computer readable medium of claim 15, wherein the operations include:

generating the subsequent search query based on a combination of;

an intermediate audio signal indicative of intermediate user speech; and

a keypad user input indicative of an entry by the user of an element of a keypad of the mobile device.

19. The computer readable medium of claim 11, wherein the multimedia search engine resides on a provider network configured to:

provide multimedia content services to the user; and

deliver multimedia content to a set top box associated with the user via a broadband network.

20. The computer readable medium of claim 11, wherein the operations include:

blocking multimedia content that violates a predetermined rating restriction.

21. The computer readable medium of claim 11, wherein the operations include:

storing the particular multimedia program locally for viewing.

22. The computer readable medium of claim 21, wherein the operations include:

determining whether to permit viewing of the locally stored particular multimedia program based on a condition selected from;

a time period condition and number of viewings condition.

23. The computer readable medium of claim 11, wherein the operations include:

automatically updating the search result.

24. The computer readable medium of claim 11, wherein the operations include:

requesting delivery of the particular program.

25. The computer readable medium of claim 24, wherein requesting the delivery of the particular multimedia program includes identifying a display device and requesting delivery of the particular multimedia program to the display device.

26. The computer readable medium of claim 25, wherein the display device is selected from:

a set top box associated with a multimedia service provider and a wireless device associated with a wireless service provider.

27. The computer readable medium of claim 11, wherein the network speech recognizer and the multimedia search engine are included in a network server of the multimedia service provider.

28. The computer readable medium of claim 27, wherein the communication network comprises a wireless communication network.

29. The computer readable medium of claim 28, wherein the wireless communication network comprises a wireless telephone network operated by a wireless telephone service provider.

30. The computer readable medium of claim 29, wherein a common provider includes the wireless telephone service provider and the multimedia service provider.

31. A multimedia search method, comprising:

initializing a local speech recognizer of a mobile device with an inactive status;

receiving, from a microphone of the mobile device, an audio signal indicative of user speech, including a search term, associated with a user and detected by the microphone;

providing the audio signal to an automated speech recognizer to identify a search term, wherein providing the audio signal to the automated speech recognizer includes;

determining a status of a local speech recognizer;

responsive to determining the local speech recognizer has an active status, providing the audio signal to the local speech recognizer; and

responsive to determining the local speech recognizer has the inactive status, providing the audio signal to a network speech recognizer via a communication network;

sending the search term to a multimedia search engine configured to search, based on the search term, a multimedia library;

receiving a search result indicating a particular multimedia program associated with the search term; and

displaying the search result on a display screen of the mobile device;

storing vocabulary entries, based on words and phrases in at least one of;

the search term and the search result, to a local vocabulary; and

activating the local speech recognizer when a number of the vocabulary entries exceeds a vocabulary threshold.