Voice activated application service architecture and delivery

• US 8,301,452 B2
• Filed: 08/12/2009
• Issued: 10/30/2012
• Est. Priority Date: 08/12/2008
• Status: Active Grant

• Current Assignee: Microsoft Technology Licensing LLC (Microsoft Corporation)
• Original Assignee: Ditech Networks, Inc. (Microsoft Corporation)
• Inventors: Simpson, Todd, Chandler, Nathan, Hawrylyshen, Alan, Madison, Chad, Lambert, Mike, Brown, Karl
• Primary Examiner(s): Godbold, Douglas
• Application Number:
  US12/540,258
• Publication Number:
  US 20100042413A1
• Time in Patent Office: 1,175 Days
• Field of Search: 704/270, 704/270.1, 704/275, 379 671- 77, 379 8801- 8828
• US Class Current: 704/270
• CPC Class Codes: G10L 15/30 Distributed recognition, e.g. in client-server systems, for mobile phones or network applications
• CPC Class Codes: H04M 2201/40 using speech recognition speech recognition per se G10L15/00
• CPC Class Codes: H04M 3/42204 Arrangements at the exchange for service or number selection by voice at the terminal H04M1/27
• CPC Class Codes: H04M 3/4938 comprising a voice browser which renders and interprets, e.g. VoiceXML

Abstract

A system and method for retrieving distributed content responsive to voice data are disclosed. Voice data is transmitted from a source client device to media server which applies a mixing table to route the voice data to one or more destinations described by the mixing table. The media server also analyzes the received voice data for one or more events. Responsive to detecting an event, the media server communicates with an application server, which modifies the mixing table so that subsequent data is also routed to a media generator which analyzes voice data received after detection of the event for a command. The media generator communicates with the application server to retrieve data from a user data source, such as a website, associated with a detected command. The media generator produces an audio representation of the retrieved data which is communicated to the source client device via the media server.

Claims

1. A system for accessing data from a remote source responsive to voice data comprising:

a client device transmitting first voice data, the first voice data including an event;

a media server coupled to the client device, the media server receiving the first voice data from the client device and applying a mixing table to the first received voice data, wherein the mixing table detects the event and identifies one or more destinations for the first voice data transmitted by the client device, the one or more destinations including a second client device;

an application server coupled to the media server, the application server receiving an application request from the media server responsive to the media server detecting the event, initializing an application in response to the application request and modifying the mixing table to identify additional destinations for second voice data transmitted by the client device responsive to the media server detecting the event;

a media generator coupled to the application server and to the media server, the media generator receiving the second voice data from the media server in response to application of the modified mixing table, analyzing the second voice data for a command associated with the application and responsive to detecting the command, communicating a description of the command to the application server; and

a user data store coupled to the application server, the user data store receiving a request for data from the application server responsive to the application server receiving the description of the command from the media generator and transmitting data to the application server responsive to the request.

2. The system of claim 1, wherein the media server includes a mixer applying the mixing table to route voice data transmitted by the client device to one or more destinations.

3. The system of claim 2, wherein the one or more destinations comprise the media server and the media generator.

4. The system of claim 1, wherein the media server includes a media store storing predefined audio data.

5. The system of claim 4, wherein the predefined audio data comprises one or more audio tones or audio signals.

6. The system of claim 1, wherein the application server further comprises an interactive voice response (IVR) controller associating commands associated with the event with audio data.

7. The system of claim 1, wherein the application server includes an application data store, the application data store associating one or more actions with the description of the command.

8. The system of claim 1, wherein the application server includes a subscriber data store, the subscriber data store associating a device identified associated with the client device with a user.

9. The system of claim 8, wherein the subscriber data store further associates the user with one or more applications or with the user data store.

10. The system of claim 9, wherein the subscriber data store includes login information for the user data store associated with the user.

11. The system of claim 1, wherein the application server includes a session controller, the session controller generating session configuration information describing the client device, a destination device and the mixing table identifying the destination device as a destination for voice data transmitted by the client device.

12. The system of claim 11, wherein the session controller generates a modified mixing table responsive to the media server detecting the event, the modified mixing table identifying the media generator and the destination device as destinations for voice data transmitted by the client device.

13. The system of claim 1, wherein:

the media generator extracts data from the second voice data received from the media server responsive to detecting a second command and transmits the extracted data to the user data store.

14. The system of claim 1, wherein the media generator includes a speech recognition engine identifying the command in the voice data.

15. The system of claim 1, wherein the media generator includes a text to speech converter generating an audio representation of the data transmitted to the application server from the user data store responsive to the request for data.

16. The system of claim 15, wherein the media generator transmits the audio representation of the data transmitted to the application server from the user data store to the media server for communication to the client device.

17. The system of claim 1, wherein the media server includes an event detector analyzing voice data transmitted by the client device for a characteristic associated with detected events.

18. The system of claim 17, wherein the characteristic associated with the event comprises a keyword, a voice stress pattern, a time-domain sample or a frequency domain pattern.

19. A computer implemented method of accessing data from a remote source responsive to voice data, the method comprising:

receiving first voice data including an event from a first client device;

applying a mixing table to the first received voice data, wherein the mixing table detects the event and identifies one or more destinations for the first voice data, the one or more destinations including a second client device;

transmitting an application request in response to detecting the event, wherein the application request initializes an application and modifies the mixing table to identify additional destinations for second voice data received from the first client device; and

transmitting a portion of the second voice data in response to application of the modified mixing table, wherein the second voice data is analyzed for a command associated with the application and the command causes the application to receive data from a user data store.

20. The method of claim 19, wherein applying the mixing table routes voice data transmitted by the first client device to one or more destinations.

21. The method of claim 19, wherein responsive to the application request, generating session configuration information describing the first client device, a destination device and the mixing table identifying the destination device as a destination for voice data transmitted by the first client device.

22. The method of claim 21, wherein responsive to the detecting of the event, generating a modified mixing table, the modified mixing table identifying a media generator and the destination device as destinations to transmit voice data received from the first client device, the media generator receiving the portion of the second voice data.

23. A non-transitory computer-readable storage medium comprising stored computer program code for accessing data from a remote source response to voice data, the computer program code when executed causes one or more processors to:

receive first voice data including an event from a first client device;

apply a mixing table to the first received voice data, wherein the mixing table detects the event and identifies one or more destinations for the first voice data, the one or more destinations including a second client device;

transmit an application request in response to detecting the event, wherein the application request initializes an application and modifies the mixing table to identify additional destinations for second voice data received from the first client device; and

transmit a portion of the second voice data in response to application of the modified mixing table, wherein the second voice data is analyzed for a command associated with the application and the command causes the application to receive data from a user data store.

24. The computer-readable storage medium of claim 23, wherein the program code to apply the mixing table further comprises program code to route voice data received from the first client device to one or more destinations.

25. The computer-readable storage medium of claim 23, wherein the computer program code further comprises instructions to generate session configuration information describing the first client device, a destination device and the mixing table identifying the destination device as a destination for voice data transmitted by the first client device.

26. The computer-readable storage medium of claim 25, the computer program code comprising further instructions to generate a modified mixing table in response to the detecting of the event, the modified mixing table identifying a media generator and the destination device as destinations for voice data transmitted by the first client device, the media generator receiving the portion of the second voice data.