Centralized speech logger analysis

• US 9,583,100 B2
• Filed: 09/05/2012
• Issued: 02/28/2017
• Est. Priority Date: 09/05/2012
• Status: Active Grant

• Current Assignee: GM Global Technology Operations LLC (General Motors Company)
• Original Assignee: GM Global Technology Operations LLC (General Motors Company)
• Inventors: Burke, Denis R., Gurovich, Danilo, Rudman, Daniel E., Fry, Keith A., McCutchen, Shane M., Carnevale, Marco T., Gupta, Mukesh
• Primary Examiner(s): Desir, Pierre-Louis
• Assistant Examiner(s): SHIN, SEONG-AH A
• Application Number:
  US13/604,246
• Publication Number:
  US 20140067392A1
• Time in Patent Office: 1,637 Days
• Field of Search: 704236-242, 704246-255, 704/235, 704/270, 704/275, 704/231, 705236-242, 705246-255
• US Class Current: 1/1
• CPC Class Codes: G10L 15/22 Procedures used during a speech recognition process, e.g. man-machine dialogue
• CPC Class Codes: G10L 15/30 Distributed recognition, e.g. in client-server systems, for mobile phones or network applications
• CPC Class Codes: G10L 2015/228 of application context

Abstract

A method of providing hands-free services using a mobile device having wireless access to computer-based services includes receiving speech in a vehicle from a vehicle occupant; recording the speech using a mobile device; transmitting the recorded speech from the mobile device to a cloud speech service; receiving automatic speech recognition (ASR) results from the cloud speech service at the mobile device; and comparing the recorded speech with the received ASR results at the mobile device to identify one or more error conditions.

Claims

1. A method of providing hands-free services using a mobile device having wireless access to computer-based services, the method comprising the steps of:

(a) receiving speech in a vehicle from a vehicle occupant using an audio user interface that is separate from the mobile device, wherein the audio user interface is communicatively linked directly to the mobile device via a short-range wireless connection;

(b) recording the speech using the mobile device;

(c) transmitting the recorded speech from the mobile device to a cloud speech service;

(d) receiving automatic speech recognition (ASR) results from the cloud speech service at the mobile device;

(e) determining that the received ASR results at the mobile device contain one or more error conditions;

(f) obtaining a local speech recognition result, in response to the determination in step (e), by performing speech recognition on the speech received at the vehicle using a speech recognition resource that includes vehicle-related and/or user-related data that was not used by the cloud speech service in generating the ASR results; and

(g) providing the local speech recognition result to the cloud speech service for use in improving subsequent recognition by the cloud speech service, wherein the local speech recognition result addresses the determined one or more error conditions.

2. The method of claim 1, further comprising the step of storing the recorded speech at the mobile device for a predetermined amount of time.

3. The method of claim 1, further comprising the step of receiving a verbal response from the vehicle occupant that triggers the comparing in step (e).

4. The method of claim 1, wherein step (f) is carried out by either the vehicle or mobile device.

5. The method of claim 1, wherein the speech recognition resource includes a local speech recognition grammar.

6. The method of claim 5, further comprising the step of optimizing the local speech recognition grammar using information gathered from a local environment or the vehicle occupant.

7. The method of claim 5, further comprising the step of recursively updating the local speech recognition grammar.

8. The method of claim 1, further comprising the step of sending the results of the comparison to a central facility.

9. The method of claim 1, further comprising the step of creating a log of the identified error conditions.

10. The method of claim 1, further comprising the steps of:

receiving error conditions from a plurality of mobile devices at the central facility and generating an aggregate log based on the received error conditions.

11. The method of claim 1, wherein the local speech recognition result is provided to the cloud speech service without a user-supplied confirmation that the local speech recognition correctly identified the speech received at the vehicle.

12. The method of claim 11, further comprising determining the local speech recognition result corrected the error condition by exceeding a predetermined threshold confidence level.

13. A method of providing hands-free services using a mobile device having wireless access to computer-based services, the method comprising:

(a) receiving speech at a mobile device from a vehicle occupant via a short-range communication connection;

(b) transmitting the received speech from the mobile device to a cloud speech service;

(c) receiving automatic speech recognition (ASR) results from the cloud speech service;

(d) determining that the received ASR results include one or more incorrectly identified speech terms;

(e) performing local speech recognition on the received speech in response to the determination in step (d), including using a speech recognition resource at the vehicle to obtain a local speech recognition result, the speech recognition resource including vehicle-related and/or user-related data that was not used by the cloud speech service in generating the ASR results;

(f) comparing the local speech recognition with the ASR results to detect when the local speech recognition correctly identified the incorrectly identified speech terms in response to the determination in step (e); and

(g) sending the results of the comparison in step (f) to a central facility located remotely from the mobile device.

14. The method of claim 13, further comprising the step of receiving a grammar from the cloud speech service along with the ASR results.

15. The method of claim 13, further comprising the step of performing local speech recognition using a local speech recognition grammar.

16. The method of claim 13, further comprising the step of creating a log of the determination in step (d).

17. The method of claim 13, further comprising the steps of:

receiving the results of step (g) from a plurality of mobile devices and generating an aggregate log based on those results.

18. A method of providing hands-free services using a mobile device having wireless access to computer-based services, the method comprising:

(a) receiving speech from a vehicle occupant at an audio interface in a vehicle;

(b) sending the received speech from the vehicle to a mobile device via a short-range communication connection;

(c) recording the received speech at the mobile device;

(d) transmitting the recorded speech from the mobile device to a cloud speech service, wherein the recorded speech includes an indication to the cloud speech service that the recorded speech originated in the vehicle;

(e) receiving automatic speech recognition (ASR) results from the cloud speech service;

(f) determining that the received ASR results include one or more speech terms that have been incorrectly identified;

(g) performing local speech recognition on the recorded speech using an automatic speech recognition (ASR) system resident on the mobile device in response to step (f) to obtain a local speech recognition result, the ASR system including data that is stored in the vehicle or mobile device and that was not used by the cloud speech service in generating the ASR results;

(h) comparing the local speech recognition with the ASR results to detect instances when the local speech recognition correctly identified the incorrectly identified speech terms; and

(i) maintaining a log of the incorrectly identified speech terms, the instances when the local speech recognition correctly identified the incorrectly identified speech terms, or both.

19. The method of claim 18, further comprising the step of receiving a grammar from the cloud speech service along with the ASR results.

20. The method of claim 18, further comprising the step of performing local speech recognition using a local speech recognition grammar.