Systems and methods for simultaneously receiving voice instructions on onboard and offboard devices

• US 10,310,808 B2
• Filed: 09/02/2015
• Issued: 06/04/2019
• Est. Priority Date: 09/08/2014
• Status: Active Grant

• Current Assignee: Google LLC (Alphabet Inc.)
• Original Assignee: Google LLC (Alphabet Inc.)
• Inventors: Foster, Andrew
• Primary Examiner(s): Opsasnick, Michael N
• Application Number:
  US14/842,986
• Publication Number:
  US 20160070533A1
• Time in Patent Office: 1,371 Days
• Field of Search: 704270, 7042701, 704275
• CPC Class Codes: G01C 21/36 Input/output arrangements for on-board computers
• CPC Class Codes: G01C 21/3608 using speech input, e.g. using speech recognition
• CPC Class Codes: G01C 21/3688 Systems comprising multiple parts or multiple output devices (not client-server), e.g. detachable faceplates, key fobs or multiple output screens
• CPC Class Codes: G06F 3/167 Audio in a user interface, e.g. using voice commands for navigating, audio feedback
• CPC Class Codes: G10L 15/22 Procedures used during a speech recognition process, e.g. man-machine dialogue
• CPC Class Codes: G10L 15/26 Speech to text systems G10L15/08 takes precedence
• CPC Class Codes: G10L 2015/223 Execution procedure of a spoken command
• CPC Class Codes: G10L 2015/228 of application context

Abstract

To allow a user to provide a voice instruction to either a portable device or a computing device embedded within a vehicle, both the portable device and the embedded computing device receive the voice instruction such as, “Direct me to Kansas City.” Moreover, both the portable device and the embedded computing device may determine the likelihoods that the portable device and the computing device, respectively, can carry out the voice instruction. The portable device and the computing device may then communicate with each other to compare the determined likelihoods. Based on the comparison, either the portable device or the computing device may respond to the voice instruction by, for example, playing a requested song, turning on the radio in the vehicle, providing navigation directions from the current location to a destination, etc.

Claims

1. A portable device comprising:

a communication interface to communicate with a computing device embedded within a vehicle via a short-range communication link;

one or more processors coupled to the communication interface; and

a non-transitory computer-readable memory coupled to the one or more processors and storing thereon instructions that, when executed by the one or more processors, cause the portable device to;

receive audio input in response to receiving, via the short-range communication link, a request to activate voice recognition from the embedded computing device,

activate voice recognition on the portable device,

perform the voice recognition on the received audio input to determine a requested action based on the received audio input,

determine a first likelihood that the portable device can carry out the requested action,

receive, via the short-range communication link, a second likelihood from the embedded computing device that the embedded computing device can carry out the requested action,

determine, based on the first likelihood and the second likelihood, whether the embedded computing device or the portable device should carry out the requested action, and

cause one of the embedded computing device or the portable device to carry out the requested action in accordance with the determination based on the first likelihood and the second likelihood.

2. The portable device of claim 1, wherein to determine whether the embedded computing device or the portable device should carry out the requested action, the instructions cause the portable device to determine whether the first likelihood differs from the second likelihood by more than a fixed threshold amount.

3. The portable device of claim 2, wherein to determine whether the embedded computing device or the portable device should carry out the requested action, the instructions further cause the portable device to:

in response to determining that the first likelihood does not differ from the second likelihood by more than a fixed threshold amount, request additional audio input,

receive the additional audio input, and

determine the first likelihood and the second likelihood based on the audio input and the additional audio input.

4. The portable device of claim 1, wherein to determine the first likelihood, the instructions cause the portable device to search one or more applications on the portable device to identify an application capable of carrying out the requested action.

5. The portable device of claim 1, wherein the instructions cause the portable device to receive the audio input from the embedded computing device, via the communication interface.

6. The portable device of claim 1, wherein the communication interface is a first communication interface, the portable device further comprising:

a second communication interface to communicate with a network server via a long-range communication link;

wherein the instructions further cause the portable device to;

transmit the received audio input to the network server via the second communication interface, and

receive an indication of the requested action via the second communication interface.

7. The portable device of claim 1, wherein the requested action includes a request for a playback of at least one of:

(i) a particular song, (ii) audio transmitted by a particular radio station, (iii) music by a particular musician, or (iv) audio from a particular device installed in the vehicle, and wherein the instructions further cause the portable device to;

identify a first application capable of audio playback on the portable device,

identify a second application capable of audio playback on the embedded computing device, and

determine the first likelihood and the second likelihood based on abilities of the first application and the second application, respectively, to provide the requested playback.

8. A portable device comprising:

a communication interface to communicate with a computing device embedded within a vehicle via a short-range communication link;

one or more processors coupled to the communication interface; and

a non-transitory computer-readable memory coupled to the one or more processors and storing thereon instructions that, when executed by the one or more processors, cause the portable device to;

receive audio input while the embedded computing device receives the same audio input in response to receiving a request to activate voice recognition from the embedded computing device, before determining whether the portable device or the embedded computing device will carry out a requested action based on the received audio input,

activate voice recognition on the portable device,

perform the voice recognition on the received audio input to determine the requested action based on the received audio input,

determine a first likelihood that the portable device can carry out the requested action,

receive, from the embedded computing device, a second likelihood that the embedded computing device can carry out the requested action,

determine, based on the first likelihood and the second likelihood, whether the embedded computing device or the portable device should carry out the requested action, and

cause one of the embedded computing device or the portable device to carry out the requested action in accordance with the determination based on the first likelihood and the second likelihood.

9. The portable device of claim 8, wherein to determine whether the embedded computing device or the portable device should carry out the requested action, the instructions cause the portable device to determine whether the first likelihood differs from the second likelihood by more than a fixed threshold amount.

10. The portable device of claim 9, wherein to determine whether the embedded computing device or the portable device should carry out the requested action, the instructions further cause the portable device to:

in response to determining that the first likelihood does not differ from the second likelihood by more than a fixed threshold amount, request additional audio input,

receive the additional audio input, and

determine the first likelihood based on the audio input and the additional audio input.

11. The portable device of claim 8, wherein to determine the first likelihood, the instructions further cause the portable device to search one or more applications on the portable device to identify an application capable of carrying out the requested action.

12. The portable device of claim 8, wherein the instructions cause the portable device to receive the audio input from the embedded computing device, via the communication interface.

13. The portable device of claim 8, wherein the communication interface is a first communication interface, the portable device further comprising:

a second communication interface to communicate with a network server via a long-range communication link;

wherein the instructions further cause the portable device to;

transmit the received audio input to the network server via the second communication interface, and

receive an indication of the requested action via the second communication interface.

14. The portable device of claim 8, wherein the requested action corresponds to a request for navigation directions from an origin to a destination, and wherein the instructions further cause the portable device to:

identify an application capable of providing navigation directions on the portable device; and

determine the first likelihood based on an ability of the first application to provide the requested navigation directions from the origin to the destination.

15. A method for determining a device for responding to a voice instruction, the method comprising:

receiving, at one or more processors in a portable device, audio input, while a computing device embedded within a vehicle receives the same audio input and in response to receiving a request to activate voice recognition from the embedded computing device, before determining whether the portable device or the embedded computing device will carry out a requested action based on the received audio input;

activating, by the one or more processors, voice recognition on the portable device,

performing, by the one or more processors, the voice recognition on the received audio input to determine the requested action based on the received audio input;

determining, by the one or more processors, a first likelihood that the portable device can carry out the requested action;

receiving, at the one or more processors and from the embedded computing device, a second likelihood that the embedded computing device can carry out the requested action,

determining, by the one or more processors and based on the first likelihood and the second likelihood, whether the embedded computing device or the portable device should carry out the requested action; and

causing, by the one or more processors, one of the embedded computing device or the portable device to carry out the requested action in accordance with the determination based on the first likelihood and the second likelihood.

16. The method of claim 15, wherein determining whether the embedded computing device or the portable device should carry out the requested action comprises determining, by the one or more processors, whether the first likelihood differs from the second likelihood by more than a fixed threshold amount.

17. The method of claim 16, wherein determining whether the embedded computing device or the portable device should carry out the requested action further comprises:

in response to determining that the first likelihood does not differ from the second likelihood by more than a fixed threshold amount, requesting, by the one or more processors, additional audio input;

receiving, at the one or more processors, the additional audio input; and

determining, by the one or more processors, the first likelihood based on the audio input and the additional audio input.

18. The method of claim 15, wherein determining the first likelihood comprises searching, by the one or more processors, one or more applications on the portable device to identify an application capable of carrying out the requested action.

19. The method of claim 15, wherein the requested action includes a request for a playback of at least one of:

(i) a particular song, (ii) audio transmitted by a particular radio station, (iii) music by a particular musician, or (iv) audio from a particular device installed in the vehicle, further comprising;

identifying, by the one or more processors, an application capable of audio playback on the portable device; and

determining, by the one or more processors, the first likelihood based on an ability of the application to provide the requested playback.