Remote invocation of mobile device actions

• US 10,152,987 B2
• Filed: 06/23/2014
• Issued: 12/11/2018
• Est. Priority Date: 06/23/2014
• Status: Active Grant

• Current Assignee: Google LLC (Alphabet Inc.)
• Original Assignee: Google LLC (Alphabet Inc.)
• Inventors: Bar-or, Omer, Huffman, Scott B., Mayer, Ida, Blume, Arthur E., Gupta, Pravir K.
• Primary Examiner(s): Mishra, Richa
• Application Number:
  US14/311,995
• Publication Number:
  US 20150371664A1
• Time in Patent Office: 1,632 Days
• Field of Search: None
• CPC Class Codes: G06F 2209/482 Application
• CPC Class Codes: G06F 2209/541 Client-server
• CPC Class Codes: G06F 2209/549 Remote execution
• CPC Class Codes: G06F 9/452 Remote windowing, e.g. X-Window System, desktop virtualisation protocols for virtual reality H04L67/131
• CPC Class Codes: G06F 9/4806 Task transfer initiation or dispatching
• CPC Class Codes: G06F 9/54 Interprogram communication
• CPC Class Codes: G10L 25/48 specially adapted for particular use

Abstract

Systems, methods and apparatus for invoking actions at a second user device from a first user device. A method includes determining that a first user device has an associated second user device; accessing specification data that specifies a set of user device actions that the second user device is configured to perform; receiving command inputs for the first user device; for each command input, determining whether the command input resolves to one of the user device actions; for each command input not determined to resolve to any of the user device actions, causing the command input to be processed at the first user device; and for each command input determined to resolve one of the user device actions causing the first user device to display in a user interface a dialog by which a user may either accept or deny invoking the user device action at the second user device.

Claims

1. A computer-implemented method performed by a data processing apparatus, comprising:

determining, at a first user device, that the first user device has an associated second user device that is separate from the first user device;

accessing, at the first user device, specification data for the second user device that specifies a set of user device actions that the second user device is configured to perform, wherein each user device action is an action performed by a respective application installed on the second user device;

receiving, at the first user device, command inputs for the first user device, each command input being a set of terms input by a user of the first user device;

for each command input, determining, at the first user device, whether the command input resolves to one of the user device actions of the set of user device actions for the second user device, the one of the user device actions being an action that can be performed by a respective application installed on the second user device; and

for each command input determined to resolve one of the user device actions at the second user device;

causing the first user device to generate and display a dialog in a user interface of the first user device by which a user of the first user device may either accept or deny invoking the user device action of the respective application at the second user device in response to the command input;

in response to an acceptance, through the dialog and by the user at the first user device, to invoke the user action at the second user device, causing the second user device to invoke the user device action of the respective application installed on the second user device and process the command input as an input for the user device action, wherein causing the second user device to invoke the user device action and process the command input as an input for the user device action comprises;

generating, at the first user device, a command communication that includes data that, when processed by an application programming interface on the second user device, causes the second user device to invoke the user device action and process the command input as the input for the user device action; and

sending, from the first user device, the command communication to the second user device; and

in response to a denial, through the dialog and by the user at the first user device, to invoke the user device action at the second user device, causing the first user device to invoke the user device action of the respective application installed on the first user device and process the command input as an input for the user device action.

2. The computer implemented method of claim 1, further comprising, for each command input not determined to resolve to any of the user device actions, causing the command input to be processed at the first user device.

3. The computer-implemented method of claim 1, wherein the second user device is a smart phone.

5. The computer-implemented method of claim 1, wherein the command inputs are voice command sentences.

12. The computer implemented method of claim 1, further comprising:

determining whether each command input resolves two or more of the user device actions of the set of user device actions for the second user device; and

for each command input determined to resolve two or more of the user device actions at the second user device and prior to causing the first user device to generate and display the dialog in the user interface of the first user device, determining a likelihood of a most likely action to be performed of the two or more of the user device actions, wherein the user device action to be displayed in the dialog in the user interface in the first device comprises the most likely action.

4. A computer-implemented method performed by a data processing apparatus, comprising:

determining, at a server computer separate from a first user device and a second user device, that the first user device is associated with the second user device that is separate from the first user device;

accessing, at the server computer, specification data for the second user device that specifies a set of user device actions that the second user device is configured to perform, wherein each user device action is an action performed by a respective application installed on the second user device;

receiving, at the server computer, command inputs input at the first user device for the first user device, each command input being a set of terms input by a user of the first user device;

for each command input, determining, at the server computer, whether the command input resolves to one of the user device actions of the set of user device actions for the second user device, the one of the user device actions being an action that can be performed by a respective application installed on the second user device; and

for each command input determined to resolve one of the user device actions at the second user device;

causing the first user device to generate and display a dialog in a user interface of the first user device by which a user of the first user device may either accept or deny invoking the user device action of the respective application at the second user device in response to the command input;

in response to an acceptance, through the dialog and by the user at the first user device, to invoke the user device action at the second user device, causing the second user device to invoke the user device action of the respective application installed on the second user device and process the command input as an input for the user device action, wherein causing the second user device to invoke the user device action and process the command input as an input for the user device action comprises;

generating, at the first user device, a command communication that includes data that, when processed by an application programming interface on the second user device, causes the second user device to invoke the user device action and process the command input as the input for the user device action; and

sending, from the first user device, the command communication to the second user device; and

in response to a denial, through the dialog and by the user at the first user device, to invoke the user device action at the second user device, causing the first user device to invoke the user device action of the respective application installed on the second user device and process the command input as an input for the user device action; and

for each command input not determined to resolve to any of the user device actions, causing the first user device to invoke the user device action and process the command input as an input for the user device action.

13. The computer-implemented method of claim 4, further comprising:

determining whether each command input resolves two or more of the user device actions of the set of user device actions for the second user device; and

for each command input determined to resolve two or more of the user device actions at the second user device and prior to causing the first user device to generate and display the dialog in the user interface of the first user device, determining a likelihood of a most likely action to be performed of the two or more of the user device actions, wherein the user device action to be displayed in the dialog in the user interface in the first device comprises the most likely action.

6. A computer-implemented method performed by a data processing apparatus, comprising:

determining that a first user device has an associated second user device that is separate from the first user device;

accessing specification data for the second user device that specifies a set of user device actions that the second user device is configured to perform, wherein each user device action is an action performed by a respective application installed on the second user device;

receiving command inputs for the first user device, each command input being a set of terms input by a user of the first user device;

for each command input, determining whether the command input resolves to one of the user device actions of the set of user device actions for the second user device, the one of the user device actions being an action that can be performed by a respective application installed on the second user device; and

for each command input determined to resolve one of the user device actions at the second user device;

causing the first user device to generate and display a dialog in a user interface of the first user device by which a user of the first user device may either accept or deny invoking the user device action of the respective application at the second user device in response to the command input;

in response to an acceptance, through the dialog and by the user at the first user device, to invoke the user action at the second user device, causing the second user device to invoke the user device action of the respective application installed on the second user device and process the command input as an input for the user device action, wherein causing the second user device to invoke the user device action and process the command input as an input for the user device action comprises;

generating, at the first user device, a command communication that includes data that, when processed by an application programming interface on the second user device, causes the second user device to invoke the user device action and process the command input as the input for the user device action; and

sending, from the first user device, the command communication to the second user device; and

in response to a denial, through the dialog and by the user at the first user device, to invoke the user device action at the second user device, causing the command input to be processed at the first user device to invoke the user device action of the respective application installed on the first user device at the first user device; and

receiving, for the second user device, update data for an update to the specification data for the second user device, the update data specifying a change to the set of user device actions that the second user device is configured to perform; and

updating the specification data for the second user device to reflect the change to the set of user device actions that the second user device is configured to perform.

7. A system, comprising:

a user device comprising a data processing apparatus and a computer storage apparatus in data communication with the data processing apparatus and storing instructions executable by the data processing apparatus and that upon such execution cause the data processing apparatus to perform operations comprising;

determining, at a first user device, that the first user device has an associated second user device that is separate from the first user device;

accessing, at the first user device, specification data for the second user device that specifies a set of user device actions that the second user device is configured to perform, wherein each user device action is an action performed by a respective application installed on the second user device;

receiving, at the first user device, command inputs for the first user device, each command input being a set of terms input by a user of the first user device;

for each command input, determining, at the first user device, whether the command input resolves to one of the user device actions of the set of user device actions for the second user device, the one of the user device actions being an action that can be performed by a respective application installed on the second user device; and

for each command input determined to resolve one of the user device actions at the second user device;

causing the first user device to generate and display a dialog in a user interface of the first user device by which a user of the first user device may either accept or deny invoking the user device action of the respective application at the second user device in response to the command input;

in response to an acceptance, through the dialog and by the user at the first user device, to invoke the user action at the second user device, causing the second user device to invoke the user device action of the respective application installed on the second user device and process the command input as an input for the user device action, wherein causing the second user device to invoke the user device action and process the command input as an input for the user device action comprises;

generating, at the first user device, a command communication that includes data that, when processed by an application programming interface on the second user device, causes the second user device to invoke the user device action and process the command input as the input for the user device action; and

sending, from the first user device, the command communication to the second user device; and

in response to a denial, through the dialog and by the user at the first user device, to invoke the user device action at the second user device, causing the first user device to invoke the user device action of the respective application installed on the first user device and process the command input as an input for the user device action.

8. The system of claim 7, the operations further comprising for each command input no determined to resolve to any of the user device actions, causing the command input to be processed at the first user device.

9. The system of claim 7, wherein the second user device is a smart phone.

10. The system of claim 7, the operations further comprising:

receiving, for the second user device, update data for an update to the specification data for the second user device, the update data specifying a change to the set of user device actions that the second user device is configured to perform; and

updating the specification data for the second user device to reflect the change to the set of user device actions that the second user device is configured to perform.

14. The system of claim 7, the operations further comprising:

determining whether each command input resolves two or more of the user device actions of the set of user device actions for the second user device; and

for each command input determined to resolve two or more of the user device actions at the second user device and prior to causing the first user device to generate and display the dialog in the user interface of the first user device, determining a likelihood of a most likely action to be performed of the two or more of the user device actions, wherein the user device action to be displayed in the dialog in the user interface in the first device comprises the most likely action.

11. A non-transitory computer readable storage medium in data communication with a data processing apparatus and storing instructions executable by the data processing apparatus and upon such execution cause the data processing to perform operations comprising:

determining, at a first user device, that the first user device has an associated second user device that is separate from the first user device;

accessing, at the first user device, specification data for the second user device that specifies a set of user device actions that the second user device is configured to perform, wherein each user device action is an action performed by a respective application installed on the second user device;

receiving, at the first user device, command inputs for the first user device, each command input being a set of terms input by a user of the first user device;

for each command input, determining, at the first user device, whether the command input resolves to one of the user device actions of the set of user device actions for the second user device, the one of the user device actions being an action that can be performed by a respective application installed on the second user device; and

for each command input determined to resolve one of the user device actions at the second user device;

causing the first user device to generate and display a dialog in a user interface of the first user device by which a user of the first user device may either accept or deny invoking the user device action of the respective application at the second user device in response to the command input;

in response to an acceptance, through the dialog and by the user at the first user device, to invoke the user action at the second user device, causing the second user device to invoke the user device action of the respective application installed on the second user device and process the command input as an input for the user device action, wherein causing the second user device to invoke the user device action and process the command input as an input for the user device action comprises;

generating, at the first user device, a command communication that includes data that, when processed by an application programming interface on the second user device, causes the second user device to invoke the user device action and process the command input as the input for the user device action; and

sending, from the first user device, the command communication to the second user device; and

in response to a denial, through the dialog and by the user at the first user device, to invoke the user device action at the second user device, causing the first user device to invoke the user device action of the respective application installed on the first user device and process the command input as an input for the user device action.