Connecting assistant device to devices
First Claim
Patent Images
1. An assistant device, comprising:
- a display screen;
a microphone;
a speaker;
a camera;
one or more processors; and
memory storing instructions, wherein the one or more processors are configured to execute instructions such that the one or more processors and memory are configured to;
receive a first image data depicting one or more devices in an environment of the assistant device from the camera;
identify the one or more devices depicted in the first image data using a visual recognition algorithm;
determine that the one or more devices in the received first image data were not previously connected to the assistant device;
determine device information representing identifying information associated with the one or more devices not previously connected to the assistant device;
determine a device type of the one or more devices based on the device information using one or more of local resources and/or one or more of cloud servers;
determine first audio installation instructions for the one or more devices based on the device type, the first audio installation instructions providing steps to install the one or more devices;
provide the first audio installation instructions via the speaker;
receive a second image data depicting a user performing the first audio installation instructions;
determine characteristics of the user performing the first audio installation instructions based on the second image data;
determine second audio installation instructions for the one or more devices based on the device type and the characteristics of the user, the second audio installation instructions providing the steps to install the one or more devices differently than the providing of the steps by the first audio installation instructions;
provide the second audio installation instructions via the speaker;
receive a user response via the microphone and the camera;
determine descriptive information associated with the one or more devices based on the user response;
set up the assistant device to operate the one or more devices, the set up including determining one or more of voice activatable commands and an adapter associated with the one or more devices;
receive a user instruction, wherein speech recognition of the user instruction includes one or more voice activatable commands and one or more descriptive information;
identify the one or more devices associated with the descriptive information; and
transmit a request to the one or more devices to perform an action associated with the one or more voice activatable commands.
1 Assignment
0 Petitions
Accused Products
Abstract
The present disclosure contemplates a variety of improved methods and systems for enabling set up of a variety of disparate IoT devices coupled to the framework and/or ambient operating system of an assistant device. The described solution includes a voice-driven assistant device setup process. An assistant device can determine the IoT devices in its physical environment and provide setup instructions to the user. The setup including determining voice activatable commands, device information and the adapter to allow the assistant device to operate the one or more IoT devices in response to user instructions.
-
Citations
15 Claims
-
1. An assistant device, comprising:
-
a display screen; a microphone; a speaker; a camera; one or more processors; and memory storing instructions, wherein the one or more processors are configured to execute instructions such that the one or more processors and memory are configured to; receive a first image data depicting one or more devices in an environment of the assistant device from the camera; identify the one or more devices depicted in the first image data using a visual recognition algorithm; determine that the one or more devices in the received first image data were not previously connected to the assistant device; determine device information representing identifying information associated with the one or more devices not previously connected to the assistant device; determine a device type of the one or more devices based on the device information using one or more of local resources and/or one or more of cloud servers; determine first audio installation instructions for the one or more devices based on the device type, the first audio installation instructions providing steps to install the one or more devices; provide the first audio installation instructions via the speaker; receive a second image data depicting a user performing the first audio installation instructions; determine characteristics of the user performing the first audio installation instructions based on the second image data; determine second audio installation instructions for the one or more devices based on the device type and the characteristics of the user, the second audio installation instructions providing the steps to install the one or more devices differently than the providing of the steps by the first audio installation instructions; provide the second audio installation instructions via the speaker; receive a user response via the microphone and the camera; determine descriptive information associated with the one or more devices based on the user response; set up the assistant device to operate the one or more devices, the set up including determining one or more of voice activatable commands and an adapter associated with the one or more devices; receive a user instruction, wherein speech recognition of the user instruction includes one or more voice activatable commands and one or more descriptive information; identify the one or more devices associated with the descriptive information; and transmit a request to the one or more devices to perform an action associated with the one or more voice activatable commands. - View Dependent Claims (2, 3, 4, 5)
-
-
6. A method, comprising:
-
receiving a first image data depicting one or more devices in an environment of an assistant device from a camera; identifying the one or more devices depicted in the first image data using a visual recognition algorithm; determining that the one or more devices in the received first image data were not previously connected to the assistant device; determining device information representing identifying information associated with the one or more devices not previously connected to the assistant device; determining a device type of the one or more devices based on the device information using one or more of local resources or one or more of cloud servers; determining first audio installation instructions for the one or more devices based on the device type, the first audio installation instructions providing steps to install the one or more devices; providing the first audio installation instructions via a speaker; receiving a second image data depicting a user performing the first audio installation instructions; determining characteristics of the user performing the first audio installation instructions based on the second image data; determining second audio installation instructions for the one or more devices based on the device type and the characteristics of the user, the second audio installation instructions providing the steps to install the one or more devices differently than the providing of the steps by the first audio installation instructions; providing the second audio installation instructions via the speaker; receiving a user response via a microphone and the camera; determining descriptive information associated with the one or more devices based on the user response; setting up the assistant device to operate the one or more devices, the set up including determining one or more of voice activatable commands and an adapter associated with the one or more devices; receiving a user instruction, wherein speech recognition of the user instruction includes one or more voice activatable commands and one or more descriptive information; identifying the one or more devices associated with the descriptive information; and transmitting a request to the one or more devices to perform an action associated with the one or more voice activatable commands. - View Dependent Claims (7, 8, 9, 10)
-
-
11. A computer program product including one or more non-transitory computer-readable media storing computer program instructions, execution of which by a processor causes a processing system to perform operations comprising:
-
receiving a first image data depicting one or more devices in an environment of an assistant device from a camera; identifying the one or more devices depicted in the first image data using a visual recognition algorithm; determining that the one or more devices in the received first image data were not previously connected to the assistant device; determining device information representing identifying information associated with the one or more devices not previously connected to the assistant device; determining a device type of the one or more devices based on the device information using one or more of local resources or one or more of cloud servers; determining first audio installation instructions for the one or more devices based on the device type, the first audio installation instructions providing steps to install the one or more devices; providing the first audio installation instructions via a speaker; receiving a second image data depicting a user performing the first audio installation instructions; determining characteristics of the user performing the first audio installation instructions based on the second image data; determining second audio installation instructions for the one or more devices based on the device type and the characteristics of the user, the second audio installation instructions providing the steps to install the one or more devices differently than the providing of the steps by the first audio installation instructions; providing the second audio installation instructions via the speaker; receiving a user response via a microphone and the camera; determining descriptive information associated with the one or more devices based on the user response; setting up the assistant device to operate the one or more devices, the set up including determining one or more of voice activatable commands and an adapter associated with the one or more devices; receiving a user instruction, wherein speech recognition of the user instruction includes one or more voice activatable commands and one or more descriptive information; identifying the one or more devices associated with the descriptive information; and transmitting a request to the one or more devices to perform an action associated with the one or more voice activatable commands. - View Dependent Claims (12, 13, 14, 15)
-
Specification