Device-layout determinations

• US 9,772,812 B1
• Filed: 03/28/2016
• Issued: 09/26/2017
• Est. Priority Date: 03/28/2016
• Status: Active Grant

• Current Assignee: Amazon Technologies, Inc. (Amazon.com, Inc.)
• Original Assignee: Amazon Technologies, Inc. (Amazon.com, Inc.)
• Inventors: Webster, Roger Robert
• Primary Examiner(s): Merkoulova, Olga
• Application Number:
  US15/082,994
• Time in Patent Office: 547 Days
• CPC Class Codes: G06F 3/1423 controlling a plurality of local displays, e.g. CRT and flat panel display
• CPC Class Codes: G06F 3/147 using display panels
• CPC Class Codes: G06F 3/16 Sound input; Sound output speech processing G10L
• CPC Class Codes: G06F 3/167 Audio in a user interface, e.g. using voice commands for navigating, audio feedback
• CPC Class Codes: G06T 7/10 Segmentation; Edge detection motion-based segmentation G06T7/215
• CPC Class Codes: G06T 7/60 Analysis of geometric attributes
• CPC Class Codes: G06T 7/74 involving reference images or patches
• CPC Class Codes: G06T 7/90 Determination of colour characteristics
• CPC Class Codes: G06V 20/20 in augmented reality scenes
• CPC Class Codes: G06V 2201/02 Recognising information on displays, dials, clocks
• CPC Class Codes: G06V 30/10 Character recognition
• CPC Class Codes: G09G 2340/14 Solving problems related to the presentation of information to be displayed
• CPC Class Codes: G09G 2340/145 related to small screens
• CPC Class Codes: G09G 2370/022 Centralised management of display operation, e.g. in a server instead of locally

Abstract

This application describes techniques for determining a constituency and a layout of a group of devices within an environment. By determining the devices that form the group, as well as the positions of these devices relative to one another, the group of devices may be used to more intelligently output content within the environment.

Claims

1. A system comprising:

one or more processors; and

one or more computer-readable media storing computer-executable instructions that, when executed, cause the one or more processors to perform acts comprising;

receiving, from an application executing on a client device within an environment, a request to determine a layout of a group of display devices within the environment;

determining a user profile associated with the client device;

determining that the user profile is associated with a first display device, a second display device, and a third display device;

sending a first instruction to the first display device to display a first identifier;

sending a second instruction to the second display device to display a second identifier;

sending a third instruction to the third display device to display a third identifier;

receiving an image captured by a camera of the client device;

performing image recognition on the image to;

(1) determine that the image depicts the first identifier and the second identifier but not the third identifier;

(2) determine a first position of the first display device, and (3) determine a second position of the second display device relative to the first position; and

storing;

(1) a first indication that the group of display devices within the environment includes the first display device and the second display device, and (2) a second indication of the layout of the group of display devices, the layout specifying the first position of the first display device and the second position of the second display device relative to the first position.

2. The system as recited in claim 1, the acts further comprising:

receiving a request to display content in the environment;

determining, based on the layout of the group of display devices, that the first and second display devices are adjacent one another; and

utilizing the first display device and the second display device as a single display by sending a third instruction to first display device to display a first portion of the content and sending a fourth instruction to the second display device to display a second portion of the content.

3. The system as recited in claim 1, wherein:

the sending the first instruction to the first display device further comprises sending the first instruction to the first display device to display a first color around a first perimeter of the first display device;

the sending the second instruction to the second display device further comprises sending the second instruction to the second display device to display a second color around a second perimeter of the second display device;

the performing the image recognition on the image is further to identify the first color around the first perimeter and the second color around the second perimeter; and

the acts further comprise determining;

(1) a first display size of the first display device based at least in part on the identifying the first color around the first perimeter, and (2) a second display size of the second display device based at least in part on the identifying the second color around the second perimeter.

4. The system as recited in claim 1, the acts further comprising:

receiving, from the application executing on the client device, a request to determine an updated layout of the group of display devices within the environment;

determining the user profile associated with the client device;

determining that the user profile is associated with the first display device, the second display device, and the third display device;

sending the first instruction to the first display device to display the first identifier;

sending the second instruction to the second display device to display the second identifier;

sending the third instruction to the third display device to display a third identifier;

receiving an updated image captured by the camera of the client device;

performing image recognition on the updated image to;

(1) determine that the updated image includes the first identifier, the second identifier, and the third identifier;

(2) determine an updated first position of the first display device;

(3) determine an updated second position of the second display device relative to the updated first position, and (4) determine a third position of the third display device relative to the updated first position; and

storing;

(1) an updated first indication to indicate that the group of display devices within the environment includes the first display device, the second display device, and the third display device, and (2) an updated second indication to indicate the updated layout of the group of display devices, the updated layout specifying the updated first position of the first display device, the updated second position of the second display device relative to the updated first position, and the third position of the third display device relative to the updated first position.

5. A system comprising:

one or more processors; and

one or more computer-readable media storing computer-executable instructions that, when executed, cause the one or more processors to perform acts comprising;

receiving image data captured by a first device within an environment, the image data representing a second device within the environment and a third device within the environment;

determining, from the image data, first visual indicia associated with the second device;

determining first information regarding the second device using at least the first visual indicia;

determining, from the image data, second visual indicia associated with the third device;

determining second information regarding the third device using at least the second visual indicia;

determining, from the image data, a first position of the second device relative to a second position of the third device;

storing the first information;

storing the second information; and

storing the first position of the second device relative to the second position of the third device.

6. The system as recited in claim 5, the acts further comprising, at least partly prior to the receiving of the image data:

sending, to the second device, a first instruction to display the first visual indicia; and

sending, to the third device, a second instruction to display the second visual indicia.

7. The system as recited in claim 5, the acts further comprising:

determining at least one of a make or model of the second device; and

determining at least one of a make or model of the third device.

8. A system as recited in claim 5, the acts further comprising:

determining at least one of a display size of the second device or a capability of the second device; and

determining at least one of a display size of the third device or a capability of the third device.

9. The system as recited in claim 5, wherein the second device comprises a display device and the third device comprises at least one of a microphone, a speaker, a camera, or a physical switch.

10. The system as recited in claim 5, the acts further comprising:

receiving a request to output visual content in the environment;

determining that the second device is configured to output the visual content;

determining that the third device is configured to output the visual content;

determining, based on the first position of the second device and the second position of the third device, that the second device and the third device are adjacent one another;

sending a first instruction to the second device to display at least a portion of the visual content; and

sending a second instruction to the third device to display at least a portion of the visual content.

11. The system as recited in claim 5, the acts further comprising:

receiving a request to output audible content in the environment;

determining that at least one of the second device or the third device is configured to output the audible content; and

sending an instruction to the at least one of the second device or the third device to output the audible content.

12. The system as recited in claim 5, wherein:

the first visual indicia comprises;

(1) at least one of a first alphanumeric string, a first shape, a first object, or a first pattern, and (2) an indication of a perimeter of a display of the second device; and

the second visual indicia comprises;

(1) at least one of a second alphanumeric string, a second shape, a second object, or a second pattern, and (2) an indication of a perimeter of a display of the third device.

13. A method comprising:

receiving image data captured by a first device within an environment, the image data representing a second device within the environment and a third device within the environment;

determining, from the image data, first visual indicia associated with the second device;

determining first information regarding the second device using at least the first visual indicia;

determining, from the image data, second visual indicia associated with the third device;

determining second information regarding the third device using at least the second visual indicia;

determining, from the image data, a first position of the second device relative to a second position of the third device;

storing the first information;

storing the second information; and

storing the first position of the second device relative to the second position of the third device.

14. The method as recited in claim 13, further comprising, at least partly prior to the receiving of the image data:

sending, to the second device, a first instruction to display the first visual indicia; and

sending, to the third device, a second instruction to display the second visual indicia.

15. The method as recited in claim 13, the acts further comprising:

determining at least one of a make or model of the second device; and

determining at least one of a make or model of the third device.

16. The method as recited in claim 13, the acts further comprising:

determining at least one of a display size of the second device or a capability of the second device; and

determining at least one of a display size of the third device or a capability of the third device.

17. The method as recited in claim 13, wherein the second device comprises a display device and the third device comprises at least one of a microphone, a speaker, a camera, or a physical switch.

18. The method as recited in claim 13, further comprising:

receiving a request to output visual content in the environment;

determining that the second device is configured to output the visual content;

determining that the third device is configured to output the visual content;

determining, based on the first position of the second device and the second position of the third device, that the second device and the third device are adjacent one another;

sending a first instruction to the second device to display at least a portion of the visual content; and

sending a second instruction to the third device to display at least a portion of the visual content.

19. The method as recited in claim 13, further comprising:

receiving a request to output audible content in the environment;

determining that at least one of the second device or the third device is configured to output the audible content; and

sending an instruction to the at least one of the second device or the third device to output the audible content.

20. The method as recited in claim 13, wherein:

the first visual indicia comprises;

(1) at least one of a first alphanumeric string, a first shape, a first object, or a first pattern, and (2) an indication of a perimeter of a display of the second device; and

the second visual indicia comprises;

(1) at least one of a second alphanumeric string, a second shape, a second object, or a second pattern, and (2) an indication of a perimeter of a display of the third device.