Complementing operation of display devices in an augmented reality environment

• US 9,607,315 B1
• Filed: 12/30/2010
• Issued: 03/28/2017
• Est. Priority Date: 12/30/2010
• Status: Expired due to Fees

• Current Assignee: Amazon Technologies, Inc. (Amazon.com, Inc.)
• Original Assignee: Amazon Technologies, Inc. (Amazon.com, Inc.)
• Inventors: Worley, William Spencer III
• Primary Examiner(s): Sigmond, Bennett
• Assistant Examiner(s): Stroud, Christopher
• Application Number:
  US12/982,519
• Time in Patent Office: 2,280 Days
• US Class Current: 1/1
• CPC Class Codes: G06Q 30/0251 Targeted advertisements
• CPC Class Codes: G06Q 30/0252 based on events or environment, e.g. weather or festivals

Abstract

Techniques for enabling an augmented reality system to complement operation of existing output devices in an environment are described herein. An augmented reality system may include one or more augmented reality functional nodes (ARFNs) that are configured to project content onto different non-powered and/or powered display mediums. The ARFN may identify existing output devices, as well as any content that these content output devices currently output. The ARFN may then either take over the display of the content when appropriate, or may project related content.

Claims

1. One or more non-transitory computer-readable media storing computer-executable instructions that, when executed, cause one or more processors to perform acts comprising:

instructing a camera of a first device to capture images of an environment, the first device including the camera and a projector;

identifying, from the images, a second device within the environment that is di splaying digital content;

detecting, within the environment, an environmental obstruction that interferes with a projection of the digital content by the projector; and

at least partly in response to identifying the second device displaying digital content and detecting, within the environment, the environmental obstruction, performing acts, including;

identifying, with reference to the images, the digital content displayed by the second device;

retrieving digital content that is related to the identified digital content;

identifying, from the images, a third device within the environment;

measuring an ambient light associated with the environment;

determining, based at least in part on the ambient light, that the digital content that is related to the identified digital content is suited for output on the third device; and

outputting the digital content that is related to the identified digital content via the third device.

2. One or more non-transitory computer-readable media as recited in claim 1, wherein the determining that the digital content that is related to the identified digital content is suited for the third device is based at least in part on a display characteristic of the third display device.

3. One or more non-transitory computer-readable media as recited in claim 1, wherein the identifying of the digital content comprises comparing the images captured by the camera to images associated with known content.

4. One or more non-transitory computer-readable media as recited in claim 1, wherein the retrieving of the digital content that is related to the identified digital content comprises requesting content from a content provider that is associated with the identified digital content but is different from a content provider that provides the identified digital content to the second device.

5. One or more non-transitory computer-readable media as recited in claim 1, wherein the retrieving comprises receiving the digital content that is related to the identified digital content from the second device or from a content provider that provides the identified digital content to the second device.

6. One or more non-transitory computer-readable media as recited in claim 1, wherein the digital content that is related comprises content that has been deemed as supplemental to the identified digital content.

7. One or more non-transitory computer-readable media as recited in claim 1, wherein the digital content that is related comprises an advertisement for an item that is associated with the identified digital content.

8. One or more non-transitory computer-readable media as recited in claim 1, wherein the third device is adjacent to the second device within the environment.

9. One or more non-transitory computer-readable media as recited in claim 1, computer-executable instructions that, when executed, cause the one or more processors to perform acts comprising:

measuring a glare on a screen associated with the third device; and

wherein determining whether to project the content that is related to the identified digital content via the projector or output the content that is related to the identified digital content via the third device is based at least in part on the glare.

10. A method comprising:

under control of one or more computing systems configured with specific executable instructions,

scanning an environment with a camera of a first device to capture images, the first device including the camera and a projector;

identifying, by the first device and from the images, content being output in the environment by a second device;

retrieving content that is related to the content being output by the second device;

identifying, from the images, a third device within the environment;

estimating, from the images, at least one dimension associated with the third device;

identifying, from the images, a brand identifier associated with the third device;

determining an identity of the third device based at least in part on the at least one dimension and the brand name;

determining to project the content that is related to the identified content via the projector, wherein determining to project the content is based, at least in part, on an environmental obstruction detected within the environment, wherein the environmental obstruction does not interfere with a projection of the content via the projector; and

in response to determining to project the content that is related to the identified content, identifying, by the first device and from the images, a projection surface to project the content onto and via the projector of the first device, projecting the related content onto the projection surface.

11. A method as recited in claim 10, wherein:

the content being output by the second device comprises visual content; and

the second device comprises a television, a mobile phone, a tablet computer, a laptop computer, a desktop computer, an electronic book reader device, a portable media player, or a personal digital assistant (PDA).

12. A method as recited in claim 10, wherein the content being output by the second device comprises audio content;

and

the second device comprises a stereo system, speakers, or a combination thereof.

13. A method as recited in claim 10, wherein the identifying of the content comprises:

comparing the captured images of the content to images of known content.

14. A method as recited in claim 10, wherein the identifying of the content comprises:

scanning the environment with the camera of the first device to identify a visual indication of the content being output by the second device; and

mapping the visual indication to the content.

15. A method as recited in claim 14, wherein the visual indication comprises a name of the content, a channel on which the content is being broadcast, or a unique identifier of the content.

16. A method as recited in claim 10, wherein the identifying of the content comprises receiving an indication of the content from the second device or from a device coupled to the second device.

17. A method as recited in claim 10, wherein the identifying of the content comprises receiving an indication of the content from a user consuming the content within the environment.

18. A method as recited in claim 10, wherein the related content comprises an advertisement that is associated with the identified content.

19. A method as recited in claim 10, further comprising receiving a request from a user within the environment to project the related content, and wherein the projecting of the related content occurs at least partly in response to receiving the request.

20. A method as recited in claim 10, wherein the project surface is a mobile projection surface having a position tracked via the camera of the first device.

21. A system comprising

one or more processors;

memory;

a camera, coupled to the one or more processors, and configured to scan an environment to capture images;

a projector, coupled to the one or more processors and configured to project content within the environment;

an ancillary device identification module, stored in or accessible by the memory and executable on the one or more processors, to identify from the images an additional output device also within the environment;

a content identification module, stored in or accessible by the memory and executable on the one or more processors, to identify content being output by the additional output device; and

a content output module, stored in or accessible by the memory and executable on the one or more processors, to selectively instruct the projector to project the content being output by the additional output device or content that is related to the content being output by the additional output device, wherein the content output module is further executable to analyze the content to determine that the content is better suited for projection rather than display on the additional output device and to instruct the projector to project the content, wherein to determine that the content is better suited for projection rather than display on the additional output device is based, at least in part, on one or more of an ambient lighting condition within the environment or an environmental obstruction detected within the environment, wherein the environmental obstruction does not interfere with a projection of the content.

22. A system as recited in claim 21, wherein the content output module is further executable to instruct the additional output device to cease outputting the content at least partly in response to instructing the projector to project the content.

23. A system as recited in claim 21, wherein the ancillary device identification module is executable to identify the additional output device with reference to the captured images, and the content identification module is executable to identify, with reference to the captured images, the content being output on the additional output device.

24. A system as recited in claim 21, wherein the ancillary device identification module is executable to identify the additional output device within the environment in response to receiving an indication of the additional output device from the additional output device or from a user within the environment.

25. A system as recited in claim 21, wherein the content output module is executable to identify the content being output on the additional output device in response to receiving an indication of the content from the additional output device or from a user within the environment.

26. A system as recited in claim 21, wherein the content output module is further executable to identify and retrieve the related content prior to, while, or after instructing the projector to project the related content onto a display medium within the environment.

27. A system as recited in claim 26, wherein the display medium within the environment is adjacent to the additional output device.

28. A system as recited in claim 21, wherein the content output module instructs the projector to project the related content, and wherein the related content comprises an advertisement.

29. A system as recited in claim 21, wherein:

the ancillary device identification module is further executable to identify multiple additional output devices within the environment from the images;

the content identification module is further executable to identify content being output by each of the multiple additional output devices; and

the content output module is further executable to selectively instruct the projector to project content being output by one or more of the multiple additional output devices or content that is related to content being output by one or more of the multiple additional output devices.