Viewer tracking image display

• US 9,626,939 B1
• Filed: 01/02/2014
• Issued: 04/18/2017
• Est. Priority Date: 03/30/2011
• Status: Active Grant

• Current Assignee: Amazon Technologies, Inc. (Amazon.com, Inc.)
• Original Assignee: Amazon Technologies, Inc. (Amazon.com, Inc.)
• Inventors: Karakotsios, Kenneth Mark, Noble, Isaac Scott
• Primary Examiner(s): Perungavoor, Sath V
• Assistant Examiner(s): Brown, Jr., Howard D
• Application Number:
  US14/146,584
• Time in Patent Office: 1,202 Days
• CPC Class Codes: G06F 3/011 Arrangements for interaction with the human body, e.g. for user immersion in virtual reality blind teaching G09B21/00
• CPC Class Codes: G06F 3/0304 Detection arrangements using opto-electronic means constructional details of pointing devices not related to the detection arrangement using opto-electronic means G06F3/033; optical digitisers G06F3/042
• CPC Class Codes: G06F 3/14 Digital output to display device ; Cooperation and interconnection of the display device with other functional units
• CPC Class Codes: G09G 2320/028 by changing the viewing angle properties, e.g. widening the viewing angle, adapting the viewing angle to the view direction
• CPC Class Codes: G09G 5/00 Control arrangements or circuits for visual indicators common to cathode-ray tube indicators and other visual indicators image data processing or generation, in general G06T
• CPC Class Codes: G09G 5/37 Details of the operation on graphic patterns G09G5/38 takes precedence
• CPC Class Codes: H04M 2250/52 including functional features of a camera constructional features concerning the integration of cameras in portable phones H04M1/0264
• CPC Class Codes: H04N 13/204 using stereoscopic image cameras stereoscopic photography G03B35/00
• CPC Class Codes: H04N 13/366 using viewer tracking

Abstract

Image information displayed on an electronic device can be adjusted based at least in part upon a relative position of a viewer with respect to a device. In some embodiments, image stabilization can be provided such that an image remains substantially consistent from the point of view of the viewer, not the display element of the device. The image can be stretched, rotated, compressed, or otherwise manipulated based at least in part upon the relative viewing position. Similarly, the viewer can move relative to the device to obtain different views, but views that are consistent with the viewer looking at an object, for example, through a piece of glass. The device can overlay information on the image that will adjust with the adjusted image. Three-dimensional modeling and display can be used to offset parallax and focus point effects.

Claims

1. A computer-implemented method of displaying an image on a computing device, comprising:

determining that a first physical object is in at least one of a first position or a first orientation relative to a display of a computing device;

determining a first perspective based on at least one of the first position or the first orientation;

determining a first representation of a virtual object that, from the first perspective, appears to be in at least one of a second position or a second orientation relative to the display;

displaying on the display a first image including the first representation of the virtual object;

determining that the first physical object is in at least one of a third position relative to the display that differs from the first position by at least a threshold distance or a third orientation that differs from the first orientation by at least a threshold angle;

determining a second perspective based on at least one of the third position or the third orientation;

determining a second representation of the virtual object that, from the second perspective, appears to be in at least one of a substantially same position as the second position or a substantially same orientation as the second orientation relative to the display; and

displaying on the display a second image including the second representation of the virtual object.

2. The computer-implemented method according to claim 1, further comprising:

capturing image information including representations of a second physical object,

wherein the first image further includes a first representation of the second physical object and the second image further includes a second representation of the second physical object.

3. The computer-implemented method of claim 1, further comprising:

capturing image information of the first physical object using at least one image capture element of the computing device.

4. The computer-implemented method of claim 1, further comprising:

capturing image information of the first physical object using at least one of a thermal imager or an infrared radiation imager.

5. The computer-implemented method of claim 1, further comprising:

determining first position information associated with the first physical object;

determining second position information associated with the first physical object; and

determining the first position as a three-dimensional position relative to the computing device based at least in part on the first position information and the second position information.

6. The computer-implemented method of claim 5, further comprising:

capturing first image information of the first physical object using a first image capture element of the computing device, wherein the first position information is determined based at least in part on the first image information; and

capturing second image information of the first physical object using a second image capture element of the computing device, wherein the second position information is determined based at least in part on the second image information.

7. The computer-implemented method of claim 1, further comprising:

determining at least one of position information or orientation information associated with at least one feature of a person near the computing device; and

determining at least one of the first position based on the position information or the first orientation based on the orientation information.

8. The computer-implemented method of claim 7, further comprising:

determining a change from at least one of the first position to the third position or the first orientation to the third orientation based on at least one of movement of the person or movement of the computing device.

9. The computer-implemented method of claim 1, further comprising:

performing at least one manipulation of the first image to generate the second image.

10. The computer-implemented method of claim 9, further comprising:

determining the at least one manipulation to perform on the first image based on at least one of stretching, compressing, or rotating the first image.

11. A computing device, comprising:

a processor;

a display; and

a memory device including instructions that, when executed by the processor, cause the computing device to;

determine that a first physical object is in at least one of a first position or a first orientation relative to the display;

determine a first perspective based on at least one of the first position or the first orientation;

determine a first representation of a virtual object that, from the first perspective, appears to be in at least one of a second position or a second orientation relative to the display;

display on the display a first image including the first representation of the virtual object;

determine that the first physical object is in at least one of a third position relative to the display that differs from the first position by at least a threshold distance or a third orientation that differs from the first orientation by at least a threshold angle;

determine a second perspective based on at least one of the third position or the third orientation;

determine a second representation of the virtual object that, from the second perspective, appears to be in at least one of a substantially same position as the second position or a substantially same orientation as the second orientation relative to the display; and

display on the display a second image including the second representation of the virtual object.

12. The computing device of claim 11, wherein the instructions when executed further cause the computing device to:

capture image information including representations of a second physical object,

wherein the first image further includes a first representation of the second physical object and the second image further includes a second representation of the second physical object.

13. The computing device of claim 11, wherein the instructions when executed further cause the computing device to:

determine at least one of the first position or the first orientation using at least one of thermal imaging, infrared radiation detection, or motion detection.

14. The computing device according to of claim 11, wherein the instructions when executed further cause the computing device to:

determine at least one of position information or orientation information associated with at least one feature of a person near the computing device; and

determine at least one of the first position based on the position information or the first orientation based on the orientation information.

15. The computing device of claim 14, wherein the instructions when executed further cause the computing device to:

determine a change from at least one of the first position to the third position or the first orientation to the third orientation based on at least one of movement of the person or movement of the computing device.

16. A non-transitory computer-readable storage medium storing instructions for displaying an image on a computing device, the instructions, when executed by a processor of the computing device, causing the computing device to perform a method comprising:

determining that a first physical object is in at least one of a first position or a first orientation relative to a display of the computing device;

determining a first perspective based on at least one of the first position or the first orientation;

determining a first representation of a virtual object that, from the first perspective, appears to be in at least one of a second position or a second orientation relative to the display;

displaying on the display a first image including the first representation of the virtual object;

determining that the first physical object is in at least one of a third position relative to the display that differs from the first position by at least a threshold distance or a third orientation that differs from the first orientation by at least a threshold angle;

determining a second perspective based on at least one of the third position or the third orientation;

determining a second representation of the virtual object that, from the second perspective, appears to be in at least one of a substantially same position as the second position or a substantially same orientation as the second orientation relative to the display; and

displaying on the display a second image including the second representation of the virtual object.

17. The non-transitory computer-readable storage medium of claim 16, wherein the instructions when executed further cause the computing device to perform:

determining the first position based at least in part on image information of the first physical object.

18. The non-transitory computer-readable storage medium of claim 17, wherein the instructions when executed further cause the computing device to perform:

determining the first position based at least in part on at least one of thermal imaging, infrared radiation detection, or motion detection.

19. The non-transitory computer-readable storage medium of claim 16, wherein the instructions when executed further cause the computing device to perform:

determining at least one of position information or orientation information associated with at least one feature of a person near the computing device; and

determining at least one of the first position based on the position information or the first orientation based on the orientation information.

20. The non-transitory computer-readable storage medium of claim 19, wherein the instructions when executed further cause the computing device to perform:

determining a change from at least one of the first position to the third position or the first orientation to the third orientation based on at least one of movement of the person or movement of the computing device.