Simulating three-dimensional features

• US 9,041,734 B2
• Filed: 08/12/2011
• Issued: 05/26/2015
• Est. Priority Date: 07/12/2011
• Status: Active Grant

• Current Assignee: Amazon Technologies, Inc. (Amazon.com, Inc.)
• Original Assignee: Amazon Technologies, Inc. (Amazon.com, Inc.)
• Inventors: Look, Howard D., Baldwin, Leo B., Karakotsios, Kenneth M., Hodge, Dennis, Noble, Isaac S., Ivanchenko, Volodymyr V., Bezos, Jeffrey P.
• Primary Examiner(s): Merouan, Abderrahim
• Application Number:
  US13/209,243
• Publication Number:
  US 20130016102A1
• Time in Patent Office: 1,383 Days
• Field of Search: 345/428, 345/853, 345/851, 345/441, 345/629, 348/46, 348/190, 348/187, 348/51, 382/103, 382/190, 382/160, 382/224, 715/739, 715/744, 715/849
• US Class Current: 345/629
• CPC Class Codes: G06T 15/20 Perspective computation
• CPC Class Codes: G06T 2207/10016 Video; Image sequence
• CPC Class Codes: G06T 2207/30201 Face
• CPC Class Codes: G06T 7/73 using feature-based methods
• CPC Class Codes: G06V 40/19 Sensors therefor

Abstract

Image information displayed on an electronic device can be modified based at least in part upon a relative position of a user with respect to a device. Mapping, topological or other types of positional data can be used to render image content from a perspective that is consistent with a viewing angle for the current relative position of the user. As that viewing angle changes, as a result of movement of the user and/or the device, the content can be re-rendered or otherwise updated to display the image content from a perspective that reflects the change in viewing angle. Simulations of effects such as parallax and occlusions can be used with the change in perspective to provide a consistent user experience that provides a sense of three-dimensional content even when that content is rendered on a two-dimensional display. Lighting, shading and/or other effects can be used to enhance the experience.

Claims

1. A method of providing viewpoint-dependent image display, comprising:

capturing a first image of a user of an electronic device using at least one imaging element of the electronic device;

determining a first relative viewpoint of the user with respect to the electronic device based at least in part upon the first image;

obtaining information for generating a graphical three-dimensional representation of a plurality of objects;

rendering a first graphical representation of the plurality of objects for display on a display element of the electronic device to correspond to the first relative viewpoint of the user, a first object of the plurality of objects being rendered to occlude a second object of the plurality of objects by a first amount;

detecting, using an orientation determining element of the electronic device, a change in orientation of the electronic device;

determining, based at least in part on the change in orientation, a second relative viewpoint of the user with respect to the electronic device, wherein the second relative viewpoint is different from the first relative viewpoint; and

rendering, based at least in part on the change in orientation, a second graphical representation of the first object and the second object to correspond to the second relative viewpoint of the user, wherein rendering the second graphical representation includes;

rendering, by utilizing three-dimensional data of the first object and three-dimensional data of the second object, the first object to occlude the second object by a second amount, the first amount being different than the second amount,

determining a shadow to be cast on the second object based at least in part on the second amount of occlusion and an intersection of virtual light from a virtual light source with the first object, and

rendering the shadow on the second object.

2. The method of claim 1, wherein one of the first amount or the second amount is zero.

3. The method of claim 1, wherein the at least one first image of the user is captured using at least two imaging elements to determine a three-dimensional relative position of the user with respect to the electronic device.

4. The method of claim 1, wherein determining the second relative viewpoint includes at least one of:

capturing at least one second image of the user, and

detecting a change in orientation using at least one orientation determining element of the electronic device.

5. The method of claim 1, wherein rendering the second graphical representation of the plurality of objects for display includes utilizing a three-dimensional model to determine the first amount and second amount of occlusion.

6. The method of claim 1, wherein rendering the second graphical representation of the plurality of objects for display includes shifting a displayed position of at least one object based upon a viewpoint change between the first relative viewpoint and the second relative viewpoint, the shifting providing a display of image content previously occluded by the first object of the plurality of objects in the at least one first image.

7. The method of claim 1, wherein rendering the second graphical representation of the plurality of objects for display includes updating at least one of a rendering or a shading of the graphical three-dimensional representation of the plurality of objects based at least in part upon at least one of a change in a relative viewpoint, a change in an orientation, or a change in a relative location of at least one light source.

8. A method of providing a graphical representation of an image of a plurality of objects for display on an electronic device, comprising:

determining a first relative viewpoint of a user with respect to the electronic device based at least in part on image data of the user;

rendering a first graphical representation of the image to be displayed on a display element of the electronic device, the first graphical representation including a first object and a second object rendered based at least in part on the first relative viewpoint, the first object at least partially occluding the second object by a first amount;

detecting, using an orientation determining element of the electronic device, a change in orientation of the electronic device;

detecting, based at least in part on the change in orientation, a change from the first relative viewpoint to a second relative viewpoint; and

rendering, based at least in part on the second relative viewpoint, a second graphical representation of the image to be displayed on the display element, the second graphical representation of the image being rendered to correspond to the change from the first relative viewpoint to the second relative viewpoint, wherein rendering includes;

rendering, by utilizing three-dimensional data of the first object and three-dimensional data of the second object, the first object to occlude the second object by a second amount, the first amount being different than the second amount,

determining a shadow to be cast on the second object based at least in part on the second amount of occlusion and an intersection of virtual light from a virtual light source with the first object, and

rendering the shadow on the second object.

9. The method of claim 8, wherein the image data of the user is obtained at least in part from an imaging element of the electronic device.

10. The method of claim 8, wherein the change from the first relative viewpoint to the second relative viewpoint is based at least in part on at least one of a movement of the user or a movement of the electronic device.

11. The method of claim 8, wherein at least one of the first relative viewpoint and the second relative viewpoint is determined based at least in part upon data obtained using at least one image capture element of the electronic device.

12. The method of claim 8, wherein at least one of the first graphical representation and the second graphical representation includes at least one of map data, topographic data, or three-dimensional position data.

13. The method of claim 8, wherein the second graphical representation includes a portion of the first object occluded by the second object in the first graphical representation.

14. The method of claim 8, wherein determining the first relative viewpoint between the user and the electronic device includes using at least two imaging elements to determine a three-dimensional relative position of the user with respect to the electronic device.

15. The method of claim 8, wherein determining at least one of the first relative viewpoint and the change from the first relative viewpoint to the second relative viewpoint includes capturing image data of the user with at least one of thermal imaging, infrared radiation detection or motion detection.

16. The method of claim 8, wherein rendering the first graphical representation of the image includes at least one of lighting or shading based at least in part upon a determined relative position of at least one light source.

17. The method of claim 16, wherein light source is at least one of a virtual light source or a physical light source near the electronic device.

18. The method of claim 8, wherein a change in a position of the second object in the second graphical representation of the image is based at least in part upon at least one of a position of the second object, a current level of zoom, or a relative user distance.

19. The method of claim 8, further comprising:

rendering at least one animation over at least a portion of the rendered second graphical representation of the image.

20. The method of claim 8, wherein at least one object in the first or second graphical representation of the image has associated data in three-dimensions, the at least one object rotatable to correspond at least in part to the change from the first relative viewpoint to the second relative viewpoint.

21. The method of claim 8, wherein the second graphical representation of the image contains a rotated representation of the second object providing additional information.

22. A computing device, comprising:

a processor;

a display element;

at least one image capture element; and

a memory device including instructions operable to be executed by the processor to perform a set of actions, enabling the computing device to;

determine, based on an image of a user obtained from the at least one image capture element, a first relative viewpoint of at least one of the user or the computing device;

render a first graphical representation of the image to be displayed on the display element, the first graphical representation including a first object and a second object rendered from a first perspective associated with the first relative viewpoint, the first object at least partially occluding the second object by a first amount;

detect, using an orientation determining element of the computing device, a change in orientation of the computing device; and

render, based at least in part on the change in orientation, a second graphical representation of the image to be displayed on the display element, the second graphical representation of the image being rendered to correspond to the change from the first relative viewpoint to a second relative viewpoint, the second graphical representation rendered from a second perspective associated with the second relative viewpoint, the rendering including;

rendering, by utilizing three-dimensional data of a first object and three-dimensional data of a second object, the first object to occlude the second object by a second amount, the first amount being different than the second amount,

determining a shadow to be cast on the second object based at least in part on the second amount of occlusion and an intersection of virtual light from a virtual light source with the first object, and

rendering the shadow on the second object.

23. The computing device of claim 22, wherein the change between the first relative viewpoint and the second relative viewpoint is determined by detecting at least one of a movement of the user a movement of the computing device.

24. The computing device of claim 22, further comprising:

at least one orientation determining element operable to determine at least one of a change in orientation or a position of the computing device.

25. The computing device of claim 22, further comprising:

at least one light sensing element for use in at least one of lighting or shading of a graphical representation to be displayed.