Realistic image lighting and shading

• US 9,123,272 B1
• Filed: 05/13/2011
• Issued: 09/01/2015
• Est. Priority Date: 05/13/2011
• Status: Active Grant

• Current Assignee: Amazon Technologies, Inc. (Amazon.com, Inc.)
• Original Assignee: Amazon Technologies, Inc. (Amazon.com, Inc.)
• Inventors: Baldwin, Leo B., Karakotsios, Kenneth M., Ivanchenko, Volodymyr V., Noble, Isaac S., Hart, Gregory M., Bezos, Jeffrey P.
• Primary Examiner(s): Wu, Xiao
• Assistant Examiner(s): ELBINGER, STEVEN Z
• Application Number:
  US13/107,710
• Time in Patent Office: 1,572 Days
• Field of Search: None
• US Class Current: 1/1
• CPC Class Codes: G09G 2320/0626 for control of overall brightness
• CPC Class Codes: G09G 2360/144 the light being ambient light
• CPC Class Codes: G09G 3/003 to produce spatial visual effects
• CPC Class Codes: G09G 5/00 Control arrangements or circuits for visual indicators common to cathode-ray tube indicators and other visual indicators image data processing or generation, in general G06T

Abstract

An electronic device can utilize one or more sensors and/or imaging elements to determine the relative position of at least one light source relative to the device. In various embodiments, occlusions can be used to cause shadows to be cast on certain sensors. By determining the relative position of each occlusion relative to the sensor, the device can determine an approximate direction of the light source. Utilizing the relative position of a light source, the electronic device can properly light or shade a graphical object to be rendered by the device or otherwise process image information captured by the device.

Claims

1. A computer-implemented method of displaying an image on an electronic device, comprising:

under control of one or more computing systems configured with executable instructions,

determining a position of a shadow region corresponding to a shadow cast from at least one of a first occlusion and a second occlusion on a sensor, the first and second occlusions being at different known positions on the electronic device, a majority of the first occlusion being positioned substantially near a first edge of the sensor, a majority of the second occlusion being positioned substantially near a second edge of the sensor that is different than the first edge, the first occlusion comprising a first shape, the second occlusion comprising a second shape, the first shape being a different shape than the second shape;

based at least in part upon the determined position of the shadow region, determining light source information, the determined light source information including at least a projection direction of a light source relative to the electronic device;

rendering the image based at least in part on the determined light source information, the rendering including at least one of a lighting process, a shading process and a glint process; and

providing the image for display on the electronic device.

2. The computer-implemented method of claim 1, wherein the image is obtained at least in part from an image capture element of the electronic device.

3. The computer-implemented method of claim 2, wherein the image capture element is a camera.

4. The computer-implemented method of claim 1, further comprising:

determining light source information from at least two positions to determine a relative location of the light source, the relative location including a distance between the electronic device and the light source.

5. The computer-implemented method of claim 4, wherein rendering the image further includes, using the determined relative location of the light source to generate a three-dimensional rendering of the image.

6. A computer-implemented method of providing content for display on an electronic device, comprising:

under control of one or more computing systems configured with executable instructions,

capturing light information using a sensor of the electronic device;

analyzing the captured light information to determine a position of a shadow cast on the sensor, the shadow cast on the sensor by at least one of a first occluding element and a second occluding element positioned at different known locations with respect to the sensor, the first occlusion being positioned substantially near a first edge of the sensor, the second occluding element being positioned substantially near a second edge of the sensor that is different than the first edge, the first occlusion comprising a first shape, the second occlusion comprising a second shape that is different than the first shape;

based at least in part upon the determined position of the shadow, determining a projection direction of a light source relative to the electronic device; and

rendering content for display on the electronic device based at least in part on the determined projection direction of the light source.

7. The computer-implemented method of claim 6, wherein the electronic device is one of a tablet computer, a smart phone, a personal computer, a personal data assistant, portable display, a mobile electronic device and a gaming device.

8. The computer-implemented method of claim 6, further comprising:

capturing image information using an imaging element of the electronic device.

9. The computer-implemented method of claim 8, wherein the image information is captured using at least two imaging elements, and further comprising:

determining a location of an object in the captured image information relative to the electronic device;

determining a three-dimensional model for at least one of lighting and shading the at least one object based at least in part upon the determined relative location of the object and the determined projection direction of the light source; and

rendering graphical information based at least in part on the determined three-dimensional model.

10. The computer-implemented method of claim 6, further comprising:

monitoring at least one of an orientation and a position of the electronic device; and

updating the projection direction of the light source relative to the electronic device, based at least in part upon changes in at least one of the orientation and the position of the electronic device.

11. The computer-implemented method of claim 6, wherein the occluding element may be selectively activated, the computer-implemented method further comprising:

selectively activating the occluding element.

12. The computer-implemented method of claim 6, further comprising:

capturing an image using the imaging element; and

processing the image to remove an occluded region corresponding to the occluding element.

13. The computer-implemented method of claim 12, wherein the occluded region is at least partially opaque in at least one color.

14. The computer-implemented method of claim 6, further comprising:

storing information for the projection direction of the light source relative to the electronic device such that a graphical element generated at a future time utilizes at least one of lighting and shading based upon the projection direction of the light source relative to the electronic device.

15. An electronic device, comprising:

a processor;

a display;

a sensor; and

memory including instructions that, when executed by the processor, cause the electronic device to;

capture light information using the sensor of the electronic device;

analyze the captured light information to determine a position of a shadow on the sensor, the shadow caused by at least one of a first occluding element and a second occluding element, the first occlusion being positioned substantially near a first edge of the sensor, the second occluding element being positioned substantially near a second edge of the sensor that is different than the first edge, the first occlusion comprising a first shape, the second occlusion comprising a second shape that is different than the first shape;

based at least in part upon the determined position of the shadow, determine a projection direction of a light source relative to the electronic device; and

render graphical information to be displayed on the display of the electronic device, including shadow information, based at least in part on the determined projection direction of the light source relative to the electronic device.

16. The electronic device of claim 15, wherein the projection direction identifies a location of the light source.

17. The electronic device of claim 15, wherein the instructions when executed further cause the electronic device to:

capture image information using at least two imaging elements;

determine a relative position of an object in the captured image information;

determine a three-dimensional model for at least one of lighting and shading the object based at least in part upon the location of the object relative to the electronic device and the projection direction of the light source relative to the electronic device; and

render the graphical information with the captured image information, based at least in part upon the three-dimensional model.

18. The electronic device of claim 17, wherein the instructions when executed further cause the electronic device to:

render a three-dimensional representation of the object.

19. The electronic device of claim 15, further comprising:

an orientation element; and

wherein the instructions when executed further cause the electronic device to;

monitor an orientation of the electronic device using the orientation element; and

determine a change in projection direction of the light source relative to the electronic device, based at least in part upon changes in the orientation of the electronic device.

20. A non-transitory computer-readable storage medium including instructions that, when executed by a processor, cause the processor to:

capture light information using a sensor;

analyze the captured light information to determine a position of a shadow on the sensor, the shadow cast on the sensor by at least one of a first occluding element and a second occluding element, the first occlusion being positioned substantially near a first edge of the sensor, the second occluding element being positioned substantially near a second edge of the sensor that is different than the first edge, the first occlusion comprising a first shape, the second occlusion comprising a second shape that is different than the first shape;

based at least in part upon the determined position of the shadow, determine a projection direction of a light source relative to the sensor; and

render graphical information to be displayed, based at least in part on the determined projection direction of the light source relative to the sensor, the graphical information including at least one of lighting information, shading information and glint information.

21. The non-transitory computer-readable storage medium of claim 20, wherein the instructions when executed further cause the processor to:

capture image information using at least two sensors;

determine a location of an object in the captured image information relative to the at least two sensors;

determine a three-dimensional model for at least one of lighting and shading the object based at least in part upon the location of the object relative to the at least two sensors and the projection direction of the light source relative to the at least two sensors; and

render the graphical information and the captured image information based at least in part on the three-dimensional model.

22. The non-transitory computer-readable storage medium of claim 20,

wherein the sensor is located on an electronic device, the electronic device including a display capable of displaying the rendered graphical information.

23. The non-transitory computer-readable storage medium of claim 22, the electronic device further comprising:

an orientation element, the orientation element enabling the processor to monitor an orientation of the electronic device, and determine a change in the projection direction of the light source relative to the sensor, based at least in part upon a change in the orientation of the electronic device.