Intensity modeling for rendering realistic images

• US 9,449,427 B1
• Filed: 05/13/2011
• Issued: 09/20/2016
• Est. Priority Date: 05/13/2011
• Status: Expired due to Fees

• Current Assignee: Amazon Technologies, Inc. (Amazon.com, Inc.)
• Original Assignee: Amazon Technologies, Inc. (Amazon.com, Inc.)
• Inventors: Baldwin, Leo B., Ivanchenko, Volodymyr V., Karakotsios, Kenneth M., Noble, Isaac S.
• Primary Examiner(s): Wu, Xiao
• Assistant Examiner(s): Elbinger, Steven
• Application Number:
  US13/107,749
• Time in Patent Office: 1,957 Days
• Field of Search: None
• US Class Current: 1/1
• CPC Class Codes: A63F 2300/6692 using special effects, generally involving post-processing, e.g. blooming
• CPC Class Codes: G06T 15/50 Lighting effects
• CPC Class Codes: G06T 2215/16 Using real world measurements to influence rendering

Abstract

An electronic device can utilize one or more sensors and/or imaging elements to determine the relative position of at least one light source relative to the device. In various embodiments, the intensity of light can be determined in various directions around the device. By determining the relative intensity around the device, an approximate direction of one or more light sources can be determined. Utilizing the relative position of each light source, the electronic device can properly light or shade a graphical object to be rendered by the device, or otherwise process image information captured by the device.

Claims

1. A computer-implemented method of displaying an image on an electronic device, comprising:

determining, using a light sensor, light information from a plurality of substantially straight different light paths of the electronic device, the light paths being configured to capture light from different primary directions with respect to the electronic device and direct the light to the light sensor, and including;

an entrance, and an exit;

determining a light type corresponding to the light information, the light type being one of a point light source or a non-point light source;

determining a relative intensity value of the light information from the plurality of light paths;

based at least in part upon the relative intensity value and the primary directions of the light paths, determining light source information including at least a projection direction of a light source relative to the electronic device;

rendering the image based at least in part on the light information and the light type, the rendering the image including at least one of a lighting process, a shading process and a glint process, wherein the image rendered based on the light type being the point light source is different than a second image that would have been rendered based on the light type being the non-point light source; and

providing the image for display on the electronic device.

2. The computer-implemented method of claim 1, further comprising:

generating an intensity model using at least the relative intensity value and the primary directions of the light paths, the intensity model representing an intensity of light from a plurality of directions around the device, and the intensity model comprises a three-dimensional model useful in determining intensity information for substantially any direction around the electronic device.

3. The computer-implemented method of claim 1, wherein each entrance of each light path has substantially a same diameter as each exit of each light path.

4. The computer-implemented method of claim 1, wherein determining intensity information for any direction around the electronic device not corresponding to a primary direction of a light path includes interpolating between light intensity values for at least two adjacent primary directions.

5. The computer-implemented method of claim 1, further comprising:

determining a relative location of the light source using the relative intensity values, the relative location including a distance between the electronic device and the light source.

6. A computer-implemented method of providing content for display on an electronic device, comprising:

determine, using a light sensor of the electronic device, light information from a plurality of substantially straight different light paths of the electronic device;

determining a light type corresponding to the light information, the light type being one of a point light source or a non-point light source;

analyzing the light information from the plurality of light paths using the light sensor to determine at least a direction of greatest light intensity relative to the electronic device; and

rendering content for display on the electronic device based at least in part on the direction of greatest light intensity and the light type, wherein the content rendered based on the light type being the point light source is different than a second content that would have been rendered based on the light type being the non-point light source.

7. The computer-implemented method of claim 6, wherein the electronic device is one of a tablet computer, a smart phone, a personal computer, a personal data assistant, portable display, a mobile electronic device and a gaming device.

8. The computer-implemented method of claim 6, further comprising:

capturing image information using at least one imaging element of the electronic device.

9. The computer-implemented method of claim 8, wherein the image information is captured using at least two imaging elements, and further comprising:

determining a location of an object in the image information relative to the electronic device;

determining a three-dimensional model for at least one of lighting and shading the at least one object based at least in part upon light intensity information determined for the plurality of light paths and a primary capture direction of each of the light paths; and

rendering graphical information for the object with at least one of lighting and shading based at least in part on the determined three-dimensional model.

10. The computer-implemented method of claim 6, wherein the plurality of light paths include a plurality of optical fibers, each optical fiber having a first end positioned to capture light from a primary direction relative to the electronic device and a second end positioned to direct at least a portion of the captured light to at least one light sensor.

11. The computer-implemented method of claim 10, wherein each of the plurality of optical fibers directs light to a single light sensor of the electronic device.

12. The computer-implemented method of claim 6, wherein the plurality of light paths include at least two sets of light paths separated with respect to a casing of the electronic device, the light paths having a length approximately equal to a thickness of the casing, the light paths directing light to an associated light sensor.

13. The computer-implemented method of claim 12, wherein the light paths comprise a transmissive material formed in a respective opening in the casing.

14. The computer-implemented method of claim 12, wherein each light path in a set of light paths is configured to collect light from a different primary direction.

15. The computer-implemented method of claim 12, wherein each set of light paths forms a two-dimensional array of light paths configured to capture light over a semi-hemispherical range of angles and direct the captured light to a two-dimensional sensor array.

16. The computer-implemented method of claim 6, further comprising:

monitoring at least one of an orientation and a position of the electronic device; and

updating at least the direction of greatest light intensity relative to the electronic device based at least in part upon changes in at least one of the orientation and the position of the electronic device.

17. The computer-implemented method of claim 6, further comprising:

storing information for at least the direction of greatest light intensity such that a graphical element generated at a future time is capable of utilizing at least one of lighting and shading based upon at least the direction of greatest light intensity.

18. The computer-implemented method of claim 6, wherein an angular capture range of each light path at least partially overlaps an angular capture range of at least one other light path.

19. The computer-implemented method of claim 6, wherein a subset of the plurality of light paths is formed by a fiber bundle including multiple individual optical fibers, each optical fiber in the fiber bundle being tapered at a receiving end to capture light from a different primary direction.

20. The computer-implemented method of claim 6, further comprising:

rendering the content with penumbral blur based at least in part on the light type.

21. An electronic device, comprising:

a processor;

a display;

a plurality of substantially straight different light paths;

a sensor; and

memory including instructions that, when executed by the processor, cause the electronic device to;

determine, using the sensor, light information from the plurality of light paths;

determining a light type corresponding to the light information, the light type being one of a point light source or a non-point light source;

analyze the light information to determine at least a direction of greatest light intensity relative to the electronic device; and

render graphical information to be displayed on the display of the electronic device, the render the graphical information including at least one of lighting and shading based at least in part at least the direction of greatest light intensity relative to the electronic device and the light type, wherein the graphical information rendered based on the light type being the point light source is different than a second graphical information that would have been rendered based on the light type being the non-point light source.

22. The electronic device of claim 21, wherein the sensor is one of ambient light camera and an intensity-detecting sensor array.

23. The electronic device of claim 21, wherein the light paths include apertures formed into a casing of the electronic device.

24. The electronic device of claim 21, wherein the apertures are formed using at least one of mechanical drilling or laser processing.

25. The electronic device of claim 21, wherein the instructions when executed further cause the electronic device to:

capture image information using at least two imaging elements;

determine a relative position of an object in the captured image information;

determine a three-dimensional model for at least one of lighting and shading the object based at least in part upon the greatest light intensity relative to the electronic device; and

render the graphical information with the captured image information, based at least in part upon the three-dimensional model.

26. The electronic device of claim 21, further comprising:

an orientation element; and

wherein the instructions when executed further cause the electronic device to;

monitor an orientation of the electronic device using the orientation element; and

determine a change in light intensity information relative to the electronic device based at least in part upon changes in the orientation of the electronic device.

27. A non-transitory computer-readable storage medium including instructions that, when executed by a processor, cause the processor to:

determine light information from a plurality of light paths using a sensor of an electronic device;

determine a light type corresponding to the light information, the light type being one of a point light source or a non-point light source;

analyze the light information to determine at least a direction of greatest light intensity relative to the electronic device;

render graphical information to be displayed on a display of the electronic device, the render the graphical information including at least one of lighting and shading based at least in part on the direction of greatest light intensity relative to the electronic device and the light type, wherein the graphical information rendered based on the light type being the point light source is different than a second graphical information that would have been rendered based on the light type being the non-point light source; and

store information for at least the direction of greatest light intensity such that a graphical element generated at a subsequent time is enabled to utilize at least one of lighting and shading based at least in part on the direction of greatest light intensity.

28. The non-transitory computer-readable storage medium of claim 27, wherein the instructions when executed further cause the processor to:

capture image information using at least two sensors;

determine a location of an object in the image information relative to the at least two sensors;

determine a three-dimensional model for at least one of lighting and shading the object based at least in part upon the direction of greatest light intensity relative to the electronic device; and

render the graphical information and the image information based at least in part on the three-dimensional model.

29. The non-transitory computer-readable storage medium of claim 28, wherein the sensor is located on the electronic device, the electronic device including a display capable of displaying the rendered graphical information.

30. The non-transitory computer-readable storage medium of claim 27, the electronic device further comprising:

an orientation element, the orientation element enabling the processor to monitor an orientation of the electronic device, and determine a change in light intensity information relative to the electronic device based at least in part upon changes in the orientation of the electronic device.