Image capture guidance to reduce specular reflection effects

• US 9,912,847 B1
• Filed: 09/25/2012
• Issued: 03/06/2018
• Est. Priority Date: 09/25/2012
• Status: Expired due to Fees

• Current Assignee: Amazon Technologies, Inc. (Amazon.com, Inc.)
• Original Assignee: Amazon Technologies, Inc. (Amazon.com, Inc.)
• Inventors: Yuan, Chang, Baldwin, Leo B., Heller, Geoffrey Scott, Bibireata, Daniel, LeGrand, III, Louis L., MacGregor, Christopher S. A., Vaughn, Charles P., Penov, Francislav P.
• Primary Examiner(s): Patel, Jay
• Assistant Examiner(s): Haghani, Shadan E
• Application Number:
  US13/626,596
• Time in Patent Office: 1,988 Days
• CPC Class Codes: H04N 13/20 Image signal generators
• CPC Class Codes: H04N 23/00 Cameras or camera modules comprising electronic image sensors; Control thereof
• CPC Class Codes: H04N 23/64 Computer-aided capture of images, e.g. transfer from script file into camera, check of taken image quality, advice or proposal for image composition or decision on when to take image
• CPC Class Codes: H04N 23/71 Circuitry for evaluating the brightness variation

Abstract

A user can capture an image of an object, using a computing device, to obtain information about that object. If a specular highlight (or other saturated region) is detected, the device can attempt to determine a location of a light source associated with the highlight. The device can then provide instructions as to a direction to move in order to reduce the presence of the specular highlight in subsequent images. Multiple images of the object can be captured and analyzed to generate a three-dimensional reconstruction of the environment, whereby a position of the light source can be determined. In other embodiments, movement of the specular reflections in response to movement of the device is used to determine a direction of the light source. In other embodiments, an image of the user is captured to determine the position of the light source based on shadows or reflections on the user.

Claims

1. A computer-implemented method of identifying a planar object, comprising:

detecting the planar object within a field of view of a first camera of a computing device;

determining that an intensity of light, reflected by the planar object and incident on at least one image sensor region of the first camera, exceeds an intensity threshold;

generating first image data of the planar object using the first camera of the computing device;

generating second image data of a planar surface of the planar object using a second camera of the computing device, the first camera being separated from the second camera by a distance;

registering, as a stereo image pair, the first image data and the second image data to substantially align a plurality of features in the first image data with a plurality of corresponding features in the second image data;

determining an orientation of the planar object, based at least on the first image data and the second image data in the stereo image pair;

determining a first location of a first region of the first image data where the determined intensity of light exceeds the intensity threshold and a second location of a second region of the second image data where the determined intensity of light exceeds the intensity threshold, the first region and the second region being associated with identifying features of the planar object represented in the first and second image data;

determining a difference in appearance of the first region and the second region;

based at least in part upon the orientation of the planar surface of the planar object, the first location, the second location and the difference in appearance, determining a direction of a light source, with respect to the computing device; and

based at least in part upon the determined direction of the light source, displaying, on a display screen of the computing device at least one instruction for changing an orientation of the computing device, relative to the determined direction of the light source, to cause an intensity of light to fall below the intensity threshold.

2. The computer-implemented method of claim 1, further comprising:

changing the orientation of the computing device;

capturing at least one subsequent image; and

providing the at least one subsequent image to at least one object recognition algorithm configured to provide information about the planar object.

3. The computer-implemented method of claim 1, wherein the at least one instruction for changing an orientation includes at least one of text instructions for adjusting the direction or a directional icon associated with a direction for adjusting the direction of the computing device, the method further comprising updating the at least one instruction based at least in part on detected movement of the computing device at least until the computing device is in a location where the intensity of light falls below the intensity threshold.

4. The computer-implemented method of claim 1, wherein the first image data and the second image data are frames of video, and wherein determining the direction of the light source is further based at least in part upon movement of the first region and the second region as detected between different frames of video.

5. A computer-implemented method comprising:

generating first image data representing a planar object using a first camera of a computing device;

generating second image data representing the planar object using a second camera of the computing device, the first camera being separated from the second camera by a distance;

generating a stereo image pair using at least the first image data and the second image data;

determining an orientation of a planar surface of the planar object, based at least on the first image data and the second image data in the stereo image pair;

detecting a location of a region in the first image data where intensity values exceed an intensity threshold, the region being associated with identifying features of the planar object in the first image data;

determining, based at least in part upon the orientation of the planar surface of the planar object and the location of the region with respect to the first image data, a direction of a light source, with respect to the computing device, that caused a specular reflection from the planar object;

determining, based at least in part upon the direction of the light source, a first movement of the computing device relative to the direction of the light source, wherein the first movement causes the intensity values in the region associated with the identifying features of the planar object in the first image data to fall below the intensity threshold; and

providing information about the first movement via the computing device.

6. The computer-implemented method of claim 5, further comprising:

detecting motion of the computing device using at least one sensor of the computing device;

capturing additional image data in response to detecting the motion;

determining an intensity of light in the region associated with the identifying features of the planar object in the additional image data falls below the intensity threshold; and

providing the additional image data for processing.

7. The computer-implemented method of claim 6, further comprising processing the additional image data using at least one of an object detection algorithm, an image matching algorithm, a facial recognition algorithm, a symbol reading algorithm, or an optical character recognition (OCR) algorithm.

8. The computer-implemented method of claim 6, further comprising detecting the motion by analyzing the additional image data, or by using at least one of an accelerometer, an electronic gyroscope, an electronic compass, or an inertial sensor.

9. The computer-implemented method of claim 5, further comprising:

detecting motion of the computing device using at least one sensor of the computing device;

capturing additional image data in response to detecting the motion;

determining an intensity of light in the region associated with the identifying features of the planar object in the additional image data remains above the intensity threshold; and

providing updated information indicating additional movements of the computing device.

10. The computer-implemented method of claim 5, further comprising:

generating additional image data representing the planar object when the first camera is at a different location;

determining a location of the first camera when generating the additional image data;

determining a location of the region associated with the identifying features of the planar object in the additional image data; and

determining the direction of the light source based at least in part upon the location of the region associated with the identifying features of the planar object in the additional image data and the location of the first camera when generating the additional image data.

11. The computer-implemented method of claim 10, further comprising:

generating additional image data from a plurality of additional images captured over a period of time;

determining a change in location of the region associated with the identifying features of the planar object in the additional image data; and

determining the direction of the light source based at least in part upon the change in location of the region between the plurality of additional images.

12. The computer-implemented method of claim 5, further comprising:

determining a location of the region associated with the identifying features of the planar object in the second image data; and

determining the direction of the light source based at least in part upon the location of the region associated with the identifying features of the planar object in the second image data.

13. The computer-implemented method of claim 5, further comprising:

detecting the location of the region associated with identifying features of the planar object in the first image data by locating pixel values that exceed an allowable pixel value.

14. The computer-implemented method of claim 5, further comprising:

providing a notification when the first image data meets at least one processing criterion.

15. A computing device, comprising:

a processor;

a first camera;

a second camera separated from the first camera by a distance; and

a memory device including instructions that, when executed by the processor, cause the computing device to;

generate first image data representing a planar object using the first camera of the computing device;

generate second image data representing the planar object using the second camera of the computing device;

generate a stereo image pair using at least the first image data and the second image data;

determine an orientation of a planar surface of the planar object, based at least on the first image data and the second image data in the stereo image pair;

detect a location of a region in the first image data where intensity values exceed an intensity threshold, the region being associated with identifying features of the planar object in the first image data;

determine, based at least in part upon the orientation of the planar surface of the planar object and the location of the region with respect to the first image data, a direction of a light source, with respect to the computing device, that caused a specular reflection from the planar object;

determine, based at least in part upon the direction of the light source, a first movement of the computing device relative to the direction of the light source, wherein the first movement causes the intensity values in the region associated with the identifying features of the planar object in the first image data to fall below the intensity threshold; and

provide information about the first movement via the computing device.

16. The computing device of claim 15, wherein the instructions when executed further cause the computing device to:

detect motion of the computing device using at least one sensor of the computing device;

capture additional image data in response to detecting the motion;

determine an intensity of light in the region associated with the identifying features of the planar object in the additional image data falls below the intensity threshold; and

provide the additional image data for processing.

17. The computing device of claim 15, wherein the instructions when executed further cause the computing device to:

detect motion of the computing device using at least one sensor of the computing device;

capture additional image data in response to detecting the motion;

determine an intensity of light in the region associated with the identifying features of the planar object in the additional image data remains above the intensity threshold; and

provide updated information indicating additional movements of the computing device.

18. The computing device of claim 15, wherein the instructions when executed further cause the computing device to:

generate additional image data representing the planar object when the first camera is at a different location;

determine a location of the first camera when generating the additional image data;

determine a location of the region associated with the identifying features of the planar object in the additional image data; and

determine the direction of the light source based at least in part upon the location of the region associated with the identifying features of the planar object in the additional image data and the location of the first camera when generating the additional image data.

19. The computing device of claim 18, wherein the instructions when executed further cause the computing device to:

generate additional image data from a plurality of additional images captured over a period of time;

determine a change in location of the region associated with the identifying features of the planar object in the additional image data; and

determine the direction of the light source based at least in part upon the change in location of the region between the plurality of additional images.

20. The computing device of claim 15, wherein the instructions when executed further cause the computing device to:

determine a location of the region associated with the identifying features of the planar object in the second image data; and

determine the direction of the light source based at least in part upon the location of the region associated with the identifying features of the planar object in the second image data.

21. A non-transitory computer-readable storage medium including instructions that, when executed by a processor of a computing device, cause the computing device to:

generate first image data representing a planar object using a first camera of the computing device;

generate second image data representing the planar object using a second camera of the computing device, the first camera being separated from the second camera by a distance;

generate a stereo image pair using at least the first image data and the second image data;

determine an orientation of a planar surface of the planar object, based at least on the first image data and the second image data in the stereo image pair;

detect a location of a region in the first image data where intensity values exceed an intensity threshold, the region being associated with identifying features of the planar object in the first image data;

determine, based at least in part upon the orientation of the planar surface of the planar object and the location of the region with respect to the first image data, a direction of a light source, with respect to the computing device, that caused a specular reflection from the planar object;

determine, based at least in part upon the direction of the light source, a first movement of the computing device relative to the direction of the light source, wherein the first movement causes the intensity values in the region associated with the identifying features of the planar object in the first image data to fall below the intensity threshold; and

provide information about the first movement via the computing device.

22. The non-transitory computer-readable storage medium of claim 21, wherein the instructions when executed further cause the computing device to:

generate additional image data representing the planar object when the computing device is at a different location;

determine a location of the computing device when generating the additional image data;

determine a location of the region associated with the identifying features of the planar object in the additional image data; and

determine the direction of the light source based at least in part upon the location of the region associated with the identifying features of the planar object in the additional image data and the location of the computing device when generating the additional image data.

23. The non-transitory computer-readable storage medium of claim 22, wherein the instructions when executed further cause the computing device to:

generate additional image data from a plurality of additional images captured over a period of time;

determine a change in location of the region associated with the identifying features of the planar object in the additional image data; and

determine the direction of the light source based at least in part upon the change in location of the region between the plurality of additional images.

24. The non-transitory computer-readable storage medium of claim 21, wherein the instructions when executed further cause the computing device to:

determine a location of the region associated with the identifying features of the planar object in the second image data; and

determine the direction of the light source based at least in part upon the location of the region associated with the identifying features of the planar object in the second image data.