Specularity determination from images

• US 9,076,032 B1
• Filed: 06/30/2014
• Issued: 07/07/2015
• Est. Priority Date: 12/16/2013
• Status: Active Grant

• Current Assignee: Google LLC (Alphabet Inc.)
• Original Assignee: Google Inc. (Alphabet Inc.)
• Inventors: Mesolongitis, Agis Iakovos, Duchaineau, Mark Alan, Jones, Jonah
• Primary Examiner(s): Seth, Manav
• Application Number:
  US14/319,024
• Publication Number:
  US 20150170369A1
• Time in Patent Office: 372 Days
• US Class Current: 1/1
• CPC Class Codes: G01N 21/55 Specular reflectivity
• CPC Class Codes: G06K 17/00 Methods or arrangements for effecting co-operative working between equipments covered by two or more of main groups G06K1/00 - G06K15/00, e.g. automatic card files incorporating conveying and reading operations
• CPC Class Codes: G06T 15/50 Lighting effects
• CPC Class Codes: G06T 17/00 Three dimensional [3D] modelling, e.g. data description of 3D objects
• CPC Class Codes: G06T 17/05 Geographic models
• CPC Class Codes: G06T 17/10 Constructive solid geometry [CSG] using solid primitives, e.g. cylinders, cubes
• CPC Class Codes: G06T 2207/10032 Satellite or aerial image; Remote sensing
• CPC Class Codes: G06T 7/507 from shading G06T7/586 takes precedence
• CPC Class Codes: G06T 7/62 of area, perimeter, diameter or volume
• CPC Class Codes: G06T 7/90 Determination of colour characteristics
• CPC Class Codes: G06V 10/40 Extraction of image or video features
• CPC Class Codes: G06V 10/60 relating to illumination properties, e.g. using a reflectance or lighting model
• CPC Class Codes: G06V 40/172 Classification, e.g. identification

Abstract

Aspects of the disclosure relate generally to determine specularity of an object. As an example an object or area of geometry may be selected. A set of images that include the area of geometry may be captured. This set of images may be filtered to remove images that do not show the area of geometry well, such as if the area is in a shadow or occluded by another object. A set of intensity values for the area are determined for each image. A set of angle values for each image is determined based on at least a direction of a camera that captured the particular image when the particular image was captured. The set of average intensities and the set of angle values are paired and fit to a curve. The specularity of the area may then be classified based on at least the fit.

Claims

1. A method for determining specularity, the method comprising:

identifying, by the one or more computing devices, a set of images including a geographic area;

for each particular image of the set determining, by the one or more computing devices, an intensity value for the geographic area for that particular image to generate a set of intensity values;

identifying, by the one or more computing devices, a specularity function based on the set of intensity values; and

classifying, by the one or more computing devices, specularity of the geographic area based on the specularity function.

2. The method of claim 1, wherein determining the intensity value for each particular image includes determining an average intensity for the geographic area in each particular image.

3. The method of claim 1, wherein the specularity function includes a function value representing an average color of an object corresponding to the geographic area.

4. The method of claim 1, wherein the specularity function includes a function value representing a height of a lobe of a curve of the specularity function relative of an offset value.

5. The method of claim 4, wherein the specularity function value is used to classify the specularity of the geographic area.

6. The method of claim 1, wherein the specularity function corresponds to an offset Gaussian curve.

7. The method of claim 1, further comprising:

receiving, from a computing device, a request for rendering an image that includes the geographic area, the request identifying a camera angle; and

providing, to the computing device, instructions to render the image based on the camera angle and the specularity classification of the area.

8. A system for determining specularity, the system comprising one or more computing devices configured to:

identify a set of images including a geographic area;

for each particular image of the set determine an intensity value for the geographic area for that particular image to generate a set of intensity values;

identify a specularity function based on the set of intensity values; and

classify specularity of the geographic area based on the specularity function.

9. The system of claim 8, wherein the one or more computing devices are configured to determine the intensity value for each particular image by determining an average intensity for the geographic area in each particular image.

10. The system of claim 8, wherein the specularity function includes a function value representing an average color of an object corresponding to the geographic area.

11. The system of claim 8, wherein the specularity function includes a function value representing a height of a lobe of a curve of the specularity function relative of an offset value.

12. The system of claim 11, wherein the function value is used to classify the specularity of the geographic area.

13. The system of claim 11, wherein the function corresponds to an offset Gaussian curve.

14. The system of claim 11, wherein the one or more computing devices are further configured to:

receive, from a computing device, a request for rendering an image that includes the geographic area, the request identifying a camera angle; and

provide, to the computing device, instructions to render the image based on the camera angle and the specularity classification of the area.

15. A system for determining specularity, the system comprising one or more computing devices configured to:

divide an object into a set of polygon-shaped surface geometry patches;

for each particular patch of the set of polygon-shaped surface geometry patches, identify a specularity function by;

identifying a set of images of the object captured at different times from different orientations;

determining an average intensity value of the particular patch for each image of the set of images; and

determining an angle value of the particular patch for each image of the set of images; and

using, by the one or more computing devices, the specularity function for each particular patch to determine a degree of shininess for that particular patch.

16. The system of claim 15, wherein the one or more computing devices are further configured to propagate the degrees of shininess between neighboring patches in a common surface of the object.

17. The system of claim 15, wherein the one or more computing devices are further configured to propagate the degrees of shininess by performing an erosion process and subsequently using a flood filling process using the degree of shininess for at least one of the particular patches.

18. The system of claim 15, wherein the one or more computing devices are further configured to filter a given set of images of a given patch by:

identifying combinations of camera and patch positions for the given set of images;

determining visibility of the given patch to the camera positions based on the combinations; and

determining visibility of the given patch to the sun at the time each image was captured.

19. The system of claim 15, wherein the one or more computing devices are further configured to determine each particular degree of shininess further based on a relative peak height of the specularity function.

20. The system of claim 15, wherein the one or more computing devices are further configured to determine each particular degree of shininess further based on a threshold value for a relative peak height of the specularity function.