Methods and systems for use of a database of three-dimensional (3D) object data models for search queries

• US 9,529,826 B2
• Filed: 12/26/2013
• Issued: 12/27/2016
• Est. Priority Date: 12/26/2013
• Status: Active Grant

• Current Assignee: Google LLC (Alphabet Inc.)
• Original Assignee: Google Inc. (Alphabet Inc.)
• Inventors: Harp, Andrew Logan, Kuffner, James Joseph, Francis, Anthony Gerald
• Primary Examiner(s): NGUYEN, KIM T
• Application Number:
  US14/141,144
• Publication Number:
  US 20150186418A1
• Time in Patent Office: 1,097 Days
• Field of Search: 707/723, 707/766
• US Class Current: 1/1
• CPC Class Codes: G06F 16/5854 using shape and object relationship

Abstract

A server may receive a search query indicative of an object. The server may identify a 3D model from within a database of 3D object data models that corresponds to the object. The 3D model may be associated with information that pertains to the object. The server may determine an updated search query based on the information associated with the 3D model. The updated search query may be indicative of one or more of an environment of the object, an orientation of the object in the environment, a position of the object in the environment, or an identification of one or more other objects in the environment. The server may provide one or more search query results that include data pertaining to the object based on the updated search query.

Claims

1. A method comprising:

receiving, at a server, a search query indicative of an object;

in response to receiving the search query, accessing a database of three-dimensional (3D) models of objects, wherein a respective 3D model in the database includes data characterizing surface geometry and texture of a respective object in 3D space;

identifying, from within the database, a 3D model that corresponds to the object indicated by the search query, wherein the database associates the 3D model with stored information pertaining to the object characterized by the 3D model;

determining, based on the stored information associated with the 3D model, an updated search query indicative of one or more of an environment of the object, an orientation of the object in the environment, a position of the object in the environment, or an identification of one or more other objects in the environment; and

providing, based on the updated search query, one or more search query results pertaining to the object.

2. The method of claim 1, wherein the updated search query comprises text included in the information associated with the 3D model that corresponds to a description of the object.

3. The method of claim 1, further comprising:

determining, by the server, a plurality of images based on the search query; and

identifying, from within the plurality of images, a given image based on the given image being associated with the information from the 3D model, wherein the one or more search query results include the given image.

4. The method of claim 1, further comprising:

arranging, by the server, the one or more search query results in an order based on the information associated with the 3D model.

5. The method of claim 1, further comprising:

determining, by the server, one or more images indicative of views of the 3D model, wherein the updated search query is based on the one or more images.

6. The method of claim 5, wherein the one or more search query results include the one or more images.

7. The method of claim 5, wherein the search query comprises an input image of the object, the method further comprising:

matching, based on a comparison between the one or more images and the input image, a given image of the one or more images to the input image, wherein identifying the 3D model that corresponds to the object is based on the matching.

8. The method of claim 7, further comprising:

determining, based on the information associated with the 3D model and a comparison between the given image and the input image, the orientation of the object, the position of the object, and lighting conditions in the environment of the object, wherein the one or more search query results are configured based on the orientation, the position, and the lighting conditions.

9. The method of claim 8, further comprising:

determining, based on the information associated with the 3D model and a comparison between the given image and the input image, a scale of the input image; and

identifying, by the server, the one or more other objects in the environment based on the orientation of the object, the position of the object, the lighting conditions, and the scale.

10. The method of claim 1, wherein the search query comprises an input image of the object and includes depth information of the input image, the method further comprising:

comparing, by the server, the depth information of the input image with corresponding depth information associated with the 3D model; and

providing the one or more search query results based on the comparison.

11. A device comprising:

one or more processors; and

data storage configured to store instructions executable by the one or more processors to cause the device to;

receive a search query indicative of an object;

in response to receiving the search query, access a database of three-dimensional (3D) models of objects, wherein a respective 3D model in the database includes data characterizing surface geometry and texture of a respective object in 3D space;

identify, from within the database, a 3D model that corresponds to the object indicated by the search query, wherein the database associates the 3D model with stored information pertaining to the object characterized by the 3D model;

determine, based on the stored information associated with the 3D model, an updated search query indicative of one or more of an environment of the object, an orientation of the object in the environment, a position of the object in the environment, or an identification of one or more other objects in the environment; and

provide, based on the updated search query, one or more search query results pertaining to the object.

12. The device of claim 11, wherein the updated search query comprises text included in the information associated with the 3D model that corresponds to a description of the object.

13. The device of claim 11, wherein the instructions executable by the one or more processors further cause the device to:

determine a plurality of images based on the search query; and

identify, from within the plurality of images, a given image based on the given image being associated with the information from the 3D model, wherein the one or more search query results include the given image.

14. The device of claim 11, wherein the instructions executable by the one or more processors further cause the device to:

arrange the one or more search query results in an order based on the information associated with the 3D model.

15. The device of claim 11, wherein the instructions executable by the one or more processors further cause the device to:

determine one or more images indicative of views of the 3D model, wherein the updated search query is based on the one or more images.

16. A non-transitory computer readable medium having stored therein instructions, that when executed by a computing device, cause the computing device to perform functions, the functions comprising:

receiving, at the computing device, a search query indicative of an object;

in response to receiving the search query, accessing a database of three-dimensional (3D) models of objects, wherein a respective 3D model in the database includes data characterizing surface geometry and texture of a respective object in 3D space;

identifying, from within the database, a 3D model that corresponds to the object indicated by the search query, wherein the database associates the 3D model with stored information pertaining to the object characterized by the 3D model;

determining, based on the stored information associated with the 3D model, an updated search query indicative of one or more of an environment of the object, an orientation of the object in the environment, a position of the object in the environment, or an identification of one or more other objects in the environment; and

providing, based on the updated search query, one or more search query results pertaining to the object.

17. The non-transitory computer readable medium of claim 16, wherein the updated search query comprises text included in the information associated with the 3D model that corresponds to a description of the object.

18. The non-transitory computer readable medium of claim 16, the functions further comprising:

determining, by the computing device, a plurality of images based on the search query; and

identifying, from within the plurality of images, a given image based on the given image being associated with the information from the 3D model, wherein the one or more search query results include the given image.

19. The non-transitory computer readable medium of claim 16, the functions further comprising:

arranging, by the computing device, the one or more search query results in an order based on the information associated with the 3D model.

20. The non-transitory computer readable medium of claim 16, the functions further comprising:

determining, by the computing device, one or more images indicative of views of the 3D model, wherein the updated search query is based on the one or more images.