Performing a visual search using a rectified image

• US 9,547,669 B2
• Filed: 05/13/2015
• Issued: 01/17/2017
• Est. Priority Date: 02/21/2013
• Status: Active Grant

• Current Assignee: Qualcomm, Inc.
• Original Assignee: Qualcomm, Inc.
• Inventors: Wagner, Daniel, Pan, Qi
• Primary Examiner(s): BEKELE, MEKONEN T
• Application Number:
  US14/711,406
• Publication Number:
  US 20150254284A1
• Time in Patent Office: 615 Days
• Field of Search: 382/103, 382/154, 382/190, 382/275, 382/300, 382/260
• US Class Current: 1/1
• CPC Class Codes: G06F 16/20 of structured data, e.g. relational data
• CPC Class Codes: G06F 16/54 Browsing; Visualisation therefor
• CPC Class Codes: G06T 17/00 Three dimensional [3D] modelling, e.g. data description of 3D objects

Abstract

Disclosed is a server that can perform a visual search using at least one rectified image. A method implemented at a server includes storing a plurality of images with the server, receiving at least one rectified image having at least one potential object of interest from a computing device for a visual search, and extracting descriptors representing features of the at least one rectified image. The extracted descriptors of the at least one rectified image are designed to be invariant to rotation, scale, and lighting without needing to be invariant to perspective or affine distortion.

Claims

1. A server comprising:

a storage medium configured to store a plurality of images; and

a processing circuit configured to execute instructions to;

receive at least one rectified image having at least one potential object of interest from a computing device for a visual search and including a modified perspective view of the at least one potential object of interest relative to a view of the at least one potential object of interest in one or more unrectified images, the at least one rectified image created based on at least a partial 3D geometric model of an environment included in the one or more unrectified images, the at least partial 3D geometric model including at least the portion of the one or more potential objects of interest within the environment, wherein the at least partial 3D geometric model is built using at least one unrectified image of the one or more unrectified images including the environment and at least a portion of one or more potential objects of interest within the environment; and

extract descriptors representing features of the at least one rectified image, wherein the extracted descriptors of the at least one rectified image are designed to be invariant to rotation, scale, and lighting without needing to be invariant to perspective or affine distortion.

2. The server of claim 1, wherein the processing circuit is further configured to execute instructions to match the extracted descriptors of the at least one rectified image with descriptors of the plurality of images stored in the database.

3. The server of claim 2, wherein the processing circuit is further configured to execute instructions to transmit information associated with at least one image stored in the storage medium that has matching descriptors.

4. The server of claim 1, wherein the extracted descriptors of the at least one rectified image are not invariant to perspective or affine distortion.

17. The server of claim 1, wherein the rectified image and the unrectified images are stored on a remote device.

5. A method implemented at a server comprising:

storing a plurality of images with the server;

receiving at least one rectified image having at least one potential object of interest from a computing device for a visual search and including a modified perspective view of the at least one potential object of interest relative to a view of the at least one potential object of interest in one or more unrectified images, the at least one rectified image created based on at least a partial 3D geometric model of an environment included in the one or more unrectified images, the at least partial 3D geometric model including at least the portion of the one or more potential objects of interest within the environment, wherein the at least partial 3D geometric model is built using at least one unrectified image a the one or more unrectified images including the environment and at least a portion of one or more potential objects of interest within the environment; and

extracting descriptors representing features of the at least one rectified image, wherein the extracted descriptors of the at least one rectified image are designed to be invariant to rotation, scale, and lighting without needing to be invariant to perspective or affine distortion.

6. The method of claim 5, further comprising:

matching the extracted descriptors of the at least one rectified image with descriptors of the plurality of images.

7. The method of claim 6, further comprising:

transmitting information associated with at least one image stored with the server that has matching descriptors.

8. The method of claim 5, wherein the extracted descriptors of the at least one rectified image are not invariant to perspective or affine distortion.

18. The server of claim 5, wherein the rectified image and the unrectified images are stored on a remote device.

9. A computer program product executed at a server comprising:

a non-transitory computer-readable storage medium comprising code for;

storing a plurality of images;

receiving at least one rectified image having at least one potential object of interest from a computing device for a visual search and including a modified perspective view of the at least one potential object of interest relative to a view of the at least one potential object of interest in one or more unrectified images, the at least one rectified image created based on at least a partial 3D geometric model of an environment included in the one or more unrectified images, the at least partial 3D geometric model including at least the portion of the one or more potential objects of interest within the environment, wherein the at least partial 3D geometric model is built using at least one unrectified image of the one or more unrectified images including the environment and at least a portion of one or more potential objects of interest within the environment; and

extracting descriptors representing features of the at least one rectified image, wherein the extracted descriptors of the at least one rectified image are designed to be invariant to rotation, scale, and lighting without needing to be invariant to perspective or affine distortion.

10. The computer program product of claim 9, further comprising code for matching the extracted descriptors of the at least one rectified image with descriptors of the plurality of images.

11. The computer program product of claim 10, further comprising code for transmitting information associated with at least one image stored in a database that has matching descriptors.

12. The computer program product of claim 9, wherein the extracted descriptors of the at least one rectified image are not invariant to perspective or affine distortion.

19. The server of claim 9, wherein the rectified image and the unrectified images are stored on a remote device.

13. A server comprising:

means for storing a plurality of images;

means for receiving at least one rectified image having at least one potential object of interest from a computing device for a visual search and including a modified perspective view of the at least one potential object of interest relative to a view of the at least one potential object of interest in one or more unrectified images, the at least one rectified image created based on at least a partial 3D geometric model of an environment included in the one or more unrectified images, the at least partial 3D geometric model including at least the portion of the one or more potential objects of interest within the environment, wherein the at least partial 3D geometric model is built using at least one unrectified image of the one or more unrectified images including the environment and at least a portion of one or more potential objects of interest within the environment; and

means for extracting descriptors representing features of the at least one rectified image, wherein the extracted descriptors of the at least one rectified image are designed to be invariant to rotation, scale, and lighting without needing to be invariant to perspective or affine distortion.

14. The server of claim 13, further comprising means for matching the extracted descriptors of the at least one rectified image with descriptors of the plurality of images.

15. The server of claim 14, further comprising means for transmitting information associated with at least one image stored in a database that has matching descriptors.

16. The server of claim 13, wherein the extracted descriptors of the at least one rectified image are not invariant to perspective or affine distortion.

20. The server of claim 13, wherein the rectified image and the unrectified images are stored on a remote device.