System and method for determining location of a device using opposing cameras

• US 9,986,208 B2
• Filed: 01/27/2012
• Issued: 05/29/2018
• Est. Priority Date: 01/27/2012
• Status: Active Grant

• Current Assignee: Qualcomm, Inc.
• Original Assignee: Qualcomm, Inc.
• Inventors: Chao, Hui, Das, Saumitra M., Poduri, Sameera
• Primary Examiner(s): Zhou, Zhihan
• Application Number:
  US13/360,372
• Publication Number:
  US 20130194428A1
• Time in Patent Office: 2,314 Days
• Field of Search: 348159
• CPC Class Codes: G01C 21/20 Instruments for performing navigational calculations G01C21/24, G01C21/26 take precedence
• CPC Class Codes: G01S 5/16 using electromagnetic waves other than radio waves
• CPC Class Codes: G06T 2207/10016 Video; Image sequence
• CPC Class Codes: G06T 7/74 involving reference images or patches
• CPC Class Codes: H04N 7/181 for receiving images from a plurality of remote sources
• CPC Class Codes: H04W 4/02 Services making use of location information
• CPC Class Codes: H04W 4/40 for vehicles, e.g. vehicle-to-pedestrians [V2P]

Abstract

System and method for determining location of a device using opposing cameras are disclosed. In one embodiment, the method includes activating a front-facing camera and a back-facing camera of the device, capturing a first pair of images from a first orientation using the front-facing camera and the back-facing camera, where the first pair of images includes a first front image and a first back image, monitoring the device being rotated to a second orientation, capturing a second pair of images from the second orientation using the front-facing camera and the back-facing camera, where the second pair of images includes a second front image and a second back image, and determining a location of the device using the first pair of images and the second pair of images.

Claims

1. A method for determining location of a device, comprising:

activating a front-facing camera and a back-facing camera of the device;

capturing a first pair of images at a location of the device from a first orientation using the front-facing camera and the back-facing camera, the first pair of images comprising a first front image and a first back image;

determining a second orientation;

capturing a second pair of images at the location of the device from the second orientation using the front-facing camera and the back-facing camera, the second pair of images comprising a second front image and a second back image;

identifying the first pair of images or the second pair of images, or both, the identifying to be initiated by searching for images in a locally cached database comprising spatially organized images of a venue, the identifying including matching the first image from the first pair of images with an image from the locally cached database followed by examination of one or more images from the locally cached database corresponding to locations opposite the matched first image to detect an image matching the second image from the first pair of images;

determining a first line connecting locations associated with the first front image and the first back image, wherein determining the first line comprises;

comparing the first front image with images in a database, wherein the database stores image features of multiple locations of the venue;

identifying an approximate location of the first front image;

comparing the first back image with the images in the database;

identifying an approximate location of the first back image; and

determining of the first line using the approximate location of the first front image and the approximate location of the first back image;

determining a second line connecting locations associated with the second front image and the second back image;

determining an approximate cross point of the first line and the second line; and

identifying the approximate cross point as an estimated location of the device.

2. The method of claim 1, wherein activating a front-facing camera and a back-facing camera of the device comprises:

activating the front-facing camera and the back-facing camera of the device with a gesture, wherein the gesture is to include applying one or more user-defined movements of the device.

3. The method of claim 1, wherein capturing the first pair of images comprises:

capturing the first pair of images simultaneously.

4. The method of claim 1, wherein capturing the first pair of images further comprises:

capturing the first pair of images within a predetermined period of time.

5. The method of claim 1, further comprising:

determining the first orientation and the second orientation using an accelerometer, a gyroscope or a magnetometer, or a combination thereof.

6. The method of claim 1, wherein comparing the first front image with the images in the database comprises:

selecting a first set of images from the database based on the first orientation of the device, wherein the first set of images covers a region in view of the front-facing camera; and

comparing the first front image to the first set of images.

7. The method of claim 1, wherein comparing the first back image with the images in the database comprises:

selecting a second set of images from the database based on the approximate location of the first front image, wherein the second set of images covers a region opposite to the approximate location of the first front image; and

comparing the first back image to the second set of images.

8. The method of claim 1, wherein determining the second line connecting locations associated with the second front image and the second back image comprises:

comparing the second front image with images in the database wherein the database stores image features of multiple locations of the venue;

identifying an approximate location of the second front image;

comparing the second back image with the images in the database;

identifying an approximate location of the second back image; and

determining the second line using the approximate location of the second front image and the approximate location of the second back image.

9. The method of claim 1, further comprising:

determining, via a positioning engine at the mobile device, a final location of the device using the estimated location.

10. A computer program product to determine a location of a device, comprising a non-transitory medium storing computer programs for execution by one or more computer systems, the computer program product comprising:

code to activate a front-facing camera and a back-facing camera of the device;

code to capture a first pair of images at a location of the device from a first orientation using the front-facing camera and the back-facing camera, the first pair of images to comprise a first front image and a first back image;

code to determine a second orientation of the device;

code to capture a second pair of images at the location of the device from the second orientation using the front-facing camera and the back-facing camera, the second pair of images to comprise a second front image and a second back image;

code to identify the first pair of images or the second pair of images, or both, the code to identify to be initiated by searching for images in a locally cached database comprising spatially organized images of a venue, the code to identify to include code to detect a match between the first image from the first pair of images and an image from the locally cached database followed by examination of one or more images from the locally cached database corresponding to locations opposite the matched first image to detect an image that matches the second image from the first pair of images;

code to determine a first line connecting locations associated with the first front image and the first back image, wherein determining the first line comprises;

code to compare the first front image with images in a database, wherein the database stores image features of multiple locations of the venue;

code to identify an approximate location of the first front image;

code to compare the first back image with the images in the database;

code to identify an approximate location of the first back image; and

code to determine of the first line using the approximate location of the first front image and the approximate location of the first back image;

code to determine a second line connecting locations associated with the second front image and the second back image;

code to determine an approximate cross point of the first line and the second line; and

code to identify the approximate cross point as an estimated location of the device.

11. The computer program product of claim 10, wherein the code to activate a front-facing camera and a back-facing camera of the device comprises:

code to activate the front-facing camera and the back-facing camera of the device with a gesture, wherein the gesture includes applying one or more user-defined movements of the device.

12. The computer program product of claim 10, wherein the code to capture a first pair of images comprises:

code to capture the first pair of images simultaneously.

13. The computer program product of claim 10, wherein the code to capture a first pair of images further comprises:

code to capture the first pair of images within a predetermined period of time.

14. The computer program product of claim 10, further comprising:

code to determine the first orientation and the second orientation using an accelerometer, a gyroscope or a magnetometer, or a combination thereof.

15. The computer program product of claim 10, wherein the code to compare the first front image comprises:

code to select a first set of images from the database based on the first orientation of the device, wherein the first set of images is to cover a region in view of the front-facing camera; and

code to compare the first front image to the first set of images.

16. The computer program product of claim 10, wherein the code to compare the first back image comprises:

code to select a second set of images from the database based on the approximate location of the first front image, wherein the second set of images covers a region opposite to the approximate location of the first front image; and

code to compare the first back image to the second set of images.

17. The computer program product of claim 10, wherein the code to determine the second line to connect locations associated with the second front image and the second back image comprises:

code to compare the second front image with images in the database wherein the database stores image features of multiple locations of the venue;

code to identify an approximate location of the second front image;

code to compare the second back image with the images in the database;

code to identify an approximate location of the second back image; and

code to determine the second line using the approximate location of the second front image and the approximate location of the second back image.

18. The computer program product of claim 10 further comprising:

code to determine, at a positioning engine of the device, a final location of the device using the estimated location.

19. A device, comprising:

one or more processors;

a dual-camera assist module, coupled to the one or more processors, the dual-camera assist module to comprise;

logic to activate a front-facing camera and a back-facing camera of the device;

logic to capture a first pair of images at a location of the device from a first orientation of the device using a front-facing camera and a back-facing camera, the first pair of images to comprise a first front image and a first back image;

logic to determine a second orientation;

logic to capture a second pair of images at the location of the device from the second orientation using the front-facing camera and the back-facing camera, the second pair of images to comprise a second front image and a second back image;

logic to identify the first pair of images or the second pair of images, or both, the logic to identify to initiate by searching for images in a locally cached database spatially organized images of a venue, the logic to identify to include logic to match a first image from the first pair of images with an image from the locally cached database followed by examination of one or more images from the locally cached database corresponding to locations opposite the matched first image to detect a match between a locally cached image and the second image from the first pair of images;

logic to determine a first line connecting locations associated with the first front image and the first back image, wherein determining the first line comprises;

logic to compare the first front image with images in a database, wherein the database stores image features of multiple locations of the venue;

logic to identify an approximate location of the first front image;

logic to compare the first back image with the images in the database;

logic to identify an approximate location of the first back image; and

logic to determine of the first line using the approximate location of the first front image and the approximate location of the first back image;

logic to determine a second line connecting locations associated with the second front image and the second back image;

logic to determine an approximate cross point of the first line and the second line; and

logic to identify the approximate cross point as an estimated location of the device.

20. The device of claim 19, wherein the logic to activate a front-facing camera and a back-facing camera of the device comprises:

logic to activate the front-facing camera and the back-facing camera of the device with a gesture, wherein the gesture includes applying one or more user-defined movements of the device.

21. The device of claim 19, wherein the logic to capture a first pair of images comprises:

logic to capture the first pair of images simultaneously.

22. The device of claim 19, wherein the logic to capture a first pair of images further comprises:

logic to capture the first pair of images within a predetermined period of time.

23. The device of claim 19, further comprising:

logic to determine the first orientation and the second orientation using an accelerometer, a gyroscope or a magnetometer, or a combination thereof.

24. The device of claim 19, wherein the logic to compare the first front image comprises:

logic to select a first set of images from the database based on the first orientation of the device, wherein the first set of images covers a region in view of the front-facing camera; and

logic to compare the first front image to the first set of images.

25. The device of claim 19, wherein the logic to compare the first back image comprises:

logic to select a second set of images from the database based on the approximate location of the first front image, wherein the second set of images is to cover a region opposite to the approximate location of the first front image; and

logic to compare the first back image to the second set of images.

26. The device of claim 19, wherein the logic to determine the second line to connect locations associated with the second front image and the second back image comprises:

logic to compare the second front image with images in the database wherein the database stores image features of multiple locations of the venue;

logic to identify an approximate location of the second front image;

logic to compare the second back image with the images in the database;

logic to identify an approximate location of the second back image; and

logic to determine the second line using the approximate location of the second front image and the approximate location of the second back image.

27. The device of claim 19, further comprising:

logic to determine, via a positioning engine at the device, a final location of the device using the estimated location.

28. The device of claim 19, further comprising:

logic to obtain a signal indicating verification of the first pair of images prior to capturing the second pair of images.

29. The device of claim 19, further comprising:

logic to indicate a request to repeat capture of the first pair of images based, at least in part, on a quality of the first pair of images or and absence of a satisfactory match between the captured first pair of images and one or more images in the database, or a combination thereof.

30. A system for determining location of a device, comprising:

one or more processors;

a dual-camera assist module, to operate with the one or more processors, the dual-camera assist module comprising;

means for activating a front-facing camera and a back-facing camera of the device;

means for capturing at a location of the device a first pair of images from a first orientation using the front-facing camera and the a back-facing camera, the first pair of images to comprise a first front image and a first back image;

means for determining a second orientation of the device;

means for capturing at the location of the device a second pair of images from the second orientation using the front-facing camera and the back-facing camera, the second pair of images to comprise a second front image and a second back image;

means for identifying the first pair of images or the second pair of images, or both, the means for identifying to be initiated by searching for images in a locally cached database comprising spatially organized images of a venue, the means for identifying including means for matching a first image from the first pair of images with an image from the locally cached database followed by examination of one or more images from the locally cached database corresponding to locations opposite the matched first image to detect an image matching the second image from the first pair of images; and

means for determining the location of the device based, at least in part, on an intersection of first and second planes;

means for to determining a first line connecting locations associated with the first front image and the first back image, wherein means for determining the first line comprises;

means for comparing the first front image with images in a database, wherein the database stores image features of multiple locations of the venue;

means for identifying an approximate location of the first front image;

means for comparing the first back image with the images in the database;

means for identifying an approximate location of the first back image; and

means for determining of the first line using the approximate location of the first front image and the approximate location of the first back image;

means for determining a second line connecting locations associated with the second front image and the second back image;

means for determining an approximate cross point of the first line and the second line; and

means for identifying the approximate cross point as an estimated location of the device.

31. The system of claim 30, wherein means for comparing the first front image comprises:

means for selecting a first set of images from the database based on the first orientation of the device, wherein the first set of images covers a region in view of the front-facing camera; and

means for comparing the first front image to the first set of images.

32. The system of claim 30, wherein means for comparing the first back image comprises:

means for selecting a second set of images from the database based on the approximate location of the first front image, wherein the second set of images covers a region opposite to the approximate location of the first front image; and

means for comparing the first back image to the second set of images.

33. The system of claim 30, wherein means for determining a second line comprises:

means for comparing the second front image with images in the database wherein the database stores image features of multiple locations of the venue;

means for identifying an approximate location of the second front image;

means for comparing the second back image with the images in the database;

means for identifying an approximate location of the second back image; and

means for determining the second line using the approximate location of the second front image and the approximate location of the second back image.

34. A method for determining location of a device, comprising:

activating a front-facing camera and a back-facing camera of the device;

obtaining a first pair of images captured at a location of the device from a first orientation of the device using a front-facing camera and a back-facing camera, the first pair of images comprises a first front image and a first back image;

obtaining a second pair of images captured at the location of the device from a second orientation of the device using the front-facing camera and the back-facing camera, the second pair of images comprises a second front image and a second back image;

identifying the first pair of images or the second pair of images, or both, wherein the identifying is to be initiated by searching for images in a locally cached database comprising spatially organized images of a venue, the identifying including matching a first image from the first pair of images with an image from the locally cached database followed by examination of one or more images from the locally cached database corresponding to locations opposite the matched first image to detect an image matching the second image from the first pair of images;

determining a first line connecting locations associated with the first front image and the first back image, wherein determining the first line comprises;

comparing the first front image with images in a database, wherein the database stores image features of multiple locations of the venue;

identifying an approximate location of the first front image;

comparing the first back image with the images in the database;

identifying an approximate location of the first back image; and

determining of the first line using the approximate location of the first front image and the approximate location of the first back image;

determining a second line connecting locations associated with the second front image and the second back image;

determining an approximate cross point of the first line and the second line; and

identifying the approximate cross point as an estimated location of the device.

35. A computer program product for determining location of a device, comprising a non-transitory medium storing computer programs for execution by one or more computer systems, the computer program product comprising:

code to activate a front-facing camera and a back-facing camera of the device;

code to obtain a first pair of images captured at a location of the device from a first orientation of the device using a front-facing camera and a back-facing camera, the first pair of images to comprise a first front image and a first back image;

code to obtain a second pair of images captured at the location of the device from a second orientation of the device using the front-facing camera and the back-facing camera, the second pair of images to comprise a second front image and a second back image;

code to identify the first pair of images or the second pair of images, or both, the code to identify to be initiated by searching for images in a locally cached database comprising spatially organized images of a venue, the code to identify to include code to detect a match between the first image from the first pair of images and an image from the locally cached database followed by examination of one or more images from the locally cached database corresponding to locations opposite the matched first image to detect an image that matches the second image from the first pair of images;

code to determine a first line connecting locations associated with the first front image and the first back image, wherein determining the first line comprises;

code to compare the first front image with images in a database, wherein the database stores image features of multiple locations of the venue;

code to identify an approximate location of the first front image;

code to compare the first back image with the images in the database;

code to identify an approximate location of the first back image; and

code to determine of the first line using the approximate location of the first front image and the approximate location of the first back image;

code to determine a second line connecting locations associated with the second front image and the second back image;

code to determine an approximate cross point of the first line and the second line; and

code to identify the approximate cross point as an estimated location of the device.

36. A server, comprising:

one or more processors;

a dual-camera positioning module configured to operate with the one or more processors, the dual-camera positioning module comprising;

logic to obtain a first pair of images captured at a location of a device from a first orientation of the device using a front-facing camera and a back-facing camera, the first pair of images to comprise a first front image and a first back image;

logic to obtain a second pair of images captured at the location of the device from a second orientation of the device using the front-facing camera and the back-facing camera, the second pair of images to comprise a second front image and a second back image;

logic to identify the first pair of images or the second pair of images, or both, the logic to identify to be initiated by searching for images in a locally cached database comprising spatially organized images of a venue, the logic to identify to include logic to match a first image from the first pair of images with an image from the locally cached database followed by examination of one or more images from the locally cached database corresponding to locations opposite the matched first image to detect a match between a locally cached image and the second image from the first pair of images;

logic to determine a first line connecting locations associated with the first front image and the first back image, wherein determining the first line comprises;

logic to compare the first front image with images in a database, wherein the database stores image features of multiple locations of the venue;

logic to identify an approximate location of the first front image;

logic to compare the first back image with the images in the database;

logic to identify an approximate location of the first back image; and

logic to determine of the first line using the approximate location of the first front image and the approximate location of the first back image;

logic to determine a second line connecting locations associated with the second front image and the second back image;

logic to determine an approximate cross point of the first line and the second line; and

logic to identify the approximate cross point as an estimated location of the device.

37. A system for determining location of a device, comprising:

one or more processors;

a dual-camera positioning module configured to operate with the one or more processors, the dual-camera positioning module to comprise;

means for obtaining a first pair of images captured at a location of the device from a first orientation of the device using a front-facing camera and a back-facing camera, the first pair of images to comprise a first front image and a first back image;

means for obtaining a second pair of images captured at the location of the device from a second orientation of the device using the front-facing camera and the back-facing camera, the second pair of images to comprise a second front image and a second back image;

means for identifying the first pair of images or the second pair of images, or both, the means for identifying to initiate by searching for images in a locally cached database comprising spatially organized images of a venue, the means for identifying to include logic to match a first image from the first pair of images with an image from the locally cached database followed by examination of one or more images from the locally cached database corresponding to locations opposite the matched first image to detect a match between a locally cached image and the second image from the first pair of images;

means for determining a first line connecting locations associated with the first front image and the first back image, wherein determining the first line comprises;

means for comparing the first front image with images in a database, wherein the database stores image features of multiple locations of the venue;

means for identifying an approximate location of the first front image;

means for comparing the first back image with the images in the database;

means for identifying an approximate location of the first back image; and

means for determining of the first line using the approximate location of the first front image and the approximate location of the first back image;

means for determining a second line connecting locations associated with the second front image and the second back image;

means for determining an approximate cross point of the first line and the second line; and

means for identifying the approximate cross point as an estimated location of the device.