ELECTRONIC DEVICE LOCALIZATION BASED ON IMAGERY

• US 20150310310A1
• Filed: 04/24/2015
• Published: 10/29/2015
• Est. Priority Date: 04/25/2014
• Status: Active Grant

• Current Assignee: Google LLC (Alphabet Inc.)
• Original Assignee: Google Technology Holdings LLC (Alphabet Inc.)
• Inventors: Hesch, Joel, Mihelich, Patrick, Nerurkar, Esha
• Granted Patent: US 9,953,243 B2
• Time in Patent Office: Days
• US Class Current: 1/1
• CPC Class Codes: G01S 19/485 whereby the further system is an optical system or imaging system
• CPC Class Codes: G06F 18/213 Feature extraction, e.g. by transforming the feature space; Summarisation; Mappings, e.g. subspace methods
• CPC Class Codes: G06F 18/253 of extracted features
• CPC Class Codes: G06T 2207/30244 Camera pose
• CPC Class Codes: G06T 7/246 using feature-based methods, e.g. the tracking of corners or segments
• CPC Class Codes: G06T 7/73 using feature-based methods
• CPC Class Codes: G06T 7/74 involving reference images or patches
• CPC Class Codes: G06V 2201/07 Target detection

Abstract

An electronic device includes one or more imaging cameras. After a reset of the device or other specified event, the electronic device identifies an estimate of the device'"'"'s pose based on location data such as Global Positioning System (GPS) data, cellular tower triangulation data, wireless network address location data, and the like. The one or more imaging cameras may be used to capture imagery of the local environment of the electronic device, and this imagery is used to refine the estimated pose to identify a refined pose of the electronic device. The refined pose may be used to identify additional imagery information, such as environmental features, that can be used to enhance the location based functionality of the electronic device.

Claims

1. A method comprising:

in response to a reset event at an electronic device determining, at the electronic device, an estimated pose of the electronic device based on received non-visual location data relative to a first frame of reference; and

refining the estimated pose of the electronic device based on an image captured at an imaging camera of the electronic device to identify a refined pose of the electronic device relative to a second frame of reference, wherein refining the estimated pose comprises;

identifying a feature of the image;

matching the feature to a first feature descriptor of a data store of feature descriptors; and

identifying the refined pose based on a pose associated with the first feature descriptor.

2. The method of claim 1, wherein the non-visual location data comprises Global Positioning System data captured at the electronic device.

3. The method of claim 1, wherein the non-visual location data comprises wireless tower triangulation information for the electronic device.

4. The method of claim 1, wherein refining the estimated pose comprises:

identifying a plurality of feature descriptors of the data store of feature descriptors based on the estimated pose; and

searching the identified plurality of feature descriptors for the first feature descriptor.

5. The method of claim 1, further comprising:

identifying the first feature descriptor based on a gravity direction of the first image identified at the electronic device.

6. The method of claim 1, further comprising:

concurrent with refining the estimated pose of the electronic device, tracking a change in position of the electronic device; and

translating the refined pose of the electronic device based on the change in positron.

7. The method of claim 6, further comprising:

translating a plurality of intermediate poses of the electronic device between an initial pose and the refined pose of the electronic device.

8. A method, comprising,

in response to a reset event at an electronic device, identifying at the electronic, device an origin in a first frame of reference;

identifying a first pose of the electronic device in a second frame of reference corresponding to a geographic frame of reference;

concurrent with identifying the first pose, identifying a change in position of the electronic device in the first frame of reference relative to the origin; and

translating the first pose of the electronic device to a second pose in the second frame of reference based on the change in position in the first frame of reference.

9. The method of claim 8, wherein identifying the first pose comprises identifying the first pose based on a first image captured by an imaging camera at the electronic device.

10. The method of claim 9, wherein identifying the first pose comprises:

identifying an estimated pose of the electronic device based on non-visual location data; and

refining the estimated pose of the electronic device based on the image captured by the imaging camera to identify the first pose.

11. The method of claim 10, wherein identifying the first pose comprises:

identifying a feature of the first image;

matching the feature to a first feature descriptor of a database of feature descriptors; and

identifying the first pose based on a pose associated with the first feature descriptor.

12. The method of claim 11, wherein refining the estimated pose comprises:

identifying a plurality of feature descriptors based on the estimated pose; and

searching the identified plurality of feature descriptors for the first feature descriptor.

13. The method of claim 8, further comprising:

identifying a second feature descriptor based on the second pose; and

providing information based on the second feature descriptor to an application executing at the electronic device.

14. An electronic device comprising:

an imaging camera to capture first imagery;

a processor to;

in response to a reset event, determine an estimated pose of the electronic device relative to a first frame of reference based on received non-visual location data; and

refine the estimated pose of the electronic device based on a first image captured at an imaging camera of the electronic device to identify a refined pose of the electronic device relative to a second frame of reference.

15. The electronic device of claim 14, wherein the non-visual location data comprises Global Positioning System data captured at the electronic device.

16. The electronic device of claim 14, wherein the non-visual location data comprises wireless tower triangulation information for the electronic device.

17. The electronic device of claim 14, wherein the processor is to refine the estimated pose by:

generating a first feature descriptor based on the first imagery;

matching the first feature descriptor to a second feature descriptor; and

identifying the refined pose based on a location associated with the second feature descriptor.

18. The electronic device of claim 17, wherein the processor is to refine the estimated pose by:

identifying a plurality of feature descriptors based on the estimated pose; and

searching the identified plurality of feature descriptors for the second feature descriptor.

19. The electronic device of claim 17. Wherein the processor is to:

identify the second feature descriptor based on a gravity direction of the first image identified at the electronic device.

20. The electronic device of claim 14, wherein the processor is to:

concurrent with refining the estimated pose of the electronic device, track a change in position of the electronic device; and

translate the refined pose of the electronic device based on the change in position.