System and method for determining position of a device

• US 10,592,744 B1
• Filed: 02/02/2018
• Issued: 03/17/2020
• Est. Priority Date: 03/12/2010
• Status: Active Grant

• Current Assignee: Google LLC (Alphabet Inc.)
• Original Assignee: Google LLC (Alphabet Inc.)
• Inventors: Rivlin, Ehud, McClendon, Brian, Bouguet, Jean-Yves
• Primary Examiner(s): Vazquez Colon, Maria E
• Application Number:
  US15/887,014
• Time in Patent Office: 774 Days
• CPC Class Codes: G06K 2019/06243 concentric-code
• CPC Class Codes: G06T 2207/10004 Still image; Photographic image
• CPC Class Codes: G06T 2207/30204 Marker
• CPC Class Codes: G06T 2207/30244 Camera pose
• CPC Class Codes: G06T 7/246 using feature-based methods, e.g. the tracking of corners or segments
• CPC Class Codes: G06T 7/285 using a sequence of stereo image pairs
• CPC Class Codes: G06T 7/73 using feature-based methods
• CPC Class Codes: G06V 20/10 Terrestrial scenes scenes under surveillance with static cameras G06V20/52; scenes perceived from the exterior of a vehicle G06V20/56; scenes perceived from the interior of a vehicle G06V20/59

Abstract

A system and method is provided for determining the location of a device based on image of objects captured by the device. In one aspect, an interior space includes a plurality of objects having discernable visual characteristics disposed throughout the space. The device captures an image containing one or more of the objects and identifies the portions of the image associated with the objects based on the visual characteristics. The visual appearance of the objects may also be used to determine the distance of the object to other objects or relative to a reference point. Based on the foregoing and the size and shape of the image portion occupied by the object, such as the height of an edge or its surface area, relative to another object or a reference, the device may calculate its location.

Claims

1. A system comprising:

one or more processors, the one or more processors configured to;

determine whether an image captured by a camera includes an image of a visual beacon;

determine an identifier of a visual beacon captured within the image, wherein the identifier of the visual beacon is visually detectable from the surface of the visual beacon, wherein the identifier includes a location of the visual beacon relative to a reference location, and wherein the identifier identifies the physical size or shape of the surface area of the visual beacon;

determine the location of the visual beacon relative to the reference location based on the identifier;

determine the physical size or shape of the surface area of the beacon based on the identifier; and

determine a location of the camera based on the reference location, and based on a comparison of the determined physical size or shape with the size or shape of the image of the visual beacon within the captured image.

2. The system of claim 1, wherein the comparison of the determined physical size or shape of the surface area of the beacon with the size or shape of the image of the beacon within the captured image further comprises computer code that determines the size or shape of a first portion of the image of the beacon within the captured image relative to the size or shape of another portion of the image of the beacon within the captured image.

3. The system of claim 1, wherein the comparison of the determined physical size or shape of the surface area of the beacon with the size or shape of the image of the beacon within the captured image further comprises computer code that determines a length of a first edge of the image of the beacon within the captured image relative to a length of a corresponding edge of determined physical size or shape of the surface area of the beacon.

4. The system of claim 1, further comprising computer code that determines an orientation of the camera relative to the location of the beacon based on a size or shape associated with the image of the beacon within the captured image.

5. The system of claim 1, wherein determining the location of the camera based on the comparison of the determined physical size or shape with the size or shape of the image of the beacon within the captured image further comprises refining the determined location of the camera to be a more precise location based on the second set of smaller visual beacons.

6. The system of claim 1, wherein the first set of visual beacons are separated from each other at a greater distance than the second set of smaller visual beacons being separated from each other.

7. The system of claim 1, wherein each beacon of the first and second sets of beacons is associated with a visual characteristic.

8. The system of claim 7, wherein the visual characteristic is a checkerboard pattern.

9. The system of claim 7, wherein the visual characteristic is a barcode.

10. The system of claim 7, wherein the visual characteristic is a 2-D shape.

11. The system of claim 7, wherein the visual characteristic is a 3-D shape.

12. A method comprising:

determining, by one or more processors, whether an image captured by a camera includes an image of a visual beacon;

determining, by the one or more processors, an identifier of a visual beacon captured within the image, wherein the identifier of the visual beacon is visually detectable from the surface of the visual beacon, wherein the identifier includes a location of the visual beacon relative to a reference location, and wherein the identifier identifies the physical size or shape of the surface area of the visual beacon;

determining the location of the visual beacon relative to the reference location based on the identifier;

determining the physical size or shape of the surface area of the visual beacon based on the identifier; and

determining a location of the camera based on the reference location, and based on a comparison of the determined physical size or shape with the size or shape of the image of the visual beacon within the captured image.

13. The method of claim 12, wherein the comparison of the determined physical size or shape of the surface area of the beacon with the size or shape of the image of the beacon within the captured image further comprises the step of determining the size or shape of a first portion of the image of the beacon within the captured image relative to the size or shape of another portion of the image of the beacon within the captured image.

14. The method of claim 12, wherein the comparison of the determined physical size or shape of the surface area of the beacon with the size or shape of the image of the beacon within the captured image further comprises the step of determining a length of a first edge of the image of the beacon within the captured image relative to a length of a corresponding edge of determined physical size or shape of the surface area of the beacon.

15. The method of claim 12, further comprising the step of determining an orientation of the camera relative to the location of the beacon based on a size or shape associated with the image of the beacon within the captured image.

16. The method of claim 12, wherein determining the location of the camera based on the comparison of the determined physical size or shape with the size or shape of the image of the beacon within the captured image further comprises refining the determined location of the camera to be a more precise location based on the second set of smaller visual beacons.

17. The method of claim 12, wherein the first set of visual beacons are separated from each other at a greater distance than the second set of smaller visual beacons being separated from each other.

18. The method of claim 12, wherein each beacon of the first and second sets of beacons is associated with a visual characteristic.