Visual localization of unmanned aerial vehicles based on marker detection and processing

• US 9,067,671 B2
• Filed: 07/25/2013
• Issued: 06/30/2015
• Est. Priority Date: 07/25/2013
• Status: Active Grant

• Current Assignee: Disney Enterprises Incorporated (The Walt Disney Company)
• Original Assignee: Disney Enterprises Incorporated (The Walt Disney Company)
• Inventors: Stark, James Alexander, Wong, Clifford
• Primary Examiner(s): Jabr, Fadey
• Assistant Examiner(s): Soofi, Yazan A
• Application Number:
  US13/951,200
• Publication Number:
  US 20150032295A1
• Time in Patent Office: 705 Days
• Field of Search: 701/3
• US Class Current: 1/1
• CPC Class Codes: B64C 19/00 Aircraft control not otherwise provided for
• CPC Class Codes: B64D 47/06 for indicating aircraft presence
• CPC Class Codes: B64U 2101/30 for imaging, photography or videography
• CPC Class Codes: B64U 70/50 Launching from storage containers, e.g. from submarine missile tubes
• CPC Class Codes: B64U 70/95 Means for guiding the landing UAV towards the platform, e.g. lighting means
• CPC Class Codes: G05D 1/104 involving a plurality of aircrafts, e.g. formation flying traffic control systems for aircraft G08G5/00
• CPC Class Codes: G06K 19/06131 the marking comprising a target pattern, e.g. for indicating the center of the bar code or for helping a bar code reader to properly orient the scanner or to retrieve the bar code inside of an image
• CPC Class Codes: G06T 2207/30204 Marker
• CPC Class Codes: G06T 7/73 using feature-based methods
• CPC Class Codes: G06V 10/245 by locating a pattern; Special marks for positioning
• CPC Class Codes: G06V 20/13 Satellite images

Abstract

A method for locating markers in an image captured by a mobile device moving about an operating space. The method includes preprocessing an image to generate a set of image data, locating fixed features of markers by tracing edges of the fixed features, and extracting variable data payloads of each of the markers associated with the located fixed features. The fixed features of each of the markers may include a pair of parallel lines extending along opposite sides of a data area containing the variable data payload, and each of the lines extends a distance beyond each exposed end of the data area to avoid missing data when markers are not arranged orthogonally to the scan direction. The preprocessing involves rotating or skewing the image to provide rotated or skewed versions of the image to facilitate locating markers regardless of their angular orientation in the image.

Claims

1. A localization marker adapted for identification with a single scan direction, comprising:

a body with an exposed surface; and

a marker pattern provided on the exposed surface, the marker pattern including;

a fixed pattern; and

a data area comprising a variable pattern, wherein the fixed pattern comprises a pair of parallel, linear features that are spaced apart a predefined distance, wherein the data area is sandwiched between the pair of linear features, wherein components of the variable pattern are positioned at predefined offsets from inner edges of the pair of the linear features, and wherein each of the linear features has a length greater than a maximum length of the components of the variable pattern.

2. The localization marker of claim 1, wherein the linear features comprise elongated bars with a width of at least 1 pixel.

3. The localization marker of claim 1, wherein a width of the linear features is at least 20 percent greater than the maximum length of the components.

4. The localization marker of claim 1, wherein each of the linear features is positioned on the exposed surface to extend a distance beyond opposite ends of the components of the variable pattern.

5. The localization marker of claim 1, wherein the components comprise lines extending parallel to each other and to the linear features and wherein each of the lines is positioned at one of the predefined offsets from the inner edges of the pair of the linear features.

6. The localization marker of claim 1, wherein the fixed pattern further comprises second and third pairs of linear features, wherein the linear features extend parallel to the pair of linear features outside of the data area, and wherein, on each side of the data area, adjacent ones of the linear features are spaced apart a second predefined distance greater than about 1 pixel.

7. The localization marker of claim 6, wherein each of the linear features of the fixed pattern has an equal length greater than the maximum length of the components of the variable pattern.

8. A system for facilitating determination of positions of mobile devices such as unmanned aerial vehicles within an operating space containing a number of objects, comprising:

at least three markers placed on exposed surfaces of the objects, wherein each of the markers includes a pattern of linear bars adjacent a data area, wherein the pattern of linear bars is identical on each of the markers and the data area has a pattern that varies between the markers and wherein the pattern of linear bars comprises at least a pair of spaced apart and parallel bars disposed on either side of the data area;

a mobile device operable to move about the operating space in response to control signals; and

a controller on the mobile device generating the control signals based on a determination of a position of the mobile device within the operating space relative to locations of the markers, wherein the controller executes a localization module to locate each of the markers, to extract data from the pattern of the data areas, and to estimate the position of the mobile device based on the extracted data.

9. The system of claim 8, wherein the localization module uses a single direction scan of an image to locate the markers in the image and wherein at least one of the markers is positioned to be oriented in the image such that the pattern of linear bars is non-orthogonal to the single direction scan.

10. The system of claim 9, wherein the localization module repeats the single direction scan on a rotated or skewed version of the image to locate the markers in the image.

11. The system of claim 8, wherein the pair of parallel bars extends a distance beyond each exposed end of the pattern in the data area.

12. The system of claim 11, wherein the pattern of each of the data areas comprises one or more linear patterns extending parallel to inner edges of the pair of spaced apart and parallel bars and at predefined distances from the inner edges.

13. The system of claim 11, wherein the pattern of linear bars comprises parallel stripes on either side of the data area and wherein adjacent pairs of the parallel stripes are spaced apart a predefined distance.

14. A method for locating markers in an image of an operating space for a mobile device, comprising:

with a processor on the mobile device, preprocessing an image to generate a set of image data;

with the processor, locating fixed features of markers by tracing edges of the fixed features; and

with the processor, extracting variable data payloads of each of the markers associated with the located fixed features,

wherein the fixed features of each of the markers comprise a pair of parallel lines extending along opposite sides of a data area containing the variable data payload.

15. The method of claim 14, wherein each of the parallel lines extends a distance beyond each exposed end of the data area.

16. The method of claim 14, wherein the preprocessing comprises rotating or skewing the image to provide a rotated or skewed version of the image with the set of image data.

17. The method of claim 14, wherein the preprocessing comprises sub-pixel edge detection.

18. The method of claim 14, wherein the locating further comprises performing a horizontal scan of the set of image data and then performing the tracing of the edges, wherein the tracing of the edges comprises performing a first vertical scan downwards and a second vertical scan upwards.

19. The method of claim 14, wherein the tracing of the edges is performed row-by-row for a single direction and wherein a predicting and updating process is performed for each pixel along a traced row.