Methods and systems for marker identification

• US 10,452,916 B2
• Filed: 09/20/2018
• Issued: 10/22/2019
• Est. Priority Date: 12/22/2015
• Status: Active Grant

• Current Assignee: Worldviz LLC
• Original Assignee: Worldviz LLC
• Inventors: Beall, Andrew C., Pusch, Matthias
• Primary Examiner(s): Wang, Carol
• Application Number:
  US16/136,687
• Publication Number:
  US 20190019034A1
• Time in Patent Office: 397 Days
• CPC Class Codes: G06T 2207/10048 Infrared image
• CPC Class Codes: G06T 2207/30204 Marker
• CPC Class Codes: G06T 7/248 involving reference images or patches
• CPC Class Codes: G06T 7/62 of area, perimeter, diameter or volume
• CPC Class Codes: G06T 7/70 Determining position or orientation of objects or cameras camera calibration G06T7/80
• CPC Class Codes: G06V 10/143 Sensing or illuminating at different wavelengths
• CPC Class Codes: G06V 10/60 relating to illumination properties, e.g. using a reflectance or lighting model
• CPC Class Codes: G06V 10/62 relating to a temporal dimension, e.g. time-based feature extraction; Pattern tracking
• CPC Class Codes: G06V 20/20 in augmented reality scenes

Abstract

A marker tracking system configured to detect light patterns (e.g., infrared light patterns) generated by one or more markers is described. A given marker is configured with a code which identifies the marker in a motion tracking camera field of view. Motion tracking camera(s) record the emitted infrared light and are configured to directly, or in conjunction with an associated computing device, computationally distinguish a given marker with high accuracy and efficiently.

Claims

1. A system comprising:

a processing system comprising at least one processing device;

a motion tracking camera, wherein the motion tracking camera detects infrared light;

a marker, the marker configured to emit a binary pattern of infrared light pulses;

non-transitory media that stores instructions readable by the processing system, that when executed by the processing system, cause the system to;

identify, within a given image, a cluster of adjacent pixels corresponding to infrared light emitted from the marker;

determine a count of how many saturated pixels are within a given identified cluster of adjacent pixels of the given image;

determine a highest count of saturated pixels from respective counts of saturated pixels of respective given identified clusters of respective given images in a plurality of images;

calculate a radius for a circular perimeter by at least multiplying the highest count of saturated pixels by a first factor;

identify a first area, bounded by the circular perimeter, of a first image, of the plurality of images, encompassing a highest number of infrared light pixels within a cluster of adjacent pixels;

for pixels within the first area, identify a first set of pixels having a highest intensity as compared to other sets of pixels in the first area, wherein the first set of pixels has a count equal to the highest count of saturated pixels;

excluding the first set of pixels, determine a sum of infrared light intensities of the first area;

determine respective sum of infrared light intensities for respective areas in the remaining one or more images of the plurality of images; and

using the determined respective sum of infrared light intensities of the respective areas of respective images of the plurality of images,

assign a binary value to the plurality of images.

2. The system of claim 1, wherein the configured marker pattern is a binary pattern of encoded one or more binary bits.

3. The system of claim 1, wherein a position in space of the marker in the first image corresponds to a center of a respective area of the first image bounded by the circular perimeter.

4. The system of claim 1, wherein the marker is user configurable to emit a plurality of infrared light pulses patterns corresponding to a plurality of binary code sets.

5. The system of claim 1, further comprising a plurality of markers configured to be attached to a user to enable at least in part, the system to determine a corresponding orientation of the user.

6. The system of claim 1, wherein the marker is configured to be attached to a user to enable at least in part, the system to determine a corresponding position of the user.

7. The system of claim 1, wherein the instructions are further configured to cause the system to determine if the respective sum of infrared light intensities for the respective areas of the respective images of the plurality of images corresponds to a set of consecutive images of low intensities followed by a set of consecutive images of high intensities.

8. The system of claim 1, wherein the instructions are further configured to cause the system to determine if the respective sum of infrared light intensities for the respective areas of the respective images of the plurality of images correspond to a set of consecutive images of medium intensities.

9. The system of claim 1, the system further configured to:

combine the binary value with other assigned binary values to a respective subsequent plurality of images to define a binary code set.

10. The system of claim 1, the system further configured to:

combine the binary value with other assigned binary values to a respective subsequent plurality of images to define a binary code set;

compare the defined binary code set to a configured binary code set of the marker; and

based at least in part on the comparison of the defined binary code set to the configured binary code set of the marker, identify the marker.

11. A system comprising:

a processing system comprising at least one processing device;

a marker, the marker configured to emit a binary pattern of infrared light pulses;

non-transitory media that stores instructions readable by the system, that when executed by the processing system, cause the system to;

identify, within a given image in a plurality of images, a cluster of adjacent pixels corresponding to infrared light emitted from the marker;

determine a count of how many saturated pixels are within a given identified cluster of adjacent pixels of a given image;

determine a highest count of saturated pixels from respective counts of saturated pixels of respective given identified clusters of respective given images;

calculate a radius for a circular perimeter by at least multiplying the highest count of saturated pixels by a first factor;

identify a first area, bounded by the circular perimeter, of a first image of the plurality of images encompassing a highest number of infrared light pixels within a cluster of adjacent pixels;

for pixels within the first area, identify a first set of pixels having a highest intensity as compared to other sets of pixels in the first area, wherein the first set of pixels has a count equal to the highest count of saturated pixels;

excluding the first set of pixels, determine a sum of infrared light intensities of the first area; and

determine respective sum of infrared light intensities for respective areas in the remaining one or more images of the plurality of images.

12. The system of claim 11, the system further configured to:

based at least in part on the determined respective sum of infrared light intensities of the respective areas of respective images of the plurality of images,

assign a binary value to the plurality of images.

13. The system of claim 11, wherein the configured marker pattern is a binary pattern of encoded one or more binary bits.

14. The system of claim 11, wherein the instructions are further configured to cause the system to determine if the respective sum of infrared light intensities for the respective areas of the respective images of the plurality of images corresponds to a set of consecutive images of low intensities followed by a set of consecutive images of high intensities.

15. The system of claim 11, wherein the instructions are further configured to cause the system to determine if the respective sum of infrared light intensities for the respective areas of the respective images of the plurality of images correspond to a set of consecutive images of medium intensities.

16. The system of claim 11, wherein a position in space of the marker in a given image corresponds to a center of a respective area bounded by the circular perimeter.

17. The system of claim 11, wherein the marker is user configurable to generate a plurality of distinguishable infrared light pulses patterns corresponding to a binary code set.

18. The system of claim 11, further comprising a plurality of markers configured to be attached to a user to enable at least in part, the system to determine a corresponding orientation of the user.

19. The system of claim 11, wherein the marker is configured to be attached to a user to enable at least in part, the system to determine a corresponding position of the user.

20. The system of claim 11, the system further configured to:

based at least in part on the determined respective sum of infrared light intensities of the respective areas of respective images of the plurality of images,

assign a binary value to the plurality of images; and

combine the binary value with other assigned binary values to a respective subsequent plurality of images to define a binary code set.

21. The system of claim 11, the system further configured to:

based at least in part on the determined respective sum of infrared light intensities of the respective areas of respective images of the plurality of images,

assign a binary value to the plurality of images;

combine the binary value with other assigned binary values to a respective subsequent plurality of images to define a binary code set;

compare the defined binary code set to a configured binary code set of the marker; and

based at least in part on the comparison of the defined binary code set to the configured binary code set of the marker, identify the marker.

22. A computer implemented method of marker identification, the method comprising:

identifying, within a given image of a plurality of images captured by a camera, a cluster of adjacent pixels corresponding to infrared light emitted from a marker;

determining a count of how many saturated pixels are within a given identified cluster of adjacent pixels of a given image of the plurality of images;

determining a highest count of saturated pixels from respective counts of saturated pixels of respective given identified clusters of respective given images;

calculating a radius for a circular perimeter by at least multiplying the highest count of saturated pixels by a first factor;

identifying a first area, bounded by the circular perimeter, of a first image of the plurality of images encompassing a highest number of infrared light pixels within a cluster of adjacent pixels;

for pixels within the first area, identifying a first set of pixels having a highest intensity as compared to other sets of pixels in the first area, wherein the first set of pixels has a count equal to the highest count of saturated pixels;

excluding the first set of pixels, determining a sum of infrared light intensities of the first area;

determining respective sum of infrared light intensities for respective areas in the remaining one or more images of the plurality of images;

using the respective sum of infrared light intensities for respective areas in tracking a user and/or object in a physical space; and

using the tracking of the user and/or object in generating a virtual and/or augmented reality experience.

23. The method of claim 22, the method further comprising:

based at least in part on the determined respective sum of infrared light intensities of the respective areas of respective images of the plurality of images,

assigning a binary value to the plurality of images.