POSITION TRACKING SYSTEM THAT EXPLOITS ARBITRARY CONFIGURATIONS TO DETERMINE LOOP CLOSURE

• US 20180113506A1
• Filed: 10/25/2016
• Published: 04/26/2018
• Est. Priority Date: 10/25/2016
• Status: Active Grant

• Current Assignee: META Platforms Technologies LLC (Meta Platforms, Inc. (f/k/a Facebook, Inc.))
• Original Assignee: Oculus VR, LLC (Meta Platforms, Inc. (f/k/a Facebook, Inc.))
• Inventors: Hall, Michael
• Granted Patent: US 9,983,665 B2
• Time in Patent Office: Days
• CPC Class Codes: G01C 21/20 Instruments for performing navigational calculations G01C21/24, G01C21/26 take precedence
• CPC Class Codes: G01S 1/7034 Mounting or deployment thereof
• CPC Class Codes: G01S 2201/01 adapted for specific applications or environments
• CPC Class Codes: G01S 5/163 Determination of attitude using inertial means G01C9/00; control of attitude G05D1/49
• CPC Class Codes: G02B 2027/0138 comprising image capture systems, e.g. camera
• CPC Class Codes: G02B 2027/014 comprising information/image processing systems
• CPC Class Codes: G02B 2027/0178 Eyeglass type eyeglass details G02C
• CPC Class Codes: G02B 27/0093 with means for monitoring data relating to the user, e.g. head-tracking, eye-tracking
• CPC Class Codes: G02B 27/017 Head mounted
• CPC Class Codes: G02B 27/0172 characterised by optical features
• CPC Class Codes: G06F 3/012 Head tracking input arrangements
• CPC Class Codes: G06F 3/0304 Detection arrangements using opto-electronic means constructional details of pointing devices not related to the detection arrangement using opto-electronic means G06F3/033; optical digitisers G06F3/042
• CPC Class Codes: G06N 20/00 Machine learning
• CPC Class Codes: G06T 19/006 Mixed reality object pose determination, tracking or camera calibration for mixed reality G06T7/00

Abstract

A position tracking system includes an array of detection pixels coupled to a head-mounted display (HMD) configured to capture light signals reflected from an environment surrounding the HMD. The position tracking system maintains, in a database, signal data related to a plurality of positions of the HMD. The position tracking system determines signal data related to a position of the HMD, based on the light signals captured during a time instant of the position of the HMD. The position tracking system matches the determined signal data to the maintained signal data, determines a present position of the HMD based on the matching, updates position data of the HMD with the determined position, and provides the updated position data of the HMD.

Claims

1. A position tracking system comprising:

an array comprising a plurality of detection pixels coupled to a head-mounted display (HMD) configured to capture light signals reflected from an environment surrounding the HMD; and

a controller, including a processor and a database, coupled to the array of sensor elements, the controller configured to;

maintain, in the database, signal data related to a plurality of positions of the HMD in the environment,

determine signal data related to a position of the HMD in the environment, based on the light signals captured during a time instant of the position of the HMD,

match the determined signal data to the maintained signal data,

determine a present position of the HMD based on the matching,

update position data of the HMD with the determined position, and

provide the updated position data of the HMD.

2. The system of claim 1, wherein the controller is further configured to:

determine that the present position of the HMD is the same as one of the plurality of positions of the HMD, if the matching is within a threshold value.

3. The system of claim 1, wherein the controller is further configured to:

determine that the present position of the HMD is a new position of the HMD in the environment, if the matching is not within the threshold value.

4. The system of claim 1, wherein:

the maintained signal data comprise a plurality of strings of binary numbers stored in the database obtained based on the light signals captured by the plurality of detection pixels during a plurality of time instants, each string has a unique timestamp corresponding to one of the plurality of time instants; and

the determined signal data comprise a string of binary numbers.

5. The system of claim 4, wherein the controller is further configured to:

perform bit-level comparisons between the string and the plurality of strings; and

determine, based on the bit-level comparisons, a timestamp corresponding to one of the plurality of time instants when the HMD was located in approximately the same position or had approximately the same orientation in the environment as during the time instant.

6. The system of claim 5, wherein the controller is further configured to:

send the determined timestamp to the HMD for associating with existing stored data of the environment.

7. The system of claim 5, wherein the controller is further configured to:

retrieve, from the HMD based on the determined timestamp, information about existing stored data of the environment; and

update the position data of the HMD based on the retrieved information.

8. The system of claim 1, wherein the processor is configured to generate a sting of binary numbers based on comparing intensities of the captured light signals with a defined threshold level.

9. The system of claim 1, wherein the plurality of detection pixels in the array is arranged in a ring placed around a user'"'"'s head where the HMD is mounted.

10. The system of claim 1, wherein the detection pixels in the array are positioned on a band of the HMD.

11. The system of claim 1, wherein the detection pixels in the array are positioned along a frame of glasses of the HMD.

12. The system of claim 1, wherein the detection pixels in the array are distributed around an exterior of a hand-held controller device that moves through the environment.

13. The system of claim 1, wherein the detection pixels in the array are positioned in fixed locations relative to each other.

14. The system of claim 1, wherein the detection pixels in the array are photodiodes.

15. The system of claim 1, wherein the detection pixels in the array measure at least one of polarization and depth.

16. The system of claim 1, wherein the plurality of detection pixels in the array comprises different types of detection pixels.

17. The system of claim 1, wherein the different types of detection pixels comprise at least two of:

infra-red (IR) sensors, color sensors, depth sensors, polarization sensors, and reflectivity sensors.

18. A position tracking system comprising:

an array comprising a plurality of detection pixels coupled to a head-mounted display (HMD) configured to capture light signals reflected from an environment surrounding the HMD;

a controller, including a processor and a database, coupled to the array of sensor elements, the controller configured to;

maintain, in the database, signal data related to a plurality of positions of the HMD in the environment,

determine signal data related to a position of the HMD in the environment, based on the light signals captured during a time instant of the position of the HMD,

match the determined signal data to the maintained signal data,

determine, based on the matching, loop closure by determining that a present position of the HMD is the same as one of the plurality of positions,

update position data of the HMD based on the determined loop closure, and

provide the updated position data of the HMD.

19. The system of claim 18, wherein:

the maintained signal data comprise a plurality of strings of binary numbers stored in the database obtained based on the light signals captured by the plurality of detection pixels during a plurality of time instants, each string has a unique timestamp corresponding to one of the plurality of time instants; and

the determined signal data comprise a string of binary numbers.

20. The system of claim 19, wherein the controller is further configured to:

perform bit-level comparisons between the string and the plurality of strings; and

determine, based on the bit-level comparisons, a timestamp corresponding to one of the plurality of time instants when the HMD was located in approximately the same position or had approximately the same orientation in the environment as during the time instant.

21. The system of claim 18, wherein the plurality of detection pixels in the array is arranged in a ring placed around a user'"'"'s head where the HMD is mounted.

22. The system of claim 18, wherein the detection pixels in the array are photodiodes.