Redundant tracking system

• US 10,504,287 B2
• Filed: 10/24/2017
• Issued: 12/10/2019
• Est. Priority Date: 10/24/2016
• Status: Active Grant

• Current Assignee: Snap, Inc.
• Original Assignee: Snap, Inc.
• Inventors: McPhee, Andrew James, Hare, Samuel Edward, Yu, Peicheng, Murphy, Robert Cornelius, Sagar, Dhritiman
• Primary Examiner(s): Broome, Said
• Application Number:
  US15/792,347
• Publication Number:
  US 20180114364A1
• Time in Patent Office: 777 Days
• Field of Search: None
• CPC Class Codes: A63F 13/211 using inertial sensors, e.g. accelerometers or gyroscopes
• CPC Class Codes: G06F 11/08 Error detection or correction by redundancy in data representation, e.g. by using checking codes
• CPC Class Codes: G06F 2203/0381 Multimodal input, i.e. interface arrangements enabling the user to issue commands by simultaneous use of input devices of different nature, e.g. voice plus gesture on digitizer
• CPC Class Codes: G06F 2203/0382 Plural input, i.e. interface arrangements in which a plurality of input device of the same type are in communication with a PC
• CPC Class Codes: G06F 3/011 Arrangements for interaction with the human body, e.g. for user immersion in virtual reality blind teaching G09B21/00
• CPC Class Codes: G06F 3/0304 Detection arrangements using opto-electronic means constructional details of pointing devices not related to the detection arrangement using opto-electronic means G06F3/033; optical digitisers G06F3/042
• CPC Class Codes: G06F 3/0346 with detection of the device orientation or free movement in a 3D space, e.g. 3D mice, 6-DOF [six degrees of freedom] pointers using gyroscopes, accelerometers or tilt-sensors
• CPC Class Codes: G06F 3/038 Control and interface arrangements therefor, e.g. drivers or device-embedded control circuitry
• CPC Class Codes: G06T 15/20 Perspective computation
• CPC Class Codes: G06T 19/003 Navigation within 3D models or images
• CPC Class Codes: G06T 19/006 Mixed reality object pose determination, tracking or camera calibration for mixed reality G06T7/00
• CPC Class Codes: G06T 7/246 using feature-based methods, e.g. the tracking of corners or segments

Abstract

A redundant tracking system comprising multiple redundant tracking sub-systems, enabling seamless transitions between such tracking sub-systems, provides a solution to this problem by merging multiple tracking approaches into a single tracking system. This system is able to combine tracking objects with six degrees of freedom (6DoF) and 3DoF through combining and transitioning between multiple tracking systems based on the availability of tracking indicia tracked by the tracking systems. Thus, as the indicia tracked by any one tracking system becomes unavailable, the redundant tracking system seamlessly switches between tracking in 6DoF and 3DoF thereby providing the user with an uninterrupted experience.

Claims

1. A system comprising:

a memory; and

at least one hardware processor coupled to the memory and comprising a redundant tracking system, the redundant tracking system comprising a set of tracking systems that include at least a simultaneous localization and mapping tracking system, and a natural feature tracking system, and one or more processors that cause the system to perform operations comprising;

rendering an object at a position in a three-dimensional space;

tracking the object at the position in the three-dimensional space via a first tracking system among the set of tracking systems, the first tracking system tracking the object based on a set of indicia;

detecting an interruption of at least an indicia from among the set of indicia tracked by the first tracking system; and

tracking the object at the position in the three-dimensional space via a second tracking system among the set of tracking systems, in response to detecting the interruption of the indicia from among the set of indicia.

2. The system of claim 1, wherein the set of indicia comprises at least six indicia.

3. The system of claim 1, wherein the first tracking system is a natural feature tracking system.

4. The system of claim 1, wherein the second tracking system is a gyroscopic tracking system.

5. The system of claim 1, wherein the first tracking system tracks pitch, yaw, roll, and translations based on the set of indicia.

6. The system of claim 1, wherein the redundant tracking system causes the system to perform operations further comprising:

maintaining a set of templates to generate objects to be displayed in the three-dimensional space;

receiving a selection of a template from among the set of templates;

receiving a selection of the position in the three-dimensional space;

generating the object based on the selected template; and

assigning the object to the position in the three-dimensional space.

7. A method for tracking an object by a redundant tracking system, the redundant tracking system comprising a set of tracking systems that include at least a simultaneous localization and mapping tracking system, and a natural feature tracking system, and the method comprising:

rendering the object at a position in a three-dimensional space;

tracking the object at the position in the three-dimensional space via a first tracking system among the set of tracking systems, the first tracking system tracking the object based on a set of indicia;

detecting an interruption of at least an indicia from among the set of indicia tracked by the first tracking system; and

tracking the object at the position in the three-dimensional space via a second tracking system among the set of tracking systems, in response to detecting the interruption of the indicia from among the set of indicia.

8. The method of claim 7, wherein the set of indicia comprises at least six indicia.

9. The method of claim 7, wherein the first tracking system is a natural feature tracking system.

10. The method of claim 7, wherein the second tracking system is a gyroscopic tracking system.

11. The method of claim 7, wherein the first tracking system tracks pitch, yaw, roll, and translations based on the set of indicia.

12. The method of claim 7, further comprising:

maintaining a set of templates to generate objects to be displayed in the three-dimensional space;

receiving a selection of a template from among the set of templates;

receiving a selection of the position in the three-dimensional space;

generating the object based on the selected template; and

assigning the object to the position in the three-dimensional space.

13. A non-transitory machine-readable storage medium including a redundant tracking system, the redundant tracking system comprising a set of tracking systems that include at least a simultaneous localization and mapping system, and a natural feature tracking system, and comprising instructions that, when executed by one or more processors of a machine, cause the machine to perform operations comprising:

rendering an object at a position in a three-dimensional space;

tracking the object at the position in the three-dimensional space via a first tracking system among the set of tracking systems, the first tracking system tracking the object based on a set of indicia;

detecting an interruption of at least an indicia from among the set of indicia tracked by the first tracking system; and

tracking the object at the position in the three-dimensional space via a second tracking system among the set of tracking systems, in response to detecting the interruption of the indicia from among the set of indicia.

14. The non-transitory machine-readable storage medium of claim 13, wherein the set of indicia comprises at least six indicia.

15. The non-transitory machine-readable storage medium of claim 13, wherein the first tracking system is a natural feature tracking system.

16. The non-transitory machine-readable storage medium of claim 13, wherein the second tracking system is a gyroscopic tracking system.

17. The non-transitory machine-readable storage medium of claim 13, wherein the first tracking system tracks pitch, yaw, roll, and translations based on the set of indicia.