Controller movement tracking with light emitters

• US 10,152,141 B1
• Filed: 08/18/2017
• Issued: 12/11/2018
• Est. Priority Date: 08/18/2017
• Status: Active Grant

• Current Assignee: Mentor Acquisition One, LLC (Magic Leap, Inc.)
• Original Assignee: Osterhout Design Group
• Inventors: Lohse, Robert Michael, Haddick, John D.
• Primary Examiner(s): Shah, Priyank
• Application Number:
  US15/680,917
• Time in Patent Office: 480 Days
• CPC Class Codes: G02B 2027/0138 comprising image capture systems, e.g. camera
• CPC Class Codes: G02B 2027/014 comprising information/image processing systems
• CPC Class Codes: G02B 2027/0178 Eyeglass type eyeglass details G02C
• CPC Class Codes: G02B 2027/0187 slaved to motion of at least a part of the body of the user, e.g. head, eye
• CPC Class Codes: G02B 27/017 Head mounted
• CPC Class Codes: G02B 27/0172 characterised by optical features
• CPC Class Codes: G06F 3/011 Arrangements for interaction with the human body, e.g. for user immersion in virtual reality blind teaching G09B21/00
• CPC Class Codes: G06F 3/013 Eye tracking input arrangements G06F3/015 takes precedence
• CPC Class Codes: G06F 3/014 Hand-worn input/output arrangements, e.g. data gloves
• CPC Class Codes: G06F 3/0308 comprising a plurality of distinctive and separately oriented light emitters or reflectors associated to the pointing device, e.g. remote cursor controller with distinct and separately oriented LEDs at the tip whose radiations are captured by a photo-detector associated to the screen
• CPC Class Codes: G06F 3/0325 using a plurality of light emitters or reflectors or a plurality of detectors forming a reference frame from which to derive the orientation of the object, e.g. by triangulation or on the basis of reference deformation in the picked up image

Abstract

A head-worn computer includes a camera system positioned to capture a surrounding environment in front of a user, a processor that identifies a position of a plurality of light emitters mounted on a hand-held controller from images captured by the camera system and tracks the position of the plurality of light emitters as the hand-held controller moves in the surrounding environment and interprets the tracked position as positional changes of the hand-held controller. The processor uses the position of the plurality of light emitters as markers in three dimensional space, the markers used as an anchor for virtual content presented in a see-through display of the head-worn computer.

Claims

1. A head-worn computer, comprising:

a camera system positioned to capture a surrounding environment in front of a user wearing the head-worn computer;

a processor adapted to identify a position of a plurality of light emitters mounted on a hand-held controller from images captured by the camera system;

the processor further adapted to track the position of the plurality of light emitters as the hand-held controller moves in the surrounding environment and interpret the tracked position as positional changes of the hand-held controller;

the processor further adapted to use the position of the plurality of light emitters as markers in three dimensional space, the markers used as an anchor for virtual content presented in a see-through display of the head-worn computer; and

wherein the camera system comprises a fisheye lens to increase a sensitivity of the processor by causing real-world movements of the plurality of light emitters to be exaggerated so small real-world movements become larger movements in the captured images.

2. The head-worn computer of claim 1, wherein the camera system comprises two or more cameras mounted separately on the head-worn computer such that each camera views the plurality of light emitters from a different perspective such that a 3D model is used to map the position of the hand-held controller.

3. The head-worn computer of claim 1, wherein the plurality of light emitters are mounted in a pre-determined pattern on the hand-held controller such that the camera system predicts a position of the hand-held controller based on a perspective view of the plurality of light emitters in the pre-determined pattern.

4. The head-worn computer of claim 1, further comprising, an inertial measurement unit (IMU), wherein the position of the hand-held controller is based on the position of the plurality of light emitters as markers and output from the IMU.

5. A head-worn computer, comprising:

a camera system positioned to capture a surrounding environment in front of a user wearing the head-worn computer;

a processor adapted to identify a position of a plurality of light emitters mounted on a hand-held controller in images captured by the camera system;

the processor further adapted to track the position of the plurality of light emitters as the hand-held controller moves in the surrounding environment and interpret the track as positional changes of the hand-held controller;

the processor further adapted to use the position of the plurality of light emitters as a prediction of the position of the hand-held controller in three dimensional space, the predicted position used as a control input for a software application operating on the head-worn computer; and

wherein the camera system comprises a fisheye lens to increase a sensitivity of the processor by causing real-world movements of the plurality of light emitters to be exaggerated so small real-world movements become larger movements in the captured images.