Detecting the pose of an out-of-range controller

• US 10,558,260 B2
• Filed: 12/15/2017
• Issued: 02/11/2020
• Est. Priority Date: 12/15/2017
• Status: Active Grant

• Current Assignee: Microsoft Technology Licensing LLC (Microsoft Corporation)
• Original Assignee: Microsoft Technology Licensing LLC (Microsoft Corporation)
• Inventors: Miller, Quentin Simon Charles
• Primary Examiner(s): Cuff, Michael A
• Application Number:
  US15/844,290
• Publication Number:
  US 20190187779A1
• Time in Patent Office: 788 Days
• Field of Search: 463 31
• CPC Class Codes: A63F 13/235 using a wireless connection, e.g. infrared or piconet
• CPC Class Codes: A63F 13/25 Output arrangements for video game devices
• CPC Class Codes: G02B 2027/0138 comprising image capture systems, e.g. camera
• CPC Class Codes: G02B 2027/014 comprising information/image processing systems
• CPC Class Codes: G02B 27/0093 with means for monitoring data relating to the user, e.g. head-tracking, eye-tracking
• CPC Class Codes: G02B 27/0172 characterised by optical features
• CPC Class Codes: G06F 3/011 Arrangements for interaction with the human body, e.g. for user immersion in virtual reality blind teaching G09B21/00
• CPC Class Codes: G06F 3/0346 with detection of the device orientation or free movement in a 3D space, e.g. 3D mice, 6-DOF [six degrees of freedom] pointers using gyroscopes, accelerometers or tilt-sensors
• CPC Class Codes: G08C 17/02 using a radio link
• CPC Class Codes: G08C 2201/32 Remote control based on movements, attitude of remote control device

Abstract

A technique is described herein for determining a pose of at least one mobile controller. The technique involves using a transmitter to initially emit a magnetic field and/or electromagnetic radiation at an elevated-power level, with respect to a normal-use power level. A receiver detects the magnetic field and/or electromagnetic radiation, to provide received signals. A tracking component determines the pose of the controller based on the received signals. Thereafter, the technique transmits the magnetic field and/or electromagnetic radiation at the normal-use power level. In one implementation, a head-mounted display (HMD) uses the above-summarized technique to find the controller when the user initially dons the HMD. This is useful because the controller may be invisible to the user who dons the HMD because it is out-of-range with respect to the normal-use operating space of the HMD.

Claims

1. A method, performed, at least in part, by a head-mounted display (HMD), the HMD providing a mixed-reality experience in conjunction with at least one controller, comprising:

determining that an initiation event has occurred that initiates a process of locating a mobile controller within a physical environment;

sending a first instruction to a transmitter in response to the initiation event, which instructs the transmitter to use a first power level in emitting a magnetic field and/or electromagnetic radiation;

receiving signals from a receiver in response to detecting the magnetic field and/or electromagnetic radiation;

determining a pose of the controller relative to the HMD based on the signals that have been received; and

sending a second instruction to the transmitter, which instructs the transmitter to emit the magnetic field and/or the electromagnetic radiation using a second power level,

the first power level being higher than the second power level, and the first power level enabling detection of the controller when the controller is out-of-range with respect to a space in which the controller operates when being manually manipulated by the user in a course of providing a mixed-reality experience,

wherein the controller further includes a local tracking component for tracking a pose of the controller, and for storing a last-known pose of the controller in a data store provided by the controller, and

wherein one initiation event corresponds to a case in which;

the user has commenced using the HMD, and the pose of the controller relative to the HMD cannot be ascertained based on a last-known pose provided by the controller.

2. The method of claim 1, wherein the transmitter is a magnetic field generator, and the receiver is a magnetic field receiver.

3. The method of claim 1, wherein the transmitter is a source of electromagnetic radiation, and the receiver includes one or more cameras for receiving the electromagnetic radiation.

4. The method of claim 1, wherein the transmitter produces the magnetic field and/or the electromagnetic radiation using the first power level at a prescribed duty cycle.

5. The method of claim 1, wherein another initiation event corresponds to a case in which the user has commenced using the HMD, and in which:

(a) the HMD determines that the controller is connected to the HMD through a data channel; and

(b) the HMD cannot initially detect the controller via the receiver.

6. The method of claim 1, wherein the method further includes terminating the process of locating the controller when a prescribed amount of time has occurred since the initiation event.

7. The method of claim 1, wherein the method further includes:

receiving handling signals from a handling-sensing device associated with the controller; and

terminating the process of locating the controller when the handling signals indicate that the controller is being handled by the user.

8. The method of claim 1, wherein the method further includes presenting an indicator to the user using the HMD that conveys the pose of the controller relative to the HMD.

9. The method of claim 1, wherein the transmitter is a component of the controller, and the receiver is a component of the HMD.

10. The method of claim 1, wherein the transmitter is a component of the HMD, and the receiver is a component of the controller.

11. A system for providing a mixed-reality experience, comprising:

a mobile controller for manipulation by a user within a physical environment;

a head-mounted display (HMD) worn by a head of the user, the HMD including at least;

a tracking component configured to generate pose information, the pose information specifying;

a pose of the HMD within the physical environment; and

a pose of the controller relative to the HMD;

a scene presentation component configured to generate a mixed-reality experience based on the pose information; and

one or more output devices for presenting the mixed-reality experience to the user;

a transmitter for emitting a magnetic field and/or electromagnetic radiation; and

a receiver for receiving the magnetic field and/or the electromagnetic radiation, to provide received signals;

the HMD further including a controller interface component that is configured to interact with the controller, the controller interface component being configured to;

instruct the transmitter to use a first power level in emitting the magnetic field and/or the electromagnetic radiation, in response to determining that an initiation event has occurred; and

instruct the transmitter to use a second power level in emitting the magnetic field and/or electromagnetic radiation when the user is manually manipulating the controller,

the first power level being higher than the second power level, wherein the controller further includes;

at least one local camera for capturing images of the physical environment, to provide image information;

a local tracking component for tracking a pose of the controller based on the image information, and for storing a last-known pose of the controller in a data store; and

an interface component for transferring the last-known pose to the HMD,

the tracking component of the HMD being configured to determine the pose of the controller relative to the HMD based on at least the signals received by the receiver when the transmitter is driven at the first power level, and when the controller is out-of-range with respect to a space in which the controller operates when being manually manipulated by the user, and when the pose of the controller relative to the HMD cannot be ascertained based on the last-known pose provided by the controller,

each component being implemented by hardware logic circuitry, the hardware logic circuitry corresponding to;

(a) one or more hardware processors that perform operations by executing machine-readable instructions stored in a memory, and/or by (b) one or more other hardware logic components that perform the operations using a task-specific collection of logic gates.

12. The system of claim 11, wherein the controller interface component is configured to instruct the transmitter to cease transmitting the magnetic field and/or the EM radiation at the first power level when a prescribed amount of time has occurred since the initiation event.

13. The system of claim 11,

wherein the controller includes a handling-sensing device that provides handling signals to the HMD, and

wherein the controller interface component is configured to instruct the transmitter to cease transmitting the magnetic field and/or the EM radiation at the first power level when the handling signals indicate that the controller is being handled by the user.

14. The system of claim 11, wherein the HMD further includes a scene presentation component that is configured to present an indicator to the user that conveys the pose of the controller relative to the HMD.

15. The system of claim 11, wherein the transmitter is a component of the controller, and the receiver is a component of the HMD.

16. The system of claim 11, wherein the transmitter is a component of the HMD, and the receiver is a component of the controller.

17. A computer-readable storage medium for storing computer-readable instructions, the computer-readable instructions, when executed by one or more hardware processors, performing a method that comprises:

determining that an initiation event has occurred that initiates a process of locating a mobile controller within a physical environment,

the controller being used in conjunction with a head-mounted display (HMD) to provide a mixed-reality experience;

sending a first instruction by the HMD via a data channel to a transmitter in response to the initiation event, which instructs the transmitter to use a first power level in emitting a magnetic field and/or electromagnetic radiation;

receiving signals by a receiver in response to detecting the magnetic field and/or electromagnetic radiation emitted by the transmitter at the first power level, the receiver receiving signals through a path other than the data channel;

determining a pose of the controller relative to the HMD based on the signals that have been received;

presenting an indicator to the user on a display device of the HMD that conveys the pose of the controller relative to the HMD; and

sending a second instruction to the transmitter via the data channel, which instructs the transmitter to emit the magnetic field and/or the electromagnetic radiation using a second power level, and stop emitting the magnetic field and/or the electromagnetic radiation at the first power level,

the first power level being higher than the second power level, and the first power level enabling detection of the controller when the controller is out-of-range with respect to a space in which the controller operates when being manually manipulated by the user,

wherein one initiation event corresponds to a case in which;

(a) the HMD determines that the controller is connected to the HMD through the data channel; and

(b) the HMD cannot initially detect the controller via the receiver, and

wherein the controller provides the transmitter and the HMD provides the receiver.

18. The computer-readable storage medium of claim 17, wherein said presenting an indicator includes presenting one or more markers that reveal the pose of the controller.

19. The computer-readable storage medium of claim 17, wherein the method further includes terminating the process of locating the controller when a prescribed amount of time has occurred since the initiation event.