Predictive RF beamforming for head mounted display

• US 10,209,771 B2
• Filed: 09/30/2016
• Issued: 02/19/2019
• Est. Priority Date: 09/30/2016
• Status: Active Grant

• Current Assignee: Sony Interactive Entertainment Inc. (Sony Group Corp.)
• Original Assignee: Sony Interactive Entertainment Inc. (Sony Group Corp.)
• Inventors: Tokubo, Todd
• Primary Examiner(s): Chang, Kent W
• Assistant Examiner(s): Brittingham, Nathaniel P
• Application Number:
  US15/283,138
• Publication Number:
  US 20180095529A1
• Time in Patent Office: 872 Days
• CPC Class Codes: A63F 13/211 using inertial sensors, e.g. accelerometers or gyroscopes
• CPC Class Codes: A63F 13/212 using sensors worn by the player, e.g. for measuring heart beat or leg activity
• CPC Class Codes: A63F 13/213 comprising photodetecting means, e.g. cameras, photodiodes or infrared cells A63F13/219, A63F13/655 take precedence
• CPC Class Codes: A63F 13/235 using a wireless connection, e.g. infrared or piconet
• CPC Class Codes: A63F 13/25 Output arrangements for video game devices
• CPC Class Codes: A63F 13/327 using wireless networks, e.g. Wi-Fi® or piconet
• CPC Class Codes: A63F 13/332 using wireless networks, e.g. cellular phone networks
• CPC Class Codes: A63F 13/5255 according to dedicated instructions from a player, e.g. using a secondary joystick to rotate the camera around a player's character
• CPC Class Codes: A63F 13/53 involving additional visual information provided to the game scene, e.g. by overlay to simulate a head-up display [HUD] or displaying a laser sight in a shooting game
• CPC Class Codes: A63F 2300/8082 Virtual reality
• CPC Class Codes: G02B 2027/0138 comprising image capture systems, e.g. camera
• CPC Class Codes: G02B 2027/014 comprising information/image processing systems
• CPC Class Codes: G02B 2027/0187 slaved to motion of at least a part of the body of the user, e.g. head, eye
• CPC Class Codes: G02B 27/017 Head mounted
• CPC Class Codes: G02B 27/0172 characterised by optical features
• CPC Class Codes: G06F 2203/0384 Wireless input, i.e. hardware and software details of wireless interface arrangements for pointing devices
• CPC Class Codes: G06F 3/011 Arrangements for interaction with the human body, e.g. for user immersion in virtual reality blind teaching G09B21/00
• CPC Class Codes: G06F 3/012 Head tracking input arrangements
• CPC Class Codes: G06F 3/0304 Detection arrangements using opto-electronic means constructional details of pointing devices not related to the detection arrangement using opto-electronic means G06F3/033; optical digitisers G06F3/042
• CPC Class Codes: H04B 7/04 using two or more spaced independent antennas
• CPC Class Codes: H04B 7/0617 : for beam forming

Abstract

A method is provided, including the following method operations: receiving captured images of an interactive environment in which a head-mounted display (HMD) is disposed; receiving inertial data processed from at least one inertial sensor of the HMD; analyzing the captured images and the inertial data to determine a predicted future location of the HMD; using the predicted future location of the HMD to adjust a beamforming direction of an RF transceiver towards the predicted future location of the HMD.

Claims

1. A method, comprising:

receiving captured image frames of an interactive environment in which a head-mounted display (HMD) is disposed;

receiving inertial data processed from at least one inertial sensor of the HMD;

analyzing the captured image frames and the inertial data to determine a predicted future location of the HMD, wherein analyzing the captured image frames and the inertial data includes identifying movement of the HMD, the predicted future location of the HMD being determined using the identified movement of the HMD, wherein analyzing the captured image frames includes examining changes within a sequence of said captured image frames to track the movement of the HMD, wherein analyzing the captured image frames includes processing the captured image frames to identify and map one or more anchors in the interactive environment and tracking the one or more anchors to enable the tracking of the motion of the HMD, the one or more anchors including one or more fixed objects in the interactive environment;

using the predicted future location of the HMD to adjust a beamforming direction of an RF transceiver towards the predicted future location of the HMD;

adjusting an angular spread of the RF transceiver based on a speed of a movement of the HMD, wherein the angular spread increases with increasing speed of the movement of the HMD.

2. The method of claim 1, wherein identifying the movement of the HMD includes determining a motion vector of the HMD, the predicted future location of the HMD being determined by applying the motion vector of the HMD to a current location of the HMD.

3. The method of claim 2, wherein a magnitude of the motion vector identifies the speed of the movement of the HMD, and wherein a direction of the motion vector identifies a direction of the movement of the HMD.

4. The method of claim 2,

wherein identifying the movement of the HMD includes identifying translational movement and/or rotational movement of the HMD;

wherein determining the motion vector includes determining an acceleration of the translational movement and/or the rotational movement.

5. The method of claim 1,

wherein the RF transceiver includes a phased array of RF emitters;

wherein adjusting the beamforming direction of the RF transceiver includes generating transceiver control data that is configured to cause adjustment of a phase or amplitude of at least one of the RF emitters of the phased array.

6. The method of claim 1, wherein the at least one inertial sensor of the HMD includes one or more of an accelerometer, a gyroscope, or a magnetometer.

7. The method of claim 1,

wherein the HMD includes a plurality of lights;

wherein analyzing the captured image frames includes identifying one or more of the plurality of lights in the captured image frames.

8. A system, comprising:

a head-mounted display (HMD), the HMD having at least one inertial sensor configured to generate inertial data;

a camera configured to capture image frames of an interactive environment in which the HMD is disposed;

an RF transceiver;

a computer configured to analyze the captured image frames and the inertial data to determine a predicted future location of the HMD, and use the predicted future location of the HMD to adjust a beamforming direction of the RF transceiver towards the predicted future location of the HMD, wherein analyzing the captured image frames and the inertial data includes identifying movement of the HMD, the predicted future location of the HMD being determined using the identified movement of the HMD, wherein analyzing the captured image frames includes examining changes within a sequence of said captured image frames to track the movement of the HMD, wherein analyzing the captured image frames includes processing the captured image frames to identify and map one or more anchors in the interactive environment and tracking the one or more anchors to enable the tracking of the motion of the HMD, the anchors including one or more fixed objects in the interactive environment;

wherein the computer is further configured to adjust an angular spread of the RF transceiver based on a radial distance of the HMD from the RF transceiver.

9. The system of claim 8, wherein identifying movement of the HMD includes determining a motion vector of the HMD, the predicted future location of the HMD being determined by applying the motion vector of the HMD to a current location of the HMD.

10. The system of claim 9, wherein a magnitude of the motion vector identifies a speed of the movement of the HMD, and wherein a direction of the motion vector identifies a direction of the movement of the HMD.

11. The system of claim 10, wherein the computer is further configured to adjust an angular spread of the RF transceiver based on the speed of the movement of the HMD.

12. The system of claim 11, wherein the angular spread increases with increasing speed of the movement of the HMD.

13. The system of claim 9,

wherein identifying movement of the HMD includes identifying translational movement and/or rotational movement of the HMD;

wherein determining the motion vector includes determining an acceleration of the translational movement and/or the rotational movement.

14. The system of claim 8,

wherein the RF transceiver includes a phased array of RF emitters;

wherein adjusting the beamforming direction of the RF transceiver includes generating transceiver control data that is configured to cause adjustment of a phase or amplitude of at least one of the RF emitters of the phased array.

15. The system of claim 8, wherein the at least one inertial sensor of the HMD includes one or more of an accelerometer, a gyroscope, or a magnetometer.

16. The system of claim 8,

wherein the HMD includes a plurality of lights;

wherein analyzing the captured image frames includes identifying one or more of the plurality of lights in the captured image frames.

17. A method, comprising:

receiving captured image frames of an interactive environment in which a head-mounted display (HMD) is disposed;

receiving inertial data processed from at least one inertial sensor of the HMD;

analyzing the captured image frames and the inertial data to determine a predicted future location of the HMD, wherein analyzing the captured image frames and the inertial data includes identifying movement of the HMD, the predicted future location of the HMD being determined using the identified movement of the HMD, wherein analyzing the captured image frames includes examining changes within a sequence of said captured image frames to track the movement of the HMD, wherein analyzing the captured image frames includes processing the captured image frames to identify and map one or more anchors in the interactive environment and tracking the one or more anchors to enable the tracking of the motion of the HMD, the one or more anchors including one or more fixed objects in the interactive environment;

using the predicted future location of the HMD to adjust a beamforming direction of an RF transceiver towards the predicted future location of the HMD;

adjusting an angular spread of the RF transceiver based on a transmission data rate of the RF transceiver.