RF beamforming for head mounted display

• US 10,514,754 B2
• Filed: 12/28/2018
• Issued: 12/24/2019
• Est. Priority Date: 09/30/2016
• Status: Active Grant

• Current Assignee: Sony Interactive Entertainment Inc. (Sony Group Corp.)
• Original Assignee: Sony Interactive Entertainment Inc. (Sony Group Corp.)
• Inventors: Tokubo, Todd
• Primary Examiner(s): Chang, Kent W
• Assistant Examiner(s): Brittingham, Nathaniel P
• Application Number:
  US16/236,283
• Publication Number:
  US 20190138087A1
• Time in Patent Office: 361 Days
• CPC Class Codes: A63F 13/211 using inertial sensors, e.g. accelerometers or gyroscopes
• CPC Class Codes: A63F 13/212 using sensors worn by the player, e.g. for measuring heart beat or leg activity
• CPC Class Codes: A63F 13/213 comprising photodetecting means, e.g. cameras, photodiodes or infrared cells A63F13/219, A63F13/655 take precedence
• CPC Class Codes: A63F 13/235 using a wireless connection, e.g. infrared or piconet
• CPC Class Codes: A63F 13/25 Output arrangements for video game devices
• CPC Class Codes: A63F 13/327 using wireless networks, e.g. Wi-Fi® or piconet
• CPC Class Codes: A63F 13/332 using wireless networks, e.g. cellular phone networks
• CPC Class Codes: A63F 13/5255 according to dedicated instructions from a player, e.g. using a secondary joystick to rotate the camera around a player's character
• CPC Class Codes: A63F 13/53 involving additional visual information provided to the game scene, e.g. by overlay to simulate a head-up display [HUD] or displaying a laser sight in a shooting game
• CPC Class Codes: A63F 2300/8082 Virtual reality
• CPC Class Codes: G02B 2027/0138 comprising image capture systems, e.g. camera
• CPC Class Codes: G02B 2027/014 comprising information/image processing systems
• CPC Class Codes: G02B 2027/0187 slaved to motion of at least a part of the body of the user, e.g. head, eye
• CPC Class Codes: G02B 27/017 Head mounted
• CPC Class Codes: G02B 27/0172 characterised by optical features
• CPC Class Codes: G06F 2203/0384 Wireless input, i.e. hardware and software details of wireless interface arrangements for pointing devices
• CPC Class Codes: G06F 3/011 Arrangements for interaction with the human body, e.g. for user immersion in virtual reality blind teaching G09B21/00
• CPC Class Codes: G06F 3/012 Head tracking input arrangements
• CPC Class Codes: G06F 3/0304 Detection arrangements using opto-electronic means constructional details of pointing devices not related to the detection arrangement using opto-electronic means G06F3/033; optical digitisers G06F3/042
• CPC Class Codes: H04B 7/04 using two or more spaced independent antennas
• CPC Class Codes: H04B 7/0617 : for beam forming

Abstract

A method is provided, including the following method operations: receiving captured images of an interactive environment in which a head-mounted display (HMD) is disposed; receiving inertial data processed from at least one inertial sensor of the HMD; analyzing the captured images and the inertial data to determine movement of the HMD; using the movement of the HMD to adjust a beamforming direction of an RF transceiver towards the HMD.

Claims

1. A method executed by a computer external to a head-mounted display (HMD), comprising:

receiving, by the computer, captured image frames of an interactive environment in which a HMD is disposed;

receiving inertial data processed from at least one inertial sensor of the HMD;

analyzing the captured image frames and the inertial data to track movement of the HMD and a gaze direction of a user that wears the HMD, wherein analyzing the captured image frames includes examining changes within a sequence of said captured image frames to track the movement of the HMD, wherein analyzing the captured image frames includes processing the captured image frames to identify and map one or more fixed objects in the interactive environment and tracking the one or more fixed objects to enable the tracking of the movement of the HMD;

using the tracked movement of the HMD and the gaze direction to adjust a beamforming direction of an RF transceiver towards the HMD, the RF transceiver being in a region local to the HMD in the interactive environment;

adjusting an angular spread of the RF transceiver based on the movement of the HMD and the gaze direction, wherein the angular spread increases with increasing lateral movement of the HMD relative to the RF transceiver.

2. The method of claim 1, wherein identifying the movement of the HMD includes determining a motion vector of the HMD, the beam forming direction and angular spread being adjusted based on the motion vector of the HMD.

3. The method of claim 2, wherein a magnitude of the motion vector identifies a speed of the movement of the HMD, and wherein a direction of the motion vector identifies a direction of the movement of the HMD.

4. The method of claim 2, wherein determining the motion vector includes determining an acceleration of the movement of the HMD.

5. The method of claim 1,

wherein identifying the movement of the HMD includes identifying translational movement and/or rotational movement of the HMD.

6. The method of claim 1,

wherein the RF transceiver includes a phased array of RF emitters;

wherein adjusting the beamforming direction of the RF transceiver includes generating transceiver control data that is configured to cause adjustment of a phase or amplitude of at least one of the RF emitters of the phased array.

7. The method of claim 1, wherein the at least one inertial sensor of the HMD includes one or more of an accelerometer, a gyroscope, or a magnetometer.

8. The method of claim 1,

wherein the HMD includes a plurality of sensors;

wherein analyzing the captured image frames includes identifying locations of one or more of a plurality of emitters in the captured image frames.

9. The method of claim 1, wherein the captured image frames are captured by one or more outward facing cameras of the HMD.

10. The method of claim 1, further comprising:

receiving feedback data indicating a quality of a signal transmitted by the RF transceiver;

wherein the beamforming direction and angular spread are adjusted based on the feedback data.

11. The method of claim 10, wherein the feedback data is transmitted over a different frequency band than the signal transmitted by the RF transceiver.

12. The method of claim 1, wherein the beamforming direction and angular spread are applied to transmit video data to the HMD, the video data being rendered to a display of the HMD to provide a view of a virtual environment through the HMD.

13. The method of claim 1, wherein the angular spread decreases with increasing radial distance of the HMD relative to the RF transceiver.

14. A method executed by a computer external to a head-mounted display (HMD), comprising:

receiving, by the computer, captured image frames of an interactive environment in which a HMD is disposed;

receiving inertial data processed from at least one inertial sensor of the HMD;

analyzing the captured image frames and the inertial data to track movement of the HMD HMD and a gaze direction of a user that wears the HMD, wherein analyzing the captured image frames includes examining changes within a sequence of said captured image frames to track the movement of the HMD, wherein analyzing the captured image frames includes processing the captured image frames to identify and map one or more fixed objects in the interactive environment and tracking the one or more fixed objects to enable the tracking of the movement of the HMD;

using the tracked movement of the HMD and the gaze direction to adjust a beamforming direction of an RF transceiver towards the HMD, the RF transceiver being in a region local to the HMD in the interactive environment;

adjusting an angular spread of the RF transceiver based on the movement of the HMD and the gaze direction, wherein the angular spread decreases with increasing radial distance of the HMD relative to the RF transceiver.

15. The method of claim 14,

wherein the RF transceiver includes a phased array of RF emitters;

wherein adjusting the beamforming direction of the RF transceiver includes generating transceiver control data that is configured to cause adjustment of a phase or amplitude of at least one of the RF emitters of the phased array.

16. The method of claim 15, wherein the at least one inertial sensor of the HMD includes one or more of an accelerometer, a gyroscope, or a magnetometer.

17. The method of claim 14,

wherein the HMD includes a plurality of emitters;

wherein analyzing the captured image frames includes identifying locations of one or more of the plurality of emitters in the captured image frames.

18. The method of claim 14, wherein the captured image frames are captured by one or more outward facing cameras of the HMD.

19. The method of claim 14, further comprising:

receiving feedback data indicating a quality of a signal transmitted by the RF transceiver;

wherein the beamforming direction and angular spread are adjusted based on the feedback data.

20. The method of claim 14, wherein the feedback data is transmitted over a different frequency band than the signal transmitted by the RF transceiver.