Virtual reality system with head-mounted display, camera and hand-held controllers

• US 9,898,091 B2
• Filed: 06/03/2015
• Issued: 02/20/2018
• Est. Priority Date: 06/03/2015
• Status: Active Grant

• Current Assignee: META Platforms Technologies LLC (Meta Platforms, Inc. (f/k/a Facebook, Inc.))
• Original Assignee: Oculus VR, LLC (Meta Platforms, Inc. (f/k/a Facebook, Inc.))
• Inventors: Bristol, Peter Wesley, Chen, Yi-yaun, Higgins, Jason Andrew, Rogoza, Benjamin E. Tunberg, Talati, Sharvil Shailesh, Konzen, Neil Warren, Webb, James S.
• Primary Examiner(s): Sasinowski, Andrew
• Application Number:
  US14/729,954
• Publication Number:
  US 20160357261A1
• Time in Patent Office: 993 Days
• CPC Class Codes: A63F 13/00 Video games, i.e. games using an electronically generated display having two or more dimensions gaming systems which provide a financial reward G07F17/32
• CPC Class Codes: A63F 13/212 using sensors worn by the player, e.g. for measuring heart beat or leg activity
• CPC Class Codes: A63F 13/213 comprising photodetecting means, e.g. cameras, photodiodes or infrared cells A63F13/219, A63F13/655 take precedence
• CPC Class Codes: A63F 13/214 for locating contacts on a surface, e.g. floor mats or touch pads
• CPC Class Codes: A63F 13/24 Constructional details thereof, e.g. game controllers with detachable joystick handles
• CPC Class Codes: G02B 2027/0138 comprising image capture systems, e.g. camera
• CPC Class Codes: G02B 2027/0178 Eyeglass type eyeglass details G02C
• CPC Class Codes: G02B 2027/0187 slaved to motion of at least a part of the body of the user, e.g. head, eye
• CPC Class Codes: G02B 27/017 Head mounted
• CPC Class Codes: G06F 1/163 Wearable computers, e.g. on a belt
• CPC Class Codes: G06F 1/1686 the I/O peripheral being an integrated camera
• CPC Class Codes: G06F 3/011 Arrangements for interaction with the human body, e.g. for user immersion in virtual reality blind teaching G09B21/00
• CPC Class Codes: G06F 3/014 Hand-worn input/output arrangements, e.g. data gloves
• CPC Class Codes: G06F 3/017 Gesture based interaction, e.g. based on a set of recognized hand gestures interaction based on gestures traced on a digitiser G06F3/04883
• CPC Class Codes: G06F 3/0219 Special purpose keyboards
• CPC Class Codes: G06F 3/0304 Detection arrangements using opto-electronic means constructional details of pointing devices not related to the detection arrangement using opto-electronic means G06F3/033; optical digitisers G06F3/042
• CPC Class Codes: G06F 3/0325 using a plurality of light emitters or reflectors or a plurality of detectors forming a reference frame from which to derive the orientation of the object, e.g. by triangulation or on the basis of reference deformation in the picked up image
• CPC Class Codes: G06F 3/0346 with detection of the device orientation or free movement in a 3D space, e.g. 3D mice, 6-DOF [six degrees of freedom] pointers using gyroscopes, accelerometers or tilt-sensors

Abstract

A virtual-reality system includes a head-mounted display (HMD), a forward-looking camera coupled to the HMD and a hand-held controller communicatively coupleable to the HMD. The hand-held controller includes a first user-input key, a grip, and an outward-facing surface coupled to the grip. The outward-facing surface includes a plurality of illumination sources which provide light that is detectable by the camera for sensing a position of the controller based on a user motion.

Claims

1. A virtual-reality system comprising:

a head-mounted display (HMD);

a forward-looking camera coupled to the HIVID; and

a hand-held controller communicatively coupleable to the HIVID, the hand-held controller comprising;

a first user-input key;

a grip;

an outward-facing surface coupled to the grip; and

a plurality of illumination sources coupled to the outward-facing surface and configured to provide light that is detectable by the camera.

2. The virtual-reality system of claim 1, wherein the forward-looking camera extends from a front surface of the HMD.

3. The virtual-reality system of claim 1, wherein:

the hand-held controller further comprises a cage coupled to the grip;

the outward-facing surface comprises an outer surface of the cage; and

the plurality of illumination sources are coupled to the outer surface of the cage.

4. The virtual-reality system of claim 3, wherein at least a portion of the plurality of illumination sources are positioned to be detectable to the forward-looking camera when the HIVID is worn by a user and the user holds the grip in a neutral position.

5. The virtual-reality system of claim 4, wherein the cage is configured to be positioned above a user'"'"'s hand when the user holds the grip in the neutral position.

6. The virtual-reality system of claim 3, wherein the hand-held controller further comprises a user-input surface that includes the first user-input key.

7. The virtual-reality system of claim 6, further comprising a structural web coupling the cage to the user-input surface.

8. The virtual-reality system of claim 7, further comprising a second user-input key, wherein the second user-input key is a trigger mounted on at least one of the structural web and the grip at a position configured to be actuated by a middle finger of the user.

9. The virtual-reality system of claim 6, wherein the user-input surface forms an inner front surface of the cage.

10. The virtual-reality system of claim 6, wherein the grip is slanted at an angle with respect to the user-input surface.

11. The virtual-reality system of claim 6, wherein:

the user-input surface comprises a plurality of user-input keys including the first user-input key, and

the user-input surface comprises a touch-sensitive surface partitioned into a plurality of sections, each section corresponding to a respective user-input key of the plurality of user-input keys and including at least one touch sensor to detect a touch on the corresponding section.

12. The virtual-reality system of claim 3, wherein the grip is integrally formed with the cage.

13. The virtual-reality system of claim 3, wherein the cage is detachably coupled to the grip.

14. The virtual-reality system of claim 1, wherein:

the plurality of illumination sources comprises a plurality of light-emitting diodes (LEDs); and

the forward-looking camera is configured to track light emitted by the LEDs.

15. The virtual-reality system of claim 14, wherein:

the plurality of LEDs comprises a plurality of infrared LEDs; and

the forward-looking camera is configured to detect infrared light emitted by the infrared LEDs.

16. The virtual-reality system of claim 14, further comprising:

a first power source to supply power to the HMD and the forward-looking camera; and

a second power source to supply power to the hand-held controller, including to the plurality of LEDs.

17. The virtual-reality system of claim 1, wherein:

the plurality of illumination sources comprises a plurality of passive reflectors; and

the camera comprises an illumination source to provide light to the passive reflectors and a sensor to detect light reflected back to the camera by the passive reflectors.

18. The virtual-reality system of claim 1, wherein the first user-input key is selected from the group consisting of a thumbstick, a button, a trigger, and a directional pad, and the button is selected from the group consisting of an A or X button, a B or Y button, a start button, a back button, a forward button, and a home button.

19. The virtual-reality system of claim 1, wherein the forward-looking camera is external to and electrically coupleable to the HMD.

20. A virtual-reality system comprising:

a head-mounted display (HMD);

a forward-looking camera coupled to the HMD; and

two hand-held controllers, each communicatively coupleable to the HMD, and each comprising;

a respective user-input key;

a respective grip;

a respective outward-facing surface coupled to the respective grip; and

a respective plurality of illumination sources coupled to the respective outward-facing surface and configured to provide light that is detectable by the camera.