System, device and method for providing user interface for a virtual reality environment

• US 10,559,113 B2
• Filed: 11/07/2017
• Issued: 02/11/2020
• Est. Priority Date: 12/18/2014
• Status: Active Grant

• Current Assignee: META Platforms Technologies LLC (Meta Platforms, Inc. (f/k/a Facebook, Inc.))
• Original Assignee: Facebook Technologies, LLC (Meta Platforms, Inc. (f/k/a Facebook, Inc.))
• Inventors: Kaminitz, Kobi, Grossinger, Nadav, Meyassed, Moshe, Romano, Nitay
• Primary Examiner(s): Johnson, Motilewa Good
• Application Number:
  US15/806,222
• Publication Number:
  US 20180068483A1
• Time in Patent Office: 826 Days
• CPC Class Codes: G06F 3/011 Arrangements for interaction with the human body, e.g. for user immersion in virtual reality blind teaching G09B21/00
• CPC Class Codes: G06F 3/017 Gesture based interaction, e.g. based on a set of recognized hand gestures interaction based on gestures traced on a digitiser G06F3/04883
• CPC Class Codes: G06F 3/0304 Detection arrangements using opto-electronic means constructional details of pointing devices not related to the detection arrangement using opto-electronic means G06F3/033; optical digitisers G06F3/042
• CPC Class Codes: G06T 11/60 Editing figures and text; Combining figures or text
• CPC Class Codes: G06T 15/005 General purpose rendering architectures
• CPC Class Codes: G06T 19/006 Mixed reality object pose determination, tracking or camera calibration for mixed reality G06T7/00
• CPC Class Codes: H04N 23/56 provided with illuminating means
• CPC Class Codes: H04N 5/2226 Determination of depth image, e.g. for foreground/background separation determining depth by image analysis in general G06T7/50; segmentation by image analysis in general G06T7/10
• CPC Class Codes: H04N 5/33 Transforming infrared radiation cameras or camera modules for generating image signals from infrared radiation H04N23/20; circuitry of SSIS for transforming infrared radiation into image signals H04N25/20

Abstract

A device connectable to a near eye display or to a virtual reality headset and to a portable computing platform having a processor is provided herein. The device may include: an illuminator configured to illuminate a proximity of a user wearing the headset or the near eye display with patterned light; and an TR camera configured to capture reflections of said patterned light coming from at least one object located in the proximity of the user, wherein the processor is configured to: establish data and power connection between the device and said poliable computing platform and said near eye display; and generate a depth map of said object based on the reflections.

Claims

1. A device connectable to a portable computing platform, the device comprising:

an illuminator configured to illuminate a scene with structured light;

a camera configured to capture reflections of the structured light coming from at least one object in the scene; and

a pre-processor configured to;

perform initial image processing of data related to the captured reflections from the camera, and

communicate the data after initial image processing to a processor of the portable computing platform to generate a depth map of the at least one object based on the reflections, the device being in communication via the pre-processor with the portable computing platform and a near eye display forming a virtual reality headset.

2. The device of claim 1, wherein the illuminator and the camera are operating at an infra-red (IR) range, the camera being covered by an IR filter.

3. The device of claim 1, wherein the device further comprising:

an interface for connecting the device to the portable computing platform, the interface supplying power to the portable computing platform.

4. The device of claim 1, wherein the portable computing platform is selected from a group comprising:

a smartphone, a laptop personal computer (PC), and a tablet PC.

5. The device of claim 1, wherein the near eye display connected to the device is configured to present a user with a virtual reality (VR) scene and at least one 3D image of the at least one object obtained based on the depth map.

6. The device of claim 5, wherein the at least one object includes at least one body part of the user.

7. The device of claim 5, wherein a field of view of the camera includes the scene surrounding the user of the near eye display.

8. The device of claim 1, wherein:

the illuminator includes a light source coupled to at least one optical element, and

the at least one optical element generates the structured light using light emitted from the light source.

9. The device of claim 8, wherein the at least one optical element comprises at least one of:

an aspheric lens, a cylindrical lens, a diffractive element, a beam splitting element, a positive optical surface, and a negative optical surface.

10. The device of claim 1, wherein the structured light includes a light pattern having a continuous feature in a first direction and a non-continuous feature in a second direction perpendicular to the first direction.

11. The device of claim 10, wherein a position of the camera relative to the illuminator is selected to cause the captured reflections of the structured light to have at least one discontinuity along the continuous feature related to at least one depth shift of the at least one object.

12. A method comprising:

connecting a device to a portable computing platform, the device comprising an illuminator and a camera;

illuminating a scene with structured light emitted from the illuminator;

capturing, by the camera, reflections of the structured light coming from at least one object in the scene;

performing initial image processing of data related to the captured reflections from the camera; and

communicating the data after initial image processing to a processor of the portable computing platform to generate a depth map of the at least one object based on the reflections, the device being in communication via the pre-processor with the portable computing platform and a near eye display forming a virtual reality headset.

13. The method of claim 12, further comprising:

providing power from the device to the portable computing platform via an interface connecting the device to the portable computing platform.

14. The method of claim 12, further comprising:

generating the structured light having a continuous feature in a first direction and a non-continuous feature in a second direction perpendicular to the first direction using at least one optical element of the illuminator, the at least one optical element comprising at least one of an aspheric lens, a cylindrical lens, a diffractive element, a beam splitting element, a positive optical surface, and a negative optical surface of the illuminator.

15. The method of claim 12, wherein the depth map includes information about at least one body part of a user wearing the near eye display coupled to the device.