SYSTEM, DEVICE AND METHOD FOR PROVIDING USER INTERFACE FOR A VIRTUAL REALITY ENVIRONMENT

• US 20180068483A1
• Filed: 11/07/2017
• Published: 03/08/2018
• Est. Priority Date: 12/18/2014
• Status: Active Grant

• Current Assignee: META Platforms Technologies LLC (Meta Platforms, Inc. (f/k/a Facebook, Inc.))
• Original Assignee: Meta Platforms, Inc. (f/k/a Facebook, Inc.)
• Inventors: Kaminitz, Kobi, Grossinger, Nadav, Meyassed, Moshe, Romano, Nitay
• Granted Patent: US 10,559,113 B2
• Time in Patent Office: Days
• CPC Class Codes: G06F 3/011 Arrangements for interaction with the human body, e.g. for user immersion in virtual reality blind teaching G09B21/00
• CPC Class Codes: G06F 3/017 Gesture based interaction, e.g. based on a set of recognized hand gestures interaction based on gestures traced on a digitiser G06F3/04883
• CPC Class Codes: G06F 3/0304 Detection arrangements using opto-electronic means constructional details of pointing devices not related to the detection arrangement using opto-electronic means G06F3/033; optical digitisers G06F3/042
• CPC Class Codes: G06T 11/60 Editing figures and text; Combining figures or text
• CPC Class Codes: G06T 15/005 General purpose rendering architectures
• CPC Class Codes: G06T 19/006 Mixed reality object pose determination, tracking or camera calibration for mixed reality G06T7/00
• CPC Class Codes: H04N 23/56 provided with illuminating means
• CPC Class Codes: H04N 5/2226 Determination of depth image, e.g. for foreground/background separation determining depth by image analysis in general G06T7/50; segmentation by image analysis in general G06T7/10
• CPC Class Codes: H04N 5/33 Transforming infrared radiation cameras or camera modules for generating image signals from infrared radiation H04N23/20; circuitry of SSIS for transforming infrared radiation into image signals H04N25/20

Abstract

A device connectable to a near eye display or to a virtual reality headset and to a portable computing platform having a processor is provided herein. The device may include: an illuminator configured to illuminate a proximity of a user wearing the headset or the near eye display with patterned light; and an TR camera configured to capture reflections of said patterned light coming from at least one object located in the proximity of the user, wherein the processor is configured to: establish data and power connection between the device and said poliable computing platform and said near eye display; and generate a depth map of said object based on the reflections.

Claims

1. A device connectable to a portable computing platform, the device comprising:

an illuminator configured to illuminate a scene with structured light;

a camera configured to capture reflections of the structured light coming from at least one object in the scene; and

a pre-processor configured to;

perform pre-processing of data related to the captured reflections, and

communicate the pre-processed data to a processor of the portable computing platform, the pre-processed data sufficient for the portable computing platform to generate a depth map of the at least one object.

2. The device of claim 1, wherein the illuminator and the camera covered by a filter are operating at an infra-red (IR) range.

3. The device of claim 1, wherein the device further comprising:

an interface for connecting the device to the portable computing platform, the interface supplying power to the portable computing platform.

4. The device of claim 1, wherein the portable computing platform is selected from a group comprising:

a smartphone, a laptop personal computer (PC), and a tablet PC.

5. The device of claim 1, wherein the device is further connected to a near eye display for presenting a user with a virtual reality (VR) scene and at least one 3D image of the at least one object obtained based on the depth map.

6. The device of claim 5, wherein the at least one object includes at least one body part of the user.

7. The device of claim 6, wherein the depth map of the at least one object includes information about one or more gestures related to the at least one body part of the user.

8. The device of claim 5, wherein a field of view of the camera includes the scene surrounding the user of the near eye display.

9. The device of claim 1, further comprising:

another camera configured to capture at least one 2D image of the at least one object, wherein

the pre-processor is further configured to;

pre-process information related to the at least one captured 2D image, and

communicate the pre-processed information to the processor, the pre-processed information sufficient for the portable computing platform to generate at least one 3D image of the at least one object based on the depth map and the pre-processed information.

10. The device of claim 1, wherein:

the illuminator includes a light source coupled to at least one optical element, and

the at least one optical element generates the structured light using light emitted from the light source.

11. The device of claim 10, wherein the at least one optical element comprises at least one of:

an aspheric lens, a cylindrical lens, a diffractive element, a beam splitting element, a positive optical surface, and a negative optical surface.

12. The device of claim 1, wherein the structured light includes a light pattern having a continuous feature in a first direction and a non-continuous feature in a second direction perpendicular to the first direction.

13. The device of claim 12, wherein a position of the camera relative to the illuminator is selected to cause the captured reflections of the structured light to have at least one discontinuity along the continuous feature related to at least one depth shift of the at least one object.

14. The device of claim 13, wherein the pre-processor is further configured to:

perform pre-processing of information related to the captured reflections having the at least one discontinuity along the continuous feature, and communicate the pre-processed information to the processor, the pre-processed information sufficient for the portable computing platform to track one or more gestures related to the at least one object.

15. A method comprising:

connecting a device to a portable computing platform, the device comprising an illuminator and a camera;

illuminating a scene with structured light emitted from the illuminator;

capturing, by the camera, reflections of the structured light coming from at least one object in the scene;

pre-processing of data related to the captured reflections;

communicating the pre-processed data to a processor of the portable computing platform, the pre-processed data sufficient for the portable computing platform to generate a depth map of the at least one object.

16. The method of claim 15, further comprising:

providing power from the device to the portable computing platform via an interface connecting the device to the portable computing platform.

17. The method of claim 15, further comprising:

capturing at least one 2D image of the at least one object;

pre-processing information related to the at least one captured 2D image; and

communicating the pre-processed information to the processor, the pre-processed information sufficient for the portable computing platform to generate at least one 3D image of the at least one object based on the depth map and the pre-processed information.

18. The method of claim 15, further comprising:

generating the structured light having a continuous feature in a first direction and a non-continuous feature in a second direction perpendicular to the first direction using at least one optical element of the illuminator, the at least one optical element comprising at least one of an aspheric lens, a cylindrical lens, a diffractive element, a beam splitting element, a positive optical surface, and a negative optical surface of the illuminator.

19. The method of claim 18, further comprising:

pre-processing of information related to the captured reflections having the at least one discontinuity along the continuous feature, and

communicating the pre-processed information to the processor, the pre-processed information sufficient for the portable computing platform to track one or more gestures related to the at least one object.

20. The method of claim 15, wherein the depth map includes information about one or more gestures related to at least one body part of a user wearing a near eye display coupled to the device.