AUGMENTED REALITY WITH MOTION SENSING

• US 20160044298A1
• Filed: 08/07/2015
• Published: 02/11/2016
• Est. Priority Date: 08/08/2014
• Status: Active Grant

• Current Assignee: Lmi Liquidating Co., LLC
• Original Assignee: Leap Motion Incorporated (Ultraleap Limited)
• Inventors: Holz, David S., Roy, Neeloy, He, Hongyuan
• Granted Patent: US 10,349,036 B2
• Time in Patent Office: Days
• US Class Current: 1/1
• CPC Class Codes: G02B 2027/0134 of stereoscopic type
• CPC Class Codes: G02B 2027/0138 comprising image capture systems, e.g. camera
• CPC Class Codes: G02B 2027/0187 slaved to motion of at least a part of the body of the user, e.g. head, eye
• CPC Class Codes: G02B 27/0093 with means for monitoring data relating to the user, e.g. head-tracking, eye-tracking
• CPC Class Codes: G02B 27/017 Head mounted
• CPC Class Codes: G06F 3/011 Arrangements for interaction with the human body, e.g. for user immersion in virtual reality blind teaching G09B21/00
• CPC Class Codes: G06F 3/017 Gesture based interaction, e.g. based on a set of recognized hand gestures interaction based on gestures traced on a digitiser G06F3/04883
• CPC Class Codes: G06T 19/006 Mixed reality object pose determination, tracking or camera calibration for mixed reality G06T7/00
• CPC Class Codes: H04N 13/239 using two 2D image sensors having a relative position equal to or related to the interocular distance H04N13/243 takes precedence
• CPC Class Codes: H04N 13/257 Colour aspects
• CPC Class Codes: H04N 13/296 Synchronisation thereof; Control thereof
• CPC Class Codes: H04N 23/11 for generating image signals from visible and infrared light wavelengths
• CPC Class Codes: H04N 23/54 Mounting of pick-up tubes, electronic image sensors, deviation or focusing coils
• CPC Class Codes: H04N 23/56 provided with illuminating means
• CPC Class Codes: H04S 2400/11 Positioning of individual sound objects, e.g. moving airplane, within a sound field H04S2420/13 takes precedence
• CPC Class Codes: H04S 2400/15 Aspects of sound capture and related signal processing for recording or reproduction
• CPC Class Codes: H04S 7/304 For headphones

Abstract

The technology disclosed relates to a motion sensory and imaging device capable of acquiring imaging information of the scene and providing at least a near real time pass-through of imaging information to a user. The sensory and imaging device can be used stand-alone or coupled to a wearable or portable device to create a wearable sensory system capable of presenting to the wearer the imaging information augmented with virtualized or created presentations of information.

Claims

1. A motion sensory and imaging device, including:

a plurality of imaging sensors arranged to provide stereoscopic imaging information for a scene being viewed;

one or more illumination sources arranged about the imaging sensors; and

a controller coupled to the imaging sensors and illumination sources to control operation thereof, acquiring imaging information of a scene, providing at least a near real time pass-through of imaging information to a user.

2. The device of claim 1, wherein the controller further provides:

capturing imaging information for control objects within the viewpoint of the imaging sensors;

wherein the imaging information for control objects of interest is used to determine gesture information indicating a command to a machine under control.

3. The device of claim 2, wherein the capturing further includes:

separating information received from pixels sensitive to IR light from information received from pixels sensitive to visible light (e.g., RGB);

processing image information from IR sensors to be used for gesture recognition; and

processing image information from RGB sensors to be provided as a live video feed via a presentation interface.

4. The device of claim 3, wherein the processing image information from RGB sensors further includes:

extracting gross features of a real world space using RGB pixels that respectively capture red, green, and blue components of illumination in the scene.

5. The device of claim 3, wherein the processing image information from IR sensors further includes:

extracting fine features of a real world space using IR pixels that capture infrared components of illumination in the scene.

6. The device of claim 5, wherein fine features of the real world space include surface texture of the real world space.

7. The device of claim 5, wherein fine features of the real world space include edges of the real world space.

8. The device of claim 5, wherein fine features of the real world space include curvatures of the real world space.

9. The device of claim 5, wherein fine features of the real world space include surface texture of objects in the real world space.

10. The device of claim 5, wherein fine features of the real world space include edges of objects in the real world space.

11. The device of claim 5, wherein fine features of the real world space include curvatures of objects in the real world space.

12. The device of claim 1, wherein the controller further provides:

determining ambient lighting conditions; and

adjusting display of output based upon the determined conditions.

determine first and second positional information of the sensor with respect to a fixed point at first and second times.

13. The device of claim 1, further including:

motion sensor; and

wherein the controller further provides;

determining difference information between a first and a second positional information from the motion sensor; and

computing movement information for the device with respect to a fixed point based upon the difference information.

14. The device of claim 1, further including:

one or more fasteners that fasten the imaging sensors and the illumination sources to a mounting surface in a wearable presentation device.

15. The device of claim 1, further including:

one or more fasteners that fasten the imaging sensors and the illumination sources to a cavity in a wearable presentation device.

16. The device of claim 1, further including:

one or more fasteners that fasten the imaging sensors and the illumination sources to a mounting surface in a portable presentation device.

17. The device of claim 1, further including:

one or more fasteners that fasten the imaging sensors and the illumination sources to a cavity in a portable presentation device.