AUTOMOTIVE AND INDUSTRIAL MOTION SENSORY DEVICE

• US 20160048725A1
• Filed: 08/13/2015
• Published: 02/18/2016
• Est. Priority Date: 08/15/2014
• Status: Active Grant

• Current Assignee: Lmi Liquidating Co., LLC
• Original Assignee: Leap Motion Incorporated (Ultraleap Limited)
• Inventors: Holz, David S., Roy, Neeloy, Schunick, Justin, Zheng, Chen, Travis, Ward
• Granted Patent: US 11,386,711 B2
• Time in Patent Office: Days
• US Class Current: 1/1
• CPC Class Codes: G01P 13/00 Indicating or recording presence, absence, or direction, of movement electric switches H01H; counting moving objects G06M7/00
• CPC Class Codes: G06F 3/005 Input arrangements through a video camera
• CPC Class Codes: G06F 3/01 Input arrangements or combined input and output arrangements for interaction between user and computer G06F3/16 takes precedence
• CPC Class Codes: G06F 3/011 Arrangements for interaction with the human body, e.g. for user immersion in virtual reality blind teaching G09B21/00
• CPC Class Codes: G06F 3/017 Gesture based interaction, e.g. based on a set of recognized hand gestures interaction based on gestures traced on a digitiser G06F3/04883
• CPC Class Codes: G06F 3/042 by opto-electronic means
• CPC Class Codes: G06T 2207/10012 Stereo images
• CPC Class Codes: G06T 2207/10024 Color image
• CPC Class Codes: G06T 2207/10048 Infrared image
• CPC Class Codes: G06T 2207/30196 Human being; Person
• CPC Class Codes: G06T 2207/30268 Vehicle interior
• CPC Class Codes: G06T 7/20 Analysis of motion motion estimation for coding, decoding, compressing or decompressing digital video signals H04N19/43, H04N19/51
• CPC Class Codes: G06T 7/70 Determining position or orientation of objects or cameras camera calibration G06T7/80
• CPC Class Codes: G06V 20/59 inside of a vehicle, e.g. relating to seat occupancy, driver state or inner lighting conditions
• CPC Class Codes: G06V 40/20 Movements or behaviour, e.g. gesture recognition recognition of facial expressions G06V40/16
• CPC Class Codes: H04N 23/11 for generating image signals from visible and infrared light wavelengths
• CPC Class Codes: H04N 7/181 for receiving images from a plurality of remote sources

Abstract

The technology disclosed relates to highly functional/highly accurate motion sensory control devices for use in automotive and industrial control systems capable of capturing and providing images to motion capture systems that detect gestures in a three dimensional (3D) sensory space.

Claims

1. An embeddable motion sensory control device, including:

a plurality of imaging sensors arranged on a first portion to provide stereoscopic imaging information for a scene being viewed;

one or more illumination sources arranged on a second portion; and

a controller coupled to the imaging sensors and illumination sources to control operation thereof, acquiring imaging information of a scene, and providing at least a near real time stream of the imaging information to a system or device under control.

2. The device of claim 1, wherein the controller further provides:

capturing imaging information for control objects within the viewpoint of the imaging sensors;

wherein the imaging information for control objects of interest is used to determine gesture information indicating a command to a machine under control.

3. The device of claim 2, wherein the capturing further includes:

separating information received from pixels sensitive to IR light from information received from pixels sensitive to visible light (e.g., RGB);

processing image information from IR sensors to be used for gesture recognition; and

processing image information from RGB sensors to be provided as a video feed via a presentation interface.

4. The device of claim 3, wherein the processing image information from RGB sensors further includes:

extracting features of a region of interest using RGB pixels that respectively capture red, green, and blue components of illumination in the scene.

5. The device of claim 3, wherein the processing image information from IR sensors further includes:

extracting features of a region of interest using IR pixels that capture infrared components of illumination in the scene.

6. The device of claim 1, wherein the second portion is selectable from a plurality of configurations, thereby enabling customizing the device to an application.

7. The device of claim 1, further including:

a port to receive information from a photo-detector sensing changes in reflectance indicating presence or absence of objects within a region of space into which the illumination sources emit light.

8. The device of claim of claim 1, further including:

one or more fasteners that fasten the first portion and the second portion to one another.

9. The device of claim 1, further including:

one or more fasteners that fasten the imaging sensors and the illumination sources to a mounting surface of a host device.

10. The device of claim 1, further including:

one or more fasteners that fasten the imaging sensors and the illumination sources to a cavity in a host device.

11. The device of claim 1, further including:

one or more fasteners that fasten the imaging sensors and the illumination sources to a mounting surface of a vehicle.

12. The device of claim 1, further including:

one or more fasteners that fasten the imaging sensors and the illumination sources to a cavity in a vehicle.

13. The device of claim 1, further including:

one or more fasteners that fasten the imaging sensors and the illumination sources to a mounting surface of a portable device interfacing with a vehicle.

14. The device of claim 1, further including:

one or more fasteners that fasten the imaging sensors and the illumination sources to a cavity in a portable device interfacing with a vehicle.

15. The device of claim 1, further including:

a presentation interface providing an augmented reality presentation produced in part from the images captured by the imaging sensors.

16. The device of claim 15, wherein the providing further includes:

augmenting a sequence of images of a scene with computer generated information.

17. The device of claim 16, wherein the augmenting further includes:

adding an icon selected from information about the sequence of images.

18. The device of claim 17, wherein the adding further includes:

selecting an icon representing an advertisement relevant to destination information determined from at least one of the sequence of images and information entered or used by an operator.

19. The device of claim 15, wherein the presentation interface further including:

a heads up display (HUD).

20. The device of claim 1, further including:

a haptic projector to provide sensations to occupants of the vehicle.