Three-dimensional imager and projection device

• US 8,570,372 B2
• Filed: 04/27/2012
• Issued: 10/29/2013
• Est. Priority Date: 04/29/2011
• Status: Active Grant

• Current Assignee: Luminar Technologies, Inc.
• Original Assignee: Austin Russell
• Inventors: Russell, Austin
• Primary Examiner(s): Dastouri, Mehrdad
• Assistant Examiner(s): HALLENBECK-HUBER, JEREMIAH CHARLES
• Application Number:
  US13/458,956
• Publication Number:
  US 20120274745A1
• Time in Patent Office: 550 Days
• Field of Search: 348/46, 348/47, 348/136
• US Class Current: 348/136
• CPC Class Codes: G02B 2027/0138 comprising image capture systems, e.g. camera
• CPC Class Codes: G02B 2027/014 comprising information/image processing systems
• CPC Class Codes: G02B 2027/0178 Eyeglass type eyeglass details G02C
• CPC Class Codes: G02B 27/017 Head mounted
• CPC Class Codes: G02C 11/10 Electronic devices other than hearing aids
• CPC Class Codes: G03B 21/26 Projecting separately subsidiary matter simultaneously with main image
• CPC Class Codes: G06F 3/013 Eye tracking input arrangements G06F3/015 takes precedence
• CPC Class Codes: G06F 3/0304 Detection arrangements using opto-electronic means constructional details of pointing devices not related to the detection arrangement using opto-electronic means G06F3/033; optical digitisers G06F3/042
• CPC Class Codes: G06F 3/0346 with detection of the device orientation or free movement in a 3D space, e.g. 3D mice, 6-DOF [six degrees of freedom] pointers using gyroscopes, accelerometers or tilt-sensors
• CPC Class Codes: G09G 3/025 with scanning or deflecting the beams in two directions or dimensions
• CPC Class Codes: H04N 13/25 using two or more image sensors with different characteristics other than in their location or field of view, e.g. having different resolutions or colour pickup characteristics; using image signals from one sensor to control the characteristics of another sensor
• CPC Class Codes: H04N 13/254 in combination with electromagnetic radiation sources for illuminating objects
• CPC Class Codes: H04N 13/271 wherein the generated image signals comprise depth maps or disparity maps

Abstract

The systems and methods described herein include a device that can scan the surrounding environment and construct a 3D image, map, or representation of the surrounding environment using, for example, invisible light projected into the environment. In some implementations, the device can also project into the surrounding environment one or more visible radiation pattern patterns (e.g., a virtual object, text, graphics, images, symbols, color patterns, etc.) that are based at least in part on the 3D map of the surrounding environment.

Claims

1. A device comprising:

one or more cameras, the one or more cameras being configured to detect visible light and invisible light from one or more fields of view, and to create one or more visible light images and one or more invisible light images of the one or more fields of view;

one or more projectors, the one or more projectors being configured to project at least invisible light into at least one of the one or more fields of view; and

one or more processors, the one or more processors being configured to

control the one or more projectors to project one or more invisible light patterns into at least one of the one or more fields of view;

analyze the one or more invisible light images detected by the one or more cameras in order to generate depth data indicative of the depth of at least one item in at least one of the one or more fields of view;

combine the one or more visible light images and the depth data to create three-dimensional data of at least one of the one or more fields of view;

identify an object in the one or more fields of view, wherein the object comprises an electronic device;

identify the location of the object using the three-dimensional data; and

control the one or more projectors to project light into a subset of volumetric space in at least one of the one or more fields of view in order to power the electronic device by wirelessly transferring energy to the electronic device via a beam of light based upon the identified location.

2. The device of claim 1, wherein the one or more cameras and the one or more projectors substantially share an overlapping field of view.

3. The device of claim 2, wherein the one or more projectors are configured to project coherent light.

4. The device of claim 3, wherein the one or more projectors are configured to project a substantially collimated beam of invisible light into the overlapping field of view, and to scan the beam of invisible light across the overlapping field of view to create the invisible light pattern.

5. The device of claim 4, wherein the invisible light comprises infrared light.

6. The device of claim 1, wherein the invisible light comprises terahertz, RF, or microwave light.

7. The device of claim 1, wherein the three-dimensional data comprises a texture mapping of the one or more visible light images onto the depth data.

8. The device of claim 1, wherein the one or more processors are further configured to transmit on/off commands to the electronic device via the beam of light.

9. The device of claim 1, wherein the one or more processors are further configured to transmit data to the electronic device via the beam of light.

10. The device of claim 1, wherein the one or more processors are configured to identify the electronic device or its location by executing object recognition software on the three-dimensional data.

11. The device of claim 1, wherein the one or more processors are configured to identify the electronic device or its location using wireless communication feedback from the electronic device.

12. The device of claim 1, wherein the one or more processors are configured to generate the depth data, at least in part, by comparing the one or more projected invisible light patterns to the one or more detected invisible light images.

13. The device of claim 1, wherein the same one or more projectors are configured to project the invisible light patterns and the beam of light used to wirelessly transfer energy to the electronic device.

14. A method comprising:

detecting visible light and invisible light from one or more fields of view, and creating one or more visible light images and one or more invisible light images of the one or more fields of view using one or more cameras;

projecting at least invisible light into at least one of the one or more fields of view using one or more projectors; and

performing processing with one or more processors to

control the one or more projectors to project one or more invisible light patterns into at least one of the one or more fields of view;

analyze the one or more invisible light images detected by the one or more cameras in order to generate depth data indicative of the depth of at least one item in at least one of the one or more fields of view;

combine the one or more visible light images and the depth data to create three-dimensional data of at least one of the one or more fields of view;

identify an object in the one or more fields of view, wherein the object comprises an electronic device;

identify the location of the object using the three-dimensional data, and

control the one or more projectors to project light into a subset of volumetric space in at least one of the one or more fields of view in order to power the electronic device by wirelessly transferring energy to the electronic device via a beam of light based upon the identified location.

15. The method of claim 14, wherein the one or more cameras and the one or more projectors substantially share an overlapping field of view.

16. The method of claim 15, wherein the one or more projectors are configured to project coherent light.

17. The method of claim 16, wherein the one or more projectors are configured to project a substantially collimated beam of invisible light into the overlapping field of view, and to scan the beam of invisible light across the overlapping field of view to create the invisible light pattern.

18. The method of claim 17, wherein the invisible light comprises infrared light.

19. The method of claim 14, wherein the invisible light comprises terahertz, RF, or microwave light.

20. The method of claim 14, wherein the three-dimensional data comprises a texture mapping of the one or more visible light images onto the depth data.

21. The method of claim 14, wherein the one or more processors are further configured to transmit on/off commands to the electronic device via the beam of light.

22. The method of claim 14, wherein the one or more processors are further configured to transmit data to the electronic device via the beam of light.

23. The method of claim 14, wherein the one or more processors are configured to identify the electronic device or its location by executing object recognition software on the three-dimensional data.

24. The method of claim 14, wherein the one or more processors are configured to identify the electronic device or its location using wireless communication feedback from the electronic device.

25. The method of claim 14, wherein the one or more processors are configured to generate the depth data, at least in part, by comparing the one or more projected invisible light patterns to the one or more detected invisible light images.

26. The method of claim 14, wherein the same one or more projectors are configured to project the invisible light patterns and the beam of light used to wirelessly transfer energy to the electronic device.