THREE-DIMENSIONAL IMAGER AND PROJECTION DEVICE

• US 20130215235A1
• Filed: 03/15/2013
• Published: 08/22/2013
• Est. Priority Date: 04/29/2011
• Status: Active Grant

• Current Assignee: Luminar Technologies, Inc.
• Original Assignee: Austin Russell
• Inventors: Russell, Austin
• Granted Patent: US 8,760,499 B2
• Time in Patent Office: Days
• US Class Current: 348/47
• CPC Class Codes: G01B 11/25 by projecting a pattern, e.g. one or more lines, moiré fringes on the object G01B11/255 takes precedence ; image analysis for depth or shape recovery G06T7/50
• CPC Class Codes: G01B 11/2545 with one projection direction and several detection directions, e.g. stereo
• CPC Class Codes: G03B 17/54 with projector
• CPC Class Codes: G03B 35/00 Stereoscopic photography
• CPC Class Codes: G06T 19/006 Mixed reality object pose determination, tracking or camera calibration for mixed reality G06T7/00
• CPC Class Codes: H02J 50/30 using light, e.g. lasers
• CPC Class Codes: H04N 13/204 using stereoscopic image cameras stereoscopic photography G03B35/00
• CPC Class Codes: H04N 13/207 using a single 2D image sensor
• CPC Class Codes: H04N 13/243 using three or more 2D image sensors
• CPC Class Codes: H04N 13/25 using two or more image sensors with different characteristics other than in their location or field of view, e.g. having different resolutions or colour pickup characteristics; using image signals from one sensor to control the characteristics of another sensor
• CPC Class Codes: H04N 13/254 in combination with electromagnetic radiation sources for illuminating objects
• CPC Class Codes: H04N 13/257 Colour aspects
• CPC Class Codes: H04N 13/271 wherein the generated image signals comprise depth maps or disparity maps
• CPC Class Codes: H04N 13/296 Synchronisation thereof; Control thereof
• CPC Class Codes: H04N 13/363 using image projection screens volumetric display H04N13/388
• CPC Class Codes: H04N 2013/0081 Depth or disparity estimation from stereoscopic image signals

Abstract

The systems and methods described herein include a device that can scan the surrounding environment and construct a 3D image, map, or representation of the surrounding environment using, for example, invisible light projected into the environment. In some implementations, the device can also project into the surrounding environment one or more visible radiation pattern patterns (e.g., a virtual object, text, graphics, images, symbols, color patterns, etc.) that are based at least in part on the 3D map of the surrounding environment.

Claims

1. A system for remotely powering an electronic device, the system comprising:

one or more sensors, the one or more sensors being configured to detect at least visible or invisible light from a field of view;

one or more projectors, the one or more projectors being configured to project at least invisible light into the field of view; and

one or more processors configured to

control the one or more projectors to project at least one or more invisible light patterns into the field of view;

analyze the invisible light detected by the one or more sensors in order to identify the location of an electronic device within the field of view; and

control the one or more projectors to project a beam of electromagnetic radiation toward the identified location in the field of view in order to power the electronic device by wirelessly transferring energy to the electronic device.

2. The system of claim 1, wherein the electronic device comprises one or more photodiodes to receive the beam of electromagnetic radiation and to produce electricity from the beam of electromagnetic radiation.

3. The system of claim 1, wherein the one or more projectors are further configured to transmit data to the electronic device by modulating the beam of electromagnetic radiation.

4. The system of claim 3, wherein the one or more projectors are configured to transmit data via spatial, temporal, spectral, polarization, and/or output power modulation of the beam of electromagnetic radiation.

5. The system of claim 3, wherein the electronic device receives data by using a photodiode or an array of photodiodes.

6. The system of claim 1, wherein the one or more processors are configured to analyze the visible or invisible light detected by the one or more sensors in order to receive data from the electronic device.

7. The system of claim 1, wherein the one or more processors are configured to analyze the invisible light in order to generate depth data used to identify the location of the electronic device in the field of view.

8. The system of claim 1, wherein the location of the electronic device is identified at least in part by establishing a data connection with the electronic device, and receiving location information from the electronic device.

9. The system of claim 1, wherein the remote device comprises an array of one or more photodetectors optimized to the spectrum of the beam of electromagnetic radiation.

10. The system of claim 1, wherein the one or more sensors comprise one or more cameras configured to create one or more visible or invisible light images of the one or more fields of view.

11. A device comprising:

one or more sensors, the one or more sensors being configured to detect light in two or more spectral bands, including light outside of the visible spectrum, from one or more fields of view, and to create one or more multispectral images of the one or more fields of view;

one or more projectors, the one or more projectors being configured to project a light pattern into at least one of the one or more fields of view; and

one or more processors configured to analyze the data in at least one of the spectral bands detected by the one or more sensors in order to generate depth data indicative of the depth of at least one item in at least one of the one or more fields of view; and

combine the one or more multispectral images and the depth data to create multi-dimensional data of at least one of the one or more fields of view.

12. The device of claim 11, wherein the one or more processors are configured to control the one or more projectors to project light into a subset of volumetric space in at least one of the one or more fields of view, the light being projected to a location designated by the one or more processors.

13. The device of claim 11, wherein the device is configured to create one or more hyperspectral images of the one or more fields of view.

14. The device of claim 11, wherein the one or more processors are configured to identify an item in the one or more fields of view based on its spectral signature.

15. The device of claim 11, wherein the device comprises a multispectral time of flight imaging system.

16. A 3D movie camera system comprising:

one or more cameras, the one or more cameras being configured to detect visible light and invisible light from a field of view, and to create visible light images and invisible light images of the field of view;

one or more projectors, the one or more projectors being configured to project invisible light into the field of view; and

one or more processors configured to control the one or more projectors to project at least one or more invisible light patterns into the field of view;

analyze the invisible light images detected by the one or more cameras in order to generate depth data indicative of the depth of at least one item in the field of view;

combine the visible light images and the depth data to create three-dimensional data of the field of view; and

execute a processing function on the three-dimensional data, the processing function being selected from the group comprising virtually moving the camera location, distinguishing the item in the field of view based on the depth data, adding a virtual light source, changing the depth of focus of the visible light images, three-dimensional content-aware fill, or creating a stereoscopic image of the field of view using at least two virtual cameras.

17. The 3D movie camera system of claim 16, wherein the one or more projectors configured to project invisible light comprise a removable accessory.

18. The 3D movie camera system of claim 16, wherein the one or more cameras are configured to create invisible light images of the field of view by using image sensor pixels corresponding to red filtering elements of a Bayer filter.

19. The 3D movie camera system of claim 16, wherein the one or more cameras are configured to create visible light images of the field of view by using image sensor pixels corresponding to green filtering elements of a Bayer filter to obtain luminance information and chrominance information.

20. A device comprising:

one or more cameras, the one or more cameras being configured to detect visible light and invisible light from one or more fields of view, and to create one or more visible light images and one or more invisible light images of the one or more fields of view;

one or more projectors, the one or more projectors being configured to project at least invisible light into at least one of the one or more fields of view; and

one or more processors, the one or more processors being configured to control the one or more projectors to project at least one or more invisible light patterns into at least one of the one or more fields of view;

analyze the one or more invisible light images detected by the one or more cameras in order to generate depth data indicative of the depth of at least one item in at least one of the one or more fields of view; and

combine the one or more visible light images and the depth data to create three-dimensional data of at least one of the one or more fields of view, wherein the one or more invisible light patterns are dynamically adjusted so as to increase the resolution of the depth data in at least one portion of the field of view.

21. The device of claim 20, wherein the one or more invisible light patterns are dynamically adjusted so as to increase resolution of the depth data in a portion of the field of view where the depth data, or the rate of change of the depth data, exceeds a threshold.