Apparatus for real-time 3D capture

• US 9,438,775 B2
• Filed: 09/17/2013
• Issued: 09/06/2016
• Est. Priority Date: 09/17/2013
• Status: Active Grant

• Current Assignee: Xrpro
• Original Assignee: Occipital, Inc.
• Inventors: Heller, Forrest, Powers, Jeffrey, Reddy, Vikas, Orsan, Mahmut Candemir, O'Keefe, Patrick, Chen, YunJa
• Primary Examiner(s): Anderson, II, James M
• Application Number:
  US14/028,868
• Publication Number:
  US 20150077517A1
• Time in Patent Office: 1,085 Days
• Field of Search: None
• US Class Current: 1/1
• CPC Class Codes: H04N 13/207 using a single 2D image sensor
• CPC Class Codes: H04N 13/254 in combination with electromagnetic radiation sources for illuminating objects
• CPC Class Codes: H04N 13/271 wherein the generated image signals comprise depth maps or disparity maps
• CPC Class Codes: H04N 2213/001 Constructional or mechanical details
• CPC Class Codes: H04N 23/54 Mounting of pick-up tubes, electronic image sensors, deviation or focusing coils
• CPC Class Codes: H04N 23/55 Optical parts specially adapted for electronic image sensors; Mounting thereof

Abstract

One variation of a real-time 3D capture system for a mobile electronic device having a camera includes an infrared projector that projects a pattern onto an imaging target; an infrared sensor that captures the pattern; a control module that controls the projector and sensor, takes data from the sensor, determines depth information from the data, and transmits the depth information to the mobile electronic device; a battery that provides power to the projector, sensor, and control module; a software module connected to the mobile electronic device that controls communication of data from the camera and depth information between the control module and the mobile electronic device; a mounting bracket that removably attaches the apparatus to the mobile electronic device such that the capture system when attached maintains alignment with the camera; and a chassis that holds the projector, sensor, control module, and battery, and attaches to the mounting bracket.

Claims

1. A real-time 3D capture system for a mobile electronic device having a camera, the capture system comprising:

an infrared projector that projects a pattern onto an imaging target;

an infrared sensor that captures the pattern;

a control module that controls the projector and sensor, takes data from the sensor, determines depth information from the data, and transmits the depth information to the mobile electronic device;

a battery that provides power to the projector, sensor, and control module;

a software module connected to the mobile electronic device that controls communication between the control module and the mobile electronic device, wherein communication comprises data from the camera and the depth information;

wherein the software module further comprises a calibration module that enables correction for rotation and translation of the infrared sensor relative to the camera for registering the depth information with the output from the camera;

a mounting bracket that removably couples the capture system to the mobile electronic device such that the capture system when coupled maintains alignment with the camera; and

a chassis that holds the projector, sensor, control module, and battery, and couples to the mounting bracket, wherein the projector, sensor, and control module are fixed to the chassis.

2. The system of claim 1 further comprising two infrared light-emitting diodes.

3. The system of claim 2 wherein the infrared sensor is a CMOS sensor with a bandpass filter having an infrared passband and the infrared light-emitting diodes emit light within the passband.

4. The system of claim 1 further comprising a wide-angle lens coupled to the mounting bracket such that the wide-angle lens covers the camera of the mobile electronic device when the mounting bracket is coupled to the mobile electronic device.

5. The system of claim 4 wherein the wide-angle lens is fixed to the mounting bracket.

6. The system of claim 1 further comprising a microphone.

7. The system of claim 1 further comprising a mechanism for rotating the infrared sensor.

8. The system of claim 1 wherein the mounting bracket is rigid and attaches to at least two corners of the mobile electronic device.

9. The system of claim 8 further comprising a layer of thermoplastic polyurethane on the surface of the mounting bracket contacting the mobile electronic device.

10. The system of claim 1 further comprising an insulating backplate coupled to the chassis and to the mobile electronic device and having a thermal conductivity of less than 5 W/mK;

wherein the chassis has a thermal conductivity of greater than 150 W/mK.

11. The system of claim 1 further comprising a thermoelectric cooler coupled to the infrared projector on one side of the cooler and the chassis on the other side of the cooler.

12. The system of claim 11 further comprising a temperature detection mechanism, wherein the control module controls the thermoelectric cooler based on output from the temperature detection mechanism.

13. The system of claim 1 wherein the battery is connected to the mobile electronic device.

14. The system of claim 13 wherein the battery is fixed to the chassis.

15. The system of claim 1 wherein the control module comprises a system microprocessor and a signal processing microprocessor, and the system microprocessor reduces power to the signal processing microprocessor when the sensor is not in use.

16. The system of claim 1 further comprising a white light-emitting diode.

17. The system of claim 1 wherein the mobile electronic device further comprises a motion sensor and data from the motion sensor is used to interpret data from the infrared sensor or the camera.

18. The system of claim 1 wherein the camera is temporally synchronized to the infrared sensor.

20. The system of claim 1, wherein the software module enables correction for rotation and translation of the infrared sensor relative to the camera by correlating pixels of image data from the camera to pixels of image data from the infrared sensor.

21. The system of claim 7, wherein the mechanism is controlled electrically and automatically via a connection to the calibration module.

22. The system of claim 3, wherein the infrared light-emitting diodes are distinct from the infrared projector and function to invisibly illuminate an imaging target to allow the infrared sensor to capture an imaging target in low ambient light conditions.

19. A real-time 3D capture system for a mobile electronic device having a camera comprising:

an infrared projector comprising a laser, lens, and diffraction gratings that projects a pseudo-random speckle pattern onto an imaging target;

a CMOS image sensor with an infrared bandpass filter that captures the pattern;

two infrared light-emitting diodes;

a signal microprocessor that takes data from the sensor, determines depth information from the data, and transmits the depth information to the mobile electronic device;

a system microprocessor that controls the projector, sensor, and signal microprocessor and reduces power to the signal microprocessor when the signal microprocessor is not in use;

a control module comprising the system microprocessor and the signal microprocessor;

a battery that provides power to the projector, sensor, light-emitting diodes, and control module;

a software module connected to the mobile electronic device that controls communication between the control module and the mobile electronic device, wherein communication comprises data from the camera and the depth information;

a calibration module that enables correction for rotation and translation of the sensor relative to the camera for registering the depth information with the output from the camera;

a chassis that holds the projector, sensor, control module, and battery, and couples to the mounting bracket, wherein the projector, sensor, and control module are fixed to the chassis; and

,

a rigid mounting bracket that removably couples the capture system to at least two corners of the mobile electronic device such that the capture system when coupled maintains alignment with the camera, and does not prevent the use of a button on the mobile electronic device.