Adaptive imaging using digital light processing

• US 20110211077A1
• Filed: 08/08/2002
• Published: 09/01/2011
• Est. Priority Date: 08/09/2001
• Status: Active Grant

• Current Assignee: Trustees Of Columbia University In The City Of New York (Columbia University)
• Original Assignee: Trustees Of Columbia University In The City Of New York (Columbia University)
• Inventors: Boult, Terrance E., Nayar, Shree K.
• Granted Patent: US 8,675,119 B2
• Time in Patent Office: Days
• US Class Current: 348/207.1
• CPC Class Codes: G02B 26/06 for controlling the phase of light G02B26/08 takes precedence , measuring optical phase difference G01J9/00
• CPC Class Codes: G02B 26/0833 the reflecting element being a micromechanical device, e.g. a MEMS mirror, DMD G02B26/0825 takes precedence; micromechanical devices in general B81B
• CPC Class Codes: H04N 23/70 Circuitry for compensating brightness variation in the scene
• CPC Class Codes: H04N 23/741 by increasing the dynamic range of the image compared to the dynamic range of the electronic image sensors
• CPC Class Codes: H04N 23/75 by influencing optical camera components
• CPC Class Codes: H04N 25/61 the noise originating only from the lens unit, e.g. flare, shading, vignetting or "cos4"
• CPC Class Codes: H04N 25/63 applied to dark current

Abstract

A system (100) for the adaptive imaging of a scene includes a digital light processing apparatus (150) adapted for controllably reflecting an image of the scene in at least a first direction to thereby reflect an intensity modulated image of the scene along at least the first direction, an image detector (140) for detecting the intensity modulated image of the scene and generating corresponding image data, and an image data processor (154) for processing the image data into control data and providing the control data to the digital light processing (150) apparatus for control thereof.

Claims

1. A system for adaptively imaging a scene, comprising:

(a) a digital light processing apparatus adapted for controllably reflecting an image of said scene in at least a first direction to thereby reflect an intensity modulated image of said scene in at least said first direction;

(b) an image detector, optically coupled to said digital light processing apparatus along said first direction of reflection, for detecting said intensity modulated image of said scene and generating corresponding detected image data, and

(c) an image data processor, coupled to said image detector and receiving said detected image data therefrom, and coupled to said digital light processing apparatus, for processing said detected image data into control data, and providing said control data to said digital light processing apparatus for adaptive control thereof.

2. The system of claim 1, wherein said digital light processing apparatus comprises a digital micromirror device.

3. The system of claim 2, further comprising an image focusing device, optically coupled to said scene and said digital micromirror device and positioned therebetween, for focusing an image of said scene onto said digital light processing apparatus.

4. The system of claim 3, wherein said image focusing device comprises a lens.

5. The system of claim 2, further comprising an image focusing device, optically coupled to said image detector and said digital micromirror device and positioned therebetween, for focusing said modulated image of said scene onto said image detector.

6. The system of claim 5, wherein said image focusing device comprises a lens.

7. The system of claim 6, wherein said image detector comprises a CCD camera.

8. The system of claim 2, wherein said image data processor comprises a general purpose computer including software for processing said detected image data into control data;

providing said control data to said digital micromirror device, and generating final image data from detected image data.

9. The system of claim 1, further comprising a second image detector, optically coupled to said digital light processing apparatus along a second direction of reflection therefrom and adapted for receiving a second intensity modulated image of said scene therefrom, for detecting said second intensity modulated image of said scene and generating corresponding detected image data, and wherein said image data processor is further coupled to said second image detector for receiving said detected image data therefrom, and adapted for processing said second detected image data into said control data.

10. The system of claim 1, further comprising (i) a beam splitter, optically coupled to said digital light processing apparatus along said first direction of reflection, for splitting said modulated image therefrom into a first partial reflection and a second partial reflection, wherein said digital image detector is optically coupled to said beam splitter along said first direction of partial reflection, and (ii) a second image detector, optically coupled to said beam splitter along said second direction of partial reflection, for detecting said intensity modulated image of said scene along said second direction of partial reflection and generating corresponding detected image data, and wherein said image data processor is further coupled to said second image detector for receiving said detected image data therefrom, and adapted for processing said second detected image data into said control data.

11. The system of claim 1, further comprising a second digital light processing apparatus adapted for controllably reflecting an image of at said scene in at least a first direction to thereby reflect an intensity modulated image of said scene along at least said first direction, optically coupled to said digital light processing apparatus along said first direction of reflection thereof, for detecting said intensity modulated image of said scene and generating a further modulated image of said scene, and wherein said digital image detector is optically coupled to said digital light processing apparatus through said first direction of reflection from said second digital light processing apparatus.

12. The system of claim 11, further comprising a second image detector, wherein said second image detector is optically coupled to said digital light processing apparatus through a second direction of reflection from said second digital light processing apparatus.

13. A system for adaptively imaging a scene, comprising:

(a) a digital light processing apparatus adapted for controllably reflecting an image of said scene in at least a first direction to thereby reflect an intensity modulated image of said scene in at least said first direction;

(b) an image focusing device, optically coupled to said scene and said digital light processing apparatus and positioned therebetween, for focusing an image of said scene onto said digital light processing apparatus;

(c) a digital image detector, optically coupled to said digital light processing apparatus along said first direction of reflection, for detecting said intensity modulated image of said scene and generating corresponding detected image data, and

(d) an image data processor, coupled to said image detector and receiving said detected image data therefrom, for processing said detected image with filter data to generate final image data from said detected image data.

14. The system of claim 13, wherein said controllable digital light processing apparatus comprises a digital micromirror device.

15. The system of claim 14, wherein said image focusing device comprises a lens.

16. The system of claim 14, further comprising an image focusing device, optically coupled to said image detector and said digital micromirror device and positioned therebetween, for focusing said modulated image of said scene onto said image detector.

17. The system of claim 16, wherein said image focusing device comprises a lens.

18. The system of claim 17, wherein said image detector comprises a CCD camera.

19. The system of claim 13, wherein said filter data comprises Laplacian filter data.

20. The system of claim 13, wherein said filter data comprises first and second optical kernal data.

21. A method for controlling the adaptive imaging of a scene using an N-level digital light processing apparatus adapted for controllably reflecting an image of said scene in at least a first direction to thereby reflect an intensity modulated image of said scene along at least said first direction, comprising the steps of:

(a) setting a digital light processing apparatus control signal corresponding to said scene to an initial level N;

(b) taking a first measurement of scene energy at one or more pixels corresponding to at least a portion of said scene using said initial control signal;

(c) revising said digital light processing apparatus control signal to a second level using said first measurement; and

(d) taking a second measurement of scene energy at said one or more pixels using said revised control signal.

22. The method of claim 21, further comprising the steps of:

(e) comparing said second measurement with a threshold value;

updating said digital light processing apparatus control signal to a third level using said comparison made in step (e); and

(g) taking a third measurement of scene energy at said one or more pixels using said updated control signal.

23. The method of claim 22, further comprising the step of:

(h) applying steps (a)-(g) to at least a different one or more pixels corresponding to a different portion of said scene at the same time said steps (a)-(g) are applied to said one or more pixels, to thereby simultaneously apply steps (a)-(g) to two or more groups of one or more pixels.

24. The method of claim 23, further comprising the steps of:

(i) comparing any most recent measurements with a threshold value;

(j) updating said digital light processing apparatus control signals to one or more new levels using said comparisons made in step (h),

(k) taking additional measurements of scene energy at said two or more groups of one or more pixels using corresponding control signals updated in step (j).

25. The method of claim 21, further comprising the steps of:

(i) determining whether said first measurement is below a first threshold value and (ii) if said first measurement is below said threshold value, determining whether said first measurement is below a second threshold value; and

wherein step (c) comprises revising said digital light processing apparatus control signal to a lower second level if said first measurement is determined to not be below said first threshold value, to a higher second level if said first measurement is determined to be below said first threshold value and below said second threshold value, and to the same level if said first measurement is determined to be below said first threshold value and not below said second threshold value.

26. The method of claim 21, wherein step (c) comprises revising said digital light processing apparatus control signal to a second level using said first measurement and a remapping value.

27. The method of claim 26, further comprising the steps of:

(e) comparing said second measurement to a threshold value;

(f) updating said digital light processing apparatus control signal to a third level using said remapping value and said comparison made in step (e);

(g) taking a third measurement of scene energy at said one or more pixels using said updated control signal;

(h) applying steps (a)-(g) to at least a different one or more pixels corresponding to a different portion of said scene at the same time said steps (a)-(g) are applied to said one or more pixels, to thereby simultaneously apply steps (a)-(g) to two or more groups of one or more pixels;

(i) comparing any most recent measurements with a threshold value;

(j) updating said digital light processing apparatus control signals to one or more new levels using said remapping value and said comparisons made in step (i); and

(k) taking additional measurements of scene energy at said two or more groups of one or more pixels using corresponding control signals updated in step (j).

28. The method of claim 21, wherein step (c) comprises revising said digital light processing apparatus control signal to a second level using said first measurement and a secondary control value.

29. The method of claim 26, The method of claim 21, further comprising the steps of:

(e) comparing said second measurement to a threshold value;

(f) updating said digital light processing apparatus control signal to a third level using said secondary control value and said comparison made in step (e);

(g) taking a third measurement of scene energy at said one or more pixels using said updated control signal;

(h) applying steps (a)-(g) to at least a different one or more pixels corresponding to a different portion of said scene at the same time said steps (a)-(g) are applied to said one or more pixels, to thereby simultaneously apply steps (a)-(g) to two or more groups of one or more pixels;

(i) comparing any most recent measurements with a threshold value;

(j) updating said digital light processing apparatus control signals to one or more new levels using said secondary control value and said comparisons made in step (i); and

(k) taking additional measurements of scene energy at said two or more groups of one or more pixels using corresponding control signals updated in step (j).

30. The method of claim 21, wherein said first measurement includes measurements from at least two differently positioned sensors and said second measurement includes measurements from said at least two differently positioned sensors.

31. The method of claim 21, wherein said initial digital light processing apparatus control signal includes two or more control signals and said revised digital light processing apparatus control signal includes two or more control signals.

32. A method for controlling the adaptive imaging of a scene using an N-level digital light processing apparatus adapted for controllably reflecting an image of said scene in at least a first direction to thereby reflect an intensity modulated image of said scene along at least said first direction, comprising the steps of:

(a) setting a digital light processing apparatus control signal corresponding to a portion of said scene to an initial level N;

(b) taking a measurement of scene energy at one or more points corresponding to said scene using said initial control signal; and

(c) convolving said measurement with filter data.

33. The method of claim 32, wherein said filter data comprises Laplacian filter data.

34. The method of claim 32, further comprising the step of:

(d) convolving said measurement with second filter data; and

(e) determining a final image from said convolution of said measurement with said filter data and said convolution of said measurement with said second filter data.

35. The method of claim 34, wherein said filter data comprises first optical kernal data and said second filter data comprises second optical kernal data.