Method, device and system for imaging

• US 8,208,129 B2
• Filed: 12/04/2007
• Issued: 06/26/2012
• Est. Priority Date: 01/14/2007
• Status: Expired due to Fees

• Current Assignee: Microsoft International Holdings BV (Microsoft Corporation)
• Original Assignee: Microsoft International Holdings BV (Microsoft Corporation)
• Inventors: Yahav, Giora, Mandelboum, David, Zigelman, Gil
• Primary Examiner(s): Alsomiri, Isam
• Assistant Examiner(s): RATCLIFFE, LUKE D
• Application Number:
  US12/523,055
• Publication Number:
  US 20100053592A1
• Time in Patent Office: 1,666 Days
• Field of Search: 356/4.01, 356/5.01, 382/103, 382/106
• US Class Current: 356/4.01
• CPC Class Codes: A63F 2300/1093 using visible light
• CPC Class Codes: G01S 17/18 wherein range gates are used
• CPC Class Codes: G01S 17/86 Combinations of lidar systems with systems other than lidar, radar or sonar, e.g. with direction finders
• CPC Class Codes: G01S 17/89 for mapping or imaging
• CPC Class Codes: G01S 7/486 Receivers

Abstract

There is provided a novel method, device and system for imaging. According to some embodiments of the present invention, a visible light image sensor array (e.g. Charge-Couple Device (“CCD”) or Complementary Metal-Oxide-Semiconductor (“CMOS”)) and a ranging sensor array may be arranged substantially facing the same direction and having a substantially overlapping field of view. According to further embodiments of the present invention, a range value derived from one or more ranging sensor elements within the ranging sensor array may be correlated with one or more imaging elements within the visible sensor array. Image processing logic may correlate the derived ranging value with the one or more imaging elements as a function of the derived range.

Claims

1. An imaging device comprising:

a visible light image sensor array of pixels and a ranging sensor array of pixels, the sensor arrays being positioned in a parallel arrangement along a sensors axis and substantially facing the same direction and having at least a partially overlapping field of view; and

image processing logic adapted to correlating a first pixel of the ranging sensor array, the first pixel capturing data from which is derived a ranging value representing a distance to an object, with a corresponding first pixel of the visible light sensor array, the corresponding first pixel capturing image data of the same object, based on the derived ranging value and the parallel arrangement of the arrays along the sensors axis.

2. The device according to claim 1, wherein said image processing logic is adapted to correlating the first pixel of the ranging sensor array, the first pixel capturing data from which is derived the ranging value representing the distance to the object with the corresponding first pixel of the visible light sensor array, the corresponding first pixel capturing image data of the same object, based on the derived ranging value and the parallel arrangement of the arrays along the sensors axis by estimating an offset value and applying the offset value to an address of the first pixel of the ranging sensor array for determining an address of the corresponding first pixel of the visible light sensor array.

3. The device according to claim 2, wherein estimating the offset value is based on the following:

a distance between optical centers of the sensor arrays;

a focusing length which is the same for the sensor arrays in a Z direction extending from the sensors axis towards the object;

coordinates in three directions including the Z direction of a focal point of the first pixel of the ranging sensor array from the optical center of the ranging sensor array;

coordinates in three directions including the Z direction of a focal point of the first pixel of the visible light sensor array from the optical center of the visible light sensor array; and

the derived ranging value from the first pixel of the ranging sensor array to the object.

4. The device according to claim 1, wherein said image processing logic is further adapted to filter background noise.

15. The device according to claim 3, wherein estimating the offset value is further based on a pixel size difference between the first pixel of the ranging sensor array and the corresponding first pixel of the visible light sensor array.

5. A method of imaging comprising:

substantially concurrently acquiring a visible light image with a visible light image sensor array of pixels and a ranging image from a ranging sensor array of pixels, which visible light image sensor array and ranging sensor array are positioned in a parallel arrangement along a sensors axis and substantially face the same direction and have at least partially overlapping fields of view; and

correlating a first pixel of the ranging sensor array, the first pixel capturing data from which is derived a ranging value representing a distance to an object, with a corresponding first pixel of the visible light sensor array, the corresponding first pixel capturing image data of the same object, based on the derived ranging value and the parallel arrangement of the arrays along the sensors axis.

6. The method according to claim 5, wherein correlating the first pixel of the ranging sensor array, the first pixel capturing data from which is derived the ranging value representing the distance to the object, with the corresponding first pixel of the visible light sensor array, the corresponding first pixel capturing image data of the same object, based on the derived ranging value and the parallel arrangement of the arrays along the sensors axis includes estimating an offset value and applying the offset value to an address of the first pixel of the ranging sensor array for determining an address of the corresponding first pixel of the visible light sensor array.

7. The method according to claim 5, further comprising filtering background noise.

8. A computing system comprising:

an image based interface, wherein the image based interface includes a visible light image sensor array of pixels and a ranging sensor array of pixels which visible light image sensor array and ranging sensor array are positioned in a parallel arrangement along a sensors axis and are substantially facing the same direction and having at least a partially overlapping field of view; and

an image processing logic adapted to correlate a first pixel of the ranging sensor array, the first pixel capturing data from which is derived a ranging value representing a distance to an object, with a corresponding first pixel of the visible light sensor array, the corresponding first pixel capturing image data of the same object, based on the derived ranging value and the parallel arrangement of the arrays along the sensors axis.

9. The system according to claim 8, wherein said image processing logic is adapted to correlate the first pixel of the ranging sensor array, the first pixel capturing data from which is derived the ranging value representing the distance to the object with the corresponding first pixel of the visible light sensor array, the corresponding first pixel capturing image data of the same object, based on the derived ranging value and the parallel arrangement of the arrays along the sensors axis by estimating an offset value and applying the offset value to an address of the first pixel of the ranging sensor array for determining an address of the corresponding first pixel of the visible light sensor array.

10. The system according to claim 8, wherein said image processing logic is further adapted to filter background noise.

11. The computing system according to claim 8, wherein the computing system is selected from the group consisting of:

a gaming console, a web-conferencing system and a security system.

12. A method of fabricating an imaging system comprising:

aligning a visible light image sensor array of pixels and a ranging sensor array of pixels in a parallel arrangement along a sensors axis for substantially facing the same direction and having at least a partially overlapping field of view; and

integrating image processing logic adapted to correlate the first pixel of the ranging sensor array, the first pixel capturing data from which is derived a ranging value representing a distance to an object, with a corresponding first pixel of the visible light sensor array, the corresponding first pixel capturing image data of the same object, based on the derived ranging value and the parallel arrangement of the arrays along the sensors axis.

13. The method according to claim 12, further comprising integrating an illumination module.

14. The method according to claim 12, further comprising integrating first and second modulators adapted to define a dynamic distance window.