Array of scanning sensors

• US 8,417,058 B2
• Filed: 09/15/2010
• Issued: 04/09/2013
• Est. Priority Date: 09/15/2010
• Status: Active Grant

• Current Assignee: Microsoft Technology Licensing LLC (Microsoft Corporation)
• Original Assignee: Microsoft Corporation
• Inventors: Tardif, John A.
• Primary Examiner(s): Cunningham, Gregory F
• Application Number:
  US12/883,012
• Publication Number:
  US 20120063637A1
• Time in Patent Office: 937 Days
• Field of Search: 235/462.32, 250/334, 348/39, 348/230.1, 348/272, 348/294, 348/298, 348/308, 348/E5.03, 348/E5.092, 352/94, 359/630, 377/10, 382/282, 396/274, 396/431, 399/231
• US Class Current: 382/282
• CPC Class Codes: A63F 2300/1093 using visible light
• CPC Class Codes: A63F 2300/8011 Ball
• CPC Class Codes: A63F 2300/8088 involving concurrently several players in a non-networked game, e.g. on the same game console
• CPC Class Codes: G06T 11/00 2D [Two Dimensional] image generation

Abstract

An array of image sensors is arranged to cover a field of view for an image capture system. Each sensor has a field of view segment which is adjacent to the field of view segment covered by another image sensor. The adjacent field of view (FOV) segments share an overlap area. Each image sensor comprises sets of light sensitive elements which capture image data using a scanning technique which proceeds in a sequence providing for image sensors sharing overlap areas to be exposed in the overlap area during the same time period. At least two of the image sensors capture image data in opposite directions of traversal for an overlap area. This sequencing provides closer spatial and temporal relationships between the data captured in the overlap area by the different image sensors. The closer spatial and temporal relationships reduce artifact effects at the stitching boundaries, and improve the performance of image processing techniques applied to improve image quality.

Claims

1. In a system of image sensors using a scanning technique to capture image data, a method of image processing image data in an overlap area comprising:

capturing image data in a first direction of traversal for a first overlap area of a first field of view (FOV) segment in an overlap time period, the first overlap area being shared with a second adjacent FOV segment; and

capturing image data in a second direction of traversal opposite to the first direction of traversal for the first overlap area of the second FOV segment in the same overlap time period.

2. The method of claim 1, wherein:

the capturing image data in a first direction of traversal further comprises starting the traversal at an edge of the first FOV segment in the shared overlap area; and

the capturing image data in a second direction of traversal further comprises starting the traversal at an edge of the second FOV segment in the shared overlap area.

3. The method of claim 1, wherein:

the capturing image data in a first direction of traversal further comprises starting the traversal at a non-overlapping edge of the first FOV segment and ending in the shared overlap area; and

wherein

the capturing image data in a second direction of traversal further comprises starting the traversal at non-overlapping edge of the second FOV segment and ending in the shared overlap area.

4. The method of claim 1 further comprising:

capturing image data in the first direction of traversal for a second overlap area of a third FOV segment adjacent to the first FOV segment in the same overlap time period, the second overlap area being shared with the first adjacent FOV segment.

5. The method of claim 1 further comprising combining data of both FOV segments in the overlap area for an image frame of data.

6. The method of claim 1, wherein:

combining data of both FOV segments in the overlap area for an image frame of data further comprises

for a selected scene space covered in a portion of the overlap area, identifying

one or more spatially corresponding data sets in the adjacent FOV segments covering the overlap area;

assigning a weighting to the spatially corresponding data sets based on criteria including how current the time of capture; and

combining the spatially corresponding data sets in the adjacent FOV segments based on their weightings.

7. A system of image sensors comprising:

a plurality of image sensors including a first image sensor and a second image sensor;

each of the first and second image sensors covering adjacent field of view (FOV) segments and sharing a first overlap area in their adjacent FOV segments;

each of the image sensors in the plurality comprising respective sets of light sensitive elements;

each of the image sensors in the plurality using a rolling capture technique to capture image data in a direction of traversal of its respective sets of light sensitive elements; and

the first and second image sensors capturing image data in the first overlap area during a same overlap time period during which the first and second image sensors use opposite directions of traversal of their respective sets of light sensitive elements.

8. The system of claim 7 wherein the image sensors are homogeneous.

9. The system of claim 7 wherein the at least two image sensors are RGB image sensors.

10. The system of claim 7 wherein the first and second image sensors using opposite directions of traversal start the traversal for a frame at respective set of light sensitive elements of the first overlap area shared with the adjacent FOV segment.

11. The system of claim 7 wherein the first and second image sensors using opposite directions of traversal start the traversal for a frame at respective set of light sensitive elements at the non-overlapping edge of the field of view segment for each sensor and end in the first overlap area.

12. The system of claim 7 further comprising:

one or more memory buffers accessible by each of the first and second image sensors to receive image data for each respective FOV segment; and

a software controlled processor communicatively coupled to the one or more memory buffers for accessing the image data and performing one or more image error correction techniques using image data for the first overlap area from a plurality of FOV segments.

13. The system of claim 7 further comprising:

a third image sensor covering a FOV segment adjacent to the FOV segment of the first image sensor and sharing a second overlap area with the first image sensor, the second overlap area covering a portion of the first overlap area; and

the first and third image sensors capturing image data in the second overlap area during the same overlap time period during which the first and third image sensors use a same direction of traversal of their respective sets of light sensitive elements.

14. The system of claim 13 further comprising:

a fourth image sensor covering a FOV segment adjacent to the FOV segment of the second image sensor and sharing a third overlap area with the second image sensor, a portion of the third overlap area covering a portion of the first overlap area and the second overlap area;

the second and fourth image sensors capturing image data in the third overlap area during the same overlap time period during which the second and fourth image sensors use a same direction of traversal of their respective sets of light sensitive elements; and

the portions of the first, second and third overlap areas covering each other forming an overlap intersection area.

15. In a system of image sensors using a scanning technique to capture image data, a method of image processing image data in an overlap area comprising:

capturing image data in a first direction of traversal for a first overlap area of a first field of view (FOV) segment in an overlap time period, the first overlap area being shared with a second adjacent FOV segment; and

capturing image data in a second direction of traversal opposite to the first direction of traversal for the first overlap area of the second FOV segment in the same overlap time period; and

performing an image stitching technique to stitch together the image data from both FOV segments for the first overlap area.

16. The method of claim 15 wherein the scanning technique is a rolling capture technique.

17. The method of claim 15 wherein

the scanning technique comprises a respective beam of light traversing each of the respective sets of light sensitive elements of each of the first, second, third and fourth image sensors in a scanning pattern;

and

the scanning pattern for each respective beam starts in a middle of the overlap intersection area.

18. The method of claim 15 further comprising:

capturing image data in the first direction of traversal for a second overlap area of a third FOV segment adjacent to the first FOV segment in the same overlap time period, the second overlap area being shared with the first adjacent FOV segment; and

performing an image stitching technique to stitch together the image data from any combination of the first and third FOV segments for the second overlap area.

19. The method of claim 15 wherein performing the image stitching technique based on data from both FOV segments for the first overlap area comprises the step of blending data from the first FOV with data from the second FOV in the first overlap area.

20. The method of claim 19, the first overlap area including a first overlap boundary nearest a center of the first FOV segment and the first overlap area including a second overlap boundary nearest a center of the second FOV segment, the step of blending comprising the steps of:

weighting data from the first FOV segment higher than data from the second FOV segment at positions in the first overlap area nearer to the first overlap boundary than the second overlap boundary,

weighting the data from the first and second FOV segments equally toward a center of the first overlap area between the first and second overlap boundaries, and

weighting data from the second FOV segment higher than data from the first FOV segment at positions in the first overlap area nearer to the second overlap boundary than the first overlap boundary.