Foveated imaging system and method

• US 9,124,808 B2
• Filed: 04/11/2013
• Issued: 09/01/2015
• Est. Priority Date: 03/04/2013
• Status: Active Grant

• Current Assignee: Raytheon Company (Rtx Corporation)
• Original Assignee: Raytheon Company (Rtx Corporation)
• Inventors: Byren, Robert W.
• Primary Examiner(s): CUTLER, ALBERT H
• Application Number:
  US13/860,893
• Publication Number:
  US 20140247372A1
• Time in Patent Office: 873 Days
• Field of Search: None
• US Class Current: 1/1
• CPC Class Codes: H04N 23/60 Control of cameras or camera modules
• CPC Class Codes: H04N 23/69 Control of means for changing angle of the field of view, e.g. optical zoom objectives or electronic zooming
• CPC Class Codes: H04N 23/951 by using two or more images to influence resolution, frame rate or aspect ratio
• CPC Class Codes: H04N 25/00 Circuitry of solid-state image sensors [SSIS]; Control thereof

Abstract

A time-varying image is synthesized having at least one time-varying region of interest therewithin. The image is spatially modulated with an array of modulating pixels to form a modulated image. The array of modulating pixels has a modulator resolution. The modulated image is sensed with an array of sensor pixels. The array of sensor pixels has a sensor resolution. The modulator resolution is finer than the sensor resolution. Each sensor pixel corresponds to a plurality of modulating pixels. Said sensor pixel and said plurality of modulating pixels subtend the same region in the image. Outside the region of interest, the modulated image is synthesized at the sensor resolution and at a sensor video frame rate. Inside the region of interest, the modulated image is synthesized at the modulator resolution and at an effective synthesized video frame rate that is less than sensor video frame rate.

Claims

1. A method of sensing a time-varying image having at least one time-varying region of interest therewithin, the method comprising:

spatially modulating the time-varying image with an array of modulating pixels to form a modulated image, the array of modulating pixels having a modulator resolution; and

sensing the modulated image with an array of sensor pixels, the array of sensor pixels having a sensor resolution, the modulator resolution being finer than the sensor resolution;

wherein each sensor pixel corresponds to a plurality of modulating pixels, said sensor pixel and said plurality of modulating pixels subtending the same region in the time-varying image;

wherein outside the region of interest, the modulated image is synthesized at the sensor resolution and at a sensor video frame rate;

wherein inside the region of interest, the modulated image is synthesized at the modulator resolution and at an effective synthesized video frame rate that is less than the sensor video frame rate;

wherein each modulating pixel is switchable between an on state and an off state;

wherein the on state comprises directing at least a portion of time-varying image light, incident on said modulating pixel, to the modulated image;

wherein the off state comprises blocking the time-varying image light, incident on said modulating pixel;

wherein inside the region of interest, said plurality of modulating pixels are modulated in an orthogonal sequence of frames; and

wherein in one frame of the orthogonal sequence, all the modulating pixels in the plurality are in the on state.

2. A method of sensing a time-varying image having at least one time-varying region of interest therewithin, the method comprising:

spatially modulating the time-varying image with an array of modulating pixels to form a modulated image, the array of modulating pixels having a modulator resolution; and

sensing the modulated image with an array of sensor pixels, the array of sensor pixels having a sensor resolution, the modulator resolution being finer than the sensor resolution;

wherein each sensor pixel corresponds to a plurality of modulating pixels, said sensor pixel and said plurality of modulating pixels subtending the same region in the time-varying image;

wherein outside the region of interest, the modulated image is synthesized at the sensor resolution and at a sensor video frame rate;

wherein inside the region of interest, the modulated image is synthesized at the modulator resolution and at an effective synthesized video frame rate that is less than the sensor video frame rate;

wherein each modulating pixel is switchable between an on state and an off state;

wherein the on state comprises directing at least a portion of time-varying image light, incident on said modulating pixel, to the modulated image;

wherein the off state comprises blocking the time-varying image light, incident on said modulating pixel;

wherein inside the region of interest, said plurality of modulating pixels are modulated in an orthogonal sequence of frames; and

wherein in all frames of the orthogonal sequence, at least two of the modulating pixels in the plurality are in the on state.

3. A method, comprising:

forming an unmodulated image of a time-varying scene;

modulating the unmodulated image with a multi-pixel spatial modulator to form a modulated image of the time-varying scene, the multi-pixel spatial modulator including a plurality of modulator pixels, each modulator pixel being switchable between an on state and an off state;

each modulator pixel in an on state contributing to the modulated image at a location of the modulator pixel with an optical power proportional to an optical power of the unmodulated image incident on the respective modulator pixel;

sensing the modulated image with a multi-pixel sensor, the sensor including a plurality of sensor pixels, the sensor pixels being fewer in number than the modulator pixels, the modulated image being synthesized at a first resolution and a first video frame rate;

identifying a region of interest within the time-varying scene; and

synthesizing the region of interest with a second resolution finer than the first resolution, and a second video frame rate slower than the first video frame rate;

wherein each sensor pixel corresponds with a plurality of the modulator pixels, said sensor pixel and said plurality of modulator pixels occupying the same position within the time-varying scene;

wherein within the region of interest, said plurality of modulator pixels are switched on and off in an orthogonal sequence;

wherein the number of modulator pixels in said plurality is X;

wherein within the region of interest, said sensor pixel records X values of optical power, the X values of optical power being orthogonal combinations of optical power from the X modulator pixels;

wherein the optical powers of the individual modulator pixels are determined from the orthogonal combinations of optical power from the modulator pixels; and

wherein in one of the orthogonal combinations, all the modulator pixels in the plurality are switched on.

4. A method, comprising:

forming an unmodulated image of a time-varying scene;

modulating the unmodulated image with a multi-pixel spatial modulator to form a modulated image of the time-varying scene, the multi-pixel spatial modulator including a plurality of modulator pixels, each modulator pixel being switchable between an on state and an off state;

each modulator pixel in an on state contributing to the modulated image at a location of the modulator pixel with an optical power proportional to an optical power of the unmodulated image incident on the respective modulator pixel;

sensing the modulated image with a multi-pixel sensor, the sensor including a plurality of sensor pixels, the sensor pixels being fewer in number than the modulator pixels, the modulated image being synthesized at a first resolution and a first video frame rate;

identifying a region of interest within the time-varying scene; and

synthesizing the region of interest with a second resolution finer than the first resolution, and a second video frame rate slower than the first video frame rate;

wherein each sensor pixel corresponds with a plurality of the modulator pixels, said sensor pixel and said plurality of modulator pixels occupying the same position within the time-varying scene;

wherein outside the region of interest, said plurality of modulator pixels are all switched on; and

wherein outside the region of interest, the optical powers of the individual modulator pixels are combined onto the same sensor pixel and are indistinguishable from one another.