Signal processing device and signal processing method, and program and recording medium

• US 7,746,359 B2
• Filed: 01/25/2007
• Issued: 06/29/2010
• Est. Priority Date: 06/27/2003
• Status: Expired due to Fees

• Current Assignee: Sony Corporation (Sony Group Corp.)
• Original Assignee: Sony Corporation (Sony Group Corp.)
• Inventors: Kanemaru, Masanori, Kondo, Tetsujiro, Nagano, Takahiro
• Primary Examiner(s): Amini; Javid A
• Application Number:
  US11/627,163
• Publication Number:
  US 20070120848A1
• Time in Patent Office: 1,251 Days
• Field of Search: 382/255, 345/63
• US Class Current: 345/633
• CPC Class Codes: G06T 2207/20201 Motion blur correction
• CPC Class Codes: G06T 3/4023 based on decimating pixels or lines of pixels; based on inserting pixels or lines of pixels
• CPC Class Codes: G06T 5/73 Deblurring; Sharpening
• CPC Class Codes: G06V 10/443 by matching or filtering

Abstract

A signal processing device, signal processing method, and program and recording medium whereby images and the like closer approximating real world signals can be obtained. The signal processing device includes a movement vector setting unit, a model generator, an equation generator, and a real world waveform estimating unit.

Claims

1. A signal processing device comprising:

a hardware sensor configured to detect a plurality of pixels;

a movement vector setting unit configured to set movement vectors for an object within image data wherein a light signal of a real world is projected on the plurality of pixels, each having a time-space integration effect, and a portion of continuity of the light signal of the real world is lost;

a model generator configured to generate a relation model modeling a relation between a pixel value of each pixel within said image data and a real world function approximated with a spline function, the pixel value of each pixel corresponding to a position in a predetermined one-dimensional direction of a spatial direction of image data is a pixel value acquired by the real world function corresponding to said real world light signals, the model generator configured to estimate the real world function by solving an equation representing that the pixel value is acquired by integrating a model as the moving spline function, which is moving in the time direction corresponding to said movement vector while phase-shifting, in said predetermined one-dimensional direction and the time direction;

an equation generator configured to generate equations by substituting pixel values of each of the pixels within said image data as to the relation model generated by said model generator, as well as to generate equations which provide conditions which constrain change of the pixel values within the pixels displayed with the spline function approximating said real world function to be a small change; and

a real world waveform estimating unit configured to estimate said real world function corresponding to said real world light signals by computing equations generated by said equation generator.

2. The signal processing device according to claim 1, wherein said equation generator is configured to generate said equations with the constraint condition that a level of said real world function outside of the processing region of said image data is a steady value.

3. The signal processing device according to claim 2, further comprising:

a weighting change unit configured to change a weighting between an equation giving a condition which constrains change of the pixel values within the pixels displayed with the spline function approximating said real world function to be the small change, and an equation giving a condition which constrains the level of said real world function outside of the processing region of said image data to be the steady value,

wherein said real world waveform estimating unit is configured to estimate said real world function by computing said equation based on said weighting.

4. The signal processing device according to claim 3, further comprising:

an image generator configured to generate an image made up of pixel values wherein movement blurring of the object within said image data is removed, by integrating the spline function approximating said real world function, serving as estimated results of said real world function, in said predetermined one-dimensional direction and the time direction in predetermined increments; and

a display configured to display the image generated with said image generator,

wherein said weighting change unit is configured to change said weighting, according to user input after images are displayed by said display.

5. A signal processing method comprising:

detecting a plurality of pixels with a hardware sensor;

setting movement vectors for an object within image data wherein a light signal of a real world is projected on the plurality of pixels, each having a time-space integration effect, and a portion of continuity of the light signal of the real world is lost;

generating a relation model modeling a relation between a pixel value of each pixel within said image data and a real world function approximated with a spline function, the pixel value of each pixel corresponding to a position in a predetermined one-dimensional direction of a spatial direction of image data is a pixel value acquired by the real world function corresponding to said real world light signals, said generating including estimating the real world function by solving an equation representing that the pixel value is acquired by integrating a model as the moving spline function, which is moving in the time direction corresponding to a movement vector while phase-shifting, in said predetermined one-dimensional direction and the time direction;

generating equations by substituting pixel values of each of the pixels within said image data as to the relation model generated in said generating a relation model, and generating equations which provide conditions which constrain change of the pixel values within the pixels displayed with the spline function approximating said real world function to be a small change; and

estimating said real world function corresponding to said real world light signals by computing equations generated in said generating equations.

6. A recording medium including computer executable instructions, wherein the instructions, when executed by a computer, cause the computer to perform a method comprising:

detecting a plurality of pixels with a hardware sensor;

setting movement vectors for an object within image data wherein a light signal of a real world is projected on the plurality of pixels, each having a time-space integration effect, and a portion of continuity of the light signal of the real world is lost;

generating a relation model modeling a relation between a pixel value of each pixel within said image data and a real world function approximated with a spline function, the pixel value of each pixel corresponding to a position in a predetermined one-dimensional direction of a spatial direction of image data is a pixel value acquired by the real world function corresponding to said real world light signals, said generating including estimating the real world function by solving an equation representing that the pixel value is acquired by integrating a model as the moving spline function, which is moving in a time direction corresponding to a movement vector while phase-shifting, in said predetermined one-dimensional direction and the time direction;

generating equations by substituting pixel values of each of the pixels within said image data as to the relation model generated in said generating a relation model, and generating equations which provide conditions which constrain change of the pixel values within the pixels displayed with the spline function approximating said real world function to be a small change; and

estimating said real world function corresponding to said real world light signals by computing equations generated in said generating equations.