Method for detecting a moving object in a sequence of images captured by a moving camera, computer system and computer program product

• US 8,629,908 B2
• Filed: 11/20/2007
• Issued: 01/14/2014
• Est. Priority Date: 11/20/2006
• Status: Active Grant

• Current Assignee: Nederlandse Organisatie Voor Toegepast-Natuurwetenschappelijk Onderzoek Tno
• Original Assignee: Nederlandse Organisatie Voor Toegepast-Natuurwetenschappelijk Onderzoek Tno
• Inventors: Schutte, Klamer, Schavemaker, Johannes Gerardus Maria, Dijk, Judith, De Lange, Dirk-Jan Jeroen
• Primary Examiner(s): BATES, KEVIN T
• Application Number:
  US12/515,445
• Publication Number:
  US 20100053333A1
• Time in Patent Office: 2,247 Days
• Field of Search: 348/169, 348/159, 348/142
• US Class Current: 348/169
• CPC Class Codes: G06T 3/4053 based on super-resolution, i.e. the output image resolution being higher than the sensor resolution
• CPC Class Codes: G06T 7/254 involving subtraction of images

Abstract

The invention relates to a method for detecting a moving object in a sequence of images captured by a moving camera. The method comprises the step of constructing a multiple number of difference images by subtracting image values in corresponding pixels of multiple pairs of images being based on captured images. Further, the method comprises the step of retrieving a moving object by extracting spatial information of pixels in the multiple number of constructed difference images having relatively large image values. In addition, from a pair of images in the construction step an image is a representation of a high resolution image having a higher spatial resolution than original captured images on which the high resolution image is based.

Claims

1. A method for detecting a moving object in a sequence of images captured by a moving camera, comprising the steps of

a. displacing a camera mounted on a platform so that a sequence of images having different views are captured;

b. processing a sequence set of images to generate a super-resolution image having a higher spatial resolution than the resolution of the original sequence set of captured images;

c. constructing a multiple number of difference images by subtracting image values in corresponding pixels of the super-resolution image and a sequence set of captured images; and

d. retrieving a moving object by extracting spatial information of pixels in the multiple number of constructed difference images having relatively large image values,

e. wherein, in construction step c. an image constructed from a pair of images is a representation of a high resolution image having a higher spatial resolution than original captured images on which the high resolution image is based with noise and/or aliasing effects suppressed; and

f. wherein the super-resolution image obtained in step b. is further processed by applying a camera model to the super-resolution image resulting in an image having real camera effects and having substantially the same spatial resolution as the original captured images on which the super-resolution image is based.

2. A method according to claim 1, wherein the difference images in the constructing step are both super-resolution images each having a higher spatial resolution than original captured images on which the super-resolution images are based.

3. A method according to claim 1, wherein original captured images on which a super-resolution image is based form temporal subsequent images in the sequence of images captured by the camera.

4. A method according to claim 1, wherein extracting spatial information comprises clustering pixels in the multiple number of constructed difference images having relatively large image values to a clustered object.

5. A method according to claim 1, further comprising the step of generating a temporal integrated difference image based on a multiple number of difference images.

6. A method according to claim 5, wherein extracting spatial information comprises applying a spatial filter to a temporal integrated difference image.

7. A method according claim 6, wherein the spatial filter has an elongate shape.

8. A method according to claim 6, wherein the spatial filter is applied in a multiple number of spatial orientations to the temporal integrated difference image.

9. A method according to claim 5, further comprising the step of tracking a clustered object in one of difference images and temporal integrated difference images.

10. Computer system comprising a processor for detecting a moving object in a sequence of images captured by a moving camera, the processor being configured to construct multiple difference images by subtracting image values in corresponding pixels of multiple pairs of images based on captured images wherein one of the images of the pair is a super-resolution image derived from captured images, and to retrieve a moving object by extracting spatial information of pixels having relatively large image values in the constructed difference images, wherein an image constructed from a pair of images is a representation of a high resolution image having a higher spatial resolution than original captured images on which the high resolution image is based with noise and/or aliasing effects preserved and wherein the processor is further configured to apply a camera model to the super-resolution image resulting in an image having real camera effects and having substantially the same spatial resolution as the original captured images on which the super-resolution image is based.

11. A non-transitory computer readable media containing non-transitory instructions for programming a computer including a processor for detecting a moving object in a sequence of images captured by a moving camera, the non-transitory instructions controlling the processor of the computer to construct a multiple number of difference images by subtracting image values in corresponding pixels of multiple pairs of images being based on captured images, wherein at least one image of a pair of images is a super-resolution image derived from captured images, and to retrieve a moving object by extracting spatial information of pixels in the multiple number of constructed difference images having relatively large image values in the of constructed difference images, wherein an image constructed from a pair of images is a representation of a high resolution image having a higher spatial resolution than original captured images on which the high resolution image is based with noise and/or aliasing effects suppressed and wherein the non-transitory instuctions control the processor of the computer to apply a camera model to the super-resolution image resulting in an image having real camera effects and having substantially the same spatial resolution as the original captured images on which the super-resolution image is based.

12. A method for detecting a moving object in a sequence of images captured by a moving camera, comprising the steps of

a. capturing a sequence of images by a moving camera;

b. constructing a multiple number of difference images by subtracting image values in corresponding pixels of multiple pairs of images based on images from the sequence of captured images, wherein at least one image of a pair is a super-resolution image derived from captured images and having a higher spatial resolution than original captured images and wherein noise levels and/or aliasing are suppressed in the difference images; and

c. retrieving a moving object by extracting spatial information of pixels in the multiple number of constructed difference images having relatively large image values, wherein the super-resolution image is further processed by applying a camera model to the super-resolution image resulting in an image having real camera effects and having substantially the same spatial resolution as the original captured images on which the super-resolution image is based.