Image processing devices and image processing methods of moving objects

• US 8,994,837 B2
• Filed: 11/26/2010
• Issued: 03/31/2015
• Est. Priority Date: 11/26/2010
• Status: Expired due to Fees

• Current Assignee: Intel Corporation
• Original Assignee: Intel Mobile Communications GmbH (Intel Corporation)
• Inventors: Wasserbauer, Andreas, Meindl, Manfred, Haas, Juergen, Gstoettner, Andreas
• Primary Examiner(s): Misleh, Justin P
• Application Number:
  US12/954,702
• Publication Number:
  US 20120133785A1
• Time in Patent Office: 1,586 Days
• Field of Search: None
• US Class Current: 348/208.6
• CPC Class Codes: G06T 3/4053 based on super-resolution, i.e. the output image resolution being higher than the sensor resolution
• CPC Class Codes: H04N 23/80 Camera processing pipelines; Components thereof

Abstract

In an embodiment, an image processing device is provided. The image processing device may include: a first image acquirer configured to acquire a first image of a first spatial resolution; a second image acquirer configured to acquire a second image of a second spatial resolution, wherein the second spatial resolution may be higher than the first spatial resolution; a determiner configured to determine in the first image a location of a part of the first image that corresponds to a pre-determined part of the second image based on a pre-determined similarity criterion; and a copying circuit configured to copy the pre-determined part of the second image to a location in an output image based on the determined location.

Claims

1. An image processing device, comprising:

a first image acquirer configured to acquire a first image over a first readout time;

a second image acquirer configured to acquire a second image over a second readout time, wherein the second readout time is longer than the first readout time;

a determiner configured to determine in the first image a location of an object in a portion of the first image that corresponds to a pre-determined part of the second image based on a pre-determined similarity criterion, wherein the pre-determined part of the second image is distorted due to movement of the object during the second readout time; and

a copying circuit configured to copy the pre-determined part of the second image to a location in an output image corresponding to the determined location of the object in the first image;

wherein the location in the output image is different from the location of the pre-determined part of the second image.

2. The image processing device of claim 1,

wherein the pre-determined similarity criterion is a criterion for comparing the first image with the second image.

3. The image processing device of claim 1, further comprising:

an upscaling circuit configured to upscale the first image to the spatial resolution of the second image;

wherein the determiner is further configured to determine in the upscaled first image a location of a part of the upscaled first image that corresponds to the pre-determined part of the second image based on a further pre-determined similarity criterion.

4. The image processing device of claim 3,

wherein the copying circuit is further configured to, for a region of the output image, to which no part of the second image has been copied, copy a region of the upscaled first image;

wherein the region of the output image corresponds to the region of the upscaled first image.

5. The image processing device of claim 3,

wherein the further pre-determined similarity criterion is a criterion for comparing the upscaled first image with the second image.

6. The image processing device of claim 1,

wherein the first image is of a first temporal resolution and the second image is of a second temporal resolution;

and

wherein the first temporal resolution is higher than the second temporal resolution.

7. The image processing device of claim 1,

wherein the second image acquirer comprises a rolling shutter sensor.

8. The image processing device of claim 1,

wherein the first image acquirer and the second image acquirer comprise a common sensor.

9. The image processing device of claim 8, further comprising:

an image sensor configurator configured to configure the common sensor for acquiring the first image to the spatial resolution of the first image and to configure the common sensor for acquiring the second image to the spatial resolution of the second image.

10. The image processing device of claim 1, further comprising:

a dividing circuit configured to divide the second image into a plurality of blocks of pre-determined size;

wherein the pre-determined part of the second image corresponds to a block of the plurality of blocks.

11. The image processing device of claim 1, further comprising:

a motion vector acquirer configured to acquire at least one motion vector based on performing motion estimation of the second image with respect to the first image; and

a generating circuit configured to generate the output image based on performing motion compensation of the second image based on the acquired at least one motion vector.

12. The device of claim 1,

wherein the copying circuit is configured to correct distortion due to movement occurring during the second readout time.

13. An image processing method, comprising:

acquiring a first image over a first readout time;

acquiring a second image over a second readout time, wherein the second readout time is longer than the first readout time;

determining in the first image a location of an object in a portion of the first image that corresponds to a pre-determined part of the second image based on a pre-determined similarity criterion, wherein the pre-determined part of the second image is distorted due to movement of the object during the second readout time; and

copying the pre-determined part of the second image to a location in an output image corresponding to the determined location of the object in the first image;

wherein the location in the output image is different from the location of the pre-determined part of the second image.

14. The image processing method of claim 13,

wherein the pre-determined similarity criterion is a criterion for comparing the first image with the second image.

15. The image processing method of claim 13, further comprising:

upscaling the first image to the spatial resolution of the second image;

wherein the determining of a part of the first image is performed by determining in the upscaled first image a location of a part of the upscaled first image that corresponds to the pre-determined part of the second image based on a further pre-determined similarity criterion.

16. The image processing method of claim 15, further comprising:

for a region of the output image, to which no part of the second image has been copied, copying a region of the upscaled first image;

wherein the region of the output image corresponds to the region of the upscaled first image.

17. The image processing method of claim 15,

wherein the further pre-determined similarity criterion is a criterion for comparing the upscaled first image with the second image.

18. The image processing method of claim 13,

wherein the first image is of a first temporal resolution and the second image is of a second temporal resolution;

and

wherein the first temporal resolution is higher than the second temporal resolution.

19. The image processing method of claim 13,

wherein the second image is acquired by a rolling shutter sensor.

20. The image processing method of claim 13,

wherein the first image and the second image are acquired by a common sensor.

21. The image processing method of claim 13, further comprising:

dividing the second image into a plurality of blocks of pre-determined size;

wherein the pre-determined part of the second image corresponds to a block of the plurality of blocks.

22. The image processing method of claim 13, further comprising:

acquiring at least one motion vector based on performing motion estimation of the second image with respect to the first image; and

generating the output image based on performing motion compensation of the second image based on the acquired at least one motion vector.