IMAGE PROCESSING DEVICES AND IMAGE PROCESSING METHODS

• US 20120133785A1
• Filed: 11/26/2010
• Published: 05/31/2012
• Est. Priority Date: 11/26/2010
• Status: Active Grant

• Current Assignee: Intel Corporation
• Original Assignee: Infineon Technologies AG
• Inventors: Wasserbauer, Andreas, Meindl, Manfred, Haas, Juergen, Gstoettner, Andreas
• Granted Patent: US 8,994,837 B2
• Time in Patent Office: Days
• US Class Current: 348/208.4
• CPC Class Codes: G06T 3/4053 based on super-resolution, i.e. the output image resolution being higher than the sensor resolution
• CPC Class Codes: H04N 23/80 Camera processing pipelines; Components thereof

Abstract

In an embodiment, an image processing device is provided. The image processing device may include: a first image acquirer configured to acquire a first image of a first spatial resolution; a second image acquirer configured to acquire a second image of a second spatial resolution, wherein the second spatial resolution may be higher than the first spatial resolution; a determiner configured to determine in the first image a location of a part of the first image that corresponds to a pre-determined part of the second image based on a pre-determined similarity criterion; and a copying circuit configured to copy the pre-determined part of the second image to a location in an output image based on the determined location.

Claims

1. An image processing device, comprising:

a first image acquirer configured to a acquiring a first image of a first spatial resolution;

a second image acquirer configured to acquire a second image of a second spatial resolution, wherein the second spatial resolution is higher than the first spatial resolution;

a determiner configured to determine in the first image a location of a part of the first image that corresponds to a pre-determined part of the second image based on a pre-determined similarity criterion; and

a copying circuit configured to copy the pre-determined part of the second image to a location in an output image based on the determined location.

2. The image processing device of claim 1,

wherein the output image is of the second spatial resolution.

3. The image processing device of claim 1,

wherein the pre-determined similarity criterion is a criterion for comparing the first image with the second image.

4. The image processing device of claim 1, further comprising:

an upscaling circuit configured to upscale the first image to the second spatial resolution;

wherein the determiner is further configured to determine in the upscaled first image a location of a part of the upscaled first image that corresponds to the pre-determined part of the second image based on a further pre-determined similarity criterion.

5. The image processing device of claim 4,

wherein the copying circuit is further configured to, for a region of the output image, to which no part of the second image has been copied, copy a region of the upscaled first image;

wherein the region of the output image corresponds to the region of the upscaled first image.

6. The image processing device of claim 4,

wherein the further pre-determined similarity criterion is a criterion for comparing the upscaled first image with the second image.

7. The image processing device of claim 1,

wherein the first image is of a first temporal resolution and the second image is of a second temporal resolution;

and

wherein the first temporal resolution is higher than the second temporal resolution.

8. The image processing device of claim 1,

wherein the second image acquirer comprises a rolling shutter sensor.

9. The image processing device of claim 1,

wherein the first image acquirer and the second image acquirer comprise a common sensor.

10. The image processing device of claim 9, further comprising:

an image sensor configurator configured to configure the common sensor for acquiring the first image to the first spatial resolution and to configure the common sensor for acquiring the second image to the second spatial resolution.

11. The image processing device of claim 1, further comprising:

a dividing circuit configured to divide the second image into a plurality of blocks of pre-determined size;

wherein the pre-determined part of the second image corresponds to a block of the plurality of blocks.

12. The image processing device of claim 1, further comprising:

a motion vector acquirer configured to acquire at least one motion vector based on performing motion estimation of the second image with respect to the first image; and

a generating circuit configured to generate the output image based on performing motion compensation of the second image based on the acquired at least one motion vector.

13. An image processing method, comprising:

acquiring a first image of a first spatial resolution;

acquiring a second image of a second spatial resolution, wherein the second spatial resolution is higher than the first spatial resolution;

determining in the first image a location of a part of the first image that corresponds to a pre-determined part of the second image based on a pre-determined similarity criterion; and

copying the pre-determined part of the second image to a location in an output image based on the determined location.

14. The image processing method of claim 13,

wherein the output image is of the second spatial resolution.

15. The image processing method of claim 13,

wherein the pre-determined similarity criterion is a criterion for comparing the first image with the second image.

16. The image processing method of claim 13, further comprising:

upscaling the first image to the second spatial resolution;

wherein the determining of a part of the first image is performed by determining in the upscaled first image a location of a part of the upscaled first image that corresponds to the pre-determined part of the second image based on a further pre-determined similarity criterion.

17. The image processing method of claim 16, further comprising:

for a region of the output image, to which no part of the second image has been copied, copying a region of the upscaled first image;

wherein the region of the output image corresponds to the region of the upscaled first image.

18. The image processing method of claim 16,

wherein the further pre-determined similarity criterion is a criterion for comparing the upscaled first image with the second image.

19. The image processing method of claim 13,

wherein the first image is of a first temporal resolution and the second image is of a second temporal resolution;

and

wherein the first temporal resolution is higher than the second temporal resolution.

20. The image processing method of claim 13,

wherein the second image is acquired by a rolling shutter sensor.

21. The image processing method of claim 13,

wherein the first image and the second image are acquired by a common sensor.

22. The image processing method of claim 13, further comprising:

dividing the second image into a plurality of blocks of pre-determined size;

wherein the pre-determined part of the second image corresponds to a block of the plurality of blocks.

23. The image processing method of claim 13, further comprising:

acquiring at least one motion vector based on performing motion estimation of the second image with respect to the first image; and

generating the output image based on performing motion compensation of the second image based on the acquired at least one motion vector.

24. An image processing device, comprising:

a first image acquirer configured to acquire a first image of a first spatial resolution;

a second image acquirer configured to acquire a second image of a second spatial resolution, wherein the second spatial resolution is higher than the first spatial resolution;

an upscaling circuit configured to upscale the first image to the second spatial resolution;

a motion vector acquirer configured to acquire at least one motion vector based on performing motion estimation of the second image with respect to the upscaled first image; and

a generating circuit configured to generate an output image based on performing motion compensation of the second image based on the acquired at least one motion vector.

25. An image processing method, comprising:

acquiring a first image of a first spatial resolution;

acquiring a second image of a second spatial resolution, wherein the second spatial resolution is higher than the first spatial resolution;

upscaling the first image to the second spatial resolution;

acquiring at least one motion vector based on performing motion estimation of the second image with respect to the upscaled first image; and

generating an output image based on performing motion compensation of the second image based on the acquired at least one motion vector.