Image processing device, image processing method, and storage medium

• US 9,836,668 B2
• Filed: 03/17/2015
• Issued: 12/05/2017
• Est. Priority Date: 03/26/2014
• Status: Active Grant

• Current Assignee: Sony Corporation (Sony Group Corp.)
• Original Assignee: Sony Corporation (Sony Group Corp.)
• Inventors: Ikeda, Nobuho, Shimauchi, Kazuhiro, Inaba, Seijiro
• Primary Examiner(s): Hausmann, Michelle M
• Application Number:
  US14/659,855
• Publication Number:
  US 20150279073A1
• Time in Patent Office: 994 Days
• Field of Search: None
• CPC Class Codes: G06F 18/22 Matching criteria, e.g. proximity measures
• CPC Class Codes: G06T 2207/10012 Stereo images
• CPC Class Codes: G06T 3/4038 Image mosaicing, e.g. composing plane images from plane sub-images
• CPC Class Codes: G06T 5/50 using two or more images, e.g. averaging or subtraction
• CPC Class Codes: G06T 5/73 Deblurring; Sharpening
• CPC Class Codes: G06T 7/97 Determining parameters from multiple pictures depth or shape recovery from multiple images G06T7/55; stereo camera calibration G06T7/85

Abstract

There is provided an image processing device including a spatial frequency characteristic adjusting unit configured to perform an adjustment on at least one of first image data corresponding to a first image and second image data corresponding to a second image to match a spatial frequency characteristic of the first image data with a spatial frequency characteristic of the second image data, the second image having an overlapping area that overlaps with an overlapping area of the first image, and an image processing unit configured to perform a process using the first image data and the second image data, on at least one of which the adjustment has been performed in the spatial frequency characteristic adjusting unit.

Claims

1. An image processing device, comprising:

one or more circuitry configured to;

adjust at least one of a first image data corresponding to a first image or a second image data corresponding to a second image to match a first spatial frequency characteristic of the first image data with a second spatial frequency characteristic of the second image data,

wherein the second image has a second overlapping area that overlaps with a first overlapping area of the first image;

adjust at least one of the first image data or the second image data based on a highest spatial frequency among a plurality of frequencies that the first spatial frequency characteristic and the second spatial frequency characteristic have in common; and

process at least one of the adjusted first image data or the adjusted second image data.

2. The image processing device according to claim 1,

wherein the one or more circuitry are further configured to detect a feature amount, in each of the first overlapping area of the first image and the second overlapping area of the second image, based on the adjusted first image data and the adjusted second image data, and process the first overlapping area and the second overlapping area based on the feature amount.

3. The image processing device according to claim 1,

wherein the one or more circuitry are further configured to filter the first image data and the second image data based on one of a low-pass filter or a band-pass filter.

4. The image processing device according to claim 1,

wherein the one or more circuitry are further configured to:

detect the first spatial frequency characteristic of the first image data and the second spatial frequency characteristic of the second image data; and

match the first spatial frequency characteristic of the first image data and the second spatial frequency characteristic of the second image data with a third spatial frequency characteristic obtained based on the detection of the first spatial frequency characteristic of the first image data and the second spatial frequency characteristic of the second image data.

5. The image processing device according to claim 1,

wherein the one or more circuitry are further configured to:

determine a positional disagreement between the first overlapping area of the first image and the second overlapping area of the second image based on the adjusted first image data and the adjusted second image data; and

combine the first image data with the second image data based on the positional disagreement to generate panoramic image data.

6. The image processing device according to claim 1,

wherein the one or more circuitry are further configured to determine, that an identical object is present in the first overlapping area of the first image and the second overlapping area of the second image, based on the adjusted first image data and the adjusted second image data.

7. The image processing device according to claim 1,

wherein the first image data is left-eye image data and the second image data is right-eye image data, and

wherein the one or more circuitry are further configured to:

obtain a positional disagreement between the first overlapping area of the first image and the second overlapping area of the second image, based on the adjusted first image data and the adjusted second image data; and

disparity adjust the first image data and the second image data based on the positional disagreement.

8. An image processing method, comprising:

adjusting at least one of a first image data corresponding to a first image or a second image data corresponding to a second image to match a first spatial frequency characteristic of the first image data with a second spatial frequency characteristic of the second image data,

wherein the second image has a second overlapping area that overlaps with a first overlapping area of the first image, and

wherein at least one of the first image data or the second image data is adjusted based on a highest spatial frequency among a plurality of frequencies that the first spatial frequency characteristic and the second spatial frequency characteristic have in common; and

processing at least one of the adjusted first image data or the adjusted second image data.

9. The image processing method according to claim 8,

wherein the processing of the at least one of the adjusted first image data or the adjusted second image data includes

detecting a feature amount, in each of the first overlapping area of the first image and the second overlapping area of the second image, based on the adjusted first image data and the adjusted second image data.

10. The image processing method according to claim 8,

wherein the adjustment includes filtering the first image data and the second image data based on one of a low-pass filter or a band-pass filter.

11. The image processing method according to claim 8,

wherein the adjustment includes detecting the first spatial frequency characteristic of the first image data and the second spatial frequency characteristic of the second image data, and matching the first spatial frequency characteristic of the first image data and the second spatial frequency characteristic of the second image data with a third spatial frequency characteristic obtained based on a result obtained by detecting the first spatial frequency characteristic of the first image data and the second spatial frequency characteristic of the second image data.

12. The image processing method according to claim 8,

wherein the processing of the at least one of the adjusted first image data or the adjusted second image data includes:

obtaining a positional disagreement between the first overlapping area of the first image and the second overlapping area of the second image based on the adjusted first image data and the adjusted second image data; and

combining the first image data with the second image data based on the positional disagreement to generate panoramic image data.

13. The image processing method according to claim 8,

wherein the processing of the at least one of the adjusted first image data or the adjusted second image data includes

determining that an identical object is present in the first overlapping area of the first image and the second overlapping area of the second image, based on the adjusted first image data and adjusted second image data.

14. The image processing method according to claim 8,

wherein the first image data is left-eye image data, and the second image data is right-eye image data, and

wherein the processing of the at least one of the adjusted first image data or the adjusted second image data includes:

obtaining a positional disagreement between the first overlapping area of the first image and the second overlapping area of the second image, based on the adjusted first image data and the adjusted second image data; and

disparity adjusting the first image data and the second image data based on the positional disagreement.

15. A non-transitory computer-readable medium storing computer-readable instructions thereon, which, when executed by a computer, cause the computer to execute operations, the operations comprising:

adjusting at least one of a first image data corresponding to a first image or a second image data corresponding to a second image to match a first spatial frequency characteristic of the first image data with a second spatial frequency characteristic of the second image data,

wherein the second image has a second overlapping area that overlaps with a first overlapping area of the first image, and

wherein at least one of the first image data or the second image data is adjusted based on a highest spatial frequency among a plurality of frequencies that the first spatial frequency characteristic and the second spatial frequency characteristic have in common; and

processing at least one of the adjusted first image data or the adjusted second image data.

16. The non-transitory computer-readable medium according to the claim 15,

wherein the processing of the at least one of the adjusted first image data or the adjusted second image data includes

detecting a feature amount, in each of the first overlapping area of the first image and the second overlapping area of the second image, based on the adjusted first image data and the adjusted second image data.

17. The non-transitory computer-readable medium according to claim 15, wherein the adjustment includes filtering of the first image data and the second image data based on one of a low-pass filter or a band-pass filter.

18. The non-transitory computer-readable medium according to claim 15,

wherein the adjustment includes detecting the first spatial frequency characteristic of the first image data and the second spatial frequency characteristic of the second image data, and matching the first spatial frequency characteristic of the first image data and the second spatial frequency characteristic of the second image data with a third spatial frequency characteristic obtained based on a result obtained by detecting the first spatial frequency characteristic of the first image data and the second spatial frequency characteristic of the second image data.

19. The non-transitory computer-readable medium according to claim 15,

wherein the processing of the at least one of the adjusted first image data or the adjusted second image data includes:

obtaining a positional disagreement between the first overlapping area of the first image and the second overlapping area of the second image based on the adjusted first image data and the adjusted second image data; and

combining the first image data with the second image data based on the positional disagreement to generate panoramic image data.

20. The non-transitory computer-readable medium according to claim 15, wherein the processing of the at least one of the adjusted first image data or the adjusted second image data includes

determining that an identical object is present in the first overlapping area of the first image and the second overlapping area of the second image, based on the adjusted first image data and the adjusted second image data.

21. The non-transitory computer-readable medium according to claim 15,

wherein the first image data is left-eye image data, and the second image data is right-eye image data, and

wherein the processing of the at least one of the adjusted first image data or the adjusted second image data includes:

obtaining a positional disagreement between the first overlapping area of the first image and the second overlapping area of the second image, based on the adjusted first image data and the adjusted second image data; and

disparity adjusting the first image data and the second image data based on the positional disagreement.