Image processing device and image processing method

• US 9,082,183 B2
• Filed: 06/05/2012
• Issued: 07/14/2015
• Est. Priority Date: 06/08/2011
• Status: Expired due to Fees

• Current Assignee: Sovereign Peak Ventures, LLC (Dominion Harbor Enterprises, LLC)
• Original Assignee: Panasonic Intellectual Property Management Co., Ltd. (Panasonic Holdings Corporation)
• Inventors: Isogai, Kuniaki, Misaki, Masayuki, Tagawa, Junichi, Kawamura, Takashi, Fujii, Takashi
• Primary Examiner(s): Le, Vu
• Assistant Examiner(s): WOLDEMARIAM, AKLILU K
• Application Number:
  US13/814,259
• Publication Number:
  US 20130136342A1
• Time in Patent Office: 1,134 Days
• Field of Search: 382/154, 382/282, 382/284, 382/285, 382/103, 382/153, 382/162, 382/164, 382/167, 382/209, 345/419, 345/424, 345/653
• US Class Current: 1/1
• CPC Class Codes: G06F 18/00 Pattern recognition
• CPC Class Codes: G06T 15/205 Image-based rendering
• CPC Class Codes: G06T 3/00 Geometric image transformations in the plane of the image
• CPC Class Codes: G06T 7/50 Depth or shape recovery
• CPC Class Codes: H04N 13/111 Transformation of image signals corresponding to virtual viewpoints, e.g. spatial image interpolation
• CPC Class Codes: H04N 13/128 Adjusting depth or disparity

Abstract

A data input unit is configured to receive an input image, depth data and a shooting parameter. A parameter input unit receives a transformation parameter as a parameter on projective transformation of a three-dimensional model, a transformed image generating unit that generate a transformed image by performing projective transformation based on the transformation parameter in the three-dimensional model obtained from the input image, the depth data, and the shooting parameter. A blank area detecting unit detects a blank area in the transformed image, the blank area being a group of blank pixels having no corresponding pixel in the input image, and an output unit is configured to output the transformed image in the case where a blank value indicating size of the blank area is smaller than or equal to a threshold value.

Claims

1. An image processing device comprising:

a data input unit configured to receive an input image, depth data indicating depth of the input image, and a shooting parameter of the input image;

a parameter input unit configured to receive a transformation parameter as a parameter on projective transformation of a three-dimensional model;

a transformed image generating unit configured to generate a transformed image by performing projective transformation based on the transformation parameter in the three-dimensional model obtained from the input image, the depth data, and the shooting parameter;

a blank area detecting unit configured to detect a blank area in the transformed image, the blank area being a group of blank pixels that do not correspond to any pixel in the input image; and

an output unit configured to output the transformed image in the case where a blank value indicating size of the blank area is smaller than or equal to a threshold value, and not to output the transformed image in the case where the blank value is larger than the threshold value,

wherein the blank value is a value weighted to increase the blank value of the blank area as the position of the blank area is closer to a center of the transformed image.

2. The image processing device according to claim 1,

wherein the blank area detecting unit configured to detect a plurality of blank areas, each of the blank areas being configured of a group of adjacent blank pixels, and

the output unit is configured to output the transformed image in the case where a largest blank value among blank values of the plurality of blank areas is smaller than or equal to the threshold value.

3. The image processing device according to claim 1,

wherein the blank value is a value weighted to increase the blank value of the blank area as an aspect ratio of the blank area is smaller.

4. The image processing device according to claim 1, further comprising

a complementing unit configured to complement pixel values of pixels of the blank area on the basis of pixel values of pixels near the blank area,

wherein the output unit is configured to output the transformed image with the complemented pixel values of the pixels of the blank area in the case where the blank value is smaller than or equal to the threshold value.

5. The image processing device according to claim 1,

wherein the parameter input unit is configured to receive an instruction from a user to change the view point with respect to the input image, as the transformation parameter, and

the transformed image generating unit performs projective transformation based on the view point identified according to the transformation parameter to generate the transformed image.

6. The image processing device according to claim 1,

wherein the output unit is further configured to output information for requesting inputting of another transformation parameter in the case where the blank value is larger than the threshold value.

7. The image processing device according to claim 6,

wherein the output unit is configured to output information for highlighting an object for receiving an instruction from a user to initialize the transformation parameter, as the information for requesting inputting of the another transformation parameter, in the case where the blank value is larger than the threshold value.

8. The image processing device according to claim 1,

wherein, the output unit is further configured to output the transformed image previously outputted, in the case where the blank value is larger than the threshold value.

9. The image processing device according to claim 1, further comprising

a parameter calculating unit configured to calculate an interpolated transformation parameter by interpolating between the transformation parameter corresponding to the input image and the transformation parameter received by the parameter input unit, in the case where the blank value is larger than the threshold value,

wherein the transformed image generating unit further generates a new transformed image by performing projective transformation based on the interpolated transformation parameter thus calculated.

10. The image processing device according to claim 9,

wherein the parameter calculating unit is configured to calculate a sequence of the interpolated transformation parameters that in the sequence gradually become closer from the transformation parameter received by the parameter input unit to the transformation parameter corresponding to the input image until the blank value becomes smaller than or equal to the threshold value,

the transformed image generating unit is configured to generate a sequence of the transformed images on the basis of the sequence of the interpolated transformation parameters, and

the output unit is configured to output the sequence of the transformed images.

11. The image processing device according to claim 1,

wherein the output unit is further configured to output information indicating a relationship between the blank value and the threshold value.

12. The image processing device according to claim 1,

wherein the image processing device is configured as an integrated circuit.

13. An image processing method comprising:

receiving an input image, depth data indicating depth of the input image, and a shooting parameter of the input image;

receiving a transformation parameter as a parameter on projective transformation of a three-dimensional model;

performing projective transformation based on the transformation parameter in the three-dimensional model obtained from the input image, the depth data, and the shooting parameter to generate a transformed image;

detecting a blank area in the transformed image, the blank area being a group of blank pixels that do not correspond to any pixel in the input image; and

outputting the transformed image in the case where a blank value indicating size of the blank area is smaller than or equal to a threshold value, and not to output the transformed image in the case where the blank value is larger than the threshold value,

wherein the blank value is a value weighted to increase the blank value of the blank area as the position of the blank area is closer to a center of the transformed image.

14. A computer-readable non-transitory recording medium having a program recorded thereon for causing a computer to perform the image processing method according to claim 13.