Image processing device, image processing method, and image processing program

• US RE46,976 E1
• Filed: 08/27/2014
• Issued: 07/31/2018
• Est. Priority Date: 06/06/2007
• Status: Active Grant

• Current Assignee: Sony Corporation (Sony Group Corp.)
• Original Assignee: Sony Corporation (Sony Group Corp.)
• Inventors: Makii, Tatsuo
• Primary Examiner(s): Ge, Yuzhen
• Application Number:
  US14/470,294
• Time in Patent Office: 1,434 Days
• CPC Class Codes: G06T 7/521 from laser ranging, e.g. using interferometry; from the projection of structured light
• CPC Class Codes: G06T 7/571 from focus
• CPC Class Codes: H04N 5/23293 Electronic viewfinders
• CPC Class Codes: H04N 5/2621 Cameras specially adapted for the electronic generation of special effects during image pickup, e.g. digital cameras, camcorders, video cameras having integrated special effects capability
• CPC Class Codes: H04N 5/2628 Alteration of picture size, shape, position or orientation, e.g. zooming, rotation, rolling, perspective, translation

Abstract

An object plane is set, and appropriate image processing is performed according to the setting of the object plane. An image processing apparatus has a controller (7). The controller (7) receives image data for a captured image and distance information on the distance to a designated point in the captured image, sets an object plane (100) containing the designated point, converts the distance information to the information on the distance from the object plane, and performs predetermined image processing according to the converted distance information.

Claims

1. An image processing apparatus comprising:

an image sensor that captures an image;

an object plane setting means for setting an object plane in said captured image based on distance information corresponding to a point position designated by a user in a displayed image based on image data of the captured image;

an object plane changing means for changing the object plane by rotating the object plane at a rotational angle specified by the user with the point position maintained in the object plane;

a distance information conversion means for converting the distance information for at least one position of the image data external to the changed object plane according to the changed object plane using the rotational angle;

and

an image processing means for performing image processing on the image data based on the converted distance information, wherein the image processing comprises;

blurring the image data not on the changed object plane with a blur amount according to the converted distance information, and not blurring the image data on the changed object plane,; and

whereina display control means for displaying a graphical indication of the changing of the object plane is displayed on a display screen with the image data.

2. The image processing apparatus according to claim 1, wherein the distance information conversion means converts is further configured to convert the distance information for each point position of the image data based on the coordinates of the designated point position.

3. The image processing apparatus according to claim 1, wherein the object plane changing means changes is further configured to change the object plane by rotating the object plane on the designated point position as rotation center.

4. The image processing apparatus according to claim 3, wherein, when the object plane is changed by the object plane changing means, the distance information conversion means converts is further configured to convert the distance information for each point position of the image data based on the coordinates of the designated point position and the angle and a direction of the rotation.

5. The image processing apparatus according to claim 1, wherein the image processing means blurs is further configured to blur the image data such that the blur amount increases as the distance from the object plane increases.

6. The image processing apparatus according to claim 1, wherein the image processing means blurs is further configured to blur the image data so that points positions within a predetermined distance from the object plane will be are blurred with the a same blur amount.

7. The image processing apparatus according to claim 1, wherein the image processing means adjusts is further configured to adjust the blur amount based on depth of field information indicating a depth of field.

8. The image processing apparatus according to claim 1, further comprising:

a storage means for storing the image data and the distance information in association with each other;

a designation means for designating the point position; and

a setting means for setting a tilt angle and a tilt direction of the object plane containing the point position with respect to an optical axis of a lens.

9. The image processing apparatus according to claim 8, wherein, when the point position is designated by the designation means and the tilt direction and the tilt angle of the object plane with respect to the optical axis of the lens is set by the setting means, the object plane setting means sets the object plane based on the tilt direction and the tilt angle.

10. The image processing apparatus according to claim 8, further comprising a ranging means for ranging distance, wherein distance information obtained by the ranging means is stored by the storage means in association with the image data.

11. The image processing apparatus according to claim 8, wherein the designation means is a touch panel, and, when a point position is designated in the captured image, the touch panel accepts the a location designation by detecting the coordinates.

12. The image processing apparatus according to claim 11, wherein the location designation by the designation means is performed by touching the touch panel a predetermined number of times.

13. The image processing apparatus according to claim 8, further comprising a wherein the display control means is for displaying the a location designated by the designation means on the a display screen along with the captured image, wherein the designation means accepts the change of the location based on the dragging of the designated point position.

14. The image processing apparatus according to claim 8, wherein the designation means and setting means are a touch panel, and wherein designating at least one point position in the captured image, changing the a designated location, and setting the tilt direction and the tilt angle of the object plane containing the designated point position with respect to the optical axis of the lens are performed by touching the touch panel a number of predetermined times.

15. The image processing apparatus according to claim 1, wherein the distance information conversion means converts the distance information for each given region including two or more pixels in the image data according to the object plane.

16. An image processing method comprising the steps of:

capturing an image with an image sensor;

an image processor setting an object plane in said captured image based on distance information corresponding to a point position designated by a user in a displayed image based on image data of the captured image;

changing the object plane by rotating the object plane at a rotational angle specified by the user with the point position maintained in the object plane, in which the rotational angle is with respect to the object plane;

converting the distance information of the image data according to the changed object plane, wherein the converting the distance information of the image data according to the changed object plane includes, for a given point Pn on the object plane having coordinates Xn, Yn and Zn, subtracting coordinates of the position designed by the user from the coordinates Xn, Yn and Zn of the given point Pn to obtain coordinates Xn′

, Yn′ and

Zn′

, and subtracting, from a first coordinate of the coordinates Xn′

, Yn′ and

Zn′

, a product of a second coordinate of the coordinates Xn′

, Yn′ and

Zn′ and

a tangent of the rotational angle, the first coordinate being different from the second coordinate;

and

performing image processing on the image data based on the converted distance information, wherein the image processing comprises;

blurring the image data of an object not on the changed object plane with a blur amount according to the converted distance information, and not blurring the image data of an object on the changed object plane,; and

whereindisplaying a graphical indication of the changing of the object plane is displayed on a display screen with the image data.

17. An image processing program stored on a A non-transitory computer-readable medium on which is stored an image processing program, the program causing a computer to execute an image processing method comprising the steps of:

capturing an image with an image sensor;

setting an object plane in said captured image based on distance information corresponding to a point position designated by a user in a displayed image based on image data of the captured image;

changing the object plane by rotating the object plane at a rotational angle specified by the user with the point position maintained in the object plane, in which the rotational angle is with respect to the object plane;

converting the distance information of the image data according to the changed object plane, wherein the converting the distance information of the image data according to the changed object plane includes, for a given point Pn on the object plane having coordinates Xn, Yn and Zn, subtracting coordinates of the position designed by the user from the coordinates Xn, Yn and Zn of the given point Pn to obtain coordinates Xn′

, Yn′ and

Zn′

, and subtracting, from a first coordinate of the coordinates Xn′

, Yn′ and

Zn′

, a product of a second coordinate of the coordinates Xn′

, Yn′ and

Zn′ and

a tangent of the rotational angle, the first coordinate being different from the second coordinate;

and

performing image processing on the image data based on the converted distance information, wherein the image processing comprises;

blurring the image data of an object not on the changed object plane with a blur amount according to the converted distance information, and not blurring the image data of an object on the changed object plane,; and

whereindisplaying a graphical indication of the changing of the object plane is displayed on a display screen with the image data.

18. An image processing apparatus comprising:

an image sensor that captures an image;

a processing device that receives the image captured by the image sensor; and

a memory including instructions which, when executed by the processing device, control;

setting an object plane in said captured image based on distance information corresponding to a position designated by a user in a displayed image based on image data of the captured image;

changing the object plane by rotating at a rotational angle specified by the user with the position maintained in the object plane in which the rotational angle is with respect to the object plane;

converting the distance information of the image data according to the changed object plane, wherein the converting the distance information of the image data according to the changed object plane includes, for a given point Pn on the object plane having coordinates Xn, Yn and Zn, subtracting coordinates of the position designed by the user from the coordinates Xn, Yn and Zn of the given point Pn to obtain coordinates Xn′

, Yn′ and

Zn′

, and subtracting, from a first coordinate of the coordinates Xn′

, Yn′ and

Zn′

, a product of a second coordinate of the coordinates Xn′

, Yn′ and

Zn′ and

a tangent of the rotational angle, the first coordinate being different from the second coordinate;

image processing on the image data based on the converted distance information, wherein the image processing comprises;

blurring the image data of an object not on the changed object plane with a blur amount according to the converted distance information, and not blurring the image data of an object on the changed object plane; and

displaying a graphical indication of the changing of the object plane with the image data.