Image processing apparatus and method, and program and recording medium used therewith

• US 7,221,863 B2
• Filed: 02/02/2005
• Issued: 05/22/2007
• Est. Priority Date: 02/12/2004
• Status: Expired due to Fees

• Current Assignee: Sony Corporation (Sony Group Corp.)
• Original Assignee: Sony Corporation (Sony Group Corp.)
• Inventors: Kondo, Tetsujiro, Tanaka, Kenji, Haga, Tsugihiko
• Primary Examiner(s): PERKEY, WILLIAM B
• Application Number:
  US11/047,751
• Publication Number:
  US 20050190989A1
• Time in Patent Office: 839 Days
• Field of Search: 396/5, 396/51, 396/287, 396/310, 396/311, 396/321, 396/429, 396/89, 396/147, 396/322, 348/207.1, 348/231.3, 348/218.1, 348/239, 348/333.03, 348/345
• US Class Current: 396/5
• CPC Class Codes: G02B 2027/0127 comprising devices increasing the depth of field
• CPC Class Codes: G02B 2027/0138 comprising image capture systems, e.g. camera
• CPC Class Codes: G02B 2027/014 comprising information/image processing systems
• CPC Class Codes: G02B 2027/0187 slaved to motion of at least a part of the body of the user, e.g. head, eye
• CPC Class Codes: G02B 27/0093 with means for monitoring data relating to the user, e.g. head-tracking, eye-tracking
• CPC Class Codes: G02B 27/017 Head mounted
• CPC Class Codes: G06T 15/20 Perspective computation

Abstract

An image processing apparatus displays a realistic image depending on a user'"'"'s viewpoint. A sensor control unit 41 detects a user'"'"'s viewpoint and a point of interest based on signals from various sensors, and outputs the detected viewpoint and point to a depth-of-field adjusting unit. The depth-of-field adjusting unit reads, from the image database, a depth of field set beforehand (or designated each time) and image data of images captured from different positions according to user'"'"'s viewpoints, and outputs the read depth of field and data to an image combining unit. The image combining unit combines a plurality of image data items output from the depth-of-field adjusting unit and displays a combined image on a display. The present invention is applicable to a television apparatus.

Claims

1. An image processing apparatus for displaying an image, comprising:

storage means for storing image data obtained by capturing a plurality of images of a subject from a plurality of positions, together with information of the positions from which the images are captured;

viewpoint position detecting means for detecting a position of a viewpoint of a user viewing said image;

focal position detecting means for detecting a focal position of the viewpoint of the user;

setting means for setting a depth of field of the image to be displayed for the user;

reading means for reading a plurality of image data items from said storage means based on the position of the viewpoint detected by said viewpoint position detecting means, the focal position detected by said focal position detecting means, and the depth of field set by said setting means;

combining means for combining the plurality of image data items read by said reading means; and

display means for displaying a combined image based on the plurality of image data items combined by said combining means.

2. The image processing apparatus according to claim 1, wherein:

the image data stored in said storage means is image data of unblurred images; and

said combining means generates a blurred image corresponding to the depth of field set by said setting means by combining the image data of unblurred images.

3. The image processing apparatus according to claim 1, wherein said focal position detecting means detects an angle of convergence based on a movement of eyes of the user, and detects the focal position based on the detected angle of convergence.

4. The image processing apparatus according to claim 1, wherein a smaller the depth of field is, a more number of image data items said reading means reads from said storage means.

5. An image processing method for an image processing apparatus for displaying an image, said image processing method comprising the steps of:

storing image data obtained by capturing a plurality of images of a subject from a plurality of positions, together with information of the positions from which the images are captured;

detecting a position of a viewpoint of a user viewing said image;

detecting a focal position of the viewpoint of the user;

setting a depth of field of the image to be displayed for the user;

reading a plurality of image data items from the image data stored in the storage step based on the position of the viewpoint detected in the viewpoint position detecting step, the focal position detected in the focal position detecting step, and the depth of field set in the setting step;

combining the plurality of image data items read in the reading step; and

displaying a combined image based on the plurality of image data items combined in the combining step.

6. A recording medium having thereon a recorded program for an image processing apparatus for displaying an image, the program comprising the steps of:

controlling detection of a position of a viewpoint of a user viewing said image;

controlling detection of a focal position of the viewpoint of the user;

controlling setting of a depth of field of the image to be displayed for the user;

controlling reading so that, based on the viewpoint detected in the viewpoint-position-detection control step, the focal position detected in the focal-position-detection control step, and the depth of field set in the setting control step, a plurality of image data items are read from image data obtained by capturing a plurality of images of a subject from a plurality of positions, the image data being stored together with information of the positions from which the images are captured;

controlling combining of the plurality of image data items read in the reading control step; and

controlling display of a combined image based on the plurality of image data items combined in the combining control step.

7. An image processing apparatus comprising:

moving means for moving a subject;

image capturing means for capturing an image of the subject moved by said moving means; and

output means for outputting image data of the image captured by said image capturing means and information of a position of the subject moved by said moving means in a form in which the image data and the information are associated with each other.

8. The image processing apparatus according to claim 7, wherein:

said moving means moves the subject in units of predetermined distances; and

whenever said moving means moves the subject, said image capturing means captures images of the subject from a plurality of positions.

9. An image processing method comprising the steps of:

moving a subject;

capturing an image of the subject moved in the moving step; and

outputting image data of the image captured in the image capturing step and information of a position of the subject moved in the moving step in a form in which the image data and the information are associated with each other.

10. A recording medium having thereon a recorded program to be executed by a computer, the program comprising the steps of:

controlling movement of a subject;

controlling capturing of an image of the subject moved in the movement control step; and

controlling output so that image data of the image captured in the image capturing control step and information of a position of the subject moved in the movement control step are output in a form in which the image data and the information are associated with each other.

11. An image processing apparatus for displaying an image, comprising:

a storage unit configured to store a plurality of image data items obtained by capturing a plurality of images of a subject from a plurality of positions, together with information of the positions from which the images are captured;

a viewpoint position detecting unit configured to detect a position of a viewpoint of a user viewing said image;

a focal position detecting unit configured to set a focal position of the viewpoint of the user;

a setting unit configured to set a depth of field of the image to be displayed for the user;

a reading unit configured to read, from said storage unit, the plurality of image data items based on the position of the viewpoint detected by said viewpoint position detecting unit, the focal position detected by said focal position detecting unit, and the depth of field set by said setting unit;

a combining unit configured to combine the plurality of image data items read from said reading unit; and

a display unit configured to display a combined image based on the plurality of image data items combined by said combining unit.

12. An image processing apparatus comprising:

a moving unit configured to move a subject;

an image capturing unit configured to capture an image of the subject moved by said moving unit; and

an output unit configured to output image data of the image captured by said image capturing unit and information of a position of the subject moved by said moving unit in a form in which the image data and the information are associated with each other.