Digital camera and method for compositing images

• US 7,221,395 B2
• Filed: 03/14/2001
• Issued: 05/22/2007
• Est. Priority Date: 03/14/2000
• Status: Expired due to Term

• Current Assignee: Fuji Photo Film Co Limited (Fujifilm Holdings Corporation)
• Original Assignee: Fuji Photo Film Co Limited (Fujifilm Holdings Corporation)
• Inventors: Kinjo, Naoto
• Primary Examiner(s): Ho; Tuan
• Assistant Examiner(s): YODER III, CHRISS S
• Application Number:
  US09/805,582
• Publication Number:
  US 20020001036A1
• Time in Patent Office: 2,260 Days
• Field of Search: 348/231.3, 348/584, 348/598, 348/586, 348/42, 348/239, 348/36, 348/39, 348/231.6, 348/333.01, 348/635, 348/637, 345/634, 345/635, 345/637, 345/629, 382/293, 382/294, 382/284
• US Class Current: 348/239
• CPC Class Codes: H04N 1/3872 Repositioning or masking
• CPC Class Codes: H04N 13/218 using spatial multiplexing
• CPC Class Codes: H04N 13/221 using the relative movement between cameras and objects
• CPC Class Codes: H04N 13/239 using two 2D image sensors having a relative position equal to or related to the interocular distance H04N13/243 takes precedence
• CPC Class Codes: H04N 13/286 having separate monoscopic and stereoscopic modes
• CPC Class Codes: H04N 13/305 using lenticular lenses, e.g. arrangements of cylindrical lenses
• CPC Class Codes: H04N 23/632 for displaying or modifying preview images prior to image capturing, e.g. variety of image resolutions or capturing parameters

Abstract

The digital camera includes an image pickup unit, an image storing unit, an image display unit, a reference image designating unit with which one or more of at least partial areas of an image to be referenced for image compositing are designated as a reference image area and an image compositing unit which produces a composite image such that a reference image within the reference image area is displayed as superposed on an image being presently photographed. The image processing method attaches first identification information to one or more of at least partial areas of the specified image and designated area information upon photographing, attaches second identification information to an image in an shooting frame to be composited with the reference image and prepares image data for a composite image obtained by compositing the image with the reference image based on the first and second identification information as well as the designated area information.

Claims

1. A digital camera, comprising:

an image pickup unit for imaging a subject to obtain second digital image data of a photographed image;

an image storing unit for storing said second digital image data of said photographed image that is obtained with said image pick up unit and first digital image data of a first image to be referenced for image compositing, or display image data of said first image having a lower resolution than said first digital image data;

an image display unit for displaying a second image being presently photographed which is not yet obtained as said photographed image, based on display image data which has a lower resolution than said second digital image data and which is not stored in said image storing unit and said first image based on said display image data having a lower resolution, which is obtained from said first digital image data read out of said image storing unit or which is directly read out of said image storing unit;

a reference image designating unit with which one or more of partial areas in said first image that is displayed on said image display unit are selected and designated as a reference image area on said image display unit; and

an image compositing unit which produces a composite image for display such that a reference image within said reference image area is displayed on said image display unit as superposed on said second image,

wherein, after said composite image for display displayed on said image display unit is confirmed, said image pick up unit photographs said second image as said photographed image and obtains said second digital image data thereof, and

said image storing unit stores the thus obtained second digital image data of said photographed image to which a second identification information indicating that said photographed image is to be composited is attached, as well as

said image storing unit stores said first digital image data or said display image data of said first image, to which a first identification information indicating that said first image is to be composited and a designated area information indicating that said designated one or more of partial areas are said reference image area designated by said reference image designating unit.

2. The digital camera according to claim 1, wherein said first digital image data of said first image is either said second digital image data of said photographed image or said first digital image data of a specified image to be quoted, and wherein said display image data of said first image is said display image data of said specified image to be quoted having a lower resolution than said first digital image data of said specified image to be quoted.

3. The digital camera according to claim 1, wherein said reference image area is displayed in a specified position, a position on said image to be referenced for image compositing or a designated position on said image display unit.

4. The digital camera according to claim 1, wherein said reference image is processed by at least one processing step selected from the group consisting of translation, rotation, resizing, density/color retouching, binarization, edge enhancement, change in painting brushwork and change in light transmittance.

5. The digital camera according to claim 1, wherein said reference image being displayed on said image display unit is automatically enlarged or reduced in accordance with a magnification of an image being presently photographed.

6. The digital camera according to claim 1, wherein said reference image designating unit performs designation of said reference image area by designating one or more of partial areas of said first image that is displayed on said image display unit.

7. The digital camera according to claim 1, wherein said image compositing unit produces said composite image for display such that said reference image within said reference image area in said first image is displayed on said image display unit as superposed on said second image at a specified light transmittance.

8. The digital camera according to claim 1, further comprising a camera control unit for performing control upon photographing such that a principal subject in said reference image and a principal subject in said image being presently photographed are equal to each other in density and color tint.

9. A digital camera comprising:

an image pickup unit for imaging a subject to obtain digital image data;

an image storing unit for storing at least said digital image data of a photographed image that is obtained with said image pick up unit;

an image display unit for displaying at least an image being presently photographed;

a reference image designating unit with which one or more of partial areas in an image to be referenced for image compositing that is displayed on said image display unit are selected and designated as a reference image area on said image display unit; and

an image compositing unit which produces a composite image such that a reference image within said reference image area is displayed on said image display unit as superposed on the image being presently photographed,

said digital camera further including a function to focus on a large number of rangefinding points, wherein a focused area of the subject is allowed to be automatically clipped out as said reference image.

10. A digital camera, comprising:

an image pickup unit for imaging a subject to obtain digital image data;

an image storing unit for storing at least said digital image data of a photographed image that is obtained with said image pick up unit;

an image display unit for displaying at least an image being presently photographed;

a reference image designating unit with which one or more of partial areas in an image to be referenced for image compositing that is displayed on said image display unit are selected and designated as a reference image area on said image display unit; and

an image compositing unit which produces a composite image such that a reference image within said reference image area is displayed on said image display unit as superposed on the image being presently photographed,

said digital camera further including a stereophotographic mode, wherein, if set to said stereophotographic mode, an area in which a focal distance is at infinity is clipped out automatically as said reference image.

11. An image processing method, comprising steps of:

photographing a subject to acquire second digital image data for a photographed image in a shooting frame with a camera; and

assembling it with at least part of first digital image data for a specified first image to be referenced for image compositing to prepare third digital image data for a composite image;

further comprising the steps of;

previously storing in a storing unit of said camera said first digital image data for said specified first image or display image data for said specified first image having a lower resolution than said first digital image data;

upon photographing, designating in said image display unit of said camera selected one or more of at least partial areas in said specified first image displayed based on said display image data which is obtained from said first digital image data read out of said storing unit or which is directly read out of said storing unit as a reference image area;

displaying a composite image for display which is produced by superposing said reference image within said reference image area on a second image being presently photographed on said image display unit;

after said composite image for display displayed on said display unit has been confirmed, photographing said second image to acciuire said photographed image in said shooting frame with said camera or further to accjuire said second digital image data thereof;

attaching in said camera to said specified first image first identification information indicating that said specified first image is to be composited, and designated area information indicating that said designated one or more of at least partial areas are said reference image area of a reference image to be composited and storing said first digital image data or said display image data of said specified first image, together with said first identification information and said designated area information; and

attaching in said camera to said photographed image in said shooting frame which is to be composited with said reference image or said second digital image data thereof second identification information indicating that said photographed image in the shooting frame is to be composited and storing in said storing unit said second identification information, or said second digital image data and said second identification information;

as well as

upon image outputting, reading out of said storing unit of said camera said first digital image data or said display image data for said specified first image, said first identification information and said reference image area, as well as said second identification information, or said second digital image data and said second identification information;

further, in case where said second digital image data and/or said first digital image data is not read out of said storing unit of said camera, separately obtaining said second digital image data from said photographed image in said shooting frame and/or said first digital image data which is previously prepared for said specified first image; and

preparing said third digital image data for said composite image obtained by compositing said photographed image in the shooting frame with said reference image based on said first and second identification information, said designated area information, as well as said first and second digital image data.

12. The image processing method according to claim 11, wherein,

upon photographing,

in addition to said first and second identification information, said storing unit in said camera stores processing information which refers to what processing step is to be performed or light transmittance information upon compositing which represents a specified light transmittance for use when compositing said reference image within said reference image area in said specified first image with said photographed image,

upon image outputting,

in addition to said first and second identification information, said processing information or said light transmittance information upon compositing is read out of said storing unit of said camera;

and

said third digital image data for the composite image is prepared based on said first and second identification information, said designated area information and said processing information or light transmittance information upon compositing, as well as said first and second digital image data.

13. The image processing method according to claim 11, wherein,

upon photographing,

in addition to said first and second identification information, said storing unit in said camera stores information about order of image compositing,

upon image outputting,

in addition to said first and second identification information, said information about order of image compositing is read out of said storing unit of said camera;

and

said information about order of image compositing in addition to said first and second identification information and said designated area information, as well as said first and second digital image data is used to prepare said third digital image data for the composite image.

14. The image processing method according to claim 11, further including a stereophotographic mode, wherein, if said stereophotographic mode is set, image data for a stereoscopic image is prepared after any positional or angular offset between image areas in which a focal distance is at infinity on frames which are to be used in the stereophotographic mode is optionally corrected.

15. The image processing method according to claim 11, wherein said third digital image data for the composite image is used for producing a composite print, recorded on an image data recording medium and delivered through a telecommunication network.

16. The image processing method according to claim 11, wherein camera control is further performed upon photographing such that a principal subject in said reference image and a principal subject in said image being presently photographed are equal to each other in density and color tint.

17. The image processing method according to claim 11, wherein adjustment is further performed when image compositing upon said image outputting such that a principal subject in said reference image and a principal subject in said image being presently photographed are equal to each other in density and color tint.

18. The image processing method according to claim 11, wherein said reference image or said image to be composited with the reference image is a motion image.

19. The digital camera according to claim 11, wherein said image compositing unit produces said composite image for display such that said reference image within said reference image area in said image to be referenced for image compositing is displayed on said image display unit as superposed on said image being presently photographed at a specified light transmittance.

20. An image processing method, comprising steps of:

photographing a subject to acciuire digital image data with a camera; and

assembling it with at least part of image data for a specified image to be referenced for image compositing to prepare image data for a composite image;

further comprising the steps of;

upon photographing, designating in said camera selected one or more of at least partial areas in said specified image to be referenced for image compositing as a reference image area;

attaching in said camera to said specified image to be referenced for image compositing first identification information indicating that said specified image to be referenced for image compositing is to be composited. and designated area information indicating that said designated one or more of at least partial areas are said reference image area of a reference image to be composited; and

attaching in said camera to an image in a shooting frame which is to be composited with said reference image second identification information indicating that said image in the shooting frame is to be composited;

as well as

upon image outputting, preparing image data for a composite image obtained by compositing said image in the shooting frame with said reference image based on said first and second identification information as well as said designated area information,

 

wherein information about a large number of rangefinding points is further obtained and a focused area of the subject is allowed to be automatically clipped out as said reference image to prepare the third digital image data for the composite image.

21. An image processing method, comprising steps of:

photographing a subject to acquire second digital image data for a photographed image in a shooting frame with a camera; and

assembling it with at least part of first digital image data for a specified first image to be referenced for image compositing to prepare third digital image data for a composite image, further comprising the steps of;

previously storing in a storing unit of said camera said first digital image data for said specified first image or display image data for said specified first image having a lower resolution than said first digital image data;

upon photographing,

designating in said image display unit of said camera selected one or more of at least partial areas in said specified first image displayed based on said display image data which is obtained from said first digital image data read out of said storing unit or which is directly read out of said storing unit as a reference image area;

displaying a composite image for display which is produced by superposing said reference image within said reference image area on a second image being presently photographed on said image display unit;

after said composite image for display displayed on said display unit has been confirmed, photographing said second image to accluire said photographed image in said shooting frame with said camera or further to acciuire said second digital image data thereof;

preparing in said camera editing information including information about a name or a frame number of said specified first image that is to be composited, designated area information representing said designated one or more of at least partial areas are said reference image area of a reference image to be composited, and information about a name or a frame number of said photographed image in said shooting frame to be composited with said reference image;

storing in said storing unit of said camera said editing information and said first digital image data or said display image data for said specified first image, further or said second digital image data for said photographed image in said shooting frame;

as well as

upon image outputting,

reading out of said storing unit in said camera said first digital image data or said display image data for said specified first image, and said editing information, or said editing information and said second digital image data;

further, in case where said second digital image data and/or said first digital image data is not read out of said storing unit of said camera, separately obtaining said second digital image data from said photographed image in said shooting frame and/or said first digital image data which is previously prepared for said specified first image; and

preparing said third digital image data for said composite image obtained by compositing said photographed image in the shooting frame with said reference image based on said editing information, as well as said first and second digital image data.

22. The image processing method according to claim 21, wherein said editing information further includes processing information which shows what processing step is to be performed to a designated area by said designated area information.

23. The image processing method according to claim 21, wherein said editing information further includes information about order of image compositing or light transmittance information upon compositing which represents a specified light transmittance for use in image compositing.

24. The digital camera according to claim 21, wherein said image compositing unit produces said composite image for display such that said reference image within said reference image area in said image to be referenced for image compositing is displayed on said image display unit as superposed on said image being presently photographed at a specified light transmittance.

25. An image processing method, comprising steps of:

photographing a subject to acquire second digital image data of each photographed image with each of a plurality of cameras; and

assembling it with at least part of first digital image data for a specified first image to be referenced for image compositing to prepare third digital image data for each composite image for each of said plurality of cameras;

further comprising the steps of;

previously storing in a storing unit in at least one of said plurality of cameras said first digital image data for said specified first image or display image data for said specified first image having a lower resolution than said first digital image data;

upon photographing,

designating at least one reference image within at least one partial area in said specified first image displayed on said image display unit of at least one of said plurality of cameras based on said display image data which is obtained from said first digital image data read out of said storing unit or which is directly read out of said storing unit with said at least one of said plurality of cameras;

attaching reference image designation data to the reference image in said at least one of said plurality of cameras and storing in said storing unit said first digital image data for the reference image or said display image data, as well as said reference image designation data;

sending and receiving said first digital image data or said display image data for said designated at least one reference image among said plurality of cameras;

displaying on respective display units in said plurality of cameras composite images for display which are produced by superposing said reference image on respective second images to be composited being presently photographed with said plurality of cameras, respectively;

after said composite images for display displayed on said respective display units in said plurality of cameras have been confirmed, respectively, photographing said respective second images to acciuire photographed images with said plurality of cameras, respectively, or further to acquire respective second digital image data thereof, respectively; and

attaching respectively in said plurality of cameras to said photographed images respectively photographed with said plurality of cameras which are to be composited with the reference image group identification information indicating that said photographed images belong to a unique group and storing in said respective storing units of said plurality of cameras said group identification information, or said respective second digital image data of said photographed images and said group identification information, as well as

on image outputting,

reading out of said storing unit in at least one of said plurality of cameras said first digital image data or said display image data for said reference image and said reference image designation data, as well as reading out of said respective storing units of said plurality of cameras said group identification information, or said respective second digital image data of said photographed images and said group identification, respectively;

further, in case where said respective second digital image data of said photographed images is not read out of said respective storing units in said plurality of cameras, respectively and/or said first digital image data is not read out of said storing unit in said at least one of said plurality of cameras, separately obtaining said respective second digital image data of said photographed images from said photographed images in said plurality of cameras, respectively, and/or said first digital image data which is previously prepared for said specified first image; and

compositing the photographed images respectively photographed with said plurality of cameras with said at least one reference image by using said respective second digital imaged data of said photographed images respectively photographed with said plurality of cameras, said first digital image data for said at least one reference image, said reference image designation data and said group identification information to obtain respective composite images.

26. An image processing method, comprising steps of:

photographing a subject to acquire digital image data with a camera; and

assembling it with at least part of image data for a specified image to be referenced for image compositing to prepare image data for a composite image;

further comprising the steps of;

upon photographing, designating in said camera selected one or more of partial areas in said specified image to be referenced for image compositing as a reference image area; and

preparing in said camera editing information including information about a name or a frame number of said specified image to be referenced for image compositing that is to be composited, designated area information representing said designated one or more of at least partial areas are said reference image area of a reference image to be compo sited, and information about a name or a frame number of an image in a shooting frame to be composited with said reference image, as well as upon image outputting, preparing image data for a composite image obtained by compositing said image in the shooting frame with said reference image based on said editing information,

wherein information about a large number of rangefinding points is further obtained and a focused area of the subject is allowed to be automatically clipped out as said reference image to prepare the third digital image data for the composite image.

27. An image processing method, comprising steps of:

photographing a subject to acquire digital image data with each of a plurality of cameras; and

assembling it with image data for a specified image to be referenced for image compositing to prepare image data for a composite image;

further comprising the steps of;

upon photographing, designating at least one reference image within at least one partial area in said specified image to be referenced for image compo siting with at least one of said plurality of cameras;

attaching reference image designation data to the reference image in said at least one of said plurality of cameras;

sending and receiving image data for said designated at least one reference image among said plurality of cameras; and

attaching respectively in said plurality of cameras to photographed images respectively photographed with said plurality of cameras which are to be composited with the reference image group identification information indicating that the photographed images belong to a unique group, as well as

on image outputting, compositing the photographed images respectively photographed with said plurality of cameras with said at least one reference image by using the photographed images respectively photographed with said plurality of cameras, said reference image designation data and said group identification information,

wherein information about a large number of rangefinding points is further obtained and a focused area of the subject is allowed to be automatically clipped out as said reference image to prepare the image data for the composite image.