Image pickup device and image synthesizing method

• US 20050036044A1
• Filed: 08/12/2004
• Published: 02/17/2005
• Est. Priority Date: 08/14/2003
• Status: Active Grant

• Current Assignee: Fujifilm Corporation (Fujifilm Holdings Corporation)
• Original Assignee: Fuji Photo Film Co Limited (Fujifilm Holdings Corporation)
• Inventors: Funakura, Hiroyuki
• Granted Patent: US 7,440,013 B2
• Time in Patent Office: Days
• US Class Current: 348/239
• CPC Class Codes: G06T 11/60 Editing figures and text; Combining figures or text
• CPC Class Codes: G06T 9/001 Model-based coding, e.g. wire frame
• CPC Class Codes: G06V 40/161 Detection; Localisation; Normalisation
• CPC Class Codes: H04M 2250/52 including functional features of a camera constructional features concerning the integration of cameras in portable phones H04M1/0264
• CPC Class Codes: H04N 19/27 involving both synthetic and natural picture components, e.g. synthetic natural hybrid coding [SNHC]
• CPC Class Codes: H04N 2007/145 Handheld terminals
• CPC Class Codes: H04N 23/45 for generating image signals from two or more image sensors being of different type or operating in different modes, e.g. with a CMOS sensor for moving images in combination with a charge-coupled device [CCD] for still images
• CPC Class Codes: H04N 23/611 where the recognised objects include parts of the human body
• CPC Class Codes: H04N 23/951 by using two or more images to influence resolution, frame rate or aspect ratio
• CPC Class Codes: H04N 5/272 Means for inserting a foreground image in a background image, i.e. inlay, outlay
• CPC Class Codes: H04N 7/147 Communication arrangements, e.g. identifying the communication as a video-communication, intermediate storage of the signals selecting H04Q

Abstract

An image pickup device is constituted by a handset of cellular phone, wherein an image pickup unit photographs a user as first object, to output image data of a first image frame. A facial region detecting circuit retrieves a facial image portion of the user as first object according to image data of the first image frame. An image synthesizing circuit is supplied with image data of a second image frame including persons as second object. The facial image portion is synthesized in a background region being defined outside the persons as second object within the second image frame.

Claims

1. An image pickup device for photographing an object, comprising:

a facial region detector for retrieving at least a facial region of a first object in a first image in which said first object is a principal object thereof; and

an image synthesizing unit for creating a synthesized image by combining a second image with said facial region from said facial region detector, said second image having a second object constituting a principal object thereof, wherein said facial region is disposed in a background region beside said second object within said second image.

2. An image pickup device as defined in claim 1, further comprising a display panel for displaying said synthesized image.

3. An image pickup device as defined in claim 2, further comprising a region determining unit for determining said background region of said second image, wherein said image synthesizing unit creates said synthesized image according to determined information from said region determining unit.

4. An image pickup device as defined in claim 3, wherein said image synthesizing unit trims said facial region from said first image by positioning a predetermined combining pattern thereon before image synthesis.

5. An image pickup device as defined in claim 4, wherein said facial region detector detects an eye position from said facial region, and said image synthesizing unit positions said combining pattern in consideration of said eye position.

6. An image pickup device as defined in claim 5, wherein said eye position is a mid point of two eyes of said facial region, and a center of said combining pattern is set at said mid point.

7. An image pickup device as defined in claim 5, further comprising a pattern memory for storing information of plural combining patterns adapted to selection of one thereof.

8. An image pickup device as defined in claim 3, wherein said image synthesizing unit further enlarges or reduces size of said facial region to be synthesized.

9. An image pickup device as defined in claim 3, further comprising:

a first image pickup unit for photographing an image pickup device user constituting said first object in a self photographing manner; and

a second image pickup unit for photographing said second object disposed in an external field in front of said first object.

10. An image pickup device as defined in claim 9, wherein said display panel displays said first and second images simultaneously according to outputs of said first and second image pickup units.

11. An image pickup device as defined in claim 10, further comprising a shutter button for driving said first and second image pickup units simultaneously to obtain said first and second images.

12. An image pickup device as defined in claim 3, wherein said image synthesizing unit refers to chromaticity or brightness of one of said first and second images, and adjusts chromaticity or brightness of a remaining one of said first and second images according to said one image, before image synthesis.

13. An image pickup device as defined in claim 3, wherein said image synthesizing unit combines said first image of said first object being photographed with a live image of said second object while said second object is framed for still photography, and causes said display panel to display said synthesized image.

14. An image pickup device as defined in claim 13, wherein said display panel is caused to display said first image of said first object at a low density or low sharpness.

15. An image pickup device as defined in claim 13, wherein said display panel displays a combining pattern in said live image, and displays at least one portion of said first object in said combining pattern.

16. An image pickup device as defined in claim 3, further comprising an image pickup unit for photographing said first and second objects to create said first and second images.

17. An image pickup device as defined in claim 3, further comprising:

an image pickup unit for photographing one of said first and second objects; and

a communication unit for receiving data of said first or second image of a remaining one of said first and second objects from an external electronic device.

18. An image pickup device as defined in claim 17, wherein said communication unit is wireless.

19. An image pickup device as defined in claim 17, wherein said communication unit is for connection with a public communication cable.

20. An image synthesizing method, comprising steps of:

retrieving at least a facial region of a first object from a first image in which said first object is recorded; and

combining said facial region being retrieved with a background region within a second image in which a second object is recorded.

21. An image synthesizing method as defined in claim 20, wherein at least one image pickup device is used to photograph said first and second objects.

22. An image synthesizing method as defined in claim 20, wherein said facial region is trimmed from said second image according to a predetermined combining pattern.

23. An image synthesizing method as defined in claim 22, further comprising steps of:

detecting an eye position in said facial region; and

positioning said combining pattern in consideration of said eye position.

24. An image synthesizing method as defined in claim 23, wherein said eye position is a mid point of two eyes of said facial region, and a center of said combining pattern is set at said mid point.

25. An image synthesizing method as defined in claim 21, wherein said image pickup device includes:

a first image pickup unit for photographing an image pickup device user constituting said first object in a self photographing manner;

a second image pickup unit for photographing said second object disposed in an external field in front of said first object;

a shutter button for driving said first and second image pickup units simultaneously to obtain said first and second images.

26. An image synthesizing method as defined in claim 20, further comprising steps of:

obtaining a first height of said first object in said first image, and a second height of said second object in said second image according to image analysis; and

enlarging or reducing a size of said facial region according to a height ratio related to said first or second height of said first or second image before image synthesis.

27. An image synthesizing method as defined in claim 20, further comprising steps of:

obtaining a first height ratio of a first height of said first object in said first image to a height of a reference photographed object;

obtaining a second height ratio of a second height of said second object in said second image to a height of said reference photographed object;

dividing one of said first and second height ratios by a remaining one of said first and second height ratios, to obtain a correcting coefficient for said first and second objects;

enlarging or reducing a size of said facial region by multiplication of said correcting coefficient before image synthesis.

28. An image synthesizing method as defined in claim 20, further comprising steps of:

displaying a live image of said object to be photographed on a display panel; and

displaying said at least facial region of said first object being photographed on said display panel by synthesis with said live image.

29. An image synthesizing method as defined in claim 20, wherein chromaticity or brightness of one of said first and second images is referred to, and chromaticity or brightness of at least one of said first and second images is adjusted according to said one image, before image synthesis.

30. An image synthesizing method as defined in claim 20, wherein one of said first and second images is photographed by a first image pickup device;

and

a remaining one of said first and second images is input to said first image pickup device by a communication unit.

31. An image synthesizing method as defined in claim 30, wherein image data of said second image, being photographed in a second image pickup device, is transmitted to said first image pickup device for photographing said first image by use of a communication unit.

32. An image synthesizing method as defined in claim 20, further comprising steps of:

photographing said first and second images by use of respectively first and second image pickup devices; and

transmitting image data of said first and second images from said first and second image pickup devices to an image synthesizing electronic device by use of a communication unit.

33. An image synthesizing method as defined in claim 20, wherein said at least facial region is constituted by a human body region.