Digital image acquisition system with portrait mode

• US 8,212,897 B2
• Filed: 03/15/2010
• Issued: 07/03/2012
• Est. Priority Date: 12/27/2005
• Status: Active Grant

• Current Assignee: FotoNation Limited (Adeia Inc.)
• Original Assignee: DigitalOptics Corporation Europe Limited (Adeia Inc.)
• Inventors: Steinberg, Eran, Zamfir, Adrian, Capata, Adrian, Corcoran, Peter
• Primary Examiner(s): Chan, Jason
• Assistant Examiner(s): FOSSELMAN, JOEL W
• Application Number:
  US12/724,391
• Publication Number:
  US 20100182458A1
• Time in Patent Office: 841 Days
• Field of Search: 348/222.1, 348/239, 348/208.4
• US Class Current: 348/239
• CPC Class Codes: H04N 5/272 Means for inserting a foreground image in a background image, i.e. inlay, outlay

Abstract

A digital image acquisition system having no photographic film comprises an apparatus for capturing digital images and a flash unit for providing illumination during image capture. The system has a portrait mode for generating an image of a foreground object against a blurred background, the portrait mode being operable to capture first, second and third images (A, B and C) of nominally the same scene. One of the first and second images (A, B) is taken with flash and the other is taken without flash, and the third image (C) is blurred compared to the first and second images. The portrait mode is further operable to determine foreground and background regions of the scene using the first and second images (A, B), and to substitute the blurred background of the third image (C) for the background of an in-focus image of the scene. In one embodiment the in-focus image is one of the first and second images. In another embodiment the in-focus image is a fourth image.

Claims

1. A portable digital image acquisition apparatus having no photographic film (hereinafter “

portable apparatus”

), the portable apparatus comprising a housing that contains an image sensor and a lens for capturing digital images and a flash unit coupled with the housing for providing illumination during image capture, the portable apparatus including a portrait mode for generating in-camera an image of a foreground object against a blurred background, the portable apparatus also including a face detector arranged to detect one or more faces in one or more of a series of digital images being captured and if so to switch to portrait mode, the portrait mode being operable to capture first, second and third images of nominally the same scene, not necessarily in the order stated, at least one of the first and second images being taken with flash, and the third image being blurred compared to the first and second images, the portrait mode further being operable to determine foreground and background regions of the scene using the first and second images, and to substitute the blurred background of the third image for the background of an in focus image of the scene, thereby generating in-camera a digital image of an in-focus foreground object against a blurred background,

wherein the in-focus image comprises one of the first and second images, wherein the first and second images have different pixel resolutions with the in-focus image having the higher resolution, and the apparatus is further configured to determine the foreground and background regions including matching pixel resolutions of the first and second images by at least one of up-sampling the image of lower resolution and sub-sampling the image of higher resolution.

2. A portable apparatus as claimed in claim 1, further being configured to determine the foreground and background regions for aligning two or more of the first, second and third images.

3. A portable apparatus according to claim 1, wherein the third image comprises a pre- or post-view image.

4. A portable apparatus according to claim 1, wherein said digital image acquisition system comprises a digital camera.

5. A portable apparatus according to claim 1, wherein said digital image acquisition system is a combination of a digital camera and an external processing device.

6. A portable apparatus as claimed in claim 5, wherein portrait mode processing to determine foreground and background regions of the scene using the first and second images and to substitute the blurred background of the third image for the background of the in-focus image of the scene is performed in the external processing device.

7. A portable apparatus according to claim 1 wherein during determination of said foreground and background regions, exposure of the foreground region of the first or second image taken without flash is adjusted to be nominally the same as exposure of foreground region of the other of the first or second image taken with flash.

8. A portable apparatus as claimed in claim 1, in which said portrait mode is manually selectable by said user.

9. A portable apparatus as claimed in claim 1 operable to analyze one or more of said first, second and third images to determine the presence of a face, and responsive to detecting a face for selecting said portrait mode.

10. A portable digital image acquisition apparatus having no photographic film (hereinafter “

portable apparatus”

), the portable apparatus comprising a housing that contains an image sensor and a lens for capturing digital images and a flash unit coupled with the housing for providing illumination during image capture, the portable apparatus having a portrait mode for generating in-camera an image of a foreground object against a blurred background, the portable apparatus also including a face detector arranged to detect one or more faces in one or more of a series of preview images being captured and if so to switch to portrait mode, the portrait mode being operable to capture first, second and third images of nominally the same scene, not necessarily in the order stated, one of the first and second images being taken with flash and the other being taken without flash, and the third image being blurred compared to the first and second images, the portrait mode further being operable to determine foreground and background regions of the scene using the first and second images, and to substitute the blurred background of the third image for the background of an in focus image of the scene, thereby generating in-camera a digital image of an in-focus foreground object against a blurred background, wherein the in-focus image comprises a fourth image captured in the portrait mode, wherein the first and second images are pre-and/or post-view images, and the apparatus is further configured to provide the image of the in-focus foreground object against the blurred background,

wherein the first and second images have different pixel resolution than the fourth image, with the in-focus image having the higher resolution, and the apparatus is further configured to determine the foreground and background regions including matching pixel resolutions of one or more images at the lower resolution of the first and/or second images with the third and/or fourth image by at least one of up-sampling the image of lower resolution and sub-sampling the image of higher resolution.

11. A portable apparatus as claimed in claim 10, further being configured to determine the foreground and background regions for aligning two or more of the first, second and third images.

12. A portable apparatus according to claim 10, wherein the third image comprises a pre- or post-view image.

13. A portable apparatus according to claim 10, wherein said digital image acquisition system comprises a digital camera.

14. A portable apparatus according to claim 10, wherein said digital image acquisition system is a combination of a digital camera and an external processing device.

15. A portable apparatus as claimed in claim 14, wherein portrait mode processing to determine foreground and background regions of the scene using the first and second images and to substitute the blurred background of the third image for the background of the in-focus image of the scene is performed in the external processing device.

16. A portable apparatus according to claim 10, wherein during determination of said foreground and background regions, exposure of the foreground region of the first or second image taken without flash is adjusted to be nominally the same as exposure of foreground region of the other of the first or second image taken with flash.

17. A portable apparatus as claimed in claim 10, in which said portrait mode is manually selectable by said user.

18. A portable apparatus as claimed in claim 10, operable to analyze one or more of said first, second and third images to determine the presence of a face, and responsive to detecting a face for selecting said portrait mode.

19. An in-camera method of generating a digital image of a foreground object against a blurred background within a portable digital image acquisition apparatus having no photographic film (hereinafter “

portable apparatus”

), the portable apparatus comprising a housing that contains an image sensor and a lens for capturing digital images and a flash unit coupled with the housing for providing illumination during image capture, the portable apparatus including a portrait mode for generating in-camera an image of a foreground object against a blurred background, the portable apparatus also including a face detector arranged to detect one or more faces in one or more of a series of preview images being captured and if so to switch to portrait mode, the method comprising;

capturing a series of digital images, detecting one or more faces within the series of digital images, switching to portrait mode as a result of said detecting said one or more faces within said series of digital images, with the apparatus thereby in portrait mode, capturing with said portable apparatus first, second and third images of nominally the same scene, not necessarily in the order stated, at least one of the first and second images being taken with flash, and the third image being blurred compared to the first and second images, determining foreground and background regions of the scene using the first and second images, and substituting the blurred background of the third image for the background of an in focus image of the scene, thereby generating in-camera a digital image of an in-focus foreground object against a blurred background,

wherein the in-focus image comprises one of the first and second images, wherein the first and second images have different pixel resolutions with the in-focus image having the higher resolution, and wherein the system is further configured to determine the foreground and background regions including matching pixel resolutions of the first and second images by at least one of up-sampling the image of lower resolution and sub-sampling the image of higher resolution.

20. The in-camera method as claimed in claim 19, further comprising determining the foreground and background regions and aligning two or more of the first, second and third images.

21. The in-camera method according to claim 20, wherein the third image comprises a pre- or post-view image.

22. The in-camera method according to claim 19, wherein during determination of said foreground and background regions, exposure of the foreground region of the first or second image is adjusted to be nominally the same as exposure of foreground region of the other of the first or second image.

23. The in-camera method as claimed in claim 19, in which said portrait mode is manually selectable by said user.

24. The in-camera method as claimed in claim 19, further comprising analyzing one or more of said first, second and third images to determine the presence of a face, and responsive to detecting a face, selecting said portrait mode.

25. The in-camera method as claimed in claim 19, wherein the in-focus image comprises one of the first and second images.

26. An in-camera method of generating a digital image of a foreground object against a blurred background within a portable digital image acquisition apparatus having no photographic film (hereinafter “

portable apparatus”

), the portable adulates comprising a housing that contains an image sensor and a lens for capturing digital images and a flash unit coupled with the housing for providing illumination during image capture, the portable apparatus including a portrait mode for generating in-camera an image of a foreground object against a blurred background, the portable apparatus also including a face detector arranged to detect one or more faces in one or more of a series of preview images being captured and if so to switch to portrait mode, the method comprising;

capturing a series of digital images, detecting one or more faces within the series of digital images, switching to portrait mode as a result of said detecting said one or more faces within said series of digital images, with the apparatus thereby in portrait mode, capturing with said portable apparatus first, second and third images of nominally the same scene, not necessarily in the order stated, one of the first and second images being taken with flash, and the third image being blurred compared to the first and second images, determining foreground and background regions of the scene using the first and second images, and substituting the blurred background of the third image for the background of an in focus image of the scene, thereby generating in-camera a digital image of an in-focus foreground object against a blurred background, and wherein the in-focus image comprises a fourth image,

wherein the first and second images have lower pixel resolution than the fourth image, wherein the first and second images are pre- and/or post-view images, and wherein the method further comprises matching pixel resolutions of one or more images at the lower resolution of the first and/or second images with the third and/or fourth image by at least one of up-sampling the image of lower resolution and sub-sampling the image of higher resolution.

27. The in-camera method as claimed in claim 26, further comprising determining the foreground and background regions and aligning two or more of the first, second and third images.

28. The in-camera method according to claim 27, wherein the third image comprises a pre- or post-view image.

29. The in-camera method according to claim 26, wherein during determination of said foreground and background regions, exposure of the foreground region of the first or second image is adjusted to be nominally the same as exposure of foreground region of the other of the first or second image.

30. The in-camera method as claimed in claim 26, in which said portrait mode is manually selectable by said user.

31. The in-camera method as claimed in claim 26, further comprising analyzing one or more of said first, second and third images to determine the presence of a face, and responsive to detecting a face, selecting said portrait mode.

32. The in-camera method as claimed in claim 26, wherein the first and second images comprise pre- and/or post-view images.

33. One or more non-transitory processor readable storage devices having processor readable code embodied thereon, said processor readable code for programming one or more processors to perform on in-camera method of generating a digital image of a foreground object against a blurred background within a portable digital image acquisition apparatus having no photographic film (hereinafter “

portable apparatus”

), the portable apparatus comprising a housing that contains an image sensor and a lens for capturing digital images and a flash unit coupled with the housing for providing illumination during image capture, the portable apparatus including a portrait mode for generating in-camera an image of a foreground object against a blurred background, the portable apparatus also including a face detector arranged to detect one or more faces in one or more of a series of preview images being captured and if so to switch to portrait mode, the method comprising;

capturing a series of digital images, detecting one or more faces within the series of digital images, switching to portrait mode as a result of said detecting said one or more faces within said series of digital images, with the apparatus thereby in portrait mode, capturing with said portable apparatus first, second and third images of nominally the same scene, not necessarily in the order stated, one of the first and second images being taken with flash and the other being taken without flash, and the third image being blurred compared to the first and second images, determining foreground and background regions of the scene using the first and second images, and substituting the blurred background of the third image for the background of an in focus image of the scene, thereby generating in-camera a digital image of an in-focus foreground object against a blurred background,

wherein the in-focus image comprises one of the first and second images, wherein the first and second images have different pixel resolutions with the in-focus image having the higher resolution, and wherein the system is further configured to determine the foreground and background regions including matching pixel resolutions of the first and second images by at least one of up-sampling the image of lower resolution and sub-sampling the image of higher resolution.

34. The one or more processor readable storage devices as claimed in claim 33, wherein the method further comprises determining the foreground and background regions and aligning two or more of the first, second and third images.

35. The one or more processor readable storage devices according to claim 34, wherein the third image comprises a pre- or post-view image.

36. The one or more processor readable storage devices according to claim 33, wherein during determination of said foreground and background regions, exposure of the foreground region of the first or second image is adjusted to be nominally the same as exposure of foreground region of the other of the first or second image.

37. The one or more processor readable storage devices as claimed in claim 33, in which said portrait mode is manually selectable by said user.

38. The one or more processor readable storage devices as claimed in claim 33, wherein the method further comprises analyzing one or more of said first, second and third images to determine the presence of a face, and responsive to detecting a face, selecting said portrait mode.

39. The one or more processor readable storage devices as claimed in claim 33, wherein the in-focus image comprises one of the first and second images.

40. One or more non-transitory processor readable storage devices having processor readable code embodied thereon, said processor readable code for programming one or more processors to perform an in-camera method of generating a digital image of a foreground object against a blurred background within a portable digital image acquisition apparatus having no photographic film (hereinafter “

portable apparatus”

), the portable apparatus comprising a housing that contains an image sensor and a lens for capturing digital images and a flash unit coupled with the housing for providing illumination during image capture, the portable apparatus including a portrait mode for generating in-camera an image of a foreground object against a blurred background, the portable apparatus also including a face detector arranged to detect one or more faces in one or more of a series of preview images being captured and if so to switch to portrait mode, the method comprising;

capturing a series of digital images, detecting one or more faces within the series of digital images, switching to portrait mode as a result of said detecting said one or more faces within said series of digital images, with the apparatus thereby in portrait mode, capturing with said portable apparatus first, second and third images of nominally the same scene, not necessarily in the order stated, one of the first and second images being taken with flash and the other being taken without flash, and the third image being blurred compared to the first and second images, determining foreground and background regions of the scene using the first and second images, and substituting the blurred background of the third image for the background of an in focus image of the scene, thereby generating in-camera a digital image of an in-focus foreground object against a blurred background, and wherein the in-focus image comprises a fourth image,

wherein the first and second images have lower pixel resolution than the fourth image, wherein the first and second images are pre- and/or post-view images, and wherein the method further comprises matching pixel resolutions of one or more images at the lower resolution of the first and/or second images with the third and/or fourth image by at least one of up-sampling the image of lower resolution and sub-sampling the image of higher resolution.

41. The one or more processor readable storage devices as claimed in claim 40, wherein the method further comprises determining the foreground and background regions and aligning two or more of the first, second and third images.

42. The one or more processor readable storage devices according to claim 41, wherein the third image comprises a pre- or post-view image.

43. The one or more processor readable storage devices according to claim 40, wherein during determination of said foreground and background regions, exposure of the foreground region of the first or second image is adjusted to be nominally the same as exposure of foreground region of the other of the first or second image.

44. The one or more processor readable storage devices as claimed in claim 40, in which said portrait mode is manually selectable by said user.

45. The one or more processor readable storage devices as claimed in claim 40, wherein the method further comprises analyzing one or more of said first, second and third images to determine the presence of a face, and responsive to detecting a face, selecting said portrait mode.

46. The one or more processor readable storage devices as claimed in claim 40, wherein the first and second images are pre- and/or post-view images.