Modification of viewing parameters for digital images using face detection information

• US 9,053,545 B2
• Filed: 03/19/2007
• Issued: 06/09/2015
• Est. Priority Date: 06/26/2003
• Status: Active Grant

• Current Assignee: FotoNation Limited (Adeia Inc.)
• Original Assignee: FotoNation Limited (Adeia Inc.)
• Inventors: Steinberg, Eran, Corcoran, Peter
• Primary Examiner(s): DULANEY, KATHLEEN YUAN
• Application Number:
  US11/688,236
• Publication Number:
  US 20070160307A1
• Time in Patent Office: 3,004 Days
• Field of Search: 382/254, 382/103, 382/118, 382/167, 382/274, 382/156, 348/239, 345/473, 345/619
• US Class Current: 1/1
• CPC Class Codes: G06T 2207/30201 Face
• CPC Class Codes: G06T 7/73 using feature-based methods
• CPC Class Codes: H04N 23/61 based on recognised objects
• CPC Class Codes: H04N 23/611 where the recognised objects include parts of the human body
• CPC Class Codes: H04N 23/67 Focus control based on electronic image sensor signals
• CPC Class Codes: H04N 23/74 by influencing the scene brightness using illuminating means

Abstract

A method of modifying the viewing parameters of digital images using face detection for achieving a desired spatial parameters based on one or more sub-groups of pixels that correspond to one or more facial features of the face. Such methods may be used for animating still images, automating and streamlining application such as the creation of slide shows and screen savers of images containing faces.

Claims

1. A method of generating one or more new digital images using an original digitally-acquired image including a face, comprising:

using a processor in

acquiring with an image sensor an original image including the face;

identifying one or more groups of pixels that correspond to two or more faces within the original image, including comparing one or more detected luminances and at least one facial orientation within a face identified within a first group of pixels with one or more expected luminances and at least one expected facial orientation of digital facial images, respectively, and identifying a second group of pixels that correspond to another face within the original image;

matching one or both of the first and second group of pixels with stored data to uniquely identify at least one of the two of more faces as a specific person or persons, including applying face recognition to the face or faces by comparing the face or faces with stored face image data identified with said specific person or persons;

generating a luminance map of the original image including the one or more groups of pixels;

selecting a portion of the original image to include the first and second groups of pixels based on the luminance map or on the comparing or both; and

automatically generating values of pixels of one or more new images based on the selected portion in a manner which includes at least one of the two or more faces within the one or more new images

wherein the automatically generating values of pixels of one or more new images comprises applying an automatic fill-flash to, or otherwise adjusting luminance of, one or more of the identified faces based on analysis of the luminance map or on the comparing or both, including brightening one or more of the identified faces relative to background pixels of the one or more new images; and

wherein the automatically generating values of pixels of one or more new images further comprises adjusting a relative rotational orientation of one or more faces of the two or more faces within the original image, wherein the adjusting the relative rotational orientation of the one or more faces comprises rotating the one or more faces relative to other portions of the image.

2. The method of claim 1, further comprising panning between the two or more faces.

3. The method of claim 1, further comprising panning from a first face to a second face of the two or more faces, and zooming the second face.

4. The method of claim 1, further comprising de-zooming a first face and panning to a second face, in which the first face and the second face are among the two or more faces.

5. The method of claim 4, further comprising zooming the second face.

6. The method of claim 1, said panning comprising identifying a panning direction parameter between said two of the two or more faces.

7. The method of claim 6, said panning further comprising sequencing along the identified panning direction between the two or more faces according to the identified panning direction parameter.

8. The method of claim 1, wherein the identifying at least one facial orientation comprises identifying and analyzing a relative spatial parameter of two or more of a nose, an ear, a hair line, a nose bridge, a chin, a neck, a torso, or one or more eyes, lips, shoulders or ears, or combinations thereof.

9. The method of claim 1, wherein the adjusting luminance of one or more identified faces comprises adjusting a shutter, sensitivity, fill-flash, or aperture, or changing ink coverage or display luminance values, or applying one or more masks each of specified strength, or combinations thereof.

10. A method of providing an option for generating one or more new digital images using an original digitally-acquired image including a face, comprising:

using a processor in

acquiring with a sensor an original image including the face;

identifying one or more groups of pixels that correspond to two or more faces within the original image, including comparing one or more detected luminances and at least one facial orientation within a first group of pixels with one or more expected luminances and at least one expected facial orientation of digital facial images, respectively, and identifying a second group of pixels that correspond to another face within the original image;

matching one or both of the first and second group of pixels with stored data to uniquely identify at least one of the two of more faces as a specific person or persons, including applying face recognition to the face or faces by comparing the face or faces with stored face image data identified with said specific person or persons;

generating a luminance map of the original image including the one or more groups of pixels;

selecting a portion of the original image to include the first and second groups of pixels based on the luminance map or on the comparing or both; and

automatically providing an option for generating values of pixels of one or more new images based on the selected portion in a manner which includes at least one of the two or more faces within the one or more new images

wherein the automatically providing an option for generating values of pixels of one or more new images comprises applying an automatic fill-flash to, or otherwise adjusting luminance of, one or more of the identified faces based on analysis of the luminance map or on the comparing or both, including brightening one or more of the identified faces relative to background pixels of the one or more new images; and

wherein the automatically providing an option for generating values of pixels further comprises adjusting a relative rotational orientation of one or more faces of the two or more faces within the original image, wherein the adjusting the relative rotational orientation of the one or more faces comprises rotating the one or more faces relative to other portions of the image.

11. The method of claim 10, further comprising panning between the two or more faces.

12. The method of claim 10, further comprising panning from a first face to a second face of the two or more faces, and zooming the second face.

13. The method of claim 10, further comprising de-zooming a first face and panning to a second face, in which the first face and the second face are among the two or more faces.

14. The method of claim 13, further comprising zooming the second face.

15. The method of claim 10, said panning comprising identifying a panning direction parameter between said two of the two or more faces.

16. The method of claim 15, said panning further comprising sequencing along the identified panning direction between the two faces according to the identified panning direction parameter.

17. The method of claim 10, wherein the identifying at least one facial orientation comprises identifying and analyzing a relative spatial parameter of two or more of a nose, an ear, a hair line, a nose bridge, a chin, a neck, a torso, or one or more eyes, lips, shoulders or ears, or combinations thereof.

18. The method of claim 10, wherein the adjusting luminance of one or more identified faces comprises adjusting a shutter, sensitivity, fill-flash, or aperture, or changing ink coverage or display luminance values, or applying one or more masks each of specified strength, or combinations thereof.

19. One or more non-transitory processor readable storage devices having processor readable code embodied thereon, said processor readable code for programming one or more processors to perform a method of generating one or more new digital images using an original digitally-acquired image including a face, the method comprising:

acquiring with a sensor an original image including the face;

identifying one or more groups of pixels that correspond to two or more faces within the original image, including comparing one or more detected luminances and at least one facial orientation within a first group of pixels with one or more expected luminances and at least one expected facial orientation of digital facial images, respectively, and identifying a second group of pixels that correspond to another face within the image;

matching one or both of the first and second group of pixels with stored data to uniquely identify at least one of the two of more faces as a specific person or persons, including applying face recognition to the face or faces by comparing the face or faces with stored face image data identified with said specific person or persons;

generating a luminance map of the original image including the one or more groups of pixels;

selecting a portion of the original image to include the first and second groups of pixels based on the luminance map or on the comparing or both; and

automatically generating values of pixels of one or more new images values of pixels of one or more new images based on the selected portion in a manner which includes at least two of the two or more faces within the one or more new images

wherein the automatically generating values of pixels of one or more new images comprises applying an automatic fill-flash to, or otherwise adjusting luminance of, one or more of the identified faces based on analysis of the luminance map or on the comparing or both, including brightening one or more of the identified faces relative to background pixels of the one or more new images; and

wherein the automatically generating values of pixels of one or more new images further comprises adjusting a relative rotational orientation of one or more faces of the two or more faces within the original image, wherein the adjusting the relative rotational orientation of the one or more faces comprises rotating the one or more faces relative to other portions of the image.

20. The one or more non-transitory processor readable storage devices of claim 19, the method further comprising panning between the two or more faces.

21. The one or more non-transitory processor readable storage devices of claim 19, the method further comprising panning from a first face to a second face of the two or more faces, and zooming the second face.

22. The one or more non-transitory processor readable storage devices of claim 19, the method further comprising de-zooming a first face and panning to a second face, in which the first face and the second face are among the two or more faces.

23. The one or more non-transitory processor readable storage devices of claim 22, the method further comprising zooming the second face.

24. The one or more non-transitory processor readable storage devices of claim 19, said panning comprising identifying a panning direction parameter between said two of the two or more faces.

25. The one or more non-transitory processor readable storage devices of claim 24, said panning further comprising sequencing along the identified panning direction between the two faces according to the identified panning direction parameter.

26. The one or more non-transitory processor readable storage devices of claim 19, wherein the identifying at least one facial orientation comprises identifying and analyzing a relative spatial parameter of two or more of a nose, an ear, a hair line, a nose bridge, a chin, a neck, a torso, or one or more eyes, lips, shoulders or ears, or combinations thereof.

27. The one or more non-transitory processor readable storage devices of claim 19, wherein the adjusting luminance of one or more identified faces comprises adjusting a shutter, sensitivity, fill-flash, or aperture, or changing ink coverage or display luminance values, or applying one or more masks each of specified strength, or combinations thereof.

28. One or more non-transitory processor readable storage devices having processor readable code embodied thereon, said processor readable code for programming one or more processors to perform a method of providing an option for generating one or more new digital images using an original digitally-acquired image including a face, the method comprising:

acquiring with a sensor an original image including the face;

identifying one or more groups of pixels that correspond to two or more faces within the image, including comparing one or more detected luminances and at least one facial orientation within a first group of pixels with one or more expected luminances and at least one expected facial orientation of digital facial images, respectively, and identifying a second group of pixels that correspond to another face within the original image;

matching one or both of the first and second group of pixels with stored data to uniquely identify at least one of the two of more faces as a specific person or persons, including applying face recognition to the face or faces by comparing the face or faces with stored face image data identified with said specific person or persons;

generating a luminance map of the image including the one or more groups of pixels;

selecting a portion of the original image to include the first and second groups of pixels based on the luminance map or on the comparing or both; and

automatically providing an option for generating values of pixels of one or more new images based on the selected portion in a manner which includes at least two of the two or more faces within the one or more new images

wherein the automatically providing an option for generating values of pixels of one or more new images comprises applying an automatic fill-flash to, or otherwise adjusting luminance of, one or more of the identified faces based on analysis of the luminance map or on the comparing or both, including brightening one or more of the identified faces relative to background pixels of the one or more new images; and

wherein the automatically providing an option for generating values of pixels further comprises adjusting a relative rotational orientation of one or more faces of the two or more faces within the original image, wherein the adjusting the relative rotational orientation of the one or more faces comprises rotating the one or more faces relative to other portions of the image.

29. The one or more non-transitory processor readable storage devices of claim 28, the method further comprising panning between the two or more faces.

30. The one or more non-transitory processor readable storage devices of claim 28, the method further comprising panning from a first face to a second face of the two or more faces, and zooming the second face.

31. The one or more non-transitory processor readable storage devices of claim 28, the method further comprising de-zooming a first face and panning to a second face, in which the first face and the second face are among the two or more faces.

32. The one or more non-transitory processor readable storage devices of claim 31, the method further comprising zooming the second face.

33. The one or more non-transitory processor readable storage devices of claim 28, said panning comprising identifying a panning direction parameter between said two of the faces.

34. The one or more non-transitory processor readable storage devices of claim 33, said panning further comprising sequencing along the identified panning direction between the two faces according to the identified panning direction parameter.

35. The one or more non-transitory processor readable storage devices of claim 28, wherein the identifying at least one facial orientation comprises identifying and analyzing a relative spatial parameter of two or more of a nose, an ear, a hair line, a nose bridge, a chin, a neck, a torso, or one or more eyes, lips, shoulders or ears, or combinations thereof.

36. The one or more non-transitory processor readable storage devices of claim 28, wherein the adjusting luminance of one or more identified faces comprises adjusting a shutter, sensitivity, fill-flash, or aperture, or changing ink coverage or display luminance values, or applying one or more masks each of specified strength, or combinations thereof.

37. A method of generating one or more new digital images using an original digitally-acquired image including a face, comprising:

using a processor in identifying one or more groups of pixels that correspond to two or more faces within an original image;

generating a luminance map of the original image including the one or more groups of pixels;

matching at least a first group of pixels with stored data to uniquely identify one or both of the two faces as a specific person or persons, including applying face recognition to the face or faces by comparing the face or faces with stored face image data identified with said specific person or persons;

determining at least one facial orientation within the original image;

selecting a portion of the original image to include the one or more groups of pixels; and

automatically generating values of pixels of one or more new images based on the selected portion in a manner which includes at least two of the two or more faces within the one or more new images,

wherein the automatically generating values of pixels of one or more new images comprises applying an automatic fill-flash to, or otherwise adjusting luminance of, one or more of the identified faces based on analysis of the luminance map, including brightening one or more of the identified faces relative background pixels of the one or more new images; and

wherein the automatically generating values of pixels of one or more new images further comprises adjusting a relative rotational orientation of one or more faces of the two or more faces within the original image, wherein the adjusting the relative rotational orientation of the one or more faces comprises rotating the one or more faces relative to other portions of the image.

38. The method of claim 37, further comprising panning between the two or more faces.

39. The method of claim 37, further comprising panning from a first face to a second face of the two or more faces, and zooming the second face.

40. The method of claim 37, further comprising de-zooming a first face and panning to a second face, in which the first face and the second face are among the two or more faces.

41. The method of claim 40, further comprising zooming the second face.

42. The method of claim 37, said panning comprising identifying a panning direction parameter between said two of the two or more faces.

43. The method of claim 42, said panning further comprising sequencing along the identified panning direction between the two or more faces according to the identified panning direction parameter.

44. The method of claim 37, wherein the adjusting luminance of one or more identified faces comprises adjusting a shutter, sensitivity, fill-flash, or aperture, or changing ink coverage or display luminance values, or applying one or more masks each of specified strength, or combinations thereof.

45. A method of providing an option for generating one or more new digital images using an original digitally-acquired image including a face, comprising:

using a processor in

identifying one or more groups of pixels that correspond to two or more faces within an original image

generating a luminance map of the original image including the one or more groups of pixels;

matching at least a first group of pixels with stored data to uniquely identify one or both of the two faces as a specific person or persons, including applying face recognition to the face or faces by comparing the face or faces with stored face image data identified with said specific person or persons;

determining at least one facial orientation within the original image;

selecting a portion of the original image to include the one or more groups of pixels based on the luminance map; and

automatically providing an option for generating values of pixels of one or more new images based on the selected portion in a manner which includes at least two of the two or more faces within the one or more new images,

wherein the automatically providing an option for generating values of pixels of one or more new images comprises applying an automatic fill-flash to, or otherwise adjusting luminance of, one or more of the identified faces based on analysis of the luminance map, including brightening one or more of the identified faces relative to background pixels of the one or more new images; and

wherein the automatically providing an option for generating values of pixels further comprises adjusting a relative rotational orientation of one or more faces of the two or more faces within the original image, wherein the adjusting the relative rotational orientation of the one or more faces comprises rotating the one or more faces relative to other portions of the image.

46. The method of claim 45, further comprising panning between the two or more faces.

47. The method of claim 45, further comprising panning from a first face to a second face of the two or more faces, and zooming the second face.

48. The method of claim 45, further comprising de-zooming a first face and panning to a second face, in which the first face and the second face are among the two or more faces.

49. The method of claim 48, further comprising zooming the second face.

50. The method of claim 45, said panning comprising identifying a panning direction parameter between said two of the two or more faces.

51. The method of claim 50, said panning further comprising sequencing along the identified panning direction between the two faces according to the identified panning direction parameter.

52. The method of claim 45, wherein the adjusting luminance of one or more identified faces comprises adjusting a shutter, sensitivity, fill-flash, or aperture, or changing ink coverage or display luminance values, or applying one or more masks each of specified strength, or combinations thereof.

53. One or more non-transitory processor readable storage devices having processor readable code embodied thereon, said processor readable code for programming one or more processors to perform a method of generating one or more new digital images using an original digitally-acquired image including a face, the method comprising:

identifying one or more groups of pixels that correspond to two or more faces within an original image

generating a luminance map of the original image including the one or more groups of pixels;

matching at least a first group of pixels with stored data to uniquely identify one or both of the two faces as a specific person or persons, including applying face recognition to the face or faces by comparing the face or faces with stored face image data identified with said specific person or persons;

determining at least one facial orientation within the original image;

selecting a portion of the image to include the one or more groups of pixels based on the luminance map; and

automatically generating values of pixels of one or more new images based on the selected portion in a manner which includes at least two of the two or more faces within the one or more new images,

wherein the automatically generating values of pixels of one or more new images comprises applying an automatic fill-flash to, or otherwise adjusting luminance of, one or more of the identified faces based on analysis of the luminance map, including brightening one or more of the identified faces relative to background pixels of the one or more new images; and

wherein the automatically generating values of pixels of one or more new images further comprises adjusting a relative rotational orientation of one or more faces within the original image, wherein the adjusting the relative rotational orientation of the one or more faces comprises rotating the one or more faces relative to other portions of the image.

54. The one or more non-transitory processor readable storage devices of claim 53, the method further comprising panning between the two or more faces.

55. The one or more non-transitory processor readable storage devices of claim 53, the method further comprising panning from a first face to a second face of the two or more faces, and zooming the second face.

56. The one or more non-transitory processor readable storage devices of claim 53, the method further comprising de-zooming a first face and panning to a second face, in which the first face and the second face are among the two or more faces.

57. The one or more non-transitory processor readable storage devices of claim 56, the method further comprising zooming the second face.

58. The one or more non-transitory processor readable storage devices of claim 53, said panning comprising identifying a panning direction parameter between said two of the two or more faces.

59. The one or more non-transitory processor readable storage devices of claim 58, said panning further comprising sequencing along the identified panning direction between the two faces according to the identified panning direction parameter.

60. The one or more non-transitory processor readable storage devices of claim 53, wherein the adjusting luminance of one or more identified faces comprises adjusting a shutter, sensitivity, fill-flash, or aperture, or changing ink coverage or display luminance values, or applying one or more masks each of specified strength, or combinations thereof.

61. One or more non-transitory processor readable storage devices having processor readable code embodied thereon, said processor readable code for programming one or more processors to perform a method of providing an option for generating one or more new digital images using an original digitally-acquired image including a face, the method comprising:

identifying one or more groups of pixels that correspond to two or more faces within an original image, including;

generating a luminance map of the original image including the one or more groups of pixels;

matching at least a first group of pixels with stored data to uniquely identify one or both of the two faces as a specific person or persons, including applying face recognition to the face or faces by comparing the face or faces with stored face image data identified with said specific person or persons;

determining at least one facial orientation within the original image;

selecting a portion of the original image to include the one or more groups of pixels based on the luminance map; and

automatically providing an option for generating values of pixels of one or more new images based on the selected portion in a manner which includes at least two of the two or more faces within the one or more new images,

wherein the automatically providing an option for generating values of pixels of one or more new images comprises applying an automatic fill-flash to, or otherwise adjusting luminance of, one or more of the identified faces based on analysis of the luminance map, including brightening one or more of the identified faces relative to background pixels of the one or more new images; and

wherein the automatically providing an option for generating values of pixels further comprises adjusting a relative rotational orientation of one or more faces of the two or more faces within the original image, wherein the adjusting the relative rotational orientation of the one or more faces comprises rotating the one or more faces relative to other portions of the image.

62. The one or more non-transitory processor readable storage devices of claim 61, the method further comprising panning between the two or more faces.

63. The one or more non-transitory processor readable storage devices of claim 61, the method further comprising panning from a first face to a second face of the two or more faces, and zooming the second face.

64. The one or more non-transitory processor readable storage devices of claim 61, the method further comprising de-zooming a first face and panning to a second face, in which the first face and the second face are among the two or more faces.

65. The one or more non-transitory processor readable storage devices of claim 64, the method further comprising zooming the second face.

66. The one or more non-transitory processor readable storage devices of claim 61, said panning comprising identifying a panning direction parameter between said two of the faces.

67. The one or more non-transitory processor readable storage devices of claim 66, said panning further comprising sequencing along the identified panning direction between the two faces according to the identified panning direction parameter.

68. The one or more non-transitory processor readable storage devices of claim 61, wherein the adjusting luminance of one or more identified faces comprises adjusting a shutter, sensitivity, fill-flash, or aperture, or changing ink coverage or display luminance values, or applying one or more masks each of specified strength, or combinations thereof.