Modification of post-viewing parameters for digital images using region or feature information

• US 8,675,991 B2
• Filed: 06/02/2006
• Issued: 03/18/2014
• Est. Priority Date: 06/26/2003
• Status: Expired due to Fees

• Current Assignee: FotoNation Limited (Adeia Inc.)
• Original Assignee: DigitalOptics Corporation Europe Limited (Adeia Inc.)
• Inventors: Steinberg, Eran, Corcoran, Peter, Prilutsky, Yury, Bigioi, Petronel
• Primary Examiner(s): DULANEY, KATHLEEN YUAN
• Application Number:
  US12/302,493
• Publication Number:
  US 20090179998A1
• Time in Patent Office: 2,846 Days
• Field of Search: 382/254, 382/103, 382/118, 382/282, 348/231.99, 345/473, 345/687
• US Class Current: 382/282
• CPC Class Codes: G06T 2207/30201 Face
• CPC Class Codes: G06T 7/73 using feature-based methods
• CPC Class Codes: H04N 23/61 based on recognised objects
• CPC Class Codes: H04N 23/611 where the recognised objects include parts of the human body
• CPC Class Codes: H04N 23/67 Focus control based on electronic image sensor signals
• CPC Class Codes: H04N 23/74 by influencing the scene brightness using illuminating means

Abstract

A method of modifying the viewing parameters of digital images using image feature detection for achieving desired parameters based on one or more sub-groups of pixels that correspond to one or more selected image features such as a face, body, pets, or a background region or a foreground region. Such methods may be used for animating still images, automating and streamlining application such as the creation of slide shows and screen savers of images containing selected image features. Two or more features such as a foreground region and a background region may be separated and independently processed.

Claims

1. A method implemented on a processor-based digital image acquisition device of generating one or more new digital images using an original digitally-acquired image including a selected image feature, comprising:

tracking a location of a selected image feature within a stream of acquired digital images;

locking focus on the selected image feature, including continuously adjusting focus based on changes of locations of said selected image feature within said stream of acquired digital images;

identifying one or more groups of pixels that correspond to the selected image feature within an original digitally-acquired image based on the location tracking and focus locking;

selecting a portion of the original image that includes the one or more groups of pixels;

determining an orientation of the selected image feature within the one or more groups of pixels;

automatically generating values of pixels of one or more new images based on the selected portion in a manner which includes the selected image feature within the one or more new images;

determining one or more simulated camera movements based on the orientation of the selected image feature within the one or more groups of pixels,

wherein the automatically generating values of pixels of one or more new images comprises simulating a changing direction of the selected image feature in two or more images in sequence based on the one or more simulated camera movements; and

transmitting, storing, or displaying at least one of the new images or a further processed version, or combinations thereof.

2. The method of claim 1, wherein said selected image feature comprises one or more faces.

3. The method of claim 2, wherein said changing direction comprises a changing direction that a subject is looking at.

4. The method of claim 3, wherein said changing direction comprises a changing direction of the eyes of the subject.

5. The method of claim 1, wherein said selected image feature comprises a human subject.

6. The method of claim 1, wherein the selected portion comprises a foreground region or a background region.

7. The method of claim 6, wherein the foreground region is determined by detection of a face.

8. The method of claim 6, wherein the determining of the foreground region by defining said selected image feature within an original image comprises determining a depth of focus.

9. The method of claim 1, wherein the identifying comprises identifying one or more groups of pixels that correspond to two or more selected image features within the original digitally-acquired image;

and wherein the automatic generating is in a manner which includes at least one of the two or more selected image features within the one or more new images or a panning intermediate image between two of the selected image features, or a combination thereof.

10. A method as recited in claim 1, further comprising identifying one or more groups of pixels within said digital image acquisition device based on information from one or more preview images.

11. A method as recited in claim 1, further comprising identifying one or more groups of pixels within said digital acquisition device using non-image data.

12. A method as recited in claim 11, wherein said non-image data comprises one or more acquisition parameters.

13. A method as recited in claim 1, wherein said generating values of pixels of one or more new images is performed by an external device to said digital acquisition device.

14. A method as recited in claim 1, further comprising:

saving a transformation between said original digitally-acquired image and said one or more new images as a movie clip.

15. A method implemented on a processor-based digital image acquisition device of generating one or more new digital images using an original digitally-acquired image including a background region or a foreground region, or both, comprising:

tracking a location of a selected image feature within a foreground region of multiple images of a stream of acquired digital images;

locking focus on the selected image feature, including continuously adjusting focus based on changes of the locations of said selected image feature within the foreground regions of the multiple images of said stream of acquired digital images;

identifying one or more groups of pixels that correspond to a background region or a foreground region, or both, within an original digitally-acquired image;

selecting a portion of the original image that includes the one or more groups of pixels;

determining an orientation of the selected image feature within the one or more groups of pixels;

automatically generating values of pixels of one or more new images based on the selected portion in a manner which includes the background region or the foreground region, or both, within each of the one or more new images;

determining one or more simulated camera movements based on the orientation of the selected image feature within the one or more groups of pixels,

wherein the automatically generating values of pixels of one or more new images comprises simulating a changing direction of the selected image feature in two or more images in sequence based on the one or more simulated camera movements; and

transmitting, storing, or displaying at least one of the new images or a further processed version, or combinations thereof.

16. The method of claim 15, further comprising separating the foreground region and the background region within the original image or the one or more new images or combinations thereof.

17. The method of claim 16, further comprising calculating a depth map of the background region or the foreground region or both.

18. The method of claim 15, further comprising independently processing the foreground region or the background region, or both.

19. The method of claim 15, wherein said changing direction comprises a changing direction that a subject is looking at.

20. The method of claim 19, wherein said changing direction comprises a changing direction of the eyes of the subject.

21. One or more non-transitory processor readable storage devices having processor readable code embodied thereon, said processor readable code for programming one or more processors to perform a method of generating one or more new digital images using an original digitally-acquired image including a selected image feature, the method comprising:

tracking a location of a selected image feature within a stream of acquired digital images;

locking focus on the selected image feature, including continuously adjusting focus based on changes of locations of said selected image feature within said stream of acquired digital images;

identifying one or more groups of pixels that correspond to the selected image feature within an original digitally-acquired image based on the location tracking and focus locking;

selecting a portion of the original image that includes the one or more groups of pixels;

determining an orientation of the selected image feature within the one or more groups of pixels;

automatically generating values of pixels of one or more new images based on the selected portion in a manner which includes the selected image feature within the one or more new images; and

determining one or more simulated camera movements based on the orientation of the selected image feature within the one or more groups of pixels, and

wherein the automatically generating values of pixels of one or more new images comprises simulating a changing direction of the selected image feature in two or more images in sequence based on the one or more simulated camera movements.

22. One or more storage devices as recited in claim 21, the method further comprising gradually displaying a transformation between said original digitally-acquired image and said one or more new images.

23. One or more storage devices as recited in claim 22, the method further comprising adjusting parameters of said transformation between said original digitally acquired image and one or more new images.

24. One or more storage devices of claim 22, wherein said transformation is different between the selected portion and remaining portions of the image.

25. One or more storage devices of claim 21, wherein said selected image feature comprises one or more faces.

26. The one or more storage devices of claim 25, wherein said changing direction comprises a changing direction that a subject is looking at.

27. The one or more storage devices of claim 26, wherein said changing direction comprises a changing direction of the eyes of the subject.

28. One or more storage devices of claim 21, wherein said selected image feature comprises a human subject.

29. One or more storage devices of claim 21, wherein said selected image feature comprises an animal.

30. One or more storage devices of claim 21, wherein the selected image feature comprises a foreground region or a background region.

31. One or more storage devices of claim 30, wherein the foreground region is determined by detection of a face.

32. One or more storage devices of claim 30, wherein the foreground region is determined by defining said selected image feature within an original image to be local sharpness, relative exposure, local color clustering, or local saturation, or combinations thereof.

33. One or more storage devices of claim 30, wherein the determining of the foreground region by defining said selected image feature within an original image comprises determining a depth of focus.

34. One or more storage devices of claim 33, the method further comprising determining a relevance or importance, or both, of the foreground region or the background region, or both.

35. One or more storage devices of claim 30, the method further comprising visually separating the foreground region and the background region within the one or more new images.

36. One or more storage devices of claim 35, the method further comprising calculating a depth map of the background region.

37. One or more storage devices of claim 30, the method further comprising independently processing the foreground region or the background region, or both.

38. One or more storage devices of claim 21, wherein the identifying comprises identifying one or more groups of pixels that correspond to two or more selected image features within the original digitally-acquired image;

and

wherein the automatic generating is in a manner which includes at least one of the two or more selected image features within the one or more new images or a panning intermediate image between two of the selected image features, or a combination thereof.

39. One or more storage devices of claim 38, the method further comprising visually shifting a center of interest between the two or more identified image features using simulated camera movement.

40. One or more storage devices of claim 38, wherein said simulated camera movement comprises identifying a direction parameter between said two of the identified image features.

41. One or more storage devices of claim 40, wherein said identifying camera movement is determined based on the determination of a vanishing point of the image.

42. One or more storage devices of claim 40, wherein said identifying camera movement is determined based on spatial orientation and line of sight of one or more faces in said image.

43. One or more storage devices as recited in claim 21, wherein said identifying one or more groups of pixels is performed within a portable digital image acquisition device.

44. One or more storage devices as recited in claim 43, wherein said identifying one or more groups of pixels within said portable digital image acquisition device is performed based on information from one or more preview images.

45. One or more storage devices as recited in claim 43, wherein said identifying one or more groups of pixels within said portable digital image acquisition device is performed using non-image data.

46. One or more storage devices as recited in claim 45, wherein said non-image data comprises one or more acquisition parameters.

47. One or more storage devices as recited in claim 43, wherein said generating values of pixels of one or more new images is performed by an external device to said digital acquisition device.

48. One or more storage devices as recited in claim 21, the method further comprising:

saving a transformation between said original digitally-acquired image and said one or more new images as a movie clip.

49. One or more non-transitory processor readable storage devices having processor readable code embodied thereon, said processor readable code for programming one or more processors to perform a method of generating one or more new digital images using an original digitally-acquired image including a background region or a foreground region, or both, the method comprising:

tracking a location of a selected image feature within a foreground region of multiple images of a stream of acquired digital images;

locking focus on the selected image feature, including continuously adjusting focus based on changes of the locations of said selected image feature within the foreground regions of the multiple images of said stream of acquired digital images;

identifying one or more groups of pixels that correspond to a background region or a foreground region, or both, within an original digitally-acquired image;

selecting a portion of the original image that includes the one or more groups of pixels;

determining an orientation of the selected image feature within the one or more groups of pixels;

automatically generating values of pixels of one or more new images based on the selected portion in a manner which includes the background region or the foreground region, or both, within each of the one or more new images; and

determining one or more simulated camera movements based on the orientation of the selected image feature within the one or more groups of pixels, and

wherein the automatically generating values of pixels of one or more new images comprises simulating a changing direction of the selected image feature in two or more images in sequence based on the one or more simulated camera movements.

50. One or more storage devices of claim 49, the method further comprising separating the foreground region and the background region within the original image or the one or more new images or combinations thereof.

51. One or more storage devices of claim 50, the method further comprising calculating a depth map of the background region or the foreground region or both.

52. One or more storage devices of claim 49, the method further comprising independently processing the foreground region or the background region, or both.

53. One or more storage devices of claim 52, wherein at least one of said new images comprises an independently processed background region or foreground region or both.

54. One or more storage devices of claim 53, wherein the independent processing comprises focusing, saturating, pixilating, sharpening, zooming, panning, tilting, geometrical distorting, cropping, exposing or combinations thereof.

55. One or more storage devices of claim 49, the method further comprising determining a relevance or importance, or both, of the foreground region or the background region, or both.

56. The one or more storage devices of claim 49, wherein said changing direction comprises a changing direction that a subject is looking at.

57. The one or more storage devices of claim 56, wherein said changing direction comprises a changing direction of the eyes of the subject.

58. A portable digital image acquisition device, comprising a lens and an electronic image sensor, a processor, and one or more processor-readable media having code embedded therein for programming the processor to perform a method of generating one or more new digital images using an original digitally-acquired image including a selected image feature, wherein the method comprises:

tracking a location of a selected image feature within a stream of acquired digital images;

locking focus on the selected image feature, including continuously adjusting focus based on changes of locations of said selected image feature within said stream of acquired digital images;

identifying one or more groups of pixels that correspond to the selected image feature within an original digitally-acquired image based on the location tracking and focus locking;

selecting a portion of the original image that includes the one or more groups of pixels;

determining an orientation of the selected image feature within the one or more groups of pixels;

automatically generating values of pixels of one or more new images based on the selected portion in a manner which includes the selected image feature within the one or more new images;

determining one or more simulated camera movements based on the orientation of the selected image feature within the one or more groups of pixels, and

wherein the automatically generating values of pixels of one or more new images comprises simulating a changing direction of the selected image feature in two or more images in sequence based on the one or more simulated camera movements; and

transmitting, storing, or displaying at least one of the new images or a further processed version, or combinations thereof.

59. The device of claim 58, wherein the method further comprises gradually displaying a transformation between said original digitally-acquired image and said one or more new images.

60. The device of claim 59, wherein said transformation is different between the selected portion and remaining portions of the image.

61. The device of claim 58, wherein said selected portion comprises one or more faces.

62. The device of claim 58, wherein said selected portion comprises an animal.

63. The device of claim 58, wherein the selected portion comprises a foreground region or a background region.

64. The device of claim 63, wherein the foreground region is determined by detection of a face.

65. The device of claim 63, wherein the determining of the foreground region by defining said selected image feature within an original image comprises determining a depth of focus.

66. The device of claim 63, wherein the method further comprises visually separating the foreground region and the background region within the one or more new images.

67. The device of claim 66, wherein the method further comprises calculating a depth map of the background region.

68. The device of claim 63, wherein the method further comprises independently processing the foreground region or the background region, or both.

69. The device of claim 63, wherein the method further comprises determining a relevance or importance, or both, of the foreground region or the background region, or both.

70. The device of claim 58, wherein the identifying comprises identifying one or more groups of pixels that correspond to two or more selected image features within the original digitally-acquired image;

and wherein the automatic generating is in a manner which includes at least one of the two or more selected image features within the one or more new images or a panning intermediate image between two of the selected image features, or a combination thereof.

71. The device of claim 58, wherein the method further comprises identifying one or more groups of pixels based on information from one or more preview images.

72. The device of claim 58, wherein the method further comprises identifying one or more groups of pixels within said a digital acquisition device using non-image data.

73. The device of claim 72, wherein said non-image data comprises one or more acquisition parameters.

74. The device of claim 58, wherein the method further comprises:

saving a transformation between said original digitally-acquired image and said one or more new images as a movie clip.

75. The device of claim 58, wherein said changing direction comprises a changing direction that a subject is looking at.

76. The device of claim 75, wherein said changing direction comprises a changing direction of the eyes of the subject.

77. A portable digital image acquisition device, comprising a lens and an electronic image sensor, a processor, and one or more processor-readable media having code embedded therein for programming the processor to perform a method of generating one or more new digital images using an original digitally-acquired image including a background region or a foreground region, or both, wherein the method comprises:

tracking a location of a selected image feature within a foreground region of multiple images of a stream of acquired digital images;

locking focus on the selected image feature, including continuously adjusting focus based on changes of the locations of said selected image feature within the foreground regions of the multiple images of said stream of acquired digital images;

identifying one or more groups of pixels that correspond to a background region or a foreground region, or both, within an original digitally-acquired image;

selecting a portion of the original image that includes the one or more groups of pixels;

determining an orientation of the selected image feature within the one or more groups of pixels;

automatically generating values of pixels of one or more new images based on the selected portion in a manner which includes the background region or the foreground region, or both, within each of the one or more new images;

determining one or more simulated camera movements based on the orientation of the selected image feature within the one or more groups of pixels, and

wherein the automatically generating values of pixels of one or more new images comprises simulating a changing direction of the selected image feature in two or more images in sequence based on the one or more simulated camera movements; and

transmitting, storing, or displaying at least one of the new images or a further processed version, or combinations thereof.

78. The device of claim 77, wherein the method further comprises separating the foreground region and the background region within the original image or the one or more new images or combinations thereof.

79. The device of claim 78, wherein the method further comprises calculating a depth map of the background region or the foreground region or both.

80. The device of claim 79, wherein the method further comprises independently processing the foreground region or the background region, or both.

81. The device of claim 80, wherein at least one of said new images comprises an independently processed background region or foreground region or both.

82. The device of claim 81, wherein the independent processing comprises focusing, saturating, pixilating, sharpening, zooming, panning, tilting, geometrical distorting, cropping, exposing or combinations thereof.

83. The device of claim 80, wherein the method further comprises determining a relevance or importance, or both, of the foreground region or the background region, or both.

84. The device of claim 77, wherein said changing direction comprises a changing direction that a subject is looking at.

85. The device of claim 84, wherein said changing direction comprises a changing direction of the eyes of the subject.