Time-lapse video capture with temporal points of interest

• US 9,324,376 B2
• Filed: 09/30/2014
• Issued: 04/26/2016
• Est. Priority Date: 09/30/2014
• Status: Active Grant

• Current Assignee: Apple Inc.
• Original Assignee: Apple Inc.
• Inventors: Doepke, Frank
• Primary Examiner(s): LAM, HUNG H
• Application Number:
  US14/502,675
• Publication Number:
  US 20160093335A1
• Time in Patent Office: 574 Days
• Field of Search: 348/222.1, 348/259
• US Class Current: 1/1
• CPC Class Codes: H04N 1/212 Motion video recording combined with still video recording television signal recording H04N5/76
• CPC Class Codes: H04N 1/2133 Recording or reproducing at a specific moment, e.g. time interval or time-lapse
• CPC Class Codes: H04N 1/32267 combined with processing of the image
• CPC Class Codes: H04N 5/772 the recording apparatus and the television camera being placed in the same enclosure
• CPC Class Codes: H04N 5/783 Adaptations for reproducing at a rate different from the recording rate
• CPC Class Codes: H04N 5/91 Television signal processing therefor

Abstract

Traditionally, time-lapse videos are constructed from images captured at given time intervals called “temporal points of interests” or “temporal POIs.” Disclosed herein are intelligent systems and methods of capturing and selecting better images around temporal points of interest for the construction of improved time-lapse videos. According to some embodiments, a small “burst” of images may be captured, centered around the aforementioned temporal points of interest. Then, each burst sequence of images may be analyzed, e.g., by performing a similarity comparison between each image in the burst sequence and the image selected at the previous temporal point of interest. Selecting the image from a given burst that is most similar to the previous selected image allows the intelligent systems and methods described herein to improve the quality of the resultant time-lapse video by discarding “outlier” or other undesirable images captured in the burst sequence around a particular temporal point of interest.

Claims

1. A non-transitory program storage device, readable by a programmable control device and comprising instructions stored thereon to cause one or more processing units to:

acquire a first digital image from an image sensor at a first time;

acquire a plurality of second digital images from the image sensor, wherein acquisition times of the plurality of second digital images are temporally centered around a determined second time;

determine a similarity value for each of the plurality of second digital images, wherein the similarity values comprise an assessment of the similarity of a respective image from the second plurality of digital images and the first digital image;

select, from the plurality of second digital images, a second digital image having the greatest similarity value; and

combine the first digital image and the selected second digital image into a time lapse video.

2. The non-transitory program storage device of claim 1, wherein the similarity values are determined based, at least in part, on an evaluation of at least one of the following characteristics of a second digital image:

a color histogram, an exposure level, a focus level, a feature detection process, an image registration process, a pixel sum process, a Δ

E process, and an average luminance level.

3. The non-transitory program storage device of claim 1, wherein the instructions to select, from the plurality of second digital images, a second digital image having the greatest similarity value further comprise instructions to:

delete each unselected digital image from the plurality of second digital images before causing the one or more processing units to combine the first digital image and the selected second digital image into a time lapse video.

4. The non-transitory program storage device of claim 1, wherein the instructions to select, from the plurality of second digital images, a second digital image having the greatest similarity value further comprise instructions to:

delete each unselected digital image from the plurality of second digital images after causing the one or more processing units to combine the first digital image and the selected second digital image into a time lapse video.

5. The non-transitory program storage device of claim 1, wherein the number of digital images in the acquired plurality of second digital images is configurable.

6. The non-transitory program storage device of claim 1, further comprising instructions stored thereon to cause the one or more processing units to:

weight at least one of the determined similarity values for a respective second digital image based, at least in part, on a content of the respective second digital image.

7. The non-transitory program storage device of claim 1, further comprising instructions stored thereon to cause the one or more processing units to:

weight at least one of the determined similarity values for a respective second digital image based, at least in part, on a time of capture of the respective second digital image with respect to the determined second time.

8. A computer-implemented method, comprising:

acquiring a first digital image from an image sensor at a first time;

acquiring a plurality of second digital images from the image sensor, wherein acquisition times of the plurality of second digital images are temporally centered around a determined second time;

determining a similarity value for each of the plurality of second digital images, wherein the similarity values comprise an assessment of the similarity of a respective image from the second plurality of digital images and the first digital image;

selecting, from the plurality of second digital images, a second digital image having the greatest similarity value; and

combining the first digital image and the selected second digital image into a time lapse video.

9. The method of claim 8, wherein the similarity values are determined based, at least in part, on an evaluation of at least one of the following characteristics of a second digital image:

a color histogram, an exposure level, a focus level, a feature detection process, an image registration process, a pixel sum process, a Δ

E process, and an average luminance level.

10. The method of claim 8, wherein the act of selecting, from the plurality of second digital images, a second digital image having the greatest similarity value further comprises:

deleting each unselected digital image from the plurality of second digital images.

11. The method of claim 8, wherein the number of digital images in the acquired plurality of second digital images is configurable.

12. The method of claim 8, further comprising weighting at least one of the determined similarity values for a respective second digital image based, at least in part, on a content of the respective second digital image.

13. The method of claim 8, further comprising weighting at least one of the determined similarity values for a respective second digital image based, at least in part, on a time of capture of the respective second digital image with respect to the determined second time.

14. An electronic device comprising:

an image sensor,

a memory operatively coupled to the image sensor, and

at least one processor operatively coupled to the memory and the image sensor, wherein the at least one processor is programmed to;

cause the camera to acquire a first digital image from an image sensor at a first time;

cause the camera to acquire a plurality of second digital images from the image sensor, wherein acquisition times of the plurality of second digital images are temporally centered around a determined second time;

determine a similarity value for each of the plurality of second digital images, wherein the similarity values comprise an assessment of the similarity of a respective image from the second plurality of digital images and the first digital image;

select, from the plurality of second digital images, a second digital image having the greatest similarity value; and

combine the first digital image and the selected second digital image into a time lapse video.

15. The electronic device of claim 14, wherein the similarity values are determined based, at least in part, on an evaluation of at least one of the following characteristics of a second digital image:

a color histogram, an exposure level, a focus level, a feature detection process, an image registration process, a pixel sum process, a Δ

E process, and an average luminance level.

16. The electronic device of claim 14, wherein the instructions to select, from the plurality of second digital images, a second digital image having the greatest similarity value that the at least one processor is programmed to perform further comprise instructions to:

delete each unselected digital image from the plurality of second digital images before the at least one processor combines the first digital image and the selected second digital image into a time lapse video.

17. The electronic device of claim 14, wherein the instructions to select, from the plurality of second digital images, a second digital image having the greatest similarity value that the at least one processor is programmed to perform further comprise instructions to:

delete each unselected digital image from the plurality of second digital images after the at least one processor combines the first digital image and the selected second digital image into a time lapse video.

18. The electronic device of claim 14, wherein the number of digital images in the acquired plurality of second digital images is configurable.

19. The electronic device of claim 14, wherein the at least one processor is further programmed to:

weight at least one of the determined similarity values for a respective second digital image based, at least in part, on a content of the respective second digital image.

20. The electronic device of claim 14, wherein the at least one processor is further programmed to:

weight at least one of the determined similarity values for a respective second digital image based, at least in part, on a time of capture of the respective second digital image with respect to the determined second time.