Systems and methods for push-button slow motion

• US 9,143,693 B1
• Filed: 06/10/2014
• Issued: 09/22/2015
• Est. Priority Date: 06/10/2014
• Status: Active Grant

• Current Assignee: Google LLC (Alphabet Inc.)
• Original Assignee: Google Inc. (Alphabet Inc.)
• Inventors: Zhou, Changyin, Levoy, Marc S, Chen, Jiawen
• Primary Examiner(s): DAGNEW, MEKONNEN D
• Application Number:
  US14/300,959
• Time in Patent Office: 469 Days
• Field of Search: 348/231.99, 348/231.9
• US Class Current: 1/1
• CPC Class Codes: H04N 23/00 Cameras or camera modules comprising electronic image sensors; Control thereof
• CPC Class Codes: H04N 23/63 by using electronic viewfinders
• CPC Class Codes: H04N 5/77 between a recording apparatus and a television camera
• CPC Class Codes: H04N 5/783 Adaptations for reproducing at a rate different from the recording rate

Abstract

Imaging systems can often gather higher quality information about a field of view than the unaided human eye. For example, telescopes may magnify very distant objects, microscopes may magnify very small objects, and high frame-rate cameras may capture fast motion. The present disclosure includes devices and methods that provide real-time vision enhancement without the delay of replaying from storage media. The disclosed devices and methods may include a live view display and image and other information enhancements, which utilize in-line computation and constant control. The disclosure includes techniques for enabling push-button slow motion effects through buffer management and the adjustment of a display frame rate.

Claims

1. A method comprising:

receiving a sequence of image frames captured by an image-capture device, wherein the sequence of image frames is initially captured at a capture frame rate;

displaying a representation of a first portion of the sequence of image frames in a live view interface at a first display frame rate, wherein the representation of the first portion is based on a real-time sampling of the sequence of image frames;

in response to receiving a control input corresponding to a slow motion effect;

storing subsequently-received image frames from the sequence to an image buffer, wherein the subsequently-received image frames are sampled from the sequence at a sampling rate that is greater than the first display frame rate;

using the image buffer to display a representation of a second portion of the sequence in the live view interface, wherein the second portion is displayed at a second display frame rate that is less than the first display frame rate, such that the displayed representation falls behind real-time;

in response to a subsequent trigger event, using the image buffer to display a representation of a third portion of the sequence in the live view interface, wherein the third portion is displayed at a third display frame rate that is greater than the first display frame rate, such that the displayed representation catches up to real-time.

2. The method of claim 1, wherein the first portion is based on a real-time sampling at the display frame rate, the method further comprising:

further in response to receiving a control input corresponding to a slow motion effect, reducing the display frame rate such that the sampling rate is greater than the display frame rate.

3. The method of claim 1, wherein the first portion is based on a real-time sampling at the display frame rate, the method further comprising:

further in response to receiving a control input corresponding to a slow motion effect, increasing the sampling rate such that the sampling rate is greater than the display frame rate.

4. The method of claim 1, wherein the first portion is based on a real-time sampling at the display frame rate, the method further comprising:

further in response to receiving a control input corresponding to a slow motion effect, storing at least one artificial image frame in the image buffer.

5. The method of claim 4, wherein the at least one artificial image frame comprises at least one duplicate frame based on at least one of the image frames.

6. The method of claim 4, wherein the at least one artificial image frame comprises at least one interpolated frame based on at least two of the image frames.

7. The method of claim 1 further comprising in response to receiving a control input corresponding to a slow motion effect, immediately reducing the second display frame rate to a minimum of a display frame rate range.

8. The method of claim 1 further comprising in response to receiving a control input corresponding to a slow motion effect, gradually reducing the second display frame rate to a minimum of a display frame rate range.

9. The method of claim 1 further comprising in response to a subsequent trigger event, immediately increasing the display frame rate to match the capture frame rate.

10. The method of claim 1 further comprising in response to a subsequent trigger event,

increasing the third display frame rate gradually such that the third display frame rate becomes greater than the capture frame rate;

and

in response to the subsequently-received image frames in the buffer reaching a buffer threshold, reducing the third display frame rate smoothly to match the capture frame rate.

11. A system comprising:

a live view interface configured to display a representation of at least a first, a second, and a third portion of a sequence of image frames, wherein the representation of the first portion is based on a real-time sampling of the sequence of image frames;

wherein the representation of the first portion is displayed at a first display frame rate;

an image buffer;

a control interface; and

a control system configured to;

a) in response to receiving a control input corresponding to a slow motion effect, store subsequently-received image frames from the sequence into the image buffer, wherein the subsequently-received image frames are sampled from the sequence at a sampling rate that is greater than the first display frame rate;

b) use the image buffer to display a representation of the second portion of the sequence in the live view interface, wherein the second portion is displayed at a second display frame rate that is less than the first display frame rate, such that the displayed representation falls behind real-time;

c) in response to a subsequent trigger event, use the image buffer to display a representation of a third portion of the sequence in the live view interface, wherein the third portion is displayed at a third display frame rate that is greater than the first display frame rate, such that the displayed representation catches up to real-time.

12. The system of claim 11 wherein the control system is configured to, in response to receiving the control input, immediately reduce the display frame rate to a minimum of the display frame rate range.

13. The system of claim 11 wherein the control system is configured to, in response to receiving the control input, gradually reduce the display frame rate to a minimum of the display frame rate range.

14. The system of claim 11 wherein the control system is configured to, in response to the subsequent trigger event, immediately increase the display frame rate to match the capture frame rate.

15. The system of claim 11 wherein the control system is configured to, in response to the subsequent trigger event:

increase the display frame rate gradually such that the display frame rate becomes greater than the capture frame rate; and

in response to the number of buffered frames reaching a buffer threshold, reduce the display frame rate smoothly to match the capture frame rate.

16. The system of claim 11 wherein the control interface comprises a touch-sensitive surface.

17. The system of claim 11 wherein the control interface comprises a button.

18. A non-transitory computer readable medium having stored therein instructions executable by a computing device to cause the computing device to perform functions comprising:

receiving a sequence of image frames captured by an image-capture device, wherein the sequence of image frames is initially captured at a capture frame rate;

causing a representation of a first portion of the sequence of image frames to be displayed in a live view interface at a first display frame rate, wherein the representation of the first portion is based on a real-time sampling of the sequence of image frames;

in response to receiving a control input corresponding to a slow motion effect;

causing subsequently-received image frames from the sequence to be stored in an image buffer, wherein the subsequently-received image frames are sampled from the sequence at a sampling rate that is greater than the first display frame rate;

causing the image buffer to display a representation of a second portion of the sequence in the live view interface, wherein the second portion is displayed at a second display frame rate that is less than the first display frame rate, such that the displayed representation falls behind real-time;

in response to a subsequent trigger event, causing the image buffer to display a representation of a third portion of the sequence in the live view interface, wherein the third portion is displayed at a third display frame rate that is greater than the first display frame rate, such that the displayed representation catches up to real-time.

19. The non-transitory computer readable medium of claim 18 further comprising in response to receiving a control input corresponding to a slow motion effect, gradually reducing the second display frame rate to a minimum of a display frame rate range.

20. The non-transitory computer readable medium of claim 18 further comprising in response to a subsequent trigger event,

increasing the third display frame rate gradually such that the third display frame rate becomes greater than the capture frame rate;

and

in response to the subsequently-received image frames in the buffer reaching a buffer threshold, reducing the third display frame rate smoothly to match the capture frame rate.