Dynamic synchronization of frame rate to a detected cadence in a time lapse image sequence

• US 9,787,900 B2
• Filed: 12/16/2015
• Issued: 10/10/2017
• Est. Priority Date: 12/16/2015
• Status: Active Grant

• Current Assignee: GoPro, Inc.
• Original Assignee: GoPro, Inc.
• Inventors: Weckel, Charles, Bhat, Vishwanatha
• Primary Examiner(s): COLEMAN, STEPHEN P
• Application Number:
  US14/970,866
• Publication Number:
  US 20170180645A1
• Time in Patent Office: 664 Days
• Field of Search: None
• CPC Class Codes: H04N 23/60 Control of cameras or camera modules
• CPC Class Codes: H04N 23/681 Motion detection
• CPC Class Codes: H04N 23/6811 based on the image signal
• CPC Class Codes: H04N 23/683 performed by a processor, e.g. controlling the readout of an image memory
• CPC Class Codes: H04N 7/0127 by changing the field or frame frequency of the incoming video signal, e.g. frame rate converter

Abstract

A frame rate is synchronized to a detected cadence in order to generate an output image sequence that is substantially stabilized. In an in-camera process, a camera receives motion data of the camera while the camera captures the sequence of image frames. A dominant frequency of motion is determined and the capture frame rate is dynamically adjusted to match the frequency of detected motion so that each image frame is captured when the camera is at approximately the same position along the axis of motion. Alternatively, in a post-processing process, frames of a captured image sequence are selectively sampled at a sampling rate corresponding to the dominant frequency of motion so that each sampled frame corresponds to an image capture that occurred when the camera is at approximately the same position along the axis of motion.

Claims

1. A method for dynamic synchronization of a frame rate to a detected cadence, the method comprising:

configuring a camera to operate according to an initial frame rate;

initiating capture of a time lapse image sequence on the camera, according to the initial frame rate;

recording motion data of the camera during the capture of the time lapse image sequence, the motion data recorded by one or more sensors tracking motion of the camera while the camera captures the time lapse image sequence;

converting the motion data in a particular window of time from time domain data to frequency domain data;

determining a dominant frequency in the frequency domain data;

determining a modified frame rate that is within a predefined tolerance range of the dominant frequency;

continuing capture of the time lapse image sequence according to the modified frame rate; and

storing the time lapse image sequence to a storage medium.

2. The method of claim 1, wherein the particular window of time comprises a predefined time window.

3. The method of claim 1, wherein the motion data comprises position data along an axis of oscillation derived from sensed data captured by a motion sensor.

4. The method of claim 1, wherein the converting the motion data from time domain data to frequency domain data is performed using Fast Fourier Transform.

5. The method of claim 1, wherein the determining the dominant frequency in the frequency domain data comprises:

determining a frequency value in the frequency domain data that has the largest amplitude of the plurality of frequency values in the frequency domain data;

selecting the frequency value as the dominant frequency.

6. The method of claim 1, wherein determining the modified frame rate comprises:

setting the modified frame rate to match the dominant frequency.

7. A non-transitory computer-readable storage medium storing instructions for dynamic synchronization of frame rate to cadence, the instructions when executed by one or more processors cause the one or more processors to:

configure a camera to operate according to an initial frame rate;

initiate capture of a time lapse image sequence on the camera, according to the initial frame rate;

record motion data of the camera during the capture of the time lapse image sequence, the motion data recorded by one or more sensors tracking motion of the camera while the camera captures the time lapse image sequence;

convert the motion data in a particular window of time from time domain data to frequency domain data;

determine a dominant frequency in the frequency domain data;

determine a modified frame rate that is within a predefined tolerance range of the dominant frequency;

continue capture of the time lapse image sequence according to the modified frame rate; and

store the time lapse image sequence to a storage medium.

8. The non-transitory computer-readable storage medium of claim 7, wherein the particular window of time comprises a predefined time window.

9. The non-transitory computer-readable storage medium of claim 7, wherein the motion data comprises position data along an axis of oscillation derived from sensed data captured by a motion sensor.

10. The non-transitory computer-readable storage medium of claim 7, wherein the conversion of the motion data from time domain data to frequency domain data is performed using Fast Fourier Transform.

11. The non-transitory computer-readable storage medium of claim 7, wherein the computer-readable storage medium stores further instructions for the determination of the dominant frequency in the frequency domain data, the instructions when executed by the one or more processors cause the one or more processors to:

determine a frequency value in the frequency domain data that has the largest amplitude of the plurality of frequency values in the frequency domain data;

select the frequency value as the dominant frequency.

12. The non-transitory computer-readable storage medium of claim 7, wherein the computer-readable storage medium stores further instructions for the determination of the modified frame rate, the instructions when executed by the one or more processors cause the one or more processors to:

set the modified frame rate to match the dominant frequency.

13. A camera for dynamic synchronization of frame rate to a detected cadence, the camera comprising:

one or more processors; and

a non-transitory computer-readable storage medium storing instructions, the instructions when executed by the one or more processors causing the one or more processors to perform steps including;

configuring the camera to operate according to an initial frame rate;

initiating capture of a time lapse image sequence on the camera, according to the initial frame rate;

recording motion data of the camera during the capture of the time lapse image sequence, the motion data recorded by one or more sensors tracking motion of the camera while the camera captures the time lapse image sequence;

converting the motion data in a particular window of time from time domain data to frequency domain data;

determining a dominant frequency in the frequency domain data;

determining a modified frame rate that is within a predefined tolerance range of the dominant frequency;

continuing capture of the time lapse image sequence according to the modified frame rate; and

storing the time lapse image sequence to a storage medium.

14. The camera of claim 13, wherein the particular window of time comprises a predefined time window.

15. The camera of claim 13, wherein the motion data comprises position data along an axis of oscillation derived from sensed data captured by a motion sensor.

16. The camera of claim 13, wherein the conversion of the motion data from time domain data to frequency domain data is performed using Fast Fourier Transform.

17. The camera of claim 13, wherein the computer-readable storage medium stores further instructions for the determination of the dominant frequency in the frequency domain data, the instructions when executed by the one or more processors causing the processors to perform steps including:

determining a frequency value in the frequency domain data that has the largest amplitude of the plurality of frequency values in the frequency domain data;

selecting the frequency value as the dominant frequency.

18. The camera of claim 13, wherein the computer-readable storage medium stores further instructions for the determination of the modified frame rate, the instructions when executed by the one or more processors causing the one or more processors to:

setting the modified frame rate to match the dominant frequency.

19. The camera of claim 13, wherein the camera is removably attachable to a user or object in motion.