Systems and methods for using a variable capture frame rate for video capture

• US 10,187,607 B1
• Filed: 04/04/2017
• Issued: 01/22/2019
• Est. Priority Date: 04/04/2017
• Status: Active Grant

• Current Assignee: GoPro, Inc.
• Original Assignee: GoPro, Inc.
• Inventors: Delachanal, Christophe
• Primary Examiner(s): Nazrul, Shahbaz
• Application Number:
  US15/478,979
• Time in Patent Office: 658 Days
• CPC Class Codes: H04N 23/6811 based on the image signal
• CPC Class Codes: H04N 23/6812 based on additional sensors, e.g. acceleration sensors
• CPC Class Codes: H04N 23/951 by using two or more images to influence resolution, frame rate or aspect ratio
• CPC Class Codes: H04N 5/144 Movement detection for video coding H04N19/503; analysis of motion in general G06T7/20
• CPC Class Codes: H04N 5/783 Adaptations for reproducing at a rate different from the recording rate
• CPC Class Codes: H04N 7/0127 by changing the field or frame frequency of the incoming video signal, e.g. frame rate converter

Abstract

An image sensor may generate visual output signals conveying visual information within a field of view of the image sensor. First and second video information defining first and video content may be generated based on the visual output signals. The first video content may be captured using a first capture frame rate and the second video content may be captured using a second capture frame rate. Motion of the image sensor and/or motion of an object within the field of view may be determined. The first capture frame rate may be changed based on the determined motion. Third video information defining third video content may be generated based on the first video information and the second video information. The third video content may include one or more frames of the first video content and one or more frames of the second video content.

Claims

1. A system using a variable capture frame rate for video capture, the system comprising:

an image sensor configured to generate visual output signals conveying visual information within a field of view of the image sensor;

a motion sensor configured to generate motion output signals conveying motion information of the image sensor;

one or more physical processors configured by machines readable instructions to;

generate first video information defining first video content based on the visual output signals, the first video content captured using a first capture frame rate, the first capture frame rate set to a first value defining a first number of frames captured per a duration of time, the first video content having a first progress length;

generate second video information defining second video content based on the visual output signals, the second video content captured using a second capture frame rate, the second capture frame rate set to a second value defining a second number of frames captured per the duration of time, the second video content having a second progress length;

determine a motion of the image sensor based on the motion output signals and/or a motion of an object within the field of view of the image sensor based on the visual output signals;

change the first capture frame rate from the first value to a third value based on the motion of the image sensor and/or the motion of the object, wherein a frame rate-change time duration exists between (1) the determination of the motion of the image sensor and/or the motion of the object and (2) the change of the first capture frame rate from the first value to the third value; and

generate third video information defining third video content based on the first video information and the second video information, the third video content including (i) one or more frames of the first video content, and (ii) one or more frames of the second video content corresponding to the frame rate-change time duration, the third video information generated based on the first value defining a lower number of frames captured per the duration of time than (1) the second value, and (2) the third value.

2. The system of claim 1, wherein the third video content includes one or more interpolated frames generated from two or more frames of the first video content and/or the second video content.

3. The system of claim 1, wherein:

the motion of the image sensor and/or the motion of the object is determined at a moment corresponding to a point in the second progress length of the second video content; and

some or all of the frames of the second video content included in the third video content corresponds to a range in the second progress length before the point in the second progress length.

4. The system of claim 1, wherein including the one or more frames of the second video content corresponding to the frame rate-change time duration in the third video content compensates for an amount of time taken to change the first capture frame rate from the first value to the third value.

5. The system of claim 1, wherein the second video information is stored in a buffer memory.

6. The system of claim 1, wherein the second progress length is shorter than the first progress length.

7. The system of claim 1, wherein the motion of the image sensor includes one or more of a linear speed, an angular speed, a linear velocity, an angular velocity, a linear acceleration, and/or an angular acceleration of the image sensor.

8. The system of claim 1, wherein the motion of the object includes one or more of a linear speed, an angular speed, a linear velocity, an angular velocity, a linear acceleration, and/or an angular acceleration of the object.

9. The system of claim 1, wherein the motion of the object is determined based on one or more motion vectors.

10. A method using a variable capture frame rate for video capture, the method comprising:

generating visual output signals conveying visual information within a field of view of an image sensor;

generating motion output signals conveying motion information of the image sensor;

generating first video information defining first video content based on the visual output signals, the first video content captured using a first capture frame rate, the first capture frame rate set to a first value defining a first number of frames captured per a duration of time, the first video content having a first progress length;

generating second video information defining second video content based on the visual output signals, the second video content captured using a second capture frame rate, the second capture frame rate set to a second value defining a second number of frames captured per the duration of time, the second video content having a second progress length;

determining a motion of the image sensor based on the motion output signals and/or a motion of an object within the field of view of the image sensor based on the visual output signals;

changing the first capture frame rate from the first value to a third value based on the motion of the image sensor and/or the motion of the object, wherein a frame rate-change time duration exists between (1) the determination of the motion of the image sensor and/or the motion of the object and (2) the change of the first capture frame rate from the first value to the third value; and

generating third video information defining third video content based on the first video information and the second video information, the third video content including (i) one or more frames of the first video content, and (ii) one or more frames of the second video content corresponding to the frame rate-change time duration, the third video information generated based on the first value defining a lower number of frames captured per the duration of time than (1) the second value, and (2) the third value.

11. The method of claim 10, wherein the third video content includes one or more interpolated frames generated from two or more frames of the first video content and/or the second video content.

12. The method of claim 10, wherein:

the motion of the image sensor and/or the motion of the object is determined at a moment corresponding to a point in the second progress length of the second video content; and

some or all of the frames of the second video content included in the third video content corresponds to a range in the second progress length before the point in the second progress length.

13. The method of claim 10, wherein including the one or more frames of the second video content corresponding to the frame rate-change time duration in the third video content compensates for an amount of time taken to change the first capture frame rate from the first value to the third value.

14. The method of claim 10, wherein the second video information is stored in a buffer memory.

15. The method of claim 10, wherein the second progress length is shorter than the first progress length.

16. The method of claim 10, wherein the motion of the image sensor includes one or more of a linear speed, an angular speed, a linear velocity, an angular velocity, a linear acceleration, and/or an angular acceleration of the image sensor.

17. The method of claim 10, wherein the motion of the object includes one or more of a linear speed, an angular speed, a linear velocity, an angular velocity, a linear acceleration, and/or an angular acceleration of the object.

18. The method of claim 10, wherein the motion of the object is determined based on one or more motion vectors.

19. A system using a variable capture frame rate for video capture, the system comprising:

an image sensor configured to generate visual output signals conveying visual information within a field of view of the image sensor;

a motion sensor configured to generate motion output signals conveying motion information of the image sensor;

one or more physical processors configured by machines readable instructions to;

generate first video information defining first video content based on the visual output signals, the first video content captured using a first capture frame rate, the first capture frame rate set to a first value defining a first number of frames captured per a duration of time, the first video content having a first progress length;

generate second video information defining second video content based on the visual output signals, the second video content captured using a second capture frame rate, the second capture frame rate set to a second value defining a second number of frames captured per the duration of time, the second video content having a second progress length, the second progress length being shorter than the first progress length, wherein the second video information is stored in a buffer memory;

determine a motion of the image sensor based on the motion output signals and/or a motion of an object within the field of view of the image sensor based on the visual output signals, the motion of the image sensor and/or the motion of the object determined at a moment corresponding to a point in the second progress length of the second video content;

change the first capture frame rate from the first value to a third value based on the motion of the image sensor and/or the motion of the object, wherein a frame rate-change time duration exists between (1) the determination of the motion of the image sensor and/or the motion of the object and (2) the change of the first capture frame rate from the first value to the third value; and

generate third video information defining third video content based on the first video information and the second video information, the third video content including (i) one or more frames of the first video content, and (ii) one or more frames of the second video content corresponding to the frame rate-change time duration, some or all of the frames of the second video content included in the third video content corresponding to a range in the second progress length before the point in the second progress length, the third video information generated based on the first value defining a lower number of frames captured per the duration of time than (1) the second value, and (2) the third value.

20. The system of claim 19, wherein the third video content includes one or more interpolated frames generated from two or more frames of the first video content and/or the second video content.