Systems and methods for determining settings for fast video capture and sensor adjustment

• US 10,469,764 B2
• Filed: 11/01/2017
• Issued: 11/05/2019
• Est. Priority Date: 11/01/2016
• Status: Active Grant

• Current Assignee: Snap, Inc.
• Original Assignee: Snap, Inc.
• Inventors: Brook, Peter
• Primary Examiner(s): Alcon, Fernando
• Application Number:
  US15/801,071
• Publication Number:
  US 20180124300A1
• Time in Patent Office: 734 Days
• Field of Search: None
• CPC Class Codes: H04N 23/45 for generating image signals from two or more image sensors being of different type or operating in different modes, e.g. with a CMOS sensor for moving images in combination with a charge-coupled device [CCD] for still images
• CPC Class Codes: H04N 23/62 Control of parameters via user interfaces
• CPC Class Codes: H04N 23/64 Computer-aided capture of images, e.g. transfer from script file into camera, check of taken image quality, advice or proposal for image composition or decision on when to take image
• CPC Class Codes: H04N 23/661 Transmitting camera control signals through networks, e.g. control via the Internet
• CPC Class Codes: H04N 23/676 Bracketing for image capture at varying focusing conditions
• CPC Class Codes: H04N 23/68 for stable pick-up of the scene, e.g. compensating for camera body vibrations
• CPC Class Codes: H04N 23/71 Circuitry for evaluating the brightness variation
• CPC Class Codes: H04N 23/73 by influencing the exposure time
• CPC Class Codes: H04N 23/74 by influencing the scene brightness using illuminating means
• CPC Class Codes: H04N 23/743 Bracketing, i.e. taking a series of images with varying exposure conditions

Abstract

Systems, methods, devices, media, and instructions are described for fast video sensor adjustment. In one embodiment, a portable electronic device with image capturing capabilities automatically captures a first plurality of frames upon selection of an input button while sensors of the device are determining camera settings for the environment. The first plurality of frames are captured using different automatic bracketing settings. Once the camera settings are determined, a second plurality of images are captured using the determined camera settings. One or more of the first plurality of images are used along with the second plurality of images for a video file. The one or more images may be selected based on a match between certain of the automatic bracketing settings and the final camera settings. A server computer may then aggregate settings from different devices to provide updated automatic bracketing settings to various devices.

Claims

1. A method for generating auto bracketing settings, the method comprising:

receiving, at a server computer from a first device, first video capture data for a first video clip captured using a first set of auto bracketing settings;

analyzing the first video capture data to determine a first set of performance values associated with the first set of auto bracketing settings;

receiving, at the server computer from a second device, second video capture data for a second video clip captured using a second set of auto bracketing settings;

analyzing the second video capture data to determine a second set of performance values associated with the second set of auto bracketing settings; and

generating, based on the first set of performance values and the second set of performance values, a set of auto bracketing update information comprising a set of update auto bracketing settings;

wherein the first set of performance values and the second set of performance values each comprise a quality value for a first frame and a responsiveness value for the first frame of a corresponding video clip.

2. The method of claim 1 further comprising:

receiving, at the server computer from a plurality of devices comprising at least the first device, the second device, and a plurality of additional devices, associated video capture data for a corresponding video clips from each device of the plurality of devices, the associated video capture data comprising corresponding auto bracketing settings for each of the corresponding video clips; and

analyzing the associated video capture data to determine performance values associated with the corresponding auto bracketing settings;

wherein the set of auto bracketing update information is further based on the performance values associated with the corresponding auto bracketing setting.

3. The method of claim 2 wherein analyzing the associated video capture data to determine the performance values comprises analyzing statistical correlation values between quality values, responsiveness values, and individual settings of the corresponding auto bracketing settings;

and

wherein generating the set of auto bracketing update information comprises analyzing quality values and responsiveness values for the corresponding video clips based on a set of weighted tradeoff criteria between quality and responsiveness to select the set of update auto bracketing settings.

4. The method of claim 3 wherein the a set of auto bracketing update information indicates a change in a number of auto bracketing settings and corresponding initial frames to be captured in order to adjust an expected responsiveness performance.

5. The method of claim 3 wherein the first video capture data comprises a number of auto bracketing settings used during capture of the first video clip, auto bracketing values for each auto bracketing setting of the number of auto bracketing settings, and an indicator of a selected auto bracketing setting of the number of auto bracketing setting used to captured the first video clip.

6. The method of claim 5 wherein the first set of auto bracketing settings comprises an International Standards Organization (ISO) setting and an exposure level setting.

7. The method of claim 6, wherein the first set of auto bracketing settings further comprises one or more settings of:

(a) a camera shutter speed based on one or more of an image brightness metric and an image sharpness metric for a first frame of the first plurality of frames;

(b) camera exposure settings based on one or more of the image brightness metric and the image sharpness metric of the first frame of the first plurality of frames;

(c) camera focus settings;

(d) camera white balance settings based on an image colorization metric;

(e) camera flash settings based on the image brightness metric; and

(f) image stabilization settings based on one or more of the image brightness metric and the image sharpness metric, so that automatic adjustment can be made to the amount of on-board processing devoted to photographic image stabilization.

8. The method of claim 7 wherein the first video capture data further comprises a data and time associated with capture of the first video clip.

9. The method of claim 1 further comprising communicating the set of auto bracketing update information to a third device different from the first device and the second device.

10. The method of claim 9 further comprising sending the auto bracketing update information to the first device and the second device.

11. The method of claim 1 wherein the first video capture data is received at the server computer as part of an ephemeral message comprising the first video clip;

wherein the first video capture data is stored in a database independent of the first video clip; and

wherein the first video clip is deleted from the server computer in response to an ephemeral message deletion trigger.

12. The method of claim 1 wherein the first video capture data is generated by a wearable device performing operations comprising:

receiving, at an input mechanism of a wearable camera device associated with the first device, a user input to initiate capture of first video data;

capturing a first plurality of frames using camera settings based on a the first set of auto bracketing settings in response to the user input, each frame of the first plurality of frames having associated camera settings which are different for each frame of the first plurality of frames, wherein the associated camera settings are determined from the first set of auto bracketing settings;

processing, in response to the user input, a set of sensor data to determine a set of camera setting values for an environment of the wearable camera device;

selecting the camera settings for the wearable camera device using the set of camera setting values after the first plurality of frames are captured; and

capturing a second plurality of frames with the camera settings selected using the set of camera setting values.

13. The method of claim 12, further comprising:

selecting at least a first frame of the first plurality of frames for inclusion with the first video data based at least in part on associated camera settings for the first frame.

14. The method of claim 12 wherein the first device comprises a smartphone configured to receive the first video data from the wearable camera device and generate the first video clip for communication to the server computer based on the first video data from the wearable device.

15. A device comprising:

memory configured store a first ephemeral message from a first device and a second ephemeral message from a second device, the first ephemeral message comprising a first video clip and first video capture data, and the second ephemeral message comprising a second video clip and second video capture data; and

one or more processors coupled to the memory and configured to;

analyze the first video capture data to determine a first set of performance values associated with a first set of auto bracketing settings;

analyze the second video capture data to determine a second set of performance values associated with a second set of auto bracketing settings;

select, based on the first set of performance values and the second set of performance values, a set of auto bracketing update information comprising a set of update auto bracketing settings; and

initiate transmission of the set of updated auto bracketing settings to one or more devices for use in capturing video data;

wherein the first set of performance values and the second set of performance values each comprise a quality value for a first frame and a responsiveness value for the first frame of a corresponding video clip.

16. The device of claim 15, wherein the a set of auto bracketing update information indicates a change in a number of auto bracketing settings and corresponding initial frames to be captured in order to adjust an expected responsiveness performance against an expected quality performance.

17. A non-transitory computer-readable storage medium having stored thereon instructions that, when executed by a machine, cause the machine to perform operations comprising:

receiving a first ephemeral message from a first device and a second ephemeral message from a second device, the first ephemeral message comprising a first video clip and first video capture data, and the second ephemeral message comprising a second video clip and second video capture data;

analyzing the first video capture data to determine a first set of performance values associated with a first set of auto bracketing settings;

analyzing the second video capture data to determine a second set of performance values associated with a second set of auto bracketing settings;

selecting, based on the first set of performance values and the second set of performance values, a set of auto bracketing update information comprising a set of update auto bracketing settings; and

initiating transmission of the set of updated auto bracketing settings to one or more devices for use in capturing video data;

wherein the first set of performance values and the second set of performance values each comprise a quality value for a first frame and a responsiveness value for the first frame of a corresponding video clip.

18. The non-transitory computer readable storage medium of claim 17 wherein the a set of auto bracketing update information indicates a change in a number of auto bracketing settings and corresponding initial frames to be captured in order to adjust an expected responsiveness performance against an expected quality performance.

19. The non-transitory computer readable storage medium of claim 17, each auto bracketing setting of the set of auto bracketing settings comprises at least an International Standards Organization (ISO) setting and an exposure level setting.

20. The non-transitory computer readable storage medium of claim 18, wherein the first set of auto bracketing settings further comprises one or more settings of:

(a) a camera shutter speed based on one or more of an image brightness metric and an image sharpness metric for a first frame of the first plurality of frames;

(b) camera exposure settings based on one or more of the image brightness metric and the image sharpness metric of the first frame of the first plurality of frames;

(c) camera focus settings;

(d) camera white balance settings based on an image colorization metric;

(e) camera flash settings based on the image brightness metric; and

(f) image stabilization settings based on one or more of the image brightness metric and the image sharpness metric, so that automatic adjustment can be made to the amount of on-board processing devoted to photographic image stabilization.