Systems and methods for fast video capture and sensor adjustment

• US 10,432,874 B2
• Filed: 11/01/2017
• Issued: 10/01/2019
• Est. Priority Date: 11/01/2016
• Status: Active Grant

• Current Assignee: Snap, Inc.
• Original Assignee: Snap, Inc.
• Inventors: Brook, Peter
• Primary Examiner(s): Tissire, Abdelaaziz
• Application Number:
  US15/801,064
• Publication Number:
  US 20180124299A1
• Time in Patent Office: 699 Days
• CPC Class Codes: H04N 23/45 for generating image signals from two or more image sensors being of different type or operating in different modes, e.g. with a CMOS sensor for moving images in combination with a charge-coupled device [CCD] for still images
• CPC Class Codes: H04N 23/62 Control of parameters via user interfaces
• CPC Class Codes: H04N 23/64 Computer-aided capture of images, e.g. transfer from script file into camera, check of taken image quality, advice or proposal for image composition or decision on when to take image
• CPC Class Codes: H04N 23/661 Transmitting camera control signals through networks, e.g. control via the Internet
• CPC Class Codes: H04N 23/676 Bracketing for image capture at varying focusing conditions
• CPC Class Codes: H04N 23/68 for stable pick-up of the scene, e.g. compensating for camera body vibrations
• CPC Class Codes: H04N 23/71 Circuitry for evaluating the brightness variation
• CPC Class Codes: H04N 23/73 by influencing the exposure time
• CPC Class Codes: H04N 23/74 by influencing the scene brightness using illuminating means
• CPC Class Codes: H04N 23/743 Bracketing, i.e. taking a series of images with varying exposure conditions

Abstract

Systems, methods, devices, media, and instructions are described for fast video sensor adjustment. In one embodiment, a portable electronic device with image capturing capabilities automatically captures a first plurality of frames upon selection of an input button while sensors of the device are determining camera settings for the environment. The first plurality of frames are captured using different automatic bracketing settings. Once the camera settings are determined, a second plurality of images are captured using the determined camera settings. One or more of the first plurality of images are used along with the second plurality of images for a video file. The one or more images may be selected based on a match between certain of the automatic bracketing settings and the final camera settings.

Claims

1. A method comprising:

receiving, at an input mechanism of a wearable camera device, a user input to initiate capture of first video data;

capturing a first plurality of frames using camera settings based on a set of auto bracketing settings in response to the user input, each frame of the first plurality of frames having associated camera settings which are different for each frame of the first plurality of frames, wherein the associated camera settings are determined from the set of auto bracketing settings;

processing, in response to the user input, a set of sensor data to determine a set of camera setting values for an environment of the wearable camera device;

selecting the camera settings for the wearable camera device using the set of camera setting values after the first plurality of frames are captured; and

capturing a second plurality of frames with the camera settings selected using the set of camera setting values;

wherein the set of sensor data comprises the first plurality of frames; and

wherein the set of camera setting values comprises;

a number of blacked out pixels for each frame of the first plurality of frames;

a number of low-luminosity pixels for each frame of the first plurality of frames, each low-luminosity pixel having a luminosity value lower than a predefined threshold;

an average luminosity value for pixels in each frame of the first plurality of frames; and

a median luminosity of pixels and macro-pixels in each frame of the first plurality of frames.

2. The method of claim 1, further comprising:

selecting at least a first frame of the first plurality of frames for inclusion with the first video data based at least in part on associated camera settings for the first frame.

3. The method of claim 2, further comprising:

storing the first video data as a data file comprising the first frame and the second plurality of frames.

4. The method of claim 3, further comprising:

receiving the set of auto bracketing settings from a network server computer system via a wireless transceiver of the wearable device prior to capture of the first plurality of frames.

5. The method of claim 4 further comprising:

communicating, to the network server computer system via the wireless transceiver, the set of camera settings.

6. The method of claim 5, further comprising:

communicating the set of camera settings to the network server computer system with a set of context data.

7. The method of claim 6, wherein the context data comprises a time of day and a location.

8. The method of claim 6, further comprising:

receiving, from the network server computer system following communication of the set of camera settings to the network server computer, updated auto bracketing settings, wherein the updated auto bracketing settings are generated by the network server computer system based on a plurality of sets of camera settings from a plurality of camera devices; and

wherein the plurality of camera devices comprises the wearable camera device.

9. The method of claim 1 wherein the camera settings comprise an International Standards Organization (ISO) setting and an exposure level setting.

10. The method of claim 9, wherein the camera settings further comprise one or more settings of:

(a) a camera shutter speed based on one or more of an image brightness metric and an image sharpness metric for a first frame of the first plurality of frames;

(b) camera exposure settings based on one or more of the image brightness metric and the image sharpness metric of the first frame of the first plurality of frames;

(c) camera focus settings;

(d) camera white balance settings based on an image colorization metric;

(e) camera flash settings based on the image brightness metric; and

(f) image stabilization settings based on one or more of the image brightness metric and the image sharpness metric, so that automatic adjustment can be made to the amount of on-board processing devoted to photographic image stabilization.

11. The method of claim 1, wherein the camera settings are selected by analyzing a frame quality for each frame of the first plurality of frames, and selecting the camera, settings based on auto bracketing settings for a highest quality frame.

12. The method of claim 1, wherein the set of auto bracketing settings are selected from a plurality of auto bracketing settings based on a device clock.

13. The method of claim 12, wherein the plurality of auto bracketing settings are received at the wearable device from a server computer.

14. The method of claim 1, wherein the set of auto bracketing settings are selected from a plurality of auto bracketing settings based on a user selection received prior to the user input.

15. A portable electronic device comprising:

a frame;

a camera mounted on the frame and configured to capture and process image data according to a plurality of image capture parameters;

an input mechanism provided on the frame and configured for receiving se input to control operation of the camera, the input mechanism being disposable by a user between an engaged condition and a disengaged condition; and

a camera controller incorporated in the frame and configured to perform an automated bracketing action with respect to a first plurality of images captured, the camera comprising;

receiving, at the input mechanism of a wearable camera device, a user input to initiate capture of first video data;

capturing the first plurality of frames using camera settings based on a set of auto bracketing settings in response to the user input, each frame of the first plurality of frames have associated camera settings which are different for each frame of the first plurality of frames, and the associated camera settings are determined from the set of auto bracketing settings;

processing, in response to the user input, a set of sensor data to determine a set of camera setting values for an environment of the wearable camera device;

selecting the camera settings for the wearable camera device using the set of camera setting values after the first plurality of frames are captured;

capturing a second plurality of frames with the camera settings selected using the set of camera setting values;

selecting at least a first frame of the first plurality of frames for inclusion with the first video data based at least in part on the associated camera settings for the first frame; and

storing the first video data as a data file comprising the first frame and the second plurality of frames;

wherein the set of sensor data comprises the first plurality of frames; and

wherein the set of camera setting values comprises;

a number of blacked out pixels for each frame of the first plurality of frames;

a number of low-luminosity pixels for each frame of the first plurality of frames, each low-luminosity pixel having a luminosity value lower than a predefined threshold;

an average luminosity value for pixels in each frame of the first plurality of frames; and

a median luminosity of pixels and macro-pixels in each frame of the first plurality of frames.

16. The device of claim 15, wherein the user input comprises placement of the input mechanism in the engaged position during a first time period for capture of the first plurality of frames and the second plurality of frames;

wherein the first video data is stored in response to the first time period with the engaged position having a duration longer than a threshold time period; and

wherein the device is configured to store image data when the input mechanism returns to a disengaged position after being in the engaged position for less than the threshold time period.

17. The device of claim 16, wherein the user input comprises a single action mechanical button.

18. A non-transitory computer-readable storage medium having stored thereon instructions that, when executed by a wearable camera device, cause the wearable camera device to perform operations comprising:

detecting a user engagement with an input mechanism forming part of a portable electronic device having an on-board camera configured to capture and process image data according to a plurality of image capture parameters, the user engagement comprising disposal of the input mechanism from a disengaged condition to an engaged condition;

processing, in response to the user input, a set of sensor data to determine a set of camera setting values for an environment of the wearable camera device;

capturing a first plurality of frames using camera settings based on a set of auto bracketing settings in response to the user input, each frame of the first plurality of frames having associated camera settings which are different frame each frame of the first plurality of frames, wherein the associated camera settings are determined from the set of auto bracketing settings;

selecting the camera settings for the wearable camera device using the set of camera setting values after the first plurality of frames are captured;

capturing a second plurality of frames with the camera settings selected using the camera settings;

selecting at least a first frame of the first plurality of frames for inclusion with the first video data based at least in part on the camera settings for the first frame; and

storing the first video data as a data file comprising the first frame and the second plurality of frames;

wherein the set of sensor data comprises the first plurality of frames; and

wherein the set of camera setting values comprises;

a number of blacked out pixels for each frame of the first plurality of frames;

a number of low-luminosity pixels for each frame of the first plurality of frames, each low-luminosity pixel having a luminosity value lower than a predefined threshold;

an average luminosity value for pixels in each frame of the first plurality of frames; and

a median luminosity of pixels and macro-pixels in each frame of the first plurality of frames.

19. The non-transitory computer readable storage medium of claim 18, wherein the instructions further cause the machine to perform operations comprising:

communicating, to a server computer system via a host device, an ephemeral message comprising the data file and a set of context data associated with the set of camera setting values.

20. The non-transitory computer readable storage medium of claim 18, wherein the camera settings further comprise one or more settings of:

(a) a camera shutter speed based on one or more of an image brightness metric and an image sharpness metric for a first frame of the first plurality of frames;

(b) camera exposure settings based on one or more of the image brightness metric and the image sharpness metric of the first frame of the first plurality of frames;

(c) camera focus settings;

(d) camera white balance settings based on an image colorization metric;

(e) camera flash settings based on the image brightness metric; and

(f) image stabilization settings based on one or more of the image brightness metric and the image sharpness metric, so that automatic adjustment can be made to the amount of on-board processing devoted to photographic image stabilization.