Automatic image capture

• US 9,596,398 B2
• Filed: 09/02/2011
• Issued: 03/14/2017
• Est. Priority Date: 09/02/2011
• Status: Active Grant

• Current Assignee: Microsoft Technology Licensing LLC (Microsoft Corporation)
• Original Assignee: Microsoft Technology Licensing LLC (Microsoft Corporation)
• Inventors: Khawand, Charbel
• Primary Examiner(s): Vu, Ngoc-Yen
• Application Number:
  US13/224,704
• Publication Number:
  US 20130057713A1
• Time in Patent Office: 2,020 Days
• Field of Search: 348/208.1, 348/61
• US Class Current: 1/1
• CPC Class Codes: H04N 23/10 for generating image signals from different wavelengths
• CPC Class Codes: H04N 23/62 Control of parameters via user interfaces
• CPC Class Codes: H04N 23/68 for stable pick-up of the scene, e.g. compensating for camera body vibrations
• CPC Class Codes: H04N 23/6812 based on additional sensors, e.g. acceleration sensors
• CPC Class Codes: H04N 23/80 Camera processing pipelines; Components thereof

Abstract

Disclosed herein are exemplary embodiments for automatically capturing images in a mobile electronic device. One embodiment comprises sensing device motion and automatically triggering image capture when the device is stationary. Use of this approach reduces image blur and avoids the need for subsequent image alteration or compensation for camera motion. Images can simply be captured in-between motions by leveraging high-resolution sensors and computational assets available to the mobile device to accurately assess when to trigger the shutter. Images can then be saved in a memory within the device. Enhancements to the disclosed method of automatic image capture include pre-selecting a set of threshold values for image acceptance.

Claims

1. A method of performing automatic image capture with a mobile device, the method comprising:

receiving data indicative of trigger criteria selected by a user for image acceptance, the user-selected trigger criteria comprising user-selected image quality trigger criteria, the user-selected image quality trigger criteria comprising thresholds for one or more of a white balance process, an auto exposure process, a de-noising process, or a gain control process;

entering an auto-capture camera mode;

determining whether the trigger criteria are satisfied for an image sensed by an image sensor;

based at least on determining that the trigger criteria are satisfied, and without further input from the user, triggering capture of the image; and

storing the image in the memory,

wherein the entering the auto-capture camera mode is based at least on the user selecting the auto-capture camera mode from a user interface displayed on the mobile device prior to each of the determining, triggering, and storing, and

wherein the triggering the capture of the image comprises continuously processing sets of image data in an image processing pipeline as the sets of image data are received at the image sensor, the processing pipeline including demosaicing processing and post-demosaicing processing, the demosaicing processing being continuously performed as the sets of image data are received but the post-demosaicing processing being restricted from being performed until the capture of the image is triggered.

2. The method of claim 1, wherein the user-selected trigger criteria further comprise a criterion indicating whether the mobile device is stationary, and wherein the method further comprises determining whether the mobile device is stationary using data received from one or more of an accelerometer or gyroscope.

3. The method of claim 1, wherein the method comprises discontinuing image processing before performing one or more of image scaling, YUV processing, or image rendering for an image sensed by the image sensor if the one or more trigger criteria are not met.

4. The method of claim 1, wherein the image sensor is part of a camera that is incorporated into the mobile device.

5. The method of claim 1, wherein the capture of the image comprises performing one or more of image scaling, YUV processing, or image rendering.

6. The method of claim 1, wherein the method further comprises receiving data indicative of a user-selected length of time to pause automatic image capture after activation of the auto-capture camera mode, and wherein the determining, triggering, and storing are delayed upon the user selecting the auto-capture camera mode for the user-selected length of time.

7. The method of claim 1, wherein the method further comprises receiving data indicative of a user-selected number of images to capture while in the auto-capture camera mode, and wherein the determining, triggering, and storing are performed continuously until the user-selected number of images has been captured and stored.

8. One or more computer-readable memory or storage devices storing computer-executable instructions which when executed by a computing device cause the computing device to perform a method of performing automatic image capture with a mobile device, the method comprising:

receiving data indicative of trigger criteria selected by a user for image acceptance, the user-selected trigger criteria comprising user-selected image quality trigger criteria, the user-selected image quality trigger criteria comprising thresholds for one or more of a white balance process, an auto exposure process, a de-noising process, or a gain control process;

entering an auto-capture camera mode;

determining whether the trigger criteria are satisfied for an image sensed by an image sensor;

based at least on determining that the trigger criteria are satisfied, and without further input from the user, triggering capture of the image; and

storing the image in the memory,

wherein the entering the auto-capture camera mode is based at least on the user selecting the auto-capture camera mode from a user interface displayed on the mobile device prior to each of the determining, triggering, and storing, and

wherein the triggering the capture of the image comprises continuously processing sets of image data in an image processing pipeline as the sets of image data are received at the image sensor, the processing pipeline including demosaicing processing and post-demosaicing processing, the demosaicing processing being continuously performed as the sets of image data are received but the post-demosaicing processing being restricted from being performed until the capture of the image is triggered.

9. A mobile device comprising:

a lens;

an electronic image sensor;

a touchscreen display;

one or more motion detectors configured to sense motion of the mobile device;

a memory; and

a processor programmed to enter an auto-capture camera mode upon receiving an indication via the touchscreen display and, after entering the auto-capture camera mode, to make a determination from data provided by the one or more motion detectors of whether the mobile device is stationary, and, based at least in part on the determination, to trigger image capture and storage of the image in the memory,

wherein the determination of whether the mobile device is stationary comprises determining whether changes to motion measurements detected by the one or more motion detectors satisfy a threshold value during a period of time, and

wherein image data is processed in an image processing pipeline, the image processing pipeline including demosaicing processing and post-demosaicing processing, the demosaicing processing being continuously performed as sets of image data are received but the post-demosaicing processing being restricted from being performed until the mobile device is determined to be stationary and image capture is triggered.

10. The mobile device of claim 9, wherein the one or more motion detectors comprise one or more of an accelerometer or a gyroscope.

11. The mobile device of claim 9, wherein the processor is further programmed to make an image quality determination by evaluating whether one or more additional image quality criteria are satisfied, and to trigger or avoid the image capture and storage of the image based at least in part on the image quality determination.

12. The mobile device of claim 11, wherein the one or more additional image quality criteria comprise one or more of a white balance value, a focus value, a de-noising value, a gain value, a sharpness value, or a flash control value.

13. The mobile device of claim 11, wherein the one or more additional image quality criteria are user-selectable.

14. The mobile device of claim 9, wherein the mobile device further comprises a hardware button, and wherein the processor is further programmed to exit an automatic image capture mode when the hardware button is depressed.

15. The mobile device of claim 9, wherein the processor is further programmed to enter and exit an automatic image capture mode when a touchscreen button on the touchscreen display is selected.

16. The mobile device of claim 9, wherein both the threshold value and the period of time over which the determination is made are variable and based on a user-selected value.

17. A method for automatic image capture using a mobile electronic device camera, the method comprising:

determining whether the camera is focused on a subject;

sensing device motion;

continuously processing sets of image data in an image processing pipeline as the sets of image data are received at an image sensor, the processing pipeline including demosaicing processing and post-demosaicing processing, the demosaicing processing being continuously performed as the sets of image data are received but the post-demosaicing processing being restricted from being performed until the device is stationary and the camera is focused on the subject; and

saving an image generated by post-demosaicing processing in a memory.

18. The method of claim 17, wherein the determining comprises comparing data from successive image frames.

19. The method of claim 17, wherein the method further comprises performing image recognition to evaluate whether or not the subject meets a user-specified criterion, and wherein the post-demosaicing processing is performed when the device is stationary, when the camera is focused on the subject, and when the subject meets the user-specified criterion.

20. The method of claim 17, wherein the post-demosaicing processing that is restricted includes scaling and YUV processing.

21. One or more computer-readable memory or storage devices storing computer-executable instructions which when executed by a computing device cause the computing device to perform a method for automatic image capture using a mobile electronic device camera, the method comprising:

determining whether the camera is focused on a subject;

sensing device motion;

continuously processing sets of image data in an image processing pipeline as the sets of image data are received at an image sensor, the processing pipeline including demosaicing processing and post-demosaicing processing, the demosaicing processing being continuously performed as the sets of image data are received but the post-demosaicing processing being restricted from being performed until the device is stationary and the camera is focused on the subject; and

saving an image generated by post-demosaicing processing in a memory.