Enhanced image capture

• US 9,813,611 B2
• Filed: 08/04/2014
• Issued: 11/07/2017
• Est. Priority Date: 05/21/2014
• Status: Active Grant

• Current Assignee: Google Technology Holdings LLC (Alphabet Inc.)
• Original Assignee: Google Technology Holdings LLC (Alphabet Inc.)
• Inventors: Petrescu, Doina I., Lay, Thomas T., Petrie, Steven R., Ryan, Bill, Sinha, Snigdha, Vanhoof, Jeffrey S.
• Primary Examiner(s): Tran, Nhan T
• Application Number:
  US14/450,461
• Publication Number:
  US 20150341547A1
• Time in Patent Office: 1,191 Days
• Field of Search: 3482071, 34820711, 3482111, 3482112, 3482201, 382116
• CPC Class Codes: H04N 23/61 based on recognised objects
• CPC Class Codes: H04N 23/611 where the recognised objects include parts of the human body
• CPC Class Codes: H04N 23/633 for displaying additional information relating to control or operation of the camera
• CPC Class Codes: H04N 23/64 Computer-aided capture of images, e.g. transfer from script file into camera, check of taken image quality, advice or proposal for image composition or decision on when to take image
• CPC Class Codes: H04N 23/661 Transmitting camera control signals through networks, e.g. control via the Internet
• CPC Class Codes: H04N 23/80 Camera processing pipelines; Components thereof
• CPC Class Codes: H04N 5/772 the recording apparatus and the television camera being placed in the same enclosure

Abstract

Disclosed are techniques that provide a “best” picture taken within a few seconds of the moment when a capture command is received (e.g., when the “shutter” button is pressed). In some situations, several still images are automatically (that is, without the user'"'"'s input) captured. These images are compared to find a “best” image that is presented to the photographer for consideration. Video is also captured automatically and analyzed to see if there is an action scene or other motion content around the time of the capture command. If the analysis reveals anything interesting, then the video clip is presented to the photographer. The video clip may be cropped to match the still-capture scene and to remove transitory parts. Higher-precision horizon detection may be provided based on motion analysis and on pixel-data analysis.

Claims

1. A method comprising:

capturing, by an image-capture device, video before receiving an indication of a capture command;

receiving, by the image-capture device and from a server remote from the image capture device, at least one recommendation based on analysis of metadata from another image capture device;

continuously analyzing, by the image-capture device and based on the received at least one recommendation, the captured video to identify a segment of video with an interest score above a threshold; and

presenting, responsive to an identified segment of video with an interest score above the threshold and by the image-capture device on a user interface of the image-capture device, a notification that the identified segment of video with an interest score above the threshold is available.

2. The method of claim 1 wherein parameters associated with the capturing of the video before receiving the indication of a capture command are based, at least in part, on levels of resources available to the image-capture device.

3. The method of claim 1 wherein the continuously analyzing is based, at least in part, on activity detection, intra-frame motion estimation, inter-frame motion estimation, face detection, motion-sensing, camera-state sensing, or scene analysis.

4. The method of claim 1 wherein the continuously analyzing is based, at least in part, on a stated preference of a user of the image-capture device, past behavior of the user of the image-capture device, or a privacy setting.

5. The method of claim 1 further comprising receiving, by the image-capture device and from the server, at least one refined operational parameter based on analysis of the metadata from the another image capture device,

wherein the continuously analyzing is further based, at least in part, on the received at least one refined operational parameter.

6. The method of claim 1 further comprising:

receiving a capture command;

wherein the continuously analyzing is based, at least in part, on a time of the capture command.

7. The method of claim 1 further comprising

storing, in a circular buffer, a most recent, finite portion of the captured video for analysis, the video being captured before receiving an indication of a capture command;

wherein the stored, finite portion of the captured video is continuously refreshed, a latest frame of the captured video replacing an oldest frame of the finite portion of the captured video in the circular buffer.

8. The method of claim 7 further comprising

in response to the continuously analyzing identifying a segment of video with an interest score above the threshold, and responsive to a user command confirming interest in the identified segment of the video by the user, copying the identified segment of the video to long-term storage outside of the circular buffer.

9. The method of claim 1 further comprising:

delimiting a segment of video, the delimiting based, at least in part, on the analyzing of the captured video or motion-sensor data.

10. An image-capture device comprising:

an image-capture system configured to capture video before receiving an indication of a capture command;

a user interface;

a communications interface configured to receive, from a server remote from the image-capture device, at least one recommendation based on analysis of metadata from another image capture device; and

a processing system operatively connected to the image-capture system, to the communications interface, and to the user interface, the processing system configured to;

continuously analyze, based on the received at least one recommendation, the captured video to identify a segment of video with an interest score above a threshold; and

present, responsive to an identified segment of video with an interest score above the threshold and via the user interface, a notification that the identified segment of video with an interest score above the threshold is available.

11. The image-capture device of claim 10 wherein the device is selected from a camera, a cellphone, or a tablet computer.

12. The image-capture device of claim 10 wherein parameters associated with the capture of the video before receiving the indication of a capture command are based on levels of resources available to the image-capture device.

13. The image-capture device of claim 10 wherein the processing system is configured to continuously analyze based, at least in part, on activity detection, intra-frame motion estimation, inter-frame motion estimation, face detection, motion-sensing, camera-state sensing, or scene analysis.

14. The image-capture device of claim 10 wherein the processing system is configured to continuously analyze based, at least in part, on a stated preference of a user of the image-capture device, past behavior of the user of the image-capture device, or a privacy setting.

15. The image-capture device of claim 10

wherein the communications interface is further configured to receive, from the server, at least one refined operational parameter based on analysis of the metadata from the another image capture device, and

wherein the continuously analyzing is further based, at least in part, on the received at least one refined operational parameter.

16. The image-capture device of claim 10 wherein

the user interface is configured to receive a capture command, and

the processing system is configured to continuously analyze based, at least in part, on a time of the capture command.

17. The image-capture device of claim 10 further comprising:

a circular buffer configured to store a most recent, finite portion of the captured video for analysis, the video captured before receiving the indication of a capture command;

wherein the stored, finite portion of the captured video is continuously refreshed, a latest frame of the captured video replacing an oldest frame of the finite portion of the captured video in the circular buffer.

18. The image-capture device of claim 10 further comprising

long-term storage outside of the circular buffer;

wherein the processing system is further configured to;

in response to identifying a segment of video with an interest score above the threshold, and in response to a user command confirming interest in the identified segment of the captured video by the user, copy the identified segment of the captured video to the long-term storage outside of the circular buffer.

19. The image-capture device of claim 10 wherein the processing system is further configured to:

delimit a segment of video based, at least in part, on the continuously analyzing of the captured video or motion-sensor data.