AUTOMATIC INTELLIGENT FOCUS CONTROL OF VIDEO

• US 20130169754A1
• Filed: 01/03/2012
• Published: 07/04/2013
• Est. Priority Date: 01/03/2012
• Status: Active Grant

• Current Assignee: Sony Mobile Communications Incorporated (Sony Group Corp.)
• Original Assignee: Sony Ericsson Mobile Communications USA Incorporated (Sony Group Corp.)
• Inventors: Aronsson, Pr-Anders, de Lon, David, Jonsson, Hkan, Klinghult, Gunnar, Thrn, Ola
• Granted Patent: US 8,941,722 B2
• Time in Patent Office: Days
• US Class Current: 348/46
• CPC Class Codes: H04N 13/128 Adjusting depth or disparity
• CPC Class Codes: H04N 13/243 using three or more 2D image sensors
• CPC Class Codes: H04N 13/383 for tracking with gaze detection, i.e. detecting the lines of sight of the viewer's eyes
• CPC Class Codes: H04N 23/63 by using electronic viewfinders
• CPC Class Codes: H04N 23/959 by adjusting depth of field during image capture, e.g. maximising or setting range based on scene characteristics

Abstract

The invention is directed to systems, methods and computer program products for providing focus control for an image-capturing device. An exemplary method includes capturing an image frame using an image-capturing device and recording video tracking data and gaze tracking data for one or more image frames following the captured image frame and/or for one or more images frames prior to the captured image frame. The exemplary method additionally includes calculating a focus distance and a depth of field based at least partially on the recorded video tracking data and recorded gaze tracking data. The exemplary method additionally includes displaying the captured image frame based on the calculated focus distance and the calculated depth of field.

Claims

1. A method for providing focus control, the method comprising:

capturing an image frame using an image-capturing device;

recording video tracking data for at least one image frame following the captured image frame;

recording gaze tracking data for the at least one image frame following the captured image frame;

calculating a focus distance and a depth of field based at least partially on the recorded video tracking data and the recorded gaze tracking data; and

displaying the captured image frame based at least partially on the calculated focus distance and the calculated depth of field.

2. The method of claim 1 further comprising:

recording video tracking data for at least one image frame prior to the captured image frame;

recording gaze tracking data for the at least one image frame prior to the captured image frame;

calculating a focus distance and a depth of field based at least partially on at least one of the recorded video tracking data for the at least one image frame prior to the captured image frame and the at least one image frame following the captured image frame, and the recorded gaze tracking data for the at least one image frame prior to the captured image frame and the at least one image frame following the captured image frame; and

displaying the captured image frame based at least partially on the calculated focus distance and the calculated depth of field.

3. The method of claim 1, wherein the recording video tracking data step further comprises determining at least one moving object based at least partially on the recorded video tracking data.

4. The method of claim 3, wherein the recording video tracking data step further comprises determining a distance from the image-capturing device to the at least one moving object in the captured image frame.

5. The method of claim 4, wherein the distance to the at least one moving object in the captured image frame is determined at a time after the capturing step.

6. The method of claim 3, wherein the recording gaze tracking data step further comprises determining movement of at least one focus area based at least partially on the recorded gaze data.

7. The method of claim 6, further comprising:

correlating the at least one moving object with the movement of the at least one focus area; and

determining, based at least partially on the correlating step, at least one object the image-capturing device is focusing on.

8. The method of claim 7, further comprising:

determining whether the image-capturing device focuses on a single object during a time frame that comprises the captured image frame and the at least one image frame following the captured image frame; and

in response to determining the image-capturing device focuses on a single object during the time frame, calculating a focus distance and a depth of field associated with the single object.

9. The method of claim 7, further comprising:

determining whether the image-capturing device shifts focus from a first object to a second object during a time frame that comprises the captured image frame and the at least one image frame following the captured image frame; and

in response to determining the image-capturing device shifts focus from the first object to the second object, initiating transition from a first focus setting and a first depth of field setting associated with the first object to a second focus setting and a second depth of field setting associated with the second object.

10. The method of claim 1, wherein the image-capturing device is a lightfield image-capturing device.

11. The method of claim 1, wherein the captured image frame and the at least one image frame following the captured image frame comprise a video.

12. The method of claim 1, wherein the image-capturing device is a stereoscopic image capturing device.

13. The method of claim 1, further comprising:

allowing a user to manually adjust the calculated focus distance and the calculated depth of field either using the image-capturing device or using a separate computing device.

14. The method of claim 1, further comprising recording video tracking data and gaze tracking data for the image frame while simultaneously capturing the image frame.

15. The method of claim 1, further comprising:

enabling the image-capturing device to predict an object'"'"'s movement following the captured image frame based at least partially on the video tracking data recorded after the captured image frame, wherein the object is positioned in the captured image frame.

16. The method of claim 1, further comprising:

determining whether a movement of the image-capturing device is an initiation of a change in focus of the image-capturing device or is a temporary movement of the image-capturing device based at least partially on the gaze tracking data recorded after the captured image frame; and

in response to determining the movement of the image-capturing device is an initiation of a change in focus of the image-capturing device, initiating a change in focus of the image-capturing device.

17. The method of claim 16, further comprising:

in response to determining the movement of the image-capturing device is a temporary movement, retaining a current focus of the image-capturing device.

18. The method of claim 1, wherein the image-capturing device determines a number of the at least one image frame that follows the captured image frame based at least partially on at least one of previously recorded video tracking data or previously recorded gaze tracking data.

19. A computer program product for providing focus control for an image-capturing device, the computer program product comprising:

a non-transitory computer readable medium comprising code configured to cause a computer to;

record video tracking data for at least one image frame following an image frame captured by an image-capturing device;

record gaze tracking data for the at least one image frame following the captured image frame;

calculate a focus distance and a depth of field based at least partially on at least one of the recorded video tracking data and the recorded gaze tracking data; and

display the captured image frame based at least partially on the calculated focus distance and the calculated depth of field.

20. An system for providing focus control, the system comprising:

an image-capturing component to capture an image frame;

a processor to initiate;

recording of video tracking data for at least one image frame following the captured image frame;

recording of gaze tracking data for the at least one image frame following the captured image frame;

calculation of a focus distance and a depth of field based at least partially on at least one of the recorded video tracking data and the recorded gaze tracking data; and

display of the captured image frame based at least partially on the calculated focus distance and the calculated depth of field.