Determination of exposure time for an image frame

• US 9,635,276 B2
• Filed: 06/10/2015
• Issued: 04/25/2017
• Est. Priority Date: 06/10/2015
• Status: Active Grant

• Current Assignee: Microsoft Technology Licensing LLC (Microsoft Corporation)
• Original Assignee: Microsoft Technology Licensing LLC (Microsoft Corporation)
• Inventors: Sen, Sumeet
• Primary Examiner(s): DIEP, TRUNG T
• Application Number:
  US14/735,682
• Publication Number:
  US 20160366326A1
• Time in Patent Office: 685 Days
• Field of Search: 3482086, 34820813- 14, 34833311, 348362
• CPC Class Codes: G06T 7/20 Analysis of motion motion estimation for coding, decoding, compressing or decompressing digital video signals H04N19/43, H04N19/51
• CPC Class Codes: G06T 7/215 Motion-based segmentation
• CPC Class Codes: H04N 23/6811 based on the image signal
• CPC Class Codes: H04N 23/73 by influencing the exposure time

Abstract

An apparatus for adjusting an exposure time for an image frame is presented. The apparatus comprises at least one processing unit and at least one memory. The at least one memory stores program instructions that, when executed by the at least one processing unit, cause the apparatus to process at least one image frame, select at least one region of interest from the at least one image frame, process at least two consecutive image frames to determine a motion field, segment the motion field into at least one motion cluster, select based on the at least one region of interest and the at least one motion cluster, the most relevant motion cluster, and adjust the exposure time based on motion information of the selected most relevant motion cluster.

Claims

1. An apparatus, comprising:

at least one processing unit;

at least one memory, wherein the at least one memory stores program instructions that, when executed by the at least one processing unit, cause the apparatus to;

process at least two consecutive image frames;

select at least one region of interest from the at least two image frames based on the at least one region of interest having a salient object therein;

process the at least one region of interest in the at least two image frames to determine a motion field;

segment the motion field into a plurality of motion clusters;

select, based on the at least one region of interest and the plurality of motion clusters, a most relevant motion cluster from the plurality of motion clusters, the most relevant motion cluster providing one of the following;

a greatest amount of movement among the plurality of motion clusters, or a least amount of movement among the plurality of motion clusters; and

adjust an exposure time based on motion information of the selected most relevant motion cluster.

2. An apparatus according to claim 1, wherein the at least one memory stores program instructions that, when executed by the at least one processing unit, cause the apparatus to:

process the at least two consecutive image frames to determine the motion field only within the at least one region of interest.

3. An apparatus according to claim 1, wherein the at least one memory stores program instructions that, when executed by the at least one processing unit, cause the apparatus to:

determine at least one salient region from the at least one image frame, the salient region comprising at least a portion of the salient object; and

select the at least one salient region as the at least one region of interest.

4. An apparatus according to claim 1, wherein the at least one memory stores program instructions that, when executed by the at least one processing unit, when selecting the region of interest, cause the apparatus to:

determine at least two salient regions of interest from the at least one image frame; and

select a salient region closest to a center of the at least one image frame and the salient object as the region of interest.

5. An apparatus according to claim 1, wherein the at least one memory stores program instructions that, when executed by the at least one processing unit, when selecting the region of interest, cause the apparatus to:

process the at least one region of interest to determine a saliency map;

weight the motion field with values of the saliency map; and

select a motion cluster having one of a highest weight or a lowest weight as the most relevant motion cluster.

6. An apparatus according to claim 1, wherein the at least one memory stores program instructions that, when executed by the at least one processing unit, when selecting the region of interest, cause the apparatus to:

determine a weight for each region of interest.

7. An apparatus according to claim 6, wherein the weight of a region of interest is based on at least one of the following:

a size of the region of interest; and

a location of the region of interest.

8. An apparatus according to claim 1, wherein the at least one memory stores program instructions that, when executed by the at least one processing unit, cause the apparatus to:

determine a weight for each motion cluster.

9. An apparatus according to claim 8, wherein the weight of each motion cluster is based on at least one of the following:

a magnitude of motion in the motion cluster;

a size of the motion cluster;

a location of the motion cluster;

a frequency of occurrence in image frames of the motion cluster;

a direction of motion in the motion cluster; and

a motion propagation across at least two consecutive image frames.

10. An apparatus according to claim 1, wherein the at least one memory stores program instructions that, when executed by the at least one processing unit, cause the apparatus to:

determine a weight for each region of interest;

determine a weight for each motion cluster;

modify a weight of a motion cluster depending on a weight of a region of interest overlapping the motion cluster; and

select a motion cluster having a highest modified weight as the most relevant motion cluster.

11. An apparatus according to claim 1, wherein the at least one memory stores program instructions that, when executed by the at least one processing unit, cause the apparatus to:

identify at least one face in the at least two image frames, the at least one face being the salient object; and

select at least one region comprising the at least one face as the at least one region of interest.

12. An apparatus according to claim 1,

wherein selecting the region of interest is further based on an indication from a user to select the region of interest.

13. An apparatus according to claim 1, wherein the at least one memory stores program instructions that, when executed by the at least one processing unit, cause the apparatus to:

adjust the at least one region of interest based on a size of the salient object.

14. An apparatus according to claim 1, wherein the at least one memory stores program instructions that, when executed by the at least one processing unit, cause the apparatus to:

determine that one or more motion clusters overlap with the at least one region of interest; and

select the most relevant motion cluster from the one or more motion clusters overlapping with the at least one region of interest.

15. An apparatus, comprising:

at least one processing unit;

at least one memory, wherein the at least one memory stores program instructions that, when executed by the at least one processing unit, cause the apparatus to;

process at least two consecutive image frames to determine a motion field;

segment the motion field into different motion clusters based on motion values in the motion field;

rank the motion clusters based on relevance of the motion;

select a most relevant motion cluster based on a direction the most relevant motion cluster is moving; and

adjust the exposure time based on motion information of the most relevant motion cluster.

16. A method comprising:

processing at least one image frame;

selecting at least one region of interest from a first image frame;

processing the first image frame and a second image frame to determine a motion field;

segmenting the motion field into at least one motion cluster;

selecting, based on the at least one region of interest and the at least one motion cluster, a most relevant motion cluster; and

adjusting the exposure time based on motion information of the most relevant motion cluster.

17. A method according to claim 16, comprising:

determining a weight for each region of interest;

determining a weight for each motion cluster;

modifying a weight of a motion cluster depending on a weight of a region of interest overlapping the motion cluster; and

selecting a motion cluster having a highest modified weight as the most relevant motion cluster.

18. A method according to claim 16, comprising:

processing the first and second frames to determine the motion field only within the at least one region of interest.

19. A method according to claim 16, comprising:

processing the at least one region of interest to determine a saliency map;

weighting the motion field with values of the saliency map; and

selecting a motion cluster having the highest weight as the most relevant motion cluster.

20. A method according to claim 16, comprising:

identifying at least one face in the at least one image frame; and

selecting at least one region comprising the at least one face as the at least one region of interest.