Automatic target selection for multi-target object tracking

• US 9,684,830 B2
• Filed: 11/14/2014
• Issued: 06/20/2017
• Est. Priority Date: 11/14/2014
• Status: Active Grant

• Current Assignee: Intel Corporation
• Original Assignee: Intel Corporation
• Inventors: Park, Minje, Kim, Tae-Hoon
• Primary Examiner(s): TSAI, TSUNG YIN
• Application Number:
  US14/541,631
• Publication Number:
  US 20160140391A1
• Time in Patent Office: 949 Days
• Field of Search: None
• CPC Class Codes: G06F 18/22 Matching criteria, e.g. proximity measures
• CPC Class Codes: G06T 7/20 Analysis of motion motion estimation for coding, decoding, compressing or decompressing digital video signals H04N19/43, H04N19/51
• CPC Class Codes: G06V 10/255 Detecting or recognising potential candidate objects based on visual cues, e.g. shapes
• CPC Class Codes: G06V 10/62 relating to a temporal dimension, e.g. time-based feature extraction; Pattern tracking
• CPC Class Codes: H04N 23/60 Control of cameras or camera modules
• CPC Class Codes: H04N 7/18 Closed-circuit television [CCTV] systems, i.e. systems in which the video signal is not broadcast

Abstract

Techniques related to automatic target object selection from multiple tracked objects for imaging devices are discussed. Such techniques may include generating one or more object selection metrics such as accumulated distances from frame center, accumulated velocities, and trajectory comparisons of predicted to actual trajectories for tracked objects and selecting the target object based on the object selection metric or metrics.

Claims

1. A method for providing automatic object selection from multiple tracked objects for image capture by an imaging device comprising:

receiving multiple image frames for object detection, object tracking, and image capture preparation;

detecting at least a first object and a second object in at least a first image frame of the multiple image frames;

tracking at least the first detected object and the second detected object across two or more second image frames of the multiple image frames;

generating a first object trajectory corresponding to the first tracked object and a second object trajectory corresponding to the second tracked object;

generating a first object selection metric associated with the first object trajectory and a second object selection metric associated with the second object trajectory, wherein the first object selection metric comprises a first weighted sum comprising a product of a first weight and an accumulated distance from frame center for the first object across at least the two or more second image frames, a product of a second weight and an accumulated velocity for the first object across at least the two or more second image frames, and a product of a third weight and an accumulated position difference between a predicted trajectory position and an actual position for the first object across at least the two or more second image frames and the second object selection metric comprises a second weighted sum comprising a product of the first weight and an accumulated distance from frame center for the second object across at least the two or more second image frames, a product of the second weight and an accumulated velocity for the second object across at least the two or more second image frames, and a product of the third weight and an accumulated position difference between a predicted trajectory position and an actual trajectory position for the second object across at least the two or more second image frames, wherein the first weight is greater than the second weight when an average velocity of the first and second detected objects is less than a threshold and the second weight is greater than the first weight when the average velocity is greater than the threshold;

selecting the first detected object as a target object responsive to the first object selection metric being less than the second object selection metric;

capturing, by a camera, an image using exposure settings determined based on the selected target object; and

storing the captured image frame to memory.

2. The method of claim 1, wherein the predicted trajectory for the first object comprises a closed-loop control predicted trajectory.

3. The method of claim 1, further comprising:

performing at least one of an auto focus adjustment, an auto exposure adjustment, or an auto white balance adjustment based at least in part on the target object to determine the exposure settings.

4. The method of claim 1, wherein detecting the first and second objects comprises at least one of a saliency based object detection, a face based object detection, or an image descriptor based object detection.

5. The method of claim 1, wherein tracking the first detected object comprises generating a filtered trajectory of the first object.

6. The method of claim 1, further comprising:

performing object recognition based on the target object.

7. The method of claim 1, further comprising:

tracking a third object and a fourth object across two or more third image frames of the multiple image frames;

generating a third object trajectory corresponding to a third tracked object and a fourth object trajectory corresponding to the second tracked object; and

generating a third object selection metric associated with the third object trajectory and a fourth object selection metric associated with the fourth object trajectory, wherein tracking and generating trajectories for the first and second objects is performed at a first frame sampling rate and tracking and generating trajectories for the third and fourth objects is performed at a second frame sampling rate less than the first frame sampling rate.

8. The method of claim 7, wherein the second frame sampling rate is less than the first frame sampling rate responsive to a power management policy or a remaining battery life.

9. A system for providing automatic object selection from multiple tracked objects for image capture by an imaging device comprising:

storage configured to store image data; and

a processor coupled to the storage, the processor to

receive multiple image frames for object detection, object tracking, and image capture preparation;

detect at least a first object and a second object in at least a first image frame of the multiple image frames;

track at least the first detected object and the second detected object across two or more second image frames of the multiple image frames;

generate a first object selection metric associated with the first object trajectory and a second object selection metric associated with the second object trajectory, wherein the first object selection metric comprises a first weighted sum comprising a product of a first weight and an accumulated distance from frame center for the first object across the at least two or more second image frames, a product of a second weight and an accumulated velocity for the first object across at least the two or more second image frames, and a product of a third weight and an accumulated position difference between a predicted trajectory position and an actual position for the first object across at least the two or more second image frames and the second object selection metric comprises a second weighted sum of a product of the first weight and an accumulated distance from frame center for the second object across at least the two or more second image frames, a product of the second weight and an accumulated velocity for the second object across at least the two or more second image frames, and a product of the third weight and an accumulated position difference between a predicted trajectory position and an actual trajectory position for the second object across at least the two or more second image frames, wherein the first weight is greater than the second weight when an average velocity of the first and second detected objects is less than a threshold and the second weight is greater than the first weight when the average velocity is greater than the threshold;

select the first detected object as a target object responsive to the first object selection metric being less than the second object selection metric; and

determine exposure settings based on the selected target object; and

a camera to capture an image using the exposure settings, wherein the storage is to store the captured image.

10. The system of claim 9, wherein the predicted trajectory for the first object comprises a closed-loop control predicted trajectory.

11. The system of claim 9, wherein to determine the exposure settings the processor is to provide at least one of an auto focus adjustment, an auto exposure adjustment, or an auto white balance adjustment based at least in part on the target object.

12. The system of claim 9, wherein the to track the first detected object, the processor is to generate a filtered trajectory of the first object.

13. The system of claim 9, wherein the processor is further to

track a third object and a fourth object across two or more third image frames of the multiple image frames;

generate a third object trajectory corresponding to a third tracked object and a fourth object trajectory corresponding to the second tracked object; and

generate a third object selection metric associated with the third object trajectory and a fourth object selection metric associated with the fourth object trajectory, wherein tracking and generating trajectories for the first and second objects is performed at a first frame sampling rate and tracking and generating trajectories for the third and fourth objects is performed at a second frame sampling rate less than the first frame sampling rate.

14. The system of claim 13, wherein the second frame sampling rate is less than the first frame sampling rate responsive to a power management policy or a remaining battery life of the system.

15. At least one non-transitory machine readable medium comprising a plurality of instructions that in response to being executed on an imaging device, cause the imaging device to provide automatic object selection from multiple tracked objects for image capture by:

receiving multiple image frames for object detection, object tracking, and image capture preparation;

detecting at least a first object and a second object in at least a first image frame of the multiple image frames;

tracking at least the first detected object and the second detected object across two or more second image frames of the multiple image frames;

generating a first object selection metric associated with the first object trajectory and a second object selection metric associated with the second object trajectory, wherein the first object selection metric comprises a first weighted sum comprising a product of a first weight and an accumulated distance from frame center for the first object across at least the two or more second image frames, a product of a second weight and an accumulated velocity for the first object across at least the two or more second image frames, and a product of a third weight and an accumulated position difference between a predicted trajectory position and an actual position for the first object across at least the two or more second image frames and the second object selection metric comprises a second weighted sum comprising a product of the first weight and an accumulated distance from frame center for the second object across at least the two or more second image frames, a product of the second weight and an accumulated velocity for the second object across at least the two or more second image frames, and a product of the third weight and an accumulated position difference between a predicted trajectory position and an actual trajectory position for the second object across at least the two or more second image frames, wherein the first weight is greater than the second weight when an average velocity of the first and second detected objects is less than a threshold and the second weight is greater than the first weight when the average velocity is greater than the threshold;

selecting the first detected object as a target object responsive to the first object selection metric being less than the second object selection metric;

capturing, by a camera, an image using exposure settings determined based on the selected target object; and

storing the captured image frame to memory.

16. The machine readable medium of claim 15, wherein the predicted trajectory for the first object comprises a closed-loop control predicted trajectory.

17. The machine readable medium of claim 15 further comprising instructions that cause the computing device to provide automatic object selection from multiple tracked objects for image capture by:

performing at least one of an auto focus adjustment, an auto exposure adjustment, or an auto white balance adjustment based at least in part on the target object to determine the exposure settings.

18. The machine readable medium of claim 15, wherein tracking the first detected object comprises generating a filtered trajectory of the first object.

19. The machine readable medium of claim 15 further comprising instructions that cause the computing device to provide automatic object selection from multiple tracked objects for image capture by:

tracking a third object and a fourth object across two or more third image frames of the multiple image frames;

generating a third object trajectory corresponding to a third tracked object and a fourth object trajectory corresponding to the second tracked object; and

generating a third object selection metric associated with the third object trajectory and a fourth object selection metric associated with the fourth object trajectory, wherein tracking and generating trajectories for the first and second objects is performed at a first frame sampling rate and tracking and generating trajectories for the third and fourth objects is performed at a second frame sampling rate less than the first frame sampling rate.

20. The machine readable medium of claim 19, wherein the second frame sampling rate is less than the first frame sampling rate responsive to a power management policy or a remaining battery life.