Subject tracking apparatus, imaging apparatus and subject tracking method

• US 8,605,942 B2
• Filed: 02/22/2010
• Issued: 12/10/2013
• Est. Priority Date: 02/26/2009
• Status: Active Grant

• Current Assignee: Nikon Corporation
• Original Assignee: Nikon Corporation
• Inventors: Takeuchi, Hiroshi
• Primary Examiner(s): Seth, Manav
• Application Number:
  US12/710,028
• Publication Number:
  US 20100232646A1
• Time in Patent Office: 1,387 Days
• Field of Search: None
• US Class Current: 382/103
• CPC Class Codes: G06T 7/248 involving reference images or patches
• CPC Class Codes: H04N 23/61 based on recognised objects
• CPC Class Codes: H04N 23/635 Region indicators; Field of view indicators
• CPC Class Codes: H04N 23/672 based on the phase difference signals
• CPC Class Codes: H04N 23/673 based on contrast or high frequency components of image signals, e.g. hill climbing method
• CPC Class Codes: H04N 5/145 Movement estimation for video coding H04N19/51

Abstract

A subject tracking apparatus includes a region extraction section extracting a region similar to a reference image in a first image based on respective feature amounts of the first image being picked up and the reference image being set, a motion vector calculating section calculating a motion vector in each of a plurality of regions in the first image using a second image and the first image, the second image being picked up at a different time from that of the first image, and a control section determining an object region of subject tracking in the first image based on an extraction result in the region extraction section and a calculation result in the motion vector calculating section.

Claims

1. A subject tracking apparatus comprising:

a region extraction section extracting a region similar to a reference image in a first image based on respective feature amounts of the first image being picked up and the reference image being set;

a motion vector calculating section calculating a motion vector in each of a plurality of regions in the first image using a second image and the first image, the second image being picked up at a different time from that of the first image; and

a control section determining an object region of subject tracking in the first image based on an extraction result in the region extraction section and a calculation result in the motion vector calculating section.

2. The subject tracking apparatus according to claim 1, wherein

the control section determines a region to be excluded from the object region of subject tracking among the plurality of regions similar to the reference image extracted by the region extraction section based on the motion vector in each of the regions of the first image calculated by the motion vector calculating section.

3. The subject tracking apparatus according to claim 2, wherein

the control section determines a region as the region to be excluded from the object region of subject tracking among the regions similar to the reference image extracted by the region extraction section, in which most of regions surrounding the region have motion vector directions nearly the same with a motion vector direction of the region.

4. The subject tracking apparatus according to claim 3, wherein

the control section determines a region existing at the same position as with the object region of subject tracking in an image being picked up before the first image to be the object region of subject tracking when having judged that the region to be excluded from the object region of subject tracking does not exist.

5. The subject tracking apparatus according to claim 1, wherein

the control section determines a region to be the object region of subject tracking among the regions similar to the reference image extracted by the region extraction section, in which most of regions surrounding the region have motion vector directions different from a motion vector direction of the region.

6. The subject tracking apparatus according to claim 1, further comprising

an extraction section extracting a region in a picked up image as the reference image.

7. The subject tracking apparatus according to claim 1, wherein

the second image is an image being picked up before the first image.

8. The subject tracking apparatus according to claim 1, further comprising

a feature amount calculating section calculating feature amounts of an image, wherein

the feature amount calculating section divides each of the reference image and the first image into regions each having a predetermined size and calculates the feature amounts of respective regions having the predetermined size, and

the region extraction section takes a correlation between the reference image and the first image based on each of the feature amounts which is calculated by the feature amount calculating section and obtained for each of the respective regions having the predetermined size in the reference image and the first image, and extracts a region having a relatively large value of the correlation in the first image as the region similar to the reference image.

9. The subject tracking apparatus according to claim 1, wherein

the motion vector calculation section divides each of the first image and the second image into the regions and calculates the motion vector in each of the regions of the first image by taking a correlation between corresponding divided regions in the first image and the second image.

10. The subject tracking apparatus according to claim 1, wherein

the control section calculates an average motion vector using a plurality of motion vectors being nearly the same as one another among the motion vectors in the respective regions of the first image and shifts each of the regions of the second image by the average motion vector, and determines the object region of subject tracking by obtaining a difference between the first image and the second image for each of the regions.

11. The subject tracking apparatus according to claim 1, wherein

each of the first image and the second image is one of a plurality of images being picked up continuously.

12. An imaging apparatus comprising:

an imaging section picking up a subject image and obtaining an image; and

the subject tracking apparatus according to claim 1.

13. A subject tracking method comprising:

extracting a region similar to a reference image in a first image based on respective feature amounts of the first image being picked up and the reference image being set;

calculating a motion vector in each of a plurality of regions in the first image using a second image and the first image, the second image being picked up at a different time from that of the first image; and

determining an object region of subject tracking in the first image based on an extraction result of the region similar to the reference image and a calculation result of the motion vector.

14. The subject tracking method according to claim 13, wherein

the process of determining the object region of subject tracking determines a region to be excluded from the object region of subject tracking among the plurality of regions being extracted and similar to the reference image based on the motion vector calculated in each of the regions of the first image.

15. The subject tracking method according to claim 14, wherein

the process of determining the object region of subject tracking determines a region, surrounded by more regions having nearly the same motion vector directions than regions not having nearly the same motion vector directions, as the region to be excluded from the object region of subject tracking among the regions being extracted and similar to the reference image.

16. The subject tracking method according to claim 15, wherein

the process of determining the object region of subject tracking determines a region existing at the same position as with the object region of subject tracking in an image being picked up before the first image to be the object region of subject tracking when having judged that the region to be excluded from the object region of subject tracking does not exist.

17. The subject tracking method according to claim 13, wherein

the process of determining the object region of subject tracking determines a region surrounded by more regions having nearly the same motion vector directions than regions not having nearly the same motion vector directions to be the object region of subject tracking among the regions being extracted and similar to the reference image.

18. The subject tracking method according to claim 13, further comprising

extracting a region in a picked up image as the reference image.

19. The subject tracking method according to claim 13, wherein

the second image is an image being picked up before the first image.

20. The subject tracking method according to claim 13, further comprising

calculating feature amounts of an image, wherein

the process of calculating the feature amount divides each of the reference image and the first image into regions each having a predetermined size and calculates the feature amounts of respective regions having the predetermined size, and

the process of extracting the region similar to the reference image takes a correlation between the reference image and the first image based on each of the feature amounts obtained for each of the respective regions having the predetermined size in the reference image and the first image, and extracts a region having a relatively large value of the correlation in the first image as the region similar to the reference image.

21. The subject tracking method according to claim 13, wherein

the process of calculating the motion vector divides each of the first image and the second image into the regions and calculates the motion vector in each of the regions of the first image by taking a correlation between corresponding divided regions in the first image and the second image.

22. The subject tracking method according to claim 13, wherein

the process of determining the object region of subject tracking calculates an average motion vector using a plurality of motion vectors nearly the same as one another among the motion vectors in the respective plural regions of the first image and shifts each of the plural regions of the second image by the average motion vector, and then determines the object region of subject tracking by obtaining a difference between the first image and the second image for each of the plural regions.

23. The subject tracking method according to claim 13, wherein

each of the first image and the second image is one of a plurality of images being picked up continuously.

24. A subject tracking apparatus comprising:

a region extraction section extracting a region similar to a reference image in a first image based on respective feature amounts of the first image being picked up and the reference image being set;

a motion calculating section calculating a motion of a predetermined region in the first image using a second image and the first image, the second image being picked up at a different time from that of the first image; and

a control section determining an object region of a subject tracking in the first image based on an extraction result in the region extraction section and a calculation result in the motion calculating section.