SUBJECT TRACKING APPARATUS, IMAGING APPARATUS AND SUBJECT TRACKING METHOD

• US 20100232646A1
• Filed: 02/22/2010
• Published: 09/16/2010
• Est. Priority Date: 02/26/2009
• Status: Active Grant

• Current Assignee: Nikon Corporation
• Original Assignee: Nikon Corporation
• Inventors: TAKEUCHI, Hiroshi
• Granted Patent: US 8,605,942 B2
• Time in Patent Office: Days
• US Class Current: 382/103
• CPC Class Codes: G06T 7/248 involving reference images or patches
• CPC Class Codes: H04N 23/61 based on recognised objects
• CPC Class Codes: H04N 23/635 Region indicators; Field of view indicators
• CPC Class Codes: H04N 23/672 based on the phase difference signals
• CPC Class Codes: H04N 23/673 based on contrast or high frequency components of image signals, e.g. hill climbing method
• CPC Class Codes: H04N 5/145 Movement estimation for video coding H04N19/51

Abstract

A subject tracking apparatus includes a region extraction section extracting a region similar to a reference image in a first image based on respective feature amounts of the first image being picked up and the reference image being set, a motion vector calculating section calculating a motion vector in each of a plurality of regions in the first image using a second image and the first image, the second image being picked up at a different time from that of the first image, and a control section determining an object region of subject tracking in the first image based on an extraction result in the region extraction section and a calculation result in the motion vector calculating section.

Claims

1. A subject tracking apparatus comprising:

a region extraction section extracting a region similar to a reference image in a first image based on respective feature amounts of the first image being picked up and the reference image being set;

a motion vector calculating section calculating a motion vector in each of a plurality of regions in the first image using a second image and the first image, the second image being picked up at a different time from that of the first image; and

a control section determining an object region of subject tracking in the first image based on an extraction result in the region extraction section and a calculation result in the motion vector calculating section.

2. The subject tracking apparatus according to claim 1, wherein

the control section determines a region to be excluded from the object region of subject tracking among the plurality of regions similar to the reference image extracted by the region extraction section based on the motion vector in each of the regions of the first image calculated by the motion vector calculating section.

3. The subject tracking apparatus according to claim 2, wherein

the control section determines a region, as the region to be excluded from the object region of subject tracking among the regions similar to the reference image extracted by the region extraction section, in which most of regions surrounding the region have motion vector directions nearly the same with a motion vector direction of the region.

4. The subject tracking apparatus according to claim 3, wherein

the control section determines a region existing at the same position as with the object region of subject tracking in an image being picked up before the first image to be the object region of subject tracking when having judged that the region to be excluded from the object region of subject tracking does not exist.

5. The subject tracking apparatus according to claim 1, wherein

the control section determines a region, to be the object region of subject tracking among the regions similar to the reference image extracted by the region extraction section, in which most of regions surrounding the region have motion vector directions different from a motion vector direction of the region.

6. The subject tracking apparatus according to claim 1, further comprising

an extraction section extracting a region in a picked up image as the reference image.

7. The subject tracking apparatus according to claim 1, wherein

the second image is an image being picked up before the first image.

8. The subject tracking apparatus according to claim 1, further comprising

a feature amount calculating section calculating feature amounts of an image, wherein

the feature amount calculating section divides each of the reference image and the first image into regions each having a predetermined size and calculates the feature amounts of respective regions having the predetermined size, and

the region extraction section takes a correlation between the reference image and the first image based on each of the feature amounts which is calculated by the feature amount calculating section and obtained for each of the respective regions having the predetermined size in the reference image and the first image, and extracts a region having a relatively large value of the correlation in the first image as the region similar to the reference image.

9. The subject tracking apparatus according to claim 1, wherein

the motion vector calculation section divides each of the first image and the second image into the regions and calculates the motion vector in each of the regions of the first image by taking a correlation between corresponding divided regions in the first image and the second image.

10. The subject tracking apparatus according to claim 1, wherein

the control section calculates an average motion vector using a plurality of motion vectors being nearly the same as one another among the motion vectors in the respective regions of the first image and shifts each of the regions of the second image by the average motion vector, and determines the object region of subject tracking by obtaining a difference between the first image and the second image for each of the regions.

11. The subject tracking apparatus according to claim 1, wherein

each of the first image and the second image is one of a plurality of images being picked up continuously.

12. An imaging apparatus comprising:

an imaging section picking up a subject image and obtaining an image; and

the subject tracking apparatus according to claim 1.

13. A subject tracking method comprising:

a first operation of extracting a region similar to a reference image in a first image based on respective feature amounts of the first image being picked up and the reference image being set;

a second operation of calculating a motion vector in each of a plurality of regions in the first image using a second image and the first image, the second image being picked up at a different time from that of the first image; and

a third operation of determining an object region of subject tracking in the first image based on an extraction result in the first operation and a calculation result in the second operation.

14. The subject tracking method according to claim 13, wherein

the third operation determines a region to be excluded from the object region of subject tracking among the plurality of regions similar to the reference image extracted by the first operation based on the motion vector in each of the regions of the first image calculated by the second operation.

15. The subject tracking method according to claim 14, wherein

the third operation determines a region, as the region to be excluded from the object region of subject tracking among the regions similar to the reference image extracted by the first operation, in which most of regions surrounding the region have motion vector directions nearly the same with a motion vector direction of the region.

16. The subject tracking method according to claim 15, wherein

the third operation determines a region existing at the same position as with the object region of subject tracking in an image being picked up before the first image to be the object region of subject tracking when having judged that the region to be excluded from the object region of subject tracking does not exist.

17. The subject tracking method according to claim 13, wherein

the third operation determines a region, to be the object region of subject tracking among the regions similar to the reference image extracted by the first operation, in which most of regions surrounding the region have motion vector directions different from a motion vector direction of the region.

18. The subject tracking method according to claim 13, further comprising

an operation of extracting a region in a picked up image as the reference image.

19. The subject tracking method according to claim 13, wherein

the second image is an image being picked up before the first image.

20. The subject tracking method according to claim 13, further comprising

a feature amount calculating operation calculating feature amounts of an image, wherein

the feature amount calculating operation divides each of the reference image and the first image into regions each having a predetermined size and calculates the feature amounts of respective regions having the predetermined size, and

the first operation takes a correlation between the reference image and the first image based on each of the feature amounts which is calculated by the feature amount calculating operation and obtained for each of the respective regions having the predetermined size in the reference image and the first image, and extracts a region having a relatively large value of the correlation in the first image as the region similar to the reference image.

21. The subject tracking method according to claim 13, wherein

the second operation divides each of the first image and the second image into the regions and calculates the motion vector in each of the regions of the first image by taking a correlation between corresponding divided regions in the first image and the second image.

22. The subject tracking method according to claim 13, wherein

the third operation calculates an average motion vector using a plurality of motion vectors nearly the same as one another among the motion vectors in the respective plural regions of the first image and shifts each of the plural regions of the second image by the average motion vector, and then determines the object region of subject tracking by obtaining a difference between the first image and the second image for each of the plural regions.

23. The subject tracking method according to claim 13, wherein

each of the first image and the second image is one of a plurality of images being picked up continuously.