Photographic subject tracking device and camera

• US 9,563,967 B2
• Filed: 07/17/2013
• Issued: 02/07/2017
• Est. Priority Date: 07/17/2012
• Status: Active Grant

• Current Assignee: Nikon Corporation
• Original Assignee: Nikon Corporation
• Inventors: Kintou, Hiroshi
• Primary Examiner(s): Harandi, Siamak
• Application Number:
  US14/413,282
• Publication Number:
  US 20150356743A1
• Time in Patent Office: 1,301 Days
• US Class Current: 1/1
• CPC Class Codes: G06F 18/22 Matching criteria, e.g. proximity measures
• CPC Class Codes: G06T 2207/10004 Still image; Photographic image
• CPC Class Codes: G06T 2207/20012 Locally adaptive
• CPC Class Codes: G06T 2207/20081 Training; Learning
• CPC Class Codes: G06T 2207/30196 Human being; Person
• CPC Class Codes: G06T 3/40 Scaling of whole images or parts thereof, e.g. expanding or contracting
• CPC Class Codes: G06T 7/248 involving reference images or patches
• CPC Class Codes: G06V 10/24 Aligning, centring, orientation detection or correction of the image
• CPC Class Codes: G06V 10/751 Comparing pixel values or logical combinations thereof, or feature values having positional relevance, e.g. template matching

Abstract

A photographic subject tracking device includes: a first degree-of-similarity calculation unit that calculates degree of similarity between a template image for tracking and an image in search area; a photographic subject position identification unit that identifies a tracked photographic subject position in the input image based on calculated degree of similarity; a second degree-of-similarity calculation unit that calculates a degree of similarity between each of multiple template images for resizing determination, which are generated based on template image for tracking, and image in search area; a matching position identification unit that identifies matching positions of the multiple template images for resizing determination, respectively, in the input image based on calculated degrees of similarity; and a size changing unit that changes an image size of template image for tracking and template images for resizing determination based on a density of the plurality of matching positions identified.

Claims

1. A photographic subject tracking device, comprising:

a memory that stores instructions; and

a processor that executes the instructions stored in the memory to;

set a search area in an input image;

calculate a first degree of similarity between a template image for tracking and an image in the search area;

identify a tracked photographic subject position in the input image based on the first degree of similarity;

calculate a second degree of similarity between each of multiple template images for resizing determination, which are generated based on the template image for tracking, and the image in the search area;

identify matching positions of the multiple template images for resizing determination, respectively, in the input image based on second degrees of similarity; and

change an image size of the template image for tracking and the template images for resizing determination based on a density of the plurality of matching positions.

2. The photographic subject tracking device according to claim 1, wherein:

when the density of the plurality of matching positions has changed by a predetermined amount or more, the processor changes the image size of the template image for tracking and the template images for resizing determination.

3. The photographic subject tracking device according to claim 1, wherein:

when the density of the plurality of matching positions has changed to a higher value, the processor reduces the image size; and

when the density of the plurality of matching positions has changed to a lower value, the processor increases the image size.

4. The photographic subject tracking device according to claim 1, wherein:

the processor selects matching positions, whose second degrees of similarity are equal to or higher than a predetermined value, from among the plurality of matching positions, and changes the image size based on the density of the selected matching positions.

5. The photographic subject tracking device according to claim 1, wherein:

each of the multiple template images for resizing determination is an image containing a different part of the template image for tracking.

6. The photographic subject tracking device according to claim 5, wherein:

each of the multiple template images for resizing determination is an image of a predetermined size, each image of the predetermined size being centered at a center position of one of a plurality of blocks into which the template image for tracking is divided.

7. The photographic subject tracking device according to claim 6, wherein:

the processor changes the image size of the template image for tracking and the template images for resizing determination based on the density of matching positions of template images for resizing determination other than one template image for resizing determination, with reference to a matching position of the one template image for resizing determination which is centered at the center position of the center block among the plurality of blocks.

8. The photographic subject tracking device according to claim 7, wherein:

when a maximum degree of similarity between the one template image for resizing determination, which is centered at the center position of the center block among the plurality of blocks, and the image in the search area is equal to or more than a predetermined value, the processor determines whether or not to change the image size of the template image for tracking and the template images for resizing determination.

11. A camera comprising the photographic subject tracking device according to claim 1.

9. A photographic subject tracking device comprising:

a memory that stores instructions; and

a processor that executes the instructions stored in the memory to;

a search area in an input image;

calculate a degree of similarity between an initial template image and an image in the search area and a degree of similarity between a synthetic template image and the image in the search area;

identify a tracked photographic subject position in the input image based on the degrees of similarity;

judge whether or not to update the synthetic template image, based on the degrees of similarity;

an image, which is obtained by synthesizing an image including the tracked photographic subject position in the input image with the initial template image, as a new synthetic template image when the processor determines to update the synthetic template image;

calculate another degree of similarity between each of multiple template images for resizing determination and the image in the search area;

identify matching positions of the multiple template images for resizing determination, respectively, in the input image based on the other degrees of similarity;

judge whether or not to change an image size of the initial template image, the synthetic template image, and the template images for resizing determination, based on a density of the plurality of matching positions; and

change the image size of the initial template image to a new image size when processor determines to change the image size;

wherein when the processor determines to change the image size, the processor sets an image of the new image size, including the tracked photographic subject position in the input image, as a new updated synthetic template image, further divides the updated synthetic template image into a plurality of blocks, and sets images of the new image size, each including one of the blocks in the input image, as new updated template images for resizing determination.

10. A photographic subject tracking device comprising:

a memory that stores instructions; and

a processor that executes the instructions stored in the memory to;

track a specified photographic subject in an image based on a first template image;

identify a position in the image based on a plurality of second template images, the plurality of second template images being generated based on the first template image,

detect a change of a size of the specified photographic subject; and

update the first template image based on a plurality of positions when the processor detects that the size of the specified photographic subject has changed by a predetermined amount or more.