Region detection device, region detection method, image processing apparatus, image processing method, program, and recording medium

• US 9,734,612 B2
• Filed: 09/08/2015
• Issued: 08/15/2017
• Est. Priority Date: 09/24/2014
• Status: Active Grant

• Current Assignee: Fujifilm Corporation (Fujifilm Holdings Corporation)
• Original Assignee: Fujifilm Corporation (Fujifilm Holdings Corporation)
• Inventors: Yamaji, Kei
• Primary Examiner(s): Lee, Jonathan S
• Application Number:
  US14/847,364
• Publication Number:
  US 20160086342A1
• Time in Patent Office: 707 Days
• CPC Class Codes: G06T 11/60 Editing figures and text; Combining figures or text
• CPC Class Codes: G06T 2207/10016 Video; Image sequence
• CPC Class Codes: G06T 7/215 Motion-based segmentation
• CPC Class Codes: G06T 7/246 using feature-based methods, e.g. the tracking of corners or segments
• CPC Class Codes: G06V 40/161 Detection; Localisation; Normalisation

Abstract

The image processing apparatus includes a region detection unit that detects a face region of the attention person, an attention person movement region of the moving image, the entire region of the attention person, and an attention person transfer region of the moving image, a region image extraction unit that extracts an image of the face region of the attention person, an image of the attention person movement region of the moving image, an image of the entire region of the attention person, and an image of the attention person transfer region of the moving image, which respectively correspond to the face region of the attention person, the attention person movement region of the moving image, the entire region of the attention person, and the attention person transfer region of the moving image, from the still image, and a composite image generation unit that generates a composite image.

Claims

1. A region detection device comprising:

at least one processor configured to;

extract a plurality of frames of still image data from moving image data,

detect an attention person, who is a person to be processed, from a plurality of still images corresponding to the plurality of frames of still image data,

track a movement of the attention person in a moving image corresponding to the moving image data, on the basis of the detection result of the attention person in the plurality of still images, and detect a movement trajectory of the attention person,

analyze a motion of the attention person in the moving image, on the basis of the movement trajectory of the attention person, and calculate an evaluation value for the motion of the attention person in each of the plurality of still images, on the basis of the analyzed motion of the attention person,

output the still image data of a still image in which the evaluation value for the motion of the attention person is equal to or greater than a threshold value from the plurality of frames of still image data,

detect an attention person movement region of the moving image captured in the still image in which the evaluation value is equal to or greater than the threshold value, on the basis of the movement trajectory of the attention person,

detect a face region of the attention person in the still image in which the evaluation value is equal to or greater than the threshold value, and

detect an entire region of the attention person in the still image in which the evaluation value is equal to or greater than the threshold value, on the basis of the attention person movement region of the moving image and the face region of the attention person.

2. The region detection device according to claim 1, wherein the at least one processor is further configured to:

detect an attention person transfer region of the moving image in the still image in which the evaluation value is equal to or greater than the threshold value, on the basis of the movement trajectory of the attention person.

3. An image processing apparatus comprising:

the region detection device according to claim 2,

wherein the at least one processor is further configured to;

extract at least one of an image of the face region of the attention person, an image of the attention person movement region of the moving image, an image of the entire region of the attention person, and an image of the attention person transfer region of the moving image, which respectively correspond to the face region of the attention person, the attention person movement region of the moving image, the entire region of the attention person, and the attention person transfer region of the moving image, from the still image in which the evaluation value is equal to or greater than the threshold value, and

generate a composite image including the at least one image extracted by the region image extraction unit.

4. The image processing apparatus according to claim 3,

wherein, when the at least one processor detects the attention person transfer region of the moving image, the at least one processor extracts the image of the attention person transfer region of the moving image from the still image in which the evaluation value is equal to or greater than the threshold value, and

the at least one processor generates a composite image including the image of the attention person transfer region of the moving image.

5. The image processing apparatus according to claim 4,

wherein, when another image is automatically superimposed and combined with the image of the attention person transfer region of the moving image, the at least one processor superimposes the image on an image of a region other than the entire region of the attention person and combines the images.

6. The image processing apparatus according to claim 4,

wherein, when another image is automatically superimposed and combined with the image of the attention person transfer region of the moving image, the at least one processor changes an image of a region, with which the image is superimposed and combined, from an image of a region other than the face region of the attention person and the attention person movement region of the moving image, an image of a region other than the entire region of the attention person, and an image of a region other than the attention person transfer region of the moving image, depending on the purpose of the composite image.

7. The image processing apparatus according to claim 4,

wherein, when another image is automatically superimposed and combined with the image of the attention person transfer region of the moving image, the at least one processor changes an image of a region, with which the image is superimposed and combined, from an image of a region other than the face region of the attention person and the attention person movement region of the moving image, an image of a region other than the entire region of the attention person, and an image of a region other than the attention person transfer region of the moving image, depending on the type of image to be superimposed and combined.

8. The image processing apparatus according to claim 4,

wherein the at least one processor is further configured to

receive an instruction input by a user, and

superimpose an image designated by the instruction on the image of the attention person transfer region of the moving image at a position designated by the instruction and combines the images.

9. The image processing apparatus according to claim 4,

wherein the at least one processor is further configured to

detect a transfer direction and a transfer speed of the attention person, and

extend an extraction range in the transfer direction of the attention person as the transfer speed of the attention person increases and extracts the image of the attention person transfer region of the moving image.

10. The image processing apparatus according to claim 9,

wherein the at least one processor is configured to extract the image of the attention person transfer region of the moving image such that an extraction range in a direction opposite to the transfer direction of the attention person is wider than the extraction range in the transfer direction of the attention person.

11. The image processing apparatus according to claim 3,

wherein the at least one processor is configured to

extract at least two of the image of the face region of the attention person, the image of the attention person movement region of the moving image, the image of the entire region of the attention person, and the image of the attention person transfer region of the moving image from one still image in which the evaluation value is equal to or greater than the threshold value, and

generate a composite image including the at least two images extracted by the region image extraction unit.

12. The image processing apparatus according to claim 3,

wherein the at least one processor is configured to perform image processing for the image extracted by the at least one processor and generate a composite image including the image subjected to the image processing.

13. The image processing apparatus according to claim 4,

wherein the at least one processor is configured to perform image processing for the image extracted by the at least one processor and generate a composite image including the image subjected to the image processing.

14. The image processing apparatus according to claim 5,

wherein the at least one processor is configured to perform image processing for the image extracted by the at least one processor and generate a composite image including the image subjected to the image processing.

15. The image processing apparatus according to claim 3,

wherein, when the attention person transfer region of the moving image is not detected by the transfer region detection unit, the at least one processor is configured to

extract the image of the entire region of the attention person from the still image in which the evaluation value is equal to or greater than the threshold value, and

generate a composite image including the image of the entire region of the attention person.

16. A region detection method comprising:

extracting a plurality of frames of still image data from moving image data;

detecting an attention person, who is a person to be processed, from a plurality of still images corresponding to the plurality of frames of still image data;

tracking a movement of the attention person in a moving image corresponding to the moving image data, on the basis of the detection result of the attention person in the plurality of still images, and detecting a movement trajectory of the attention person;

analyzing a motion of the attention person in the moving image, on the basis of the movement trajectory of the attention person, and calculating an evaluation value for the motion of the attention person in each of the plurality of still images, on the basis of the analyzed motion of the attention person;

outputting the still image data of a still image in which the evaluation value for the motion of the attention person is equal to or greater than a threshold value from the plurality of frames of still image data;

detecting an attention person movement region of the moving image captured in the still image in which the evaluation value is equal to or greater than the threshold value, on the basis of the movement trajectory of the attention person;

detecting a face region of the attention person in the still image in which the evaluation value is equal to or greater than the threshold value; and

detecting an entire region of the attention person in the still image in which the evaluation value is equal to or greater than the threshold value, on the basis of the attention person movement region of the moving image and the face region of the attention person.

17. The region detection method according to claim 16, further comprising:

detecting an attention person transfer region of the moving image in the still image in which the evaluation value is equal to or greater than the threshold value, on the basis of the movement trajectory of the attention person.

18. An image processing method comprising:

extracting the face region of the attention person, the attention person movement region of the moving image, the entire region of the attention person, and the attention person transfer region of the moving image using the region detection method according to claim 17;

extracting at least one of an image of the face region of the attention person, an image of the attention person movement region of the moving image, an image of the entire region of the attention person, and an image of the attention person transfer region of the moving image, which respectively correspond to the face region of the attention person, the attention person movement region of the moving image, the entire region of the attention person, and the attention person transfer region of the moving image, from the still image in which the evaluation value is equal to or greater than the threshold value; and

allowing a composite image generation unit to generate a composite image including the at least one extracted image.

19. A non-transitory computer-readable recording medium storing a program that causes a computer to perform the region detection method according to claim 16.

20. A non-transitory computer-readable recording medium storing a program that causes a computer to perform the image processing method according to claim 18.