REGION DETECTION DEVICE, REGION DETECTION METHOD, IMAGE PROCESSING APPARATUS, IMAGE PROCESSING METHOD, PROGRAM, AND RECORDING MEDIUM

• US 20160086342A1
• Filed: 09/08/2015
• Published: 03/24/2016
• Est. Priority Date: 09/24/2014
• Status: Active Grant

• Current Assignee: Fujifilm Corporation (Fujifilm Holdings Corporation)
• Original Assignee: Fujifilm Corporation (Fujifilm Holdings Corporation)
• Inventors: YAMAJI, Kei
• Granted Patent: US 9,734,612 B2
• Time in Patent Office: Days
• US Class Current: 1/1
• CPC Class Codes: G06T 11/60 Editing figures and text; Combining figures or text
• CPC Class Codes: G06T 2207/10016 Video; Image sequence
• CPC Class Codes: G06T 7/215 Motion-based segmentation
• CPC Class Codes: G06T 7/246 using feature-based methods, e.g. the tracking of corners or segments
• CPC Class Codes: G06V 40/161 Detection; Localisation; Normalisation

Abstract

The image processing apparatus includes a region detection unit that detects a face region of the attention person, an attention person movement region of the moving image, the entire region of the attention person, and an attention person transfer region of the moving image, a region image extraction unit that extracts an image of the face region of the attention person, an image of the attention person movement region of the moving image, an image of the entire region of the attention person, and an image of the attention person transfer region of the moving image, which respectively correspond to the face region of the attention person, the attention person movement region of the moving image, the entire region of the attention person, and the attention person transfer region of the moving image, from the still image, and a composite image generation unit that generates a composite image.

Claims

1. A region detection device comprising:

a still image data extraction unit that extracts a plurality of frames of still image data from moving image data;

an attention person detection unit that detects an attention person, who is a person to be processed, from a plurality of still images corresponding to the plurality of frames of still image data;

a movement trajectory detection unit that tracks the movement of the attention person in a moving image corresponding to the moving image data, on the basis of the detection result of the attention person in the plurality of still images, and detects a movement trajectory of the attention person;

a motion analysis unit that analyzes a motion of the attention person in the moving image, on the basis of the movement trajectory of the attention person, and calculates an evaluation value for the motion of the attention person in each of the plurality of still images, on the basis of the analyzed motion of the attention person;

a still image data output unit that outputs the still image data of a still image in which the evaluation value for the motion of the attention person is equal to or greater than a threshold value from the plurality of frames of still image data;

a movement region detection unit that detects an attention person movement region of the moving image captured in the still image in which the evaluation value is equal to or greater than the threshold value, on the basis of the movement trajectory of the attention person;

a face region detection unit that detects a face region of the attention person in the still image in which the evaluation value is equal to or greater than the threshold value; and

a person region detection unit that detects the entire region of the attention person in the still image in which the evaluation value is equal to or greater than the threshold value, on the basis of the attention person movement region of the moving image and the face region of the attention person.

2. The region detection device according to claim 1, further comprising:

a transfer region detection unit that detects an attention person transfer region of the moving image in the still image in which the evaluation value is equal to or greater than the threshold value, on the basis of the movement trajectory of the attention person.

3. An image processing apparatus comprising:

the region detection device according to claim 2;

a region image extraction unit that extracts at least one of an image of the face region of the attention person, an image of the attention person movement region of the moving image, an image of the entire region of the attention person, and an image of the attention person transfer region of the moving image, which respectively correspond to the face region of the attention person, the attention person movement region of the moving image, the entire region of the attention person, and the attention person transfer region of the moving image, from the still image in which the evaluation value is equal to or greater than the threshold value; and

a composite image generation unit that generates a composite image including the at least one image extracted by the region image extraction unit.

4. The image processing apparatus according to claim 3,

wherein, in case where the transfer region detection unit detects the attention person transfer region of the moving image, the region image extraction unit extracts the image of the attention person transfer region of the moving image from the still image in which the evaluation value is equal to or greater than the threshold value, and

the composite image generation unit generates a composite image including the image of the attention person transfer region of the moving image.

5. The image processing apparatus according to claim 4,

wherein, in case where another image is automatically superimposed and combined with the image of the attention person transfer region of the moving image, the composite image generation unit superimposes the image on an image of a region other than the entire region of the attention person and combines the images.

6. The image processing apparatus according to claim 4,

wherein, in case where another image is automatically superimposed and combined with the image of the attention person transfer region of the moving image, the composite image generation unit changes an image of a region, with which the image is superimposed and combined, from an image of a region other than the face region of the attention person and the attention person movement region of the moving image, an image of a region other than the entire region of the attention person, and an image of a region other than the attention person transfer region of the moving image, depending on the purpose of the composite image.

7. The image processing apparatus according to claim 4,

wherein, in case where another image is automatically superimposed and combined with the image of the attention person transfer region of the moving image, the composite image generation unit changes an image of a region, with which the image is superimposed and combined, from an image of a region other than the face region of the attention person and the attention person movement region of the moving image, an image of a region other than the entire region of the attention person, and an image of a region other than the attention person transfer region of the moving image, depending on the type of image to be superimposed and combined.

8. The image processing apparatus according to claim 4, further comprising:

an instruction input unit that receives an instruction input by a user,

wherein the composite image generation unit superimposes an image designated by the instruction on the image of the attention person transfer region of the moving image at a position designated by the instruction and combines the images.

9. The image processing apparatus according to claim 4,

wherein the movement trajectory detection unit further detects a transfer direction and a transfer speed of the attention person, and

the region image extraction unit extends an extraction range in the transfer direction of the attention person as the transfer speed of the attention person increases and extracts the image of the attention person transfer region of the moving image.

10. The image processing apparatus according to claim 9,

wherein the region image extraction unit extracts the image of the attention person transfer region of the moving image such that an extraction range in a direction opposite to the transfer direction of the attention person is wider than the extraction range in the transfer direction of the attention person.

11. The image processing apparatus according to claim 3,

wherein the region image extraction unit extracts at least two of the image of the face region of the attention person, the image of the attention person movement region of the moving image, the image of the entire region of the attention person, and the image of the attention person transfer region of the moving image from one still image in which the evaluation value is equal to or greater than the threshold value, and

the composite image generation unit generates a composite image including the at least two images extracted by the region image extraction unit.

12. The image processing apparatus according to claim 3,

wherein the composite image generation unit performs image processing for the image extracted by the region image extraction unit and generates a composite image including the image subjected to the image processing.

13. The image processing apparatus according to claim 4,

wherein the composite image generation unit performs image processing for the image extracted by the region image extraction unit and generates a composite image including the image subjected to the image processing.

14. The image processing apparatus according to claim 5,

wherein the composite image generation unit performs image processing for the image extracted by the region image extraction unit and generates a composite image including the image subjected to the image processing.

15. The image processing apparatus according to claim 3,

wherein, in case where the attention person transfer region of the moving image is not detected by the transfer region detection unit, the region image extraction unit extracts the image of the entire region of the attention person from the still image in which the evaluation value is equal to or greater than the threshold value, and

the composite image generation unit generates a composite image including the image of the entire region of the attention person.

16. A region detection method comprising:

a step of allowing a still image data extraction unit to extract a plurality of frames of still image data from moving image data;

a step of allowing an attention person detection unit to detect an attention person, who is a person to be processed, from a plurality of still images corresponding to the plurality of frames of still image data;

a step of allowing a movement trajectory detection unit to track the movement of the attention person in a moving image corresponding to the moving image data, on the basis of the detection result of the attention person in the plurality of still images, and to detect a movement trajectory of the attention person;

a step of allowing a motion analysis unit to analyze a motion of the attention person in the moving image, on the basis of the movement trajectory of the attention person, and to calculate an evaluation value for the motion of the attention person in each of the plurality of still images, on the basis of the analyzed motion of the attention person;

a step of allowing a still image data output unit to output the still image data of a still image in which the evaluation value for the motion of the attention person is equal to or greater than a threshold value from the plurality of frames of still image data;

a step of allowing a movement region detection unit to detect an attention person movement region of the moving image captured in the still image in which the evaluation value is equal to or greater than the threshold value, on the basis of the movement trajectory of the attention person;

a step of allowing a face region detection unit to detect a face region of the attention person in the still image in which the evaluation value is equal to or greater than the threshold value; and

a step of allowing a person region detection unit to detect the entire region of the attention person in the still image in which the evaluation value is equal to or greater than the threshold value, on the basis of the attention person movement region of the moving image and the face region of the attention person.

17. The region detection method according to claim 16, further comprising:

a step of allowing a transfer region detection unit to detect an attention person transfer region of the moving image in the still image in which the evaluation value is equal to or greater than the threshold value, on the basis of the movement trajectory of the attention person.

18. An image processing method comprising:

a step of extracting the face region of the attention person, the attention person movement region of the moving image, the entire region of the attention person, and the attention person transfer region of the moving image using the region detection method according to claim 17,

a step of allowing a region image extraction unit to extract at least one of an image of the face region of the attention person, an image of the attention person movement region of the moving image, an image of the entire region of the attention person, and an image of the attention person transfer region of the moving image, which respectively correspond to the face region of the attention person, the attention person movement region of the moving image, the entire region of the attention person, and the attention person transfer region of the moving image, from the still image in which the evaluation value is equal to or greater than the threshold value; and

a step of allowing a composite image generation unit to generate a composite image including the at least one image extracted by the region image extraction unit.

19. A non-transitory computer-readable recording medium storing a program that causes a computer to perform each step of the region detection method according to claim 16.

20. A non-transitory computer-readable recording medium storing a program that causes a computer to perform each step of the image processing method according to claim 18.