Tracking apparatus, tracking method, and storage medium to store tracking program

• US 8,929,598 B2
• Filed: 06/27/2012
• Issued: 01/06/2015
• Est. Priority Date: 06/29/2011
• Status: Active Grant

• Current Assignee: OM Digital Solutions Corporation
• Original Assignee: Olympus Imaging Corporation (Olympus Corporation)
• Inventors: Yoneyama, Hisashi
• Primary Examiner(s): Kassa, Yosef
• Application Number:
  US13/534,075
• Publication Number:
  US 20130004020A1
• Time in Patent Office: 923 Days
• Field of Search: 382/103, 382/209, 382/219, 382/278, 348/143, 348/144, 348/152, 348/611
• US Class Current: 382/103
• CPC Class Codes: G06V 40/162 using pixel segmentation or colour matching
• CPC Class Codes: H04N 23/61 based on recognised objects
• CPC Class Codes: H04N 23/611 where the recognised objects include parts of the human body
• CPC Class Codes: H04N 23/62 Control of parameters via user interfaces
• CPC Class Codes: H04N 23/635 Region indicators; Field of view indicators
• CPC Class Codes: H04N 23/673 based on contrast or high frequency components of image signals, e.g. hill climbing method

Abstract

A tracking apparatus includes a face detection unit, a face corresponding region detection unit, a face tracking unit, a peripheral part tracking unit, a tracking switch unit. The face tracking unit tracks the face in accordance with a result detected by the face detection unit or a result detected by the face corresponding region detection unit. The peripheral part tracking unit tracks, as a part other than the face, a part of the subject having a preset positional relationship with the face. The tracking switch unit switches to the tracking of a part other than the face by the peripheral part tracking unit when the face is not tracked by the face tracking unit.

Claims

1. A tracking apparatus comprising:

a face detection unit which detects a face of a subject from image data containing the subject;

a face corresponding region detection unit which detects a region corresponding to the face in the image data;

a face tracking unit which tracks the face in accordance with a result detected by the face detection unit or a result detected by the face corresponding region detection unit;

a peripheral part tracking unit which tracks, as a part other than the face, a part of the subject having a preset positional relationship with the face;

a tracking switch unit which switches to the tracking of the part other than the face by the peripheral part tracking unit when the face is not tracked by the face tracking unit;

a color information detection unit which detects a region having color information corresponding to a peripheral of the image data other than the face of the subject;

a face direction determination unit which determines a direction of the face tracked by the face tracking unit; and

a color information update processing unit which updates color information corresponding to the peripheral part when the face direction determination unit determines that the tracked face is directed forward, and does not update the color information corresponding to the peripheral part when the face direction determination unit determines that the tracked face is not direction forward.

2. The tracking apparatus according to claim 1,

wherein the face corresponding region detects color information for the region corresponding to the face in the image data, and

the tracking switch unit returns to the tracking of the face by the face tracking unit from the tracking by the peripheral part tracking unit when a region at a position estimated from the position of the part other than the face is detected as the face or when the color information corresponding to the face is detected.

3. The tracking apparatus according to claim 2,

wherein the tracking switch unit previously stores the color information corresponding to the face, and when color information in the region estimated to be the face corresponds to the previously stored color information, the tracking switch unit recognizes that the color information corresponding to the face is detected and then returns to the tracking of the face by the face tracking unit.

4. The tracking apparatus according to claim 2,

wherein the face tracking unit gives priority to the detection of the face by the face detection unit over the detection of the color information by the face corresponding region detection unit.

5. The tracking apparatus according to claim 2,

wherein the tracking switch unit determines that the face is not tracked by the face tracking unit when the face is not detected by at least the face detection unit and when the color information is not detected by the face corresponding region detection unit.

6. The tracking apparatus according to claim 5,

wherein the tracking switch unit determines that the face is not tracked by the face tracking unit when the image data includes no data for the face and when the face is not tracked in accordance with the region having the color information.

7. The cracking apparatus according to claim 2,

wherein the face corresponding region detection unit detects a flesh color as the color information.

8. A tracking method comprising:

causing a face detection unit to detect a face of a subject from image data containing the subject;

causing a face corresponding region detection unit to detect a region corresponding to the face in the image data;

causing a face tracking unit to track the face in accordance with a result detected by the face detection unit or a result detected by the face corresponding region detection unit;

causing a peripheral part tracking unit to track, as a part other than the face, a part of the subject having a preset positional relationship with the face;

switching to the tracking of the part other than the face by the peripheral part tracking unit when the face is not tracked by the face tracking unit;

causing a color information detection to detect a region having color information corresponding to a peripheral part to the image data other than the face of the subject;

causing a face direction determination to determine a direction of the face tracked by the face tracking unit; and

causing a color information update processing unit to update color information corresponding to the peripheral part when the face direction determination unit determines that the tracked face is directed forward, and does not update the color information corresponding to the peripheral part when the face direction determination unit determines that the tracked face is not directed forward.

9. The tracking method according to claim 8,

wherein the switch of the tracking comprises returning to the tracking of the face by the face tracking unit when the face is detected in a region estimated to be the face from peripheral parts of the face or when color information for the region corresponding to the face is detected.

10. A storage medium to store a tracking program processable by a computer, the tracking program comprising:

a face detection function which detects a face of a subject from image data containing the subject;

a face corresponding region detection function which detects a region corresponding to the face in the image data;

a face tracking function which tracks the face in accordance with a result detected by the face detection function or a result detected by the face corresponding region detection function;

a peripheral part tracking function which tracks, as a part other than the face, a part of the subject having a preset positional relationship with the face;

a tracking switch function which switches to the tracking of the part other than the face by the peripheral part tracking function when the face is not tracked by the face tracking function;

a color information which detects a region having color information corresponding to a peripheral part of the image data other than the face of the subject;

a face direction determination function which determines a direction of the face tracked by the face tracking function; and

a color information update processing function which updates color information corresponding to the peripheral part when the face direction determination function determines that the tracked face is directed forward, and does not update the color information corresponding to the peripheral part when the face direction determination function determines that the tracked face is not directed forward.

11. The storage medium according to claim 10,

wherein the tracking switch function returns to the tracking of the face by the face tracking function when the face is detected at a position estimated to be the face from the position of the peripheral part other than the face or when the color information for the region corresponding to the face is detected.

12. An imaging device comprising:

an imaging unit which images a subject and then outputs image data;

a face detection unit which detects a face of the subject from the image data;

a face tracking unit which tracks the face of the subject in the image data in accordance with the image data;

a peripheral part tracking unit which tracks a face peripheral part of the subject in the image data in accordance with the image data;

a display unit which displays an image based on the image data;

a control unit superposing a display frame indicating the position of the face on the image displayed on the display unit when the detection of the face by the face detection unit or the tracking of the face by the face tracking unit is effective, and superposing a display frame indicating the position of the face peripheral part on the image displayed on the display unit when the detection of the face by the face detection unit and the tracking of the face by the face tracking unit are ineffective;

a color information detection unit which detects a region having color information corresponding to a peripheral part of the image data other that the face of the subject;

a face direction determination unit which determines a direction of the face tracked by the face tracking unit; and

a color information update processing unit which updates color information corresponding to the peripheral part when the face direction determination unit determines that the tracked face is directed forward, and does not update the color information corresponding to the peripheral part when the face direction determination unit determines that the tracked face is not directed forward.

13. The imaging device according to claim 12,

wherein in response to a change from a first condition in which the detection of the face by the face detection unit or the tracking of the face by the face tracking unit is effective to a second condition in which the detection of the face by the face detection unit and the tracking of the face by the face tracking unit are ineffective, the control unit stops the superposition of the display frame which indicates the position of the face and which is superposed on the display unit.

14. The imaging device according to claim 13,

wherein when the detection of the face by the face detection unit and the tracking of the face by the face tracking unit are effective, the control unit superposes, on the image displayed on the display unit, a first display frame indicating the position of the face detected by the face detection unit.

15. The imaging device according to claim 13,

wherein when the detection of the face by the face detection unit and the tracking of the face by the face tracking unit are effective, the control unit superposes, on the image displayed on the display unit, the first display frame indicating the position of the face detected by the face detection unit and a second display frame indicating the position of the face tracked by the face tracking unit.

16. The imaging device according to claim 13,

wherein when the detection of the face by the face detection unit is ineffective and the operation of tracking the face by the face tracking unit is effective, the control unit superposes, on the image displayed on the display unit, a second display frame indicating the position of the face tracked by the face tracking unit.

17. The imaging device according to claim 12,

wherein the peripheral part tracking unit tracks the face by estimating the position of a breast of the subject to be the position of the face peripheral part in accordance with the position of the face detected by the face detection unit or the position of the face tracked by the face tracking unit.

18. The imaging device according to claim 12,

wherein the face tracking unit comprises a first color tracking unit which detects color information for the image data to track the face of the subject, and

a first luminance tracking unit which detects luminance information for the image data to track the face of the subject, and

the peripheral part tracking unit comprises a second color tracking unit which detects the color information for the image data to track the face peripheral part of the subject, and

a second luminance tracking unit which detects luminance information for the image data to track the face peripheral part of the subject.

19. The imaging device according to claim 12,

wherein the control unit displays a display frame indicating the position of the face and a display frame indicating the position of the face peripheral part so that these display frames are different in size or color.

20. The imaging device according to claim 12,

wherein the control unit superposes a display frame which comprises the face and the face peripheral part on the image displayed on the display unit.