Face Tracking Method and Device

• US 20190311181A1
• Filed: 01/04/2019
• Published: 10/10/2019
• Est. Priority Date: 04/10/2018
• Status: Active Grant

• Current Assignee: Beijing BOE Optoelectronics Technology Co Ltd. (BOE Technology Group Co., Ltd.), BOE Technology Group Co., Ltd.
• Original Assignee: Beijing BOE Optoelectronics Technology Co Ltd. (BOE Technology Group Co., Ltd.), BOE Technology Group Co., Ltd.
• Inventors: CHU, Minglei, ZHANG, Hao, CHEN, Lili, SUN, Jian, GUO, Ziqiang, YAN, Guixin, SUN, Jiankang, LI, Xi, CHEN, Yinwei, QIN, Ruifeng, LU, Yuanjie
• Granted Patent: US 11,275,926 B2
• Time in Patent Office: Days
• CPC Class Codes: G06T 2207/30201 Face
• CPC Class Codes: G06T 7/248 involving reference images or patches
• CPC Class Codes: G06T 7/74 involving reference images or patches
• CPC Class Codes: G06V 10/446 using Haar-like filters, e.g. using integral image techniques
• CPC Class Codes: G06V 40/162 using pixel segmentation or colour matching
• CPC Class Codes: G06V 40/165 using facial parts and geometric relationships
• CPC Class Codes: G06V 40/171 Local features and components; Facial parts eye characteristics G06V40/18; Occluding parts, e.g. glasses; Geometrical relationships
• CPC Class Codes: G06V 40/173 face re-identification, e.g. recognising unknown faces across different face tracks

Abstract

Disclosed is face tracking method and device. The method includes: acquiring an initial facial image in a to-be-tracked picture; performing binarization processing on the initial facial image according to a standard range of color parameter and an actual value of the color parameter of each pixel in the initial facial image, to obtain a binarized facial image; acquiring a position of a preset organ in the binarized facial image; and acquiring a position of a final facial image according to the position of the preset organ and a position of the initial facial image.

Claims

1. A face tracking method, comprising steps of:

acquiring an initial facial image in a to-be-tracked picture;

performing binarization processing on the initial facial image according to a standard range of color parameter and an actual value of the color parameter of each pixel in the initial facial image, to obtain a binarized facial image;

acquiring a position of a preset organ in the binarized facial image; and

acquiring a position of a final facial image according to the position of the preset organ and a position of the initial facial image.

2. The face tracking method according to claim 1, wherein

the face tracking method includes at least one tracking period in which face tracking is sequentially performed for each one of to-be-tracked pictures, and in which the to-be-tracked pictures are pictures including the same face.

3. The face tracking method according to claim 2, wherein

in each of the tracking periods, in addition to the last to-be-tracked picture, after the step of acquiring a position of a final facial image, the face tracking method further comprises:

updating the standard range of color parameter according to the actual value of the color parameter of each pixel of the initial facial image.

4. The face tracking method according to claim 3, wherein the position of the preset organ is a position of region where a preset organ feature is satisfied.

5. The face tracking method according to claim 4, wherein the color parameter comprises hue, saturation, and brightness;

the step of updating the standard range of color parameter according to the actual value of the color parameter of each pixel of the initial facial image in the to-be-detected picture comprises;

using the initial facial image in the to-be-tracked picture as a to-be-processed image, and performing the image processing on the initial facial image in the to-be-detected picture by a preset standard range calculation method, to obtain a standard range of hue, a standard range saturation, and a standard range brightness.

6. The face tracking method according to claim 5, wherein the preset standard range calculation method comprises:

presetting a plurality of groups of reference ranges, each group of reference ranges including a reference range of hue, a reference range of saturation, and a reference range of brightness, wherein reference ranges of brightness in different groups are different from each other;

performing binarization processing on the to-be-processed image according to each group of reference ranges, wherein when performing binarization processing on the to-be-processed image, if the actual values of the hue, saturation and brightness of a pixel are within the corresponding reference ranges, greyscale of the pixel is set to 255, otherwise, the greyscale of the pixel is set to 0;

judging whether there is a region in each of the binarized images that satisfies the preset organ feature, and if yes, recording a current group of reference ranges; and

for the groups of reference ranges recorded, averaging the reference ranges of the hue, the reference ranges of the saturation, and the reference ranges of the brightness, respectively, to obtain a standard range of hue, a standard range of saturation, and a standard range of brightness corresponding to the to-be-processed image.

7. The face tracking method according to claim 6, wherein in each group of reference ranges, the reference range of the hue is 0-180, the reference range of the saturation is 20-255, and the reference range of the brightness is V-255, wherein V is an integer greater than or equal to zero and less than 255 and in different groups of the reference range, values of V are different from each other.

8. The face tracking method according to claim 7, wherein from the first group to the last group of reference ranges, the values of V are increased by a predetermined step length.

9. The face tracking method according to claim 8, wherein the predetermined step length is 10.

10. The face tracking method according to claim 5, wherein the preset organ includes a mouth, and the region that satisfies the preset organ feature is a rectangular region formed of successive pixel points having the greyscale of 0, and the rectangular region has a dimension in a width direction of the face three times larger than in a length direction of the face.

11. The face tracking method according to claim 5, wherein in each of the tracking period, the method further comprises performing a face detection process before the first to-be-tracked picture is received, the face detection process including:

acquiring the initial facial image in the to-be-detected picture, wherein the to-be-detected picture is a picture that is located before the to-be-tracked picture and includes the same face as the to-be-tracked picture; and

using the initial facial image in the to-be-detected picture as the to-be-processed image, and performing image processing on the initial facial image in the to-be-detected picture by the preset standard range calculation method, to obtain an initial standard range of color parameter, wherein the initial standard range of color parameter is used to binarize the initial facial image in the first to-be-tracked picture.

12. The face tracking method according to claim 11, wherein the to-be-detected picture is an RGB image and the step of acquiring the initial facial image in the to-be-detected picture comprises:

acquiring an image of an initial face region in the to-be-detected picture;

pre-processing the image acquired to eliminate noises in the image; and

converting a pre-processed image from an RGB image to an HSV image.

13. The face tracking method according to claim 11, wherein the image of the initial face region in the to-be-detected picture is acquired by haar feature detection.

14. The face tracking method according to claim 1, wherein the to-be-track picture is an RGB image, and the step of acquiring an initial facial image in a to-be-tracked picture comprises:

acquiring an image of an initial face region in the to-be-tracked picture;

pre-processing the image acquired to eliminate noises in the image; and

converting a pre-processed image from an RGB image to an HSV image.

15. The face tracking method according to claim 14, wherein the image of the initial face area in the to-be-tracked picture is acquired by the camshift algorithm.

16. A face tracking device, comprising a processor and a computer readable storage medium in which a computer program is stored, wherein the steps of the face tracking method according to claim 1 are implemented when the computer program is executed by the processor.

17. The face tracking device according to claim 16, further comprising an image capture component configured to continuously capture facial images.

18. The face tracking device according to claim 16, wherein

the face tracking device operates in at least one tracking period in which face tracking is sequentially performed for each one of to-be-tracked pictures, and in which the to-be-tracked pictures are pictures including the same face.