OBJECT TRACKING

• US 20120288142A1
• Filed: 11/22/2011
• Published: 11/15/2012
• Est. Priority Date: 05/09/2011
• Status: Active Grant

• Current Assignee: Google LLC (Alphabet Inc.)
• Original Assignee: Google Inc. (Alphabet Inc.)
• Inventors: Gossweiler, Richard C. III, Zhao, Yong
• Granted Patent: US 8,306,267 B1
• Time in Patent Office: Days
• US Class Current: 382/103
• CPC Class Codes: G06V 40/164 using holistic features
• CPC Class Codes: G06V 40/167 using comparisons between temporally consecutive images

Abstract

In general, the subject matter described in this specification can be embodied in methods, systems, and program products. A computing system accesses an indication of a first template that includes a region of a first image. The region of the first image includes a graphical representation of a face. The computing system receives a second image. The computing system identifies indications of multiple candidate templates. Each respective candidate template from the multiple candidate templates includes a respective candidate region of the second image. The computing system compares at least the first template to each of the multiple candidate templates, to identify a matching template from among the multiple candidate templates that includes a candidate region that matches the region of the first image that includes the graphical representation of the face.

Claims

1. A computer-implemented method, comprising:

accessing, by a computing system, an indication of a first template that includes a region of a first image, the region of the first image including a graphical representation of a human face;

receiving, by the computing system, a second image;

identifying, by the computing system, indications of multiple candidate templates, each respective candidate template from the multiple candidate templates including a respective candidate region of the second image, wherein;

the multiple candidate templates are normalized representations of their respective regions of the second image,

normalizing the respective regions of the second image to generate the multiple candidate templates includes changing a viewpoint of at least one of the respective regions of the second image,

the human face in the at least one respective region of the second image is not directed forward, and

changing the viewpoint of the at least one respective region includes changing the viewpoint so that the human face in the respective candidate template is directed forward; and

comparing, by the computing system, at least the first template to each of the multiple candidate templates, to identify a matching template from among the multiple candidate templates that includes a candidate region that matches the region of the first image that includes the graphical representation of the human face.

2. The computer-implemented method of claim 1, wherein comparing at least the first template to each of the multiple candidate templates includes performing a pixel-wise sum of the absolute differences between the at least first template and one of the multiple candidate templates.

3. The computer-implemented method of claim 1, further comprising designating, by the computing system and as a region of the second image that includes a graphical representation of the human face in the second image, the region of the second image that is included in the matching template.

4. The computer-implemented method of claim 1, further comprising:

receiving the first image at the computing system as a first frame of a video that is captured by a camera;

wherein;

the second image is a second frame of the video that is captured by the camera, and

the second frame is an immediately next frame in the video after the first frame.

5. The computer-implemented method of claim 1, further comprising:

requesting, by the computing system, an object detection process on the first image to identify the region of the first image as including the graphical representation of the human face, the object detection process analyzing one or more portions of the first image that are not included in the region of the first image;

wherein comparing at least the first template to each of the multiple candidate templates does not include analyzing one or more portions of the second image that are outside of the respective multiple candidate templates.

6. The computer-implemented method of claim 1, further comprising:

requesting, by the computing system, an object detection process on the first image to identify the region of the first image as including the graphical representation of the human face, wherein;

the object detection process performs one or more feature detection processes on the first image; and

comparing the first template to each individual candidate template of the multiple candidate templates does not include performing the one or more feature detection processes on each individual candidate template.

7. The computer-implemented method of claim 1, further comprising:

requesting, by the computing system, an object detection process on the first image to identify the region of the first image as including the graphical representation of the human face, wherein the object detection process is performed by a central processing unit of the computing system or another computing system and comparing at least the first template to each of the multiple candidate templates is performed by a graphics processing unit of the computing system.

8. The computer-implemented method of claim 1, further comprising:

generating, by the computing system, multiple additional first templates that include respective regions of different images, the regions of the different images including respective graphical representations the human face; and

wherein comparing at least the first template to each of the multiple candidate templates includes comparing the multiple additional first templates to each of the multiple candidate templates to identify the matching template from among the multiple candidate templates that matches a region included in one of the first templates.

9. The computer-implemented method of claim 1, wherein the region of the first image includes mouth, nose, and eyes of the human face.

10. The computer-implemented method of claim 1, wherein:

the region of the first image is a non-rectangular region;

the first template is a normalized representation of the region of the first image; and

generating the normalized representation of the region of the first image includes generating a rectangular representation of the region of the first image.

11. The computer-implemented method of claim 1, wherein the first template has same dimensions as each of the multiple candidate templates.

12. The computer-implemented method of claim 1, wherein:

the first template is a normalized representation of the region of the first image; and

normalizing the region of the first image to generate the first template includes changing a viewpoint of the region of the first image.

15. The computer-implemented method of claim 1, further comprising, before identifying the matching template that includes the candidate region that matches the region of the first image, verifying that the matching template represents a human face by providing the matching template or a graphical representation included in the matching template to a face verification system.

16. The computer-implemented method of claim 1, further comprising using a location of the candidate region, of the matching template, in the second image to truck at least a portion of a view on a graphical display of the computing system so as to at least partially track movement of the human face with the at least portion of the view on the graphical display.

17. The computer-implemented method of claim 1, wherein identifying the indications of the multiple candidate templates includes selecting, as the regions of the second image to include in the respective multiple candidate templates, regions of the second image based on a determined trajectory of the human face in the first image and images that were received before the first image.

13. (canceled)

14. (canceled)

18. A computer-implemented method, the method comprising:

receiving, by a computing system, a first frame of a video;

requesting, by the computing system, an object detection process on the first frame of the video in order to identify an object;

receiving, by the computing system and in response to requesting the object detection process, an indication of a first region of the first frame that includes a graphical representation of the object;

receiving, by the computing system, a second frame of the video;

requesting, by the computing system, identification of multiple candidate regions in the second frame;

requesting, by the computing system, indications of multiple candidate templates that represent versions of the multiple candidate regions, at least one of the multiple candidate templates including a graphical representation of a viewpoint of the represented candidate region that has been changed from a graphical representation of a viewpoint of the represented candidate region as included in the second frame so as to be directed forward and different than the graphical representation of the viewpoint of the represented candidate region as included in the second frame, the graphical representation of the viewpoint of the represented candidate region as included in the second frame not being directed forward;

requesting, by the computing system, comparison of the multiple candidate templates to the first region or a first template that is generated from the first region to identify a matching template that has been identified matching the first region or the first template; and

designating, by the computing system, the region of the second frame from which the matching template was generated as a region of the second frame that includes a graphical representation of the object.

19. A computerized system, including:

a camera adapted to receive multiple frames of video;

one or more tangible computer-readable memory devices to store an indication of a first region, in a first frame of the video, that includes a graphical representation of a human face;

a computerized face candidate generator to (i) identify, in a second frame of the video that is subsequent to the first frame of the video, multiple candidate regions, and (ii) create multiple candidate templates from at least some of the multiple candidate regions, at least some of the multiple candidate templates illustrating the respective candidate regions from viewpoints that are different than viewpoints of the respective candidate regions so that the human faces in the at least some multiple candidate templates are directed forward, the human faces in the respective candidate regions not being directed forward; and

a computerized match determiner to compare the multiple candidate regions and the multiple candidate templates to the first region in order to identify one of the multiple candidate regions as including a graphical representation of the human face in the second frame of the video.

20. The computerized system of claim 19, wherein:

each of the multiple candidate template is a representation of the respective candidate region that has been normalized to a standard image size; and

the computerized match determiner'"'"'s comparing the multiple candidate regions to the first region includes comparing the multiple candidate templates to the first region or a template that is generated from the first region.

21. A computer-implemented method, comprising:

accessing, by a computing system, an indication of a first region of a first frame of a video, the first region of the first frame of the video including a graphical representation of a human face;

receiving, by the computing system, a second frame of the video;

identifying, by the computing system, multiple candidate regions of the second image;

changing a viewpoint of a candidate region, of the multiple candidate regions, in which the human face is not directed forward so that the human face in the candidate region is directed forward; and

comparing, by the computing system, the first region to each of the multiple candidate regions, including the candidate region in which the viewpoint of the candidate region was changed so that the human face was directed forward, to identify a matching region from among the multiple candidate regions that has been identified matching the first region.

22. The computer-implemented method of claim 21, wherein the matching region is the candidate region in which the viewpoint of the candidate region was changed so that the human face in the candidate region is directed forward.