Method and means for detecting people in image sequences

• US 5,987,154 A
• Filed: 11/22/1996
• Issued: 11/16/1999
• Est. Priority Date: 07/19/1993
• Status: Expired due to Term

• Current Assignee: Lucent Technologies, Inc. (Nokia Corporation)
• Original Assignee: Lucent Technologies, Inc. (Nokia Corporation)
• Inventors: Gibbon, David C., Shahraray, Behzad, Segen, Jakub
• Primary Examiner(s): Boudreau, Leo H.
• Assistant Examiner(s): Tadayon, Bijan
• Application Number:
  US08/754,910
• Time in Patent Office: 1,089 Days
• Field of Search: 382/115, 382/116, 382/117, 382/118, 382/119, 382/120, 382/124, 382/181, 382/190, 382/209, 382/312, 382/171, 382/103, 382/107, 382/142, 382/199, 382/201, 382/203, 382/204 , 382/217, 382/224, 382/225, 382/226, 382/227, 382/228, 382/266, 382/268, 382/269, 382/286, 382/291, 340/825.34, 340/825.31, 902/3, 348/77, 348/143, 348/161, 348/171, 348/418, 235/382.5
• US Class Current: 382/115
• CPC Class Codes: G06T 2207/10016 Video; Image sequence
• CPC Class Codes: G06T 2207/30196 Human being; Person
• CPC Class Codes: G06T 2207/30201 Face
• CPC Class Codes: G06T 7/251 involving models
• CPC Class Codes: G06T 7/254 involving subtraction of images
• CPC Class Codes: G06V 40/10 Human or animal bodies, e.g. vehicle occupants or pedestrians; Body parts, e.g. hands
• CPC Class Codes: G06V 40/103 Static body considered as a whole, e.g. static pedestrian or occupant recognition
• CPC Class Codes: G06V 40/167 using comparisons between temporally consecutive images
• CPC Class Codes: H04N 19/20 using video object coding
• CPC Class Codes: H04N 7/15 Conference systems
• CPC Class Codes: H04N 7/18 Closed-circuit television [CCTV] systems, i.e. systems in which the video signal is not broadcast

Abstract

The head in a series of video images is identified by digitizing sequential images, subtracting a previous image from an input image to determine moving objects, calculating boundary curvature extremes of regions in the subtracted image, comparing the extremes with a stored model of a human head to find regions shaped like a human head, and identifying the head with a surrounding shape.

Claims

1. A method of locating a person in a video picture, comprising the steps of:

forming a differential image from time displaced video images to extract differential figures from the video images;

comparing local boundary curvature extremes of outlines of the differential figures with outlines of a stored model of at least a part of a human body which includes a head and a portion of the body immediately below the head; and

in response to the comparing step identifying a region in the video picture from the step of comparing the local boundary curvature extremes of outlines of the differential figures with outlines of the stored model of the head and the portion of the body immediately below the head;

the portion of the body immediately below the head being a neck.

2. A method as in claim 1, wherein the step of forming the differential image includes digitizing the video images and subtracting one image from another time displaced image.

3. A method as in claim 1, wherein the forming step includes digitizing the video images to form two dimensional arrays of digital image data and subtracting one image from another time displaced image.

4. A method as in claim 3, wherein the forming step includes forming a threshold and taking absolute values of the digital image data and comparing the absolute values with a threshold.

5. A method as in claim 1, wherein the step of comparing the local boundary curvature extremes includes calculating outlines of the local boundary curvature extremes before comparing the outlines of the curvature extremes with outlines of the stored model.

7. A method as in claim 1, wherein the step of identifying includes the step of fitting a surrounding band to the portion of the region of the outlines of the differential figures corresponding to outlines of the head.

8. A method as in claim 7, wherein the surrounding band is an annulus.

9. A method as in claim 1, further comprising the step of responding to the identifying step and sensing data from a subregion of one of the video images corresponding to the outlines of a pattern of a head for display of at least a portion of the subregion.

10. A method as in claim 9, wherein the display of at least a portion of the subregion includes display of outlines of a pattern of a head.

11. A method as in claim 9, further comprising the step of responding to the step of selecting includes the step of transmitting data of outlines of the head within said subregion.

12. A method as in claim 1, further comprising responding to the identifying step and then performing the step of sensing data from a subregion of one of the video images corresponding to a pattern of a head to transmit outlines of a pattern of a head and achieve the effect of a camera being manually pointed at the head.

13. A method as in claim 12, wherein the sensing of the region includes controlling a mechanical system for pointing a camera to keep the pattern of the head within one of the video images.

14. A method as in claim 9, wherein the step of sensing includes allocating a greater portion of transmission bandwidth to same subregion.

15. A method as in claim 9, wherein the step of sensing includes the step of selecting one of several cameras in a system on a basis of the sensing so as to display the camera with the subregion with the pattern of said head in its field of view.

16. A method as in claim 9, wherein the step of sensing a subregion includes the step of forming a computer input to permit rendering a three dimensional representation.

17. A method as in claim 9, wherein the step of sensing includes the step of limiting an area of the sensing to the subregion to permit increase of speed and efficiency of recognizing a face on said head.

18. A method as in claim 9, wherein the step of sensing includes storing statistical data about motion and presence of heads in a scene.

19. A method as in claim 1, wherein the step of identifying includes centering the head in a video image.

20. A method as in claim 1, wherein the step of identifying includes the step of placing a pattern of a head at a predetermined position in at least one of the video images.

21. A method as in claim 1, wherein the step of forming a differential image from video images includes the step of subtracting immediately successive images.

22. A method as in claim 1, wherein the step of forming includes the step of subtracting images separated from each other by a time τ

and varying the time τ

to adjust the figures.

23. A as in claim 22, wherein the step of varying the time τ

is done manually.

47. A method as in claim 1, further comprising controlling a camera on the basis of said identifying step so as to place the head within a desired part of the image.

48. A method as in claim 1, further comprising electronically controlling a display on a basis of said identifying step so as to place the head within a desired part of the display.

6. An apparatus for locating a person in a video picture, comprising:

a video device containing video images of a scene;

a processor responsive to said video device for forming a differential image from a time displaced video images to extract differential figures;

comparing local boundary curvature extremes of outlines of the differential figures with outlines of a stored model of at least a part of a human body which includes a head and a portion of the body immediately below said head; and

identifying a region in the video picture in response to the comparing of the local boundary curvature extremes of outlines of the differential figures with outlines of the stored model of the head and the portion of the body immediately below the head;

the portion of the body immediately below the head being a neck.

24. An apparatus for locating a person in a video picture, comprising:

a video device containing video images of a scene;

a processor responsive to said video device for forming a differential image from a time displaced video images to extract differential figures;

comparing local boundary curvature extremes of outlines of the differential figures with outlines of a stored model of at least a part of a human body which includes a head and a portion of the body immediately below said head; and

identifying a region in the video picture in response to the comparing of the local boundary curvature extremes of outlines of the differential figures with outlines of the stored model of the head and the portion of the body immediately below the head;

wherein said processor fits a surrounding shape to the portion of the region boundary corresponding to the head in the video images and the surrounding shape is an annulus.

25. An apparatus as in claim 24, wherein the processor digitizes the video images and subtracts a video image from a time displaced video image.

26. An apparatus as in claim 24, wherein the processor digitizes the images to form two dimensional arrays of digital data and subtracts one of said video images from a time displaced video image.

27. An apparatus as in claim 26, wherein the processor forms a threshold and takes absolute values of the digital image data and compares the absolute values with a threshold.

28. An apparatus as in claim 24, wherein said processor calculates outlines of the local boundary curvature extremes before comparing the outlines of the local boundary curvature extremes with outlines of the stored model.

29. An apparatus as in claim 24, wherein said processor determines in the model normals to the outlines of the head and the portion of the body immediately below the head to identify the regions of a neck;

and compares the outlines in the model with outlines in the differential figures.

30. An apparatus as in claim 24, wherein said device is a video display responsive to said processor for electronically placing the region in a desired position of an image.

31. An apparatus as in claim 24, wherein the portion of the body below the head is a neck.

32. An apparatus as in claim 24, wherein said device is a video display, said processor detects data from a subregion of a differential image corresponding a pattern of the human head and a portion of a body immediately below the head and transmits the data from the subregion to said video display.

33. An apparatus as in claim 32, wherein the video display displays said head.

34. An apparatus as in claim 32, further comprising a recorder for recording data of the head in said subregion.

35. An apparatus as in claim 24, said processor senses data from a subregion of one of said video images corresponding to a pattern of a human head and the portion of a body immediately below the head to transmit a human head and electronically produces the effect of a camera being manually pointed at the head.

36. An apparatus as in claim 35, wherein the device is a video camera responsive to said processor, and the processor in sensing the region points the camera to keep the head within the one of the video images.

37. An apparatus as in claim 32, said processor allocates a greater portion of transmission bandwidth to the subregion that contains the pattern of the head.

38. An apparatus as in claim 32, wherein the device includes a plurality of cameras and said processor when sensing selects one of said cameras on a basis of the sensing so as to use the camera with the pattern of the head in its field of view.

39. An apparatus as in claim 32, wherein the processor forms a computer input to permit rendering a three dimensional representation.

40. An apparatus as in claim 32, wherein the processor limits areas of the sensing to the subregion to permit increase of speed and efficiency of recognizing faces on the pattern of said head.

41. An apparatus as in claim 32, wherein the processor stores statistical data about motion and presence of patterns of heads in a scene.

42. An apparatus as in claim 24, wherein the processor centers the pattern of a head in a video picture.

43. An apparatus as in claim 24, wherein the processor places an image of the pattern of the head at a predetermined position in one of the video pictures.

44. An apparatus as in claim 24, wherein the processor subtracts immediately successive ones of said time displaced video images.

45. An apparatus as in claim 24, wherein the processor subtracts time displaced images separated from each other by a time τ

and varies the time τ

to adjust the figures.

46. An Apparatus as in claim 45, wherein said processor varies the time τ

in response to manual operation.

49. An apparatus as in claim 24, wherein said device is a camera responsive to said processor for placing the region in a desired position in an image.