Object tracking apparatus

• US 5,473,369 A
• Filed: 02/23/1994
• Issued: 12/05/1995
• Est. Priority Date: 02/25/1993
• Status: Expired due to Term

• Current Assignee: Sony Corporation (Sony Group Corp.)
• Original Assignee: Sony Corporation (Sony Group Corp.)
• Inventors: Abe, Keiko
• Primary Examiner(s): Britton, Howard W.
• Application Number:
  US08/200,710
• Time in Patent Office: 650 Days
• Field of Search: 348/169, 348/170, 348/171, 348/172
• US Class Current: 348/169
• CPC Class Codes: G01S 3/7864 T.V. type tracking systems
• CPC Class Codes: G06T 7/20 Analysis of motion motion estimation for coding, decoding, compressing or decompressing digital video signals H04N19/43, H04N19/51

Abstract

An object detecting and tracking apparatus detects a tracking target object from a moving image photographed by a television camera and tracks the same, wherein the movement of the object is detected reliably and with high accuracy to automatically track the target object. When tracking is started after putting the target object in a region designating frame "WAKU" displayed on a screen in such a way as to be variable in size and position, a video signal input from a photographic optical system is Y/C-separated. After that, the target object is specified from a tracking region histogram, and movement vectors are obtained from a color-time-space image, color-time-space differential image, and/or luminance-time-space image thereof, thereby making it possible to more reliably detect even the movement of varied objects as compared with the conventional block matching.

Claims

1. An object tracking apparatus for tracking a target object within a designated region, comprising:

photographing means for photographing an object, and generating a video signal;

histogram processing means for calculating the histogram of said video signal within designated region on the initial screen;

time-space image processing means for obtaining a time-space image with respect to the range having higher frequency of said histogram; and

control means for controlling said photographing means based on the locus of said time-space image.

2. An object tracking apparatus according to claim 1, wherein:

said control means controls the zooming of said photographing means based on the change of width of the locus of said time-space image.

3. An object tracking apparatus according to claim 1, further comprising:

hue detecting means for detecting the hue of said video signal, wherein;

said histogram processing means calculates the histogram of said hue.

4. An object tracking apparatus according to claim 1, further comprising:

hue detecting means for detecting the hue of said video signal; and

a color filter for filtering for said hue, wherein;

said time-space image processing means calculates said time-space image with respect to the colors until upper n-th of said color filter output.

5. An object tracking apparatus according to claim 1, wherein:

said control means judges crossing of target objects or disappearance from the screen, based on the locus of said time-space image.

6. An object tracking apparatus according to claim 5, wherein:

said control means performs pattern-matching with the pattern of a target object region and the present screen, if occurred crossing of said target objects or disappearance from the screen, and judges re-appearance of said target object on the basis of the error of said pattern-matching.

7. An object tracking apparatus according to claim 1, wherein:

said control means detects the first moving amount based on the locus of said time-space image, and controls said photographing means in the photographing direction on the basis of said first moving amount.

8. An object tracking apparatus according to claim 7, further comprising:

hue detecting means for detecting the hue of said video signal, wherein;

said histogram processing means calculates the histogram of said hue and the histogram of a luminance;

said time-space image processing means generates a luminance-time-space image and a color-time-space image, with respect to the ranges having respectively higher frequency of the histogram of said hue and the histogram of said luminance; and

said control means detects the second and third moving amounts based on the locus of said luminance-time-space image and color-time-space image, to detect said first moving amount corresponding to said second and third moving amounts.

9. An object tracking apparatus according to claim 7, further comprising:

hue detecting means for detecting the hue of said video signal, wherein;

said histogram processing means calculates the histogram of said hue;

said time-space image processing means generates a color-time-space image and a color-time-space differential image, with respect to the range having higher frequency of the histogram of said hue; and

said control means detects the second and third moving amounts based on the locus of said color-time-space image and said color-time-space differential image, to determine said first moving amount corresponding to said second and third moving amounts.

10. An object tracking apparatus for tracking a target object within designated region, comprising:

histogram processing means for calculating the histogram of a video signal within designated region on the initial screen;

time-space image processing means for obtaining a time-space image with respect to the range having higher frequency in said histogram; and

control means for controlling said designating region based on the locus of said time-space image.

11. An object tracking apparatus according to claim 10, wherein:

said control means controls the size of said designating region on the basis of change of width of the locus of said time-space image.

12. An object tracking apparatus according to claim 10, further comprising:

hue detecting means for detecting the hue of said video signal, wherein;

said histogram processing means calculates the histogram of said hue.

13. An object tracking apparatus according to claim 10, further comprising:

hue detecting means for detecting the hue of said video signal; and

a color filter for filtering for said hue, wherein;

said time-space image processing means calculates said time-space image with respect to colors until upper n-th of said color filter output.

14. An object tracking apparatus according to claim 10, wherein:

said control means judges crossing of target objects and disappearance from the screen, based on the locus of said time-space image.

15. An object tracking apparatus according to claim 14, wherein:

said control means performs pattern-matching with the pattern of a target object region and the present screen, if occurred crossing of said target objects or disappearance from the screen, and judges re-appearance of said target object on the basis of the error of said pattern-matching.

16. An object tracking apparatus according to claim 10, wherein:

said control means detects the first moving amount based on the locus of said time-space image, and moves said designating region corresponding to said first moving amount.

17. An object target apparatus according to claim 16, further comprising:

hue detecting means for detecting the hue of said video signal, wherein;

said histogram processing means calculates the histograms of said hue and the histogram of a luminance;

said time-space image processing means generates a luminance-time-space image and a color-time-space image, with respect to the ranges having higher frequency respectively, of the histogram of said Hue and the histogram of said luminance; and

said control means detects the second and third moving amounts based on the loci of said luminance-time-space image and color-time-space image, to determine said first moving amount corresponding to said second and third moving amounts.

18. An object target apparatus according to claim 16, further comprising:

hue detecting means for detecting the hue of said video signal, wherein;

said histogram processing means calculates the histogram of said hue;

said time-space image processing means generates a color-time-space image and a color-time-space differential image, with respect to the range having higher frequency of the histogram of said hue; and

said control means detects the second and third moving amounts based on the loci of said color-time-space image and said color-time-space differential image, to determine said first moving amount corresponding to said second and third moving amounts.

19. An object tracking apparatus for tracking a target object within a designated region, comprising:

photographing means for photographing an object, and generating a video signal;

histogram processing means for calculating the histogram of said video signal within designated region on the initial screen;

time-space image processing means for obtaining a time-space image with respect to the range having higher frequency of said histogram; and

control means for controlling said photographing means based on the locus of said time-space image;

wherein said control means detects the first moving amount based on the locus of said time-space image, and controls said photographing means in the photographing direction on the basis of said first moving amount.

20. An object tracking apparatus according to claim 19, further comprising:

hue detecting means for detecting the hue of said video signal, wherein;

said histogram processing means calculates the histogram of said hue and the histogram of a luminance;

said time-space image processing means generates a luminance-time-space image and a color-time-space image, with respect to the ranges having respectively higher frequency of the histogram of said hue and the histogram of said luminance; and

said control means detects the second and third moving amounts based on the locus of said luminance time-space image and color-time-space image, to detect said first moving amount corresponding to said second and third moving amounts.

21. An object tracking apparatus according to claim 19, further comprising:

hue detecting means for detecting the hue of said video signal, wherein;

said histogram processing means calculates the histogram of said hue;

said time-space image processing means generates a color-time-space image and a color-time-space differential image, with respect to the range having higher frequency of the histogram of said hue; and

said control means detects the second and third moving amounts based on the locus of said color-time-space image and said color-time-space differential image, to determine said first moving amount corresponding to said second and third moving amounts.

22. An object tracking apparatus for tracking a target object within a designated region, comprising:

photographing means for photographing an object, and generating a video signal;

histogram processing means for calculating the histogram of said video signal within designated region on the initial screen;

time-space image processing means for obtaining a time-space image with respect to the range having higher frequency of said histogram; and

control means for controlling said photographing means based on the locus of said time-space image;

wherein said control means controls the zooming of said photographing means based on the change of width of the locus of said time-space image.

23. An object tracking apparatus for tracking a target object within a designated region, comprising:

photographing means for photographing an object, and generating a video signal;

histogram processing means for calculating the histogram of said video signal within designated region on the initial screen;

time-space image processing means for obtaining a time-space image with respect to the range having higher frequency of said histogram; and

control means for controlling said photographing means based on the locus of said time-space image; and

hue detecting means for detecting the hue of said video signal, wherein said histogram processing means calculates the histogram of said hue.

24. An object tracking apparatus for tracking a target object within a designated region, comprising:

photographing means for photographing an object, and generating a video signal;

histogram processing means for calculating the histogram of said video signal within designated region on the initial screen;

time-space image processing means for obtaining a time-space image with respect to the range having higher frequency of said histogram; and

control means for controlling said photographing means based on the locus of said time-space image;

wherein said control means judges crossing of target objects or disappearance from the screen, based on the locus of said time-space image.

25. An object tracking apparatus according to claim 24, wherein:

said control means performs pattern-matching with the pattern of a target object region and the present screen, if occurred crossing of said target objects or disappearance from the screen, and judges re-appearance of said target object on the basis of the error of said pattern-matching.

26. An object tracking apparatus for tracking a target object within a designated region, comprising:

photographing means for photographing an object, and generating a video signal;

histogram processing means for calculating the histogram of said video signal within designated region on the initial screen;

time-space image processing means for obtaining a time-space image with respect to the range having higher frequency of said histogram;

control means for controlling said photographing means based on the locus of said time-space image;

hue detecting means for detecting the hue of said video signal; and

a color filter for filtering for said hue, wherein said time-space image processing means calculates said time-space image with respect to the colors until upper n-th of said color filter output.

27. An object tracking apparatus for tracking a target object within designated region, comprising:

histogram processing means for calculating the histogram of a video signal within designated region on the initial screen;

time-space image processing means for obtaining a time-space image with respect to the range having higher frequency in said histogram; and

control means for controlling said designating region based on the locus of said time-space image;

wherein said control means detects the first moving amount based on the locus of said time-space image, and moves said designating region corresponding to said first moving amount.

28. An object tracking apparatus according to claim 27, further comprising:

hue detecting means for detecting the hue of said video signal, wherein;

said histogram processing means calculates the histograms of said hue and the histogram of a luminance;

said time-space image processing means generates a luminance-time-space image and a color-time-space image, with respect to the ranges having higher frequency respectively, of the histogram of said hue and the histogram of said luminance; and

said control means detects the second and third moving amounts based on the loci of said luminance-time-space image and color-time-space image, to determine said first moving amount corresponding to said second and third moving amounts.

29. An object tracking apparatus according to claim 27, further comprising:

hue detecting means for detecting the hue of said video signal, wherein;

said histogram processing means calculates the histogram of said hue;

said time-space image processing means generates a color-time-space image and a color-time-space differential image, with respect to the range having higher frequency of the histogram of said hue; and

said control means detects the second and third moving amounts based on the loci of said color-time-space image and said color-time-space differential image, to determine said first moving amount corresponding to said second and third moving amounts.

30. An object tracking apparatus for tracking a target object within designated region, comprising:

histogram processing means for calculating the histogram of a video signal within designated region on the initial screen;

time-space image processing means for obtaining a time-space image with respect to the range having higher frequency in said histogram; and

control means for controlling said designating region based on the locus of said time-space image;

wherein said control means controls the size of said designating region on the basis of change of width of the locus of said time-space image.

31. An object tracking apparatus for tracking a target object within designated region, comprising:

histogram processing means for calculating the histogram of a video signal within designated region on the initial screen;

time-space image processing means for obtaining a time-space image with respect to the range having higher frequency in said histogram;

control means for controlling said designating region based on the locus of said time-space image; and

hue detecting means for detecting the hue of said video signal, wherein said histogram processing means calculates the histogram of said hue.

32. An object tracking apparatus for tracking a target object within designated region, comprising:

histogram processing means for calculating the histogram of a video signal within designated region on the initial screen;

time-space image processing means for obtaining a time-space image with respect to the range having higher frequency in said histogram; and

control means for controlling said designating region based on the locus of said time-space image;

wherein said control means judges crossing of target objects and disappearance from the screen, based on the locus of said time-space image.

33. An object tracking apparatus according to claim 32, wherein:

said control means performs pattern-matching with the pattern of a target object region and the present screen, if occurred crossing of said target objects or disappearance from the screen, and judges re-appearance of said target object on the basis of the error of said pattern-matching.

34. An object tracking apparatus for tracking a target object within designated region, comprising:

histogram processing means for calculating the histogram of a video signal within designated region on the initial screen;

time-space image processing means for obtaining a time-space image with respect to the range having higher frequency in said histogram;

control means for controlling said designating region based on the locus of said time-space image;

hue detecting means for detecting the hue of said video signal; and

a color filter for filtering for said hue, wherein said time-space image processing means calculates said time-space image with respect to colors until upper n-th of said color filter output.