Video signal follow-up processing system

• US 5,546,125 A
• Filed: 07/06/1994
• Issued: 08/13/1996
• Est. Priority Date: 07/14/1993
• Status: Expired due to Term

• Current Assignee: Sony Corporation (Sony Group Corp.)
• Original Assignee: Sony Corporation (Sony Group Corp.)
• Inventors: Tomitaka, Tadafusa, Sekiya, Tsuneo
• Primary Examiner(s): Chin, Tommy P.
• Assistant Examiner(s): Le, Vu
• Application Number:
  US08/268,125
• Time in Patent Office: 769 Days
• Field of Search: 348/169-172, 348/135-137, 348/149-157, 348/649-652, 348/214, 348/32, 348/30
• US Class Current: 348/169
• CPC Class Codes: G01S 3/7865 using correlation of the live video image with a stored image

Abstract

A video signal follow-up processing system for adaptively tracking to the moving of a subject. A detection feature pattern is formed through acquisition of brightness and hue frequency characteristic data based on pixel information in a detection measurement frame, a similarity calculation method, which can distinguish a reference measurement frame from other areas, is selected on the screen, and the position of a detection measurement frame with a feature pattern having the highest similarity with the standard feature pattern obtained from the reference measurement frame is determined, in order to change and control an image to be projected on the display screen based on the positional information of the detection measurement frame, so that the video signal follow-up processing system can adaptively track to the moving of the subject.

Claims

1. A camera apparatus comprising:

pixel information forming means for forming pixel information constituting a display screen based on pickup output signal obtained through a lens block;

reference measurement frame setting means for setting a reference measurement frame with a predetermined size on a predetermined position on said display screen;

detection measurement frame setting means for setting a detection measurement frame with a predetermined size on said display screen;

standard frequency characteristic data forming means for forming standard frequency characteristic data for a brightness level and/or hue angle based on brightness and/or hue information on an image in said reference measurement frame;

detection frequency characteristic data forming means for forming detection frequency characteristic data for a brightness level and/or hue angle based on brightness and/or hue information on an image in said detection measurement frame;

similarity calculating means for calculating the similarity of said standard frequency characteristic data and/or said detection frequency characteristic data by several methods;

effective similarity calculation method selection means for selecting the most effective similarity data on the screen among a plurality of similarities obtained from said similarity calculation;

detection measurement frame determination means for determining said detection measurement frame with a higher similarity by using the similarity calculation method selected by said effective similarity calculation method selection means; and

pixel information modification control means for controlling said pixel information forming means so as to match the position of said pixel information on the determined detection measurement frame with the position of said reference measurement frame.

2. The camera apparatus according to claim 1, wherein:

said detection measurement frame setting means sets a plurality of detection measurement frame at predetermined positions on said display screen which are different from each other;

said detection frequency characteristic data forming means forms said detection frequency characteristic data from each of said plurality of detection measurement frames;

said similarity calculating means finds similarity of the standard frequency characteristic data and/or said detection frequency characteristic data by plural way of calculation method;

said effective similarity calculation method selection means selects the most effective similarity calculation method on the screen;

said detection measurement frame determination means finds similarity by using the calculation method selected by said effective similarity selection means to determine a detection measurement frame having detection frequency characteristic data with the highest similarity; and

said pixel information modification control means controls said pixel information formation means so as to correspond the position of image information of the above determined detection frame to the position of said reference measurement frame.

3. The camera apparatus according to claim 2, wherein:

said pixel information formation means decreases the frequency band by filtering the brightness signal and the hue signal and thins out, and then converts said color difference signal into hue signal and saturation signal.

4. The camera apparatus according to claim 3, wherein:

said pixel information modification control means moves a field of view image projected in said display screen in such a manner that an image projected in said detection measurement frame determined by said detection measurement frame detection means is projected at the position of said reference measurement frame.

5. The camera apparatus according to claim 4, wherein:

said pixel information modification control means moves a field of view image projected in said display screen in such a manner that an image projected in said detection measurement frame determined by said detection measurement frame detection means is projected at the position of said reference measurement frame, by panning and tilt-controlling said lens block unit of said pixel information formation means based on the result of determination of said detection measurement frame detection means.

6. The camera apparatus according to claim 2, wherein:

said standard frequency characteristic data forming means and/or said detection frequency characteristic data forming means use a membership function when the frequency of pixel data obtained by said pixel information forming means are counted.

7. The camera apparatus according to claim 2, wherein:

said pixel information forming means sends out brightness, hue, and saturation pixel information as said pixel information, and said standard frequency characteristic data forming means for hue signal and/or said detection frequency characteristic data forming means counts the frequency of said hue pixel information, said hue pixel information being uncounted because of judging that the color is absent when the saturation pixel information is at a predetermined threshold value or less.

8. The camera apparatus according to claim 2, wherein:

said similarity calculation means defines each of the standard frequency characteristic data and the detection frequency characteristic data as vector to calculate Euclidean distance, Hamming distance, and integral distance.

9. The camera apparatus according to claim 2, wherein:

said effective similarity calculation method selection means selects a calculation method which, with regard to the distance between the standard frequency characteristic data and the detection frequency characteristic data on a screen, the ratio of the first minimum distance representing the smallest distance (highest similarity) to the second minimum distance representing the second smallest distance (the second minimum distance/the first minimum distance) becomes largest.

10. The camera apparatus according to claim 2, wherein:

said effective similarity calculation method selection means selects a calculation method which, with regard to the distance between the standard frequency characteristics data and the detection frequency characteristic data, the ratio of the average distance representing an average value of the distance in one screen to the minimum distance representing the smallest distance (the average distance/the minimum distance) becomes largest.

11. The camera apparatus according to claim 2, wherein:

said effective similarity calculation method selection means selects a calculation method which, with regard to the distance between the standard frequency characteristic data and the detection frequency characteristic data on a screen, the ratio (the maximum distance/the minimum distance) of the minimum distance representing the smallest distance (that is, highest similarity) to the maximum distance representing the largest distance (that is, lowest similarity) becomes the largest.

12. The camera apparatus according to claim 2, wherein:

said reference measurement frame setting means forms said reference measurement frame at the center of said display screen.

13. The camera apparatus according to claim 2, wherein:

said detection measurement frame detection means calculates the distance of said detection frequency characteristic data to said standard frequency characteristic data for each of said plurality of detection measurement frames with the distance calculation method selected by said effective similarity calculation method selection means, and determines a detection measurement frame with small distance as said detection measurement frame with high similarity.

14. A video signal reproducing apparatus comprising:

pixel information formation means for forming pixel information obtained through a video signal reproduction device, which constitutes a display screen, based on reproducing video signal to send out as video output signal;

reference measurement frame setting means for setting a reference measurement frame with a predetermined size at a predetermined position on said display screen;

detection measurement frame setting means for setting a detection measurement frame with a predetermined size on said display screen;

standard frequency characteristic data formation means for forming standard frequency characteristic data for a brightness level and/or hue angle based on brightness and/or hue information on an image in said reference measurement frame;

detection frequency characteristic data formation means for forming detection frequency data for a brightness level and/or hue angle based on brightness and/or hue information on an image in said detection measurement frame;

similarity calculating means for calculating the similarity of said standard frequency characteristic data and/or said detection frequency characteristic data by several methods;

effective similarity calculation method selection means for selecting the most effective similarity data on the screen among a plurality of similarities obtained by said similarity calculation;

detection measurement frame determination means for determining said detection measurement frame with a higher similarity by using the similarity calculation method selected by said effective similarity calculation method selection means; and

pixel information modification control means for controlling said pixel information forming means so as to correspond the position of said pixel information on the determined detection measurement frame with the position of said reference measurement frame.

15. The video signal reproducing apparatus according to claim 14, wherein:

said detection measurement frame setting means sets a plurality of detection measurement frame at predetermined positions on said display screen which are different from each other;

said detection frequency characteristic data formation means forms said detection frequency characteristic data from each of said plurality of detection measurement frames;

said similarity calculating means determines similarity of the standard frequency characteristic data and/or said detection frequency characteristic data;

said effective similarity calculation method selection means selects the most effective similarity calculation method on the screen;

said detection measurement frame determination means finds similarity by using the calculation method selected by said effective similarity selection means to determine a detection measurement frame having detection similarity characteristic data with the highest similarity; and

said pixel information formation means is controlled so as to correspond the position of image information of the above detection measurement frame determined by said pixel information modification control means, to the position of said reference measurement frame.

16. The video signal reproducing apparatus according to claim 15, wherein:

said pixel information formation means decreases the frequency band by filtering the brightness signal and the hue signal and thins out, and then converts said color difference signal into hue signal and saturation signal.

17. The video signal reproducing apparatus according to claim 15, wherein:

said pixel information modification control means moves a field of view image projected in said display screen in such a manner that an image projected in said detection measurement frame determined by said detection measurement frame detection means is projected at the position of said reference measurement frame.

18. The video reproducing apparatus according to claim 15, wherein:

said pixel information modification control means moves a field of view image projected in said display screen in such a manner that an image projected in said detection measurement frame determined by said detection measurement frame detection means is projected at the position of said reference measurement frame, by panning and tilt-controlling said lens block unit of said pixel information formation means based on the result of determination of said detection measurement frame detection means.

19. The video signal reproducing apparatus according to claim 15, wherein:

said standard frequency characteristic data forming means and/or said detection frequency characteristic data forming means use a membership function when the frequency of pixel data obtained by said pixel information forming means are counted.

20. The video signal reproducing apparatus according to claim 15, wherein:

said pixel information forming means sends out brightness, hue, and saturation pixel information as said pixel information, and said standard frequency characteristic data forming means for hue signal and/or said detection frequency characteristic data forming means counts the frequency of said hue pixel information, which is uncounted when the saturation pixel information is at a predetermined threshold value or less because of judging that the color is absence.

21. The video signal reproducing apparatus according to claim 15, wherein:

said similarity calculation means defines each of the standard frequency characteristic data and the detection frequency characteristic data as vector to calculate Euclidean distance, Hamming distance, and integral distance.

22. The video signal reproducing apparatus according to claim 15, wherein:

said effective similarity calculation method selection means selects a calculation method which, with regard to the distance between the standard frequency characteristic data and the detection frequency characteristic data on a screen, the ratio of the first minimum distance representing the smallest distance (highest similarity) to the second minimum distance representing the second smallest distance (the second minimum distance/the first minimum distance) becomes largest.

23. The video signal reproducing apparatus according to claim 15, wherein:

said effective similarity calculation method selection means selects a calculation method which, with regard to the distance between the standard frequency characteristics data and the detection frequency characteristic data, the ratio of the average distance representing an average value of the distance in one screen to the minimum distance representing the smallest distance (the average distance/the minimum distance) becomes largest.

24. The video signal reproducing apparatus according to claim 15, wherein:

said effective similarity calculation method selection means selects a calculation method which, with regard to the distance between the standard frequency characteristic data and the detection frequency characteristic data on a screen, the ratio (the maximum distance/the minimum distance) of the minimum distance representing the smallest distance (that is, highest similarity) to the maximum distance representing the largest distance (that is, lowest similarity) becomes the largest.

25. A video signal reproducing apparatus according to claim 15, wherein:

said reference measurement frame setting means forms said reference measurement frame at the center of said display screen.

26. The video signal reproducing apparatus according to claim 15, wherein:

said detection measurement frame detection means calculates the distance of said detection frequency characteristic data to said standard frequency characteristic data for each of said plurality of detection measurement frames with the distance calculation method selected by said effective similarity calculation method selection means, and determines a detection measurement frame with small distance as said detection measurement frame with high similarity.

27. A video signal conversion apparatus comprising:

pixel information formation means for forming pixel information constituting a display screen based on video input signal arriving at a video input terminal from outside, to send out as conversion video output signal from a video output terminal;

reference measurement frame setting means for setting a reference measurement frame with a predetermined size on a predetermined position on said display screen;

detection measurement frame setting means for setting a detection measurement frame with a predetermined size on said display screen;

standard frequency characteristic data formation means for forming standard frequency characteristic data for a brightness level and/or hue angle based on brightness and/or hue information on an image in said reference measurement frame;

detection frequency characteristic data formation means for forming detection frequency characteristic data for a brightness level and/or hue angle based on brightness and/or hue information on an image in said detection measurement frame;

similarity calculating means for calculating the similarity of said standard frequency characteristic data and/or said detection frequency characteristic data by several methods;

effective similarity calculation method selection means for selecting the most effective similarity data on the screen among a plurality of similarities obtained from said similarity calculation;

detection measurement frame determination means for determining said detection measurement frame with a higher similarity by using the similarity calculation method selected by said effective similarity calculation method selection means; and

pixel information modification control means for controlling said pixel information formation means so as to match the position of the above pixel information on the determined detection measurement frame with the position of said reference measurement frame.

28. The video signal conversion apparatus according to claim 27, wherein:

said detection measurement frame setting means sets a plurality of detection measurement frame at predetermined positions on said display screen which are different from each other;

said detection frequency characteristic data forming means forms said detection frequency characteristic data from each of said plurality of detection measurement frames;

said similarity calculating means finds similarity of the standard frequency characteristic data and/or said detection frequency characteristic data by plural way of calculation method;

said effective similarity calculation method selection means selects the most effective similarity calculation method on the screen;

said detection measurement frame determination means finds similarity by using the calculation method selected by said effective similarity selection means to determine a detection measurement frame having detection frequency characteristic data with the highest similarity; and

said pixel information modification control means controls said pixel information formation means so as to correspond the position of image information of the above determined detection frame to the position of said reference measurement frame.

29. The video signal conversion apparatus according to claim 28, wherein:

said pixel information formation means decreases the frequency band by filtering the brightness signal and the hue signal and thins out, and then converts said color difference signal into hue signal and saturation signal.

30. The video signal conversion apparatus according to claim 28, wherein:

said pixel information modification control means moves a field of view image projected in said display screen in such a manner that an image projected in said detection measurement frame determined by said detection measurement frame detection means is projected at the position of said reference measurement frame.

31. The video signal conversion apparatus according to claim 28, wherein:

said pixel information modification control means moves a field of view image projected in said display screen in such a manner that an image projected in said detection measurement frame determined by said detection measurement frame detection means is projected at the position of said reference measurement frame, by panning and tilt-controlling said lens block unit of said pixel information formation means based on the result of determination of said detection measurement frame detection means.

32. The video signal conversion apparatus according to claim 28, wherein:

said standard frequency characteristic data forming means and/or said detection frequency characteristic data forming means use a membership function when the frequency of pixel data obtained by said pixel information forming means are counted.

33. The video signal conversion apparatus according to claim 28, wherein:

said pixel information forming means sends out brightness, hue, and saturation pixel information as said pixel information, and said standard frequency characteristic data forming means for hue signal and/or said detection frequency characteristic data forming means counts the frequency of said hue pixel information, which is uncounted when the saturation pixel information is at a predetermined threshold value or less because of judging that the color is absence.

34. The video signal conversion apparatus according to claim 28, wherein:

said similarity calculation means defines each of the standard frequency characteristic data and the detection frequency characteristic data as vector to calculate Euclidean distance, Hamming distance, and integral distance.

35. The video signal conversion apparatus according to claim 28, wherein:

said effective similarity calculation method selection means selects a calculation method which, with regard to the distance between the standard frequency characteristic data and the detection frequency characteristic data on a screen, the ratio of the first minimum distance representing the smallest distance (highest similarity) to the second minimum distance representing the second smallest distance (the second minimum distance/the first minimum distance) becomes largest.

36. The video signal conversion apparatus according to claim 28, wherein:

said effective similarity calculation method selection means selects a calculation method which, with regard to the distance between the standard frequency characteristics data and the detection frequency characteristic data, the ratio of the average distance representing an average value of the distance in one screen to the minimum distance representing the smallest distance (the average distance/the minimum distance) becomes largest.

37. The video signal conversion apparatus according to claim 28, wherein:

said effective similarity calculation method selection means selects a calculation method which, with regard to the distance between the standard frequency characteristic data and the detection frequency characteristic data on a screen, the ratio (the maximum distance/the minimum distance) of the minimum distance representing the smallest distance (that is, highest similarity) to the maximum distance representing the largest distance (that is, lowest similarity) becomes the largest.

38. The video signal conversion apparatus according to claim 28, wherein:

said reference measurement frame setting means forms said reference measurement frame at the center of said display screen.

39. The video signal conversion apparatus according to claim 28, wherein:

said detection measurement frame detection means calculates the distance of said detection frequency characteristic data to said standard frequency characteristic data for each of said plurality of detection measurement frames with the distance calculation method selected by said effective similarity calculation method selection means, and determines a detection measurement frame with small distance as said detection measurement frame with high similarity.