Picture processing apparatus with object tracking

• US 5,552,823 A
• Filed: 07/17/1995
• Issued: 09/03/1996
• Est. Priority Date: 02/15/1992
• Status: Expired due to Fees

• Current Assignee: Sony Corporation (Sony Group Corp.)
• Original Assignee: Sony Corporation (Sony Group Corp.)
• Inventors: Kageyama, Koji
• Primary Examiner(s): Chin, Tommy P.
• Assistant Examiner(s): Le, Vu
• Application Number:
  US08/503,191
• Time in Patent Office: 414 Days
• Field of Search: 348/155, 348/169, 348/699, 348/416, 348/405, 348/407
• US Class Current: 348/155
• CPC Class Codes: G01S 3/7864 T.V. type tracking systems
• CPC Class Codes: G06T 2207/10016 Video; Image sequence
• CPC Class Codes: G06T 7/231 using full search
• CPC Class Codes: H04N 19/112 according to a given display mode, e.g. for interlaced or progressive display mode
• CPC Class Codes: H04N 19/51 Motion estimation or motion compensation
• CPC Class Codes: H04N 19/527 Global motion vector estimation
• CPC Class Codes: H04N 19/61 in combination with predictive coding
• CPC Class Codes: H04N 5/144 Movement detection for video coding H04N19/503; analysis of motion in general G06T7/20

Abstract

In a picture processing apparatus such as a camera, monitor camera, television and the like, an imaging device is controlled to follow a desired target object and image it in the best state. A distribution of motion vectors within a tracking vector detection area is detected, and then the most characteristic motion vector is detected. In a second embodiment a power of components in a high frequency band is detected per unit of predetermined blocks, difference data between picture data of motion pictures of the present frame and the previous frame is obtained, the power of the difference data is detected per unit of predetermined blocks, and an edge having movement is detected on the basis of the detection result.

Claims

1. A target object following apparatus comprising:

imaging means having a focus and an iris, for picking up an image of an imaging area and supplying a video signal corresponding to data produced therefrom as picture data including a target object;

imaging area changing means for changing said imaging area of said imaging means on a basis of a predetermined control signal;

a picture data blocking circuit for dividing said picture data corresponding to said image of said imaging area which is sequentially input into blocks of a predetermined number of pixels in both a horizontal direction and a vertical direction, said picture data being supplied from said image means;

an input circuit for setting a position of an area containing said target object in said picture data according to an input by a user and supplying a position signal produced therefrom;

a detection area setting circuit for setting a tracking vector detection area including a predetermined number of said blocks in the horizontal and the vertical directions according to said position signal;

a motion detection circuit for detecting motion vectors for respective ones of said blocks in said tracking vector detection area and supplying the detected motion vectors produced therefrom;

a tracking vector detection circuit for detecting a distribution of said detected motion vectors of said respective ones of said blocks in said tracking vector detection area and then detecting a tracking vector with a moving direction and a moving quantity of a portion of a picture in said tracking vector detection area on a basis of said distribution detection result;

a detection area correction circuit for moving said tracking vector detection area in correlation with a motion of said target object in said picture data, said tracking vector detection area being moved on a basis of said tracking vector with said moving direction and said moving quantity detected by said tracking vector detection circuit;

a window-frame data generation circuit for generating window-frame picture data of a window frame corresponding to said tracking vector detection area, and for supplying said predetermined control signal to said imaging area changing means;

a picture composing circuit for superimposing said window frame represented by said window-frame picture data upon said video signal, and generating a composite video signal therefrom;

display means for receiving said composite signal, and displaying a picture with said window frame superimposed; and

control means for detecting a signal level of said video signal for said tracking vector detection area, and for supplying a second predetermined control signal to said imaging means and thereby controlling said focus and said iris of said image means on a basis of said detected signal level of said video signal.

2. The target object following apparatus according to claim 1, wherein said motion detection circuit includes means for computing said plurality of motion vectors by performing a block matching method.

3. The target object following apparatus according to claim 1 or 2 further comprising:

a second motion detection circuit for detecting a plurality of motion vectors of respective ones of a plurality of blocks forming said picture data;

a picture data processing circuit for motion compensating said plurality of blocks forming picture data on the basis of said plurality of motion vectors and outputting residual data produced herefrom;

a motion field detection circuit for detecting a disorder of each one of said plurality of motion vectors for a respective one of said plurality of blocks relative to motion vectors of respective surrounding blocks adjacent to said respective one of said plurality of blocks and producing an indication of said detected disorder;

a quantizing circuit receiving said indication of said detected disorder and for quantizing said residual data and outputting quantized data produced therefrom; and

a buffer circuit for storing said quantized data and outputting said stored quantized data as transmission data,

wherein said quantizing circuit changes a quantizing step size greatly when said indication of said detected disorder of said one of said plurality of motion vectors for said respective one of said plurality of blocks is high.

9. The target object following apparatus according to claim 1 or 4, wherein said tracking vector detection circuit stores into tables said detected motion vectors of said respective ones of said blocks in said tracking vector detection area, said detected motion vectors having respective moving directions and moving quantities,

and determines said tracking vector with said moving direction and said moving quantity using a vector having a largest count in said distribution of said detected motion vectors of said respective ones of said blocks in said tracking vector detection area stored in said tables.

4. A target object following apparatus comprising:

a spatial edge detection circuit for outputting a spatial edge detection result for a motion picture comprising a plurality of sequentially input frames;

a time edge detection circuit for outputting a time edge detection result for said motion picture; and

an edge judging means for outputting a motion edge detection result, on a basis of said spatial edge detection result and said time edge detection result;

wherein said spatial edge detection circuit comprises a spatial power detection circuit for outputting a spatial power detection result detecting powers of high frequency components for a plurality of predetermined blocks of said frames; and

a first comparison circuit for obtaining a first comparison result comparing said spatial power detection result with a first comparison reference, and for outputting said spatial edge detection result when said spatial power detection result is larger than said first comparison reference; and

wherein said time edge detection circuit comprises a time power detection circuit for obtaining a difference data between a present frame and a previous frame of said motion picture, and for detecting a power of said difference data to output a time power detection result for each of said predetermined blocks; and

a second comparison circuit for obtaining a second comparison result comparing said time power detection result with a second comparison reference, and then, when said time power detection result is larger than said second comparison reference, for outputting said time edge detection result;

wherein said edge judging means judges an edge as having a motion when both of said spatial edge detection result and said time edge detection result are obtained, and then for outputting said motion edge detection result.

5. The target object following apparatus according to claim 4, further comprising:

a picture data blocking circuit for dividing said sequentially input frames into said plurality of predetermined blocks and another plurality of predetermined blocks of a predetermined number of pixels in a horizontal direction and a vertical direction;

a detection area setting circuit for setting a tracking vector detection area including a predetermined number of said another plurality of predetermined blocks in the horizontal and the vertical directions;

a motion detection circuit for detecting a motion vector for respective ones of said predetermined number of said another plurality of predetermined blocks within said tracking vector detection area; and

a tracking vector detection circuit for detecting a distribution of said motion vectors within said tracking vector detection area on the basis of the detection results of said spatial edge detection circuit in said target object following apparatus, and then detecting a moving direction and a moving quantity of said picture within said tracking vector detection area on the basis of said distribution detection result.

6. The target object following apparatus according to claim 5, further comprising:

an imaging means for imaging the target object, and for outputting said plurality of sequentially input frames into said target object following apparatus;

an imaging area changing means for changing an imaging area of said imaging means on the basis of a predetermined control signal; and

a detection area correction circuit for tracking the movement of said picture within said tracking vector detection area on the basis of the detection result of said tracking vector detection circuit within said target object following apparatus.

7. The target object following apparatus according to claim 5 or 6, wherein said motion detection circuit includes means for computing said plurality of motion vectors by performing a block matching method.

8. The target object following apparatus according to claim 4, 5 or 6 further comprising:

a second motion detection circuit for detecting a plurality of motion vectors of respective ones of a plurality of blocks forming one of said sequentially input frames;

a picture data processing circuit for motion compensating said plurality of blocks forming said one of said sequentially input frames on the basis of said respective plurality of motion vectors and outputting residual data produced therefrom;

a motion field detection circuit for detecting a disorder of each one of said plurality of motion vectors for a respective one of said plurality of blocks relative to motion vectors of respective surrounding blocks adjacent to said respective one of said plurality of blocks and providing an indication of said detected disorder;

a quantizing circuit responsive to said indication of said detected disorder and for quantizing said residual data and outputting quantized data produced therefrom; and

a buffer circuit for storing said quantized data and outputting said stored quantized data as transmission data,

wherein said quantizing circuit changes a quantizing step size greatly when said indication of said detected disorder of said one of said plurality of motion vectors for said respective one of said plurality of blocks is high.

10. A picture conversion apparatus comprising:

a motion detection circuit for detecting a plurality of motion vectors for a plurality of respective blocks forming picture data, for calculating a sum of absolute values for each block of picture data, and for generating a selection signal for the block of picture data according to a comparison of said corresponding sum of absolute values and corresponding motion vector of the block of picture data;

a picture data processing circuit for motion compensating said plurality of blocks forming picture data on the basis of said plurality of motion vectors and outputting residual data produced therefrom for each motion vector;

a selection circuit for switching between each block of picture data and said corresponding residual data for the block according to said selection signal and supplying a pre-quantized video signal therefrom;

a motion field detection circuit for detecting a disorder of each one of said plurality of motion vectors relative to motion vectors of respective surrounding blocks adjacent to a block corresponding to the motion vector for which said disorder is detected, and producing an indication of said detected disorder;

a quantizing circuit receiving said indication of said detected disorder and for quantizing said pre-quantized video signal and outputting quantized data produced therefrom, said quantizing circuit changing a quantizing step size according to said indication of said detected disorder of the motion vector; and

a buffer circuit for storing said quantized data and outputting said stored quantized data as transmission data.

11. The picture conversion apparatus according to claim 10, wherein said motion field detection circuit calculates said disorder of the motion vector by executing an outer-product of the motion vector and each one of said motion vectors of said respective adjacent surrounding blocks, and then executing an absolute summation of outer-product results corresponding to said respective adjacent surrounding blocks,

wherein the motion vector has a magnitude of A and the one of said motion vectors of said respective adjacent surrounding blocks has a magnitude of B, and

the magnitude of the outer product equals A times B times a sine of an angle between the motion vector and said one of said motion vectors of said respective adjacent surrounding blocks.