Image processing method, a program of an image processing method, a recording medium on which a program of an image processing method is recorded and an image processing circuit

• US 8,243,807 B2
• Filed: 08/22/2007
• Issued: 08/14/2012
• Est. Priority Date: 08/22/2006
• Status: Expired due to Fees

• Current Assignee: Sony Corporation (Sony Group Corp.)
• Original Assignee: Sony Corporation (Sony Group Corp.)
• Inventors: Yokoyama, Kazuki, Asano, Mitsuyasu, Ueda, Kazuhiko
• Primary Examiner(s): Dastouri, Mehrdad
• Assistant Examiner(s): ANDERSON II, JAMES M
• Application Number:
  US11/843,028
• Publication Number:
  US 20080049145A1
• Time in Patent Office: 1,819 Days
• Field of Search: None
• US Class Current: 375/240.16
• CPC Class Codes: H04N 23/695 Control of camera direction for changing a field of view, e.g. pan, tilt or based on tracking of objects

Abstract

An image processing method in which a video signal is processed to detect a conspicuous area which is thought that a camera person intends to shoot, the method includes the steps of: detecting a motion vector in each portion of the video signal; detecting a representative value that represents a motion in two places in an outer part of a screen; detecting a motion component caused by zooming operations from the representative value; detecting a conspicuous degree by zooming in which a motion component caused by zooming operations is detected in each portion of the video signal based on the motion component caused by zooming operations; correcting a motion vector; detecting a motion component caused by making a pan and tilt from the representative value; detecting a conspicuous degree caused by making a pan and tilt; and detecting a total conspicuous degree.

Claims

1. A computer-implemented method, comprising:

identifying motion vectors associated with a plurality of portions of a video signal;

identifying, for the video signal, at least two boundary portions that correspond to an outer part of a screen, the boundary portions facing each other and being disposed such that a center of the screen is between the boundary portions;

generating, using a processor, a value representative of at least one of a zooming motion or a pan and tilt motion within the boundary portions, the representative value being based on at least the detected motion vectors associated with corresponding ones of the boundary portions, wherein generating the representative value comprises;

generating a histogram of the motion vectors of the boundary portions;

identifying a motion vector within the histogram having a largest distribution; and

establishing the identified motion vector as the representative value;

obtaining, based on the representative value, a first motion component caused by zooming operations within the boundary portions;

computing, based on the at first motion component, a first conspicuous degree associated with zooming operations;

generating, based on the first motion component, modified motion vectors for the video signal portions, the modified motion vectors correcting for zooming operations within the boundary portions;

obtaining, based on the representative value, a second motion component caused by pan and tilt operations within the boundary portions;

computing, based on the second motion component and the modified motion vectors, a second conspicuous degree associated with pan and tilt operations; and

determining a total conspicuous degree for the video signal potions based on a sum of the first and second conspicuous degrees, the total conspicuous degree being indicative of a probability that a conspicuous area is disposed within corresponding ones of the video signal portions.

2. The method of claim 1, wherein the first motion component corresponds to an antiphase component of the representative value.

3. The method of claim 1, wherein the second motion component corresponds to an in-phase component of the representative value.

4. The method of claim 1, wherein computing the first conspicuous degree comprises interpolating the first motion component across variable screen positions within the video signal portions.

5. The method of claim 1, further comprising applying a smoothing process to the first and second conspicuous degrees to determine the total conspicuous degree.

9. The method of claim 1, wherein determining the modified motion vectors comprises subtracting the first motion component from corresponding ones of the identified motion vectors.

10. The method of claim 1, wherein computing the second conspicuous degree comprises subtracting the second motion component from corresponding ones of the modified motion vectors.

6. A tangible, non-transitory computer-readable medium storing instructions that, when executed by at least one processor, cause the processor to perform a method, comprising:

identifying motion vectors associated with a plurality of portions of a video signal;

identifying, for the video signal, at least two boundary portions that correspond to an outer part of a screen, the boundary portions facing each other and being disposed such that a center of the screen is between the boundary portions;

generating a value representative of at least one of a zooming motion or a pan and tilt motion within the boundary portions, the representative value being based on at least the detected motion vectors associated with corresponding ones of the boundary portions, wherein generating the representative value comprises;

generating a histogram of the motion vectors of the boundary portions;

identifying a motion vector within the histogram having a largest distribution; and

establishing the identified motion vector as the representative value;

obtaining, based on the representative value, a first motion component caused by zooming operations within the boundary portions;

computing, based on the first motion component, a first conspicuous degree associated with zooming operations;

generating, based on the first motion component, modified motion vectors for the video signal portions, the modified motion vectors correcting for zooming operations within the boundary portions;

obtaining, based on the representative value, a second motion component caused by pan and tilt operations within the boundary portions;

computing, based on the second motion component and the modified motion vectors, a second conspicuous degree associated with pan and tilt operations; and

determining a total conspicuous degree for the video signal potions based on a sum of the first and second conspicuous degrees, the total conspicuous degree being indicative of a probability that a conspicuous area is disposed within corresponding ones of the video signal portions.

7. An image processing apparatus, comprising:

a motion vector detecting part configured to;

identify motion vectors associated with a plurality of portions of a video signal; and

identify, for the video signal, at least two boundary portions that correspond to an outer part of a screen, boundary portions facing each other and being disposed such that a center of the screen is between the boundary portions;

a representative value detecting part configured to generate a value representative of at least one of a zooming motion or a pan and tilt motion within the boundary portions, the representative value being based on at least the detected motion vectors associated with corresponding ones of the boundary portions, wherein the representative value detecting part is further configured to;

generate a histogram of the motion vectors of the boundary portions;

identify a motion vector within the histogram having a largest distribution; and

establish the identified motion vector as the representative Value;

a first motion component detecting part configured to obtain, based on the representative value, a first motion component caused by zooming operations within the boundary portions;

a first conspicuous degree detecting configured to compute, based on the first motion component, a first conspicuous degree associated with zooming operations;

a motion vector correcting part configured to generate, based on the first motion component, modified motion vectors for the video signal portions, the modified motion vectors correcting for zooming operations within the boundary portions;

a second motion component detecting part configured to obtain, based on the representative value, a second motion component caused by pan and tilt operations within the boundary portions;

a second conspicuous degree detecting part configured to compute, based on the second motion component and the modified motion vectors, a second conspicuous degree associated with pan and tilt operations; and

a total conspicuous degree detecting part configured to determine a total conspicuous degree for the video signal potions based on a sum of the first and second conspicuous degrees the total conspicuous degree being indicative of a probability that a conspicuous area is disposed within corresponding ones of the video signal portions.

8. An apparatus, comprising:

a storage device; and

at least one processor coupled to the storage device, wherein the storage device stores a program for controlling the at least one processor, and wherein the at least one processor, being operative with the program, is configured to;

identify motion vectors associated with a plurality of portions of a video signal;

identify, for the video signal, at least two boundary portions that correspond to an outer part of a screen;

generate a value representative of at least one of a zooming motion or a pan and tilt motion within the boundary portions, the representative value being based on at least the identified motion vectors associated with corresponding ones of the boundary portions, wherein the processor is further configured to;

generate a histogram of the motion vectors of the boundary portions;

identify a motion vector within the histogram having a largest distribution; and

establish the identified motion vector as the representative value;

obtain, based on the representative value, at least one first motion component caused by zooming operations within the boundary portions, and at least one second motion component caused by pan and tilt operations within the boundary portions;

compute, based on the first and second motion components, a first conspicuous degree associated with zooming operations and a second conspicuous degree associated with pan and tilt operations; and

determine a total conspicuous degree for the video signal potions based on a sum of the first and second conspicuous degrees, the total conspicuous degree being indicative of a probability that a conspicuous area is disposed within corresponding ones of the video signal portions.