Image analysis method, medium and apparatus and moving image segmentation system

• US 20080095436A1
• Filed: 09/14/2007
• Published: 04/24/2008
• Est. Priority Date: 10/18/2006
• Status: Active Grant

• Current Assignee: Samsung Electronics Co. Ltd.
• Original Assignee: Samsung Electronics Co. Ltd.
• Inventors: Kim, Yoon-kyung, Moon, Young-su, Kim, Jun-mo, Yoo, Yong-seok
• Granted Patent: US 8,285,045 B2
• Time in Patent Office: Days
• US Class Current: 382/173
• CPC Class Codes: G06T 2207/10016 Video; Image sequence
• CPC Class Codes: G06T 7/215 Motion-based segmentation
• CPC Class Codes: G06T 7/579 from motion
• CPC Class Codes: G06V 10/28 Quantising the image, e.g. histogram thresholding for discrimination between background and foreground patterns
• CPC Class Codes: H04N 19/137 Motion inside a coding unit, e.g. average field, frame or block difference
• CPC Class Codes: H04N 19/142 Detection of scene cut or scene change
• CPC Class Codes: H04N 19/186 the unit being a colour or a chrominance component
• CPC Class Codes: H04N 19/20 using video object coding

Abstract

An image analysis method, medium and apparatus for segmentation of a moving image, and a moving image segmentation system. The image analysis method includes receiving an image signal representing an image, detecting features of the image by calculating a difference between the current frame of the image signal and its previous frame, analyzing the image signal based on the detected features of the image, and performing segmentation on the image signal according to the analysis result, thereby separately performing segmentation on all types of moving images. In other words, by using an appropriate segmentation method according to a feature of an image, effective segmentation can be achieved.

Claims

1. An image analysis method for an image signal representing an image, the method comprising:

detecting a feature of the image;

determining a segmentation algorithm based on the detected feature of the image; and

segmenting the image signal using the determined segmentation algorithm.

2. The image analysis method of claim 1, wherein the feature of the image includes at least one of motion of the image, motion of a camera taking the image, and whether a foreground object that moves independently in the image exists.

3. The image analysis method of claim 1, further comprising receiving an image signal representing an image, and wherein the detecting comprises:

calculating a difference between a current frame of the image signal and a previous frame of the image signal;

detecting motion of the image based on the calculation result; and

detecting motion of a camera taking the image based on the detected motion of the image.

4. The image analysis method of claim 3, further comprising detecting a foreground object that moves independently in the image based on the detected motion of the camera taking the image, wherein the detection of the foreground object is performed by determining a quantity of outliers whose motion is not consistent with the motion of the camera.

5. The image analysis method of claim 4, further comprising detecting a peak value of a depth distribution by analyzing the depth distribution of the image based on a detection result of the detecting of the foreground object, wherein if the number of outliers is less than a predetermined number, the detecting of the peak value of the depth distribution is performed.

6. The image analysis method of claim 4, further comprising determining whether a number of peak values is greater than 2.

7. The image analysis method of claim 4, wherein the detecting of the foreground object comprises:

tracking feature points in the image;

calculating motion of the camera from the tracked feature points;

detecting, among the feature points, outliers whose motion is not consistent with the calculated motion of the camera; and

determining whether a quantity of the detected outliers is greater than a predetermined number.

8. The image analysis method of claim 5, wherein the detecting of the peak value of the depth distribution comprises:

estimating depths of the image signal;

analyzing a distribution of the estimated depths; and

determining a quantity of peak values of the analyzed depth distribution.

9. The image analysis method of claim 1, further comprising detecting a color change in the image, wherein image analysis with respect to a current image signal is stopped and image analysis with respect to a next image signal is performed upon the detected color change being higher than a predetermined level.

13. At least one medium comprising computer readable code to control at least one processing element to implement the method of any one of claims 1, 10 and 12.

10. An image analysis method comprising:

receiving an image signal representing an image and calculating a difference between two consecutive frames of the image signal;

detecting motion of a camera taking the image based on the calculated difference;

tracking feature points of the image and comparing the feature points with the motion of the camera in order to determine a quantity of outliers that are not associated with the motion of the camera;

analyzing the image signal based on the motion of the camera and the number of outliers; and

performing segmentation with respect to the image signal according to the analysis result.

11. The image analysis method of claim 10, wherein if the determined quantity of outliers is greater than a predetermined number the tracking and comparing comprises estimating depths of the image signal, analyzing a distribution of the estimated depths, and determining a quantity of peak values of the analyzed depth distribution, and the analyzing the image signal comprises analyzing the image signal based on the number of peak values.

12. An image analysis method for an image signal provided by a moving image recording device, the method comprising:

detecting features of the image signal including whether at least one of the image and a camera taking the image have motion; and

segmenting the image signal into a foreground region and a background region using a predetermined segmentation algorithm selected according to the detected features of the image signal.

14. An image analysis apparatus for an image signal representing an image, the apparatus comprising:

a feature detection unit to detect a feature of the image;

an image analysis unit to determine a segmentation algorithm based on the detected feature of the image; and

an image segmentation unit to segment the image signal using the segmentation algorithm determined by the image analysis unit.

15. The image analysis apparatus of claim 14, wherein the image segmentation unit segments the image signal into a foreground region and a background region using the segmentation algorithm.

16. The image analysis apparatus of claim 14, wherein the feature detection unit comprises:

receiving an image signal representing the image;

a motion detection unit to detect motion of the image and motion of a camera taking the image; and

a foreground object detection unit to detect a foreground object that moves independently in the image,

wherein the foreground object detection unit determines a quantity of outliers of the image whose motion is not consistent with the motion of the camera and detects the foreground object based on the quantity of outliers, upon the motion of the image and the motion of the camera being detected by the motion detection unit.

17. The image analysis apparatus of claim 16, wherein the feature detection unit further comprises a depth detection unit to detect a peak value of a depth distribution based on depths of the image, wherein the depth detection unit detects the depths of the image signal upon the quantity of outliers detected by the foreground object detection unit being less than a predetermined number.

18. The image analysis apparatus of claim 17, wherein the depth detection unit comprises:

a depth estimation unit to estimate the depths of the image signal;

a depth analysis unit to analyze a distribution of the estimated depths of the image signal; and

a peak value determination unit to determine whether a quantity of peak values of the depth distribution is greater than 1.

19. The image analysis apparatus of claim 16, wherein the motion detection unit comprises:

a frame calculation unit to receive the image signal and calculate a difference between a current frame of the image signal and a previous frame of the image signal;

an image motion detection unit to detect motion of the image based on the difference received from the frame calculation unit; and

a camera motion calculation unit to calculate motion of the camera taking the image based on a difference received from the frame calculation unit,

wherein if the motion of the image is detected by the image motion detection unit, the camera motion calculation unit calculates the motion of the camera.

20. The image analysis apparatus of claim 16, wherein the foreground object detection unit comprises:

a feature point extraction unit to extract feature points from the image signal;

a motion calculation unit to calculate the motion of the camera, detected by the motion detection unit, based on the feature points received from the feature point extraction unit; and

an outlier detection and determination unit to detect a quantity of outliers whose motion is not consistent with the calculated motion of the camera and to determine the quantity of outliers,

wherein the outlier detection and determination unit determines whether the number of outliers is greater than a predetermined number.

21. The image analysis apparatus of claim 16, wherein the image analysis unit comprises:

a detection result determination unit to determine the feature of the image based on at least one of the motion of the image and the motion of the camera detected by the motion detection unit, the foreground object received from the foreground object detection unit, and the peak value received from the depth detection unit; and

wherein the image segmentation unit includes an image segmentation apparatus selection unit receiving the determination result of the detection result determination unit and selecting a segmentation apparatus based on the determination result.

22. The image analysis apparatus of claim 16, wherein the feature detection unit comprises a scene change detection unit to detect a scene change in the image based on a color change in the image, wherein the scene change detection unit stops image mode classification for a current image signal and performs image mode classification for a next image signal upon the detected color change being higher than a predetermined level.

23. A moving image segmentation system to segment an image signal received from the image signal input apparatus into a foreground region and a background region, the system comprising:

an image signal input apparatus; and

an image analysis apparatus of claim 14.

24. A moving image segmentation system to segment an image signal input received the image signal input apparatus into a foreground region and a background region, the system comprising:

an image signal input apparatus; and

an image mode classifying apparatus to classify the image signal into an image mode suitable for segmentation of the image signal based on the feature of the image signal received from the image signal input apparatus and to output the image signal to a segmentation apparatus corresponding to the mode into which the image signal is classified.

25. A system analyzing an image signal provided by a moving image recording device, the system comprising:

a feature detection unit to detect features of the image signal including whether at least one of the image and a camera taking the image have motion; and

an image segmentation unit to segment the image signal into a foreground region and a background region using a predetermined segmentation algorithm selected according to the detected features of the image signal.