Method and apparatus for segmenting object in image

• US 10,235,761 B2
• Filed: 09/19/2016
• Issued: 03/19/2019
• Est. Priority Date: 08/27/2013
• Status: Active Grant

• Current Assignee: Samsung Electronics Co. Ltd.
• Original Assignee: Samsung Electronics Co. Ltd.
• Inventors: Jumpasut, Arin, Lee, Seong-oh, Jeong, Moon-sik, Choi, Sung-do
• Primary Examiner(s): Carter, Aaron W
• Application Number:
  US15/269,388
• Publication Number:
  US 20170004628A1
• Time in Patent Office: 911 Days
• Field of Search: 382173, 382171, 382190, 382199, 382202, 382284, 345 6, 345419, 348 42, 358 19
• CPC Class Codes: G06T 2207/20072 Graph-based image processing
• CPC Class Codes: G06T 2207/20092 Interactive image processing based on input by user
• CPC Class Codes: G06T 2207/20101 Interactive definition of point of interest, landmark or seed
• CPC Class Codes: G06T 2207/20112 Image segmentation details
• CPC Class Codes: G06T 7/11 Region-based segmentation
• CPC Class Codes: G06T 7/162 involving graph-based methods
• CPC Class Codes: G06T 7/194 involving foreground-background segmentation

Abstract

An object in a first image is segmented by obtaining the first image including the object; receiving a first input signal including first information about a first position in the first image; selecting at least one pixel included in the first image, based on the first information about the first position; generating a second image by dividing the first image into several areas, using the selected at least one pixel; and segmenting the object in the first image by using the first image and the second image.

Claims

1. A method of segmenting an object in a first image, the method comprising:

obtaining the first image including the object;

receiving a first input signal including first information about a first position in the first image;

selecting at least one pixel included in the first image, based on the first information about the first position;

generating, using the selected at least one pixel, a second image by dividing the first image into several areas including a first area as a foreground area, a second area as a possible foreground area, a third area as a background area, a fourth area as a possible background area, and a fifth area as an unknown area;

regenerating the second image based on a user input by refining each of the several areas; and

segmenting the object in the first image by using the first image and the second image.

2. The method of claim 1, wherein segmenting the object comprises segmenting the object in the first image based on color information of pixels included in the first image and information of several areas included in the second image.

3. The method of claim 2, further comprising:

obtaining information about the segmented object and information about the several areas that are updated using the information about the segmented object; and

updating the information about the segmented object based on color information and the information about the several areas that are updated using the information about the segmented object,

wherein updating the information about the segmented object is performed a predetermined number of times.

4. The method of claim 2, wherein segmenting the object in the first image based on the color information comprises:

generating a foreground model and a background model based on the color information and the information about the several areas;

constructing a graph of an energy function of the pixels included in the first image by combining a data term indicating a similarity between the pixels included in the first image with a smoothness term indicating a similarity between adjacent pixels; and

segmenting the object in the first image by applying a graph cut algorithm to the constructed graph.

5. The method of claim 3, wherein updating the information about the segmented object comprises:

updating the several areas included in the second image by using the information about the segmented object;

updating a foreground model and a background model that are previously generated, based on the color information and the updated information about the several areas;

reconstructing a graph of an energy function of pixels included in the first image by combining a data term indicating a similarity between the pixels included in the first image and a smoothness term indicating a similarity between adjacent pixels; and

updating the information about the segmented object by applying a graph cut algorithm to the reconstructed graph.

6. The method of claim 2, further comprising:

receiving a second input signal including second information about a position in the foreground area from the several areas; and

obtaining shape prior information based on the second input signal,

wherein segmenting the object in the first image further comprises adding the shape prior information.

7. The method of claim 6, wherein obtaining the shape prior information comprises:

selecting at least one pixel included in the foreground area based on the second information;

calculating visibilities of pixels indicating the object based on an initially selected pixel from among the selected at least one pixel; and

obtaining the shape prior information using the visibilities.

8. The method of claim 6, further comprising:

obtaining information about the segmented object and information about the several areas that are updated using the information about the segmented object; and

updating the information about the segmented object based on color information, the shape prior information, and the information about the several areas that are updated using the information about the segmented object.

9. The method of claim 1, further comprising outputting an image in which an area indicating the segmented object and an area indicating an area not including the segmented object are distinguished from each other.

10. The method of claim 1, further comprising:

applying a predetermined effect to pixels indicating an area not including the segmented object; and

outputting an image including the area indicating the segmented object to which the predetermined effect is applied.

11. The method of claim 10, wherein the predetermined effect includes at least one of a background blur, a motion blur, an edge filter, a background swapping, and a black and white background.

12. The method of claim 1, wherein the first input signal comprises at least one of a closed loop signal, a stroke signal, and a signal that designates the first position in the first image.

13. The method of claim 1, wherein receiving the first input signal comprises receiving a user input being drawn as a closed loop at the first position in the first image, and

wherein selecting the at least one pixel comprises selecting pixels corresponding to a track of the closed loop.

14. The method of claim 1, wherein receiving the first input signal comprises receiving a user input drawn as a stroke at the first position in the first image, and

wherein selecting the at least one pixel comprises selecting pixels corresponding to a track of the stroke.

15. The method of claim 1, wherein receiving the first input signal comprises receiving a user input through which the first position in the first image is designated, and

wherein selecting the at least one pixel comprises selecting a pixel corresponding to the designated position.

16. The method of claim 1, wherein generating the second image comprises:

generating the second image by using pixels selected in the first image based on the first input signal; and

correcting the second image by using the selected pixels included in the first image,

wherein the first input signal is input by a user and includes at least one of a closed loop signal, a stroke signal, and a signal that designates the first position in the first image.

17. The method of claim 16, wherein correcting the second image comprises repeatedly correcting the second image a number of times corresponding to a number of times that the first input signal is received.

18. A non-transitory computer readable recording medium having embodied thereon a program for executing a method comprising the steps of:

obtaining a first image including the object;

receiving an input signal including information about a position in the first image;

selecting at least one pixel included in the first image, based on the information about the position;

generating, using the selected at least one pixel, a second image by dividing the first image into several areas including a first area as a foreground area, a second area as a possible foreground area, a third area as a background area, a fourth area as a possible background area, and a fifth area as an unknown area;

regenerating the second image based on a user input by refining each of the several areas; and

segmenting the object in the first image by using the first image and the second image.

19. An apparatus for segmenting an object in a first image, the apparatus comprising:

an interface configured for obtaining the first image including the object and receiving an input signal including information about a position in the first image;

an image generator configured for selecting at least one pixel included in the first image, based on the information about the position, generating, using the selected at least one pixel, a second image by dividing the first image into several areas including a first area as a foreground area, a second area as a possible foreground area, a third area as a background area, a fourth area as a possible background area, and a fifth area as an unknown area, and regenerating the second image based on a user input by refining each of the several areas; and

a segmentor configured for segmenting the object in the first image by using the first image and the second image.