APPARATUS AND METHOD FOR DETECTING FOREGROUND IN IMAGE

• US 20170213100A1
• Filed: 01/13/2017
• Published: 07/27/2017
• Est. Priority Date: 01/22/2016
• Status: Active Grant

• Current Assignee: Samsung Electronics Co. Ltd., Seoul National University R&DB Foundation (Seoul National University)
• Original Assignee: Samsung Electronics Co. Ltd., Seoul National University R&DB Foundation (Seoul National University)
• Inventors: YUN, Ki Min, LIM, Jong In, CHOI, Jin Young
• Granted Patent: US 10,311,579 B2
• Time in Patent Office: Days
• CPC Class Codes: G06T 2207/10016 Video; Image sequence
• CPC Class Codes: G06T 2207/20081 Training; Learning
• CPC Class Codes: G06T 7/11 Region-based segmentation
• CPC Class Codes: G06T 7/174 involving the use of two or more images
• CPC Class Codes: G06T 7/194 involving foreground-background segmentation
• CPC Class Codes: G06T 7/215 Motion-based segmentation
• CPC Class Codes: G06V 10/26 Segmentation of patterns in the image field; Cutting or merging of image elements to establish the pattern region, e.g. clustering-based techniques; Detection of occlusion
• CPC Class Codes: G06V 10/28 Quantising the image, e.g. histogram thresholding for discrimination between background and foreground patterns

Abstract

An apparatus and method for detecting a foreground in an image is provided, and the foreground detecting apparatus includes a context information estimator configured to estimate context information on a scene from an image frame of the image, a background model constructor configured to construct a background model of the image frame using the estimated context information, and a foreground detector configured to detect a foreground from the image frame based on the constructed background model.

Claims

1. An apparatus for detecting a foreground in an image, the apparatus comprising:

a processor configured to;

estimate context information on a scene from an image frame of the image;

construct a background model of the image frame using the estimated context information; and

detect a foreground from the image frame based on the constructed background model.

2. The apparatus of claim 1, wherein the processor comprises:

a context information estimator configured to estimate the context information on the scene from the image frame of the image;

a background model constructor configured to construct the background model of the image frame using the estimated context information; and

a foreground detector configured to detect the foreground from the image frame based on the constructed background model.

3. The apparatus of claim 2, wherein the context information comprises any one or any combination of background motion, a foreground speed, and an illumination change.

4. The apparatus of claim 3, wherein the context information estimator comprises:

a background motion estimator configured to calculate a speed using an optical flow of the image frame and a previous image frame, and to generate a projective transform matrix that represents the background motion using the speed;

a foreground speed estimator configured to estimate a foreground speed by correcting the speed based on the generated projective transform matrix; and

an illumination change estimator configured to estimate the illumination change based on a difference between a mean of brightness intensities of the image frame and a mean of brightness intensities of the background model.

5. The apparatus of claim 3, wherein the background model constructor comprises a background model generator configured to generate the background model of the image frame using a background model of a previous image frame and the background motion.

6. The apparatus of claim 5, wherein the background model generator is further configured to determine locations on the previous image frame based on a location of the previous image frame which corresponds to a first location of the image frame, and to generate a background model of the first location by performing weighted summation on background models of the determined locations.

7. The apparatus of claim 6, wherein the background model generator is further configured to correct a variance of the generated background model, in response to a value of a context variable based on the foreground speed being equal to or larger than a block size.

8. The apparatus of claim 7, wherein the background model generator is further configured to correct a variance of the background model of the first location by reflecting differences between a mean of the background model of the first location and means of the background models at the locations.

9. The apparatus of claim 5, wherein the background model constructor further comprises a background model updater configured to update the generated background model based on the illumination change.

10. The apparatus of claim 9, wherein the background model updater is further configured to update a mean of the generated background model by reflecting the illumination change, in response to value of a context variable being smaller than a block size, and to update the mean and a variance of the generated background model by reflecting the illumination change and additional information, in response to the value of the context variable being equal to or larger than the block size.

11. The apparatus of claim 10, wherein the additional information comprises any one or any combination of a mean of block-specific brightness intensities and information on a difference between a mean of the generated background model and the image frame.

12. The apparatus of claim 10, wherein the background model updater calculates a value of a time-variant variable in units of blocks starting from an initial image frame input from the camera to the image frame, and adjusts an update intensity of the generated background model using the value of the time-variant variable.

13. The apparatus of claim 2, wherein the foreground detector comprises:

a foreground probability map generator configured to generate a foreground probability map by calculating a foreground probability for each pixel of the image frame based on the constructed background model; and

a foreground extractor configured to extract the foreground from the image frame using the generated foreground probability map.

14. The apparatus of claim 13, wherein the foreground extractor is further configured to classify the each pixel of the image frame into at least one of the foreground, a candidate, and a background based on a comparison of the foreground probability of the each pixel of the image frame with a threshold value, and to categorize pixels classified into the candidate into the foreground or the background based on applying watershed segmentation to the pixels classified into the candidate.

15. The apparatus of claim 1, further comprising:

a memory configured to store instructions; and

wherein the processor is further configured to execute the instructions to estimate context information on a scene from an image frame of the image, to construct a background model of the image frame using the estimated context information, and to detect a foreground from the image frame based on the constructed background model.

16. The apparatus of claim 4, wherein the previous image frame comprises the immediately previous image frame.

17. A method of detecting a foreground in an image, the method comprising:

estimating context information on a scene from an image frame of the input image;

constructing a background model of the image frame using the estimated context information; and

detecting a foreground from the image frame based on the constructed background model.

18. The method of claim 17, wherein the context information comprises any one or any combination of background motion, a foreground speed, and an illumination change.

19. The method of claim 18, wherein the constructing of the background model comprises generating the background model of the image frame using a background model of a previous image frame based on information on the background motion.

20. The method of claim 19, wherein the constructing of the background model further comprises correcting a variance of the generated background model in response to a value of a context variable based on the foreground speed being equal to or larger than a block size.

21. The method of claim 19, wherein the constructing of the background model further comprises updating the generated background model based on the illumination change.

22. The method of claim 21, wherein the updating of the background model comprises, updating a mean of the generated background model by reflecting the illumination change, in response to a value of a context variable being smaller than a block size, and updating a mean and a variance of the generated background model by reflecting the illumination change and additional information, in response to the value of the context variable being equal to or larger than the block size.

23. The method of claim 17, wherein the detecting of a foreground comprises:

generating a foreground probability map by calculating a foreground probability for each pixel of the image frame based on the constructed background model; and

extracting the foreground from the image frame using the generated foreground probability map.

24. A non-transitory computer-readable storage medium storing instructions that, when executed by a processor, cause the processor to perform the method of claim 17.

25. A digital device comprising:

an antenna;

a cellular radio configured to transmit and receive data via the antenna according to a cellular communications standard;

a touch-sensitive display;

a camera configured to capture an image;

a memory configured to store instructions; and

a processor is configured to execute the instructions to receive the image from the camera, estimate context information on a scene from an image frame of the image, construct a background model of the image frame using the estimated context information, and detect a foreground from the image frame based on the constructed background model.