Methods and systems for real-time user extraction using deep learning networks
First Claim
1. A method comprising:
- obtaining a first frame of color pixel data;
checking whether a reset flag is cleared or set at a first time;
generating a trimap for the first frame, wherein;
if the reset flag is cleared at the first time, then generating the trimap for the first frame comprises;
obtaining a user-extraction contour that is based on an immediately preceding frame; and
generating the trimap for the first frame based on the obtained user-extraction contour;
if the reset flag is set at the first time, then generating the trimap for the first frame comprises;
detecting at least one persona feature in the first frame;
generating an alpha mask at least in part by aligning an intermediate persona contour with the detected at least one persona feature, wherein the intermediate persona contour is based on a result of a color-based flood-fill operation having been performed on a previous frame of color pixel data that had been segmented by a machine-learning-segmentation (MLS) process; and
generating the trimap for the first frame based on the generated alpha mask; and
outputting the generated trimap for use in extracting a user persona from the first frame.
3 Assignments
0 Petitions
Accused Products
Abstract
Methods and systems for real-time user extraction using deep learning networks. In one embodiment, user extraction comprises obtaining a given frame of color pixel data, checking whether a reset flag is cleared or set, and generating a trimap for the given frame. If the reset flag is cleared, generating the trimap comprises: obtaining a user-extraction contour based on a preceding frame; and generating the trimap based on the obtained user-extraction contour. If the reset flag is set, generating the trimap comprises: detecting at least one persona feature in the given frame; generating an alpha mask by aligning an intermediate contour with the detected persona feature(s), wherein the intermediate contour is based on a color-based flood-fill operation performed on a previous frame which was segmented by a machine-learning-segmentation process; and generating the trimap based on the generated alpha mask. The generated trimap is output for extracting a user persona.
226 Citations
22 Claims
-
1. A method comprising:
-
obtaining a first frame of color pixel data; checking whether a reset flag is cleared or set at a first time; generating a trimap for the first frame, wherein; if the reset flag is cleared at the first time, then generating the trimap for the first frame comprises; obtaining a user-extraction contour that is based on an immediately preceding frame; and generating the trimap for the first frame based on the obtained user-extraction contour; if the reset flag is set at the first time, then generating the trimap for the first frame comprises; detecting at least one persona feature in the first frame; generating an alpha mask at least in part by aligning an intermediate persona contour with the detected at least one persona feature, wherein the intermediate persona contour is based on a result of a color-based flood-fill operation having been performed on a previous frame of color pixel data that had been segmented by a machine-learning-segmentation (MLS) process; and generating the trimap for the first frame based on the generated alpha mask; and outputting the generated trimap for use in extracting a user persona from the first frame. - View Dependent Claims (2, 3, 4, 5, 6, 7, 8, 9, 10, 11)
-
-
12. A method comprising:
-
receiving a first segmented frame of color pixel data, the first segmented frame comprising an initial-segmentation persona contour that was identified by a machine-learning-segmentation (MLS) process; defining an eroded MLS persona contour and a dilated MLS persona contour, both based on the initial-segmentation persona contour; defining an intermediate persona contour at least in part by performing a bidirectional color-based flood-fill operation outward from the eroded MLS persona contour and inward from the dilated MLS persona contour; obtaining a second frame of color pixel data from an input source; detecting at least one second-frame persona feature in the second frame; generating a second-frame alpha mask at least in part by aligning the intermediate persona contour with the detected at least one second-frame persona feature; generating a second-frame trimap based on the generated second-frame alpha mask; and outputting the generated second-frame trimap for use in extracting a user persona from the second frame. - View Dependent Claims (13, 14, 15, 16, 17, 18, 19, 20, 21)
-
-
22. An apparatus comprising:
-
a communication interface; a processor; and non-transitory computer readable data storage containing instructions executable by the processor for causing the system to carry out a set of functions, the set of functions comprising; obtaining a first frame of color pixel data; checking whether a reset flag is cleared or set at a first time; generating a trimap for the first frame, wherein; if the reset flag is cleared at the first time, then generating the trimap for the first frame comprises; obtaining a user-extraction contour that is based on an immediately preceding frame; and generating the trimap for the first frame based on the obtained user-extraction contour; if the reset flag is set at the first time, then generating the trimap for the first frame comprises; detecting at least one persona feature in the first frame; generating an alpha mask at least in part by aligning an intermediate persona contour with the detected at least one persona feature, wherein the intermediate persona contour is based on a result of a color-based flood-fill operation having been performed on a previous frame of color pixel data that had been segmented by a machine-learning-segmentation (MLS) process; and generating the trimap for the first frame based on the generated alpha mask; and outputting the generated trimap for use in extracting a user persona from the first frame.
-
Specification