Visual Conditioning for Augmented-Reality-Assisted Video Conferencing

• US 20130265382A1
• Filed: 12/31/2012
• Published: 10/10/2013
• Est. Priority Date: 04/09/2012
• Status: Active Grant

• Current Assignee: Futurewei Technologies Incorporated (Huawei Investment & Holding Co., Ltd.)
• Original Assignee: Futurewei Technologies Incorporated (Huawei Investment & Holding Co., Ltd.)
• Inventors: Guleryuz, Onur G., Kalker, Antonius
• Granted Patent: US 9,154,732 B2
• Time in Patent Office: Days
• US Class Current: 348/14.08
• CPC Class Codes: G06T 11/60 Editing figures and text; Combining figures or text
• CPC Class Codes: H04N 7/15 Conference systems

Abstract

An apparatus comprising a processor configured to receive a video teleconferencing image, wherein the video teleconferencing image has a background and at least one foreground, wherein the foreground has a boundary, determine the boundary of the foreground, overlay the boundary of the foreground with a drawing area, wherein the drawing area obscures the boundary, and replace the background of the image with a new background.

Claims

1. An apparatus comprising:

a processor configured to;

receive a video teleconferencing image, wherein the video teleconferencing image has a background and at least one foreground, wherein the foreground has a boundary;

determine the boundary of the foreground;

overlay the boundary of the foreground with a drawing area, wherein the drawing area obscures the boundary; and

replace the background of the image with a new background.

2. The apparatus of claim 1, wherein the processor is further configured to determine the drawing area, and wherein determining the drawing area comprises:

establishing a contour following flow field;

estimating one or more error spreads associated with a contour following flow field; and

deriving the drawing area utilizing the contour following flow field and the one or more error spreads.

3. The apparatus of claim 1, wherein the drawing area contains one or more textures, and wherein the textures of the texturized drawing area are selected from a group consisting of:

random textures, semi-random textures, patterned textures, and pattern fields selected to match the local structure of the foreground.

4. The apparatus of claim 1, wherein the processor is further configured to select the new background from a plurality of new backgrounds, and wherein the selection includes harmonizing the color of the new background with the color of the foreground.

5. The apparatus of claim 1, wherein the processor is further configured to:

receive a second video teleconferencing image, wherein the second video teleconferencing image has a background and a foreground, wherein the foreground has a boundary;

determine the boundary of the foreground of the second video teleconferencing image;

overlay the boundary of the foreground of the second video teleconferencing image with a drawing area, wherein the drawing area of the second video teleconferencing image obscures the boundary of the foreground of the second video teleconferencing image; and

replace the background of the second video teleconferencing image with a new background, wherein the new background of the second video teleconferencing image is distinct from the new background of the video teleconferencing image in color, texture, or both.

6. The apparatus of claim 1, wherein the processor is further configured to select the new background from a plurality of new backgrounds, and wherein the selection includes selecting a texture to facilitate a smooth visual transition from the foreground to the new background.

7. The apparatus of claim 1, wherein the processor is further configured to:

establish a contour following flow field, wherein the contour following flow field is a linear fit to the boundary;

estimate segmentation error spreads in the direction of the contour following flow field and in the direction orthogonal to the contour following flow field;

obtain an uncertainty kernel from a look-up-table using the segmentation error spreads; and

derive the drawing region from the uncertainty kernel, the boundary, and the contour following flow field.

8. A computer program product comprising computer executable instructions stored on a non-transitory medium that when executed by a processor cause the processor to perform the following:

receive a video conferencing image, wherein the image has a foreground and a background;

select at least one dominant foreground color;

identify a new background, wherein the new background has at least one dominant background color matching the dominant foreground color; and

replace the background with the new background.

9. The computer program product of claim 8, wherein selecting the dominant foreground color is accomplished by passing at least a portion of the image through a first and second low pass filter, and wherein the first low pass filter has a lower pass band frequency than the second low pass filter.

10. The computer program product of claim 8, wherein the instructions further cause the processor to modulate the new background over time.

11. The computer program product of claim 10, wherein the modulation is accomplished using base background frames formed from pre-existing video.

12. The computer program product of claim 8, wherein identifying a new background further includes selecting a texture for the new background, and wherein the texture facilitates the visual transition from the foreground to the new background.

13. The computer program product of claim 8, wherein the foreground has two dominant foreground colors, wherein the new background has two dominant background colors, and wherein the first dominant background color matches the first dominant foreground color and the second dominant background color matches the second dominant foreground color.

14. A method for visual conditioning for augmented-reality-assisted video conferencing, comprising:

separating a video conferencing image into a foreground and a first background, wherein separation results in a foreground having at least one segmentation;

deriving a drawing area around the segmentation;

designing one or more textures to be rendered in the drawing area;

selecting a second background; and

combining the foreground, the one or more textures, the drawing area, and the second background to form a composite image.

15. The method of claim 14, wherein deriving the drawing area comprises:

determining pixels that are at the boundary of the foreground;

fixing a neighborhood around each boundary pixel to create an approximate boundary;

obtaining a contour following flow field by fitting the approximate boundary to a smoothing function;

estimating segmentation error spreads;

obtaining an uncertainty kernel; and

obtaining the drawing area using the contour following flow field and the uncertainty kernel.

16. The method of claim 14, further comprising:

identifying at least one dominant foreground color; and

selecting a background color for the second background, wherein the background color harmonizes with the foreground color.

17. The method of claim 14, wherein selecting comprises:

averaging the background pixel colors in the first background to obtain an average first background color; and

determining the color of the second background based on the average first background color, wherein the color of the second background is analogous or split-analogous in color scheme to the average first background color.

18. The method of claim 14, further comprising selecting a texture to facilitate a smooth visual transition from the foreground to the new background.

19. The method of claim 14, wherein selecting the second background comprises:

forming a plurality of base background frames using a pre-existing video; and

modulating the plurality of base background frames to form time varying backgrounds.

20. The method of claim 14, wherein designing one or more textures comprises approximating the local structure of the foreground image using textures designed using one or more techniques selected from a group consisting of:

random field generation, semi-random field generation, patterned field generation, auto-regressive moving average processes, and texture-resampling processes.