Video communication systems and methods

• US 8,294,823 B2
• Filed: 08/04/2006
• Issued: 10/23/2012
• Est. Priority Date: 08/04/2006
• Status: Active Grant

• Current Assignee: Apple Inc.
• Original Assignee: Apple Inc.
• Inventors: Ciudad, Jean-Pierre, Stochosky, Michael, Knight, Kurt, Forstall, Scott, Brunner, Ralph, Zhang, Chendi
• Primary Examiner(s): Fan, Chieh M
• Assistant Examiner(s): Trehan, Akshay
• Application Number:
  US11/462,608
• Publication Number:
  US 20080030621A1
• Time in Patent Office: 2,272 Days
• Field of Search: 348/14.01, 348/14.1, 348/14.08, 348/14.09, 348/220.1, 348/239, 348/586, 382/100, 382/154
• US Class Current: 348/586
• CPC Class Codes: H04N 7/147 Communication arrangements, e.g. identifying the communication as a video-communication, intermediate storage of the signals selecting H04Q

Abstract

An image of a background is stored and thereafter removed from a video stream. Substitute background video data is merged into the video stream and provided to videoconference participants.

Claims

1. A method, comprising:

storing by a computing system background video data of a videoconference participant environment in a data store, the background video data defining, for each of a plurality of pixels, color data values of the pixel and an alpha channel value of the pixel;

receiving by the computing system additional video data of the videoconference participant environment, the additional video data defining, for each of a plurality of pixels, color data values of the pixel and an alpha channel value of the pixel;

determining by the computing system whether a pixel of the additional video data is a match to a corresponding pixel of the background video data, wherein a match indicates that both color data values of a pixel of the additional video data and color data values of a corresponding pixel of the background video data are within a threshold range;

if a pixel of the additional video data is determined to be a match to a corresponding pixel of the background video data, then setting an alpha channel value for the pixel to a first value that designates the pixel of the additional video data as a transparent pixel;

if a pixel of the video data is determined not to be a match to a corresponding pixel of the background video data, then setting an alpha channel value for the pixel to a second value that designates the pixel of the additional video data as an opaque pixel;

generating by the computing system a first layer from the additional video data having pixels designated as transparent and pixels designated as opaque based on the respective alpha channel values, each of the pixels including the color data of the additional video data;

generating by the computing system a second layer from a substitute background video data item, the second layer including data defining, for each of a plurality of pixels, color data of the pixel;

merging by the computing system the first layer and the second layer to generate composite video data, the composite video data include each of the first layer and the second layer, the second layer being subordinate to the first layer so that only the pixels of the second layer that correspond to transparent pixels of the first layer are shown from the second layer when the composite video data is presented for display; and

transmitting by the computing system the composite video data to a videoconference for display to other video conference participants.

2. The method of claim 1, further comprising:

prompting a user to select from a plurality of substitute background video data items; and

receiving a selection for one of the substitute background video data items; and

wherein generating a second layer from a substitute background video data item comprises generating a second layer from the selected one of the substitute background video data items.

3. The method of claim 1, wherein:

generating by the computing system a second layer from substitute background video data item comprises generating second layers from substitute background video data items, wherein at least one of the second layers is generated from a first substitute background video data item that is different from another second background video data item that is used for generating an other second layer; and

merging by the computing system the first layer and the second layers to generate respective composite video data comprises merging the first layer and the second layers to generate respective composite video data for video conference participants, the respective composite video data for each video conference participant including the first layer and one of the second layers, the second layer being subordinate to the first layer so that only the pixels of the second layer that correspond to transparent pixels of the first layer are shown from the second layer when the composite video data is presented for display; and

transmitting by the computing system the composite video data to a videoconference for display to other video conference participants comprises transmitting to each videoconference participant the respective composite video data for display; and

further comprising;

receiving, for each of the video conference participants, a selection of a respective substitute background video data item from the plurality of background video data items;

associating each respective substitute background video data item with each of the videoconference participants for which it was selected; and

for each of the videoconference participants, generating the respective composite video data based on the respective substitute background video data item and the video data and transmitting the respective composite video data to the videoconference participant.

4. The method of claim 1, further comprising:

storing video data files;

storing image data files; and

prompting a user to select one of the stored video data files or one of the stored image data files as the substitute background video data item.

5. The method of claim 1, wherein:

storing background video data of a videoconference participant environment comprises;

prompting a videoconference participant to move out of the videoconference participant environment; and

storing image data of the videoconference participant environment without the participant present.

6. The method of claim 5, further comprising:

prompting the videoconference participant to indicate when the videoconference participant has moved out of the videoconference environment.

7. The method of claim 1, further comprising:

receiving selections of a plurality of substitute background video data items; and

periodically selecting one of the selected plurality of substitute background video data items for generation of the second layer.

8. The method of claim 1, further comprising:

receiving composite video data of another video conference participant environment;

providing for display to a video conference participant in the video conference participant environment the received composite video data of the another video conference participant environment in a first subsection of a display device; and

providing for display to the video conference participant the transmitted composite video data in a second subsection of a display device, the second subsection being subordinate to the first subsection.

9. The method of claim 1, wherein:

at least one of the substitute background video data items specify a slide show that includes a plurality of other video data items, and wherein the substitute background video data item that specifies the slide show causes each of the plurality of other video data items to be automatically selected periodically as the substitute background video data item for use in generating the second layer.

10. A system, comprising:

a video camera configured to generate video data;

a data store storing background data files;

a communication subsystem to receive and transmit video data; and

a video processing device configured to

capture background video data of a videoconference environment and receive additional video data of the video conference environment, wherein the background video data and the additional video data define, for each of a plurality of pixels, color data values of the pixel and an alpha channel value of the pixel;

determine whether a pixel of the additional video data is a match to a corresponding pixel of the background video data, wherein a match indicates that both color data values of a pixel of the additional video data and color data values of a corresponding pixel of the background video data are within a threshold range;

set an alpha channel value of a pixel of the additional video data to a minimum value to designate the pixel as transparent if the pixel of the additional video data is determined to be a match to a corresponding pixel of the background video data;

set an alpha channel value of a pixel of the additional video data to a maximum value to designate the pixel as opaque if the pixel of the additional video data is determined not to be a match to a corresponding pixel of the background video data;

generate a first layer from the additional video data having pixels designated as transparent and pixels designated as opaque based on the respective alpha channel values, each of the pixels including the color data of the additional video data;

generate a second layer from a background data file, the second layer including data defining, for each of a plurality of pixels, color data of the pixel; and

merge the first layer and the second layer to generate composite video data including each of the first and second layers so that pixels in the second layer are shown through corresponding pixels in the first layer according to the alpha channel values of the corresponding pixels, and transmit the composite video data to a videoconference.

11. The system of claim 10, wherein:

the background data files comprise video data files and image data files.

12. The system of claim 10, wherein:

the background data files comprise slideshow data files from which different background data files are automatically selected periodically as the background data file for use in generating the second layer.

13. The system of claim 10, wherein:

the video processing device is configured to generate second layers of background data files, wherein at least one of the second layers is generated from a first substitute background video data file that is different from another second background video data file that is used for generating an other second layer, to merge the first layer and the second layers to generate respective composite video data for each video conference participant including the first one of the second layers so that pixels in the second layer are shown through corresponding pixels in the first layer according to the alpha channel values of the corresponding pixels, and transmit the respective composite video data to respective videoconference participants in a videoconference; and

the video processing device is further configured to prompt a user to select a respective background data file and for each respective videoconference participant and associate the respective background data files with the respective videoconference participants and for each respective videoconference participant generate associated composite video data based on the respectively associated background data file.

14. The system of claim 10, wherein:

the processing device is further configured to prompt a user to select one or more background data files to define slideshow data and automatically and periodically select the one or more background data files from the slideshow data for substitution to generate generation of the composite video data.

15. The system of claim 10, wherein:

the background video data comprises streamed video data received over the communication subsystem.

16. A system, comprising:

means for storing background video data of a videoconference participant environment, the background video data defining, for each of a plurality of pixels, color data values of the pixel and an alpha channel value of the pixel;

means for receiving additional video data of the videoconference participant environment, the additional video data defining, for each of a plurality of pixels, color data values of the pixel and an alpha channel value of the pixel;

means for determining if a pixel of the additional video data is a match to a corresponding pixel of the background video data, wherein a match indicates that both a color data value of a pixel of the video data and a color data value of a corresponding pixel of the background video data are within a threshold rang;

setting an alpha channel value for the pixel to a first value that designates the pixel of the additional video data as a transparent pixel if a pixel of the additional video data is determined to be a match to a corresponding pixel of the background video data, and setting an alpha channel value for the pixel to a second value that designates the pixel of the additional video data as an opaque pixel if a pixel of the additional video data is determined not to be a match to a corresponding pixel of the background video data;

means for generating a first layer from the additional video data having pixels designated as transparent and pixels designated as opaque based on the respective alpha channel values, each of the pixels including the color data of the additional video data, generating a second layer from substitute background video data, the second layer including data defining, for each of a plurality of pixels, color data of the pixel, and merging the first layer and the second layer to generate composite video data, the second layer being subordinate to the first layer so that only the pixels of the second layer that correspond to transparent pixels of the first layer are shown when the composite video data is presented for display; and

means for transmitting the composite video data to a videoconference.