Transmitting video and sharing content via a network

• US 10,325,172 B2
• Filed: 07/05/2016
• Issued: 06/18/2019
• Est. Priority Date: 12/31/2013
• Status: Active Grant

• Current Assignee: Personify Incorporated (Wilson Human Capital Group, Inc.)
• Original Assignee: Personify Incorporated (Wilson Human Capital Group, Inc.)
• Inventors: Nguyen, Quang, Do, Minh, Truong, Long, Lin, Dennis, Kowshik, Sumant, Patel, Sanjay
• Primary Examiner(s): Patel, Jayesh A
• Application Number:
  US15/202,478
• Publication Number:
  US 20160314369A1
• Time in Patent Office: 1,078 Days
• Field of Search: None
• CPC Class Codes: H04N 19/00 Methods or arrangements for coding, decoding, compressing or decompressing digital video signals
• CPC Class Codes: H04N 19/124 Quantisation
• CPC Class Codes: H04N 19/162 User input
• CPC Class Codes: H04N 19/17 the unit being an image region, e.g. an object
• CPC Class Codes: H04N 19/186 the unit being a colour or a chrominance component
• CPC Class Codes: H04N 19/21 with binary alpha-plane coding for video objects, e.g. context-based arithmetic encoding [CAE]
• CPC Class Codes: H04N 19/46 Embedding additional information in the video signal during the compression process H04N19/517, H04N19/68, H04N19/70 take precedence

Abstract

Embodiments disclose systems and methods for transmitting user-extracted video and content more efficiently by recognizing that user-extracted video provides the potential to treat parts of a single frame of a user-extracted video differently. An alpha mask of the image part of the user-extracted video is used when encoding the image part so that it retains a higher quality upon transmission than the remainder of the user-extracted video.

Claims

1. A method performed by a computer system comprising at least one processor, the method comprising:

identifying, by the computer system, in a video comprising a plurality of video frames, a foreground portion and a background portion by generating an alpha mask that is indicative of the foreground portion and the background portion, wherein the identified foreground portion includes a user persona;

selecting, by the computer system, a first chroma-key color not found in the identified foreground portion in a first sequence of video frames;

generating, by the computer system, a modified first sequence of video frames at least in part by replacing the identified background portion with the selected first chroma-key color, and outputting the modified first sequence of video frames for transmission;

determining, by the computer system, that the selected first chroma-key color is present in the identified foreground portion in a second, subsequent, sequence of video frames, and responsively selecting a second chroma-key color not found in the identified foreground portion in the second sequence of video frames; and

generating, by the computer system, a modified second sequence of video frames at least in part by replacing the identified background portion in the second sequence of video frames with the second chroma-key color, and outputting the modified second sequence of video frames for transmission.

2. The method of claim 1, wherein determining that the selected first chroma-key color is present in the identified foreground portion in the second sequence of video frames comprises determining that the identified foreground portion in the second sequence of video frames includes a color that is within a predetermined color distance L of the selected first chroma-key color.

3. The method of claim 1, wherein selecting the second chroma-key color comprises:

building a 3D YUV histogram of color points from the identified foreground portion in the second sequence of video frames;

identifying an empty box in the 3D YUV histogram; and

selecting the second chroma-key color from the identified empty box in the 3D YUV histogram.

4. The method of claim 3, wherein the identified empty box is cubic.

5. The method of claim 3, wherein selecting the second chroma-key color further comprises selecting a color from a middle of the identified empty box.

6. The method of claim 3, wherein identifying the empty box comprises identifying an empty box having at least a predetermined size.

7. The method of claim 6, wherein the 3D YUV histogram initially does not include the empty box of at least the predetermined size, the method further comprising modifying at least one color point in the 3D YUV histogram such that, after modifying the at least one color point, the 3D YUV histogram includes the empty box of at least the predetermined size.

8. The method of claim 1, wherein outputting the modified first and second sequences of video frames for transmission comprises Z-encoding the modified first and second sequences of video frames for transmission.

9. The method of claim 8, wherein Z-encoding comprises compressing the identified foreground and background portions differently.

10. A non-transitory computer readable storage medium storing a set of instructions that, when executed by a computer system comprising at least one processor, cause the computer system to perform functions including:

identifying, by the computer system, in a video comprising a plurality of video frames, a foreground portion and a background portion by generating an alpha mask that is indicative of the foreground portion and the background portion, wherein the identified foreground portion includes a user persona;

selecting a first chroma-key color not found in the identified foreground portion in a first sequence of video frames;

generating a modified first sequence of video frames at least in part by replacing the identified background portion with the selected first chroma-key color, and outputting the modified first sequence of video frames for transmission;

determining that the selected first chroma-key color is present in the identified foreground portion in a second, subsequent, sequence of video frames, and responsively selecting a second chroma-key color not found in the identified foreground portion in the second sequence of video frames; and

generating a modified second sequence of video frames at least in part by replacing the identified background portion in the second sequence of video frames with the second chroma-key color, and outputting the modified second sequence of video frames for transmission.

11. The non-transitory computer readable storage medium of claim 10, wherein for determining that the selected first chroma-key color is present in the identified foreground portion of the second sequence of video frames, the storage medium further comprises instructions for determining that the identified foreground portion in the second sequence of video frames includes a color that is within a predetermined color distance L of the selected first chroma-key color.

12. The non-transitory computer readable storage medium of claim 10, wherein for selecting the second chroma-key color, the storage medium further comprising instructions for:

building a 3D YUV histogram of color points from the identified foreground portion in the second sequence of video frames;

identifying an empty box in the 3D YUV histogram; and

selecting the second chroma-key color from the identified empty box.

13. The non-transitory computer readable storage medium of claim 12, wherein the identified empty box is cubic.

14. The non-transitory computer readable storage medium of claim 12, wherein for selecting the second chroma-key color, the storage medium further comprises instructions for selecting a color from a middle of the identified empty box.

15. The non-transitory computer readable storage medium of claim 12, wherein for identifying the empty box, the storage medium further comprises instructions for identifying an empty box having at least a predetermined size.

16. The non-transitory computer readable storage medium of claim 15, wherein the 3D YUV histogram initially does not include the empty box of at least the predetermined size, the storage medium further comprising instructions for modifying at least one color point in the 3D YUV histogram such that, after modifying the at least one color point, the 3D YUV histogram includes the empty box of at least the predetermined size.

17. The non-transitory computer readable storage medium of claim 10, wherein for outputting the modified first and second sequences of video frames for transmission, the storage medium further comprises instructions for Z-encoding the modified first and second sequences of video frames for transmission.

18. The non-transitory computer readable storage medium of claim 17, wherein for Z-encoding, the storage medium further comprises instructions for compressing the identified foreground and background portions differently.