AUTOMATED CINEMATIC DECISIONS BASED ON DESCRIPTIVE MODELS

• US 20190313056A1
• Filed: 10/05/2018
• Published: 10/10/2019
• Est. Priority Date: 04/10/2018
• Status: Active Grant

• Current Assignee: Meta Platforms, Inc. (f/k/a Facebook, Inc.)
• Original Assignee: Meta Platforms, Inc. (f/k/a Facebook, Inc.)
• Inventors: Harrison, Jason Francis, Hwang, Eric W., Nallamothu, Rahul, Razzaq, Shahid
• Granted Patent: US 10,511,808 B2
• Time in Patent Office: Days
• CPC Class Codes: G06F 18/22 Matching criteria, e.g. proximity measures
• CPC Class Codes: G06T 2207/10016 Video; Image sequence
• CPC Class Codes: G06T 2207/20044 Skeletonization; Medial axis transform
• CPC Class Codes: G06T 2207/20221 Image fusion; Image merging
• CPC Class Codes: G06T 7/12 Edge-based segmentation
• CPC Class Codes: G06T 7/194 involving foreground-background segmentation
• CPC Class Codes: G06T 7/246 using feature-based methods, e.g. the tracking of corners or segments
• CPC Class Codes: G06T 7/73 using feature-based methods
• CPC Class Codes: G06V 10/50 by performing operations within image blocks; by using histograms, e.g. histogram of oriented gradients [HoG]; by summing image-intensity values; Projection analysis
• CPC Class Codes: G06V 10/56 relating to colour
• CPC Class Codes: G06V 40/10 Human or animal bodies, e.g. vehicle occupants or pedestrians; Body parts, e.g. hands
• CPC Class Codes: G06V 40/103 Static body considered as a whole, e.g. static pedestrian or occupant recognition
• CPC Class Codes: G06V 40/168 Feature extraction; Face representation
• CPC Class Codes: H04N 21/23418 involving operations for analysing video streams, e.g. detecting features or characteristics television picture signal circuitry for scene change detection H04N5/147; filtering for image enhancement G06T5/00; methods or arrangements for recognising scenes G06V20/00; arrangements characterised by components specially adapted for monitoring, identification or recognition of video in broadcast systems H04H60/59
• CPC Class Codes: H04N 21/4223 Cameras H04N23/00 takes precedence
• CPC Class Codes: H04N 21/44008 involving operations for analysing video streams, e.g. detecting features or characteristics in the video stream arrangements characterised by components specially adapted for monitoring, identification or recognition of video in broadcast systems H04H60/59
• CPC Class Codes: H04N 21/45455 applied to a region of the image
• CPC Class Codes: H04N 21/4728 for selecting a Region Of Interest [ROI], e.g. for requesting a higher resolution version of a selected region
• CPC Class Codes: H04N 21/4788 communicating with other users, e.g. chatting arrangements for providing for computer conferences, e.g. chat rooms, to substation in data switching networks H04L12/1813; distributed application using peer-to-peer [P2P] networks H04L67/104
• CPC Class Codes: H04N 23/611 where the recognised objects include parts of the human body
• CPC Class Codes: H04N 23/64 : Computer-aided capture of images, e.g. transfer from script file into camera, check of taken image quality, advice or proposal for image composition or decision on when to take image
• CPC Class Codes: H04N 23/69 : Control of means for changing angle of the field of view, e.g. optical zoom objectives or electronic zooming
• CPC Class Codes: H04N 23/695 : Control of camera direction for changing a field of view, e.g. pan, tilt or based on tracking of objects
• CPC Class Codes: H04N 5/262 : Studio circuits, e.g. for mixing, switching-over, change of character of image, other special effects ; Cameras specially adapted for the electronic generation of special effects
• CPC Class Codes: H04N 5/2628 : Alteration of picture size, shape, position or orientation, e.g. zooming, rotation, rolling, perspective, translation
• CPC Class Codes: H04N 5/272 : Means for inserting a foreground image in a background image, i.e. inlay, outlay
• CPC Class Codes: H04N 7/147 : Communication arrangements, e.g. identifying the communication as a video-communication, intermediate storage of the signals selecting H04Q
• CPC Class Codes: H04N 7/15 : Conference systems
• CPC Class Codes: H04N 7/155 : involving storage of or access to video conference sessions tracking arrangements for later retrieval of a computer conference content or participants activities H04L12/1831
• CPC Class Codes: H04R 1/406 : microphones
• CPC Class Codes: H04R 3/005 : for combining the signals of two or more microphones specially adapted for hearing aids H04R25/407

Abstract

In one embodiment, a method includes, at a first time during an audio-video communication session: (1) determining that a first participant is located in an environment; (2) locating a first body region of the first participant; (3) generating a first color histogram of the first body region that represents a first distribution of colors of the first body region. The method also includes, at a second time: (1) determining that a second participant is located in the environment; (2) locating a second body region of the second participant; (3) generating a second color histogram of the second body region that represents a second distribution of one or more colors of the second body region. The method also includes determining that the first and second color histograms are substantially alike, and based on the determination, determining that the first participant is the same as the second participant.

Claims

1. A method comprising, by a computing device:

at a first time during an audio-video communication session;

determining that a first participant is located in an environment associated with the audio-video communication session;

locating a first body region of the first participant;

generating a first color histogram of the first body region, wherein the first color histogram represents a first distribution of one or more colors of the first body region;

at a second time during the audio-video communication;

determining that a second participant is located in the environment;

locating a second body region of the second participant that is the same as the first body region;

generating a second color histogram of the second body region, wherein the second color histogram represents a second distribution of one or more colors of the second body region;

determining that the first color histogram is the same as the second color histogram;

based on the determination that the first color histogram is the same as the second color histogram, determining that the first participant is the same as the second participant.

2. The method of claim 1, wherein the first color histogram and the second color histogram each comprise a number of pixel buckets, wherein the pixel buckets comprise pixels whose color value is within a span associated with the respective pixel bucket.

3. The method of claim 1, further comprising identifying the first participant and storing the first color histogram in a social graph associated with the first participant.

4. The method of claim 1, further comprising determining one or more body ratios for the first participant and the second participant, the body ratios comprising a shoulder-to-torso ratio or a head-to-torso ratio, wherein the determining that the first participant is the same as the second participant is further based on the one or more body ratios.

5. The method of claim 1, further comprising:

determining a first current location for the first participant and a first trajectory for the first participant;

determining a second current location for the second participant and a second trajectory for the second participant, wherein the determining that the first participant is the same as the second participant is further based on the first location, first trajectory, second location, and second trajectory.

6. The method of claim 5, further comprising assigning a plurality of weights to the first trajectory, second trajectory, first location, second location, first color histogram, and second color histogram, wherein at a first weight of the plurality of weights is greater than a second weight of the plurality of weights.

7. The method of claim 1, wherein the first body region and the second body region are a torso region.

8. One or more computer-readable non-transitory storage media embodying software that is operable when executed to:

at a first time during an audio-video communication session;

determine that a first participant is located in an environment associated with the audio-video communication session;

locate a first body region of the first participant;

generate a first color histogram of the first body region, wherein the first color histogram represents a first distribution of one or more colors of the first body region;

at a second time during the audio-video communication;

determine that a second participant is located in the environment;

locate a second body region of the second participant that is the same as the first body region;

generate a second color histogram of the second body region, wherein the second color histogram represents a second distribution of one or more colors of the second body region;

determine that the first color histogram is the same as the second color histogram;

based on the determination that the first color histogram is the same as the second color histogram, determining that the first participant is the same as the second participant.

9. The media of claim 8, wherein the first color histogram and the second color histogram each comprise a number of pixel buckets, wherein the pixel buckets comprise pixels whose color value is within a span associated with the respective pixel bucket.

10. The media of claim 8, wherein the software is further operable when executed to identify the first participant and storing the first color histogram in a social graph associated with the first participant.

11. The media of claim 8, wherein the software is further operable when executed to determine one or more body ratios for the first participant and the second participant, the body ratios comprising a shoulder-to-torso ratio or a head-to-torso ratio, wherein the determining that the first participant is the same as the second participant is further based on the one or more body ratios.

12. The media of claim 8, wherein the software is further operable when executed to:

determine a first current location for the first participant and a first trajectory for the first participant; and

determine a second current location for the second participant and a second trajectory for the second participant, wherein the determining that the first participant is the same as the second participant is further based on the first location, first trajectory, second location, and second trajectory.

13. The media of claim 8, wherein the software is further operable when executed to assign a plurality of weights to the first trajectory, second trajectory, first location, second location, first color histogram, and second color histogram, wherein at a first weight of the plurality of weights is greater than a second weight of the plurality of weights.

14. The media of claim 8, wherein the first body region and the second body region are a torso region.

15. A system comprising:

one or more processors; and

one or more computer-readable non-transitory storage media coupled to one or more of the processors and comprising instructions operable when executed by one or more of the processors to cause the system to;

at a first time during an audio-video communication session;

determine that a first participant is located in an environment associated with the audio-video communication session;

locate a first body region of the first participant;

generate a first color histogram of the first body region, wherein the first color histogram represents a first distribution of one or more colors of the first body region;

at a second time during the audio-video communication;

determine that a second participant is located in the environment;

locate a second body region of the second participant that is the same as the first body region;

generate a second color histogram of the second body region, wherein the second color histogram represents a second distribution of one or more colors of the second body region;

determine that the first color histogram is the same as the second color histogram;

based on the determination that the first color histogram is the same as the second color histogram, determining that the first participant is the same as the second participant.

16. The system of claim 15, wherein the first color histogram and the second color histogram each comprise a number of pixel buckets, wherein the pixel buckets comprise pixels whose color value is within a span associated with the respective pixel bucket.

17. The system of claim 15, wherein the processors are further operable when executing the instructions to identify the first participant and storing the first color histogram in a social graph associated with the first participant.

18. The system of claim 15, wherein the processors are further operable when executing the instructions to determine one or more body ratios for the first participant and the second participant, the body ratios comprising a shoulder-to-torso ratio or a head-to-torso ratio, wherein the determining that the first participant is the same as the second participant is further based on the one or more body ratios.

19. The system of claim 15, wherein the processors are further operable when executing the instructions to:

determine a first current location for the first participant and a first trajectory for the first participant; and

determine a second current location for the second participant and a second trajectory for the second participant, wherein the determining that the first participant is the same as the second participant is further based on the first location, first trajectory, second location, and second trajectory.

20. The system of claim 15, wherein the processors are further operable when executing the instructions to assign a plurality of weights to the first trajectory, second trajectory, first location, second location, first color histogram, and second color histogram, wherein at a first weight of the plurality of weights is greater than a second weight of the plurality of weights.