AUTOMATED CINEMATIC DECISIONS BASED ON DESCRIPTIVE MODELS

• US 20190311480A1
• Filed: 10/05/2018
• Published: 10/10/2019
• Est. Priority Date: 04/10/2018
• Status: Active Grant

• Current Assignee: Meta Platforms, Inc. (f/k/a Facebook, Inc.)
• Original Assignee: Meta Platforms, Inc. (f/k/a Facebook, Inc.)
• Inventors: Harrison, Jason Francis, Hwang, Eric W., Nallamothu, Rahul, Razzaq, Shahid
• Granted Patent: US 10,979,669 B2
• Time in Patent Office: Days
• CPC Class Codes: G06F 18/22 Matching criteria, e.g. proximity measures
• CPC Class Codes: G06T 2207/10016 Video; Image sequence
• CPC Class Codes: G06T 2207/20044 Skeletonization; Medial axis transform
• CPC Class Codes: G06T 2207/20221 Image fusion; Image merging
• CPC Class Codes: G06T 7/12 Edge-based segmentation
• CPC Class Codes: G06T 7/194 involving foreground-background segmentation
• CPC Class Codes: G06T 7/246 using feature-based methods, e.g. the tracking of corners or segments
• CPC Class Codes: G06T 7/73 using feature-based methods
• CPC Class Codes: G06V 10/50 by performing operations within image blocks; by using histograms, e.g. histogram of oriented gradients [HoG]; by summing image-intensity values; Projection analysis
• CPC Class Codes: G06V 10/56 relating to colour
• CPC Class Codes: G06V 40/10 Human or animal bodies, e.g. vehicle occupants or pedestrians; Body parts, e.g. hands
• CPC Class Codes: G06V 40/103 Static body considered as a whole, e.g. static pedestrian or occupant recognition
• CPC Class Codes: G06V 40/168 Feature extraction; Face representation
• CPC Class Codes: H04N 21/23418 involving operations for analysing video streams, e.g. detecting features or characteristics television picture signal circuitry for scene change detection H04N5/147; filtering for image enhancement G06T5/00; methods or arrangements for recognising scenes G06V20/00; arrangements characterised by components specially adapted for monitoring, identification or recognition of video in broadcast systems H04H60/59
• CPC Class Codes: H04N 21/4223 Cameras H04N23/00 takes precedence
• CPC Class Codes: H04N 21/44008 involving operations for analysing video streams, e.g. detecting features or characteristics in the video stream arrangements characterised by components specially adapted for monitoring, identification or recognition of video in broadcast systems H04H60/59
• CPC Class Codes: H04N 21/45455 applied to a region of the image
• CPC Class Codes: H04N 21/4728 for selecting a Region Of Interest [ROI], e.g. for requesting a higher resolution version of a selected region
• CPC Class Codes: H04N 21/4788 communicating with other users, e.g. chatting arrangements for providing for computer conferences, e.g. chat rooms, to substation in data switching networks H04L12/1813; distributed application using peer-to-peer [P2P] networks H04L67/104
• CPC Class Codes: H04N 23/611 where the recognised objects include parts of the human body
• CPC Class Codes: H04N 23/64 : Computer-aided capture of images, e.g. transfer from script file into camera, check of taken image quality, advice or proposal for image composition or decision on when to take image
• CPC Class Codes: H04N 23/69 : Control of means for changing angle of the field of view, e.g. optical zoom objectives or electronic zooming
• CPC Class Codes: H04N 23/695 : Control of camera direction for changing a field of view, e.g. pan, tilt or based on tracking of objects
• CPC Class Codes: H04N 5/262 : Studio circuits, e.g. for mixing, switching-over, change of character of image, other special effects ; Cameras specially adapted for the electronic generation of special effects
• CPC Class Codes: H04N 5/2628 : Alteration of picture size, shape, position or orientation, e.g. zooming, rotation, rolling, perspective, translation
• CPC Class Codes: H04N 5/272 : Means for inserting a foreground image in a background image, i.e. inlay, outlay
• CPC Class Codes: H04N 7/147 : Communication arrangements, e.g. identifying the communication as a video-communication, intermediate storage of the signals selecting H04Q
• CPC Class Codes: H04N 7/15 : Conference systems
• CPC Class Codes: H04N 7/155 : involving storage of or access to video conference sessions tracking arrangements for later retrieval of a computer conference content or participants activities H04L12/1831
• CPC Class Codes: H04R 1/406 : microphones
• CPC Class Codes: H04R 3/005 : for combining the signals of two or more microphones specially adapted for hearing aids H04R25/407

Abstract

In one embodiment, a method includes accessing foreground visual data that comprises a set of coordinate points that correspond to a plurality of surface points of a person in an environment; generating a bounding box for the set of coordinate points, wherein the bounding box comprises every coordinate point in the set of coordinate points; providing instructions to collect background visual data for an area in the environment that is outside of the bounding box; and providing the foreground visual data and the background visual data to an intelligent director associated with the computing device.

Claims

1. A method comprising, by a computing device:

accessing foreground visual data that comprises a set of coordinate points that correspond to a plurality of surface points of a person in an environment;

generating a bounding box for the set of coordinate points, wherein the bounding box comprises every coordinate point in the set of coordinate points;

providing instructions to collect background visual data for an area in the environment that is outside of the bounding box; and

providing the foreground visual data and the background visual data to an intelligent director associated with the computing device.

2. The method of claim 1, further comprising assigning a foreground tag to the bounding box and a background tag to the background visual data.

3. The method of claim 1, wherein the set of coordinate points inside the bounding box is updated at regular intervals.

4. The method of claim 1, further comprising assigning a foreground tag to the bounding box and a background tag to the background visual data, wherein the foreground tag remains assigned to the bounding box until the bounding box moves out of the environment.

5. The method of claim 1, wherein the background visual data comprises a plurality of pixels, wherein each pixel:

comprises a red/green/blue (RGB) color value; and

corresponds to a unique location within the environment.

7. The method of claim 1, further comprising:

receiving a user input that indicates that the user has tapped on a location of a display screen of an intelligent communication device that corresponds to the bounding box;

providing instructions to crop out at least some of the background visual data to render the person for display on the display screen.

6. The method of claim 6, further comprising, before providing the instructions to collect the background visual data, assigning each pixel of the of the plurality of pixels a randomly generated RGB color value.

8. One or more computer-readable non-transitory storage media embodying software that is operable when executed to:

access foreground visual data that comprises a set of coordinate points that correspond to a plurality of surface points of a person in an environment;

generate a bounding box for the set of coordinate points, wherein the bounding box comprises every coordinate point in the set of coordinate points;

provide instructions to collect background visual data for an area in the environment that is outside of the bounding box; and

provide the foreground visual data and the background visual data to an intelligent director associated with the computing device.

9. The media of claim 8, wherein the software is further operable when executed to assign a foreground tag to the bounding box and a background tag to the background visual data.

10. The media of claim 8, wherein the set of coordinate points inside the bounding box is updated at regular intervals.

11. The media of claim 8, wherein the software is further operable when executed to assign a foreground tag to the bounding box and a background tag to the background visual data, wherein the foreground tag remains assigned to the bounding box until the bounding box moves out of the environment.

12. The media of claim 8, wherein the background visual data comprises a plurality of pixels, wherein each pixel:

comprises a red/green/blue (RGB) color value; and

corresponds to a unique location within the environment.

13. The media of claim 12, wherein the software is further operable when executed to, before providing the instructions to collect the background visual data, assigning each pixel of the of the plurality of pixels a randomly generated RGB color value.

14. The media of claim 8, wherein the software is further operable when executed to:

receive a user input that indicates that the user has tapped on a location of a display screen of an intelligent communication device that corresponds to the bounding box;

provide instructions to crop out at least some of the background visual data to render the person for display on the display screen.

15. A system comprising:

one or more processors; and

one or more computer-readable non-transitory storage media coupled to one or more of the processors and comprising instructions operable when executed by one or more of the processors to cause the system to;

access foreground visual data that comprises a set of coordinate points that correspond to a plurality of surface points of a person in an environment;

generate a bounding box for the set of coordinate points, wherein the bounding box comprises every coordinate point in the set of coordinate points;

provide instructions to collect background visual data for an area in the environment that is outside of the bounding box; and

provide the foreground visual data and the background visual data to an intelligent director associated with the computing device.

16. The system of claim 15, wherein the processors are further operable when executing the instructions to assign a foreground tag to the bounding box and a background tag to the background visual data.

17. The system of claim 15, wherein the set of coordinate points inside the bounding box is updated at regular intervals.

18. The system of claim 15, wherein the processors are further operable when executing the instructions to assign a foreground tag to the bounding box and a background tag to the background visual data, wherein the foreground tag remains assigned to the bounding box until the bounding box moves out of the environment.

19. The system of claim 15, wherein the background visual data comprises a plurality of pixels, wherein each pixel:

comprises a red/green/blue (RGB) color value; and

corresponds to a unique location within the environment.

20. The system of claim 19, wherein the processors are further operable when executing the instructions to, before providing the instructions to collect the background visual data, assigning each pixel of the of the plurality of pixels a randomly generated RGB color value.