SELECTED REPLACEMENT OF DIGITAL IMAGERY PORTIONS USING AUGMENTED REALITY

• US 20190313133A1
• Filed: 06/28/2018
• Published: 10/10/2019
• Est. Priority Date: 04/04/2018
• Status: Active Grant

• Current Assignee: Dish Technologies LLC (Dish Network Corporation)
• Original Assignee: Dish Technologies LLC (Dish Network Corporation)
• Inventors: MCSCHOOLER, JEFFREY
• Granted Patent: US 11,006,154 B2
• Time in Patent Office: Days
• CPC Class Codes: G11B 27/036 Insert-editing
• CPC Class Codes: H04N 21/2187 Live feed
• CPC Class Codes: H04N 21/23412 for generating or manipulating the scene composition of objects, e.g. MPEG-4 objects
• CPC Class Codes: H04N 21/23418 involving operations for analysing video streams, e.g. detecting features or characteristics television picture signal circuitry for scene change detection H04N5/147; filtering for image enhancement G06T5/00; methods or arrangements for recognising scenes G06V20/00; arrangements characterised by components specially adapted for monitoring, identification or recognition of video in broadcast systems H04H60/59
• CPC Class Codes: H04N 21/812 involving advertisement data advertising per se G06Q30/02

Abstract

Systems, devices and automated processes augment television broadcasts with digital content that automatically replaces static portions of the captured imagery. In a video broadcast of a live event, for example, signs or other advertisements that are physically present in the event venue can be automatically replaced in television broadcasts of the event by virtual imagery of other signs or objects, as desired.

Claims

1. A process executable by a computing device to automatically replace static imagery in live video content, the process comprising:

receiving live video content from a camera that captures an event occurring in a physical space;

initially identifying at least one location in the received live video content that represents static imagery for image replacement;

receiving subsequent live video content from the camera that captures the event occurring in the physical space;

identifying dynamic imagery in the subsequent live video occurring within the at least one replacement location;

adapting the subsequent live video to replace the static imagery at the least one replacement location with a replacement image while retaining the dynamic imagery in the subsequent live video as an overlay of the static image; and

providing the adapted subsequent live video comprising the replacement image as an output.

2. The process of claim 1 wherein the camera captures a substantially hemispherical field of view that encompasses the physical space.

3. The process of claim 1 at least one location in the received live video content that represents static imagery for image replacement is identified by the computing device comparing pixel values between a plurality of frames in the live video content to automatically identify consistent features between the plurality of frames.

4. The process of claim 1 wherein the adapting comprises the computing device automatically adjusting a shape of the replacement image to adapt to changes in a shape of the location that represents the static imagery.

5. The process of claim 1 wherein the camera captures a substantially hemispherical field of view that encompasses the physical space, and wherein the at least one location representing the static imagery is defined by coordinates defining boundaries of the static imagery within a three-dimensional space that represents the substantially hemispherical field of view.

6. The process of claim 5 wherein the adapting comprises the computing device automatically adjusting the coordinates defining the boundaries of the static imagery to adapt to changes in a shape of the location resulting from movement of the camera relative to the static imagery.

7. The process of claim 6 wherein the coordinates defining the three dimensional space comprise a first coordinate defining a azimuthal position within the hemispherical field and a second coordinate defining a position on elevation within the hemispherical field.

8. The process of claim 7 wherein adapting comprises replacing the imagery within the boundaries defined by the coordinates with the replacement image.

9. The process of claim 8 wherein the wherein the coordinates defining the three dimensional space further comprise a third coordinate representing an amount of zoom applied in the live video content.

10. The process of claim 9 wherein the adapting comprises replacing the imagery within the boundaries defined by the coordinates with the replacement image and scaling a size of the replacement image according to the third coordinate representing the amount of zoom.

11. The process of claim 1 wherein the adapting comprises adjusting the luminance of the replacement image based upon a change in a brightness of the static imagery.

12. A computer system comprising:

an interface to receive live video content from a camera; and

a processor configured to automatically replace static imagery in the live video content by performing a process that comprises;

receiving live video content from a camera that captures an event occurring in a physical space;

initially identifying at least one location in the received live video content that represents static imagery for image replacement;

receiving subsequent live video content from the camera that captures the event occurring in the physical space;

identifying dynamic imagery in the subsequent live video occurring within the at least one replacement location;

adapting the subsequent live video to replace the static imagery at the least one replacement location with a replacement image while retaining the dynamic imagery in the subsequent live video as an overlay of the static image; and

providing the adapted subsequent live video as an output.

13. The computer system of claim 12 wherein the camera captures a substantially hemispherical field of view that encompasses the physical space.

14. The computer system of claim 12 at least one location in the received live video content that represents static imagery for image replacement is identified by the computing device comparing pixel values between a plurality of frames in the live video content to identify consistent features between the plurality of frames.

15. The computer system of claim 12 at least one location in the received live video content that represents static imagery for image replacement is identified by the computing device comparing averaging pixel values across a plurality of frames in the live video content to identify imagery that does not change between frames.

16. The computer system of claim 12 wherein the camera captures a substantially hemispherical field of view that encompasses the physical space, and wherein the at least one location representing the static imagery is defined coordinates defining a three-dimensional space within the hemispherical field of view.

17. The computer system of claim 16 wherein the coordinates define boundaries of the at least one location representing the static imagery.

18. The computer system of claim 17 wherein the imagery within the boundaries defined by the coordinates is replacement with the replacement image.

19. The computer system of claim 18 wherein the processor further adapts the coordinates that define the boundaries of the at least one location representing the static imagery in response to relative movement between the camera and an object captured in the static imagery so that a shape of the replacement image is adapted in response to changes in a shape of the static imagery.

20. The computer system of claim 12 wherein the processor further adapts a luminance of the replacement image in response to a change in a brightness of the static imagery.