System and Method for Multi-Device Video Image Display and Modification

• US 20170041677A1
• Filed: 10/21/2016
• Published: 02/09/2017
• Est. Priority Date: 06/03/2014
• Status: Active Grant

• Current Assignee: Disney Enterprises Incorporated (The Walt Disney Company)
• Original Assignee: Disney Enterprises Incorporated (The Walt Disney Company)
• Inventors: Anderson, Hans C., Thiel, Aaron, Shi, Li, Morris, Mathieu, Swartz, Jeremiah Michael
• Granted Patent: US 9,736,540 B2
• Time in Patent Office: Days
• US Class Current: 1/1
• CPC Class Codes: G06F 40/169 Annotation, e.g. comment data or footnotes
• CPC Class Codes: H04N 21/42204 User interfaces specially adapted for controlling a client device through a remote control device; Remote control devices therefor constructive details of casings for the remote control device H01H9/0235; user interfaces for controlling a tuning device of a television receiver through a remote control H03J9/00; remote control of peripheral devices connected to a television receiver through the remote control device of the television receiver H04B1/205
• CPC Class Codes: H04N 21/47205 for manipulating displayed content, e.g. interacting with MPEG-4 objects, editing locally
• CPC Class Codes: H04N 21/478 Supplemental services, e.g. displaying phone caller identification, shopping application
• CPC Class Codes: H04N 21/4788 communicating with other users, e.g. chatting arrangements for providing for computer conferences, e.g. chat rooms, to substation in data switching networks H04L12/1813; distributed application using peer-to-peer [P2P] networks H04L67/104
• CPC Class Codes: H04N 5/445 for displaying additional information H04N5/50 takes precedence

Abstract

Methods and systems for allowing multiple devices to display and annotate (or modify) a video image are described, where a content renderer device displays a video image that is also displayed (or viewed) by one or more content viewer devices, which can both display and annotate (or modify) the same video image at the same time, and the content renderer selectively determines whether or not a GUI menu is displayed on its display screen. Annotation commands (e.g., screen touch data) on the viewer devices are captured, packaged and sent to the content renderer, which creates a combined modified video image having the annotations from the viewer devices, displays it to the renderer display and sends it to the viewer devices for display thereon. Also, the viewer device may selectively display video content from multiple other peer devices as a composite video image, may annotate (or modify) each displayed video content independently, and may send a screen video image, or a portion thereof, to any other peer device. Any peer device may operate as a content renderer or content viewer, or both, at any given time, and peer devices may be running different operating systems.

Claims

1. A method for modifying a displayed video image, comprising:

transmitting by a content renderer device a first video image to a renderer display;

transmitting a supplemented video image by the content renderer device to at least one user device, the supplemented video image having the first video image and having selected GUI control graphics, the GUI control graphics displaying options to modify the first image;

receiving by the content renderer device user input data from the at least one user device indicative of a corresponding modification of the first image to be displayed on a user display of the at least one user device;

wherein the user input data indicative of the modification to be displayed being intercepted in realtime prior to the user device acting on the user input data, and being sent by the user device to the content renderer device in realtime;

transmitting in realtime by the content renderer device a first modified video image to the renderer display, the first modified video image having the first video image and the modifications, and not having the selected GUI control graphics;

transmitting in realtime by the content renderer device a second modified video image to the at least one user display, the second modified video image having the supplemented video image and the modifications;

wherein the content renderer device and the user device each having a peer-sharing master software application that is adapted to provide user input data in realtime from the user device to the content renderer device and adapted to transmit the first and second modified video images from the content renderer device;

wherein the user device receives and displays the second modified video image; and

wherein a latency time between the user input data being received by the content renderer device and the user device display of the second modified video image is less than a predetermined acceptable latency time.

2. The method of claim 1, wherein the content renderer receives the user input data from a plurality of user devices and the first and second modified video images are indicative of a compilation of the modifications of the first video image on the user displays of the plurality of user devices.

3. The method of claim 2, wherein the peer sharing software application is device platform agnostic and at least two of the user devices are running different operating systems from each other.

4. The method of claim 1, further comprising translating, by the content renderer, the user input data into modification commands and wherein the first and second modified video images are based on the modification commands.

5. The method of claim 1, further comprising receiving the first video image by the content renderer.

6. The method of claim 1, wherein the user input data is indicative of at least one of:

a screen touch, a mouse click, a mouse pad touch, a keyboard press, and a light pen press.

7. The method of claim 1, wherein the user input data comprises at least one of:

state data, common language data, and command data.

8. The method of claim 1, wherein the modification comprises at least one of:

a video telestration, an annotation, a video edit, a video control, an application control, a line drawing, dragging and dropping, a text character, an eraser, a highlighter, a solid line, a dashed line, an arrow, a circular shape image, a rectangular shape image, and an animated blinking circle image.

9. The method of claim 1 wherein the at least one user device is operating as a content viewer.

10. The method of claim 1 wherein the first video image is generated by an application running on the content renderer and wherein the video image from the application is displayed on the at least one user device and the application is running on the content renderer and not running on the user device.

11. The method of claim 10, wherein the application is capable of receiving user inputs to modify the first video image.

12. The method of claim 10, wherein the application comprises at least one of:

a web browser application, a Video Telestration application, a word processor application, a presentation application, a spreadsheet application, a sports Draft application, and a video clip browser application.

13. The method of claim 1, further comprising:

receiving a plurality of video images from a corresponding plurality of the user devices;

creating a compilation video image from each of the plurality of video images indicative of the combination of each of said plurality of video images; and

causing the display of the compilation video image on the renderer display.

14. The method of claim 13, further comprising:

causing the display of a plurality of user device options associated with the video images from the corresponding user devices; and

accepting the selection of at least two of the plurality of user device options for creating the compilation video image.

15. The method of claim 1, further comprising causing the display of at least one user device option associated with video images from a corresponding at least one user device;

accepting the selection of the least one user device option as a selected video image; and

causing, upon the selection of the device option, the first video image to become the selected video image.

16. The method of claim 1, wherein the predetermined acceptable latency time comprises one of:

1.0 seconds, 0.5 seconds, 0.25 seconds, and 0.1 seconds.

17. The method of claim 1, wherein the user input data is converted into common language data which is sent to the content renderer and wherein the common language data is independent of the operating system running on the user device.

18. The method of claim 1, wherein the peer sharing software application is device platform agnostic.

19. A non-transitory computer readable medium having a computer executable component for causing a computer system to perform a method, comprising:

transmitting by a content renderer device a first video image to a renderer display;

transmitting a supplemented video image by the content renderer device to at least one user device, the supplemented video image having the first video image and having selected GUI control graphics;

receiving by the content renderer device user input data from the at least one user device indicative of modifications on a user display of the at least one user device;

wherein the user input data indicative of the modification to be displayed being intercepted in realtime prior to the user device acting on the user input data, and being sent by the user device to the renderer device in realtime;

transmitting in realtime by the content renderer device a first modified video image to the renderer display, the first modified video image having the first video image and having the modifications;

transmitting in realtime by the content renderer device a second modified video image to the at least one user device, the second modified video image having the supplemented video image and the modifications;

wherein the content renderer device and the user device each having a peer-sharing master software application that is adapted to provide user input data in realtime from the user device to the content renderer device and adapted to transmit the first and second modified video images from the content renderer device;

wherein the user device receives and displays the second modified video image; and

wherein a latency time between the user input data being received by the content renderer device and the user device display of the second modified video image is less than a predetermined acceptable latency time.

20. A system for modifying and displaying a video image comprising:

a content renderer;

a renderer display;

a first user device;

a second user device;

wherein;

the first and second user devices are adapted to receive user inputs and send state data in realtime reflecting the user inputs, to the content renderer, prior to the user device acting on the respective user inputs;

the content renderer is adapted to;

transmit a video image to the renderer display; and

transmit the video image, supplemented with a graphical user interface, to the first and second user devices, the graphical user interface displaying to a user options for modifying the video image;

receive the state data reflecting the user inputs, based on options displayed by the graphical user interface, from the first and second user devices;

convert in realtime the state data to instructions for modifying the video image;

generate in realtime a modified video image based on the instructions;

transmit in realtime the modified video image, without the graphical user interface, to the renderer display; and

transmit in realtime the modified video image supplemented with the graphical user interface, to the first and second user devices;

wherein the content renderer and the first and second user devices each having a peer-sharing master software application that is adapted to send in realtime the respective state data reflecting the user inputs from the respective user device to the content renderer and adapted to transmit the modified video image to the renderer display and to the first and second user devices;

wherein the first and second user devices each receives and displays the modified video image supplemented with the graphical user interface on a user display of the respective first and second user devices; and

wherein a latency time between the user input data being received by the content renderer and the user device display of the modified video image supplemented with the graphical user interface, is less than a predetermined acceptable latency time.