Systems and methods of visualizing multimedia content

• US 10,402,438 B2
• Filed: 07/20/2015
• Issued: 09/03/2019
• Est. Priority Date: 07/20/2015
• Status: Active Grant

• Current Assignee: Disney Enterprises Incorporated (The Walt Disney Company)
• Original Assignee: Disney Enterprises Incorporated (The Walt Disney Company)
• Inventors: Drake, Edward, Mutalipassi, Anthony, Wright, Andrew
• Primary Examiner(s): Lee, Ting Z
• Application Number:
  US14/804,150
• Publication Number:
  US 20170024385A1
• Time in Patent Office: 1,506 Days
• CPC Class Codes: G06F 16/435 Filtering based on additional data, e.g. user or group profiles
• CPC Class Codes: G06F 16/48 Retrieval characterised by using metadata, e.g. metadata not derived from the content or metadata generated manually
• CPC Class Codes: G06F 40/117 Tagging; Marking up details of markup languages G06F40/143; Designating a block; Setting of attributes style sheets, e.g. eXtensible Stylesheet Language Transformation [XSLT], G06F40/154
• CPC Class Codes: G11B 27/322 used signal is digitally coded
• CPC Class Codes: H04N 21/258 Client or end-user data management, e.g. managing client capabilities, user preferences or demographics, processing of multiple end-users preferences to derive collaborative data arrangements for services using the result on the distributing side of broadcast systems H04H60/66; profiles in network data switching protocols H04L67/30
• CPC Class Codes: H04N 21/25891 being end-user preferences retrieval of video data in a video database based on user preferences G06F16/739; arrangements for recognizing users' preferences H04H60/46; user profiles in network data switching protocols H04L67/306; processing of user preferences or user profiles in wireless networks H04W8/18
• CPC Class Codes: H04N 21/45 Management operations performed by the client for facilitating the reception of or the interaction with the content or administrating data related to the end-user or to the client device itself, e.g. learning user preferences for recommending movies, resolving scheduling conflicts
• CPC Class Codes: H04N 21/454 Content or additional data filtering, e.g. blocking advertisements
• CPC Class Codes: H04N 21/462 Content or additional data management, e.g. creating a master electronic program guide from data received from the Internet and a Head-end, controlling the complexity of a video stream by scaling the resolution or bit-rate based on the client capabilities
• CPC Class Codes: H04N 5/76 Television signal recording
• CPC Class Codes: H04N 9/8205 involving the multiplexing of an additional signal and the colour video signal

Abstract

Systems and methods for creating dynamic, real-time visual representations of multimedia content. A temporal metadata visualization module receives a plurality of temporal metadata tags associated with a time code reference, representing a plurality of attributes of the multimedia content at the particular time. During playback, the temporal metadata virtualization module applies a set of display preferences and creates a visual representation of attributes identified by the temporal metadata tags at a particular time code reference. The visual representation changes based on the temporal metadata tags of a point within playback of the multimedia content, enhancing a user'"'"'s experience and engagement with the multimedia content.

Claims

1. A method of visualizing metadata of multimedia content, comprising:

a temporal metadata visualization module receiving a plurality of temporal metadata tags of a multimedia content, the plurality of temporal metadata tags identifying one or more content attributes of the multimedia content;

receiving a set of content attribute filters, wherein the set of content attribute filters is based on the one or more content attributes of the multimedia content such that the set of content attribute filters allows a user to select the one or more content attributes;

receiving user input selecting one or more of the set of content attribute filters;

creating a visual representation of the one or more filtered content attributes, wherein the filtered content attributes are filtered by a user-selected set of content attribute filters;

playing the multimedia content;

displaying the visual representation of the one or more filtered content attributes on a display of the user device, wherein displaying occurs via a multimedia application running on the user device during playback of the multimedia content, wherein the visual representation comprises;

a visual tag cloud, wherein individual ones of the one or more filtered content attributes are represented by a respective visual indicator within the visual tag cloud; and

dynamically changing the respective visual indicator in real time within the visual tag cloud to visually represent the one or more filtered content attributes associated with a current portion of the multimedia content being displayed, wherein the one or more filtered content attributes associated with a current portion of the multimedia content are determined based on a time code reference corresponding to the current portion of the multimedia content.

2. The method of claim 1, wherein the multimedia application comprises the temporal metadata visualization module.

3. The method of claim 2, wherein the user device is communicatively coupled to a multimedia content distribution system, the multimedia content is streamed from the multimedia content distribution system via the multimedia application running on the user device, and receiving the plurality of temporal metadata tags comprises:

the user device receiving the multimedia content from the multimedia content distribution system, wherein the multimedia content comprises a content portion and a metadata tag portion; and

the temporal metadata visualization module identifying the metadata tag portion, the metadata tag portion comprising the plurality of temporal metadata tags.

4. The method of claim 2, wherein the one or more filtered content attributes are generated by:

filtering, by the temporal metadata visualization module, the plurality of temporal metadata tags according to the user-selected set of content attribute filters; and

the temporal metadata visualization module creating a visual representation of the one or more filtered content attribute filters.

5. The method of claim 4, wherein filtering the plurality of temporal metadata tags comprises:

the temporal metadata visualization module identifying the user-selected set of content attribute filters via an attribute filtering interface of the multimedia application running on the user device; and

the temporal metadata visualization module applying the user-selected set of content attribute filters to the plurality of temporal metadata tags.

6. The method of claim 4, wherein receiving the plurality of temporal metadata tags comprises:

the multimedia content distribution system retrieving the multimedia content from a content storage location, wherein the multimedia content comprises a content portion and a metadata tag portion; and

the temporal metadata visualization module identifying the metadata tag portion, the metadata tag portion comprising the plurality of temporal metadata tags.

7. The method of claim 1, wherein a multimedia content distribution system comprises the temporal metadata visualization module.

8. The method of claim 7, wherein the one or more filtered content attributes are generated by:

filtering, by the temporal metadata visualization module, the plurality of temporal metadata tags according to the user-selected set of content attribute filters; and

the temporal metadata visualization module creating a visual representation of the one or more filtered content attribute filters.

9. The method of claim 8, wherein filtering the plurality of temporal metadata tags comprises:

the temporal metadata visualization module identifying the user-selected set of content attribute filters via an attribute filtering interface of the multimedia application running on the user device; and

the temporal metadata visualization module applying the user-selected set of content attribute filters to the plurality of temporal metadata tags.

10. The method of claim 1, wherein the visual tag cloud is a mixed indicator-textual tag cloud.

11. The method of claim 10, further comprising the temporal metadata visualization module displaying a multimedia content display area, wherein the visual representation and the multimedia content display area are displayed such that the multimedia content and the visual representation are visible simultaneously.

12. The method of claim 1, wherein the visual representation is overlaid on the multimedia content, such that the multimedia content and the visual representation are visible simultaneously.

13. The method of claim 1, wherein a second user device is communicatively coupled to a first user device, the multimedia content is displayed on a display of the first user device, and displaying the visual representation comprises displaying the visual representation on a display of the second user device.

14. The method of claim 1, wherein the multimedia content comprises one or more of:

a movie;

a television show;

a song;

a music album;

an animated video;

or an audio book.

28. The method of claim 1, wherein the one or more visual indicators within the visual tag cloud are selectable, wherein the method further comprises:

receiving user input selecting one of the visual indicators; and

in response to receiving the user input, accessing and displaying additional information about the one or more filtered content attributes represented by the selected visual indicator.

15. A system for visually representing metadata of multimedia content, comprising:

a user device comprising a display; and

a non-transitory computer-readable medium having instructions stored thereon that, when executed by a processor, cause the system to;

run a multimedia application on the user device, the multimedia application configured to playback a multimedia content having a plurality of temporal metadata tags;

receive a set of content attribute filters, wherein the set of content attribute filters is based on the one or more content attributes of the multimedia content such that the set of content attribute filters allows a user to select the one or more content attributes;

receive user input selecting one or more of the set of content attribute filters;

create a visual representation of the one or more filtered content attributes, wherein the filtered content attributes are filtered by a user-selected set of content attribute filters, wherein the visual representation comprises;

a visual tag cloud, wherein individual ones of the one or more filtered content attribute are represented by a respective visual indicator within the visual tag cloud;

display the visual representation on the display of the user device via the multimedia application during playback of the multimedia content; and

dynamically change the visual representation in real time based on a current moment within the multimedia content, wherein the one or more filtered content attributes associated with the current moment of the multimedia content are determined based on a time code reference corresponding to the current moment of the multimedia content; and

wherein the respective visual indicator dynamically changes at least one of a size, shape, and location based on at least one of the one or more filtered content attributes associated with the time code reference corresponding to the current moment of the multimedia content.

16. The system of claim 15, wherein the visual representation comprises:

a visual tag cloud, wherein the one or more filtered content attributes are represented by a respective visual indicator within the visual tag cloud, and wherein dynamically changing the visual representation comprises dynamically changing the visual indicators within the visual tag cloud.

17. The system of claim 16, wherein the instructions, when executed by the processor, further cause the system to display a multimedia content display area on the display of the user device, wherein the visual representation and the multimedia content display area are displayed such that the multimedia content and the visual representation are visible simultaneously.

18. The system of claim 16, wherein the one or more visual indicators within the visual tag cloud are selectable, wherein the instructions, when executed by the processor, further cause the system:

to receive user input selecting one of the visual indicators; and

in response to receiving the user input, accessing and displaying additional information about the one or more filtered content attributes represented by the selected visual indicator.

19. The system of claim 15, wherein the visual representation is overlaid on the multimedia content, such that the multimedia content and the visual representation are visible simultaneously.

20. The system of claim 15, wherein the visual representation is displayed immediately when playback of the multimedia content by the multimedia application.

21. The system of claim 15, wherein the multimedia application comprises a user interface accessible by a user of the user device during playback of the multimedia content.

22. The system of claim 21, wherein the user interface comprises a visualization mode button, the visualization mode button configured to initiate a temporal metadata visualization mode, wherein the visual representation is displayed on the display of the user device when the temporal metadata visualization mode is initiated and not displayed when the temporal metadata visualization mode is not initiated.

23. The system of claim 21, wherein the user interface comprises an attribute filtering interface, the attribute filtering interface enabling a user to create the set of content attribute filters.

24. The system of claim 15, the multimedia application further comprising a temporal metadata visualization mode, wherein the visual representation is displayed only when the temporal metadata visualization mode is initiated.

25. The system of claim 24, the user device further comprising an input device, wherein the temporal metadata visualization mode is initiated by an input action of a user.

26. The system of claim 25, wherein the input device comprises a touch screen, and the input action comprises a swipe gesture.

27. The system of claim 25, wherein the input device comprises a mouse, and the input gesture comprises clicking on a visualization viewer button.