Unobtrusively enhancing video content with extrinsic data

• US 9,800,951 B1
• Filed: 03/10/2015
• Issued: 10/24/2017
• Est. Priority Date: 06/21/2012
• Status: Active Grant

• Current Assignee: Amazon Technologies, Inc. (Amazon.com, Inc.)
• Original Assignee: Amazon Technologies, Inc. (Amazon.com, Inc.)
• Inventors: Carlson, Adam, Goetz, Jeromey Russell, Hungarter, Douglas Andrew, Tritschler, Charles G., Larsen, Peter A.
• Primary Examiner(s): Shepard, Justin
• Application Number:
  US14/644,006
• Time in Patent Office: 959 Days
• Field of Search: None
• CPC Class Codes: G06F 16/7867 using information manually generated, e.g. tags, keywords, comments, title and artist information, manually generated time, location and usage information, user ratings
• CPC Class Codes: H04N 21/23 Processing of content or additional data; Elementary server operations; Server middleware
• CPC Class Codes: H04N 21/234 Processing of video elementary streams, e.g. splicing of video streams or manipulating encoded video stream scene graphs
• CPC Class Codes: H04N 21/23418 involving operations for analysing video streams, e.g. detecting features or characteristics television picture signal circuitry for scene change detection H04N5/147; filtering for image enhancement G06T5/00; methods or arrangements for recognising scenes G06V20/00; arrangements characterised by components specially adapted for monitoring, identification or recognition of video in broadcast systems H04H60/59
• CPC Class Codes: H04N 21/2343 involving reformatting operations of video signals for distribution or compliance with end-user requests or end-user device requirements video transcoding H04N19/40; media packet handling at the source H04L65/762
• CPC Class Codes: H04N 21/23439 for generating different versions
• CPC Class Codes: H04N 21/235 Processing of additional data, e.g. scrambling of additional data or processing content descriptors
• CPC Class Codes: H04N 21/2355 involving reformatting operations of additional data, e.g. HTML pages optimising the visualization of content for information retrieval from the Internet G06F16/9577; tracking of instant messages H04L51/234; media packet handling at the source H04L65/762
• CPC Class Codes: H04N 21/2356 by altering the spatial resolution
• CPC Class Codes: H04N 21/2358 for generating different versions, e.g. for different recipient devices
• CPC Class Codes: H04N 21/25 Management operations performed by the server for facilitating the content distribution or administrating data related to end-users or client devices, e.g. end-user or client device authentication, learning user preferences for recommending movies maintenance or administration in data networks H04L41/00
• CPC Class Codes: H04N 21/25435 involving characteristics of content or additional data, e.g. video resolution or the amount of advertising
• CPC Class Codes: H04N 21/258 Client or end-user data management, e.g. managing client capabilities, user preferences or demographics, processing of multiple end-users preferences to derive collaborative data arrangements for services using the result on the distributing side of broadcast systems H04H60/66; profiles in network data switching protocols H04L67/30
• CPC Class Codes: H04N 21/25825 involving client display capabilities, e.g. screen resolution of a mobile phone optimising the visualisation of content during browsing in the Internet G06F16/9577; processing of terminal status or physical abilities in wireless networks H04W8/22; authentication in wireless network security H04W12/06
• CPC Class Codes: H04N 21/25833 involving client hardware characteristics, e.g. manufacturer, processing or storage capabilities allocation of resources considering hardware capabilities in multiprogramming arrangements G06F9/5044; allocation of resources considering software capabilities in multiprogramming arrangements G06F9/5055
• CPC Class Codes: H04N 21/26603 for automatically generating descriptors from content, e.g. when it is not made available by its provider, using content analysis techniques
• CPC Class Codes: H04N 21/40 Client devices specifically adapted for the reception of or interaction with content, e.g. set-top-box [STB]; Operations thereof
• CPC Class Codes: H04N 21/41 Structure of client; Structure of client peripherals
• CPC Class Codes: H04N 21/41265 having a remote control device for bidirectional communication between the remote control device and client device unidirectional remote control device H04N21/42204
• CPC Class Codes: H04N 21/422 Input-only peripherals , i.e. input devices connected to specially adapted client devices, e.g. global positioning system [GPS] input devices also receiving signals from specially adapted client devices H04N21/4104
• CPC Class Codes: H04N 21/42201 : biosensors, e.g. heat sensor for presence detection, EEG sensors or any limb activity sensors worn by the user input arrangements for interaction with the human body based on nervous system activity detection G06F3/015
• CPC Class Codes: H04N 21/42203 : sound input device, e.g. microphone
• CPC Class Codes: H04N 21/42209 : for displaying non-command information, e.g. electronic program guide [EPG], e-mail, messages or a second television channel
• CPC Class Codes: H04N 21/4223 : Cameras H04N23/00 takes precedence
• CPC Class Codes: H04N 21/43 : Processing of content or additional data, e.g. demultiplexing additional data from a digital video stream; Elementary client operations, e.g. monitoring of home network or synchronising decoder's clock; Client middleware real-time communication protocols in data switching networks H04L65/00
• CPC Class Codes: H04N 21/431 : Generation of visual interfaces for content selection or interaction; Content or additional data rendering
• CPC Class Codes: H04N 21/4312 : involving specific graphical features, e.g. screen layout, special fonts or colors, blinking icons, highlights or animations
• CPC Class Codes: H04N 21/4314 : for fitting data in a restricted space on the screen, e.g. EPG data in a rectangular grid
• CPC Class Codes: H04N 21/4316 : for displaying supplemental content in a region of the screen, e.g. an advertisement in a separate window
• CPC Class Codes: H04N 21/435 : Processing of additional data, e.g. decrypting of additional data, reconstructing software from modules extracted from the transport stream
• CPC Class Codes: H04N 21/4394 : involving operations for analysing the audio stream, e.g. detecting features or characteristics in audio streams arrangements characterised by components specially adapted for monitoring, identification or recognition of audio in broadcast systems H04H60/58
• CPC Class Codes: H04N 21/44008 : involving operations for analysing video streams, e.g. detecting features or characteristics in the video stream arrangements characterised by components specially adapted for monitoring, identification or recognition of video in broadcast systems H04H60/59
• CPC Class Codes: H04N 21/4415 : using biometric characteristics of the user, e.g. by voice recognition or fingerprint scanning cryptography using biological data H04L9/3231; authentication in networks using biometric H04L63/0861
• CPC Class Codes: H04N 21/44218 : Detecting physical presence or behaviour of the user, e.g. using sensors to detect if the user is leaving the room or changes his face expression during a TV program methods or arrangements for recognising human body or animal bodies or body parts G06V40/10; methods or arrangements for acquiring or recognising human faces, facial parts, facial sketches, facial expressions G06V40/16; methods or arrangements for recognising movements or behaviour G06V40/20; arrangements for identifying users in broadcast systems H04H60/45
• CPC Class Codes: H04N 21/4532 : involving end-user characteristics, e.g. viewer profile, preferences monitoring of user activities for profile generation for accessing a video database G06F16/739; user profiles in network data switching protocols H04L67/306; processing of user preferences or user profiles in wireless networks H04W8/18
• CPC Class Codes: H04N 21/4722 : for requesting additional data associated with the content
• CPC Class Codes: H04N 21/47815 : Electronic shopping payment schemes, payment architectures or payment protocols for electronic shopping systems G06Q20/12
• CPC Class Codes: H04N 21/4826 : using recommendation lists, e.g. of programs or channels sorted out according to their score
• CPC Class Codes: H04N 21/4828 : for searching program descriptors retrieval of video data G06F16/739
• CPC Class Codes: H04N 21/4882 : for displaying messages, e.g. warnings, reminders arrangements for providing short real-time information to substation in data switching networks H04L12/1895
• CPC Class Codes: H04N 21/8133 : specifically related to the content, e.g. biography of the actors in a movie, detailed information about an article seen in a video program
• CPC Class Codes: H04N 21/8146 : involving graphical data, e.g. 3D object, 2D graphics
• CPC Class Codes: H04N 21/84 : Generation or processing of descriptive data, e.g. content descriptors systems specially adapted for using meta-information in broadcast systems H04H60/73

Abstract

The playback of video content upon a display is enhanced through the unobtrusive presentation of extrinsic data upon the same display. A video content feature is rendered on a display. It is determined whether to present extrinsic data relevant to a current time in the video content feature based at least in part on appearance control metadata. A graphical element presenting at least a portion of the extrinsic data is rendered on the display while the video content feature is also being rendered on the display in response to determining to present the extrinsic data.

Claims

1. A non-transitory computer-readable medium embodying a program, that when executed by at least one computing device, causes the at least one computing device to at least:

cause a video content feature to be rendered on a display;

determine whether a user account history indicates that a corresponding user has viewed the video content feature previously;

determine a resolution used by the display;

determine an estimated distance between the display and a viewer of the display;

determine extrinsic data relevant to a current time in the video content feature including at least one of;

character information, music information, or trivia information, a quantity of the extrinsic data being based at least in part on whether the user account history indicates that the corresponding user has viewed the video content feature previously; and

cause a graphical element to be rendered upon the display while the video content feature is also being rendered on the display, the graphical element presenting at least a portion of the extrinsic data, a size of the graphical element being determined as a function of the resolution, the estimated distance, and the quantity of the extrinsic data determined.

2. The non-transitory computer-readable medium of claim 1, wherein the program further causes the at least one computing device to determine a region in which the graphical element is to be rendered based at least in part on an analysis of the video content feature.

3. The non-transitory computer-readable medium of claim 1, wherein the quantity of the extrinsic data is greater and the size of the graphical element is larger when the user account history indicates that the corresponding user has not viewed the video content feature previously.

4. A system, comprising:

a data store; and

at least one computing device comprising a processor and a memory, the at least one computing device in communication with the data store and being configured to at least;

render a video content feature on a display;

determine whether a user account history indicates that a corresponding user has viewed the video content feature previously;

determine a resolution of the display;

determine an estimated distance between the display and a viewer of the display;

determine extrinsic data relevant to a current scene in the video content feature including at least one of;

character information, music information, or trivia information, a quantity of the extrinsic data being based at least in part on whether the user account history indicates that the corresponding user has viewed the video content feature previously; and

render a graphical element presenting at least a portion of the extrinsic data while the video content feature is also being rendered on the display, wherein a size of the graphical element is determined as a function of the resolution, the estimated distance, and the quantity of the extrinsic data determined.

5. The system of claim 4, wherein appearance control metadata is generated based at least in part on a user account history and a determination to render the graphical element is based in part on the appearance control metadata.

6. The system of claim 4, wherein the at least one computing device is further configured to determine whether to present the extrinsic data further based at least in part on a user account configuration setting that is enabled by default.

7. The system of claim 4, wherein the at least one computing device is further configured to render the graphical element in a region selected so as not to obscure a visual element of the video content feature.

8. The system of claim 7, wherein the visual element corresponds to a face of a character, and the region is selected based at least in part on a detected location of the face.

9. The system of claim 7, wherein the region is selected based at least in part on an analysis of an entirety of the video content feature or an analysis of a frame of the video content feature.

10. The system of claim 4, wherein the at least one computing device is further configured to determine whether an appearance of a character corresponds to a first recognizable appearance of the character in the video content feature.

11. The system of claim 4, wherein the video content feature corresponds to an episode of a series, and the at least one computing device is further configured to determine whether to present the extrinsic data based at least in part on whether a user account history indicates that the corresponding user has already viewed another episode of the series.

12. The system of claim 4, wherein the extrinsic data comprises information about a character that appears in the video content feature, and the at least one computing device is further configured to determine whether to present the extrinsic data based at least in part on whether the current scene corresponds to a first recognizable appearance of the character in the video content feature.

13. The system of claim 4, wherein the extrinsic data comprises information about a music track that is featured in the video content feature, and the at least one computing device is further configured to determine whether to present the extrinsic data based at least in part on whether the current scene corresponds to a play of the music track in the video content feature.

14. The system of claim 4, wherein the at least one computing device is further configured to determine whether an offline mode is supported, wherein the extrinsic data comprises all images and data necessary to render the graphical element when the offline mode is supported.

15. The system of claim 14, wherein the at least one computing device is further configured to decompress the extrinsic data to extract the images and data necessary to render the graphical element.

16. A method, comprising:

causing, via at least one computing device comprising a processor and a memory, a video content feature to be rendered on a display;

determining, via the at least one computing device, whether a user account history indicates that a corresponding user has viewed the video content feature previously;

determining, via the at least one computing device, a resolution used by the display;

determining, via the at least one computing device, an estimated distance between the display and a viewer of the display;

determining, via the at least one computing device, extrinsic data that is relevant to a current scene in the video content feature including at least one of;

character information, music information, or trivia information, a quantity of the extrinsic data being based at least in part on whether the user account history indicates that a corresponding user has viewed the video content feature previously; and

causing, via the at least one computing device, a graphical element to be rendered on the display while the video content feature is also being rendered on the display, the graphical element including at least a portion of the extrinsic data, a size of the graphical element being determined as a function of the resolution, the estimated distance, and the quantity of the extrinsic data determined.

17. The method of claim 16, wherein causing the graphical element to be rendered further comprises determining, via the at least one computing device, that a configuration setting that is enabled by default has not been disabled.

18. The method of claim 16, wherein the graphical element includes a name of a character, a name of a cast member who is performing the character, and an image of the character.

19. The method of claim 16, further comprising causing, via the at least one computing device, the graphical element to be positioned at a location upon the display based at least in part on a visual analysis of the video content feature.

20. The method of claim 16, further comprising;

determining, via the at least one computing device, that an appearance of a character in the current scene of the video content feature corresponds to a first recognizable appearance of the character in the video content feature; and

refraining, via the at least one computing device, from rendering information about the character upon the display upon a subsequent appearance of the character in a subsequent scene of the video content feature.