Method and system for automatic content recognition (ACR) integration for smartTVs and mobile communication devices

• US 9,282,346 B2
• Filed: 12/27/2013
• Issued: 03/08/2016
• Est. Priority Date: 12/28/2012
• Status: Active Grant

• Current Assignee: Turner Broadcasting System Incorporated (Warner Bros. Discovery, Inc.)
• Original Assignee: Turner Broadcasting System Incorporated (Warner Bros. Discovery, Inc.)
• Inventors: Sinha, Nishith Kumar, Giles, Matthew Paul, Loheide, Donald Jude
• Primary Examiner(s): HANCE, ROBERT J
• Application Number:
  US14/141,931
• Publication Number:
  US 20140282657A1
• Time in Patent Office: 802 Days
• US Class Current: 1/1
• CPC Class Codes: H04H 20/93 which locates resources of other pieces of information, e.g. URL [Uniform Resource Locator]
• CPC Class Codes: H04H 60/37 for identifying segments of broadcast information, e.g. scenes or extracting programme ID
• CPC Class Codes: H04H 60/40 for identifying broadcast time
• CPC Class Codes: H04H 60/82 the transmission system being the Internet
• CPC Class Codes: H04N 21/23 Processing of content or additional data; Elementary server operations; Server middleware
• CPC Class Codes: H04N 21/234 Processing of video elementary streams, e.g. splicing of video streams or manipulating encoded video stream scene graphs
• CPC Class Codes: H04N 21/23418 involving operations for analysing video streams, e.g. detecting features or characteristics television picture signal circuitry for scene change detection H04N5/147; filtering for image enhancement G06T5/00; methods or arrangements for recognising scenes G06V20/00; arrangements characterised by components specially adapted for monitoring, identification or recognition of video in broadcast systems H04H60/59
• CPC Class Codes: H04N 21/23424 involving splicing one content stream with another content stream, e.g. for inserting or substituting an advertisement
• CPC Class Codes: H04N 21/236 Assembling of a multiplex stream, e.g. transport stream, by combining a video stream with other content or additional data, e.g. inserting a URL [Uniform Resource Locator] into a video stream, multiplexing software data into a video stream; Remultiplexing of multiplex streams; Insertion of stuffing bits into the multiplex stream, e.g. to obtain a constant bit-rate; Assembling of a packetised elementary stream multiplexing of data packets for data networks, e.g. RTP/UDP H04L65/00
• CPC Class Codes: H04N 21/2407 Monitoring of transmitted content, e.g. distribution time, number of downloads arrangements for monitoring programmes for broadcast or for distribution combined with broadcast H04H20/14
• CPC Class Codes: H04N 21/4122 additional display device, e.g. video projector digital output for controlling a plurality of local displays G06F3/1423
• CPC Class Codes: H04N 21/4126 The peripheral being portable, e.g. PDAs or mobile phones
• CPC Class Codes: H04N 21/41407 embedded in a portable device, e.g. video client on a mobile phone, PDA, laptop constructional details of equipment or arrangements specially adapted for portable computer application G06F1/1626; arrangements specially adapted for mobile receivers in broadcast systems H04H20/57
• CPC Class Codes: H04N 21/43079 of additional data with content streams on multiple devices
• CPC Class Codes: H04N 21/431 Generation of visual interfaces for content selection or interaction; Content or additional data rendering
• CPC Class Codes: H04N 21/4316 for displaying supplemental content in a region of the screen, e.g. an advertisement in a separate window
• CPC Class Codes: H04N 21/4325 by playing back content from the storage medium reproduction of recorded television signals H04N5/76; reproduction of recorded television signals H04N9/79
• CPC Class Codes: H04N 21/435 Processing of additional data, e.g. decrypting of additional data, reconstructing software from modules extracted from the transport stream
• CPC Class Codes: H04N 21/44008 involving operations for analysing video streams, e.g. detecting features or characteristics in the video stream arrangements characterised by components specially adapted for monitoring, identification or recognition of video in broadcast systems H04H60/59
• CPC Class Codes: H04N 21/442 Monitoring of processes or resources, e.g. detecting the failure of a recording device, monitoring the downstream bandwidth, the number of times a movie has been viewed, the storage space available from the internal hard disk arrangements for monitoring broadcast services or broadcast-related services H04H60/29; arrangements for identifying or recognising characteristics with a direct linkage to broadcast information H04H60/35; monitoring of user activities for profile generation for accessing a video database G06F16/739; monitoring in wireless networks H04W24/00
• CPC Class Codes: H04N 21/44204 : Monitoring of content usage, e.g. the number of times a movie has been viewed, copied or the amount which has been watched monitoring of user activities for profile generation for accessing a video database G06F16/739; protecting generic digital content where the protection is independent of the precise nature of the content G06F21/10; arrangements for monitoring the use made of the broadcast services in broadcast systems H04H60/31
• CPC Class Codes: H04N 21/472 : End-user interface for requesting content, additional data or services; End-user interface for interacting with content, e.g. for content reservation or setting reminders, for requesting event notification, for manipulating displayed content end-user interfaces for retrieving video data from a database G06F16/739; network services for supporting unicast streaming H04L65/612
• CPC Class Codes: H04N 21/47205 : for manipulating displayed content, e.g. interacting with MPEG-4 objects, editing locally
• CPC Class Codes: H04N 21/6125 : involving transmission via Internet transmission by internet of broadcast information H04H60/82
• CPC Class Codes: H04N 21/6175 : involving transmission via Internet broadcast-related systems characterised by the transmission system being the Internet H04H60/82
• CPC Class Codes: H04N 21/64322 : IP
• CPC Class Codes: H04N 21/812 : involving advertisement data advertising per se G06Q30/02
• CPC Class Codes: H04N 21/8173 : End-user applications, e.g. Web browser, game
• CPC Class Codes: H04N 21/84 : Generation or processing of descriptive data, e.g. content descriptors systems specially adapted for using meta-information in broadcast systems H04H60/73
• CPC Class Codes: H04N 21/858 : Linking data to content, e.g. by linking an URL to a video object, by creating a hotspot

Abstract

A second screen communication device that is paired with an automatic content recognition (ACR) enabled display communication device launches an interactive application in response to an invitation cue generated and presented by the ACR-enabled display communication device while the ACR-enabled display communication device is displaying linear broadcast content. The linear broadcast content is a non-IP based broadcast content. The interactive application launched on the second screen communication device enables interaction with interactive content displayed by a user interface on the second screen communication device based on the invitation cue. The second screen communication device may generate one or more control elements that are utilized to control the interaction with the interactive content. The displaying of IP-based digital stream content corresponding to the linear broadcast content on the automatic content recognition enabled display may be resumed on the ACR enabled communication device utilizing the generated one or more control elements.

Claims

1. A method, comprising:

in a second screen communication device that is paired with an automatic content recognition enabled display communication device that communicates with an automatic content recognition system that assigns same interactive event identifiers to different sets of video fingerprints that are generated from same video content by different video fingerprint technologies from multiple vendors based on timing of assignment of the interactive event identifiers to multiple sequences of the video fingerprints that are generated from the same video content;

in response to and/or based on an invitation cue generated by and presented on said automatic content recognition enabled display communication device, based on an automatic content recognition event triggered by an interactive event identifier, while said automatic content recognition enabled display communication device is displaying linear broadcast content, launching an interactive application on said second screen communication device, wherein said interactive application enables interaction with interactive content displayed by a user interface on said second screen communication device based on said invitation cue presented on said automatic content recognition enabled display communication device;

generating one or more control elements that enables said interaction with said interactive content displayed by said user interface on said second screen communication device;

controlling said interaction with said interactive content utilizing said generated one or more control elements; and

resuming display of an IP-based digital stream content that corresponds to said linear broadcast content on said automatic content recognition enabled display communication device utilizing said generated one or more control elements, wherein said resumption is based on automatic content recognition parameters comprising time code and content information.

2. The method according to claim 1, wherein said controlling of said interaction with said interactive content utilizing said generated one or more control elements to cause said interactive content displayed by said user interface on said second screen communication device to be concurrently displayed on said automatic content recognition enabled display communication device.

3. The method according to claim 1, comprising interacting with additional interactive content, which is displayed by said user interface on said second screen communication device and is associated with said invitation cue, subsequent to said resuming of said display of said IP-based digital stream content on said automatic content recognition enabled display communication device.

4. The method according to claim 1, comprising receiving a notification on said second screen communication device that indicates that said interactive content is available for display by said user interface on said second screen communication device.

5. The method according to claim 1, comprising, if said interactive application is not installed on said second screen communication device prior to said launching, downloading and/or installing said interactive application on said second screen communication device.

6. The method according to claim 1, comprising deferring said interaction with said interactive content utilizing said generated one or more control elements.

7. The method according to claim 1, wherein said invitation cue is generated based on an advertisement, which comprises a portion of said linear broadcast content that is displayed by said automatic content recognition enabled display communication device.

8. The method according to claim 7, wherein said interactive content displayed on said second screen communication device comprises subject matter that is representative of said advertisement.

9. The method according to claim 1, wherein said invitation cue comprises a visual and/or an audio cue.

10. The method according to claim 1, wherein:

said second screen communication device comprises one of a smart phone and a tablet; and

said automatic content recognition enabled display communication device comprises an automatic content recognition enabled smart television.

11. A system, comprising:

a second screen communication device that is paired with an automatic content recognition enabled display communication device that communicates with an automatic content recognition system that assigns same interactive event identifiers to different sets of video fingerprints that are generated from same video content by different video fingerprint technologies from multiple vendors based on timing of assignment of the interactive event identifiers to multiple sequences of the video fingerprints that are generated from the same video content, said second screen communication device being operable to;

in response to and/or based on an invitation cue generated by and presented on said automatic content recognition enabled display communication device, based on an automatic content recognition event triggered by an interactive event identifier, while said automatic content recognition enabled display communication device is displaying linear broadcast content, launch an interactive application on said second screen communication device, wherein said interactive application enables interaction with interactive content displayed by a user interface on said second screen communication device based on said invitation cue presented on said automatic content recognition enabled display communication device;

generate one or more control elements that enables said interaction with said interactive content displayed by said user interface on said second screen communication device;

control said interaction with said interactive content utilizing said generated one or more control elements; and

resume display of said IP-based digital stream that corresponds to said linear broadcast content on said automatic content recognition enabled display communication device utilizing said generated one or more control elements, wherein said resumption is based on automatic content recognition parameters comprising time code and content information.

12. The system according to claim 11, wherein said second screen communication device is operable to control said interaction with said interactive content utilizing said generated one or more control elements to cause said interactive content displayed by said user interface on said second screen communication device to be concurrently displayed on said automatic content recognition enabled display communication device.

13. The system according to claim 11, wherein said second screen communication device is operable to interact with additional interactive content, which is displayed by said user interface on said second screen communication device and is associated with said invitation cue, subsequent to said resuming of said display of said IP-based digital stream content on said automatic content recognition enabled display communication device.

14. The system according to claim 11, wherein said second screen communication device is operable to receive a notification that indicates that said interactive content is available for display by said user interface on said second screen communication device.

15. The system according to claim 11, wherein said second screen communication device is operable to download and/or install said interactive application on said second screen communication device, if said interactive application is not installed on said second screen communication device prior to said launching.

16. The system according to claim 11, wherein said second screen communication device is operable to defer said interaction with said interactive content utilizing said generated one or more control elements.

17. The system according to claim 11, wherein said invitation cue is generated based on an advertisement, which comprises a portion of said linear broadcast content that is displayed by said automatic content recognition enabled display communication device.

18. The system according to claim 17, wherein said interactive content displayed on said second screen communication device comprises subject matter that is representative of said advertisement.

19. The system according to claim 11, wherein:

said second screen communication device comprises one of a smart phone and a tablet; and

said automatic content recognition enabled display communication device comprises an automatic content recognition enabled smart television.

20. The non-transitory computer-readable medium according to claim 11, wherein said invitation cue comprises a visual and/or an audio cue.

21. A non-transitory computer-readable medium having stored thereon, a computer program having at least one code section being executable by a machine for causing the machine to perform steps comprising:

in a second screen communication device that is paired with an automatic content recognition enabled display communication device that communicates with an automatic content recognition system that assigns same interactive event identifiers to different sets of video fingerprints that are generated from same video content by different video fingerprint technologies from multiple vendors based on timing of assignment of the interactive event identifiers to multiple sequences of the video fingerprints that are generated from the same video content;

in response to and/or based on an invitation cue generated by and presented on said automatic content recognition enabled display communication device, based on an automatic content recognition event triggered by an interactive event identifier, while said automatic content recognition enabled display communication device is displaying linear broadcast content, launching an interactive application on said second screen communication device, wherein said interactive application enables interaction with interactive content displayed by a user interface on said second screen communication device based on said invitation cue presented on said automatic content recognition enabled display communication device;

generating one or more control elements that enables said interaction with said interactive content displayed by said user interface on said second screen communication device;

controlling said interaction with said interactive content utilizing said generated one or more control elements; and

resuming display of said IP-based digital stream content that corresponds to said linear broadcast content on said automatic content recognition enabled display communication device utilizing said generated one or more control elements, wherein said resumption is based on automatic content recognition parameters comprising time code and content information.

22. The non-transitory computer-readable medium according to claim 21, wherein said controlling of said interaction with said interactive content utilizing said generated one or more control elements to cause said interactive content displayed by said user interface on said second screen communication device to be concurrently displayed on said automatic content recognition enabled display communication device.

23. The non-transitory computer-readable medium according to claim 21, comprising interacting with additional interactive content, which is displayed by said user interface on said second screen communication device and is associated with said invitation cue, subsequent to said resuming of said display of said IP-based digital stream content on said automatic content recognition enabled display communication device.

24. The non-transitory computer-readable medium according to claim 21, comprising receiving a notification on said second screen communication device that indicates that said interactive content is available for display by said user interface on said second screen communication device.

25. The non-transitory computer-readable medium according to claim 21, comprising, if said interactive application is not installed on said second screen communication device prior to said launching, downloading and/or installing said interactive application on said second screen communication device.

26. The non-transitory computer-readable medium according to claim 21, comprising deferring said interaction with said interactive content utilizing said generated one or more control elements.

27. The non-transitory computer-readable medium according to claim 21, wherein said invitation cue is generated based on an advertisement, which comprises a portion of said linear broadcast content that is displayed by said automatic content recognition enabled display communication device.

28. The non-transitory computer-readable medium according to claim 27, wherein said interactive content displayed on said second screen communication device comprises subject matter that is representative of said advertisement.

29. The non-transitory computer-readable medium according to claim 21, wherein said invitation cue comprises a visual and/or an audio cue.

30. The non-transitory computer-readable medium according to claim 21, wherein:

said second screen communication device comprises on of a smart phone and a tablet; and

said automatic content recognition enabled display communication device comprises an automatic content recognition enabled smart television.

31. A method, comprising:

in a second screen communication device that is paired with an automatic content recognition enabled display communication device that communicates with an automatic content recognition system that assigns same interactive event identifiers to different sets of video fingerprints that are generated from same video content by different video fingerprint technologies from multiple vendors based on timing of assignment of the interactive event identifiers to multiple sequences of the video fingerprints that are generated from the same video content, and the interactive event identifiers are utilized to trigger one or more interactive events;

in response to and/or based on an invitation cue generated by and presented on said automatic content recognition enabled display communication device, based on the triggered one or more interactive events, while said automatic content recognition enabled display communication device is displaying linear broadcast content, launching an interactive application on said second screen communication device, wherein said interactive application enables interaction with interactive content displayed by a user interface on said second screen communication device based on said invitation cue presented on said automatic content recognition enabled display communication device;

generating one or more control elements that enables said interaction with said interactive content displayed by said user interface on said second screen communication device;

controlling said interaction with said interactive content utilizing said generated one or more control elements; and

resuming display of an IP-based digital stream content that corresponds to said linear broadcast content on said automatic content recognition enabled display communication device utilizing said generated one or more control elements, wherein said resumption is based on automatic content recognition parameters comprising time code and content information.