Time synchronizing of distinct video and data feeds that are delivered in a single mobile IP data network compatible stream

• US 10,327,044 B2
• Filed: 01/13/2017
• Issued: 06/18/2019
• Est. Priority Date: 12/13/2006
• Status: Active Grant

• Current Assignee: DIRECTV, LLC (AT&T, Inc.)
• Original Assignee: Quickplay Media Inc (AT&T, Inc.)
• Inventors: Walker, Torin, Martinez, Michael M.
• Primary Examiner(s): Sheleheda, James R
• Application Number:
  US15/405,805
• Publication Number:
  US 20170134830A1
• Time in Patent Office: 886 Days
• CPC Class Codes: G06F 16/487 using geographical or spatial information, e.g. location
• CPC Class Codes: G06F 16/739 in form of a video summary, e.g. the video summary being a video sequence, a composite still image or having synthesized frames
• CPC Class Codes: G06F 21/1011 to devices
• CPC Class Codes: G06Q 10/06 Resources, workflows, human or project management; Enterprise or organisation planning; Enterprise or organisation modelling
• CPC Class Codes: G06Q 10/10 Office automation; Time management
• CPC Class Codes: G06Q 30/0244 Optimization
• CPC Class Codes: G06Q 30/0267 Wireless devices
• CPC Class Codes: G06Q 30/0269 based on user profile or attribute
• CPC Class Codes: G06Q 30/0277 Online advertisement
• CPC Class Codes: H04H 20/26 Arrangements for switching distribution systems
• CPC Class Codes: H04H 60/91 Mobile communication networks for accessing the Internet H04H60/85
• CPC Class Codes: H04L 65/1059 End-user terminal functionalities specially adapted for real-time communication
• CPC Class Codes: H04L 65/611 for multicast or broadcast systems for broadcast or conference H04L12/18; arrangements for broadcast or distribution combined with broadcast H04H20/00; arrangements for broadcast applications with a direct linkage to broadcast information or to broadcast space-time H04H60/00; selective distribution of broadcast services, e.g. multimedia broadcast multicast service [MBMS], H04W4/06
• CPC Class Codes: H04L 65/612 for unicast
• CPC Class Codes: H04L 65/70 Media network packetisation
• CPC Class Codes: H04L 65/752 adapting media to network capabilities
• CPC Class Codes: H04L 65/756 adapting media to device capabilities
• CPC Class Codes: H04L 65/762 at the source reformatting of additional data in video distribution servers H04N21/2355
• CPC Class Codes: H04L 65/765 intermediate
• CPC Class Codes: H04L 65/80 Responding to QoS
• CPC Class Codes: H04L 67/30 : Profiles
• CPC Class Codes: H04N 21/2187 : Live feed
• CPC Class Codes: H04N 21/23424 : involving splicing one content stream with another content stream, e.g. for inserting or substituting an advertisement
• CPC Class Codes: H04N 21/2343 : involving reformatting operations of video signals for distribution or compliance with end-user requests or end-user device requirements video transcoding H04N19/40; media packet handling at the source H04L65/762
• CPC Class Codes: H04N 21/2365 : Multiplexing of several video streams
• CPC Class Codes: H04N 21/242 : Synchronization processes, e.g. processing of PCR [Program Clock References] arrangements for synchronising broadcast or distribution via plural systems in broadcast distribution systems H04H20/18
• CPC Class Codes: H04N 21/2665 : Gathering content from different sources, e.g. Internet and satellite
• CPC Class Codes: H04N 21/41407 : embedded in a portable device, e.g. video client on a mobile phone, PDA, laptop constructional details of equipment or arrangements specially adapted for portable computer application G06F1/1626; arrangements specially adapted for mobile receivers in broadcast systems H04H20/57
• CPC Class Codes: H04N 21/6112 : involving terrestrial transmission, e.g. DVB-T
• CPC Class Codes: H04N 21/6125 : involving transmission via Internet transmission by internet of broadcast information H04H60/82
• CPC Class Codes: H04N 21/6131 : involving transmission via a mobile phone network wireless downlink channel access H04W74/006
• CPC Class Codes: H04N 21/6408 : Unicasting
• CPC Class Codes: H04N 21/64322 : IP
• CPC Class Codes: H04N 21/8547 : involving timestamps for synchronizing content

Abstract

Multiple video and/or content feeds are combined and synchronized in real-time into a single mobile IP data network compatible stream for presenting as a coherent display on a mobile device.

Claims

1. A method comprising:

accessing, by a processing system including a processor, a plurality of distinct feeds including a video feed of a live event;

accessing, by the processing system, time stamp information for at least a portion of the plurality of distinct feeds;

determining, by the processing system, by comparing each time stamp information to a reference clock a relative order of, and a relative time difference among, the portion of the plurality of distinct feeds;

determining, by the processing system, a synchronizing point in each of the plurality of distinct feeds at which each is synchronized relative to the reference clock;

accumulating, by the processing system, event data to generate a video object representing at least a portion of the event data;

generating, by the processing system at a predetermined interval, a succession of frames each comprising a state of the video object; and

combining, by the processing system, the succession of frames and the plurality of distinct feeds to form a single stream comprising a plurality of synchronized distinct feeds based on the determined synchronizing point in each of the plurality of distinct feeds, wherein the succession of frames is not synchronized.

2. The method of claim 1, further comprising encoding, by the processing system, the single stream to form a composite mobile device compatible stream.

3. The method of claim 1, wherein the single stream comprising a plurality of synchronized distinct feeds comprises a mosaic of the plurality of the distinct feeds.

4. The method of claim 1, further comprising replacing, by the processing system, at least one of the distinct feeds in the single stream with other content.

5. The method of claim 1, further comprising replacing, by the processing system, at least one of the distinct feeds in the single stream with an advertisement.

6. The method of claim 1, wherein at least one of the plurality of distinct feeds provides content associated with another one of the plurality of distinct feeds, and wherein the at least one of the plurality of distinct feeds is not synchronized to ensure fine grain coherency.

7. A system comprising:

an ingestion facility of a media processing platform configured to receive a plurality of distinct feeds including a video feed of a live event;

a media processor of the media processing platform configured to accumulate event data to generate a video object representing at least a portion of the event data, and further configured to generate, at a predetermined interval, a succession of frames each comprising a state of the video object; and

a feed combining engine that is operationally connected to the ingestion facility and that produces a single stream comprising the succession of frames and a portion of the received plurality of distinct feeds that are provided by the ingestion facility, the feed combining engine further;

accessing a feed clock associated with a source of at least one of the plurality of distinct feeds;

accessing a reference clock; and

determining by comparing the feed clock to the reference clock a latency between the at least one of the portion of the plurality of received distinct feeds and the reference clock, the portion of the received plurality of distinct feeds in the single stream being synchronized relative to the reference clock based on the determined latency, wherein the succession of frames is not synchronized.

8. The system of claim 7, wherein a frame of at least one of the distinct feeds comprises a synchronization point for a portion of the plurality of distinct feeds.

9. The system of claim 7, further comprising an encoding engine that encodes the single stream to form a composite mobile device compatible stream.

10. The system of claim 7, wherein the single stream comprises a mosaic of the portion of the plurality of the received distinct feeds.

11. The system of claim 7, wherein the feed combining engine replaces at least one of the distinct feeds in the single stream with other content.

12. The system of claim 7, wherein the feed combining engine replaces at least one of the distinct feeds in the single stream with an advertisement.

13. The system of claim 7, wherein at least one of the plurality of distinct feeds provides content associated with another one of the plurality of distinct feeds, and wherein the at least one of the plurality of distinct feeds is not synchronized to ensure fine grain coherency.

14. The system of claim 7, further comprising configurations options that are accessible by the feed combining engine to determine a number of the plurality of distinct feeds to combine and sources of the plurality of distinct feeds to combine.

15. A method comprising:

receiving, by a processing system including a processor, a single stream comprising a plurality of distinct feeds including a video feed of a live event, and distinct feed data time information, wherein the single stream takes no more bandwidth than one of the plurality of distinct feeds, the single stream further comprising a succession of frames generated at a predetermined interval, each comprising a state of a video object generated from accumulated event data;

determining, by the processing system, by comparing the time information for each of the distinct feeds a relative order of, and a relative time difference among, the plurality of distinct feeds;

determining, by the processing system, a synchronizing point in each of the plurality of distinct feeds at which each is synchronized; and

presenting, by the processing system, the plurality of distinct feeds and the succession of frames, wherein the plurality of distinct feeds is synchronized in time based on the synchronizing point and the succession of frames is not synchronized.

16. The method of claim 15, wherein the synchronizing point in each of the plurality of distinct feeds is associated with a single time value.

17. The method of claim 15, wherein the single stream is presented as a mosaic of the plurality of the distinct feeds.

18. The method of claim 15, further comprising presenting, by the processing system, other content associated with at least one of the distinct feeds in the single stream.

19. The method of claim 15, further comprising presenting, by the processing system, an advertisement associated with at least one of the distinct feeds in the single stream.

20. The method of claim 15, wherein at least one of the plurality of distinct feeds provides content associated with another one of the plurality of distinct feeds, and wherein the at least one of the plurality of distinct feeds is not synchronized to ensure fine grain coherency.