Point of view video processing and curation platform

• US 10,622,020 B2
• Filed: 09/30/2015
• Issued: 04/14/2020
• Est. Priority Date: 10/03/2014
• Status: Active Grant

• Current Assignee: FieldCast, LLC
• Original Assignee: FieldCast, LLC
• Inventors: Anwar, Saeed Z, Tshering, Tenzing P.
• Primary Examiner(s): Pontius, James M
• Application Number:
  US14/870,101
• Publication Number:
  US 20160099025A1
• Time in Patent Office: 1,658 Days
• CPC Class Codes: A42B 3/0433 Detecting, signalling or lighting devices
• CPC Class Codes: G06V 20/41 Higher-level, semantic clustering, classification or understanding of video scenes, e.g. detection, labelling or Markovian modelling of sport events or news items segmenting video sequences G06V20/49
• CPC Class Codes: G06V 20/46 Extracting features or characteristics from the video content, e.g. video fingerprints, representative shots or key frames
• CPC Class Codes: G11B 27/005 Reproducing at a different information rate from the information rate of recording for television signals H04N5/783
• CPC Class Codes: G11B 27/036 Insert-editing
• CPC Class Codes: G11B 27/10 Indexing; Addressing; Timing or synchronising; Measuring tape travel
• CPC Class Codes: H04N 1/00209 Transmitting or receiving image data, e.g. facsimile data, via a computer, e.g. using e-mail, a computer network, the internet, I-fax
• CPC Class Codes: H04N 19/44 Decoders specially adapted therefor, e.g. video decoders which are asymmetric with respect to the encoder
• CPC Class Codes: H04N 21/21805 enabling multiple viewpoints, e.g. using a plurality of cameras
• CPC Class Codes: H04N 21/2187 Live feed
• CPC Class Codes: H04N 21/43072 of multiple content streams on the same device
• CPC Class Codes: H04N 21/84 Generation or processing of descriptive data, e.g. content descriptors systems specially adapted for using meta-information in broadcast systems H04H60/73
• CPC Class Codes: H04N 21/8455 involving pointers to the content, e.g. pointers to the I-frames of the video stream
• CPC Class Codes: H04N 23/698 for achieving an enlarged field of view, e.g. panoramic image capture
• CPC Class Codes: H04N 23/90 Arrangement of cameras or camera modules, e.g. multiple cameras in TV studios or sports stadiums
• CPC Class Codes: H04N 7/181 for receiving images from a plurality of remote sources

Abstract

Embodiments of the present disclosure may provide methods and systems enabled to receive a plurality of streams comprising at least one video stream and at least one telemetry data stream associated with the at least one video stream; and create metadata corresponding to the at least one video stream and the at least one telemetry data stream, wherein creating the metadata comprises: processing at least one frame of the at least one video stream to detect at least one artifact within the frame, processing the at least one telemetry data stream to determine telemetry data associated with the at least one frame of the at least one video stream, and annotating the at least one artifact and the telemetry data associated with the at least one frame.

Claims

1. A method comprising:

receiving a plurality of streams from a plurality of different sources, each of the plurality of streams comprising a video stream, a telemetry data stream, and a location data stream associated with each of the plurality of different sources;

decoding each of the plurality of streams to generate at least one decoded video stream; and

creating metadata corresponding to each of the streams, wherein creating the metadata comprises;

processing at least one frame of the at least one decoded video stream to detect at least one artifact within the frame,

processing the at least one telemetry data stream to determine telemetry data associated with the at least one frame of the at least one decoded video stream,

designating at least one rule for associating the at least one frame with at least one other frame based on at least one of the following;

the at least one artifact detected in the at least one frame, and

the telemetry data associated with the at least one frame, and

annotating the at least one frame to indicate the at least one rule.

2. The method of claim 1, wherein processing the at least one frame of the at least one decoded video stream to detect the at least one artifact within the frame comprises:

accessing a library of comprising a plurality of artifacts, and

determining if any elements in the at least one decoded frame correspond to any one of the plurality of artifacts in the library.

3. The method of claim 2, wherein determining if any elements in the at least one frame correspond to any one of the plurality of artifacts comprises analyzing the at least one decoded frame for at least one of the following:

lighting, shading, physical objects, marked reference points, and text.

4. The method of claim 1, wherein processing the at least one telemetry data stream to determine the telemetry data associated with the at least one frame of the at least one decoded video stream comprises:

synchronizing the at least one decoded video stream with the at least one telemetry data stream.

5. The method of claim 4, wherein synchronizing the at least one decoded video stream with the at least one telemetry data stream comprises:

averaging the telemetry data across a plurality of frames.

6. The method of claim 1, providing the metadata along with the corresponding at least one video stream to at least one rules engine, wherein the metadata provides annotations for frames within the at least one video stream, the annotations enabling the rules engine to perform at least one of the following:

temporal stitching based on the annotations of the frames from the plurality of different sources, and

spatial stitching based on the annotations of the frames from the plurality of different sources.

7. The method of claim 1, further comprising:

processing the plurality of streams to create a logical view, wherein processing the plurality of video streams to create the logical view comprises at least one of the following;

analyzing the metadata associated with each of the plurality of video streams, and

stitching at least two portions of at least two video streams of the plurality of streams based on the set of rules and the analyzed metadata.

8. The method of claim 7, wherein stitching the at least two portions of the at least two video streams comprises at least one of the following:

temporal stitching, and

spatial stitching.

9. The method of claim 1, further comprising:

establishing a plurality of rules associated with a plurality of logical views, wherein each rule specifies required characteristics of the metadata associated with the at least one frame in order to designate each rule to the at least one frame.

10. The method of claim 7, further comprising:

encoding the logical view, and

transmitting the logical view to at least one of the following;

a content distribution network, and

a client device.

11. The method of claim 7, further comprising:

transmitting the logical view, and

displaying the logical view.

12. The method of claim 11, wherein displaying the logical view comprises:

providing a plurality of logical views for selection,

receiving a selection of the logical view, and

displaying the selected logical view.

13. The method of claim 9, further comprising:

receiving the logical view metadata,

receiving the logical view, and

receiving a specification in displaying the logical view.

14. The method of claim 13, wherein receiving the specification in displaying the logical view comprises receiving at least one of the following:

a pan specification,

a zoom specification, and

a directional lock specification.

15. The method of claim 13, wherein receiving the specification in displaying the logical view comprises receiving at least one of the following:

a pause request,

a fast-forward request,

a rewind request, and

a slow-motion request.

16. The method of claim 13, further comprising:

determining a set of rules for providing the desired specification,

processing the logical view based on the logical view metadata and the set of rules, and

displaying a specified logical view.

17. The method of claim 7, further comprising determining the logical view to create based on a received request for the logical view from a client device.

18. A method for creating a logical view comprising:

receiving a plurality of streams from a plurality of sources;

decoding a video data of the plurality of streams;

creating metadata corresponding to the decoded video data, wherein creating the metadata comprises;

scanning at least one decoded video frame to detect at least one object within the at least one decoded video frame,

selecting at least one rule to be applied to the at least one decoded video frame based on the at least one objected detected within the at least one decoded frame, and

annotating the at least one decoded video frame to indicate the at least one rule and a detection of the at least one object,

wherein the metadata is configured to be employed in a curation of a logical view, wherein the logical view is configured to be comprised of video frames from the plurality of sources, wherein the video frames in the logical view are configured to be stitched together based on a processing of the metadata and the at least one rule associated with each video frame.

19. The method of claim 18, wherein selecting the at least one rule comprises, prior to selecting the at least one rule, establishing a plurality of rules associated with a plurality of logical views.

20. The method of claim 19, wherein each rule of the plurality of rules specifies required characteristics of the at least one frame in order to designate each rule to the at least one frame.

21. The method of claim 18, further comprising processing the decoded video data based on each video data frame'"'"'s association with the detected at least one object, wherein processing the decoded video data comprises stitching at least one first video frame of a first stream with at least one second video frame of a second stream.

22. The method of claim 21, wherein stitching comprises temporally stitching the at least one first video frame and the at least one second video frame to generate a constant feed of the detected object.

23. The method of claim 18, wherein receiving the plurality of streams comprises receiving the plurality of streams from a plurality of capturing devices.

24. The method of claim 23, wherein the plurality of capturing devices are associated with a plurality of integrated helmets.

25. The method of claim 24, wherein stitching the at least one first video frame with the at least one second video frame comprises stitching a first content captured from a first integrated helmet with a second content captured from a second integrated helmet.

26. The method of claim 24, further comprising stitching a plurality of single frames received from the plurality of capturing devices associated with a same integrated helmet into panoramic frames.

27. The method of claim 18, wherein receiving the plurality of streams comprises receiving at least one of the following:

audio data,

video data,

telemetry data, and

location data.

28. The method of claim 18, wherein decoding the at least one video stream comprises creating a pixel map for the video frames within the video component.

29. The method of claim 18, further comprising pre-processing the at least one decoded video frame with algorithms configured to perform at least one of the following:

stabilization,

color &

polarization,

linear perspective, relative and known size,

texture gradient,

light and shadow distribution, and

aerial perspective.

30. The method of claim 18, wherein scanning the at least one decoded video frame to detect the at least one object comprises detecting at least one of the following:

a ball;

a football snap;

a player'"'"'s face, a player'"'"'s shirt, a sideline, a line marker, a down marker, an infrared marker, a hash mark, a 5-yard mark, a 10-yard mark, an end, an end-zone, an upright, a game clock, a referee, a brand icon, a team staff, a team owner, a cheerleader, and a fan.