Synchronization of video and data

• US 20030182620A1
• Filed: 05/22/2002
• Published: 09/25/2003
• Est. Priority Date: 03/19/2002
• Status: Active Grant

• Current Assignee: Sharp Kabushiki Kaisha (Hon Hai Precision Industry Co., Ltd.)
• Original Assignee: Sharp Kabushiki Kaisha (Hon Hai Precision Industry Co., Ltd.)
• Inventors: Errico, James, Li, Baoxin
• Granted Patent: US 8,214,741 B2
• Time in Patent Office: Days
• US Class Current: 715/500.1
• CPC Class Codes: H04N 21/23614 Multiplexing of additional data and video streams arrangements for simultaneous broadcast of plural pieces of information H04H20/28
• CPC Class Codes: H04N 21/4348 Demultiplexing of additional data and video streams
• CPC Class Codes: H04N 21/84 Generation or processing of descriptive data, e.g. content descriptors systems specially adapted for using meta-information in broadcast systems H04H60/73
• CPC Class Codes: H04N 21/8547 involving timestamps for synchronizing content
• CPC Class Codes: H04N 21/8549 Creating video summaries, e.g. movie trailer retrieval in video databases by using presentations in form of a video summary G06F16/739

Abstract

A system including a video stream and a data stream. The system synchronizes the data stream to different portions of the video stream.

Claims

1. A method of synchronizing information descriptive of the content of a video with said video comprising:

(a) identifying a plurality of segments of said video, wherein at least one of said segments contains a sports activity, wherein said plurality of segments includes fewer frames than said video;

(b) identifying first synchronization data from at least one of said identifying said plurality of segments and processing of said video;

(c) identifying second synchronization data from said information;

(d) associating portions of said information with different said segments based upon said first and second synchronization data in a manner free from manual association of said portions of said information with said different said segments.

2. The method of claim 1 wherein said video is a summarization of another video.

3. The method of claim 1 wherein said first synchronization data from said processing of said video includes scoring information.

4. The method of claim 2 wherein said first synchronization data from said identifying said plurality of events results from said summarization of said video.

5. The method of claim 4 wherein said summarization is automatically generated based upon said another video.

6. A method of synchronizing information descriptive of the content of a video with said video comprising:

(a) identifying a plurality of events of said video, wherein at least one of said events contains a sports activity;

(b) identifying first synchronization data from at least one of said identifying said plurality of events and processing of said video;

(c) identifying second synchronization data from said information;

(d) associating portions of said information with different said events based upon said first and second synchronization data in a manner free from manual association of said portions of said information with said different said events.

7. The method of claim 6 wherein said plurality of events includes fewer frames than said video.

8. The method of claim 6 wherein said video is a summarization of another video.

9. The method of claim 6 wherein said first synchronization data from said processing of said video includes scoring information.

10. The method of claim 6 wherein said first synchronization data from said identifying said plurality of events results from the summarization of said video.

11. The method of claim 6 wherein a first portion of said information is associated with a first portion of said video, wherein a second portion of said information is associated with a second portion of said video, wherein a third portion of said information temporally between said first portion and said second portion of said information is associated with a third portion of said video temporally between said first portion and said second portion of said video.

12. The method of claim 11 wherein said first and second portions of said information are associated with said first and second portions of said video with a degree of confidence greater than the confidence of said third portion of said information associated with said third portion of said video.

13. The method of claim 6 wherein a first portion of said information is associated with a first portion of said video, wherein a second portion of said information is associated with a second portion of said video, wherein a third portion of said information temporally between said first portion and said second portion of said information is free from being associated with a third portion of said video temporally between said first portion and said second portion of said video.

14. The method of claim 6 wherein a first portion of said information is associated with a first portion of said video with a first degree of confidence, wherein a second portion of said information is associated with a second portion of said video with a second degree of confidence, wherein a third portion of said information is associated with a third portion of said video with a third degree of confidence, wherein at least two of said first, second, and third degrees of confidence are different.

15. The method of claim 14 wherein said first degree of confidence is based, at least in part, upon the likelihood of proximity to the proper associated portion of said video.

16. A method of synchronizing information descriptive of the content of a video with said video comprising:

(a) identifying first synchronization data from said video;

(b) identifying second synchronization data from said information; and

(c) associating portions of said information with different portions of said video based upon said first and second synchronization data in a manner free from manual association of said portions of said information with said different said events, wherein said data is characterized by at least one of;

(i) said video and said data having a different time base;

(ii) said video and said data having a different frequency.

17. The method of claim 16 wherein said video is a summarization of another video.

18. The method of claim 17 wherein said another video is characterized by at least one of:

(i) said another video and said video having a different time base; and

(ii) said another video and said video having a different frequency;

19. The method of claim 16 wherein said video includes sporting events.

20. The method of claim 16 wherein said first synchronization data from said processing of said video includes scoring information.

21. The method of claim 16 wherein said first synchronization data from said identifying said plurality of events results from the summarization of said video.

22. A method of synchronizing information descriptive of the content of a video with said video comprising:

(a) identifying first synchronization data from said video;

(b) identifying second synchronization data from said information;

(c) associating portions of said information with different portions of said video based upon said first and second synchronization data in a manner free from manual association of said portions of said information with said different said events, wherein associating is performed in a non-deterministic manner.

23. The method of claim 22 wherein said data is characterized by at least one of:

(i) said video and said data having a different time base;

(ii) said video and said data having a different frequency.

24. The method of claim 22 wherein said video is a summarization of another video.

25. The method of claim 24 wherein said another video is characterized by at least one of:

(i) said another video and said video having a different time base; and

(ii) said another video and said video having a different frequency;

26. The method of claim 22 wherein said video includes sporting events.

27. A system for synchronizing information descriptive of the content of a video with said video comprising:

(a) a video analysis module identifies a plurality of segments of said video, wherein at least one of said segments contains a sports activity, wherein said plurality of segments includes fewer frames than said video;

(b) a video pre-process module that identifies first synchronization data from said identification of said plurality of segments;

(c) a filter module that identifies second synchronization data in the form of scoring information from processing of said video;

(d) a data pre-process module that identifies third synchronization data from said information;

(e) a synchronization module that associates portions of said information with different said segments based upon said first, second, and third synchronization data in a manner free from manual association of said portions of said information with said different said segments.

28. A method of analyzing video comprising:

(a) identifying a plurality of events of said video, wherein at least one of said events contains a sports activity;

(b) identifying a region of at least one frame of said video containing scoring related information;

(c) identifying at least one value of said scoring related information based upon character recognition; and

(d) associating said at least one value with at least one of said events.

29. The method of claim 28 further comprising:

(a) identifying a region of at least another one frame of said video containing additional scoring related information;

(b) identifying at least one value of said additional scoring related information based upon character recognition;

(c) associating said at least one value of said additional scoring related information with at least another one of said events; and

(d) modifying at least a portion of said scoring related information based upon said scoring related information.

30. The method of claim 29 further comprising:

(a) identifying a region of at least another one frame of said video containing additional scoring related information;

(b) identifying at least one value of said additional scoring related information based upon character recognition; and

(c) associating said at least one value of said additional scoring related information with said at least one of said events.

31. The method of claim 29 further comprising low pass filtering said scoring related information and said additional scoring related information.

32. The method of claim 29 further comprising analyzing said scoring related information and said additional scoring related information to determine appropriate scoring related information.

33. The method of claim 29 further comprising modifying at least one of said scoring related information and said additional scoring related information based upon the type of content of said video.

34. A method of using a video with associated data comprising:

(a) providing a video that includes a plurality of segments wherein a plurality of said segments include associated data descriptive of the content thereof; and

(b) modifying at least one of a transmission and a storage of a plurality of said segments of said video based upon said associated data.

35. The method of claim 34 wherein said modification includes allocating an unequal number of bits, based at least in part upon said associated data, to said segments.

36. The method of claim 35 wherein said segments whose contents are relatively less in importance are allocated relatively less number of said bits than segments whose contents are greater in relative importance.

37. The method of claim 35 wherein modification includes employing unequal error protection in error prone channels.

38. The method of claim 37 wherein said relatively higher error protection is applied to relatively higher priority segments.

39. The method of claim 37 further comprising retransmission of relatively important segments in case of data loss.

40. The method of claim 37 further comprising modifying a transmitter power.

41. The method of claim 40 wherein said modifying said transmitter power is responsive to power restrictions of a transmitting device.

42. The method of claim 40 wherein said modifying said transmitter power includes dropping segments of less importance.

43. A method of presenting a video with associated data comprising:

(a) providing a video that includes a plurality of segments wherein a plurality of said segments include associated data descriptive of the content thereof, wherein said associated data was originally unsynchronized to said video; and

(b) presenting said video to a viewer in a manner such that while a selected one of said segments is being said presented said associated data to said selected one of said segments is simultaneously presented to said viewer.

44. The method of claim 43 wherein said video includes sporting activity.

45. The method of claim 43 further comprising synchronizing said associated data with said plurality of segments.

46. A method of presenting a video with associated data comprising:

(a) providing a video that includes a plurality of segments wherein a plurality of said segments include associated data descriptive of the content thereof;

(b) presenting said video to a viewer in a manner such that while a selected one of said segments is being said presented said associated data to said selected one of said segments is simultaneously presented to said viewer; and

(c) selecting a previously viewed segment of said video for presentation to said viewer and simultaneously presenting to said viewer said associated data to said previously viewed segment.

47. The method of claim 46 wherein said video includes sporting activity.

48. The method of claim 46 further comprising synchronizing said associated data with said plurality of segments.

49. A method of presenting a video with associated data comprising:

(a) providing a video that includes a plurality of segments wherein a plurality of said segments include associated data descriptive of the content thereof; and

(b) modifying the amount of said associated data that is presented to a viewer based upon a user'"'"'s preferences.

50. The method of claim 49 further comprising a selector that results in said modifying.

51. A method of presenting a video with associated data comprising:

(a) providing a video that includes a plurality of segments wherein a plurality of said segments include associated data descriptive of the content thereof; and

(b) modifying the amount of said associated data that is presented to a viewer based upon, at least in part, the duration of said segment.

52. The method of claim 51 wherein a segment of a relatively shorter duration is presented relatively less said associated data.

53. A method of presenting a video with associated data comprising:

(a) providing a video that includes a plurality of segments wherein a plurality of said segments include associated data descriptive of the content thereof; and

(b) modifying the speed at which said associated data that is presented to a viewer based upon at least one of the duration of said segment and the amount of said associated data.

54. The method of claim 53 wherein said speed is based upon said duration.

55. The method of claim 53 wherein said speed is based upon said amount.

56. A method of presenting a video with associated data comprising:

(a) providing a video that includes a plurality of segments wherein a plurality of said segments include associated data descriptive of the content thereof; and

(b) modifying the associated data that is presented a viewer based upon a profile of the viewer.

57. A method of presenting a video with associated data comprising:

(a) providing a video that includes a plurality of segments wherein a plurality of said segments include associated data descriptive of the content thereof; and

(b) modifying the number of lines of said associated data that are presented to a viewer.

58. The method of claim 57 wherein a viewer may modify said number of lines.

59. A method of using a video with associated data comprising:

(a) providing a video that includes a plurality of segments wherein a plurality of said segments include associated data descriptive of the content thereof; and

(b) storing said video in a database wherein said video is indexed using said associated data.

60. The method of claim 59 further comprising placing queries for specific play events and retrieving from said database corresponding video segments containing said play events.