System for presenting audio-video content

• US 7,904,814 B2
• Filed: 12/13/2001
• Issued: 03/08/2011
• Est. Priority Date: 04/19/2001
• Status: Expired due to Fees

• Current Assignee: Sharp Kabushiki Kaisha (Hon Hai Precision Industry Co., Ltd.)
• Original Assignee: Sharp Laboratories of America Incorporated (Hon Hai Precision Industry Co., Ltd.)
• Inventors: Kerofsky, Louis J., Errico, James, van Beek, Petrus J. L., Li, Baoxin, Ferman, Ahmet Mufit
• Primary Examiner(s): Pillai; Namitha
• Application Number:
  US10/016,941
• Publication Number:
  US 20020180774A1
• Time in Patent Office: 3,372 Days
• Field of Search: 715/721, 715/719, 715/720, 715/723, 715/716, 715/727, 715/725, 715/202
• US Class Current: 715/725
• CPC Class Codes: G11B 27/005 Reproducing at a different information rate from the information rate of recording for television signals H04N5/783
• CPC Class Codes: G11B 27/105 of operating discs
• CPC Class Codes: G11B 27/34 Indicating arrangements indicating means incorporated in magazine or cassette G11B23/046 and G11B23/0875; indicating measured values in general G01D

Abstract

A system for presenting a summarization of audio and/or visual content having a plurality of segments to a user together with a graphical user interface that preferably indicates to the viewer the relative temporal position of video segments viewed in the summary within the content from which the summary was derived.

Claims

1. A method of presenting information regarding a video comprising a plurality of frames comprising:

(a) summarizing a video, said summarization comprising a plurality of segments of said video, based upon an event characterized by a semantic event that includes a sports play, where each of said segments includes a plurality of sequential frames of said video;

(b) displaying said summarization in a first portion of a display; and

(c) displaying a graphical user interface on a second portion of said display, simultaneously with said summarization, said interface sequentially indicating the relative location of each of said plurality of segments within said summarization relative to at least one other of said segments as each of said plurality of segments is displayed, each of said plurality of segments represented by a bounded spatial region on said second portion of said display, said bounded spatial region having a respective size based on the number of sequential frames included in the respective segment represented by said bounded spatial region;

(d) displaying to a user said relative location for a first semantic characterization of a said sports play in said video using a first visual indication and displaying said relative location for a second semantic characterization of a said sports play in said video using a second visual indication different from said first visual indication, where said first and second semantic characterizations are each individually distinguishable when their associated visual indications graphically overlap;

(e) receiving from said user, by interaction with said graphical user interface, a selection of one of said plurality of segments;

(f) in response to said selection, presenting a selected one of said plurality of segments and not presenting at least one other of said plurality of segments; and

(g) wherein a user selects a portion of said video not included within said plurality of segments, in response thereto, said system presents said selected portion not included within said plurality of segments, and wherein after presenting said selected portion not included within said plurality of segments presents said selected plurality of segments in temporal order without portions of said video not included within said plurality of segments, and wherein said user selects a portion of said video included within said plurality of segments, in response thereto, said system presents said portion of said video within said plurality of segments.

2. The method of claim 1 wherein said first and second semantic characterizations of a said sports play temporally overlap in said summarization.

3. The method of claim 1 wherein said graphical user interface includes a generally rectangular region where each of said plurality of segments is indicated within said generally rectangular region.

4. The method of claim 1 wherein the size of each of said plurality of segments is indicated in a manner such that said plurality of segments with a greater number of frames are larger than said plurality of segments with a lesser number of frames.

5. The method of claim 4 wherein the size of the regions between each of said plurality of segments is indicated in a manner such that said regions between with a greater number of frames are larger than said plurality of segments with a lesser number of frames.

6. The method of claim 4 where said user selects one of said plurality of segments by interacting with said graphical user interface at a point within the displayed bounded spatial region associated with the selected one of said plurality of segments.

7. The method of claim 6 wherein presentation of a selected one of said plurality of segments begins at the first frame of said segment irrespective of which point within said displayed bounded spatial region that said user interacted with.

8. The method of claim 7 including a user-moveable scroll bar on said graphical user interface indicating the relative temporal location of currently-presented frames of said summary, wherein said user selects one of said plurality of segments by moving said scroll bar over the selected one of said plurality of segments, and where said scroll bar snaps to the beginning of the selected one of said plurality of segments.

9. The method of claim 6 wherein presentation of a selected one of said plurality of segments begins at a frame of said segment temporally corresponding to the point within said displayed bounded spatial region that said user interacted with.

10. The method of claim 6 including a selector by which said user may alternatively select a chosen one of (i) presentation of a selected one of said plurality of segments beginning at the first frame of said segment irrespective of which point within said displayed bounded spatial region that said user interacted with;

and (ii) presentation of a selected one of said plurality of segments beginning at a frame of said segment temporally corresponding to the point within said displayed bounded spatial region that said user interacted with.

11. The method of claim 1 wherein at least two of said plurality of segments are temporally overlapping.

12. The method of claim 11 wherein said temporally overlapping segments are visually indicated in a manner such that each of said overlapping segments are independently identifiable.

13. The method of claim 1 wherein a user selects a portion of said video not included within said plurality of segments, wherein in response thereto, said system presents one of said plurality of segments.

14. The method of claim 13 wherein said one of said plurality of segments is the segment most temporally adjacent to said portion of said video.

15. The method of claim 13 wherein said one of said plurality of segments is the next temporally related segment.

16. The method of claim 13 wherein said one of said plurality of segments is the previous temporally related segment.

17. The method of claim 1 wherein a user selects a portion of said video included within said plurality of segments, wherein in response thereto, said system presents said portion of said video from the start thereof.

18. The method of claim 1 wherein a user selects a portion of said video not included within said plurality of segments, wherein in response thereto, said system presents one of said plurality of segments, and wherein said user selects a portion of said video included within said plurality of segments, wherein in response thereto, said system presents said portion of said video within said plurality of segments.

19. The method of claim 1 wherein a user selects a portion of said video not included within said plurality of segments, wherein in response thereto, said system presents one of said plurality of segments, and wherein said user selects a portion of said video included within said plurality of segments, wherein in response thereto, said system presents said portion of said video within said plurality of segments starting from the beginning thereof.

20. The method of claim 1 wherein said temporal information is hierarchical and is displayed in such a manner to retain a portion of its hierarchical structure.

21. The method of claim 1 wherein said temporal information relates to overlapping time periods and said temporal information is displayed in such a manner to maintain the differentiation of said overlapping time periods.

22. The method of claim 1 wherein said temporal information is displayed within a time line, wherein the temporal information is presented in a plurality of layers in a direction perpendicular to the length of said time line.

23. The method of claim 1 wherein said temporal information is displayed within a time line, wherein textual information is included within said time line.

24. The method of claim 1 wherein said temporal information is displayed within a time line, wherein additional textual information is displayed upon selecting a portion of said time line.

25. The method of claim 1 wherein said temporal information is displayed together with a time line, wherein additional textual information is displayed together with selecting a portion of said time line.

26. The method of claim 1 wherein said temporal information is displayed within a time line, wherein additional audio annotation is presented upon presenting a portion of said time line.

27. A method of presenting information regarding a video comprising a plurality of frames comprising:

(a) identifying a plurality of different segments of said video, where each of said segments includes a plurality of frames of said video;

(b) displaying, simultaneously with a said segment of said video, a graphical user interface including information regarding the temporal location of a said segment relative to at least one other of said segments of said video;

(c) displaying in an interactive display said temporal location for a first semantic characterization of an event in said video using a first visual indication and displaying said temporal location for a second semantic characterization of an event in said video using a second visual indication different from said first visual indication, where said first and second semantic characterizations are each individually distinguishable when their associated visual indications graphically overlap;

(d) displaying to a user at least one selector by which said user may interact with said interactive display to select for viewing selective identified ones of said plurality of segments;

(e) receiving user-selections of identified ones of said plurality of different segments;

(f) presenting user-selected ones of said plurality of different segments; and

(g) wherein a user selects a portion of said video not included within said plurality of different segments, in response thereto, said system presents said selected portion not included within said plurality of different segments, and wherein after presenting said selected portion not included within said plurality of different segments presents said selected plurality of different segments in temporal order without portions of said video not included within said plurality of different segments, and wherein said user selects a portion of said video included within said plurality of different segments, in response thereto, said system presents said portion of said video within said plurality different of segments.

28. The method of claim 27 wherein said graphical user interface includes a generally rectangular region where each of said plurality of segments is indicated within said generally rectangular region.

29. The method of claim 27 wherein the size of each of said plurality of segments is indicated in a manner such that said plurality of segments with a greater number of frames are larger than said plurality of segments with a lesser number of frames.

30. The method of claim 29 wherein the size of the regions between each of said plurality of segments is indicated in a manner such that said regions between with a greater number of frames are larger than said plurality of segments with a lesser number of frames.

31. The method of claim 27 further comprising an indicator that indicates the current position within said temporal information of a currently displayed portion of said video.

32. The method of claim 31 wherein said indicator changes location relative to said temporal information as the portion of said currently displayed portion of said video changes.

33. The method of claim 27 further comprising

(a) indicating with an indicator the current position within said temporal information of a currently displayed portion of said video;

and

(b) modifying the position of said indicator within said temporal information which modifies the displayed portion of said video.

34. The method of claim 33 wherein said indicator is modified to a portion of said video that is not included within said plurality of segments.

35. The method of claim 27 wherein at least two of said plurality of segments are temporally overlapping.

36. The method of claim 35 wherein said temporally overlapping segments are visually indicated in a manner such that each of said overlapping segments are independently identifiable.

37. The method of claim 27 wherein a user selects a portion of said video not included within said plurality of segments, wherein in response thereto, said system presents one of said plurality of segments.

38. The method of claim 37 wherein said one of said plurality of segments is the segment most temporally adjacent to said portion of said video.

39. The method of claim 37 wherein said one of said plurality of segments is the next temporally related segment.

40. The method of claim 37 wherein said one of said plurality of segments is the previous temporally related segment.

41. The method of claim 27 wherein a user selects a portion of said video included within said plurality of segments, wherein in response thereto, said system presents said portion of said video from the start thereof.

42. The method of claim 27 wherein a user selects a portion of said video not included within said plurality of segments, wherein in response thereto, said system presents one of said plurality of segments, and wherein said user selects a portion of said video included within said plurality of segments, wherein in response thereto, said system presents said portion of said video within said plurality of segments.

43. The method of claim 27 wherein a user selects a portion of said video not included within said plurality of segments, wherein in response thereto, said system presents one of said plurality of segments, and wherein said user selects a portion of said video included within said plurality of segments, wherein in response thereto, said system presents said portion of said video within said plurality of segments starting from the beginning thereof.

44. The method of claim 27 wherein said temporal information is hierarchical and is displayed in such a manner to retain a portion of its hierarchical structure.

45. The method of claim 27 wherein said temporal information relates to overlapping time periods and said temporal information is displayed in such a manner to maintain the differentiation of said overlapping time periods.

46. The method of claim 27 wherein said temporal information is displayed within a time line, wherein the temporal information is presented in a plurality of layers in a direction perpendicular to the length of said time line.

47. The method of claim 27 wherein said temporal information is displayed within a time line, wherein textual information is included within said time line.

48. The method of claim 27 wherein said temporal information is displayed within a time line, wherein additional textual information is displayed upon selecting a portion of said time line.

49. The method of claim 27 wherein said temporal information is displayed together with a time line, wherein additional textual information is displayed together with selecting a portion of said time line.

50. The method of claim 27 wherein said temporal information is displayed within a time line, wherein additional audio annotation is presented upon presenting a portion of said time line.

51. The method of claim 27 wherein a user selectable skip function skips a set of frames to a modified location of said video in at least one of a forward temporal direction or a reverse temporal direction, and displays said video at said modified location.

52. The method of claim 27 wherein a user selectable skip function skips to a later temporal segment or a previous temporal segment, and displays said video at said later temporal segment or said previous temporal segment, respectively.

53. The method of claim 27 wherein a user selectable scan function skips a set of frames to a modified location of said video in at least one of a forward temporal direction or a reverse temporal direction, and displays said video at said modified location, and thereafter automatically skips another set of frames to another modified location of said video in at least one of said forward temporal direction or said reverse temporal direction, and displays said video at said another modified location.

54. The method of claim 53 wherein at least one of said forward temporal direction and said reverse temporal direction are consistent with said different segments.

55. The method of claim 54 wherein said display of said video is at the start of the respective one of said different segments.

56. The method of claim 54 wherein said display of said video is at a predetermined offset within the respective one of said different segments.

57. The method of claim 56 wherein said respective image associated with the currently presented said different segments is visually highlighted.

58. The method of claim 27 wherein said graphical user interface displays a respective image associated with at least a plurality of said different segments.

59. The method of claim 58 wherein during presentation of said video said visually highlighted respective images are said highlighted in a substantially regular interval while the sequence of said presentation of said video is at substantially irregular intervals.

60. A method of presenting information regarding audio comprising:

(a) identifying a plurality of different segments of said audio, where each of said segments includes a temporal duration of said audio;

(b) displaying simultaneously with said segment of said audio a graphical user interface including information regarding the temporal location of a said segment relative to at least one other of said segment of said audio;

(c) displaying in an interactive display said temporal location for a first semantic characterization of an event in said audio using a first visual indication and displaying said temporal location for a second semantic characterization of an event in said audio using a second visual indication different from said first visual indication, where said first and second semantic characterizations are each individually distinguishable when their associated visual indications graphically overlap;

(d) displaying to a user at least one selector by which said user may interact with said display to select for listening selective identified ones of said plurality of segments;

(e) receiving user-selections of identified ones of said plurality of different segments; and

(f) presenting user-selected ones of said plurality of different segments; and

(g) wherein a user selects a portion of said audio not included within said plurality of different segments, in response thereto, said system presents said selected portion not included within said plurality of different segments, and wherein after presenting said selected portion not included within said plurality of different segments presents said selected plurality of different segments in temporal order without portions of said audio not included within said plurality of different segments, and wherein said user selects a portion of said audio included within said plurality of different segments, in response thereto, said system presents said portion of said audio within said plurality different of segments.

61. The method of claim 60 further comprising (a) indicating with an indicator the current position within said temporal information of a currently displayed portion of said audio;

and

(b) modifying the position of said indicator within said temporal information which modifies the displayed portion of said audio.

62. The method of claim 61 wherein said indicator is modified to a portion of said audio that is not included within said plurality of segments.

63. The method of claim 60 wherein at least two of said plurality of segments are temporally overlapping.

64. The method of claim 63 wherein said temporally overlapping segments are visually indicated in a manner such that each of said overlapping segments are independently identifiable.

65. The method of claim 60 wherein a user selects a portion of said audio not included within said plurality of segments, wherein in response thereto, said system presents one of said plurality of segments.

66. The method of claim 65 wherein said one of said plurality of segments is the segment most temporally adjacent to said portion of said audio.

67. The method of claim 65 wherein said one of said plurality of segments is the next temporally related segment.

68. The method of claim 62 wherein said one of said plurality of segments is the previous temporally related segment.

69. The method of claim 60 wherein a user selects a portion of said audio included within said plurality of segments, wherein in response thereto, said system presents said portion of said audio from the start thereof.

70. The method of claim 60 wherein a user selects a portion of said audio not included within said plurality of segments, wherein in response thereto, said system presents one of said plurality of segments, and wherein said user selects a portion of said audio included within said plurality of segments, wherein in response thereto, said system presents said portion of said audio within said plurality of segments.

71. The method of claim 60 wherein a user selects a portion of said audio not included within said plurality of segments, wherein in response thereto, said system presents one of said plurality of segments, and wherein said user selects a portion of said audio included within said plurality of segments, wherein in response thereto, said system presents said portion of said audio within said plurality of segments starting from the beginning thereof.

72. The method of claim 60 wherein said temporal information is hierarchical and is displayed in such a manner to retain a portion of its hierarchical structure.

73. The method of claim 60 wherein said temporal information relates to overlapping time periods and said temporal information is displayed in such a manner to maintain the differentiation of said overlapping time periods.

74. The method of claim 60 wherein said temporal information is displayed within a time line, wherein the temporal information is presented in a plurality of layers in a direction perpendicular to the length of said time line.

75. The method of claim 60 wherein said temporal information is displayed within a time line, wherein textual information is included within said time line.

76. The method of claim 60 wherein said temporal information is displayed within a time line, wherein additional textual information is displayed upon selecting a portion of said time line.

77. The method of claim 60 wherein said temporal information is displayed together with a time line, wherein additional textual information is displayed together with selecting a portion of said time line.

78. The method of claim 60 wherein said temporal information is displayed within a time line, wherein additional audio annotation is presented upon presenting a portion of said time line.

79. The method of claim 60 wherein the presentation of said different segments may be modified by a plurality of different functions, and wherein the user may customize another function, not previously explicitly provided, by combining a plurality of said plurality of different functions into a single function.

80. A method of presenting information regarding a video comprising a plurality of frames comprising:

(a) identifying a plurality of different segments of said video, where each of said segments includes a plurality of frames of said video;

(b) displaying, simultaneously with a said segment of said video, a graphical user interface including information regarding the temporal location of a said segment relative to at least one other of said segments of said video;

(c) displaying in an interactive display said temporal location for a first semantic characterization of an event in said video using a first visual indication and displaying said temporal location for a second semantic characterization of an event in said video using a second visual indication different from said first visual indication;

(d) displaying to a user at least one selector by which said user may interact with said interactive display to select for viewing selective identified ones of said plurality of segments;

(e) receiving user-selections of identified ones of said plurality of segments;

(f) presenting user-selected ones of said plurality of different segments;

wherein

(g) wherein a user selects a portion of said video not included within said plurality of segments, and wherein in response thereto, said system presents said selected portion not included within said plurality of segments, and wherein after presenting said selected portion not included within said plurality of segments presents said selected plurality of segments in temporal order without portions of said video not included within said plurality of segments, and wherein said user selects a portion of said video included within said plurality of segments, wherein in response thereto, said system presents said portion of said video within said plurality of segments.