PLAYING AND EDITING LINKED AND ANNOTATED AUDIOVISUAL WORKS

• US 20100293190A1
• Filed: 05/13/2010
• Published: 11/18/2010
• Est. Priority Date: 05/13/2009
• Status: Active Grant

• Current Assignee: Coincident TV Incorporated
• Original Assignee: Coincident TV Incorporated
• Inventors: ROSENBERG, CARL, KAISER, DAVID H., REYES, LUIS, SCHWARTZ, BRUCE
• Granted Patent: US 9,170,700 B2
• Time in Patent Office: Days
• US Class Current: 707/769
• CPC Class Codes: G06F 16/40 of multimedia data, e.g. slideshows comprising image and additional audio data retrieval of still image data G06F16/50; retrieval of audio data G06F16/60; retrieval of video data G06F16/70
• CPC Class Codes: G06F 16/748 Hypervideo linking data to content, e.g. by linking an URL to a video object in the context of video distribution systems H04N21/858
• CPC Class Codes: G06F 3/048 Interaction techniques based on graphical user interfaces [GUI]
• CPC Class Codes: H04N 21/2353 specifically adapted to content descriptors, e.g. coding, compressing or processing of metadata
• CPC Class Codes: H04N 21/4312 involving specific graphical features, e.g. screen layout, special fonts or colors, blinking icons, highlights or animations
• CPC Class Codes: H04N 21/4782 Web browsing , e.g. WebTV
• CPC Class Codes: H04N 21/8456 by decomposing the content in the time domain, e.g. in time segments
• CPC Class Codes: H04N 21/854 Content authoring

Abstract

In one embodiment, a method includes obtaining metadata that relates to a video program and that defines, for one or more specified time points in the video program, one or more web services to be invoked at those time points; generating and displaying, on a computer, a video window that is configured to play the video program; during playing the video program on a computer, detecting that the video program is playing at one of the time points and determining an identifier of a particular web service associated with that one of the time points; in response to the detecting, causing the computer to invoke the particular web service.

Claims

1. A non-transitory computer-readable data storage medium storing instructions which, when executed by one or more processors, cause the one or more processors to perform:

obtaining metadata that relates to a video program and that defines, for one or more specified time points in the video program, one or more web services to be invoked at those time points;

generating and displaying, on a computer, a video window that is configured to play the video program;

obtaining and playing the video program on the computer;

during playing the video program, detecting that the video program is at one of the time points and determining an identifier of a particular web service associated with that one of the time points;

in response to the detecting, causing the computer to invoke the particular web service.

2. The computer-readable data storage medium of claim 1, instructions which when executed cause invoking a particular web service that is configured to obtain a cellular radiotelephone number and to dispatch a text message to the number.

3. The computer-readable data storage medium of claim 1, instructions which when executed cause obtaining metadata that further defines a database query, and invoking the particular web service by submitting the database query to the web service, including invoking the particular web service that is configured to execute the database query with a database, determine a result set, and return the result set in a response to the computer.

4. The computer-readable data storage medium of claim 1, instructions which when executed cause invoking a particular web service that is configured to submit information about the video program over a network to a social networking service.

5. The computer-readable data storage medium of claim 1, further comprising instructions which when executed cause obtaining the video program from any of local storage on the computer, or a first networked server computer that is separate from the computer.

6. The computer-readable data storage medium of claim 1, further comprising instructions which when executed cause obtaining the video program from any of local storage on the computer, or a first networked server computer that is separate from the computer, and obtaining the metadata from a different second networked server computer.

7. A non-transitory computer-readable data storage medium storing instructions which, when executed by one or more processors, cause the one or more processors to perform:

obtaining metadata that relates to a video program and that defines, for one or more specified time points in the video program, one or more annotations to be invoked at those time points, wherein each of the annotations comprises a stored association of a graphical image and a network location identifier;

during playing the video program on a computer, detecting that the video program is playing at a particular time point among the time points;

in response to the detecting, causing the computer to display the graphical image associated with the particular time point, obtain an online electronic document associated with the network location identifier, and display the online electronic document with the video program.

8. The computer-readable data storage medium of claim 7, further comprising instructions which when executed cause the computer, in response to the detecting, to display the video program in a reduced size video window logically layered above a browser window, and to display the online electronic document in the browser window concurrently.

9. The computer-readable data storage medium of claim 7, further comprising instructions which when executed cause obtaining the particular time point that is associated with a song playing as part of the video program, and in response to the detecting, causing the computer to display, as the online electronic document, online information about how to obtain or purchase the song.

10. The computer-readable data storage medium of claim 7, further comprising instructions which when executed cause obtaining the particular time point that is associated with an appearance of a person in the video program, and cause the computer, in response to the detecting, to display, as the online electronic document, online information about the person.

11. The computer-readable data storage medium of claim 7, further comprising instructions which when executed cause obtaining the online information about the person from a remotely located server computer over a network.

12. The computer-readable data storage medium of claim 7, further comprising instructions which when executed cause obtaining, from the metadata, an identifier of an electronic document for a background for the video window;

loading the electronic document over a network;

displaying the electronic document as a background of a player window that includes the video window.

13. A non-transitory computer-readable data storage medium storing instructions which, when executed by one or more processors, cause the one or more processors to perform:

obtaining a link to a stored video program;

obtaining metadata that relates to the video program and that defines, for a specified time point in the video program, one or more annotations to be invoked at the specified time point, wherein each of the annotations comprises an association of a graphic image and a reference to a video program segment or an online electronic document;

causing the computer to play the video program from the link;

during playing the video program on a computer, detecting that the video program is playing at the specified time point;

in response to the detecting, causing the computer to display one or more graphic images associated with the one or more annotations for the specified time point, obtain input selecting one of the graphic images, and initiate playing the video program segment that is associated with the selected one of the graphic images.

14. The computer-readable data storage medium of claim 13, further comprising instructions which when executed cause the computer, in response to the detecting, to display one or more graphic images associated with the one or more annotations for the specified time point, obtain input selecting one of the graphic images, obtain a copy of the online electronic document that is associated with the selected one of the graphic images, display the video program in a reduced size video window logically layered above a browser window, and to display the copy of the online electronic document in the browser window concurrently.

15. The computer-readable data storage medium of claim 13, wherein the video program segment is in the video program.

16. The computer-readable data storage medium of claim 13, wherein the reference to the video program segment is a particular time point in the video program.

17. The computer-readable data storage medium of claim 13, wherein the video program segment is a second video program that is obtainable over a network.

18. The computer-readable data storage medium of claim 13, further comprising instructions which when executed cause displaying the one or more graphic images associated with the one or more annotations for the specified time point continuously after the time point is reached.

19. The computer-readable data storage medium of claim 13, further comprising instructions which when executed cause displaying the one or more graphic images in a border rectangle that is adjacent to a video rectangle, wherein both the border rectangle and video rectangle are within a video window of a player window, and wherein the video program plays within the video rectangle.

20. The computer-readable data storage medium of claim 19, comprising instructions which when executed cause displaying the one or more graphic images in the border rectangle in a position laterally adjacent to the video rectangle.

21. The computer-readable data storage medium of claim 13, further causing instructions which when executed cause the computer to display the one or more graphic images only in response to determining that a user of the computer is a paid subscriber, authenticated, or authorized to view the graphic images.

22. The computer-readable data storage medium of claim 13, wherein the graphic images comprise at least one graphic image that is not selectable and that is associated with a video program segment that is to be available in the future.

23. The computer-readable data storage medium of claim 13, further comprising instructions which when executed cause obtaining the video program from any of local storage on the computer, or a first networked server computer that is separate from the computer, and obtaining the metadata from a different second networked server computer.

24. A non-transitory computer-readable data storage medium storing instructions which, when executed by one or more processors, cause the one or more processors to perform:

obtaining a stored video file having one or more video segments and one or more cue points in the video file;

obtaining a metadata file that contains an internal reference to the video file;

generating and displaying a video window on a display unit;

playing a first video segment of the video file in the video window;

during the playing, in response to reaching a particular cue point among the cue points that are defined in the video file;

obtaining from the metadata file one or more metadata values identifying for the particular cue point a cue point type, and one or more values of attributes that are associated with a particular cue point type, the cue point type and attribute values defining features of an action to perform at the cue point during playing of the video file;

performing the action using the attribute values to determine and perform particular features, displays, or controls associated with the action.

25. The storage medium of claim 24, further comprising instructions which when executed cause generating and displaying in the graphical user interface a graphical directory of the video file in which each of the cue points is a selectable directory element.

26. The storage medium of claim 24, further comprising instructions which when executed cause performing the action by playing a second video segment of the video file that begins at a point other than the cue point.

27. The storage medium of claim 24, further comprising instructions which when executed cause performing the action by displaying images representing a set of video segments or networked resources and prompting a selection of one of the video segments or network resources.

28. The storage medium of claim 27, wherein one of the images represents a web site.

29. The storage medium of claim 24, further comprising instructions which when executed cause performing the action by automatically selecting from among a plurality of other video segments defined in the metadata file based upon a result of a call to a web service that is identified in the metadata file.

30. The storage medium of claim 24, further comprising instructions which when executed cause performing the action by returning to a different cue point and playing another video segment starting at the different cue point.

31. The storage medium of claim 24, further comprising instructions which when executed cause performing the action by displaying a graphical button overlaid in the video window;

receiving user input selecting the graphical button;

in response to the user input, playing a second video segment, then returning to an end of the first video segment and playing a next video segment after the end of the first video segment.

32. The storage medium of claim 24, further comprising instructions which when executed cause performing the action by reading a MXML (Macromedia XML) user interface markup language script from the metadata file, parsing and interpreting the MXML script, and generating and displaying one or more overlay graphical elements in the video window based on the parsing and interpreting.

33. The storage medium of claim 24, further comprising instructions which when executed cause performing the action by automatically pausing the playing the video file, switching to displaying a web browser interface, connecting to a web site, retrieving one or more web pages, and displaying the one or more web pages.

34. The storage medium of claim 33, further comprising instructions which when executed cause, while displaying the web browser interface, receiving user input requesting resuming video playback, and in response to the user input, automatically switching to displaying the first video segment in the video window and resuming playing the first video segment at a point at which the switching to displaying the web browser interface occurred.

35. The storage medium of claim 24, further comprising instructions which when executed cause playing the first video segment, reaching a modal story branch cue point at a first end of the first video segment, displaying a plurality of graphic images representing a plurality of second video segments, receiving user input specifying a selected one of the second video segments, in response to the user input playing the selected one of the second video segments until reaching a “

goto”

type cue point, and playing a third video segment in response to reaching the “

goto”

type cue point.

36. The storage medium of claim 24, further comprising instructions which when executed cause displaying, in the graphical user interface, a timeline that includes a plurality of lines, each line representing one of the video segments, a plurality of points, each of the points representing one of the cue points, and one or more arcs, each of the arcs representing a branch from one of the cue points to one of the video segments.

37. The storage medium of claim 36, further comprising instructions which when executed cause displaying the arcs in the timeline of the graphical user interface based on the cue point type of the one of the cue points.

38. The storage medium of claim 24, further comprising instructions which when executed cause receiving user input requesting related web content, and in response, connecting to a web site that is identified in the metadata file, retrieving one or more web pages, and displaying the one or more web pages in the video window while continuing playing the video file.

39. The storage medium of claim 38, further comprising instructions which when executed cause playing the video file in a reduced size video window that is overlaid on the web pages.

40. A non-transitory computer-readable data storage medium storing instructions which, when executed by one or more processors, cause the one or more processors to perform:

obtaining first data identifying a stored video file having one or more video segments;

creating and storing a list of cue points based upon either user input or predefined cue points in the video file;

obtaining second data identifying one of the cue points of a first video segment;

obtaining one or more metadata values identifying a cue point type, and one or more values of attributes that are associated with a particular cue point type, the cue point type and attribute values defining features of an action to perform at the cue point during playing of the video file;

storing the first data, second data, and metadata values in association with one another in a metadata file that references the video file.

41. The data storage medium of claim 40 further comprising instructions which when executed cause generating and displaying a graphical user interface that comprises all of:

a video player window configured to play the video file;

a list of the cue points;

a cue point type input field; and

one or more input fields and one or more display fields associated with attributes of particular cue point types.

42. The storage medium of claim 41, further comprising instructions which when executed cause generating and displaying in the graphical user interface, at least one of:

an annotation value field configured to receive annotation metadata values that specifies whether a player should generate and display a selectable graphical image with the video file;

a web service field configured to receive a name of a web service that can control selection among a plurality of video segments.

43. The storage medium of claim 40, wherein the cue point type and attribute values specify an operation selected from the group consisting of:

displaying a second video segment;

displaying a set of video segments or networked resources and prompting a selection of one of the video segments or network resources;

selecting from among a plurality of other video segments based upon a result of a call to a web service; and

returning to a different cue point.

44. The storage medium of claim 43, further comprising instructions which when executed cause receiving a web service identifier value that specifies the web service, and storing the web service identifier value in the metadata file.

45. The storage medium of claim 40, wherein the cue point type and attribute values represent any of commands to a video player to play a different video file;

commands to the video player to play a different portion of the same video file;

commands to a web browser client to retrieve and display a file from a web server;

commands to the video player to display a menu of user options.

46. The storage medium of claim 40, further comprising instructions which when executed cause playing and displaying the video file during any of the receiving steps.

47. The storage medium of claim 40 further comprising storing the cue point type and attribute values in the metadata file in extensible markup language (XML) format.

48. The storage medium of claim 40, further comprising instructions which when executed cause receiving an annotation value and storing the annotation value in the metadata file, wherein the annotation value specifies whether a player should generate and display a graphical annotation of the video file in which each of the cue points is a selectable annotation element.

49. The storage medium of claim 40, wherein the cue point type and attribute values specify playing a second video segment of the video file that begins at a point other than the cue point.

50. The storage medium of claim 40, wherein the cue point type and attribute values specify displaying images representing a set of video segments or networked resources and prompting a selection of one of the video segments or network resources.

51. The storage medium of claim 50, wherein one of the images represents a web site.

52. The storage medium of claim 40, wherein the cue point type and attribute values specify automatically selecting from among a plurality of other video segments defined in the metadata file based upon a result of a call to a web service that is identified in the metadata file.

53. The storage medium of claim 40, wherein the cue point type and attribute values specify returning to a different cue point and playing another video segment starting at the different cue point.

54. The storage medium of claim 40, wherein the cue point type and attribute values specify displaying a graphical button overlaid in the video window;

receiving user input selecting the graphical button;

in response to the user input, playing a second video segment, then returning to an end of the first video segment and playing a next video segment after the end of the first video segment.

55. The storage medium of claim 40, wherein the cue point type and attribute values specify reading a MXML (Macromedia XML) user interface markup language script from the metadata file, parsing and interpreting the MXML script, and generating and displaying one or more overlay graphical elements in the video window based on the parsing and interpreting.

56. The storage medium of claim 40, wherein the cue point type and attribute values specify automatically pausing the playing the video file, switching to displaying a web browser interface, connecting to a web site, retrieving one or more web pages, and displaying the one or more web pages.

57. A non-transitory computer-readable data storage medium storing instructions which, when executed by one or more processors, cause the one or more processors to perform:

obtaining first data identifying a stored video file having one or more video segments;

obtaining second data specifying one or more cue points for the video file, wherein each of the cue points comprises an association of a playback time value, a cue point type, and one or more values of attributes that are associated with a particular cue point type, wherein the cue point type and attribute values define an action to perform at the cue point during playing of the video file, wherein the action includes at least one of;

obtaining and displaying a networked electronic document, and invoking a web service;

storing the first data and second data in association with one another in a metadata file.

58. The storage medium of claim 57, further comprising instructions which when executed cause:

obtaining annotation data defining an annotation for one or more cue points, wherein the annotation data comprises an annotation name, annotation type, annotation position, and one or more of text, a graphical image, and a graphical image to display at the time of a user mouse over operation, and wherein the annotation defines information for a video player to display over a video program when the video program is at a particular one of the cue points;

obtaining association data defining an association of the annotation to the particular one of the cue points;

storing the annotation data and the association data in the metadata file.

59. The storage medium of claim 57, further comprising instructions which when executed cause:

obtaining web reference data defining an online electronic document for association with one or more cue points, wherein the web reference data comprises at least a network resource identifier of an electronic document to display in a browser window of a video player at the time of that the one or more cue points is reached in playing the video program;

obtaining association data defining an association of the web reference data to at least one of the cue points;

storing the annotation data and the association data in the metadata file.

60. The storage medium of claim 59, further comprising instructions which when executed cause obtaining, as part of the web reference data, a query string for submitting to an internet search engine for generating search results in which a viewer may have interest.

61. The storage medium of claim 59, further comprising instructions which when executed cause obtaining, as part of the web reference data, an interest URL to which a viewer may be directed at playback time in response to receiving input at a player indicating interest in other information relating to the video program.

62. The storage medium of claim 59, further comprising instructions which when executed cause obtaining, as part of the web reference data, a target at which processing should continue in response to receiving user input indicating closing the electronic document or completing viewing the electronic document.

63. The storage medium of claim 62, wherein the target comprises a second URL.

64. The storage medium of claim 59, further comprising instructions which when executed cause obtaining, as part of the web reference data, web view layout data that defines a visual layout of a browser window and a video window within a player window for use at playback time to concurrently display both the electronic document and the video program.

65. The storage medium of claim 59, further comprising instructions which when executed cause obtaining player layout data that defines a visual layout of a player window for use at playback time to concurrently display both the electronic document and the video program, wherein the player layout data defines a size and position of a browser window and a video window within the player window.

66. The storage medium of claim 59, further comprising instructions which when executed cause obtaining player layout data that defines a visual layout of a player window for use at playback time to concurrently display both the electronic document and the video program, wherein the player layout data defines a size and position of a browser window, a video window, and a background within the player window.

67. The storage medium of claim 66 comprising instructions which when executed cause obtaining player layout data that defines the video window as reduced size and logically over the browser window.

68. The storage medium of claim 66 comprising instructions which when executed cause obtaining player layout data that defines the video window and the browser window as adjacent and both surrounded by the background.

69. The storage medium of claim 57, further comprising instructions which when executed cause storing the metadata file in a first location that is different than a second location of the stored video file.

70. A computer-readable data storage medium storing a metadata file produced by the process of:

receiving first data identifying a stored video file having one or more video segments and one or more cue points in the video file;

reading the video file and forming a list of the cue points that are defined in the video file;

receiving second data identifying one of the cue points of a first video segment of the video data file;

receiving one or more metadata values identifying a cue point type, and one or more values of attributes that are associated with a particular cue point type, wherein the cue point type and attribute values define features of an action to perform at the cue point during playing of the video file;

storing the first data, second data, and metadata values in association with one another in a metadata file that references the video file.

71. The storage medium of claim 70, wherein the metadata file further comprises at least one of:

an annotation value that specifies whether a player should generate and display a graphical annotation of the video file in which each of the cue points is a selectable annotation element;

a name of a web service that can control selection among a plurality of video segments.

72. The storage medium of claim 70, wherein the cue point type and attribute values in the metadata file specify any selected from the group consisting of:

displaying a second video segment;

displaying a set of video segments or networked resources and prompting a selection of one of the video segments or network resources;

selecting from among a plurality of other video segments based upon a result of a call to a web service; and

returning to a different cue point.

73. The storage medium of claim 72, wherein the metadata file further comprises a web service identifier value that specifies the web service.

74. The storage medium of claim 70, wherein the cue point type and attribute values represent any of commands to a video player to play a different video file;

commands to the video player to play a different portion of the same video file;

commands to a web client to retrieve and display a file from a web server;

commands to the video player to display a menu of user options.

75. The storage medium of claim 70 wherein the cue point type and attribute values are stored in the metadata file in extensible markup language (XML) format.

76. The storage medium of claim 70, wherein the metadata file further comprises the cue point type and attribute values specifying playing a second video segment of the video file that begins at a point other than the cue point.

77. The storage medium of claim 70, wherein the metadata file further comprises the cue point type and attribute values specifying displaying images representing a set of video segments or networked resources and prompting a selection of one of the video segments or network resources.

79. The storage medium of claim 70, wherein the metadata file further comprises the cue point type and attribute values specifying automatically selecting from among a plurality of other video segments defined in the metadata file based upon a result of a call to a web service that is identified in the metadata file.

80. The storage medium of claim 70, wherein the cue point type and attribute values specify returning to a different cue point and playing another video segment starting at the different cue point.

81. The storage medium of claim 70, wherein the cue point type and attribute values specify displaying a graphical button overlaid in the video window;

receiving user input selecting the graphical button;

in response to the user input, playing a second video segment, then returning to an end of the first video segment and playing a next video segment after the end of the first video segment.

82. The storage medium of claim 70, wherein the cue point type and attribute values specify reading a MXML (Macromedia XML) user interface markup language script from the metadata file, parsing and interpreting the MXML script, and generating and displaying one or more overlay graphical elements in the video window based on the parsing and interpreting.

83. The storage medium of claim 70, wherein the cue point type and attribute values specify automatically pausing the playing the video file, switching to displaying a web browser interface, connecting to a web site, retrieving one or more web pages, and displaying the one or more web pages.

78. The storage medium of claim 78, wherein one of the images represents a web site.