Techniques for generating a static representation for time-based media information

• US 7,703,044 B2
• Filed: 12/23/2004
• Issued: 04/20/2010
• Est. Priority Date: 11/19/2001
• Status: Active Grant

• Current Assignee: Ricoh Company Limited
• Original Assignee: Ricoh Company Limited
• Inventors: Graham, Jamey
• Primary Examiner(s): Bautista; X. L
• Application Number:
  US11/021,303
• Publication Number:
  US 20050216838A1
• Time in Patent Office: 1,944 Days
• Field of Search: 715/700, 715/716, 715/717, 715/719, 715/727, 715/764, 715/835, 715/838, 715/839, 715/855, 715/200, 715/201, 715/202, 715/203, 715/205, 715/234, 715/273, 705/55, 358/1.1, 358/1.15, 358/1.18, 358/530, 358/540, 382/173
• US Class Current: 715/838
• CPC Class Codes: G06F 3/1206 resulting in increased flexibility in input data format or job format or job type
• CPC Class Codes: G06F 3/1268 Job submission, e.g. submitting print job order or request not the print data itself
• CPC Class Codes: G06F 3/1288 in client-server-printer device configuration
• CPC Class Codes: H04N 21/4117 for generating hard copies of the content, e.g. printer, electronic paper interfaces to printers G06F3/12; printing data G06K15/02
• CPC Class Codes: H04N 21/4312 involving specific graphical features, e.g. screen layout, special fonts or colors, blinking icons, highlights or animations
• CPC Class Codes: H04N 21/4314 for fitting data in a restricted space on the screen, e.g. EPG data in a rectangular grid
• CPC Class Codes: H04N 21/44008 involving operations for analysing video streams, e.g. detecting features or characteristics in the video stream arrangements characterised by components specially adapted for monitoring, identification or recognition of video in broadcast systems H04H60/59
• CPC Class Codes: H04N 21/440236 by media transcoding, e.g. video is transformed into a slideshow of still pictures, audio is converted into text
• CPC Class Codes: H04N 21/4532 involving end-user characteristics, e.g. viewer profile, preferences monitoring of user activities for profile generation for accessing a video database G06F16/739; user profiles in network data switching protocols H04L67/306; processing of user preferences or user profiles in wireless networks H04W8/18
• CPC Class Codes: H04N 21/4755 for defining user preferences, e.g. favourite actors or genre retrieval personalisation and generation of user profiles for the retrieval of video data G06F16/739; user profiles in network data switching protocols H04L67/306
• CPC Class Codes: H04N 21/8456 by decomposing the content in the time domain, e.g. in time segments
• CPC Class Codes: H04N 7/163 by receiver means only

Abstract

Techniques for generating a static representation of time-based media information. A static representation is generated that comprises a timeline representing the duration of the time-based media information. Occurrences of one or more events that occur in the time-based representation are indicated along the timeline in the static representation. The static representation may be printed on a paper medium.

Claims

1. A computer-implemented method of generating static representations of time-based multimedia information, the method comprising:

receiving, at a data processing system, time-based multimedia information electronically stored in one or more multimedia files, the time-based multimedia information characterized by a start time and an end time;

receiving, at the data processing system, event criteria specifying one or more types of multimedia events that occur in multimedia information;

receiving, at the data processing system, textual information;

analyzing, by a processor of the data processor system, the time-based multimedia information in response to the textual information to identify using one or more content recognition techniques audio information or video information in the time-based multimedia information that includes the textual information;

determining, by the processor of the data processing system, a set of multimedia events occurring in the identified audio information or video information that satisfy the event criteria, each multimedia event in the set of multimedia events being of the specified one or more types of multimedia events and occurring as a multimedia event in the time-based multimedia information between a first time and a second time;

determining, by the processor of the data processing system, for each multimedia event in the set of multimedia events occurring in the time-based multimedia information between the first time and the second time, a time between the first time and the second time when the multimedia event occurs in the time-based multimedia information;

generating, by the processor of the data processing system, a static representation for the time-based multimedia information, wherein the static representation comprises;

a timeline having a start point corresponding to the first time and an end point corresponding to the second time;

an indication along the timeline, for each multimedia event in the set of multimedia events occurring in the time-based multimedia information between the first time and the second time, of the time between the first time and the second time when the multimedia event occurs in the time-based multimedia information;

a tag for each multimedia event in the set of multimedia events occurring in the time-based multimedia information between the first time and the second time, each tag for a multimedia event corresponding to the time determined between the first time and the second time for the multimedia event in the time-based multimedia information, wherein each tag is configured to be readable by a reading device associated with a computer system that enables access based on the tag to the time-based media information stored in the multimedia document substantially from the time corresponding to the tag; and

printing the static representation using the data processing device on a paper medium.

2. The method of claim 1 further comprising:

generating a visual representation for each multimedia event in the set of multimedia events being of the specified one or more types of multimedia events and occurring in the time-based multimedia information between the first time and the second time;

wherein the static representation for the time-based multimedia information comprises the visual representation for each multimedia event in the set of multimedia events; and

wherein the indication along the timeline, for each multimedia event in the set of multimedia events being of the specified one or more types of multimedia events and occurring in the time-based multimedia information between the first time and the second time, of the time between the first time and the second time when the multimedia event occurs in the time-based multimedia information comprises a mark on the timeline corresponding to the time determined for the multimedia event and information correlating the mark on the timeline to the visual representation generated for the multimedia event.

3. The method of claim 1 further comprising:

receiving information at the computer system from the reading device indicating selection of a first tag associated with the static representation printed on the paper medium;

determining, using the computer system, a first time corresponding to the first tag; and

initiating playback of a portion of the time-based multimedia information using the computer system from the first time.

4. A computer-implemented method of generating a representation of time-based media information, the method comprising:

receiving, at a data processing system, time-based multimedia information electronically stored in one or more multimedia files, the time-based multimedia information characterized by a start time and an end time;

receiving, at the data processing system, information indicative of one or more search terms;

analyzing, by a processor of the data processing system, the time-based multimedia information in response to the one or more search terms to identify using one or more content recognition techniques portions of content in the time-based multimedia information that include at least one of the one or more search terms;

determining, by the processor of the data processing system, a set of multimedia events occurring in the time-based multimedia information to be included in a static representation of the time-based multimedia information based on the portions of content in the time-based multimedia information that include at least one of the one or more search terms, the set of multimedia events comprising one or more multimedia events being of one or more types of multimedia events and occurring in the time-based media information between a first time and a second time;

determining, by the processor of the data processing system, for each multimedia event in the set of multimedia events occurring in the time-based multimedia information between the first time and the second time, a time between the first time and the second time when the multimedia event occurs in the time-based multimedia information;

generating, by the processor of the data processing system, a static representation for the time-based multimedia information, wherein the static representation comprises;

a timeline having a start point corresponding to the first time and an end point corresponding to the second time, the timeline comprising a plurality of sections, each timeline section corresponding to a segment of time between the first time and the second time, each timeline section having an associated start time and an end time;

an indication along the timeline, for each multimedia event in the set of multimedia events occurring in the time-based multimedia information between the first time and the second time, of the time between the first time and the second time when the multimedia event occurs in the time-based multimedia information;

a tag for each multimedia event in the set of multimedia events occurring in the time-based multimedia information between the first time and the second time, each tag for a multimedia event corresponding to the time determined between the first time and the second time for the multimedia event, wherein each tag is configured to be readable by a reading device associated with a computer system that enables access based on the tag to the time-based multimedia information from the time corresponding to the tag; and

printing the static representation using the data processing device on a paper medium.

5. The method of claim 4 wherein determining, by the processor of the data processing system, the set of multimedia events occurring in the time-based multimedia information to be included in a static representation of the time-based multimedia information based on the portions of content in the time-based multimedia information that include at least one of the one or more search terms comprises:

determining, by the processor of the data processing system, a plurality of multimedia events occurring in the time-based multimedia information;

determining, by the processor of the data processing system, for each timeline section, one or more multimedia events in the plurality of multimedia events that occur during the time segment corresponding to the timeline section; and

selecting, by the processor of the data processing system, for each timeline section, at most a threshold number of multimedia events from the one or more multimedia events determined for the timeline section; and

wherein the set of multimedia events identified by the data processing device to be included in the static representation comprises multimedia events selected by the data processing device for the plurality of timeline sections.

6. The method of claim 5 wherein selecting, by the processor of the data processing system, at most a threshold number of events for each timeline section comprises:

determining, by the processor of the data processing device, whether the number of multimedia events in the one or more multimedia events determined to occur during the time segment corresponding to the timeline section exceeds the threshold number; and

selecting, by the processor of the data processing device, upon determining that the number of multimedia events in the one or more multimedia events exceeds the threshold number, at most the threshold number of multimedia events from the one or more multimedia events that are determined to occur during the time segment corresponding to the timeline section.

7. A system for generating a representation of time-based media information, the system comprising:

a processor; and

a printer;

wherein the processor is configured to;

receive time-based multimedia information electronically stored in one or more multimedia files, the time-based multimedia information characterized by a start time and an end time;

receive event criteria specifying one or more types of multimedia events that occur in multimedia information;

receive textual information;

analyze the time-based multimedia information in response to the textual information to identify using one or more content recognition techniques audio information or video information in the time-based multimedia information that includes the textual information;

determine a set of multimedia events occurring in the identified audio information or video information that satisfy the event criteria, each multimedia event in the set of multimedia events being of the specified one or more types of multimedia events and occurring in the time-based multimedia information between a first time and a second time;

determine, for each multimedia event in the set of multimedia events occurring in the time-based multimedia information between the first time and the second time, a time between the first time and the second time when the multimedia event occurs in the time-based multimedia information;

generate a static representation for the time-based multimedia information, wherein the static representation comprises;

a timeline having a start point corresponding to the first time and an end point corresponding to the second time;

an indication along the timeline, for each multimedia event in the set of multimedia events occurring in the time-based multimedia information between the first time and the second time, of the time between the first time and the second time when the multimedia event occurs in the time-based multimedia information;

a tag for each multimedia event in the set of multimedia events occurring in the time-based multimedia information between the first time and the second time, each tag for a multimedia event corresponding to the time determined between the first time and the second time for the multimedia event in the time-based multimedia information, wherein each tag is configured to be readable by a reading device associated with a computer system that enables access based on the tag to the time-based media information stored in the multimedia document substantially from the time corresponding to the tag; and

wherein the printer is configured to print the static representation on a paper medium.

8. The system of claim 7 wherein:

the processor is further configured to generate a visual representation for each multimedia event in the set of multimedia events being of the specified one or more types of multimedia events and occurring in the time-based multimedia information between the first time and the second time;

the static representation for the time-based multimedia information comprises the visual representation for each multimedia event in the set of multimedia events; and

the indication along the timeline, for each multimedia event in the set of multimedia events being of the specified one or more types of multimedia events and occurring in the time-based multimedia information between the first time and the second time, of the time between the first time and the second time of when the multimedia event occurred comprises a mark on the timeline corresponding to the time determined for the multimedia event and information correlating the mark on the timeline to the visual representation generated for the event.

9. The system of claim 7 wherein the processor is configured to:

receive information from the reading device indicating selection of a first tag associated with the static representation printed on the paper medium;

determine a first time corresponding to the first tag; and

initiate playback of a portion of the time-based multimedia information from the first time.

10. A system for generating a representation of time-based media information, the system comprising:

a processor; and

a printer;

wherein the processor is configured to;

receive time-based multimedia information electronically stored in one or more multimedia files, the time-based multimedia information characterized by a start time and an end time;

receive information indicative of one or more search terms;

analyze the time-based multimedia information in response to the one or more search terms to identify using one or more content recognition techniques portions of content in the time-based multimedia information that include at least one of the one or more search terms;

determine a set of multimedia events occurring in the time-based multimedia information to be included in a static representation of the time-based multimedia information, the set of multimedia events comprising one or more multimedia events being of one or more types of multimedia events and occurring in the time-based media information between a first time and a second time;

determine, for each multimedia event in the set of multimedia events occurring in the time-based multimedia information between the first time and the second time, a time between the first time and the second time when the multimedia event occurs in the time-based multimedia information;

generate a static representation for the time-based multimedia information, wherein the static representation comprises;

a timeline having a start point corresponding to the first time and an end point corresponding to the second time, the timeline comprising a plurality of sections, each timeline section corresponding to a segment of time between the first time and the second time, each timeline section having an associated start time and an end time;

an indication along the timeline, for each multimedia event in the set of multimedia events occurring in the time-based multimedia information between the first time and the second time, of the time between the first time and the second time when the multimedia event occurs in the time-based multimedia information;

a tag for each multimedia event in the set of multimedia events occurring in the time-based multimedia information between the first time and the second time, each tag for a multimedia event corresponding to the time determined between the first time and the second time for the multimedia event, wherein each tag is configured to be readable by a reading device associated with a computer system that enables access based on the tag to the time-based multimedia information from the time corresponding to the tag; and

wherein the printer is configured to print the static representation on a paper medium.

11. The system of claim 10 wherein:

the processor is configured to determine the set of multimedia events by;

determining a plurality of multimedia events occurring in the time-based multimedia information;

determining, for each timeline section, one or more multimedia events from the plurality of multimedia events that occur during the time segment corresponding to the timeline section; and

selecting, for each timeline section, at most a threshold number of multimedia events from the one or more events determined for the timeline section; and

the set of multimedia events determined to be included in the static representation comprises multimedia events selected for the plurality of timeline sections.

12. The system of claim 11 wherein the processor is configured to select at most a threshold number of multimedia events for each timeline section by:

determining whether the number of multimedia events in the one or more multimedia events determined to occur during the time segment corresponding to the timeline section exceeds the threshold number; and

selecting, upon determining that the number of multimedia events in the one or more multimedia events exceeds the threshold number, at most the threshold number of multimedia events from the one or more multimedia events that are determined to occur during the time segment corresponding to the timeline section.

13. A computer-readable storage medium storing a plurality of instructions for controlling a data processor to generate a representation of time-based media information, the computer-readable storage medium comprising:

instructions that cause the data processor to receive time-based multimedia information electronically stored in one or more multimedia files, the time-based multimedia information characterized by a start time and an end time;

instructions that cause the data processor to receive event criteria specifying one or more types of multimedia events that occur in multimedia information;

instructions that cause the data processor to receive textual information;

instructions that cause the data processor to analyze the time-based multimedia information in response to the textual information to identify using one or more content recognition techniques audio information or video information in the time-based multimedia information that includes the textual information;

instructions that cause the data processor to determine a set of multimedia events occurring in the identified audio information or video information that satisfy the event criteria, each multimedia event in the set of multimedia events being of the specified one or more types of multimedia events and occurring in the time-based multimedia information between a first time and a second time;

instructions that cause the data processor to determine, for each multimedia event in the set of multimedia events occurring in the time-based multimedia information between the first time and the second time, a time between the first time and the second time when the multimedia event occurs in the time-based multimedia information;

instructions that cause the data processor to generate a static representation for the time-based multimedia information, wherein the static representation comprises;

a timeline having a start point corresponding to the first time and an end point corresponding to the second time;

an indication along the timeline, for each multimedia event in the set of multimedia events occurring in the time-based multimedia information between the first time and the second time, of the time between the first time and the second time when the multimedia event occurs in the time-based multimedia information;

a tag for each multimedia event in the set of multimedia events occurring in the time-based multimedia information between the first time and the second time, each tag for a multimedia event corresponding to the time determined between the first time and the second time for the multimedia event in the time-based multimedia information, wherein each tag is configured to be readable by a reading device associated with a computer system that enables access based on the tag to the time-based media information stored in the multimedia document substantially from the time corresponding to the tag; and

instructions that cause the data processor to initiate printing of the static representation on a paper medium.

14. A computer-readable storage medium storing a plurality of instructions for controlling a data processor to generate a representation of time-based media information, the computer-readable storage medium comprising:

instructions that cause the data processor to receive time-based multimedia information electronically stored in one or more multimedia files, the time-based multimedia information characterized by a start time and an end time;

instructions that cause the data processor to receive information indicative of one or more search terms;

instructions that cause the data processor to analyze the time-based multimedia information in response to the one or more search terms to identify using one or more content recognition techniques portions of content in the time-based multimedia information that include at least one of the one or more search terms;

instructions that cause the data processor to determine a set of multimedia events occurring in the time-based multimedia information to be included in a static representation of the time-based multimedia information based on the portions of content in the time-based multimedia information that include at least one of the one or more search terms, the set of multimedia events comprising one or more multimedia events being of one or more types of multimedia events and occurring in the time-based media information between a first time and a second time;

instructions that cause the data processor to determine, for each multimedia event in the set of multimedia events occurring in the time-based multimedia information between the first time and the second time, a time between the first time and the second time when the multimedia event occurs in the time-based multimedia information;

instructions that cause the data processor to generate a static representation for the time-based multimedia information, wherein the static representation comprises;

a timeline having a start point corresponding to the first time and an end point corresponding to the second time, the timeline comprising a plurality of sections, each timeline section corresponding to a segment of time between the first time and the second time, each timeline section having an associated start time and an end time;

an indication along the timeline, for each multimedia event in the set of multimedia events occurring in the time-based multimedia information between the first time and the second time, of the time between the first time and the second time when the multimedia event occurs in the time-based multimedia information;

a tag for each multimedia event in the set of multimedia events occurring in the time-based multimedia information between the first time and the second time, each tag for a multimedia event corresponding to the time determined between the first time and the second time for the multimedia event, wherein each tag is configured to be readable by a reading device associated with a computer system that enables access based on the tag to the time-based multimedia information from the time corresponding to the tag; and

instructions that cause the data processor to initiate printing of the static representation on a paper medium.

15. The computer-readable storage medium of claim 14 wherein:

the instructions that cause the data processor to determine the set of multimedia events comprise;

instructions that cause the data processor to determine a plurality of multimedia events occurring in the time-based multimedia information;

instructions that cause the data processor to determine, for each timeline section, one or more multimedia events in the plurality of multimedia events that occur during the time segment corresponding to the timeline section; and

instructions that cause the data processor to select, for each timeline section, at most a threshold number of multimedia events from the one or more multimedia events determined for the timeline section; and

the set of multimedia events determined to be included in the static representation comprises multimedia events selected for the plurality of timeline sections.

16. The computer-readable storage medium of claim 15 wherein the instructions that cause the data processor to select at most a threshold number of multimedia events for each timeline section comprise:

instructions that cause the data processor to determine whether the number of multimedia events in the one or more multimedia events determined to occur during the time segment corresponding to the timeline section exceeds the threshold number; and

instructions that cause the data processor to select, upon determining that the number of multimedia events in the one or more multimedia events exceeds the threshold number, at most the threshold number of multimedia events from the one or more multimedia events that are determined to occur during the time segment corresponding to the timeline section.