Printing system with embedded audio/video content recognition and processing

• US 20050008221A1
• Filed: 03/30/2004
• Published: 01/13/2005
• Est. Priority Date: 11/19/2001
• Status: Active Grant

• Current Assignee: Ricoh Company Limited
• Original Assignee: Ricoh Company Limited
• Inventors: Piersol, Kurt W., Graham, Jamey, Hart, Peter E., Van Olst, Daniel G., Hull, Jonathan J., Erol, Berna, Lee, Dar-Shyang
• Granted Patent: US 7,424,129 B2
• Time in Patent Office: Days
• US Class Current: 382/161
• CPC Class Codes: G06F 40/106 Display of layout of documents; Previewing
• CPC Class Codes: G06F 40/169 Annotation, e.g. comment data or footnotes
• CPC Class Codes: G10H 1/00 Details of electrophonic musical instruments
• CPC Class Codes: G10H 2210/086 for transcription of raw audio or music data to a displayed or printed staff representation or to displayable MIDI-like note-oriented data, e.g. in pianoroll format
• CPC Class Codes: G10H 2240/056 MIDI or other note-oriented file format
• CPC Class Codes: G10H 2240/061 MP3, i.e. MPEG-1 or MPEG-2 Audio Layer III, lossy audio compression

Abstract

A printing system includes embedded functionality for performing media content recognition and processing. Based on the results of the content recognition and processing performed on the media content, the printing system can generate a paper-based or other printed representation thereof. The printing system may further produce a corresponding electronic representation of the media content, which in combination with the printed representation can be used to view and access the media content. A printing system may also include a media renderer application having a print functionality for printing selected media content.

Claims

1. A system for printing time-based media based on content therein, the printing system comprising:

an interface for receiving media data;

a content recognition module that segments the media data according to a content recognition algorithm; and

an output system that produces a printed output based on the segmented media data, the printed output including a plurality of samples of the media data that correspond to the segmented media data.

2. The system of claim 1, wherein the content recognition module further generates meta data that describe how the media data are segmented, the meta data associating the printed output with an electronic version of the media data.

3. The system of claim 2, wherein the output system further produces an electronic version of the media data.

4. The system of claim 3, wherein the electronic version of the media data includes the meta data.

5. The system of claim 1, wherein the media data include audio data.

6. The system of claim 1, wherein the media data include video data.

7. The system of claim 6, wherein the content recognition module identifies a plurality of key frames for the video data.

8. The system of claim 7, wherein the printed output includes the identified key frames and, for each key frame, a corresponding indicia that relates the key frame to an electronic version of the media data.

9. The system of claim 8, wherein the indicia for a key frame comprises a machine-readable bar code.

10. The system of claim 8, wherein the indicia for each key frame associates the key frame with its position in the media data.

11. The system of claim 1, wherein the content recognition algorithm performs video event detection.

12. The system of claim 1, wherein the content recognition algorithm performs video foreground/background segmentation.

13. The system of claim 1, wherein the content recognition algorithm performs face detection.

14. The system of claim 1, wherein the content recognition algorithm performs face image matching.

15. The system of claim 1, wherein the content recognition algorithm performs face recognition.

16. The system of claim 1, wherein the content recognition algorithm performs face cataloging.

17. The system of claim 1, wherein the content recognition algorithm performs video text localization.

18. The system of claim 1, wherein the content recognition algorithm performs video optical character recognition.

19. The system of claim 1, wherein the content recognition algorithm performs language translation.

20. The system of claim 1, wherein the content recognition algorithm performs frame classification.

21. The system of claim 1, wherein the content recognition algorithm performs clip classification.

22. The system of claim 1, wherein the content recognition algorithm performs image stitching.

23. The system of claim 1, wherein the content recognition algorithm performs audio reformatting.

24. The system of claim 1, wherein the content recognition algorithm performs speech recognition.

25. The system of claim 1, wherein the content recognition algorithm performs audio event detection.

26. The system of claim 1, wherein the content recognition algorithm performs audio waveform matching.

27. The system of claim 1, wherein the content recognition algorithm performs audio-caption alignment.

28. The system of claim 1, wherein the content recognition algorithm performs caption alignment.

29. A system for printing media content, the system comprising:

a media renderer for viewing a selected media item, the media renderer including a print option;

a printer driver configured to send the selected media item to a printer responsive to user selection of the print option of the media renderer; and

a printer coupled to the printer driver to receive the media item, the printer configured to segment the media item according to a content recognition algorithm and produce a printed output based on the segmented media item, the printed output including a plurality of samples of the media item and information related to the samples.

30. The system of claim 29, wherein the media renderer comprises a computer program product configured to run on a computer system coupled to the printer.

31. The system of claim 29, wherein the media renderer comprises a plug-in software module that provides the print option.

32. The system of claim 29, wherein the printer is configured to generate meta data that describe how the media item is segmented, the meta data associating the printed output with an electronic version of the media item.

33. The system of claim 32, wherein the printer is further configured to produce an electronic version of the media data.

34. The system of claim 33, wherein the electronic version of the media data includes the meta data.

35. The system of claim 29, wherein the media item is an audio file.

36. The system of claim 29, wherein the media item is a video file.

37. The system of claim 29, wherein the printer identifies a plurality of key frames for the media item when the printer segments the media item.

38. The system of claim 37, wherein the printed output includes the identified key frames and, for each key frame, a corresponding indicia that relates the key frame to an electronic version of the media item.

39. The system of claim 38, wherein the indicia for a key frame comprises a machine-readable bar code.

40. The system of claim 38, wherein the indicia for each key frame associates the key frame with its position in the media item.

41. The system of claim 29, wherein the content recognition algorithm performs a function selected from a group consisting of:

video event detection, video foreground/background segmentation, face detection, face image matching, face recognition, face cataloging, video text localization, video optical character recognition, language translation, frame classification, clip classification, image stitching, audio reformatting, speech recognition, audio event detection, audio waveform matching, audio-caption alignment, caption alignment.

42. A method of printing media content, the method comprising:

receiving time-based media data that represent the media content;

a step for segmenting the media data based on the media content, the step for segmenting performed at least in part by a printing system; and

producing a printed output based on the segmented media data, the printed output displaying a plurality of samples of the media content.

43. The method of claim 42, further comprising:

generating meta data that describe how the media data are segmented, the meta data associating the printed output with an electronic version of the media data.

44. The method of claim 43, further comprising:

producing an electronic version of the media data.

45. The method of claim 44, wherein the electronic version of the media data includes the meta data.

46. The method of claim 42, wherein the media data include audio data.

47. The method of claim 42, wherein the media data include video data.

48. The method of claim 47, wherein the step for segmenting the media data includes identifying a plurality of key frames for the video data.

49. The method of claim 48, wherein the printed output includes the identified key frames and, for each key frame, a corresponding indicia that relates the key frame to an electronic version of the media data.

50. The method of claim 49, wherein the indicia for a key frame comprises a machine-readable bar code.

51. The method of claim 49, wherein the indicia for each key frame associates the key frame with its position in the media data.

52. The method of claim 42, wherein the step for segmenting includes a content recognition algorithm that performs a function selected from a group consisting of:

video event detection, video foreground/background segmentation, face detection, face image matching, face recognition, face cataloging, video text localization, video optical character recognition, language translation, frame classification, clip classification, image stitching, audio reformatting, speech recognition, audio event detection, audio waveform matching, audio-caption alignment, caption alignment.