Television-based visualization and navigation interface

• US 7,954,056 B2
• Filed: 06/17/2002
• Issued: 05/31/2011
• Est. Priority Date: 12/22/1997
• Status: Expired due to Fees

• Current Assignee: Ricoh Company Limited
• Original Assignee: Ricoh Company Limited
• Inventors: Graham, Jamey
• Primary Examiner(s): Huynh; Ba
• Assistant Examiner(s): TRAN, MYLINH T
• Application Number:
  US10/174,522
• Publication Number:
  US 20030184598A1
• Time in Patent Office: 3,270 Days
• Field of Search: 715/716, 715/810, 715/853, 715/719, 715/721, 715/836, 348/13, 348/906, 348/654
• US Class Current: 715/716
• CPC Class Codes: G06F 16/30 of unstructured textual data document management systems G06F16/93
• CPC Class Codes: G06F 16/34 Browsing; Visualisation therefor
• CPC Class Codes: G06F 16/40 of multimedia data, e.g. slideshows comprising image and additional audio data retrieval of still image data G06F16/50; retrieval of audio data G06F16/60; retrieval of video data G06F16/70
• CPC Class Codes: G06F 16/483 using metadata automatically derived from the content
• CPC Class Codes: G11B 27/034 on discs G11B27/036, G11B27/038 take precedence
• CPC Class Codes: G11B 27/105 of operating discs

Abstract

Techniques for retrieving and displaying multimedia information are provided. A television (TV) interface is provided that displays multimedia information that may be stored in a multimedia document. The interface enables a user to navigate through multimedia information stored on the multimedia document.

Claims

1. A method for displaying a multimedia document on a television display, the multimedia document being stored on a storage system as a file, the method comprising:

retrieving the multimedia document from the storage system;

displaying a section of multimedia information comprising the multimedia document in a first area of the display;

displaying a navigation bar in a second area of the display, the navigation bar including a plurality of thumbnail images and a progress bar image representative of a total playing time of the multimedia information, wherein the thumbnail images are extracted from the multimedia information at substantially equal intervals of time between the beginning and end of the multimedia information, wherein the progress bar image includes a symbol displayed at a position on the progress bar image indicative of a time corresponding to the section of the multimedia information being displayed in the first area of the display;

displaying a location box on the navigation bar, the location box associated with at least one thumbnail image in the thumbnail images;

receiving input from a user indicative of selection of a selected thumbnail image, including, receiving user input to move the location box from a first position associated with a first thumbnail image to a second position associated with the selected thumbnail image; and

in response to selection of the selected thumbnail image, displaying in the first area of the display a section of the multimedia information at a point in time corresponding to the selected thumbnail image, including displaying the location box in the second position associated with the selected thumbnail image.

2. The method of claim 1, wherein at least one of the thumbnail images is selectable during playback of the multimedia information in the first area of the display.

3. The method of claim 1, wherein the at least one thumbnail image is selectable using a television remote control.

4. The method of claim 1, wherein the progress bar comprises a symbol related to the playback of the multimedia information in the first area of the display, wherein the symbol comprises at least one of a pause sign, a play sign, a rewind sign, and a fast forward sign.

5. The method of claim 1, further comprising receiving user input for selecting the at least one thumbnail image.

6. The method of claim 5, further comprising displaying an indication that the selected at least one thumbnail has been selected.

7. The method of claim 6, wherein the indication is a highlighted location box including the selected at least one thumbnail.

8. The method of claim 1, further comprising displaying a second navigation bar in a third area of the display, the second navigation bar including one or more preview thumbnail images.

9. The method of claim 8, wherein the preview thumbnail images are extracted from the multimedia information in an interval related to a current time of the section of multimedia information being displayed in the first viewing area.

10. The method of claim 1, wherein the multimedia information comprises video information.

11. The method of claim 1, wherein a thumbnail image in the plurality of thumbnail images is a keyframe of multimedia information.

12. The method of claim 1, further comprising:

receiving topic of interest information;

determining one or more thumbnail images related to the topic of interest information from one or more thumbnail images; and

displaying a visual indication with the determined one or more thumbnail images that are related to the topic of interest.

13. A method for displaying a multimedia document on a television display, the multimedia document being stored on a storage system as a file, the method comprising:

retrieving the multimedia document from the storage system;

displaying a section of multimedia information contained in the multimedia document in a first area of the display;

displaying a navigation bar in a second area of the display, the navigation bar including a plurality of keyframes and a progress bar image representative of a total playing time of the multimedia information, wherein the keyframes are extracted from the multimedia information at substantially equal intervals of time between the beginning and end of the multimedia information, wherein the progress bar image includes a symbol displayed at a position on the progress bar image indicative of a time corresponding to the section of the multimedia information being displayed in the first area of the display;

displaying a location box on the navigation bar, the location box associated with at least one keyframe in the keyframes;

receiving input from a user indicative of selection of a selected keyframe, including, receiving user input to move the location box from a first position associated with a first keyframe to a second position associated with the selected keyframe; and

in response to selection of the selected keyframe, displaying in the first area of the display a section of the multimedia information at a point in time corresponding to the selected keyframe, including displaying the location box in the second position associated with the selected keyframe,

wherein displaying a navigation bar includes;

identifying a second area of the display in which the navigation bar will be displayed;

determining one or more timestamps based on the second area of the display and the length of time of the multimedia document;

for each timestamp, selecting a keyframe from the multimedia information of the multimedia document using time information associated with said each timestamp as a basis for selecting said keyframe;

configuring the navigation bar, the navigation bar including keyframes extracted for said each timestamp; and

displaying the configured navigation bar in the second area of the display, wherein at least one keyframe included in the navigation bar is selectable by a user while the section of multimedia information in the first area of the display is being displayed.

14. The method of claim 13, wherein at least one keyframe is selectable during playback of the multimedia information in the first area of the display.

15. The method of claim 13, wherein the at least one keyframe is selectable using a television remote control.

16. The method of claim 13, wherein determining one or more timestamps comprises determining a seconds multiplier.

17. The method of claim 16, wherein determining the seconds multiplier comprises dividing a length of the multimedia information by a width of the second area of the display.

18. The method of claim 16, wherein determining the seconds multiplier comprises dividing a length of the multimedia information by a height of the second area of the display.

19. The method of claim 16, wherein determining the one or more timestamps comprises multiplying the seconds multiplier by one or more positions in the second area of the display.

20. The method of claim 19, wherein the one or more positions are one or more X axis positions in the second area of the display.

21. The method of claim 19, wherein the one or more positions are one or more Y axis positions in the second area of the display.

22. The method of claim 13, wherein determining one or more timestamps comprises:

determining a timestamp for a first keyframe at a first position in the second area of the display; and

determining a timestamp for a second keyframe at a second position in the second area of the display,

wherein the second position is greater than the first position.

23. The method of claim 22, wherein determining one or more timestamps comprises:

determining a timestamp for a third keyframe at a third position in the second area of the display,

wherein the second position is greater than the first position but less than the third position.

24. The method of claim 13, wherein the second area of the display includes one or more cell positions.

25. The method of claim 24, wherein the keyframe extracted for the one or more timestamps are displayed in the one or more cell positions.

26. The method of claim 13, wherein the progress bar comprises a symbol related to the playback of the multimedia information in the first area of the display, wherein the symbol comprises at least one of a pause sign, a play sign, a rewind sign, and a fast forward sign.

27. The method of claim 13, further comprising receiving user input for selecting the at least one keyframe.

28. The method of claim 27, further comprising in response to the user input selecting the at least one keyframe, displaying a section of the multimedia information at a time corresponding to the selected at least one keyframe in the first area of the display.

29. The method of claim 27, further comprising displaying an indication that the selected at least one keyframe has been selected.

30. The method of claim 29, wherein the indication is a highlighted location box including the selected at least one keyframe.

31. The method of claim 13, further comprising displaying a second navigation bar in a third area of the display, the second navigation bar including one or more preview keyframes.

32. The method of claim 31, wherein the preview keyframes are extracted from the multimedia information in an interval related to a current time of the section of multimedia information being displayed in the first viewing area.

33. The method of claim 13, wherein the multimedia information comprises video information.

34. The method of claim 13, further comprising:

receiving topic of interest information;

determining one or more keyframes in the navigation bar related to the topic of interest information; and

displaying a visual indication with the determined one or more keyframes that are related to the topic of interest.

35. A data processing system for displaying a multimedia document on a television display, the data processing system comprising:

a processor;

a data storage system for storing a plurality of data files, the multimedia document being one of the data files;

a memory coupled to the processor, the memory configured to store one or more code modules for execution by the processor, the one or more code modules comprising;

a module for retrieving the multimedia document from the data storage system;

a module for displaying a section of multimedia information comprising the multimedia document in a first area of the display;

a module for displaying a navigation bar in a second area of the display, the navigation bar including a plurality of thumbnail images and a progress bar image representative of a total playing time of the multimedia information, wherein the thumbnail images are extracted from the multimedia information at substantially equal intervals of time between the beginning and end of the multimedia information, wherein the progress bar image includes a symbol displayed at a position on the progress bar image indicative of a time corresponding to the section of the multimedia information being displayed in the first area of the display, wherein when a user selects a selected thumbnail image, then the section of the multimedia information that is being displayed in the first area of the display starts from a point in time corresponding to the selected thumbnail image;

a module for displaying a location box on the navigation bar, the location box associated with at least one thumbnail image in the thumbnail images;

a module for receiving input from a user indicative of selection of a selected thumbnail image, including, a module for receiving user input to move the location box from a first position associated with a first thumbnail image to a second position associated with the selected thumbnail image; and

a module for displaying, in response to selection of the selected thumbnail image, in the first area of the display a section of the multimedia information at a point in time corresponding to the selected thumbnail image, including displaying the location box in the second position associated with the selected thumbnail image.

36. The data processing system of claim 35, wherein at least one of the thumbnail images is selectable during playback of the multimedia information in the first area of the display.

37. The data processing system of claim 35, wherein the at least one thumbnail image is selectable using a television remote control.

38. The data processing system of claim 35, further comprising a module for displaying a second navigation bar in a third area of the display, the second navigation bar including one or more preview thumbnail images.

39. A data processing system for displaying a multimedia document on a television display, the data processing system comprising:

a processor;

a data storage system for storing a plurality of data files, the multimedia document being one of the data files;

a memory coupled to the processor, the memory configured to store one or more code modules for execution by the processor, wherein the one or more code modules comprising;

a module for retrieving the multimedia document from the data storage system;

a module for displaying in a first area of the display a section of multimedia information comprising the multimedia document;

a module for displaying a navigation bar in a second area of the display, the navigation bar including a plurality of keyframes and a progress bar image representative of a total playing time of the multimedia information, wherein the keyframes are extracted from the multimedia information at substantially equal intervals of time between the beginning and end of the multimedia information, wherein the progress bar image includes a symbol displayed at a position on the progress bar image indicative of a time corresponding to the section of the multimedia information being displayed in the first area of the display;

a module for displaying a location box on the navigation bar, the location box associated with at least one keyframe in the keyframes;

a module for receiving input from a user indicative of selection of a selected keyframe, including, a module for receiving user input to move the location box from a first position associated with a first keyframe to a second position associated with the selected keyframe; and

a module for displaying, in response to selection of the selected keyframe, in the first area of the display a section of the multimedia information at a point in time corresponding to the selected keyframe, including displaying the location box in the second position associated with the selected keyframe,

wherein the module for displaying displaying a navigation bar includes;

a module for identifying a second area of the display in which the navigation bar will be displayed;

a module for determining one or more timestamps based on the second area of the display and the length of time of the multimedia document;

a module for selecting, for each timestamp, a keyframe from the multimedia information of the multimedia document using time information associated with said each timestamp as a basis for selecting said keyframe;

a module for configuring the navigation bar, the navigation bar including keyframes extracted for said each timestamp; and

a module for displaying the configured navigation bar in the second area of the display, wherein at least one keyframe included in the navigation bar is selectable while the section of multimedia information in the first area of the display is being displayed.

40. The data processing system of claim 39, wherein at least one keyframe is selectable during playback of the multimedia information in the first area of the display.

41. The data processing system of claim 39, wherein the at least one keyframe is selectable using a television remote control.

42. The data processing system of claim 39, wherein determining one or more timestamps comprises determining a seconds multiplier.

43. The data processing system of claim 42, wherein determining the one or more timestamps comprises multiplying the seconds multiplier by one or more positions in the second area of the display.

44. The data processing system of claim 39, wherein the module for determining one or more timestamps comprises:

a module for determining a timestamp for a first keyframe at a first position in the second area of the display; and

a module for determining a timestamp for a second keyframe at a second position in the second area of the display,

wherein the second position is greater than the first position.

45. The data processing system of claim 44, wherein the module for determining one or more timestamps comprises:

a module for determining a timestamp for a third keyframe at a third position in the second area of the display,

wherein the second position is greater than the first position but less than the third position.

46. A system comprising

a television;

a data storage system for storing a plurality of data files;

a data processing system, wherein the data processing system is configured to;

obtain a multimedia document stored on the data storage system;

display a section of multimedia information in a first area of the display, the multimedia information comprising the multimedia document;

display a navigation bar in a second area of the display, the navigation bar including a plurality of thumbnail images and a progress bar image representative of a total playing time of the multimedia information, wherein the thumbnail images are extracted from the multimedia information at substantially equal intervals of time between the beginning and end of the multimedia information, wherein the progress bar image includes a symbol displayed at a position on the progress bar image indicative of a time corresponding to the section of the multimedia information being displayed in the first area of the display, wherein when a user selects a selected thumbnail image, then the multimedia information that is being displayed in the first area of the display starts from a point in time corresponding to the selected thumbnail image;

display a location box on the navigation bar, the location box associated with at least one thumbnail image in the thumbnail images;

receive input from a user indicative of selection of a selected thumbnail image, including, receiving user input to move the location box from a first position associated with a first thumbnail image to a second position associated with the selected thumbnail image; and

display, in response to selection of the selected thumbnail image, in the first area of the display a section of the multimedia information at a point in time corresponding to the selected thumbnail image, including displaying the location box in the second position associated with the selected thumbnail image.

47. The system of claim 46, wherein at least one thumbnail image in the thumbnail images is selectable during playback of the multimedia information in the first area of the display.

48. The system of claim 46, wherein the at least one thumbnail image is selectable using a television remote control.

49. A system comprising

a television;

a data storage system for storing a plurality of data files, the multimedia document being one of the data files;

a data processing system, wherein the data processing is configured to;

retrieve the multimedia document from the data storage system;

display a section of multimedia information from a multimedia document in a first area of the display;

display a navigation bar in a second area of the display, the navigation bar including a plurality of keyframes and a progress bar image representative of a total playing time of the multimedia information, wherein the keyframes are extracted from the multimedia information at substantially equal intervals of time between the beginning and end of the multimedia information, wherein the progress bar image includes a symbol displayed at a position on the progress bar image indicative of a time corresponding to the section of the multimedia information being displayed in the first area of the display;

display a location box on the navigation bar, the location box associated with at least one keyframe in the keyframes;

receive input from a user indicative of selection of a selected keyframe, including, receive user input to move the location box from a first position associated with a first keyframe to a second position associated with the selected keyframe; and

in response to selection of the selected kevframe, display in the first area of the display a section of the multimedia information at a point in time corresponding to the selected keyframe, including displaying the location box in the second position associated with the selected keyframe,

wherein to display the navigation bar, the data processing is configured to;

identify a second area of the display in which the navigation bar will be displayed;

determine one or more timestamps based on the second area of the display and the length of time of the multimedia document;

for each timestamp, selecting a keyframe from the multimedia information of the multimedia document using time information associated with said each timestamp as a basis for selecting said keyframe;

configure the navigation bar, the navigation bar including keyframes extracted for said each timestamp; and

display the configured navigation bar in the second area of the display, wherein at least one keyframe included in the navigation bar is selectable while the section of multimedia information in the first area of the display is being displayed.

50. The system of claim 49, wherein at least one keyframe is selectable during playback of the multimedia information in the first area of the display.

51. The system of claim 49, wherein the at least one keyframe image is selectable using a television remote control.

52. A system for displaying a multimedia document on a television display, the system comprising:

a processing unit, the processing unit including;

means for displaying a section of multimedia information comprising the multimedia document in a first area of the display;

means for displaying a navigation bar in a second area of the display, the navigation bar including a plurality of thumbnail images and a progress bar image representative of a total playing time of the multimedia information, wherein the thumbnail images are extracted from the multimedia information at substantially equal intervals of time between the beginning and end of the multimedia information, wherein the progress bar image includes a symbol displayed at a position on the progress bar image indicative of a time corresponding to the section of the multimedia information being displayed in the first area of the display, wherein when a selected thumbnail image is selected by a user, then the section of the multimedia information that is being displayed in the first area of the display starts from a point in time corresponding to the selected thumbnail image;

means for receiving input from a user indicative of selection of a selected thumbnail image, including;

means for displaying a location box on the navigation bar, the location box associated with at least one thumbnail image in the thumbnail images; and

means for receiving user input to move the location box from a first position associated with a first thumbnail image to a second position associated with the selected thumbnail image; and

means for displaying, in response to selection of the selected thumbnail image, in the first area of the display a section of the multimedia information at a point in time corresponding to the selected thumbnail image, including displaying the location box in the second position associated with the selected thumbnail image.

53. A system for displaying a multimedia document on a television display, the multimedia document being stored on a storage system as a file, the system comprising:

a data processing unit, the data processing unit including;

means for retrieving the multimedia document from the storage system;

means for displaying a section of multimedia information contained in the multimedia document in a first area of the display;

means for displaying a navigation bar in a second area of the display, the navigation bar including a plurality of keyframes and a progress bar image representative of a total playing time of the multimedia information, wherein the keyframes are extracted from the multimedia information at substantially equal intervals of time between the beginning and end of the multimedia information, wherein the progress bar image includes a symbol displayed at a position on the progress bar image indicative of a time corresponding to the section of the multimedia information being displayed in the first area of the display;

means for displaying a location box on the navigation bar, the location box associated with at least one keyframe in the keyframes;

means for receiving input from a user indicative of selection of a selected keyframe, including, means for receiving user input to move the location box from a first position associated with a first keyframe to a second position associated with the selected keyframe; and

means for displaying, in response to selection of the selected keyframe, in the first area of the display a section of the multimedia information at a point in time corresponding to the selected keyframe, including displaying the location box in the second position associated with the selected keyframe,

wherein the means for displaying a navigation bar includes;

means for identifying a second area of the display in which the navigation bar will be displayed;

means for determining one or more timestamps based on the second area of the display and the length of time of the multimedia document;

means for selecting, for each timestamp, a keyframe from the multimedia information of the multimedia document using time information associated with said each timestamp as a basis for selecting said keyframe;

means for configuring the navigation bar, the navigation bar including keyframes extracted for said each timestamp; and

means for displaying the configured navigation bar in the second area of the display, wherein at least one keyframe included in the navigation bar is selectable while the section of multimedia information in the first area of the display is being displayed.

54. A computer program product for displaying a multimedia document on a television display, the multimedia document being stored on a storage system as a file, the computer program product comprising:

a storage subsystem having stored thereon executable program code, the executable program code including;

code for retrieving the multimedia document from the storage system;

code for displaying a section of multimedia information comprising the multimedia document in a first area of the display; and

code for displaying a navigation bar in a second area of the display, the navigation bar including a plurality of thumbnail images and a progress bar image representative of a total playing time of the multimedia information, wherein the thumbnail images are extracted from the multimedia information at substantially equal intervals of time between the beginning and end of the multimedia information, wherein the progress bar image includes a symbol displayed at a position on the progress bar image indicative of a time corresponding to the section of the multimedia information being displayed in the first area of the

display, wherein when a selected thumbnail image is selected by the user, then the section of the multimedia information that is being displayed in the first area of the display starts from a point in time corresponding to the selected thumbnail image;

code for displaying a location box on the navigation bar, the location box associated with at least one thumbnail image in the thumbnail images;

code for receiving input from a user indicative of selection of a selected thumbnail image, including code for receiving user input to move the location box from a first position associated with a first thumbnail image to a second position associated with the selected thumbnail image; and

code for displaying, in response to selection of the selected thumbnail image, in the first area of the display a section of the multimedia information at a point in time corresponding to the selected thumbnail image, including displaying the location box in the second position associated with the selected thumbnail image.

55. The computer program product of claim 54, wherein at least one thumbnail image in the one or more thumbnail images is selectable during playback of the multimedia information in the first area of the display.

56. The computer program product of claim 54, wherein the at least one thumbnail image is selectable using a television remote control.

57. A computer program product for displaying a multimedia document on a television display, the multimedia document being stored on a storage system as a file, the computer program product comprising:

a storage subsystem having stored thereon executable program code, the executable program code including;

code for retrieving the multimedia document from the storage system;

code for displaying a navigation bar in a second area of the display, the navigation bar including a plurality of keyframes and a progress bar image representative of a total playing time of the multimedia information, wherein the keyframes are extracted from the multimedia information at substantially equal intervals of time between the beginning and end of the multimedia information, wherein the progress bar image includes a symbol displayed at a position on the progress bar image indicative of a time corresponding to the section of the multimedia information being displayed in the first area of the display;

code for displaying a location box on the navigation bar, the location box associated with at least one keyframe in the keyframes;

code for receiving input from a user indicative of selection of a selected keyframe, including, code for receiving user input to move the location box from a first position associated with a first keyframe to a second position associated with the selected keyframe; and

code for displaying, in response to selection of the selected keyframe, in the first area of the display a section of the multimedia information at a point in time corresponding to the selected keyframe, including displaying the location box in the second position associated with the selected keyframe,

wherein the code for displaying a navigation bar includes;

code for displaying a section of multimedia information comprising the multimedia document in a first area of the display;

code for identifying a second area of the display in which the navigation bar will be displayed;

code for determining one or more timestamps based on the second area of the display and the length of time of the multimedia document;

code for selecting, for each timestamp, a keyframe from the multimedia information of the multimedia document using time information associated with said each timestamp as a basis for selecting said keyframe;

keyframes extracted for said each timestamp; and

code for displaying the configured navigation bar in the second area of the display, wherein at least one keyframe included in the navigation bar is selectable while the section of multimedia information in the first area of the display is being displayed.

58. The computer program product of claim 57, wherein at least one keyframe is selectable during playback of the multimedia information in the first area of the display.

59. The computer program product of claim 57, wherein the at least one keyframe is selectable using a television remote control.