Automatic generation of presentations from "path-enhanced" multimedia

• US 20040218894A1
• Filed: 04/30/2003
• Published: 11/04/2004
• Est. Priority Date: 04/30/2003
• Status: Active Grant

• Current Assignee: Hewlett-Packard Development Company, L.P. (HP Inc.)
• Original Assignee: Hewlett-Packard Development Company, L.P. (HP Inc.)
• Inventors: Samadani, Ramin, Harville, Michael
• Granted Patent: US 7,149,961 B2
• Time in Patent Office: Days
• US Class Current: 386/46
• CPC Class Codes: G06F 16/29 Geographical information databases
• CPC Class Codes: G11B 2220/213 Read-only discs
• CPC Class Codes: G11B 2220/2545 CDs
• CPC Class Codes: G11B 2220/2562 DVDs [digital versatile discs]; Digital video discs; MMCDs; HDCDs
• CPC Class Codes: G11B 27/034 on discs G11B27/036, G11B27/038 take precedence
• CPC Class Codes: H04N 5/9201 involving the multiplexing of an additional signal and the video signal

Abstract

A “path-enhanced” presentation is automatically generated from previously recorded “path-enhanced” multimedia based on time and location information associated with the multimedia. The path based on the time and location information is rendered over a two-dimensional map. The presentation indicates position and progress along the path, and at the same time, previously recorded multimedia corresponding to points or segments of that path are automatically played. When the moving icon reaches a location that is associated with multimedia, that multimedia is automatically played.

Claims

1. A method of generating a multimedia presentation from “

path enhanced”

multimedia data including essentially continuous path information, having associated time and location data, and at least one multimedia data file associated with at least one point along the path, the method comprising;

obtaining map rendering information dependent on the path information;

generating information for rendering the path and sequentially rendering the at least one multimedia data file associated with the at least one point along the rendered path dependent on the path information and the associated at least one multimedia data file;

wherein a multimedia presentation is renderable according to the map rendering information and the information for rendering the path and the at least one multimedia data file.

2. The method as described in claim 1 further comprising generating information for rendering the path and the at least one multimedia data file by one of default and user selection.

3. The method as described in claim 1 further comprising obtaining map rendering information by forming a query to a map database using the path information.

4. The method as described in claim 1 wherein generating information for rendering the path and the at least one multimedia data file further comprises generating a rendering description of icons for representing the associated at least one multimedia data file at the at least one point along the rendered path.

5. The method as described in claim 4 further comprising representing with a single cluster icon a plurality of multimedia data files each associated with a corresponding point along the path.

6. The method as described in claim 1 wherein generating information for rendering the path and the at least one multimedia data file further comprises generating a rendering description of how the at least one multimedia data file is played at each point along the path during the presentation.

7. The method as described in claim 1 wherein generating information for rendering the path and the at least one multimedia data file further comprises constructing a rendering description of the path including path color and line style.

8. The method as described in claim 1 wherein generating information for rendering the path and the at least one multimedia data file further comprises constructing a rendering description of a location indicator representing progress along the path.

9. The method as described in claim 1 wherein generating information for rendering the path and the at least one multimedia data file further comprises constructing a rendering description of a cluster of at least one multimedia data file associated with more than one point along the path.

10. The method as described in claim 1 further comprising generating information for the rendering of at least one of an introduction, a conclusion, and an audio accompaniment.

11. A system for generating a data structure corresponding to a multimedia presentation from “

path-enhanced”

multimedia including essentially continuous path information, having associated time and location data, and at least one multimedia data file associated with at least one point along the path, the system comprising;

“

path-enhanced”

multimedia analyzer for obtaining map rendering information dependent on the path information;

at least one playlist generator for generating information for rendering the path and sequentially rendering the at least one multimedia data file associated with the at least one point along the rendered path dependent on the path information and the associated at least one multimedia data file.

12. The system as described in claim 11 further comprising a multimedia data cluster grouper for identifying and grouping adjacent points along the path and their associated multimedia in the “

path-enhanced”

multimedia data into a multimedia cluster.

13. The system as described in claim 12 wherein the playlist generator comprises a context playlist generator for generating a description of the rendering of at least one of the multimedia clusters, the map, a location indicator representing a progress along the path, transition type between the playing of multimedia, icons representing the at least one multimedia data file associated with at least one point along the path and multimedia clusters, and the path.

14. The system as described in claim 12 wherein the playlist generator comprises a cluster playlist generator for generating a description of the rendering of at least one of the map, a location indicator representing a progress along the path, transition type between the playing of multimedia, icons representing the at least one multimedia data file associated with the at least one point along the path, the path, and field-of-view.

15. The system as described in claim 11 wherein the playlist generator comprises an audio playlist generator for generating a description of the playing of a an audio accompaniment including a list of audio elements, timing for playing the audio elements, timing of audio silences, and play control of the audio elements.

16. The system as described in claim 11 wherein the playlist generator comprises an introduction and conclusion playlist generator for generating a description of the rendering of at least one of still images and video to be played during the introduction and conclusion of the presentation.

17. The system as described in claim 11 further comprising a device for receiving the presentation data structure and playing the presentation according to the generated playlist.

18. A machine-implemented multimedia method, comprising:

receiving path-enhanced multimedia (PEM) data including a sequence of points corresponding to at least one spatiotemporal path and links between at least one multimedia object and at least one respective point on the at least one spatiotemporal path, wherein at least one spatiotemporal path includes at least one point unlinked to any respective multimedia object;

generating a structured specification for rendering a presentation including at least a portion of the PEM data, wherein at least a portion of a given spatiotemporal path is represented by a substantially continuous sequence of points overlaid on a corresponding portion of a map of a geographic region encompassing spatial bounds of at least a portion of the received PEM data; and

storing the structured specification on a machine readable medium.

19. The method of claim 18, wherein each point on a given spatiotemporal path is associated with time data, at least one point on a given spatiotemporal path additionally is associated with geographic location data, and at least one point on a given spatiotemporal path additionally is associated with a multimedia object reference component for linking a point to an associated multimedia object.

20. The method of claim 18, further comprising computing the spatial bounds of the geographic region from the received PEM data.

21. The method of claim 20, further comprising querying a database for map information corresponding to the computed spatial bounds.

22. The method of claim 18, further comprising computing temporal bounds of at least a portion of the received PEM data.

23. The method of claim 18, wherein generating the structured specification comprises constructing from the received PEM data a description of at least a portion of the given spatiotemporal path.

24. The method of claim 23, wherein constructing the path description comprises constructing an ordered list of two-dimensional coordinates registered to coordinates of the map from spatial component data of points on at least a portion of the given spatiotemporal path.

25. The method of claim 23, wherein constructing the path description comprises computing a set of parameters describing at least one approximation to at least a portion of the path from spatial component data of points on at least a portion of the given spatiotemporal path.

26. The method of claim 23, further comprising processing at least a portion of the received PEM data to smooth at least a portion of the given spatiotemporal path.

27. The method of claim 26, wherein processing PEM data comprises computing geographic location data for a given point unassociated with geographic location data based on geographic location data associated with points neighboring the given point.

28. The method of claim 23, further comprising removing at least one path loop from at least a portion of the received PEM data.

29. The method of claim 18, wherein generating the structured specification comprises associating graphical icons with multimedia objects linked to points on the given spatiotemporal path.

30. The method of claim 18, wherein generating the structured specification comprises specifying a graphical representation of a field-of-view for a given visual multimedia object linked to a point on the given spatiotemporal path.

31. The method of claim 30, wherein the field-of-view representation includes a graphical arrow icon associated with the given visual multimedia object.

32. The method of claim 31, wherein the graphical arrow icon has an orientation corresponding to an orientation of a camera that captured the given visual multimedia object.

33. The method of claim 31, wherein the graphical arrow icon has a length representative of distance to a point-of-focus of a camera that captured the given visual multimedia object.

34. The method of claim 30, wherein the field-of-view representation includes an indication of a spatial extent of the field-of-view of the given visual multimedia object.

35. The method of claim 30, wherein the field-of-view representation includes of a tilt angle of the field-of-view of the given visual multimedia object.

36. The method of claim 30, wherein the field-of-view representation corresponds to a union of multiple fields-of-view captured by respective video frames of the given visual multimedia object.

37. The method of claim 18, wherein generating the structured specification comprises grouping PEM data points into clusters.

38. The method of claim 37, wherein PEM data points associated with multimedia objects are grouped into clusters based on values for at least one of the spatial components and the temporal components of the associated points

39. The method of claim 38, wherein grouping PEM data points comprises comparing a threshold to differences between component values of pairs of PEM data points.

40. The method of claim 39, further comprising adjusting the threshold to achieve clusters whose respective numbers of points linked to multimedia objects fall within a specified range.

41. The method of claim 37, further comprising computing mean location and time values for PEM data points within each cluster based on the spatial and temporal components associated with the PEM data points.

42. The method of claim 37, wherein generating the structured specification comprises associating graphical icons with clusters.

43. The method of claim 37, wherein the structured specification includes a specification for rendering a sequence of views of a cluster.

44. The method of claim 43, wherein the sequence of views contains at least one view containing a graphical icon representing the cluster and at least one view containing a set of graphical icons representative of multimedia objects in the given cluster.

45. The method of claim 44, wherein the cluster rendering specification includes a map resolution threshold for determining when to transition between rendering a cluster view containing a cluster icon and rendering a cluster view containing a set of graphical icons representative of multimedia objects in the given cluster.

46. The method of claim 18, wherein the structured specification prescribes respective rendering times for multimedia objects linked to the given spatiotemporal path.

47. The method of claim 46, wherein the structured specification prescribes that all visual multimedia objects in the given spatiotemporal path are to be rendered in a sequence ordered in accordance with capture times respectively associated with the visual multimedia objects.

48. The method of claim 46, wherein the structured specification prescribes that each multimedia object consisting of a single image is to be rendered for a specified time period.

49. The method of claim 46, wherein the structured specification associates a given audio multimedia object to a given visual multimedia object previously unassociated with any audio multimedia objects.

50. The method of claim 49, further comprising associating the given audio multimedia object to the given visual multimedia object based on at least one of the spatial components and the temporal components of the points associated with the given audio multimedia object and the given visual multimedia object.

51. The method of claim 50, wherein the structured specification prescribes that the given visual multimedia object is to be rendered at least partially concurrently with the rendering of the given audio multimedia object.

52. The method of claim 18, wherein the structured specification prescribes at least one transition effect between renderings of a map view of a path and a multimedia object linked to the path.

53. The method of claim 18, wherein generating the structured specification comprises defining at least one path progress indicator.

54. The method of claim 53, wherein a path progress indicator definition specifies at least one graphical icon to be rendered at a current path location.

55. The method of claim 54, wherein defining at least one path progress indicator comprises associating different graphical icons to different respective path segments of the given spatiotemporal path.

56. The method of claim 55, wherein a given graphical icon is associated with a given path segment based on temporal extent of the given path segment.

57. The method of claim 55, wherein a given graphical icon is associated with a given path segment based on spatial extent of the given path segment.

58. The method of claim 55, wherein a given graphical icon is associated with a given path segment based on a capture time associated with the given path segment.

59. The method of claim 55, wherein a given graphical icon is associated with a given path segment based on a terrain type over which the given path segment is superposed.

60. The method of claim 55, wherein a given graphical icon is associated with a given path segment based on a temperature measure associated with the given path segment.

61. The method of claim 55, wherein a given graphical icon is associated with a given path segment based on a number of multimedia objects linked to the given path segment.

62. The method of claim 53, wherein the path progress indicator definition specifies dynamic modification of path appearance during rendering.

63. The method of claim 62, wherein the path progress indicator definition specifies a first style for rendering path segments already traversed and a second style different from the first style for rendering path segments yet to be traversed.

64. The method of claim 53, wherein the path progress indicator definition specifies an animation style for a given path progress indicator.

65. The method of claim 64, wherein the animation style specifies how fast the given path progress indicator traverses a given path segment.

66. The method of claim 65, wherein the animation style specifies a rate of path traversal proportional to a rate at which points on the given path segment were traversed by a path recording device.

67. The method of claim 65, wherein the animation style specifies a fixed time period for traversing path portions between multimedia rendering events.

68. The method of claim 18, wherein generating the structured specification comprises associating non-PEM data to at least one point of the spatiotemporal path.

69. The method of claim 18, wherein generating the structured specification comprises automatically querying a database based on the received PEM data, and incorporating data received in response to the querying into the structured specification.

70. The method of claim 18, wherein generating the structured specification comprises generating a specification for rendering at least one of an introduction and a conclusion for the presentation.

71. The method of claim 18, wherein the structured specification includes a data structure including at least one playlist describing at least one rendering action selected from:

play a multimedia file;

draw at least a portion of the map;

draw a representation of a path segment;

draw an indicator of a current location along a path;

draw a representation of a multimedia capture location; and

draw a representation of a field-of-view associated with a visual multimedia object.

72. The method of claim 18, wherein the structured specification represents movement of a map-based view of the presentation with a set of time-stamped keyframes.

73. The method of claim 18, wherein the stored structured specification includes a frame-by-frame video of the presentation and an associated audio track for the presentation.

74. The method of claim 18, wherein generating the structured specification comprises prompting a user to input information.

75. The method of claim 18, further comprising rendering the presentation as specified by the structured specification.

76. The method of claim 75, wherein rendering the presentation comprises displaying a graphical representation of at least a portion of the given spatiotemporal path overlaid on an image of a corresponding portion of the map.

77. The method of claim 76, wherein the presentation further comprises displaying a path progress indicator on the graphical path representation.

78. The method of claim 77, wherein rendering the presentation further comprises displaying the path progress indicator traversing the graphical path representation over time.

79. The method of claim 78, wherein rendering the presentation comprises rendering a multimedia object associated with a point on the given spatiotemporal path when the path progress indicator reaches a corresponding point on the graphical path representation.

80. The method of claim 78, wherein rendering the presentation comprises scrolling the rendered portions of the given spatiotemporal path and the map so that the path progress indicator remains in a central region of a display area.

81. The method of claim 78, wherein rendering the presentation comprises rendering the portion of the given spatiotemporal path and the map at a variable resolution level selected based on numbers of renderable objects near the path progress indicator as the path progress indicator traverses the graphical path representation.

82. The method of claim 78, wherein rendering the presentation comprises rendering at least one multimedia object associated with a point on the given spatiotemporal path while the path progress indicator traverses the graphical path representation.

83. The method of claim 75, further comprising rendering background audio content concurrently with rendering of at least a non-audio portion of the PEM data.

84. The method of claim 77, further comprising allowing a user to control rendering of the presentation, including at least one of pausing the rendering of the presentation, rewinding the rendering of the presentation, fast-forwarding through the rendering of the presentation, and beginning the rendering of the presentation at a selected point.

85. A multimedia system, comprising at least one processing module operable to:

receive path-enhanced multimedia (PEM) data including a sequence of points corresponding to at least one spatiotemporal path and links between at least one multimedia object and at least one respective point on the at least one spatiotemporal path, wherein at least one spatiotemporal path includes at least one point unlinked to any respective multimedia object;

generate a structured specification for rendering a presentation including at least a portion of the PEM data, wherein at least a portion of a given spatiotemporal path is represented by a substantially continuous sequence of points overlaid on a corresponding portion of a map of a geographic region encompassing spatial bounds of at least a portion of the received PEM data; and

store the structured specification on a machine readable medium.

86. A machine-readable medium storing machine-readable instructions for causing a machine to:

receive path-enhanced multimedia (PEM) data including a sequence of points corresponding to at least one spatiotemporal path and links between at least one multimedia object and at least one respective point on the at least one spatiotemporal path, wherein at least one spatiotemporal path includes at least one point unlinked to any respective multimedia object;

generate a structured specification for rendering a presentation including at least a portion of the PEM data, wherein at least a portion of a given spatiotemporal path is represented by a substantially continuous sequence of points overlaid on a corresponding portion of a map of a geographic region encompassing spatial bounds of at least a portion of the received PEM data; and

store the structured specification on a machine readable medium.

87. A multimedia system, comprising:

a machine-implemented accessing module configured to receive path-enhanced multimedia (PEM) data including a sequence of points corresponding to at least one spatiotemporal path and links between at least one multimedia object and at least one respective point on the at least one spatiotemporal path, wherein at least one spatiotemporal path includes at least one point unlinked to any respective multimedia object;

a machine-implemented presentation generation module configured to generate a structured specification for rendering a presentation including at least a portion of the PEM data, wherein at least a portion of a given spatiotemporal path is represented by a substantially continuous sequence of points overlaid on a corresponding portion of a map of a geographic region encompassing spatial bounds of at least a portion of the received PEM data; and

a machine-implemented rendering module configured to render the presentation.