Narrative-based media organizing system for transforming and merging graphical representations of digital media within a work area

• US 9,460,068 B2
• Filed: 02/03/2011
• Issued: 10/04/2016
• Est. Priority Date: 02/03/2010
• Status: Active Grant

• Current Assignee: Google LLC (Alphabet Inc.)
• Original Assignee: Google Inc. (Alphabet Inc.)
• Inventors: Kokemohr, Nils
• Primary Examiner(s): Paula, Cesar
• Assistant Examiner(s): Faber, David
• Application Number:
  US13/576,918
• Publication Number:
  US 20130198602A1
• Time in Patent Office: 2,070 Days
• Field of Search: 715/231, 715/233, 715/243, 715/273
• US Class Current: 1/1
• CPC Class Codes: G06F 16/40 of multimedia data, e.g. slideshows comprising image and additional audio data retrieval of still image data G06F16/50; retrieval of audio data G06F16/60; retrieval of video data G06F16/70
• CPC Class Codes: G06F 16/41 Indexing; Data structures therefor; Storage structures
• CPC Class Codes: G06F 16/4393 Multimedia presentations, e.g. slide shows, multimedia albums
• CPC Class Codes: G06F 16/93 Document management systems
• CPC Class Codes: G06F 3/04845 for image manipulation, e.g. dragging, rotation, expansion or change of colour
• CPC Class Codes: G06F 40/106 Display of layout of documents; Previewing
• CPC Class Codes: G06F 40/169 Annotation, e.g. comment data or footnotes

Abstract

A computer system for a narrational media organizer for transforming digital media into a personal, memorable story with minimal user input having a processor and storage with instructions for creating a narrational media organizer (NMO) environment, where a user can annotate one or more than one digital media file or graphical representations of the digital media files using a user interface; and an NMO data structure for storing the digital media and annotations of the NMO environment.

Claims

1. A computer system for a narrational media organizer to transform digital media into a personal, memorable story with minimal user input, the system comprising:

a) one or more processors;

b) a non-transitory machine readable medium coupled to the one or more processors;

c) a set of computer instructions stored in the machine readable medium and operable on the one or more processors for creating a narrational media organizer (NMO) environment causing performance of operations comprising;

displaying digital media files of a first set as graphical representations of each of the digital media files in an arrangement along a timeline in a work area;

receiving from a first user, a first command to shift the arrangement, including a user selection of an annotation location between two of the displayed graphical representations of the digital media files and a textual annotation;

in response to the first command, placing a graphical instance of the textual annotation at the selected annotation location and shifting the arrangement of graphical representations to make room for the textual annotation;

obtaining one or more digital media files of a second set of a second user, each of the one or more digital media files of the second set associated with a timestamp, wherein a first digital media file of the one or more digital media files of the second set was non-destructively excluded from the NMO environment; and

merging the digital media files of the second set with the first set of digital media files, wherein;

a first graphical representation of at least one of the digital media files of the second set is automatically inserted between two of the displayed graphical representations of the digital media files of the first set into the arrangement along the timeline based on the timestamp associated with the at least one of the digital media files of the second set, and

a second graphical representation of the first digital media file of the second set is automatically inserted along the timeline and the second graphical representation includes a minimized version of the first digital media file;

d) a user interface operably connected to the one or more processors, the user interface effective to transmit one or more commands including the first command to the one or more processors; and

e) a storage operably connected to the one or more processors, the storage effective for storing a data structure, wherein the data structure includes the digital media files of the first set, the digital media files of the second set, and the textual annotation.

2. The system of claim 1, wherein the NMO environment is stored in a data structure that includes multiple versions of a story that include the digital media files of the first set and digital media files of the second set.

3. The system of claim 2, wherein the operations further comprise creating a paragraph using an input device, a gesture or both an input device and a gesture, thereby, adding space between the digital media files in a form of a new paragraph.

4. The system of claim 3, wherein creating the paragraph includes adding space in the NMO environment for a headline.

5. The system of claim 3, wherein the gesture is performed by the first user by taking the graphical representation of the digital media files with a pointing device and dragging it down.

6. The system of claim 1, wherein the first digital media file is excluded from an NMO output that includes the digital media files of the first set and the digital media files of the second set.

7. The system of claim 6, wherein the operations further comprise re-including one of the excluded at least one of the digital media files in the NMO environment.

8. The system of claim 1, wherein (3) the first digital media file is excluded from an NMO output that includes the digital media files of the first set and the digital media files of the second set.

9. The system of claim 1, wherein the first digital media file of the one or more digital media files of the second set is displayed to the first user.

10. The system of claim 1, wherein the textual annotation from the first user describes an event occurring before a time of capture of one of the digital media files and after a time of capture of a preceding one of the digital media files, where none of the digital media files represent the event.

11. The system of claim 1, wherein the textual annotation from the first user relates to contents of a single one of the digital media files.

12. The system of claim 1, wherein the textual annotation from the first user relates to one or more than one detail within the digital media files.

13. The system of claim 1, wherein the textual annotation from the first user is related to contents of a following numbered series of digital media files.

14. The system of claim 1, wherein the textual annotation from the first user is related to contents of a following unnumbered series of digital media files.

15. The system of claim 1, wherein the operations for receiving textual annotations from the first user further comprise instructions to receive a paragraph headline annotation related to contents of a following section of digital media files.

16. The system of claim 1, wherein the operations further comprise positioning a cursor at the annotation location.

17. The system of claim 16, wherein the cursor position can be controlled by arrow keys, keyboard key combinations, or by a pointing device.

18. The system of claim 1, wherein the operations further comprise receiving from the user, a second command to position a cursor at a second annotation location between a second two of the displayed graphical representations, wherein the cursor position can be controlled by a return key to create a paragraph, and thereby cause the digital media files subsequent to the cursor position to be moved down and form a new paragraph.

19. The system of claim 18, wherein the formation of a new paragraph also includes adding space in the NMO environment for a headline.

20. The system of claim 1, wherein the operations further comprise providing the first user control over a cursor positioned between digital media files;

where the position of the cursor can be controlled by the first user to add a textual input by the first user; and

where the textual input will trigger instructions to the NMO environment to receive the textual input as the textual annotation and place it at the position of the cursor at the annotation location.

21. The system of claim 1, wherein the operations further comprise presenting a user interface menu option for switching between a first mode and a second mode, where the first mode is suitable to review the digital media files with a current status of textual annotations, and the second mode is suitable to receive the textual annotations or digital media exclusion information from the first user.

22. The system of claim 1, wherein the operations further comprise:

causing a display of digital media files and narration as a sequence of icons of the digital media files and other elements based on;

a first algorithm configured to display all digital media files at a same on-screen height in an arrangement in a work area;

a second algorithm configured to display a textual comment in-between the digital media files at a comment location, in response to receiving the first command from the first user to shift the arrangement, including a user selection of the comment location and the textual comment;

graphical marks configured to indicate different types of narrational annotations; and

vertical spatial inserts between the digital media files chosen by the first user to indicate that a section or paragraph is to be displayed at a paragraph location, leaving room for a paragraph headline at the paragraph location.

23. The system of claim 22, wherein receiving a selection of one or more the displayed digital media files causes an increase in a size of the one or more displayed digital media files that were selected.

24. The system of claim 22, wherein one or more of the displayed digital media files comprise a flag to emphasize the displayed digital media files.

25. The system of claim 22, wherein the operations further comprise non-destructively excluding one or more of the digital media files.

26. The system of claim 25, wherein non-destructively excluding is performed by clicking on an icon superimposed on each of the digital media files, thereby minimizing the digital media files.

27. The system of claim 26, wherein the minimized digital media files are represented by a smaller version of the digital media files.

28. The system of claim 25, wherein non-destructively excluding is performed by initially displaying the digital media files on a side bar, a top bar or a gutter, and then selecting and positioning each selected digital media file on a large space suitable to receive and arrange the selected digital media files and textual annotations.

29. The system of claim 25, wherein non-destructively excluding is performed by initially displaying the digital media files in a sequence and providing a side bar, a top bar or a gutter where the user can move the digital media files to be excluded from the narration.

30. The system of claim 1, wherein the operations further comprise cropping one or more of the digital media files within the NMO environment.

31. The system of claim 30, wherein the operations further comprise rescaling the digital media files in the NMO environment and to rearrange, if necessary, subsequent digital media files in the NMO environment.

32. The system of claim 1, wherein the data structure comprises:

a) one or more first fields that store a first list of digital media elements information;

b) one or more second fields that store annotation information; and

c) one or more third fields that store paragraph information.

33. The system of claim 32, wherein the annotation information comprises a data field that stores a type of the textual annotation, where the data field also comprises a hyperlink field configured to link the textual annotation to one or more digital media parameters.

34. The system of claim 33, wherein the digital media parameters are selected from the group comprising a digital media detail, a series of digital media files and a single digital media file.

35. The system of claim 32, wherein the first list of digital media elements information includes actual digital media data, a storage location of the digital media data, or both the actual digital media data and the storage location of the digital media data.

36. The system of claim 32, wherein the first list of digital media elements information includes the minimized version to indicate an exclusion of one or more of the digital media files from an NMO output.

37. The system of claim 32, wherein the first list of digital media elements information includes an icon to indicate whether or not the digital media files will be printed at a larger size in an NMO output.

38. The system of claim 32, wherein the data structure further comprises one or more fields configured to store a second list of digital media elements information where the second list is created when the first user triggers the system to output an NMO output.

39. The system of claim 38, wherein operations further include receiving from the first user a choice between the first and second lists.

40. The system of claim 39 wherein the first list represents a first version of a user narrated story and the second list represents a second version of the user narrated story for the digital media files.

41. The system of claim 1, wherein the data structure comprises a first list and a second list, where the first list and the second list comprise one or more of the following:

a) the digital media files; and

b) a reference to digital media data.

42. The system of claim 41, wherein the first list is a list of elements pointing to elements in the second list, and wherein the first or the second list contain information or pointers to the textual annotation and information regarding the type of the textual annotation, the information being configured to link the textual annotation to one or more of the following:

a digital media detail, a series of the digital media files, and one of the digital media files.

43. The system of claim 1, wherein the operations further comprise (vi) receiving one or more favorited selections of corresponding digital media files;

and (vii) providing a first option to print the one or more favorited selections and a second option to print remaining digital media files that were not selected as favorites.

44. A method for a narrational media organizer to transform digital media into a personal, memorable story with minimal user input, the method comprising the steps of:

a) presenting to a first user a collection of digital media files in an arrangement in a work area;

b) receiving from the first user a selection of a marker to indicate an exclusion of a first digital media file from the collection of digital media files;

c) presenting to the first user the collection of digital media files, wherein the first digital media file is displayed as a minimized version;

d) receiving from the first user more than one different types of digital media annotations, a user selected annotation location for each of the digital media annotations, user instructions to shift the arrangement to insert at least one of the digital media annotations at the annotation location between two of the digital media files, a user placeholder location between two of the digital media files or between a digital media annotation and a digital media file, and a user request for a second user to provide a digital media file for the placeholder location;

e) receiving a second digital media file of a second user;

f) inserting the digital media file of the second user into the placeholder location;

g) providing to the second user the collection of digital media files and the second digital media file, wherein the first digital media file is displayed as the minimized version; and

h) producing graphics that comprise both one or more versions of the digital media files and the digital media annotations that are suitable to be uploaded or mailed or shared to form a digital media narration, wherein the first digital media file is excluded from the graphics.

45. The method of claim 44, wherein the collection is transformed into smaller resolution versions of an original of the digital media files.

46. The method of claim 45, further comprising the step of i) uploading, emailing, or both uploading and emailing the smaller resolution versions of the digital media files and adding a link to an internet accessible data store comprising the original of the digital media files.

47. A computer system for a narrational media organizer to transform digital media into a personal, memorable story with minimal user input, the system comprising:

a) one or more processors;

b) a non-transitory machine readable medium connected to the one or more processors;

c) a set of computer instructions stored in the machine readable medium and operable on the one or more processors for creating a narrational media organizer (NMO) environment, causing performance of operations comprising;

displaying a first set of more than one digital media files as graphical representations of each of the digital media files in an arrangement along a timeline in a work area, wherein each digital media file includes content;

receiving from a first user, a first command to shift the arrangement, including a user selection of an annotation location between two of the displayed graphical representations of the digital media files and a textual annotation;

receiving from the first user, a second command to create a paragraph in the arrangement, including a user selection of a paragraph location between two of the displayed graphical representations of the digital media files and a headline annotation related to the contents of a following section of digital media files;

in response to the first and second commands, placing a graphical instance of the textual annotation at the selected annotation location, placing a graphical instance of the headline annotation at the selected paragraph location and shifting the arrangement of graphical representations to make room for the textual annotation and headline annotation;

obtaining, a second set of one or more digital media files of a second user, each of the one or more digital media files of the second set associated with a timestamp, wherein a first digital media file of the second set was non-destructively excluded from the NMO environment; and

merging the digital media files of the second set with the first set of more than one digital media files, wherein;

a first graphical representation of at least one of the digital media files of the second set is automatically inserted between two of the displayed graphical representations of the digital media files of the first set into the arrangement along the timeline based on the timestamp associated with the at least one of the digital media files of the second set and

a second graphical representation of the first digital media file of the second set is automatically inserted along the timeline and the second graphical representation includes a minimized version of the first digital media file;

d) a user interface operably connected to the one or more processors, the user interface effective to transmit one or more commands to the processor; and

e) a storage operably connected to the processor, the storage effective for storing a data structure, wherein the data structure includes the digital media files and the textual annotation.

48. The system of claim 47, wherein the operations further comprise displaying a selection of the NMO environment that excludes the first digital media file.

49. The system of claim 47, wherein the operations further comprise:

receiving a selection from the second user of the first digital media file; and

displaying a selection of the NMO environment that includes the first digital media file.

50. The system of claim 49, wherein the first digital media file is restored to a full graphical representation.

51. The system of claim 47, wherein the operations further comprise displaying a selection of the NMO to destructively exclude the digital media files.