Image with audio conversation system and method

• US 9,977,591 B2
• Filed: 02/13/2014
• Issued: 05/22/2018
• Est. Priority Date: 10/01/2013
• Status: Active Grant

• Current Assignee: Filmstrip, Inc.
• Original Assignee: Ambient Consulting, LLC
• Inventors: Grossman, Andrew, Krutsch, Kenneth F.
• Primary Examiner(s): Taylor, Nicholas
• Assistant Examiner(s): Kim, Chong G
• Application Number:
  US14/179,602
• Publication Number:
  US 20150095804A1
• Time in Patent Office: 1,559 Days
• Field of Search: 709206
• CPC Class Codes: G06F 2203/04806 Zoom, i.e. interaction techniques or interactors for controlling the zooming operation
• CPC Class Codes: G06F 3/0482 Interaction with lists of selectable items, e.g. menus
• CPC Class Codes: G06F 3/04845 for image manipulation, e.g. dragging, rotation, expansion or change of colour
• CPC Class Codes: G06F 3/04883 for inputting data by handwriting, e.g. gesture or text
• CPC Class Codes: H04L 51/10 Multimedia information

Abstract

A system and method are presented to allow audio communication between users concerning an image. The originator of the communication uses a mobile device app to select an image and record an audio commentary. The image, audio commentary, and metadata are submitted to a cloud server for storage. The app uses the server to analyze a recipient address to determine the preferred mode of delivery. If the recipient is a known user of the app, the file is delivered without combining the image, audio commentary, and metadata into a standard movie file. Otherwise, the originator'"'"'s app delivers the file through MMS or e-mail for the recipient as a movie file for viewing using a standard video player.

Claims

1. A computerized method comprising:

a) at a first mobile device, presenting a plurality of still visual content items on a touchscreen display of the first mobile device;

b) at the first mobile device and while presenting the visual content, recording an audio commentary through a microphone on the first mobile device;

c) at the first mobile device and while recording the audio commentary;

i) identifying user touch input on the touchscreen display at a particular time relative to the audio commentary wherein the user touch input comprises a plurality of user requests to transition to a new one of the plurality of still visual content items, and

ii) presenting on the touchscreen display, in response to the user touch input, a visual augmentation of the visual content wherein the visual augmentation comprises visual transitions between the plurality of still images;

d) at the first mobile device, encoding the visual content, the audio commentary and the visual augmentation into a video file;

e) at the first mobile device, generating metadata defining the visual augmentation and the timing of the visual augmentation with respect to the audio commentary, wherein the visual augmentation is defined in metadata so as to allow the visual augmentation to be recreated solely through the metadata;

f) determining that a second mobile device is not operating a custom app capable of rendering the visual content, the audio commentary, and the visual augmentation outside of the video file and determining that a third mobile device is operating the custom app;

g) at the first mobile device and based on the determining step, transmitting the video file to the second mobile device and separately transmitting the visual content, the audio commentary, and the metadata without the video file to the third mobile device, wherein metadata is sufficient to allow the third mobile device to present the visual transitions at particular times in the audio commentary corresponding to the user touch input during recording of the audio commentary.

12. The method of claim 1, wherein the still visual content items comprise still photographic images.

13. The method of claim 1, further comprising receiving input at the first mobile device of a URL to present at a selected time during playback of the audio commentary, wherein the step of transmitting the visual content, the audio commentary, and the visual augmentation to the second mobile device further comprises sending the URL to allow display of the URL by the second mobile device at the selected time.

2. A computerized method comprising:

a) at a first mobile device, presenting a plurality of still visual content items on a touchscreen display of the first mobile device;

b) at the first mobile device and while presenting the visual content, recording an audio commentary through a microphone on the first mobile device;

c) at the first mobile device and while recording the audio commentary;

i) identifying user touch input on the touchscreen display at a particular time relative to the audio commentary, wherein the user touch input comprises a plurality of user requests to transition to a new one of the plurality of still visual content items, and

ii) presenting on the touchscreen display, in response to the user touch input, a visual augmentation of the visual content wherein the visual augmentation comprises visual transitions between the plurality of still images;

d) at the first mobile device, generating metadata defining the visual augmentation and the timing of the visual augmentation with respect to the audio commentary, wherein the visual augmentation is defined in metadata so as to allow the visual augmentation to be recreated solely through the metadata;

e) at the first mobile device, transmitting the visual content, the audio commentary, and the metadata to a second mobile device,

f) at the second mobile device, using the metadata to recreate the visual augmentation at the particular time relative to the audio commentary when presenting the visual content and the audio commentary, wherein the second mobile device presents the visual transitions at particular times in the audio commentary corresponding to the user touch input during recording of the audio commentary.

3. The method of claim 2, wherein the first mobile device transmits the visual content items, the audio commentary, and the metadata to the second mobile device as an attachment to a message in a standard messaging protocol selected from a set consisting of:

e-mail, SMS messaging, and MMS messaging.

4. The method of claim 2, wherein the first mobile device transmits the visual content items, the audio commentary, and the metadata to a remote server to be stored under a unique identifier;

wherein the first mobile device further transmits the unique identifier to the second mobile device to allow the second mobile device to access the visual content items, the audio commentary, and the metadata from the remote server.

5. The method of claim 4, wherein the unique identifier is sent as a stub file attachment to a message in a standard messaging protocol selected from a set consisting of:

e-mail, SMS messaging, and MMS messaging.

6. The method of claim 4, wherein the unique identifier is sent as part of a message sent in a standard messaging protocol selected from a set consisting of:

e-mail, SMS messaging, and MMS messaging.

7. The method of claim 4, wherein the visual content items, the audio commentary, and the metadata are combined into a single file that is transmitted to the remote server for later download to the second mobile device.

8. The method of claim 7, wherein the single file includes a plurality of additional audio commentaries on the visual content items in addition to the audio commentary recorded by the first mobile device, wherein the metadata within the single file contains information concerning each commentary so as to allow each commentary to be separately accessible and playable.

9. The method of claim 8, wherein the visual content items are separately identified in the metadata stored in the single file.

10. The method of claim 2, wherein the metadata defines different transition timing between the separate visual content items for each of the commentaries stored in the single file.

11. The method of claim 8, wherein the metadata defines different visual augmentations for each of the commentaries stored in the single file.

14. A system for transmitting audio commentaries on video content comprising:

a) a mobile device having

i) a microphone,

ii) a touchscreen display,

iii) a processor,

iv) a network interface,

v) non-transitory, physical memory, and

vi) a cellular interface for communicating cellular messages with a remote mobile device;

b) cellular messaging programming on the non-transitory, physical memory providing instructions that program the processor to transmit and receive the cellular messages via the cellular interface, and to maintain a list of incoming cellular messages, the cellular messaging programming instructions including an application programming interface to receive content and commands from other programming on the mobile device and to submit content and the messages to other programming on the mobile device;

c) app programming on the non-transitory, physical memory comprising instructions that program the processor to;

i) present a plurality of still visual content items on the touchscreen display;

ii) while presenting the visual content, record an audio commentary through the microphone;

iii) while recording the audio commentary, identifying a plurality of user touch input requests to transition to a new one of the plurality of still visual content items;

iv) generating metadata defining the timing of the transitions between the still visual content items with respect to the audio commentary, wherein the metadata allows the transitions to be recreated in sync with the audio commentary solely through the metadata;

iv) submit messaging data including the metadata to the application programming interface for transmission to the remote mobile device through the cellular interface to allow the remote mobile device to play the transitions between the plurality of still visual content items at particular times in the audio commentary corresponding to the user touch input requests made during recording of the audio commentary.

15. The system of claim 14, wherein the app programming further programs the processor to transmit the visual content, metadata, and audio commentary via the network interface to a remote server for storage in connection with a unique message identifier, further wherein the messaging data submitted to the application programming interface comprises the unique message identifier.

16. The system of claim 15, wherein the app programming further programs the processor to receive incoming data from the remote mobile device through the application programming interface, the incoming data comprising a second message identifier, wherein the app programming programs the processor to request content from the remote server associated with the second message identifier.

17. The system of claim 16, wherein the incoming data was formatted so as to inform the cellular messaging programming to pass the incoming data to the app programming via the application programming interface.