Methods and systems for audio-visual synchronization

• US 8,861,925 B1
• Filed: 07/28/2010
• Issued: 10/14/2014
• Est. Priority Date: 07/28/2010
• Status: Active Grant

• Current Assignee: Intuit, Inc.
• Original Assignee: Intuit, Inc.
• Inventors: Ohme, Phillip J.
• Primary Examiner(s): Vaughn, Jr., William C
• Assistant Examiner(s): ADAMS, EILEEN M
• Application Number:
  US12/845,683
• Time in Patent Office: 1,539 Days
• Field of Search: None
• US Class Current: 386/207
• CPC Class Codes: H04N 5/76 Television signal recording
• CPC Class Codes: H04N 5/932 Regeneration of analogue synchronisation signals
• CPC Class Codes: H04N 9/8211 the additional signal being a sound signal H04N9/835 takes precedence

Abstract

Audio-visual synchronization using retrieval of visual data through a network to enhance the content presented to the user and to provide the user with a more meaningful and engaging audio-visual experience. An audio player such as an electronic book reader or stereo plays an audio file. The audio file, whether in analog form as played through a speaker or in digital form, is provided to a controller, which scans the audio file to detect a trigger or cue identifying visual data stored in a data store or database that is local or accessible through a network. The controller accesses the data store or database, downloads the pre-determined visual data, and controls the audio player and display such that playing of audio data and display of the retrieved visual data are synchronized with each other.

Claims

1. A computer-implemented method for audio-visual synchronization, the method comprising computer-implemented steps of:

receiving an audio file from a first source, wherein an audio player of a user also receives the audio file;

identifying, at a pre-determined location within the audio file, a trigger indicating when stored pre-determined visual data should be retrieved from a second source different than the first source;

accessing the second source through a network, the second source comprising data store or database storing visual data;

searching the data store or database of the second source to identify visual data associated with the identified trigger indicating when stored pre-determined visual data should be retrieved from the second source different from the first source;

receiving the identified pre-determined visual data from the second source through the network; and

coordinating playing of the audio file with the audio player and providing the received pre-determined visual data to a display such that the visual data is displayed with a pre-determined segment of the audio file played by the audio player.

2. The method of claim 1, the audio file being received from the first source comprising server of a website.

3. The method of claim 1, the audio file being received from the first source comprising a storage device of an automobile.

4. The method of claim 3, the playing of the audio file being coordinated with display of the pre-determined visual data by a heads up display unit on a windshield of the automobile.

5. The method of claim 1, the audio file comprising data of an audio podcast.

6. The method of claim 1, the audio file being played by an audio player comprising a mobile communication device.

7. The method of claim 1, the pre-determined visual data comprising an image or a series of images.

8. The method of claim 1, the trigger comprising a pre-determined time into the audio file.

9. The method of claim 1, the trigger comprising an original, unmodified pre-determined word or phrase of the audio file.

10. The method of claim 9, the audio player comprising a speaker, wherein sounds generated by the speaker during playing of the audio file are detected by a microphone, and an output of the microphone is received and analyzed to identify the trigger within the audio file.

11. The method of claim 9, the trigger being identified based at least in part upon the audio file data and independently of sounds generated by a speaker of the audio player.

12. The method of claim 9, the trigger comprising a pre-determined word within the audio file.

13. The method of claim 9, the trigger comprising a group of words.

14. The method of claim 9, wherein non-overlapping audio file segments are scanned to identify the trigger such that each word or group of words in the audio file is scanned a single time.

15. The method of claim 1, further comprising modifying the audio file by embedding the trigger within the audio file at the pre-determined location.

16. The method of claim 1, wherein the trigger is associated with a pre-determined segment of the audio file, and the pre-determined visual data is retrieved from the data store or database and displayed before playing of the pre-determined segment begins.

17. The method of claim 1, wherein the trigger is associated with a first segment of the audio file, and the pre-determined visual data is retrieved from the data store or database and displayed during playing of the first segment.

18. The method of claim 1, the audio player playing a first segment of the audio file, the first segment being received, wherein the trigger of the first segment is identified, the data store or database is accessed, and the pre-determined visual data is identified and displayed during playing of the first segment.

19. The method of claim 1, the audio player playing a first segment of the audio file, a second segment of the audio file being received while the first segment is playing, wherein the trigger of the second segment is identified, the data store or database is accessed and the pre-determined visual data is identified during playing of the first segment and before playing of the second segment.

20. The method of claim 19, further comprising storing pre-determined visual data retrieved from the data store or database during playing of the first segment.

21. The method of claim 1, the pre-determined visual data being displayed for a pre-determined amount of time.

22. The method of claim 1, the audio file comprising a second trigger indicating a length of time the pre-determined visual data should be displayed.

23. The method of claim 1, the pre-determined visual data being identified without input by the user.

24. The method of claim 1, the data of the audio file being continuously analyzed to identify triggers, access the data store or database, and identify respective pre-determined visual data in real time while receiving the audio file.

25. The method of claim 1, the audio player comprising

a display, wherein playing of the audio file and display of the pre-determined visual data by the same audio player are coordinated.

26. The method of claim 1, wherein playing of the audio file by the audio player and display of the pre-determined visual data by a different and separate device are coordinated.

27. The method of claim 1, the pre-determined visual data comprising an advertisement.

28. The method of claim 1, the audio file being received from the first source comprising a portable electronic or optical storage media.

29. The method of claim 1, the audio file comprising data of the first source comprising an audio book.

30. The method of claim 1, the audio file comprising data of the first source comprising a mp3 file.

31. The method of claim 1, the audio file being played by the first source comprising an electronic book reading device capable of wireless or cellular communication.

32. The method of claim 1, wherein overlapping audio file segments are scanned utilizing a sliding function such overlapping words or groups of words are scanned to identify the trigger.