Method and apparatus to synchronize multi-media events

• US 20050070241A1
• Filed: 09/30/2003
• Published: 03/31/2005
• Est. Priority Date: 09/30/2003
• Status: Active Grant

• Current Assignee: Sony Ericsson Mobile Communications AB (Sony Group Corp.)
• Original Assignee: Sony Ericsson Mobile Communications AB (Sony Group Corp.)
• Inventors: Zak, Robert A., Northcutt, John W., Yates, Charles Randall
• Granted Patent: US 7,966,034 B2
• Time in Patent Office: Days
• US Class Current: 455/265
• CPC Class Codes: G10H 1/0008 Associated control or indicating means
• CPC Class Codes: G10H 2230/021 Mobile ringtone, i.e. generation, transmission, conversion or downloading of ringing tones or other sounds for mobile telephony; Special musical data formats or protocols therefor
• CPC Class Codes: G10H 2240/056 MIDI or other note-oriented file format
• CPC Class Codes: G10H 2240/325 Synchronizing two or more audio tracks or files according to musical features or musical timings
• CPC Class Codes: H04M 1/72409 by interfacing with external accessories hands-free H04M1/60
• CPC Class Codes: H04M 1/72442 for playing music files
• CPC Class Codes: H04M 19/041 Encoding the ringing signal, i.e. providing distinctive or selective ringing capability
• CPC Class Codes: H04M 2250/52 including functional features of a camera

Abstract

A mobile communications device has a wireless transceiver to facilitate communications, memory, and a processor. The processor may be configured to analyze an audio file stored in the memory, and extract or generate synchronizing information from the audio file. The processor synchronizes one or more complementary multi-media effects, such as lights, a camera, or a tactile feedback generator, with the audio file based on the synchronizing information. Alternatively, a user may input synchronizing information about the audio file via an interface on the communications device. The processor may then control the complementary multi-media effects in synchronization with the input synchronizing information.

Claims

1. A mobile communications device comprising:

a wireless transceiver;

a memory to store an audio file; and

a processor to analyze the audio file and extract or generate synchronizing information, and to synchronize a complementary multi-media effect with the audio file based on the synchronizing information.

2. The mobile communications device of claim 1 wherein said information is timing information.

3. The mobile communications device of claim 1 wherein said information is instrument type information.

4. The mobile communications device of claim 1 wherein said processor stores the synchronizing information in said memory.

5. The mobile communications device of claim 1 wherein said processor generates a control signal during playback of the audio file to control the complementary multi-media effect according to the synchronizing information.

6. The mobile communications device of claim 5 wherein the complementary multi-media effect comprises tactile feedback, and said control signal controls a tactile feedback generator in synchronization with the synchronizing information.

7. The mobile communications device of claim 5 wherein the complementary multi-media effect comprises one or more lights, and said control signal controls the one or more lights in synchronization with the synchronizing information.

8. The mobile communications device of claim 5 wherein the complementary multi-media effect comprises a camera, and said control signal activates the camera to take a picture in synchronization with the synchronizing information.

9. The mobile communications device of claim 5 wherein the complementary multi-media effect comprises a video sequence, and said control signal controls a display to playback said video sequence in synchronization with the synchronizing information.

10. The mobile communications device of claim 1 further comprising a system interface, and wherein said processor generates a control signal during playback of the audio file to control an external device connected via the system interface according to the synchronizing information.

11. A mobile communications device comprising:

memory to store an audio file;

an interface to input synchronizing information regarding the audio file by a user of the mobile communication device; and

a processor to synchronize a complementary multi-media effect with the audio file based on the synchronizing information.

12. The mobile communications device of claim 11 wherein said interface comprises a menu displaying the synchronizing information.

13. The mobile communications device of claim 11 wherein the synchronizing information is a timing signature.

14. The mobile communications device of claim 11 wherein the synchronizing information is an instrument type.

15. The mobile communications device of claim 11 wherein said processor associates user-selected synchronizing information with the audio file and stores the synchronizing information in said memory.

16. The mobile communications device of claim 11 wherein said processor generates a control signal during playback of the audio file to control the complementary multi-media effect according to user-selected synchronizing information.

17. The mobile communications device of claim 16 wherein the complementary multi-media effect comprises tactile feedback, and said control signal controls a tactile feedback generator in synchronization with the user-selected synchronizing information.

18. The mobile communications device of claim 16 wherein the complementary multi-media effect comprises one or more lights, and said control signal controls the one or more lights in synchronization with the user-selected synchronizing information.

19. The mobile communications device of claim 16 wherein the complementary multi-media effect comprises a camera, and said control signal controls the camera to take a picture in synchronization with the user-selected synchronizing information.

20. The mobile communications device of claim 16 wherein the complementary multi-media effect comprises a video sequence, and said control signal controls a display to playback said video sequence in synchronization with the user-selected synchronizing information.

21. The mobile communications device of claim 11 further comprising a system interface, and wherein said processor generates a control signal during playback of the audio file to control an external device connected via the system interface according to the synchronizing information.

22. A method to synchronize multi-media effects with an audio file in a mobile communications device, the method comprising:

analyzing an audio file stored in memory of the mobile communications device to extract or generate synchronizing information; and

synchronizing a complementary multi-media effect in the mobile communications device with the audio file based on the synchronizing information.

23. The method of claim 22 wherein the synchronizing information comprises timing information.

24. The method of claim 22 wherein the synchronizing information comprises instrument type information.

25. The method of claim 22 further comprising storing the synchronizing information in memory.

26. The method of claim 22 wherein synchronizing a complementary multi-media effect comprises generating a control signal during playback of the audio file to control the complementary multi-media effect according to the synchronizing information.

27. The method of claim 26 further comprising controlling a tactile feedback generator in synchronization with the synchronizing information.

28. The method of claim 26 further comprising controlling one or more lights in synchronization with the synchronizing information.

29. The method of claim 26 further comprising controlling a camera to take one or more pictures in synchronization with the synchronizing information.

30. The method of claim 26 further comprising displaying a sequence of one or more video frames in synchronization with the synchronizing information.

31. A method of synchronizing one or more multi-media effects with an audio file in a mobile communications device, the method comprising:

selecting synchronizing information regarding an audio file stored in memory of the mobile communications device; and

synchronizing a complementary multi-media effect in the mobile communications device with the audio file based on the synchronizing information.

32. The method of claim 31 wherein selecting information regarding an audio file comprises displaying a menu option to a user of the mobile communications device, said menu option comprising synchronizing information to be associated with the audio file.

33. The method of claim 32 wherein said menu option comprises a list of timing signatures.

34. The method of claim 32 wherein said menu option comprises a list of instrument types.

35. The method of claim 31 further comprises storing the synchronizing information in memory.

36. The method of claim 31 wherein synchronizing a complementary multi-media effect comprises generating a control signal during playback of the audio file to control the complementary multi-media effect according to the selected synchronizing information.

37. The method of claim 36 further comprising controlling a tactile feedback generator in synchronization with the selected synchronizing information.

38. The method of claim 36 further comprising controlling one or more lights in synchronization with the selected synchronizing information.

39. The method of claim 36 further comprising controlling a camera to take one or more pictures in synchronization with the selected synchronizing information.

40. The method of claim 36 further comprising displaying a sequence of one or more video frames in synchronization with the selected synchronizing information.

41. A method of synchronizing one or more complementary multi-media effects with an audio file in a mobile communications device, the method comprising:

selecting a sample from an audio file stored in memory in a mobile communications device;

analyzing said sample to determine synchronizing information; and

synchronizing the one or more complementary multi-media effects in the mobile communications device with the audio file based on the synchronizing information.

42. The method of claim 41 wherein analyzing said sample to determine synchronizing information comprises sectioning said sample into a plurality of measures, each said measure comprising an equivalent number of notes.

43. The method of claim 42 wherein analyzing said sample to determine synchronizing information further comprises determining the notes that occur within a desired beat in each said measure.

44. The method of claim 43 wherein analyzing said sample to determine synchronizing information further comprises calculating a weight value by summing a velocity parameter of a corresponding note-on event for each said note that occurs within said desired beat in each said measure.

45. The method of claim 41 wherein analyzing said sample to determine synchronizing information comprises calculating a first value and a second value based on a first candidate time signature and a second candidate time signature, respectively.

46. The method of claim 45 wherein analyzing said sample to determine synchronizing information further comprises selecting the highest value from said first and second values to select a time signature.

47. The method of claim 46 wherein synchronizing the one or more complementary multi-media effects with the audio file comprises associating said selected time signature with the audio file.

48. The method of claim 41 wherein analyzing said sample to determine synchronizing information comprises sampling the output of a MIDI synthesizer in the mobile communications device.

49. The method of claim 48 wherein analyzing said sample to determine synchronizing information further comprises detecting a peak amplitude within said sample.

50. The method of claim 49 further comprising comparing said detected peak amplitude to a threshold value.

51. The method of claim 50 wherein synchronizing the one or more complementary multi-media effects with the audio file comprises generating a control signal based on the comparison of said detected peak amplitude and said threshold value.

52. The method of claim 50 wherein synchronizing the one or more multi-media effects with the audio file varying the intensity of the one or more complementary multi-media effects based on the comparison of said detected peak amplitude and said threshold value.

53. The method of claim 50 wherein synchronizing the one or more complementary multi-media effects comprises varying the duration of activation of the one or more complementary multi-media effects based on the comparison of said detected peak amplitude and said threshold value.

54. The method of claim 41 further comprising overwriting selected information in the audio file with the synchronizing information.

55. The method of claim 41 further comprising storing the synchronizing information in memory of the mobile communications device.

56. The method of claim 41 wherein synchronizing the one or more complementary multi-media effects with the audio file comprises generating a control signal to the one or more multi-media effects based on the synchronizing information during playback of the audio file.

57. A circuit comprising:

a microprocessor programmed to extract or generate synchronizing information from an audio stream, and to synchronize one or more complementary multi-media effects with the audio stream during playback of the audio stream.