Method and apparatus for providing multimedia data using event index

• US 8,718,447 B2
• Filed: 09/25/2006
• Issued: 05/06/2014
• Est. Priority Date: 10/17/2005
• Status: Active Grant

• Current Assignee: Samsung Electronics Co. Ltd.
• Original Assignee: Samsung Electronics Co. Ltd.
• Inventors: Yang, Gyung-hye, Chung, Seung-Nyung, Choi, Ki-wan, Yoo, Myung-hyun
• Primary Examiner(s): JONES, HEATHER RAE
• Application Number:
  US11/525,896
• Publication Number:
  US 20070088833A1
• Time in Patent Office: 2,780 Days
• Field of Search: None
• US Class Current: 386/278
• CPC Class Codes: G06F 16/41 Indexing; Data structures therefor; Storage structures
• CPC Class Codes: G06F 16/436 using biological or physiological data of a human being, e.g. blood pressure, facial expression, gestures

Abstract

A method and apparatus for providing multimedia data using an event index are provided. The method includes; to detect a biometric signal of a user when the user captures a multimedia data, digitize the detected biometric signal and indexing the multimedia data using the result of the digitization, and selectively edit the indexed multimedia data and playing back the result of the selectively editing. Accordingly, it is possible to facilitate the management of moving data files by indexing part of a moving data corresponding to an emotional event of a user who has captured the moving data, editing the moving data according to the results of the indexing, and playing back the edited moving data to provide a preview of the moving data.

Claims

1. A method of providing a moving image in a mobile device, comprising:

detecting a biometric signal of a user of the mobile device while the user captures the moving image;

digitizing the detected biometric signal and synchronizing the digitized biometric signal with the captured moving image;

receiving an input for setting a threshold for the digitized biometric signal; and

filtering out, based on the set threshold, at least a portion of the captured moving image,

wherein the digitizing the detected biometric signal and synchronizing the digitized biometric signal with the captured moving image comprises;

obtaining a signal processing parameter from the digitized biometric signal;

comparing when a peak value of the signal processing parameter is detected with when a user behavioural response to the moving image occurs; and

determining an emotional event period for the signal processing parameter based upon the comparison,

wherein the user behavioural response to the moving image comprises a facial response and/or a vocal response made by the user.

2. The method of claim 1, further comprising:

playing back a remaining portion of the moving image after the filtering out.

3. The method of claim 1, wherein the biometric signal of the user is detected by a sensor, the sensor is a photoplethysmography (PPG) sensor, and the detected biometric signal is synchronized with a button used for capturing the moving image.

4. The method of claim 1, wherein the digitizing the detected biometric signal and synchronizing the digitized biometric signal with the capture moving image further comprises:

recording the moving image in synchronization with the emotional event period.

5. The method of claim 4, wherein the determining the emotional event period for the signal processing parameter comprises:

determining whether an overlapping time period of a first interval before and after a peak value of the signal processing parameter is detected and a second interval before and after a user behavioral response to the moving image occurs meets a predetermined standard; and

determining the overlapping time period of the first interval before and after the peak value of the signal processing parameter is detected and the second interval before and after the user behavioural response to the moving image occurs as the emotional event period for the signal processing parameter if the corresponding overlapping time period meets the predetermined standard.

6. The method of claim 5, wherein the predetermined standard is whether the interval between the time when the peak value of the signal processing parameter is detected and the time when the user behavioural response to the moving image occurs is within a predetermined value, which is determined in consideration of the characteristic of the human body in that the time when a biometric signal is detected is not likely to coincide with the time when a user behavioural response occurs.

7. The method of claim 6, wherein the signal processing parameter is SDNN10, which is a standard deviation of heartbeat intervals of the user within ten seconds.

8. The method of claim 5, wherein the predetermined standard is whether the time when the user behavioural response to the moving image occurs prior to the detection of the peak value of the signal processing parameter and the interval between the time when the peak value of the signal processing parameter is detected and the time when the user behavioural response to the moving image occurs is within a predetermined value.

9. The method of claim 8, wherein the signal processing parameter is SDNN10, which is a standard deviation of heartbeat intervals of the user within ten seconds.

10. The method of claim 5, wherein the filtering out comprises:

adjusting the amount of the remaining part of the moving image after the filtering out, to be played back.

11. The method of claim 10, further comprises:

playing back the remaining part of the moving image after the filtering out according to the results of the adjustment performed.

12. The method of claim 11, wherein the playing back comprises:

indicating the times when the peak value of the signal processing parameter is respectively detected by displaying a plurality of pointers on a progress bar of a multimedia player, which plays back the moving image; and

if the user chooses one of the peak value detection times respectively indicated by the pointers, playing back part of the moving image corresponding to the peak value detection time chosen by the user.

13. The method of claim 11, wherein the playing back comprises:

representing the emotional event period as a bar, which is filled with shades of a color and is located above a progress bar of a multimedia player, which plays back the moving image; and

if the user chooses a location on the bar, playing back part of the moving image corresponding to the chosen location.

14. The method of claim 10, wherein the signal processing parameter is SDNN10, which is a standard deviation of heartbeat intervals of the user within ten seconds.

15. The method of claim 10, wherein the adjusting the amount of the remaining part of the moving image after the filtering out to be played back comprises adjusting a summarization level using a graphic user interface (GUI) of a multimedia player, which plays back the moving image, the summarization level representing the ratio of the length of the emotional event period and the length of the moving image.

16. The method of claim 15, wherein the adjusting the amount of the remaining part of the moving image after the filtering out to be played back comprises gradually or discontinuously varying the ratio of the amount of part of the moving image to be played-back and the amount of the filtered out part of the moving image not to be played back according to the summarization level,

wherein the ratio is represented by a progress bar of the multimedia player.

17. The method of claim 16, wherein the user determines the summarization level by directly typing in a desired summarization rate and a desired summarization time to the multimedia player.

18. The method of claim 15, wherein the user determines the summarization level by directly typing in a desired summarization rate and a desired summarization time to the multimedia player.

19. The method of claim 5, wherein the signal processing parameter is SSNN10, which is a standard deviation of heartbeat intervals of the user within ten seconds.

20. The method of claim 4, wherein the signal processing parameter is SDNN10, which is a standard deviation of heartbeat intervals of the user within ten seconds.

21. The method of claim 1, wherein the moving image in synchronization with the digitized biometric signal is selectively edited and played back.

22. The method of claim 1, wherein the filtering out comprises summarizing the captured moving image in synchronization with the digitized biometric signal, using the set threshold.

23. A non-transitory computer-readable recording medium storing a computer program to execute the method of claim 1.

24. An apparatus for providing a moving image in a mobile device, comprising:

a biometric signal detection unit to detect a biometric signal of a user of the mobile device while the user captures the moving image;

an indexing unit to digitize the detected biometric signal and to synchronize the digitized biometric signal with the captured moving image;

a user interface to receive an input for setting a threshold for the digitized biometric signal; and

a playback unit to filter out, based on the set threshold, at least a portion of the captured moving image,

wherein the indexing unit comprises;

a parameter calculator to obtain a signal processing parameter from the digitized biometric signal; and

an event period generator to determine an emotional event period for the signal processing parameter, wherein the event period generator;

compares when a peak value of the signal processing parameter is detected with when a user behavioural response to the moving image occurs; and

determines an emotional event period for the signal processing parameter based upon the comparison,

wherein the user behavioural response to the moving image comprises a facial response and/or a vocal response made by the user.

25. The apparatus of claim 24, wherein the detecting unit includes a sensor to detect the biometric signal of the user,

wherein the sensor is a photoplethysmography (PPG) sensor and the biometric signal is synchronized with a button used for capturing the moving image.

26. The apparatus of claim 25, wherein the indexing unit further comprises:

a synchronization recorder which records the moving image in synchronization with the emotional event period.

27. The apparatus of claim 26, wherein the event period generator determines the emotional event period by determining whether an overlapping time period of a first interval before and after a peak value of the signal processing parameter is detected and a second interval before and after a user behavioural response to the moving image occurs meets a predetermined standard;

and

determines the overlapping time period of the first interval before and after the peak value of the signal processing parameter is detected and the second interval before and after the user behavioural response to the moving image occurs as the emotional event period for the signal processing parameter if the corresponding overlapping time period meets the predetermined standard.

28. The apparatus of claim 27, wherein the predetermined standard is whether the interval between the time when the peak value of the signal processing parameter is detected and the time when the user behavioural response to the moving image occurs is within a predetermined value, which is determined in consideration of the characteristic of the human body in that the time when a biometric signal is detected is not likely to coincide with the time when a user behavioural response occurs.

29. The apparatus of claim 28, wherein the signal processing parameter is SSNN10, which is a standard deviation of heartbeat intervals of the user within ten seconds.

30. The apparatus of claim 27, wherein the predetermined standard is whether the time when the user behavioural response to the moving image occurs prior to the detection of the peak value of the signal processing parameter and the interval between the time when the peak value of the signal processing parameter is detected and the time when the user behavioural response to the moving image occurs is within a predetermined value.

31. The apparatus of claim 30, wherein the signal processing parameter is SSNN10, which is a standard deviation of heartbeat intervals of the user within ten seconds.

32. The apparatus of claim 27, wherein the playback unit comprises an adjustment unit which adjusts the amount of a remaining part of the moving image after the filtering out, to be played back,

wherein the playback unit plays back the moving image according to the results of the adjustment performed by the adjustment unit.

33. The apparatus of claim 32, wherein the signal processing parameter is SDNN10, which is a standard deviation of heartbeat intervals of the user within ten seconds.

34. The apparatus of claim 32, wherein the adjustment unit adjusts a summarization level using a graphic user interface (GUI) of a multimedia player, which plays back the moving image, wherein the summarization level represents the ratio of the length of the emotional event period and the length of the moving image.

35. The apparatus of claim 34, wherein the adjustment unit gradually or discontinuously varies the ratio of the amount of the remaining part of the moving image after the filtering out to be played back and the amount of the filtered out part of the moving image not to be played back according to the summarization level, wherein the ratio is represented by a progress bar of the multimedia player.

36. The apparatus of claim 35, wherein the user determines the summarization level by directly typing in a desired summarization rate and a desired summarization time to the multimedia player.

37. The apparatus of claim 34, wherein the user determines the summarization level by directly typing in a desired summarization rate and a desired summarization time to the multimedia player.

38. The apparatus of claim 32, wherein the playback unit indicates the times when the peak value of the signal processing parameter is respectively detected by displaying a plurality of pointers on a progress bar of a multimedia player, which plays back the moving image;

and if the user chooses one of the peak value detection times respectively indicated by the pointers, plays back part of the moving image corresponding to the peak value detection time chosen by the user.

39. The apparatus of claim 32, wherein the playback unit represents the emotional event period as a bar, which is filled with shades of a color and is located above a progress bar of a multimedia player, which plays back the moving image;

and if the user chooses a location on the bar, playing back part of the moving image corresponding to the chosen location.

40. The apparatus of claim 27, wherein the signal processing parameter is SSNN10, which is a standard deviation of heartbeat intervals of the user within ten seconds.

41. The apparatus of claim 26, wherein the signal processing parameter is SSNN10, which is a standard deviation of heartbeat intervals of the user within ten seconds.

42. The apparatus of claim 24, wherein the moving image in synchronization with the digitized biometric signal is selectively edited and played back.

43. A method of recording and/or replaying a moving image in a mobile device, comprising:

detecting a biometric signal of a user while the user captures the moving image;

digitizing the detected biometric signal;

synchronizing the digitized biometric signal with the moving image;

receiving an input for setting a threshold for the digitized biometric signal; and

filtering out, based on the set threshold, at least a portion of the captured moving image;

recording the captured moving image with the digitized biometric signal in synchronization with the moving image; and

selectively replaying the moving image, by using the result of the filtering out,

wherein the digitizing the detected biometric signal and synchronizing the digitized biometric signal with the moving image comprises;

obtaining a signal processing parameter from the digitized biometric signal;

comparing when a peak value of the signal processing parameter is detected with when a user behavioural response to the moving image occurs; and

determining an emotional event period for the signal processing parameter based upon the comparison,

wherein the user behavioural response to the moving image comprises a facial response and/or a vocal response made by the user.

44. The method of claim 43, wherein the filtering out comprises filtering out the captured moving image automatically.

45. The method of claim 43, wherein in the biometric signal is a heart rate signal of the user of the mobile device.

46. The method of claim 43, wherein the biometric signal of the user of the mobile device is detected by using a sensor, the sensor is a Photoplethysmography (PPG) sensor.

47. A mobile apparatus for providing a moving image, comprising:

a biometric signal detection unit to detect a biometric signal of a user of the mobile apparatus, while the user captures the moving image;

an indexing unit to digitize the detected biometric signal and to index the captured moving image using the digitized biometric signal;

a user interface to receive an input for setting a threshold for the digitized biometric signal; and

a playback unit to filter out, based on the set threshold, at least a portion of the indexed moving image, and to play back only the remaining portion of the moving image after the filtering out,

wherein the playback unit filters out a portion of the indexed moving image, if the digitized biometric signal intensity in synchronization with the portion is less than the set threshold,

wherein the indexing unit comprises;

a parameter calculator to obtain a signal processing parameter from the digitized biometric signal; and

an event period generator to determine an emotional event period for the signal processing parameter, wherein the event period generator;

compares when a peak value of the signal processing parameter is detected with when a user behavioural response to the moving image occurs; and

determines an emotional event period for the signal processing parameter based upon the comparison,

wherein the user behavioural response to the moving image comprises a facial response and/or a vocal response made by the user.

48. A method of recording and/or reproducing a moving image using a mobile device, comprising:

detecting a biometric signal of a user of the mobile device, while the user captures the moving image;

digitizing the detected biometric signal;

synchronizing the digitized biometric signal with the captured moving image;

receiving an input for setting a threshold for the digitized biometric signal;

filtering out, based on the set threshold, at least a portion of the captured moving image;

selectively recording the captured moving image in synchronization with the digitized biometric signal; and

reproducing the recorded moving image,

wherein the digitizing the detected biometric signal and synchronizing the digitized biometric signal with the moving image comprises;

obtaining a signal processing parameter from the digitized biometric signal;

comparing when a peak value of the signal processing parameter is detected with when a user behavioural response to the moving image occurs; and

determining an emotional event period for the signal processing parameter based upon the comparison,

wherein the user behavioural response to the moving image comprises a facial response and/or a vocal response made by the user.

49. An apparatus for recording and/or playing back a moving image, comprising:

a sensor;

a biometric signal detection unit to detect a biometric signal of a user of the apparatus using the sensor while the user captures a moving image;

an indexing unit to digitize the detected biometric signal and to index the captured moving image by synchronizing the digitized biometric signal with the captured moving image, while recording the captured moving image;

a user interface to receive an input for setting a threshold for the digitized biometric signal;

an editing unit to filter out a portion of the indexed moving image, based on the set threshold; and

a playback unit to vary the playback time of the moving image using the digitized biometric signal in synchronization with the moving image,

wherein the editing unit filters out a portion of the indexed moving image, if the synchronized digitized biometric signal intensity of the portion is less than the set threshold,

wherein the indexing unit comprises;

a parameter calculator to obtain a signal processing parameter from the digitized biometric signal; and

an event period generator to determine an emotional event period for the signal processing parameter, wherein the event period generator;

compares when a peak value of the signal processing parameter is detected with when a user behavioural response to the moving image occurs; and

determines an emotional event period for the signal processing parameter based upon the comparison,

wherein the user behavioural response to the moving image comprises a facial response and/or a vocal response made by the user.