System, method and medium editing moving pictures using biometric signals

• US 8,157,729 B2
• Filed: 03/09/2007
• Issued: 04/17/2012
• Est. Priority Date: 05/09/2006
• Status: Expired due to Fees

• Current Assignee: Samsung Electronics Co. Ltd.
• Original Assignee: Samsung Electronics Co. Ltd.
• Inventors: Yang, Gyung-hye, Choi, Ki-wan, Chung, Seung-Nyung, Kim, Youn-ho, Lee, Yong-beom
• Primary Examiner(s): Johnson, III, Henry M
• Assistant Examiner(s): Rajan, Kai
• Application Number:
  US11/715,907
• Publication Number:
  US 20070263978A1
• Time in Patent Office: 1,866 Days
• Field of Search: 600/300, 600/301, 128903-905, 128/920, 386/278, 386/E5.003, 386/52
• US Class Current: 600/300
• CPC Class Codes: G11B 27/034 on discs G11B27/036, G11B27/038 take precedence
• CPC Class Codes: G11B 27/11 by using information not detectable on the record carrier

Abstract

A system, method and medium editing a moving picture using biometric signals is provided. The system includes a biometric signal generation module to measure signals that reflect an emotional state of a user while capturing a moving picture, and to generate a first and a second biometric signal based on the measured signals, an event section extraction module to extract a first event section that reflects preferences of the user from a playback section of the moving picture based on the first biometric signal, extract a second event section that reflects preferences of the user from the playback section of the moving picture based on a second biometric signal, and extract a final event section based on the first and second event sections, and an indexing module to edit the moving picture by indexing the final event section in synchronization with the playback section of the moving picture.

Claims

1. A system editing a moving picture using biometric signals, the system comprising a processor and processor-executable modules comprising:

a biometric signal generation module to measure signals that reflect an emotional state of a user who captures a moving picture, and to generate a first and a second biometric signal based on the measured signals;

an event section extraction module to extract a first event section that reflects preferences of the user from a playback section of the moving picture based on the first biometric signal, extract a second event section that reflects preferences of the user from the playback section of the moving picture based on the second biometric signal, and extract a final event section based on where the first and second event sections overlap each other; and

an indexing module, controlled by the processor, to edit the moving picture such that the final event section is indexed in synchronization with the playback section of the moving picture.

2. The system of claim 1, wherein the signals that reflect the emotional state of the user comprise a signal that reflect variations in the amount of blood flow resulting from variations in the heart rate of the user and a signal that reflects variations in the skin resistance and the degree of perspiration of the user.

3. The system of claim 1, wherein the first biometric signal is generated by a photoplethysmograghy (PPG) sensor, and the second biometric signal is generated by a galvanic skin response (GSR) sensor, wherein the PPG sensor and the GSR sensor are attached to a portion of a digital device configured to capture the moving picture.

4. The system of claim 3, wherein the biometric signal generation module comprises:

a first biometric signal generation unit to generate the first biometric signal using the PPG sensor that is placed in contact with the skin of the user;

a second biometric signal generation unit to generate the second biometric signal using the GSR sensor that is placed in contact with the skin of the user;

a PPG noise section setting unit to set a PPG noise section using PPG data, for the first biometric signal; and

a GSR noise section setting unit to set a GSR noise section using GSR data, for the second biometric signal.

5. The system of claim 4, wherein the PPG noise section setting unit sets an RR interval which is a time interval between peaks in the PPG data, removes noise from the RR interval, and sets a time interval between T_RR(n)+RR(n)/2 and T_RR(n+1)−

RR(n+1)12 (where RR(n) indicates n-th data of the RR interval, T_RR(n) indicates the time of detection of RR(n), RR(n+1) indicates (n+1)-th data of the RR interval, and T_RR(n+1) indicates the time of detection of RR(n+1)) as the PPG noise section if T_RR(n+1)−

T_RR(n)>

{RR(n)+RR(n+1)}/2°

C. (where C is a constant between 1 and

     3).

6. The system of claim 4, wherein the GSR data comprises SIL data which indicates absolute skin resistance measured as a signal and SIR data which indicates the rate of change in the SIL data over time, and the GSR noise section setting module extracts a SIR count value indicating the number of peaks in the SIR data from the SIR data, sets a point where SIL(t+Δ

t1)−

SIL(t)>

Th1 (where SIL(t) indicates SIL data measured at a time t, Δ

t1 indicates a predetermined time interval, and Th1 indicates a first threshold) as a beginning point of the GSR noise section, sets a point where 0<

SIL(t)−

SIL(t+Δ

t2)<

Th2 and SIL(t)<

Th3 (where Δ

t2 indicates a predetermined time interval and Th2 and Th3 respectively indicate second and third thresholds) as an ending point of the GSR noise section, and then removes noise from the GSR noise section.

7. The system of claim 6, wherein Δ

t1 is within the range of 0.1-0.5, Δ

t2 is within the range of 0.2-1.0, and Th1 is within the range of 20-60 kΩ

, Th2 is within the range of 0-5 kΩ

, and Th3 is higher than 2MΩ

when the SIL data ranges between 10 kΩ and

2 MΩ

.

8. The system of claim 3, wherein the event section extraction module comprises:

a first event section extraction unit to extract n first event sections respectively having first through n-th priorities from the PPG noise section;

a second event section extraction unit to extract m second event sections respectively having first through m-th priorities from the GSR noise section; and

a final event section extraction module to determine overlapping sections between the n first event sections and the m second event sections as first priority overlapping sections, and to extract a first priority overlapping section that is longer than a minimum required time set by the user as the final event section.

9. The system of claim 8, wherein, if none of the first priority overlapping sections are longer than the minimum required time, the final event section extraction module determines overlapping sections between the n first event sections and the GSR noise section as second priority overlapping sections and extracts a second priority overlapping section that is longer than the minimum required time as the final event section.

10. The system of claim 9, wherein, if none of the second priority overlapping sections are longer than the minimum required time, the final event section extraction module determines overlapping sections between the m second event sections and the PPG noise section as third priority overlapping sections, and extracts a third priority overlapping section that is longer than the minimum required time as the final event section.

11. The system of claim 10, wherein, if none of the third priority overlapping sections are longer than the minimum required time, the final event section extraction module prioritizes remaining first and second event sections to be generated as final event sections, and extracts the remaining first or second event section with a highest priority as the final event section.

12. The system of claim 3, wherein the first event section comprises a time interval including the time of detection of a peak in the first biometric signal, the second event section comprises a time interval including the time of detection of a SIR count value higher than a SIR threshold that is determined based on the average and the standard deviation of SIR data measured during a predetermined time interval.

13. The system of claim 1, wherein the captured moving picture is encoded together with the first and second biometric signals.

14. The system of claim 1, wherein the final event extraction module determines the overlapping section of the first and second event sections as the final event section when the overlapping section exceeds a minimum required time.

15. The method of claim 1, wherein the overlapping section of the first and second event sections is determined as the final event section when the overlapping section exceeds a minimum required time.

16. A method of editing a moving picture using biometric signals, the method comprising:

measuring signals that reflect an emotional state of a user who captures a moving picture;

generating, by way of a processor, a first and a second biometric signal based on the measured signals;

extracting a first event section that reflects preferences of the user from a playback section of the moving picture based on the first biometric signal;

extracting a second event section that reflects preferences of the user from the playback section of the moving picture based on the second biometric signal;

extracting a final event section based on where the first and second event sections overlap each other; and

editing the moving picture, by way of the processor, by indexing the final event section in synchronization with the playback section of the moving picture.

17. The method of claim 16, wherein the signals that reflect the emotional state of the user comprise a signal that reflect variations in the amount of blood flow resulting from variations in the heart rate of the user and a signal that reflects variations in the skin resistance and the degree of perspiration of the user.

18. The method of claim 16, wherein the first biometric signal is generated by a photoplethysmograghy (PPG) sensor, and the second biometric signal is generated by a galvanic skin response (GSR) sensor, wherein the PPG sensor and the GSR sensor are attached to a predetermined portion of a digital device capturing the moving picture.

19. The method of claim 18, wherein the measuring of the signals comprises:

generating the first biometric signal using the PPG sensor that is placed in contact with the skin of the user, and generating the second biometric signal using the GSR sensor that is placed in contact with the skin of the user;

setting a PPG noise section using PPG data, wherein the PPG data is the first biometric signal; and

setting a GSR noise section using GSR data, wherein the GSR data is the second biometric signal.

20. The method of claim 19, wherein the setting of the PPG noise section using PPG data comprises:

setting an RR interval which is a time interval between peaks in the PPG data;

removing noise from the RR interval; and

setting a time interval between T_RR(n)+RR(n)/2 and T_RR(n+1)—

RR(n+1)/2 (where RR(n) indicates n-th data of the RR interval, T_RR(n) indicates the time of detection of RR(n), RR(n+1) indicates (n+1)-th data of the RR interval, and T_RR(n+1) indicates the time of detection of RR(n+1)) as the PPG noise section if T_RR(n+1)−

T_RR(n)>

{RR(n)+RR(n+1)}/2*C (where C is a constant between 1 and

     3).

21. The method of claim 19, wherein the GSR data comprises SIL data, which indicates absolute skin resistance measured as a signal and SIR data which indicates the rate of change in the SIL data over time, and the setting of the GSR noise section using GSR data comprises:

extracting a SIR count value indicating the number of peaks in the SIR data from the SIR data;

setting a point where SIL(t+Δ

t1)−

SIL(t)>

Th1 (where SIL(t) indicates SIL data measured at a time t, Δ

t1 indicates a predetermined time interval, and Th1 indicates a first threshold) as a beginning point of the GSR noise section, and setting a point where 0<

SIL(t)−

SIL(t+Δ

t2)<

Th2 and SIL(t)<

Th3 (where Δ

t2 indicates a predetermined time interval and Th2 and Th3 respectively indicate second and third thresholds) as an ending point of the GSR noise section; and

removing noise from the GSR noise section.

22. The method of claim 21, wherein Δ

t1 is within the range of 0.1-0.5, Δ

t2 is within the range of 0.2-1.0, and Th1 is within the range of 20-60 kΩ

, Th2 is within the range of 0-5 kΩ

, and Th3 is higher than 2MΩ

when the SIL data ranges between 10 kΩ and

2MΩ

.

23. The method of claim 18, wherein the extracting of the first event section comprises:

extracting n first event sections respectively having first through n-th priorities from the PPG noise section, and extracting m second event sections respectively having first through m-th priorities from the GSR noise section; and

determining the overlapping sections between the n first event sections and the m second event sections as first priority overlapping sections, and extracting a first priority overlapping section that is longer than a minimum required time set by the user as the final event section.

24. The method of claim 23, wherein the extracting of the first event section further comprises:

if none of the first priority overlapping sections are longer than the minimum required time, determining overlapping sections between the n first event sections and the GSR noise section as second priority overlapping sections and extracting a second priority overlapping section that is longer than the minimum required time as the final event section;

if none of the second priority overlapping sections are longer than the minimum required time, determining overlapping sections between the m second event sections and the PPG noise section as third priority overlapping sections, and extracting a third priority overlapping section that is longer than the minimum required time as the final event section; and

if none of the third priority overlapping sections are longer than the minimum required time, prioritizing remaining first and second event sections to be generated as final event sections, and extracting the remaining first or second event section with a highest priority as the final event section.

25. The method of claim 18, wherein the first event section comprises a time interval including the time of detection of a peak in the first biometric signal, the second event section comprises a time interval including the time of detection of a SIR count value higher than a SIR threshold that is determined based on the average and the standard deviation of SIR data measured during a predetermined time interval.

26. At least one medium comprising computer readable code to control at least one processing element to implement the method of claim 16.

27. The method of claim 16, wherein the captured moving picture is encoded together with the first and second biometric signals.

28. A moving picture capturing device editing a moving picture using biometric signals, the device including a processor and processor-executable modules comprising:

a first biometric sensor attached to a first portion of the moving picture capturing device and measuring a first biometric during capture of a moving picture;

a second biometric sensor attached to a second portion of the moving picture capturing device and measuring a second biometric during capture of the moving picture;

a biometric signal generation module generating a first and a second biometric signal based on the measured first and second biometrics;

an event section extraction module extracting a first event section that reflects preferences of the user from a playback section of the moving picture based on the first biometric signal, extracting a second event section that reflects preferences of the user from the playback section of the moving picture based on the second biometric signal, and extracting a final event section based on where the first and second event sections overlap; and

an indexing module, controlled by the processor, to edit the moving picture such that the final event section is indexed in synchronization with the playback section of the moving picture.