Audio signal de-identification

• US 20060190263A1
• Filed: 02/23/2005
• Published: 08/24/2006
• Est. Priority Date: 02/23/2005
• Status: Active Grant

• Current Assignee: Multimodal Technologies Incorporated (3M Company)
• Original Assignee: Multimodal Technologies Incorporated (3M Company)
• Inventors: Koll, Detlef, Finke, Michael
• Granted Patent: US 7,502,741 B2
• Time in Patent Office: Days
• US Class Current: 704/270
• CPC Class Codes: G10L 15/1822 Parsing for meaning understanding
• CPC Class Codes: G16H 10/20 for electronic clinical trials or questionnaires
• CPC Class Codes: G16H 15/00 ICT specially adapted for medical reports, e.g. generation or transmission thereof

Abstract

Techniques are disclosed for automatically de-identifying spoken audio signals. In particular, techniques are disclosed for automatically removing personally identifying information from spoken audio signals and replacing such information with non-personally identifying information. De-identification of a spoken audio signal may be performed by automatically generating a report based on the spoken audio signal. The report may include concept content (e.g., text) corresponding to one or more concepts represented by the spoken audio signal. The report may also include timestamps indicating temporal positions of speech in the spoken audio signal that corresponds to the concept content. Concept content that represents personally identifying information is identified. Audio corresponding to the personally identifying concept content is removed from the spoken audio signal. The removed audio may be replaced with non-personally identifying audio.

Claims

1. A method comprising steps of:

(A) identifying a first portion of an original audio signal, the first portion representing sensitive information; and

(B) producing a modified audio signal in which the identified first portion is protected against unauthorized disclosure.

2. The method of claim 1, wherein the sensitive information comprises personally identifying information.

3. The method of claim 1, wherein the step (B) comprises a step of removing the identified first portion from the original audio signal to produce the modified audio signal, whereby the modified audio signal does not include the identified first portion.

4. The method of claim 1, wherein the step (B) comprises a step of applying a security measure to the identified first portion to produce the modified audio signal, wherein the identified first portion in the modified audio signal is protected against unauthorized disclosure.

5. The method of claim 1, wherein the step of applying the security measure comprises a step of encrypting the identified first portion.

6. The method of claim 1, wherein the step (A) comprises steps of:

(A)(1) identifying a candidate portion of the original audio signal;

(A)(2) determining whether the candidate portion represents personally identifying information; and

(A)(3) identifying the candidate portion as the first portion if the candidate portion represents personally identifying information.

7. The method of claim 6, wherein the step (A) further comprises a step of:

(A)(4) repeating steps (A)(1)-(A)(3) for a plurality of candidate portions until a candidate portion representing personally identifying information is identified.

8. The method of claim 6, further comprising a step of:

(C) repeating steps (A) and (B) for a plurality of portions of the audio signal.

9. The method of claim 8, wherein the step (C) comprises a step of repeating steps (A) and (B) for all portions of the audio signal, thereby producing a de-identified audio signal not representing any personally identifying information.

10. The method of claim 6, wherein the step (A)(2) comprises steps of:

(A)(2)(a) identifying a concept represented by the candidate portion; and

(A)(2)(b) determining that the candidate portion represents personally identifying information if the concept is a personally identifying concept.

11. The method of claim 10, wherein the step (A)(2)(b) comprises a step of determining that the candidate portion represents personally identifying information if the concept is of a type that is in a predetermined set of personally identifying concept types.

12. The method of claim 11, wherein the predetermined set of personally identifying concept types includes at least one concept type selected from the list consisting of a personal name, personal address, personal telephone number, gender, birthday, medical diagnosis, drug prescription, and social security number.

13. The method of claim 1, further comprising a step of:

(C) replacing the first audio signal portion with a second audio signal portion that does not include personally identifying information.

14. The method of claim 13, wherein the step (C) comprises a step of replacing the first audio signal portion with a non-speech audio signal.

15. The method of claim 1, wherein the step (C) comprises a step of replacing the first audio signal portion with a beep audio signal.

16. The method of claim 13, wherein the step (C) comprises a step of replacing the identified portion with an audio signal representing a type of concept represented by the identified portion.

17. The method of claim 16, wherein the type of concept is one of a personal name, personal address, personal telephone number, gender, birthday, medical diagnosis, drug prescription, and social security number.

18. The method of claim 16, wherein the step (C) comprises a step of replacing the first audio signal portion with an audio signal representing both a type of concept represented by the first audio signal portion and an identifier of the first audio signal portion which distinguishes the first audio signal portion from other portions of the original audio signal representing the same type of concept.

19. The method of claim 1, wherein the step (A) comprises a step of identifying a header portion of the original audio signal, and wherein the step (B) comprises a step of removing the identified header portion from the original audio signal.

20. The method of claim 1, wherein the step (A) comprises steps of:

(A)(1) generating a report, the report comprising;

(a) content representing information in the original audio signal, and (b) at least one timestamp indicating at least one temporal position of at least one portion of the original audio signal corresponding to the content;

(A)(2) identifying a first personally identifying concept in the report;

(A)(3) identifying a first timestamp in the report corresponding to the first personally identifying concept; and

(A)(4) identifying a portion of the original audio signal corresponding to the first personally identifying concept by using the first timestamp.

21. The method of claim 20, further comprising a step of repeating steps (A)(2)-(A)(4) for all of the personally identifying concepts in the report, and wherein the step (B) comprises a step of removing portions of the original audio signal corresponding to all of the personally identifying concepts in the report.

22. The method of claim 20, further comprising a step of:

(C) removing the first personally identifying concept from the report to produce a de-identified report.

23. The method of claim 22, further comprising a step of:

(D) transcribing the de-identified audio signal based on the de-identified audio signal and the de-identified report.

24. The method of claim 1, further comprising a step of:

(C) transcribing the de-identified audio signal to produce a transcript of the de-identified audio signal.

25. The method of claim 24, wherein the transcript comprises a literal transcript of the de-identified audio signal.

26. The method of claim 24, wherein the transcript comprises a non-literal transcript of the de-identified audio signal.

27. The method of claim 24, wherein the step (C) comprises a step of transcribing the de-identified audio signal using an automated speech recognizer.

28. The method of claim 1, wherein the original audio signal comprises a live audio signal.

29. The method of claim 1, wherein the original audio signal comprises a recorded audio signal.

30. A method comprising steps of:

(A) identifying a first portion of an original audio signal, the first portion representing personally identifying information, by performing steps of;

(A)(1) generating a report, the report comprising;

(a) content representing information in the original audio signal, and (b) at least one timestamp indicating at least one temporal position of at least one portion of the original audio signal corresponding to the content;

(A)(2) identifying a first personally identifying concept in the report;

(A)(3) identifying a first timestamp in the report corresponding to the first personally identifying concept;

(A)(4) identifying a portion of the original audio signal corresponding to the first personally identifying concept by using the first timestamp; and

(B) removing the identified first portion from the original audio signal to produce a modified audio signal, which does not include the identified first portion.

31. The method of claim 30, further comprising a step of:

(C) transcribing the de-identified audio signal to produce a transcript of the de-identified audio signal.

32. A system comprising:

sensitive information identification means comprising means identifying a first portion of an original audio signal, the first portion representing sensitive information; and

modified audio signal production means comprising means for producing a modified audio signal in which the identified first portion is protected against unauthorized disclosure.

33. The system of claim 32, wherein the sensitive information comprises personally identifying information.

34. The system of claim 32, wherein the modified audio signal production means comprises means for removing the identified first portion from the original audio signal to produce the modified audio signal, whereby the modified audio signal does not include the identified first portion.

35. The system of claim 32, wherein the modified audio signal production means comprises means for applying a security measure to the identified first portion to produce the modified audio signal, wherein the identified first portion in the modified audio signal is protected against unauthorized disclosure.

36. The system of claim 32, wherein the means for applying the security measure comprises a step of encrypting the identified first portion.

37. The system of claim 32, wherein the sensitive information identification means comprises:

means for identifying a candidate portion of the original audio signal;

means for determining whether the candidate portion represents personally identifying information; and

means for identifying the candidate portion as the first portion if the candidate portion represents personally identifying information.

38. The system of claim 32, further comprising:

means for replacing the first audio signal portion with a second audio signal portion that does not include personally identifying information.

39. The system of claim 32, wherein the sensitive information identification means comprises means for identifying a header portion of the original audio signal, and wherein the modified audio signal production means comprises means for removing the identified header portion from the original audio signal.

40. The system of claim 32, wherein the sensitive information identification means comprises:

means for generating a report, the report comprising;

(a) content representing information in the original audio signal, and (b) at least one timestamp indicating at least one temporal position of at least one portion of the original audio signal corresponding to the content;

means for identifying a first personally identifying concept in the report;

means for identifying a first timestamp in the report corresponding to the first personally identifying concept; and

means for identifying a portion of the original audio signal corresponding to the first personally identifying concept by using the first timestamp.

41. The system of claim 32, further comprising:

means for transcribing the de-identified audio signal to produce a transcript of the de-identified audio signal.

43. The system of claim 32, further comprising:

means for transcribing the de-identified audio signal to produce a transcript of the de-identified audio signal.

42. A system comprising:

identification means comprising means for identifying a first portion of an original audio signal, the first portion representing personally identifying information, the identification means comprising;

means for generating a report, the report comprising;

(a) content representing information in the original audio signal, and (b) at least one timestamp indicating at least one temporal position of at least one portion of the original audio signal corresponding to the content;

means for identifying a first personally identifying concept in the report;

means for identifying a first timestamp in the report corresponding to the first personally identifying concept;

means for identifying a portion of the original audio signal corresponding to the first personally identifying concept by using the first timestamp; and

means for removing the identified first portion from the original audio signal to produce a modified audio signal, which does not include the identified first portion.