Physiologic audio fingerprinting

• US 9,445,763 B2
• Filed: 05/28/2014
• Issued: 09/20/2016
• Est. Priority Date: 04/18/2013
• Status: Active Grant

• Current Assignee: Digimarc Corporation
• Original Assignee: Digimarc Corporation
• Inventors: Davis, Bruce L., Rodriguez, Tony F., Conwell, William Y., Thagadur Shivappa, Shankar, Sharma, Ravi K., Gibson, Richard F.
• Primary Examiner(s): GUERRA-ERAZO, EDGAR X
• Application Number:
  US14/289,364
• Publication Number:
  US 20150006186A1
• Time in Patent Office: 846 Days
• Field of Search: 704/226, 704/270, 704/270.1, 704/275
• US Class Current: 1/1
• CPC Class Codes: A61B 2576/02 specially adapted for a particular organ or body part
• CPC Class Codes: A61B 5/0075 by spectroscopy, i.e. measuring spectra, e.g. Raman spectroscopy, infrared absorption spectroscopy A61B5/0071 takes precedence
• CPC Class Codes: A61B 5/0077 Devices for viewing the surface of the body, e.g. camera, magnifying lens
• CPC Class Codes: A61B 5/1032 Determining colour for diagnostic purposes measuring colour in general G01J3/46
• CPC Class Codes: A61B 5/1034 by means of colour cards
• CPC Class Codes: A61B 5/441 Skin evaluation, e.g. for skin disorder diagnosis
• CPC Class Codes: A61B 5/444 Evaluating skin marks, e.g. mole, nevi, tumour, scar
• CPC Class Codes: A61B 5/445 Evaluating skin irritation or skin trauma, e.g. rash, eczema, wound, bed sore
• CPC Class Codes: A61B 5/4806 Sleep evaluation A61B5/4821 takes precedence; devices for inducing sleep A61M21/02
• CPC Class Codes: A61B 5/6898 Portable consumer electronic devices, e.g. music players, telephones, tablet computers
• CPC Class Codes: A61B 5/7246 using correlation, e.g. template matching or determination of similarity
• CPC Class Codes: A61B 5/7278 Artificial waveform generation or derivation, e.g. synthesising signals from measured signals A61B5/327 takes precedence
• CPC Class Codes: A61B 5/7282 Event detection, e.g. detecting unique waveforms indicative of a medical condition A61B5/4094, A61B5/4818, A61B5/349, A61B5/0823 take precedence
• CPC Class Codes: A61B 5/7425 Displaying combinations of multiple images regardless of image source, e.g. displaying a reference anatomical image with a live image geometric transforms for image registration G06T3/14; enhancement using image fusion G06T5/50; algorithms for registration of images G06T7/30
• CPC Class Codes: A61B 5/743 Displaying an image simultaneously with additional graphical information, e.g. symbols, charts, function plots
• CPC Class Codes: A61B 5/7485 Automatic selection of region of interest
• CPC Class Codes: G06F 16/245 Query processing
• CPC Class Codes: G06F 16/248 Presentation of query results
• CPC Class Codes: G06T 2207/30088 Skin; Dermal
• CPC Class Codes: G06T 5/40 using histogram techniques
• CPC Class Codes: G06T 7/0012 : Biomedical image inspection
• CPC Class Codes: G10L 19/018 : Audio watermarking, i.e. embedding inaudible data in the audio signal
• CPC Class Codes: G16H 30/20 : for handling medical images, e.g. DICOM, HL7 or PACS
• CPC Class Codes: G16H 30/40 : for processing medical images, e.g. editing
• CPC Class Codes: G16H 40/67 : for remote operation
• CPC Class Codes: G16H 50/20 : for computer-aided diagnosis, e.g. based on medical expert systems
• CPC Class Codes: G16H 50/70 : for mining of medical data, e.g. analysing previous cases of other patients
• CPC Class Codes: G16Z 99/00 : Subject matter not provided for in other main groups of this subclass
• CPC Class Codes: Y02A 50/30 : Against vector-borne diseases, e.g. mosquito-borne, fly-borne, tick-borne or waterborne diseases whose impact is exacerbated by climate change
• CPC Class Codes: Y02A 90/10 : Information and communication technologies [ICT] supporting adaptation to climate change, e.g. for weather forecasting or climate simulation

Abstract

Audio sounds are captured from a subject'"'"'s body, e.g., using a smartphone or a worn array of microphones. Plural features are derived from the captured audio, and serve as fingerprint information. One such feature may be a time interval over which a threshold part of spectral energy in the audio is expressed. Another may be a frequency bandwidth within which a second threshold part of the spectral energy is expressed. Such fingerprint information is provided to a knowledge base that contains reference fingerprint data and associated metadata. The knowledge base matches the fingerprint with reference fingerprint data, and provides associated metadata in return—which can comprise diagnostic information related to the captured sounds. In some arrangements, an audio signal or pressure waveform stimulates the body at one location, and is sensed at another, to discern information about the intervening transmission medium. A great variety of other features and arrangements are also detailed.

Claims

1. A method comprising the acts:

collecting physiologic information in a data structure, the collected information corresponding to physiologic sensor data gathered by plural non-professional users;

collecting, in said data structure, professional evaluation information corresponding to at least some of the physiologic information;

receiving query information, the query information corresponding to physiologic sensor data gathered by a non-professional user;

consulting the data structure in determining result information; and

communicating at least some of the result information to said non-professional user;

wherein the result information depends on correlation between the query information, the collected physiology information, and the professional evaluation information;

wherein the physiologic sensor data comprises audio data.

2. A method comprising the acts:

capturing audio sounds from a subject'"'"'s body, using a worn or held portable apparatus;

deriving plural features from the captured audio, the plural features comprising fingerprint information corresponding to the captured sounds;

providing said fingerprint information to a knowledge base, the knowledge base containing reference fingerprint data and metadata associated therewith;

receiving metadata associated with one or more of the reference fingerprint data in the knowledge base, to which the provided fingerprint information corresponds; and

providing physiologic- or health-related information based on the received metadata.

3. The method of claim 2 in which the received metadata includes candidate diagnostic information related to said captured audio sounds.

4. The method of claim 2 in which the capturing comprises capturing pulmonary, respiratory, or bowel sounds from the subject'"'"'s body, and the provided information relates to said pulmonary, respiratory or bowel sounds.

5. The method of claim 2 in which the sounds captured from the subject'"'"'s body are cough sounds, and the provided information indicates whether the cough sounds originate in the subject'"'"'s upper or lower airway.

6. The method of claim 2 in which the portable apparatus comprises plural microphones, and the method includes processing signals from said microphones so that the microphones serve as a beam-forming array, the method including determining from the beam-forming array a relative location of a source of the sounds from within the subject'"'"'s body.

7. The method of claim 2 in which the audio sounds comprise speech from the subject, and the provided physiologic- or health-related information concerns subject depression.

8. The method of claim 2 in which the audio sounds comprise stimulus externally introduced to the body and sensed by said apparatus.

9. The method of claim 2 in which capturing audio sounds from the subject'"'"'s body comprises capturing audio sounds by a sensor-equipped belt worn around the subject'"'"'s belly.

10. A non-transitory computer readable medium containing software instructions operative to cause an associated system programmed by said instructions to perform acts including:

capturing audio sounds from a subject'"'"'s body, using a worn or held portable sensor apparatus;

deriving plural features from the captured audio, the plural features comprising fingerprint information corresponding to the captured sounds;

providing said fingerprint information to a knowledge base, the knowledge base containing reference fingerprint data and metadata associated therewith;

receiving metadata associated with one or more of the reference fingerprint data in the knowledge base, to which the provided fingerprint information corresponds; and

providing physiologic- or health-related information based on the received metadata.

11. The medium of claim 10 in which the capturing comprises capturing pulmonary, respiratory, or bowel sounds from the subject'"'"'s body, and the provided information relates to said pulmonary, respiratory or bowel sounds.

12. The medium of claim 10 in which the sensor comprises plural microphones, and the acts include processing signals from said microphones so that the microphones serve as a beam-forming array, the acts further including determining from the beam-forming array a relative location of a source of the sounds from within the subject'"'"'s body.