Learning algorithm to detect human presence in indoor environments from acoustic signals

• US 10,515,654 B2
• Filed: 07/01/2019
• Issued: 12/24/2019
• Est. Priority Date: 06/30/2014
• Status: Active Grant

• Current Assignee: Rajeev Conrad Nongpiur
• Original Assignee: Rajeev Conrad Nongpiur
• Inventors: Nongpiur, Rajeev Conrad
• Primary Examiner(s): Sarpong, Akwasi M
• Application Number:
  US16/459,094
• Publication Number:
  US 20190325893A1
• Time in Patent Office: 176 Days
• Field of Search: 704231
• CPC Class Codes: G10L 17/02 Preprocessing operations, e.g. segment selection; Pattern representation or modelling, e.g. based on linear discriminant analysis [LDA] or principal components; Feature selection or extraction
• CPC Class Codes: G10L 17/26 Recognition of special voice characteristics, e.g. for use in lie detectors; Recognition of animal voices
• CPC Class Codes: G10L 25/15 the extracted parameters being formant information
• CPC Class Codes: G10L 25/51 for comparison or discrimination
• CPC Class Codes: G10L 25/90 Pitch determination of speech signals

Abstract

A system is described that constantly learns the sound characteristics of an indoor environment to detect the presence or absence of humans within that environment. A detection model is constructed and a decision feedback approach is used to constantly learn and update the statistics of the detection features and sound events that are unique to the environment in question. The learning process may not only rely on acoustic signal, but may also make use of signals derived from other sensors such as range sensor, motion sensors, pressure sensors, and video sensors.

Claims

1. A method for detecting presence of one or more individual human in an indoor environment comprising:

refining at least one of models of a plurality of models based at least in part on a determination that a first individual human is present in the indoor environment, so that the models over time become more accurate and more specific to respective characteristics of acoustic features of each of a set of individual human occupants and to the particular indoor environment, wherein refining the models includes;

starting with generic models to detect, from acoustic features, the presence and absence of each of one or more individual human occupants in the indoor environment; and

during high-detection probability of presence of a particular individual human occupant, modifying a human-presence model for that particular individual human occupant so that relevant acoustic features specific to the particular individual human occupant and the indoor environment are emphasized, thereby improving accuracy of the human-presence model for that particular individual human occupant over time.

2. The method of claim 1, wherein the determination that the first individual human is present in the indoor environment comprises:

comparing acoustic events with each of the plurality of models; and

computing a likelihood score for each of the plurality of models based at least on the comparing the acoustic events with each of the plurality of models.

3. The method of claim 2, wherein the determination that the first individual human is present in the indoor environment further comprises:

identifying a first model of the plurality of models having a greatest likelihood score, the first model associated with the first individual human; and

determining that the first individual human is present in the indoor environment based at least in part on the first model associated with the first individual human being identified as having a greatest likelihood score.

4. The method of claim 1, further comprising detecting acoustic events in an audio signal associated with the first individual human present in the indoor environment, the detecting comprising:

extracting directions of the audio signals;

extracting magnitude of the audio signals;

extracting frequency spectrogram of the audio signals;

extracting autocorrelation coefficients of the audio signals; and

extracting time information of the audio signals.

5. The method of claim 1, wherein refining at least one of the models of the plurality of models based at least in part on the determination that the first individual human is present in the indoor environment includes:

initializing a generic model for human-presence or human-absence;

computing a probabilistic score for presence or absence of each individual human occupant from a joint probability of acoustic events; and

updating a prior probability distribution of input features for a human-presence model or a human-absence model for each individual human occupant.

6. The method for claim 5, wherein the updating the prior probability distribution of the input features for the human-presence model or the human-absence model of each individual human occupant includes:

confirmation of human presence or absence from one or more sensors that is different than a sensor that generated an audio signal from the indoor environment; and

compute a score of presence or absence of each occupant from the human-presence model or human-absence model.

7. The method of claim 6 further comprising:

when presence or absence of an individual human occupant is confirmed or the score of presence and absence for an occupant exceeds a prescribed threshold;

updating joint probability of acoustic events and human presence or absence for the individual human occupant; and

updating a prior distribution of input features for the human-presence model or human absence model for the individual human occupant.

8. The method of claim 1, further comprising computing a likelihood score for presence or absence of each occupant in the indoor environment comprising:

computing, using the respective models, detection probabilities of detected acoustic events during presence or absence of a respective individual human occupant; and

computing from a human-presence model or a human-absence model the likelihood score for the presence or absence of the respective individual human occupant, from the detection probabilities of the detected acoustic events.

9. The method of claim 8, wherein computing the detection probabilities of the detected acoustic events during presence or absence of each individual human occupant includes:

computing the acoustic features corresponding to the acoustic events from microphone array signals obtained from a microphone array in the indoor environment; and

computing a probability of acoustic features during presence and absence of each occupant by taking into account prior probability distributions of acoustic features during determined presence and absence of each respective individual human occupant in the indoor environment.

10. A computer implemented method comprising:

refining at least one model of a plurality of models based at least in part on a determination that a first individual is present in an indoor environment, the refining such that the models become more accurate in determining presence of specific individuals being present in the indoor environment, the refining of the model of the plurality of models including;

during high-detection probability of presence of a particular individual occupant, modifying an occupant-presence model for that particular individual occupant so that relevant features specific to the particular individual occupant and the indoor environment are emphasized to improve accuracy of the occupant-presence model for that particular individual occupant.

11. The computer implemented method of claim 10, wherein each model of the plurality of models is associated with a respective individual of a plurality of individuals.

12. The computer implemented method of claim 10, wherein the determining that the first individual is present in the indoor environment is based at least in part on a first model associated with the first individual being identified as having a greatest likelihood score for being present in the indoor environment.

13. The computer implemented method of claim 10, wherein refining at least one of the models of the plurality of models based at least in part on the determining that the first individual is present in the indoor environment, refining the at least one model a plurality of successive instances over time to become increasingly more accurate and more specific to characteristics of the features of each of the respective individual occupants and the particular indoor environment.

14. The computer implemented method of claim 10, wherein determining that a first individual is present in the indoor environment is based at least in part on determining a separate probability score for:

the absence of each of the individual occupants in the indoor environment, and

the presence of each of the individual occupants in the indoor environment.

15. The computer implemented method of claim 10, wherein the refining of a model of the plurality of models further includes starting with a generic occupant-presence model to detect presence of at least one specific individual in the indoor environment and during high-detection probability of presence of a particular individual occupant, modifying the generic occupant-presence model for the particular individual occupant so that relevant features specific to the particular individual occupant and the indoor environment are emphasized to improve accuracy of the occupant-presence model for that particular individual occupant.

16. The computer implemented method of claim 10, further comprising obtaining one or more signal from the indoor environment, the one or more signal obtained from the indoor environment comprising a signal obtained from at least one of a range sensor, motion sensor, pressure sensor, or video sensor.

17. The computer implemented method of claim 10, wherein refining at least one of the models of the plurality of models comprises determining a pitch and vocal-tract length of a plurality of individuals.

18. The computer implemented method of claim 10, wherein refining at least one of the models of the plurality of models based at least in part on the determining that the first individual is present in the indoor environment includes:

initializing a generic model for individual occupant presence or absence;

computing a probabilistic score for presence or absence of each individual occupant from a joint probability of acoustic events; and

updating a prior probability distribution of input features for a presence model or an absence model for each individual occupant.

19. The computer implemented method of claim 18, wherein the updating the prior probability distribution of the input features for the presence model or the absence model of each individual occupant includes:

confirmation of individual occupant presence or absence from one or more sensors that is different than a sensor that obtained an audio signal from the indoor environment; and

computing a score of presence or absence of each occupant from the presence model or absence model.

20. The computer implemented method of claim 19, further comprising:

when presence or absence of an individual occupant is confirmed or the score of presence and absence for an occupant exceeds a prescribed threshold;

updating joint probability of events and presence or absence for the individual occupant; and

updating a prior distribution of input features for the presence model or absence model for the individual occupant.