Acoustic map command contextualization and device control

• US 9,812,132 B2
• Filed: 12/31/2015
• Issued: 11/07/2017
• Est. Priority Date: 12/31/2015
• Status: Active Grant

• Current Assignee: General Electric Company
• Original Assignee: General Electric Company
• Inventors: Vandroux, Stphane, Haddad, Mireille, Riou, Patrice
• Primary Examiner(s): Baker, Charlotte M
• Application Number:
  US14/986,613
• Publication Number:
  US 20170194008A1
• Time in Patent Office: 677 Days
• Field of Search: 600407, 600424, 600427, 342 14, 342 53, 89 111, 725 14, 725 18, 324338, 704275
• CPC Class Codes: G10L 15/005 Language recognition
• CPC Class Codes: G10L 15/1822 Parsing for meaning understanding
• CPC Class Codes: G10L 15/22 Procedures used during a speech recognition process, e.g. man-machine dialogue
• CPC Class Codes: G10L 17/00 Speaker identification or verification techniques
• CPC Class Codes: G10L 17/22 Interactive procedures; Man-machine interfaces
• CPC Class Codes: G10L 2015/223 Execution procedure of a spoken command
• CPC Class Codes: G10L 21/028 using properties of sound source
• CPC Class Codes: G10L 25/93 Discriminating between voiced and unvoiced parts of speech signals G10L25/90 takes precedence
• CPC Class Codes: H04R 3/005 for combining the signals of two or more microphones specially adapted for hearing aids H04R25/407

Abstract

Provided is a system where users performing a coordinated process are localized in a complex environment based upon audio input. Audio commands are detected and executed based on system user vocalization. Available commands are limited by user status, location, process type and process progress. Command execution is limited by the presence and locations of system users, non-users, or extraneous equipment.

Claims

1. A system comprising:

acoustical sensors disposed as an array, coupled to an analog to digital converter and a digital data output transceiver, forming an acoustic array sensor;

at least one acoustic array sensor operatively connected to an audio analysis processor;

the audio analysis processor operatively connected to a command database and a control module;

the command and control module operatively connected to one or more automated machines;

wherein multiple array sensors are distributed throughout a designated control area.

2. The system claim 1 operatively connected to a machine vision system configured to track the locations of passive and/or non-vocalizing active users.

3. The system of claim 1 wherein the audio analysis processor is configured to receive inputs from multiple audio acquisition arrays, and parse the inputs to determine at least one of the presence, identity, and location of multiple users relative to the positions of the audio acquisition arrays.

4. The system of claim 1 wherein the database is interchangeably localized to specific languages.

5. The system of claim 1 wherein the database is interchangeably configured to contain commands relevant to a particular process.

6. The system of claim 1 wherein the acoustic array sensors are distributed throughout multiple distinct environments containing at least one of a multiplicity of individual users and machines.

7. The system of claim 1 wherein non-vocal audio signals are used to at least one of identify, localize, and command automated machines responsive to the command and control module.

8. The system of claim 1 comprising at least two audio sensors oriented to cover a separated designated primary area, further oriented to allow for localization of an audio source based on the difference in signal arrival time.

9. An audio command and control method comprising:

placing at least one acoustic array sensor in each of at least one distinct environment, said environment containing a multiplicity of users and automated equipment, said users engaged in a coordinated process;

parsing the signals from at least one acoustic array sensor to determine the identities and locations of users and equipment;

presenting to users a specialized command list tailored to location, user authority, and coordinated process timeline;

engaging and positioning automated equipment in response to user command;

limiting the engagement and position of automated equipment in response to a user, non-user, or equipment presence and location;

presenting to users a specialized command list tailored to location, user authority, and coordinated process timeline;

engaging and positioning automated equipment in response to user command; and

limiting the engagement and position of automated equipment in response to user presence and location.

10. A process comprising:

grouping audio sensors into an acoustical array sensor;

placing at least one acoustical array sensor in proximity to multiple users;

registering a first audio input from the at least one acoustical array sensor;

recognizing a first user based upon the first audio input;

localizing a first user based upon the first audio input into at least one acoustical array sensor;

assigning an authority level to the first authorized user based upon the first audio input;

presenting a list of commands available to the first authorized user based on the first audio input;

enacting a chosen command, from the list of commands, based upon a second audio input from the first user, wherein enacting the chosen command comprises moving instrumentation;

recognizing and localizing at least one additional user through vocal or non-vocal audio signal; and

limiting the movement of the instrumentation based upon the locations of either the first user or at least one additional user.

11. The process of claim 10 wherein the audio sensors utilize non-vocal audio emissions from equipment to automate and track movement.

12. The process of claim 10 wherein the multiple users are engaged in a coordinated process.

13. The process of claim 12 wherein commands available to an individual user are based on at least one of location, user function, process type, and overall progress of the coordinated process.

14. The process of claim 10 wherein the enactment of a chosen command comprises at least one of altering information displayed to a user, altering a process variable, dictating a report, and recording a notation.

15. The process of claim 14 wherein the execution of the command is altered by the presence of one or more additional users proximate to the issuer of the command.

16. The process of claim 10 wherein multiple users are distributed between at least first and second environments with limited visibility or accessibility between the environments.

17. The process of claim 16 wherein available commands are localized to a particular environment.

18. The process of claim 16 wherein commands issued by a system user in a first environment are executed in the at least one additional environment.

19. The process of claim 18 wherein execution of commands in at least one additional environment are limited by the presence and locations of at least one additional user in at least one additional environment.