LOCALIZATION SYSTEM, ROBOT, LOCALIZATION METHOD, AND SOUND SOURCE LOCALIZATION PROGRAM

• US 20090262604A1
• Filed: 08/20/2007
• Published: 10/22/2009
• Est. Priority Date: 08/30/2006
• Status: Abandoned Application

• Current Assignee: NEC Corporation
• Original Assignee: NEC Corporation
• Inventors: Funada, Junichi
• Application Number:
  US12/438,781
• Publication Number:
  US 20090262604A1
• Time in Patent Office: Days
• US Class Current: 367/127
• CPC Class Codes: G01S 11/16 using difference in transit time between electrical and acoustic signals
• CPC Class Codes: G01S 17/08 for measuring distance only indirect measurement G01S17/46; active triangulation systems G01S17/48
• CPC Class Codes: G01S 5/30 Determining absolute distances from a plurality of spaced points of known location

Abstract

To measure an accurate positional relationship between an ultrasonic tag and a microphone and identify a sound source position, even if an object is present between the ultrasonic tag and the microphone. When a radio transmission unit transmits a radio wave, an ultrasonic wave transmission unit of an ultrasonic tag receives it and transmits an ultrasonic wave. Then, a plurality of microphones in an ultrasonic wave reception array unit receive the ultrasonic wave. A propagation time calculation unit calculates a time from when the radio wave is transmitted by the radio transmission unit till when an ultrasonic wave reaches each of the microphones in the ultrasonic wave reception array unit. A position estimation unit calculates the position (sound source) of the ultrasonic tag according to the arrival time at each of the microphones and the result of object detection while considering reflection of the ultrasonic wave.

Claims

1. A localization system for identifying a position of a sound source using propagation times of a sound wave propagating from the sound source to a plurality of microphones, comprising:

an object detection unit which detects a position, a shape, and the like of a surrounding object present around the plurality of microphones; and

a sound source position estimation unit which estimates a position of the sound source based on the propagation times, wherein

the sound source position estimation unit has a reflection wave path estimation function for estimating a reflection wave path and its distance from the surrounding object identified by information from the object detection unit, and based on the reflection wave path and the distance, the sound source position estimation unit estimates the position of the sound source.

2. The localization system, according to claim 1, further comprising:

a radio wave transmission unit which transmits a radio wave including an operational command to transmit an ultrasonic wave;

an ultrasonic wave reception array unit including a plurality of microphones which receive an ultrasonic wave from the sound source, the ultrasonic wave being transmitted by being urged by the command to transmit an ultrasonic wave;

a propagation time calculation unit which calculates a time period required from a time that the radio wave is transmitted from the radio wave transmission unit until a time that an ultrasonic wave reaches each of the microphones of the ultrasonic wave reception array unit; and

an object detection unit which detects a surrounding object present around the ultrasonic wave reception array unit.

3. The localization system, according to claim 2, wherein the object detection unit has a relative position detecting function for detecting a shape and a position of an object reflecting an ultrasonic wave and a relative position of the object with respect to a microphone array, based on the microphone array included in the ultrasonic wave reception array unit and a surrounding environment where the sound source is placed.

4. The localization system, according to claim 2 or 3, wherein the sound source position estimation unit includes:

a shortest reflection path calculation unit which estimates a reflection path using information of the object for each of the microphones configuring a microphone array included in the ultrasonic wave reception array unit and obtains an area where a shortest path length of the ultrasonic wave corresponds to the time calculated by the propagation time calculation unit as a candidate area for an ultrasonic wave transmission unit; and

a sound source position calculation unit which calculates the position of the sound source from a relationship between candidate areas obtained for respective microphones.

5. The localization system, according to claim 1, wherein

the object detection unit is configured as to measure and detect the position and the shape of the object using at least one of;

a method of performing shape measurement with a range sensor using laser light;

a method of estimating a three-dimensional shape from the position of the object and an observation result by causing a range sensor capable of measuring a distance on a two-dimensional plane to function;

a method of performing shape restoration by stereo view using a plurality of cameras;

a method of performing shape restoration by a factorization method using movement of a camera; and

a method of performing shape restoration from gradation on a surface of an object using an image captured by a camera.

6. The localization system, according to claim 1, wherein

the object detection unit has a sensor for detecting a surrounding object and a sensor moving function for moving the sensor, a surrounding map creating function for creating a surrounding object map by detecting a surrounding object at a plurality of locations based on movement of the sensor moving mechanism, and an object position identifying function for identifying the position of the object using the surrounding object map created.

7. The localization system according to claim 1, wherein

the object detection unit has an object matching unit which detects in advance what the surrounding object is, and outputs shape information regarding a stored surrounding object upon request, and

the object detection unit has an off-range map creating function for creating an object map regarding the surrounding object in an area outside of a measurement range of a sensor provided to the object detection unit, using the shape information from the object matching unit.

8. The localization system, according to claim 7, wherein the object matching unit has a function of detecting a tag such as an RFID tag, an ultrasonic tag, or an image marker attached to an object, and acquiring information for identifying a surrounding object to which the tag is attached based on detected tag information and transmitting the information to the object detection unit.

9. The localization system, according to claim 7, wherein the object detection unit has a function of identifying a surrounding object by performing image matching, and transmitting image information regarding the surrounding object to the object detection unit.

10. The localization system, according to claim 7, wherein the object matching unit has a function of detecting a tag such as an RFID tag, an ultrasonic tag, or an image marker, identifying a position and orientation of the surrounding object based on the information obtained from the tag, and transmitting information regarding the identified surrounding object to the object detection unit.

11. The localization system, according to claim 7, wherein in a case where at least one of an RFID tag, an ultrasonic tag and an image marker is set as the tag and multiple pieces of the tags are attached to the surrounding object, the object matching unit has a function of identifying a position and orientation of the surrounding object by detecting positions of the attached tags, and transmitting information regarding the identified surrounding object to the object detection unit.

12. The localization system according to claim 7, wherein the object matching unit detects a position and orientation of an object by performing matching with a shape of an object observed by the object detection unit.

13. An object searching robot having the localization system according to claim 1.

14. A localization method for measuring propagation times of an ultrasonic wave propagating from a sound source to a plurality of microphones configuring a microphone array and identifying a position of the sound source based on the propagation times, comprising

a radio wave transmission step for transmitting a radio wave including an ultrasonic wave transmission command to an ultrasonic tag provided to the sound source;

an ultrasonic wave reception step for receiving, by the plurality of microphones, an ultrasonic wave transmitted from the ultrasonic tag in response to the ultrasonic wave transmission command;

a propagation time calculation step for calculating propagation times required from a time that the radio wave is transmitted until times that the ultrasonic wave reaches the plurality of microphones; and

a sound source position estimation step for estimating a reflective propagation path of the ultrasonic wave based on positional information and the like of the surrounding object which has been detected and identified regarding the surrounding object present around the microphone array and the propagation time for each of the microphones calculated in the propagation time calculation step, calculating a position of the ultrasonic tag, and estimating the position of the sound source.

15. The localization method, according to claim 14, comprising, before the radio wave transmission step, a surrounding object detection step for detecting a surrounding object present around the microphone array and generating an object map.

16. The localization method, according to claim 15, wherein the surrounding object detection step includes an object information storing step for detecting in advance a position, a shape and a size of the surrounding object present around the microphone array and storing as surrounding map information, and an object map generating step for generating an object map for identifying a relative position with the surrounding object viewed from the respective microphones configuring the microphone array based on the surrounding map information stored.

17. The localization method according to claim 16, wherein in the object information storing step, a movable sensor provided to the object detection unit, having been disposed separately, detects positional information and shape information of a surrounding object present in a wide range around the microphone array, and the positional information an the shape information are stored as surrounding map information.

18. The localization method according to claim 16, wherein in the object map generation step in the surrounding object detection step, shape information of an object corresponding to the surrounding object is extracted from an object matching unit which has detected and stored the position, the shape, and the size of the surrounding object present around the microphone array, and arranged on the object map identified.

19. A computer readable recording medium storing a sound source localization program for calculating propagation times of an ultrasonic wave propagating from a sound source to a plurality of microphones, and calculating a position of the sound source based on respective propagation times of a plurality of different ultrasonic waves detected by the plurality of microphones, the program causing a computer to perform:

a transmitting operation control function for controlling a radio wave transmitting operation of a radio wave transmission unit which transmits a radio wave including an ultrasonic wave transmission command to an ultrasonic tag;

a propagation time calculation function for calculating propagation times of an ultrasonic wave transmitted from the ultrasonic tag and received by the plurality of microphones, from a time that the radio wave is transmitted until a time that the ultrasonic wave reaches the respective microphones; and

a position estimation computing function for, if a position of a reflective object present around a microphone array has been detected in advance, estimating a reflective propagation path of the ultrasonic wave from a detection result of the reflective object, and calculating a position of the ultrasonic tag based on the estimated reflective propagation path of the ultrasonic wave and the propagation time for each of the microphones calculated by the propagation time calculation function.

20. The computer readable recording medium storing the sound source localization program, according to claim 19, further causing the computer to perform, when performing the position estimation computing function, a surrounding object identifying function for storing in advance positional information and shape information of a surrounding object present around the microphone array detected by an object detection unit provided separately, and generating an object map based on a detection result.

21. The computer readable recording medium storing the sound source localization program, according to claim 20, further causing the computer to perform, when performing the surrounding object identifying function, an object information storing function for, if information regarding a position, a shape, and a size of the surrounding object present around the microphone array has been detected by the object detection unit provided separately, storing the information in a form of surrounding map information, and an object map generation function for generating an object map for identifying a relative position with the surrounding object viewed from the plurality of microphones configuring the microphone array, based on the surrounding map information stored.

22. The computer readable recording medium storing the sound source localization program, according to claim 20, causing the computer to perform the object map generation function by extracting shape information of an object corresponding to the surrounding object from an object matching unit which has detected and stored a position, a shape, and a size of the surrounding object present around the microphone array, and arranging the shape information on the object map.

23. A localization system for identifying a position of a sound source using propagation times of a sound wave propagating from the sound source to a plurality of microphones, comprising:

object detection means for detecting a position, a shape, and the like of a surrounding object present around the plurality of microphones; and

sound source position estimation means for estimating a position of the sound source based on the propagation times, wherein

the sound source position estimation means has a reflection wave path estimation function for estimating a reflection wave path and its distance from the surrounding object identified by information from the object detection means, and based on the reflection wave path and the distance, the sound source position estimation means estimates the position of the sound source.