Motion detection of audio sources to facilitate reproduction of spatial audio spaces

• US 10,225,680 B2
• Filed: 07/30/2013
• Issued: 03/05/2019
• Est. Priority Date: 07/30/2013
• Status: Active Grant

• Current Assignee: Jawbone Innovations, LLC
• Original Assignee: JAWB Acquisition LLC
• Inventors: Donaldson, Thomas Alan
• Primary Examiner(s): Kim, Paul S
• Assistant Examiner(s): Yu, Norman
• Application Number:
  US13/954,367
• Publication Number:
  US 20150036848A1
• Time in Patent Office: 2,044 Days
• Field of Search: 381 1, 381303, 381300, 381306, 381307, 381310, 381 17- 18, 381 77, 381120
• CPC Class Codes: H04M 3/568 audio processing specific to telephonic conferencing, e.g. spatial distribution, mixing of participants echo suppression in two-way loud-speaking telephone systems H04M9/02; sound field processing per se H04S7/30
• CPC Class Codes: H04N 7/15 Conference systems
• CPC Class Codes: H04S 2400/01 Multi-channel, i.e. more than two input channels, sound reproduction with two speakers wherein the multi-channel information is substantially preserved
• CPC Class Codes: H04S 2400/15 Aspects of sound capture and related signal processing for recording or reproduction
• CPC Class Codes: H04S 7/303 Tracking of listener position or orientation

Abstract

Embodiments of the invention relate generally to electrical and electronic hardware, computer software, wired and wireless network communications, and wearable/mobile computing devices configured to facilitate production and/or reproduction of spatial audio and/or one or more audio spaces. More specifically, disclosed are systems, components and methods to acoustically determine displacements of audios sources (or portions thereof), such as a subset of speaking users, for providing audio spaces and spatial sound field reproduction, for example, for a remote listener. In one embodiment, a media device includes transducers to emit audible acoustic signals into a region including one or more audio sources, acoustic probe transducers configured to emit ultrasonic signals and acoustic sensors configured to sense received ultrasonic signals reflected from an audio source. A controller can determine a displacement of the audio source. Examples of displacement include locomotion, gesture-related motion and orientation changes.

Claims

1. A method comprising:

generating unique ultrasonic signals, at least a unique ultrasonic signal being generated by emission from a corresponding acoustic probe transducer;

emitting the unique ultrasonic signal in a direction associated with the acoustic probe transducer;

sensing reflected ultrasonic signals from one or more surfaces, a subset of surfaces being associated with an object;

identifying a position of the object relative to a point in space as a function of characteristics of the reflected ultrasonic signals;

mapping an environment to detect relatively immobile surfaces within the environment in which the object is substantially positioned to modify a power level associated with the unique ultrasonic signal using a power adjuster in an audio device configured to emit the unique ultrasonic signal, the power adjuster being configured to modify the power level based on a temperature associated with ambient air in the environment using a temperature sensor in the audio device;

determining a displacement of at least a portion of the object;

identifying an action responsive to the displacement; and

performing the action to modify operation of the audio system.

2. The method of claim 1, wherein performing the action comprises:

changing directivity of sound beams configured to provide spatial audio to an audio space including a recipient of audio as the object.

3. The method of claim 1, wherein performing the action comprises:

detecting a gesture based on the displacement of the at least a portion of the object; and

modifying generation of spatial audio based on the gesture.

4. The method of claim 1, wherein determining the displacement comprises:

determining values representative of modified characteristics.

5. The method of claim 4, wherein determining the values representative of the modified characteristics comprises:

detecting a variation in a direction.

6. The method of claim 5, wherein detecting the variation in the direction comprises:

detecting a variation in an angle.

7. The method of claim 4, wherein determining the values representative of the modified characteristics comprises:

detecting a variation in a distance.

8. The method of claim 1, further comprising:

detecting locomotion of the object based on the displacement.

9. The method of claim 1, further comprising:

sensing reflected ultrasonic signals from the subset of surfaces associated with the object;

modifying, using the power adjuster, the power level of the unique ultrasonic signal based on an orientation of at least one of the plurality of acoustic probe transducers in the environment relative to the object using a motion sensor in the audio device;

calculating one or more variations of distances between the point in space and the subset of surface to form one or more calculated distance variations; and

identifying a next position of the portion of the object based on the one or more calculated distance variations that is either closer or farther relative to the point in space.

10. The method of claim 1, further comprising:

sensing a first subset of reflected ultrasonic signals from the subset of surfaces associated with the object at a first time point;

sensing a second subset of reflected ultrasonic signals from the subset of surfaces associated with the object at a second time point;

calculating one or more variations of directions between the subset of surfaces at the first time point and the subset of surfaces at the second time point to form one or more calculated direction variations; and

identifying a next position of the portion of the object based on the one or more calculated direction variations that is either in a first lateral direction or in a second lateral direction relative to the point in space.

11. The method of claim 10, wherein the one or more variations of directions comprise:

one or more angles, wherein the one or more calculated direction variations is either in

a negative angular direction or in a positive angular direction relative to the point in space.

12. The method of claim 1, further comprising:

detecting a gesture based on the displacement of the at least a portion of the object, detecting the gesture comprising;

determining a first subset of the unique ultrasonic signals associated with another portion of the object;

identifying the first subset of the unique ultrasonic signals indicate that the another portion of the object is non-transitory;

determining a second subset of the unique ultrasonic signals associated with the portion of the object; and

identifying the second subset of the unique ultrasonic signals indicate that the another portion of the object is transitory.

13. The method of claim 12, further comprising:

characterizing motion of the another portion of the object to form characterized motion;

comparing the characterized motion against data representing patterns of motion each of which is associated with a gesture; and

detecting a match of the characterized motion to a pattern of motion to determine an identified gesture.

14. The method of claim 13, further comprising:

translating the identified gesture to a control action; and

generating control action signal associated with the control action,

wherein the control action signal is configured to cause modified operation of the audio system.

15. The method of claim 12, further comprising:

characterizing motion of the another portion of the object to form characterized motion;

comparing the characterized motion against data representing patterns of motion each of which is associated with a gesture;

detecting no match of the characterized motion to the patterns of motion; and

identifying that the characterized motion is indicative of a change in orientation of the object.

16. An apparatus comprising:

a plurality of transducers configured to emit audible acoustic signals into a region including one or more audio sources;

a plurality of acoustic probe transducers configured to emit ultrasonic signals, at least a subset of the acoustic probe transducers each is configured to emit a unique ultrasonic signal;

a power adjuster configured to modify a power level of the unique ultrasonic signal based on a temperature associated with ambient air in an environment in which the one or more audio sources are substantially positioned;

a temperature sensor coupled with the power adjuster and configured to generate a signal indicative of the temperature;

a plurality of acoustic sensors configured to sense received ultrasonic signals reflected from the one or more audio sources;

a controller configured to determine a displacement of at least a portion of an audio source of the one or more audio sources; and

an environmental mapper configured to determine spatial characteristics of relatively immobile surfaces within the environment.

17. The apparatus of claim 16, wherein the controller comprises:

a locomotion detector configured to detect locomotion of the audio source based on the displacement; and

a gesture detector configured to detect a gesture based on the displacement of the portion of the audio source.

18. The apparatus of claim 17, further comprising:

a motion sensor coupled with the power adjuster, the motion sensor including accelerometers configured to generate signals indicative of an orientation of at least one of the plurality of acoustic probe transducers in the environment, and wherein the power adjuster being configured to modify the power level of the unique ultrasonic signal based on the orientation of the at least one of the plurality of acoustic probe transducers in the environment relative to the one or more audio sources;

a motion correlator configured to correlate the displacement with data representing a gesture;

a gesture identifier configured to identify the gesture;

a gesture translator configured to translate the gesture to a control action command; and

a control action data generator configured to generate the control action command to control operation of the apparatus.

19. The apparatus of claim 16, further comprising:

a phase-shift key signal modulator configured to generate the unique ultrasonic signal.