Techniques for providing non-verbal speech recognition in an immersive playtime environment

• US 9,855,497 B2
• Filed: 01/20/2015
• Issued: 01/02/2018
• Est. Priority Date: 01/20/2015
• Status: Active Grant

• Current Assignee: Disney Enterprises Incorporated (The Walt Disney Company)
• Original Assignee: Disney Enterprises Incorporated (The Walt Disney Company)
• Inventors: Goslin, Michael P., Haseltine, Eric C., Olson, Joseph L.
• Primary Examiner(s): Laneau, Ronald
• Application Number:
  US14/601,083
• Publication Number:
  US 20160206955A1
• Time in Patent Office: 1,078 Days
• Field of Search: 463 30- 36, 463 42
• CPC Class Codes: A63F 13/215 comprising means for detecting acoustic signals, e.g. using a microphone
• CPC Class Codes: A63F 13/424 involving acoustic input signals, e.g. by using the results of pitch or rhythm extraction or voice recognition
• CPC Class Codes: G06F 2203/0381 Multimodal input, i.e. interface arrangements enabling the user to issue commands by simultaneous use of input devices of different nature, e.g. voice plus gesture on digitizer
• CPC Class Codes: G06F 3/017 Gesture based interaction, e.g. based on a set of recognized hand gestures interaction based on gestures traced on a digitiser G06F3/04883
• CPC Class Codes: G06F 3/0346 with detection of the device orientation or free movement in a 3D space, e.g. 3D mice, 6-DOF [six degrees of freedom] pointers using gyroscopes, accelerometers or tilt-sensors
• CPC Class Codes: G06F 3/167 Audio in a user interface, e.g. using voice commands for navigating, audio feedback
• CPC Class Codes: G10L 17/26 Recognition of special voice characteristics, e.g. for use in lie detectors; Recognition of animal voices
• CPC Class Codes: G10L 25/18 the extracted parameters being spectral information of each sub-band

Abstract

An immersive play environment platform including techniques describing recognizing non-verbal vocalization gestures from a user is disclosed. A headset device receives audio input from a user. The headset device transmits the audio input to a controller device. The controller device evaluates characteristics of the audio input (e.g., spectral features over a period of time) to determine whether the audio input corresponds to a predefined non-verbal vocalization, such as a humming noise, shouting noise, etc. The controller device may perform an action in response to detecting such non-verbal vocalizations, such as engaging a play object (e.g., an action figure, an action disc) in the play environment.

Claims

1. An immersive play experience platform providing an interactive environment, comprising:

an audio device configured to receive audio input; and

a controller device configured to perform an operation for recognizing non-verbal vocalizations, the operation comprising;

receiving audio input transmitted by the audio device,

determining that the audio input matches spectral features of one or more predefined non-verbal vocalizations characterizing gameplay within the interactive environment, and

causing an action corresponding to the one or more predefined non-verbal vocalizations to be performed on an interactive gameplay object within the interactive environment.

2. The platform of claim 1, further comprising:

a physical device configured to receive instructions from the controller device to perform the corresponding action; and

wherein the interactive gameplay object is connected with the physical device.

3. The platform of claim 2, wherein performing the corresponding action comprises:

causing the interactive gameplay object to disconnect from the physical device.

4. The platform of claim 1, wherein the predefined non-verbal vocalizations include at least one of a humming gesture or a shouting gesture.

5. The platform of claim 1, further comprising:

a sensor device having an accelerometer and a gyroscope, the sensor device configured to receive position and orientation input and to transmit the position and orientation input to the controller device.

6. The platform of claim 5, wherein the one or more predefined non-verbal vocalizations are associated with a choreographed sequence having one or more gestures corresponding to specified position and orientation input transmitted from the sensor device.

7. The platform of claim 1, wherein the predefined non-verbal vocalizations are further determined based on amplitude and timing thresholds measured in the audio input.

8. A method for recognizing non-verbal vocalizations, comprising:

receiving, by a controller device executing on an immersive play experience platform providing an interactive environment, audio input;

determining, by the controller device, that the audio input matches spectral features of one or more predefined non-verbal vocalizations characterizing gameplay within the interactive environment; and

causing an action corresponding to the one or more predefined non-verbal vocalizations to be performed on an interactive gameplay object within the interactive environment.

9. The method of claim 8, further comprising, transmitting instructions to a physical device for performing the corresponding action.

10. The method of claim 9, wherein performing the corresponding action comprises:

causing the interactive gameplay object to disconnect from the physical device.

11. The method of claim 8, wherein the predefined non-verbal vocalizations include at least one of a humming gesture or a shouting gesture.

12. The method of claim 8, further comprising, receiving position and orientation input.

13. The method of claim 12, wherein the one or more predefined non-verbal vocalizations are associated with a choreographed sequence having one or more gestures corresponding to specified position and orientation input.

14. The method of claim 8, wherein the predefined non-verbal vocalizations are further determined based on amplitude and timing thresholds measured in the audio input.

15. A non-transitory computer-readable storage medium storing instructions, which, when executed on a processor, perform an operation for recognizing non-verbal vocalizations, the operation comprising:

receiving, by a controller device executing on an immersive play experience platform providing an interactive environment, audio input;

determining, by the controller device, that the audio input matches spectral features of at least a first predefined non-verbal vocalization characterizing gameplay within the interactive environment;

determining that the matched audio input is received as a part of a predefined sequence of gestures including at least a first physical gesture and the first predefined non-verbal vocalization, and

causing an action corresponding to the predefined sequence of gestures to be performed on an interactive gameplay object.

16. The computer-readable storage medium of claim 15, wherein the operation further comprises, transmitting instructions to a physical device for performing the corresponding action.

17. The computer-readable storage medium of claim 16, wherein performing the corresponding action comprises:

causing, via the physical device, the interactive gameplay object to disconnect from the physical device.

18. The computer-readable storage medium of claim 15, wherein the operation further comprises, receiving position and orientation input.

19. The computer-readable storage medium of claim 18, wherein the at least the first physical gesture matches specified position and orientation input.

20. The computer-readable storage medium of claim 15, wherein the at least the first predefined non-verbal vocalization is further determined based on amplitude and timing thresholds measured in the audio input.