TILED GRAMMAR FOR PHRASE SPOTTING WITH A PERSISTENT COMPANION DEVICE

• US 20160171979A1
• Filed: 02/12/2016
• Published: 06/16/2016
• Est. Priority Date: 03/15/2013
• Status: Abandoned Application

• Current Assignee: Ntt Disruptions Co.
• Original Assignee: Jibo, Inc.
• Inventors: Pieraccini, Roberto, Makachev, Maxim, Breazeal, Cynthia
• Application Number:
  US15/043,027
• Publication Number:
  US 20160171979A1
• Time in Patent Office: Days
• US Class Current: 1/1
• CPC Class Codes: B25J 11/0005 Manipulators having means for high-level communication with users, e.g. speech generator, face recognition means
• CPC Class Codes: B25J 11/001 with emotions simulating means
• CPC Class Codes: B25J 11/0015 Face robots, animated artificial faces for imitating human expressions
• CPC Class Codes: B25J 9/0003 Home robots, i.e. small robots for domestic use
• CPC Class Codes: B25J 9/1694 characterised by use of sensors other than normal servo-feedback from position, speed or acceleration sensors, perception control, multi-sensor controlled systems, sensor fusion
• CPC Class Codes: G10L 15/197 Probabilistic grammars, e.g. word n-grams
• CPC Class Codes: G10L 15/22 Procedures used during a speech recognition process, e.g. man-machine dialogue
• CPC Class Codes: G10L 15/32 Multiple recognisers used in sequence or in parallel; Score combination systems therefor, e.g. voting systems
• CPC Class Codes: G10L 2015/223 Execution procedure of a spoken command
• CPC Class Codes: H04N 23/611 where the recognised objects include parts of the human body

Abstract

Tiled grammar-based phrase spotting includes orienting at least one segment of a multi-segment robot to facilitate capturing speech of a user arriving at the robot, and configuring a plurality of processing threads of a multi-threaded processing environment into distinct tiles, wherein at least a portion of the plurality of processing threads operate simultaneously on the captured speech to recognize a phrase type using a speech recognition grammar that is associated with the corresponding tile, wherein at least two of the tiles employ different speech recognition grammars to recognize different content in the captured speech.

Claims

1. A method comprising:

orienting at least one segment of a multi-segment robot to facilitate capturing speech of a user arriving at the robot; and

configuring a plurality of processing threads of a multi-threaded processing environment into distinct tiles, wherein at least a portion of the plurality of processing threads operate simultaneously on the captured speech to recognize a phrase type using a speech recognition grammar that is associated with the corresponding tile, wherein at least two of the tiles employ different speech recognition grammars to recognize different content in the captured speech.

2. The method of claim 1, wherein the audio is speech of a user in proximity to the robot.

3. The method of claim 1, further comprising triggering execution of a second phrase spotting recognizer based on recognition of a phrase by a first phrase spotting recognizer.

4. The method of claim 3, wherein the first phrase spotting recognizer comprises at least one of the processing threads.

5. The method of claim 3, wherein phrase spotting recognizers for different phrases are arranged according to a graph data structure that organizes the order of triggering of at least one different phrase spotting recognizer based on recognition of a particular phrase by at least one other phrase spotting recognizer.

6. The method of claim 1, further comprising causing an interaction action to be taken by the robot based on a determined phrase detected from the captured audio as a result of at least one iteration of operation of at least two recognizers.

7. The method of claim 1, wherein a final phrase is determined based on a plurality of candidate phrases determined by the plurality of recognizers.

8. The method of claim 7, wherein the final phrase is a compound phrase consisting of portions determined by each of a plurality of recognizers.

9. The method of claim 1, wherein determination of at least one portion of a phrase is determined by comparing speech recognition confidence levels of a plurality of recognizers.

10. The method of claim 1, wherein at least one phrase spotting recognizer employs natural language processing on the captured speech.

11. The method of claim 1, wherein the phase recognizing grammar is adapted to be statistically-based in response detecting a plurality of determined phrases.

12. A system comprising:

a multi-segment robot configured to orient at least one segment to facilitate capture of audio arriving at the robot;

a multi-threaded processing environment of a processor associated with the robot, the environment for facilitating simultaneous operation of threads in the environment; and

a plurality of processing threads deployed in the multi-threaded processing environment, the threads configured into distinct tiles, at least a portion of the plurality of processing threads operating simultaneously on the captured audio to recognize a phrase type using a speech recognition grammar that is associated with the corresponding tile, wherein at least two of the tiles employ different speech recognition grammars to recognize different content in the captured audio.

13. The system of claim 12, wherein the audio is speech of a user in proximity to the robot.

14. The system of claim 12, further comprising a second processing thread for a second tile that is triggered based on recognition of a phrase by a processing thread for a first tile.

15. The system of claim 14, wherein tile-specific processing threads for recognizing different phrases are arranged according to a graph data structure that organizes the order of triggering of at least one different tile-specific processing thread based on recognition of a particular phrase by at least one other tile-specific processing thread.

16. The system of claim 12, further comprising an interaction action that is taken by the robot based on a determined phrase detected from the captured audio as a result of at least one iteration of operation of at least two tile-specific processing threads.

17. The system of claim 12, wherein a final phrase is based on a plurality of candidate phrases determined by the plurality of processing threads.

18. The system of claim 17, wherein the final phrase is a compound phrase consisting of portions determined by each of a plurality of processing threads.

19. The system of claim 12, wherein recognition of at least one portion of a phrase type is determined by comparing speech recognition confidence levels of a plurality of processing threads.

20. The system of claim 12, wherein the speech recognizing grammar is adapted to be statistically-based in response to detecting a plurality of phrase types.