Enabling voice control of voice-controlled apparatus

• US 20020105575A1
• Filed: 12/04/2001
• Published: 08/08/2002
• Est. Priority Date: 12/05/2000
• Status: Active Grant

• Current Assignee: Hewlett-Packard Development Company, L.P. (HP Inc.)
• Original Assignee: Hewlett-Packard Development Company, L.P. (HP Inc.)
• Inventors: Pollard, Stephen B., Hunter, Andrew Arthur, Hinde, Stephen John, Wilkinson, Timothy Alan Heath
• Granted Patent: US 6,970,824 B2
• Time in Patent Office: Days
• US Class Current: 348/14.01
• CPC Class Codes: G06F 3/167 Audio in a user interface, e.g. using voice commands for navigating, audio feedback
• CPC Class Codes: G10L 15/24 Speech recognition using non-acoustical features
• CPC Class Codes: G10L 2025/783 based on threshold decision

Abstract

Voice-controlled apparatus is provided which minimises the risk of activating more than one such apparatus at a time where multiple voice-controlled apparatus exist in close proximity. To start voice control of the apparatus, a user needs to be looking at the apparatus when speaking. Preferably, after the user stops looking at the apparatus, continuing voice control can only be effected whilst the user continues speaking without breaks longer than a predetermined duration. Detection of whether the user is looking at the apparatus can be effected in a number of ways including by the use of camera systems, by a head-mounted directional transmitter, and by detecting the location and direction of facing of the user.

Claims

1. A method of enabling voice control of voice-controlled apparatus, involving at least:

(a) detecting when the user is looking towards the apparatus; and

(b) initially enabling the apparatus for voice control only when the user is detected in (a) as looking towards the apparatus.

2. A method according to claim 1, wherein the apparatus only remains enabled for voice control whilst the user continues to be detected in (a) as looking towards the apparatus.

3. A method according to claim 1, further involving:

detecting when the user is speaking, and

where the user is detected as speaking whilst the apparatus is initially enabled for voice control, continuing enablement of the apparatus for voice control following the user ceasing to look towards the apparatus but only whilst the user continues speaking and for a timeout period thereafter, recommencement of speaking by the user during this timeout period continuing enablement of voice control with timing of the timeout period being reset.

4. A method according to claim 1, further involving:

detecting when the user is speaking and determining characteristics of the user'"'"'s voice;

where the user is detected as speaking whilst the apparatus is initially enabled for voice control, continuing enablement of the apparatus for voice control following the user ceasing to look towards the apparatus but only in respect of a voice having the same characteristics as that of the voice detected whilst the apparatus was initially enabled, and only whilst that voice continues speaking and for a timeout period thereafter, recommencement of speaking by the same voice during this timeout period continuing enablement of voice control with timing of the timeout period being reset.

5. A method according to claim 1, wherein (a) is effected using a camera system mounted on the apparatus, images produced by the camera system being processed to determine if the user is looking towards the apparatus.

6. A method according to claim 1, wherein (a) is effected using a camera system comprising one or more cameras mounted off the apparatus in fixed positions, images produced by the camera system being processed to determine if the user is looking towards the apparatus.

7. A method according to claim 1, wherein (a) is effected using a camera system mounted on a user'"'"'s head and arranged to point in the direction the user is facing or looking, images produced by the camera system being processed by an image processing subsystem to determine if the user is looking towards the apparatus.

8. A method according to claim 7, wherein the apparatus carries an identifying mark that is used to identify the apparatus to the image processing subsystem.

9. A method according to claim 8, wherein the identifying mark takes the form of a perspective invariant bar code.

10. A method according to claim 8, wherein the identifying mark takes the form of an encoded optical or infrared signal.

11. A method according to claim 8, wherein the identifying mark encodes a communications address at which the apparatus can be contacted.

12. A method according to claim 1, wherein (a) is effected using a directional transmitter mounted on a user'"'"'s head and arranged to point in the direction the user is facing, the apparatus having a receiver for detecting emissions from the directional transmitter.

13. A method according to claim 1, wherein (a) is effected by detecting the position of the user and using a direction sensor mounted on a user'"'"'s head for sensing the direction of facing of the user, the output of this sensor and the position of the user being used to determine whether the user is facing towards a known position of the apparatus.

14. A method according to claim 1, wherein speech recognition means of the apparatus ignores voice input from the user unless whilst the user is looking towards the apparatus, the user speaks a predetermined key word.

15. A method according to claim 1, wherein the initial enabling of the apparatus for voice control is only effected for specific persons as identified by the sending of an identifying code by a transmitting device associated with that person.

16. A method according to claim 1, wherein the initial enabling of the apparatus for voice control is only effected for specific persons as identified by the recognition of the face of the person by an imaging system

17. A method according to claim 16, wherein amongst multiple persons facing the apparatus, a person speaking to the apparatus is discerned by the detection of lip movement by an image processing subsystem, voice control of the apparatus only being enabled if the person speaking is a said specific person.

18. A method according to claim 1, wherein the enabling of the apparatus for voice control is only effected for specific persons as identified by the recognition of the voice of the person by a voice characteristics recognizer.

19. A method according to claim 18, wherein amongst multiple persons facing the apparatus, a person speaking to the apparatus is discerned by the detection of lip movement by an image processing subsystem, voice control of the apparatus only being enabled if the person speaking is a said specific person.

20. A method according to claim 1, further involving:

detecting lip movement by a user detected as facing the apparatus, and

only effecting initial enablement of the apparatus for voice control where the user detected as facing the apparatus is also moving their lips, indicating that the user is speaking.

21. A method according to claim 1, further involving detecting when a user is speaking, initial enablement of the apparatus for voice control requiring that a user is detected as speaking simultaneously with a user being detected as looking at the apparatus.

22. An arrangement for enabling voice control of voice-controlled apparatus, comprising:

detection means for detecting when the user is looking towards the apparatus; and

enablement control means for initially enabling the apparatus for voice control only if the detection means indicate that the user is looking towards the apparatus.

23. An arrangement according to claim 22, wherein the control means is operative to keep the apparatus enabled for voice control only whilst the detection means continues to detect the user looking towards the apparatus.

24. An arrangement according to claim 22, further comprising a speaking detector for detecting when a user is speaking, the control means comprising:

initial-enablement means for effecting the said initial enabling of the apparatus for voice control;

delayed-disablement means including timing means for timing a timeout period; and

means for activating the delayed-disablement means upon the speaking detector detecting a user speaking whilst the apparatus is initially enabled by the initial enablement means;

the delayed-disablement means, when activated, being operative to keep the apparatus enabled for voice control following the detection means ceasing to detect that the user is looking towards the apparatus but only whilst the speaking detector continues to detect that the user is speaking and for the duration thereafter of the said timeout period as timed by the timing means, the delayed-disablement means being responsive to the speaking detector detecting recommencement of speaking by the user during this timeout period to reset timing of the timeout period.

25. An arrangement according to claim 22, further comprising a speaking detector for detecting when a user is speaking, and a voice analyzer for determining characteristics of the user'"'"'s voice, the control means comprising:

initial-enablement means for effecting the said initial enabling of the apparatus for voice control;

delayed-disablement means including timing means for timing a timeout period; and

means for activating the delayed-disablement means upon the speaking detector detecting a user speaking whilst the apparatus is initially enabled by the initial enablement means;

the delayed-disablement means, when activated, being operative to keep the apparatus enabled for voice control following the detection means ceasing to detect that the user is looking towards the apparatus but only;

in respect of a voice having the same characteristics, as determined by the voice analyser, as that of the detected voice giving rise to activation of the delayed disablement means; and

whilst the voice continues without a break greater than said timeout period as timed by the timing means.

26. An arrangement according to claim 22, wherein the detection means comprises:

a camera system mounted on the apparatus, and

image processing means for processing images produced by the camera system to determine if the user is looking towards the apparatus.

27. An arrangement according to claim 22, wherein the detection means comprises:

a camera system comprising one or more cameras mounted off the apparatus in fixed positions, and

image processing means for processing images produced by the camera system to determine if the user is looking towards the apparatus.

28. An arrangement according to claim 22, wherein the detection means comprises:

a camera system mounted on a user'"'"'s head and arranged to point in the direction the user is facing or looking, and

image processing means for processing images produced by the camera system to determine if the user is looking towards the apparatus.

29. An arrangement according to claim 28, wherein the image processing means is operative to recognise identifying marks of a predetermined type carried by the apparatus.

30. An arrangement according to claim 29, wherein the said predetermined type of identifying mark is an invariant bar code.

31. An arrangement according to claim 29, wherein the said predetermined type of identifying mark is an encoded optical or infrared signal.

32. An arrangement according to claim 28, wherein the said predetermined type of identifying mark is an encoded communications address at which the apparatus can be contacted.

33. An arrangement according to claim 22, wherein the detection means comprises:

a directional transmitter for mounting on a user'"'"'s head such as to point in the direction the user is facing, and

a receiver, at the apparatus, for detecting emissions from the directional transmitter.

34. An arrangement according to claim 22, wherein the detection means comprises:

user-position detection system for detecting the position of the user,

a direction sensor for mounting on a user'"'"'s head to sense the direction of facing of the user,

processing means, responsive to the user position detected by the user-position detection system and the user direction of facing sensed by the direction sensor, to determine whether the user is facing towards a known position of the apparatus.

35. An arrangement according to claim 22, further comprising speech recognition means for recognizing the speaking of a predetermined key word by a user, the control means being operative to inhibit voice control of the apparatus unless whilst the user is looking towards the apparatus as detected by the detection means, the user speaks said predetermined key word.

36. An arrangement according to claim 22, further comprising a receiver for receiving a code sent by a user-carried transmitting device, the control means being arranged to initially enable the apparatus for voice control only if said code is received by the receiver substantially contemporaneously with the detection means detecting a user looking towards the apparatus.

37. An arrangement according to claim 22, wherein the detection means comprises an imaging system comprising:

a camera subsystem, and

first image processing means for determining when a person is facing the apparatus;

the imaging system further comprising second image processing means for recognizing specific persons from their image as produced by the camera subsystem; and

the control means being operative to initially enable the apparatus for voice control only where the imaging system detects a said specific person facing the apparatus.

38. An arrangement according to claim 37, wherein the imaging system further comprises third image processing means for determining which person, amongst multiple persons facing the apparatus, is speaking to the apparatus by detecting lip movement, the control means being operative to initially enable voice control of the apparatus only if the person determined as speaking to the apparatus is a said specific person.

39. An arrangement according to claim 22, further comprising a voice analyzer for recognizing the voice of specific persons, the control means being operative to initially enable the apparatus for voice control only where the voice analyzer recognizes a said specific person substantially contemporaneously with the detection means detecting a user looking towards the apparatus.

40. An arrangement according to claim 39, wherein the detection means comprises an imaging system comprising:

a camera subsystem, and

first image processing means for determining when a person is facing the apparatus;

the image system further comprising second image processing means for determining which person, amongst multiple persons facing the apparatus, is speaking to the apparatus by detecting lip movement; and

the control means being operative to initially enable voice control of the apparatus only if the person determined as speaking to the apparatus by an image processing means is a said specific person.

41. An arrangement according to claim 22, wherein the detection means comprises an imaging system comprising:

a camera subsystem, and

first image processing means for determining when a person is facing the apparatus;

the imaging system further comprising second image processing means for detecting lip movement by the person facing the apparatus, and the control means being operative to initially enable the apparatus for voice control only where the image processing means detects lip movement by a person facing the apparatus.

42. An arrangement according to claim 22, further comprising means for detecting when a user is speaking, the control means being operative to initially enable the apparatus for voice control only upon a user being detected as speaking simultaneously with a user being detected as looking at the apparatus.

43. An installation for accommodating at least one voice-controlled apparatus, the installation including an arrangement according to claim 22 wherein the control means includes a communication subsystem for transmitting enablement signals to said apparatus.

44. An installation for accommodating at least one voice-controlled apparatus, the installation including an arrangement according to claim 27 wherein the control means includes a communication subsystem for transmitting enablement signals to said apparatus.

45. An installation for accommodating at least one voice-controlled apparatus, the installation including an arrangement according to claim 28 wherein the control means includes a communication subsystem for transmitting enablement signals to said apparatus.

46. An installation for accommodating at least one voice-controlled apparatus, the installation including an arrangement according to claim 34 wherein the control means includes a communication subsystem for transmitting enablement signals to said apparatus.

47. Voice-controlled apparatus including an arrangement according to claim 22.

48. Voice-controlled apparatus including an arrangement according to claim 26.

49. Voice-controlled apparatus including an arrangement according to claim 33.