Voice communication concerning a local entity

• US 7,113,911 B2
• Filed: 11/21/2001
• Issued: 09/26/2006
• Est. Priority Date: 11/25/2000
• Status: Active Grant

• Current Assignee: Valtrus Innovations Limited (f/k/a Dolya Holdco 9 Limited) (Key Patent Innovations Limited)
• Original Assignee: Hewlett-Packard Development Company, L.P. (HP Inc.)
• Inventors: Brittan, Paul St John, Hickey, Marianne, Wilcock, Lawrence, Belrose, Guillaume, Thomas, Andrew, Hinde, Stephen John
• Primary Examiner(s): McFadden, Susan
• Assistant Examiner(s): Vo, Huyen X.
• Application Number:
  US09/990,768
• Publication Number:
  US 20020082839A1
• Time in Patent Office: 1,770 Days
• Field of Search: 715/700, 340/825.29, 340/825.69, 704/275, 704/231, 704/258, 704/270.1, 345/700, 345/473, 701/211, 709/205, 348/14.12
• US Class Current: 704/270.1
• CPC Class Codes: G10L 15/30 Distributed recognition, e.g. in client-server systems, for mobile phones or network applications
• CPC Class Codes: H04M 2201/40 using speech recognition speech recognition per se G10L15/00
• CPC Class Codes: H04M 2201/60 Medium conversion
• CPC Class Codes: H04M 3/4938 comprising a voice browser which renders and interprets, e.g. VoiceXML

Abstract

A local entity without its own means of voice communication is provided with the semblance of having a voice interaction capability. This is done by providing a beacon device at or near the entity, the beacon device transmitting, over a short-range communication link, contact data identifying a voice service associated with, but hosted separately from, the entity. The transmitted contact data is picked up by equipment carried by a nearby person and used to contact the voice service over a wireless network. The person then interacts with the voice service, the latter acting as a voice proxy for the local entity. The contact data can be presented to the user in other ways, for example, by being inscribed on the local entity for scanning or user input into the equipment.

Claims

1. A system for enabling verbal communication on behalf of a local entity with a nearby user, the system comprising:

user equipment, intended to be carried by a user, comprising a wireless communication subsystem, and a contact-data input subsystem for receiving contact data;

a contact-data providing arrangement located at the local entity for making available to a user near the local entity or to the user equipment carried by that user, contact data identifying a voice service associated with the entity but separately hosted;

a communications infrastructure comprising at least a wireless network for communicating with the wireless communication subsystem of the user equipment;

an audio output arrangement forming part of the user equipment, or located in the locality of the local entity and connected to the communication infrastructure;

an audio input arrangement forming part of the user'"'"'s equipment, or located in the locality of said entity and connected to said communications infrastructure;

a voice service arrangement for providing said voice service, the voice service arrangement being connected to said communications infrastructure such as to enable the user'"'"'s equipment to contact it over the wireless network using said contact data, the voice service arrangement being operative, in response to being contacted by the user equipment, to act as voice proxy for the local entity by signals exchanged over the communications infrastructure with the audio input and output arrangements thereby enabling a user to interact with the voice service through spoken dialog with voice input by the user through the audio input arrangement and voice output to the user through the audio output arrangement;

controllable functionality associated with the local entity; and

a short-range-communications arrangement comprising complimentary elements at the local entity and in the user equipment for establishing a short range wireless link between the user equipment and said controllable functionality;

the user equipment being arranged to receive control data from the voice service arrangement in the course of the latter acting as a voice proxy for the local entity, and to pass on the control data via said short-range wireless link to said controllable functionality whereby to enable this functionality to be controlled by the voice service arrangement in coordination with said voice output.

2. A system according to claim 1, wherein the contact-data providing arrangement is a beacon device located at or near the local entity and operative to communicate with the contact-data input subsystem of the user'"'"'s equipment over said short-range wireless link.

3. A system according to claim 2, wherein the beacon device is operative to include parameter values relating to the state of said local entity in said contact data, the user equipment being operative to pass these parameter values via the wireless network to the voice service arrangement for use in conditioning the output of the voice service.

4. A system according to claim 2, wherein multiple beacon devices are associated with the entity with each beacon device being arranged to communicate different contact data, the voice service arrangement being arranged to adapt the voice output of the voice service delivered in respect of said local entity in dependence on the contact data of the beacon device first or most recently picked up by the user equipment.

5. A system according to claim 1, wherein both the audio input and output arrangement form part of the user equipment, the user equipment being operative to exchange said voice input and voice output with the voice service as voice signals passed across the wireless network.

6. A system according to claim 5, wherein the user equipment includes a mobile phone providing the said wireless communication subsystem and said audio input and output arrangements.

7. A system according to claim 5, wherein the voice service arrangement comprises:

a voice page server for serving voice pages in the form of text with embedded voice markup tags; and

a voice browser comprising;

a speech recognizer for carrying out speech recognition of user voice input received as voice signals;

a dialog manager for effecting dialog control on the basis of output from the speech recognizer and pages served by the voice page server; and

a text-to-speech converter operative to convert voice pages into voice output signals under the control of the dialog manager;

the user equipment including a mobile phone providing the said wireless communication subsystem and said audio input and output arrangements, the wireless network being a mobile phone wireless network.

8. A system according to claim 7, wherein the voice browser is not part of the user'"'"'s equipment and the contact data comprises a telephone number usable by the mobile phone to connect over a voice circuit of the wireless network to the voice browser, the voice browser being responsive to being connected to by the mobile phone to access the voice page server and to thereafter use said voice circuit for the exchange of voice input and/or output between the user and voice browser.

9. A system according to claim 7, wherein the voice browser is not part of the user'"'"'s equipment and the contact data is in the form of a URL, the mobile phone being operable to pass this URL, via a data-capable bearer service of the wireless network, to the voice service arrangement;

the voice service arrangement being operative to thereupon use its voice browser to call back the user on the mobile phone using a voice circuit of the wireless network that is then used for voice input and/or output between the user and voice browser.

10. A system according to claim 7, wherein the voice browser is not part of the user'"'"'s equipment and the contact data is in the form of a URL, the mobile phone being operable to pass this URL, via a data-capable bearer service of the wireless network, to the voice service arrangement;

the voice service arrangement being thereafter operative to use the data-capable bearer service for voice input and/or output between the user and voice browser using a packetized voice protocol.

11. A system according to claim 7, wherein the voice browser is part of the user'"'"'s equipment and the contact data is in the form of a URL, the voice browser being operative to use this URL to access, via a data-capable bearer service of the mobile-phone wireless network, the voice page server;

the voice service arrangement being thereafter operative to use the data-capable bearer service for passing text based input and/or output between the voice browser and voice page server.

12. A system according to claim 1, wherein said audio output arrangement comprises multiple sound output devices spaced from said local entity, and a controller for controlling excitation of these devices such as to produce a sound output that it appears to the user to emanate from the location of said local entity independently of the user'"'"'s position and head orientation relative to the entity.

13. A system according to claim 12, wherein said multiple sound output devices are headphones worn by the user, the controller being arranged to control excitation of the headphones in dependence on the relative positions of the user and entity and rotations of the user'"'"'s head.

14. A system according to claim 12, wherein said multiple sound output devices are loudspeakers associated with the locality of the entity rather than with the user and connected with the voice service through a communications infrastructure, the controller being arranged to control excitation of the loudspeaker in dependence on the relative positions of the users and entity.

15. A system according to claim 1, wherein said audio output arrangement comprises headphones forming part of the user equipment, said controllable functionality comprising a mouth representation device associated with the local entity and arranged to present a mouth representation that is movable dependence on the control data from the voice service whereby to operate in synchronism with voice output from the voice service.

16. A system according to claim 15, wherein the mouth representation device iselectro-mechanical in form with movable mouth parts, said controllable functionality further comprising electrically-powered actuators for moving said mouth parts.

17. A system according to claim 15, wherein the mouth representation device comprises an electronic display for displaying a mouth image.

18. A system according to claim 1, wherein the contact-data providing arrangement comprise markings that are located on or adjacent the entity and represent the contact data, the contact-data input subsystem of the user'"'"'s equipment comprising a scanner for reading the markings.

19. A system according to claim 1, wherein the contact-data providing arrangement is arranged to present the contact data to the user visually or audibly, the contact-data input subsystem of the user'"'"'s equipment comprising a user-operable input arrangement through which the user can input the contact data into their equipment.

20. A system according to claim 1, wherein in said dialog the entity is represented in first person terms through the voice service.

21. A system according to claim 1, wherein both the audio input and output arrangements are located in the locality of said entity apart from the user equipment, the voice service arrangement being operative to exchange said voice input and voice output with the audio input and output devices as voice signals passed across the communications infrastructure.

22. A system according to claim 1, wherein the audio input arrangement forms part of the user equipment and the latter is arranged to pass said voice input as voice signals across the wireless network to the voice service, the audio output arrangement being located in the locality of said entity apart from the user equipment and the voice service arrangement being arranged to pass said voice output as voice signals to the audio output arrangement across the communications infrastructure.

23. A system according to claim 1, wherein the voice service arrangement comprises:

a voice page server for serving voice pages in the form of text with embedded voice markup tags; and

a voice browser comprising;

a speech recognizer for carrying out speech recognition of user voice input received as voice signals;

a dialog manager for effecting dialog control on the basis of output from the speech recognizer and pages served by the voice page server; and

a text-to-speech converter operative to convert voice pages into voice output signals under the control of the dialog manager.

24. A system according to claim 1, wherein the wireless network is a proprietary-space local network hosting the voice service arrangement, the local entity being located in the proprietary-space concerned.

25. A system according to claim 5, wherein the wireless network is a proprietary-space local network hosting the voice service arrangement, the local entity being located in the proprietary-space concerned.

26. A system according to claim 25, wherein said audio output arrangement comprises headphones worn by the user and a controller for controlling excitation of the headphones in dependence on the relative positions of the user and entity and rotations of the user'"'"'s head such that the sound output appears to the user to emanate from the location of said local entity independently of the user'"'"'s position and head orientation relative to the entity.

27. A system according to claim 1, further comprising an arrangement for determining the orientation of the local entity as perceived from the user'"'"'s current location, the voice service being operative to condition its output in dependence on the determined orientation of the local entity.

28. A system according to claim 1, further comprising an arrangement for determining the orientation of the user relative to the entity, the voice service being operative to condition its output in dependence on the user'"'"'s determined orientation.

29. A system according to claim 1, further comprising an arrangement for determining the line of approach or departure of the user relative to the entity, the voice service being operative to condition its output in dependence on the user'"'"'s line of approach or departure.

30. A method of interacting with a local entity wherein:

(a) upon a user approaching the local entity, contact data, identifying a voice service associated with the entity but separately hosted, is presented to the user or to user-carried equipment

(b) the contact data is used by the user'"'"'s equipment to contact the voice service over a wireless network;

(c) the user interacts with the voice service through spoken dialog with both voice input by the user and voice output by the service;

(d) the voice service controls the operation of functionality associated with the local entity by means of control data passed to the functionality over a short-range wireless link from the user-carried equipment whereby to coordinate operation of the functionality with said voice output.

34. A method according to claim 30, wherein sound output is through multiple sound output devices spaced from said local entity and controlled so that the sound appears to the user to emanate from the location of said local entity independently of the user'"'"'s position and head orientation relative to the entity.

35. A method according to claim 34, wherein said multiple sound output devices are headphones worn by the user, excitation of the headphones being controlled to take account of the relative positions of the user and entity and rotations of the user'"'"'s head.

36. A method according to claim 34, wherein said multiple sound output devices are loudspeakers associated with the locality of the entity rather than with the user and connected with the voice service through the communications infrastructure, excitation of the loudspeakers being controlled in dependence on the relative positions of the user and entity.

37. A method according to claim 30, wherein:

the voice service is effected by the serving of voice pages in the form of text with embedded voice markup tags to a voice browser, the voice browser interpreting these pages and carrying out speech recognition of user voice input, text to speech conversion to generate voice output, and dialog management;

the voice browser being disposed between a voice page server and the user; and

the user equipment includes a mobile phone, (b) involving contacting the voice service using the mobile phone and (c) involving the mobile phone to transfer voice service input and output to and from the user.

38. A method according to claim 37, wherein the voice browser is not part of the user'"'"'s equipment and the contact data comprises a telephone number which when dialled by the mobile phone connects over a voice circuit to the voice browser and causes the latter to access the voice page server, the voice circuit being subsequently used for the exchange of voice input and/or output between the user and voice browser.

39. A method according to claim 37, wherein the voice browser is not part of the user'"'"'s equipment and the contact data is in the form of a URL which in (b) the mobile phone passes, via a data-capable bearer service of the mobile-phone wireless network, to the voice service;

the voice service then using the voice browser to call back the user on the mobile phone using a voice circuit that is then used in (c) for voice input and/or output between the user and voice browser.

40. A method according to claim 37, wherein the voice browser is not part of the user'"'"'s equipment and the contact data is in the form of a URL which in (b) the mobile phone passes, via a data-capable bearer service of the mobile-phone wireless network, to the voice service;

the data-capable bearer service being subsequently used in (c) for voice input and/or output between the user and voice browser using a packetized voice protocol.

41. A method according to claim 37, wherein the voice browser is part of the user'"'"'s equipment and the contact data is in the form of a URL which in (b) the voice browser uses to access, via a data-capable bearer service of the mobile-phone wireless network, the voice page server;

the data-capable bearer service being subsequently used in (c) for passing text based input and/or output between the voice browser and voice page server.

42. A method according to claim 30, wherein the wireless network is a proprietary-space local network hosting the voice service, the local entity being located in the proprietary-space concerned.

43. A method according to claim 42, wherein the user equipment includes a wireless headset which in (c) is used for exchanging voice input and output with the voice service over the same wireless network as used in (b).

44. A method according to claim 30, wherein the voice output of the voice service is output via headphones of the user equipment, said controllable functionality comprising a mouth representation device associated with the local entity and arranged to present a mouth representation that is movable in dependence on the control data from the voice service whereby to operate in synchronism with voice output from the voice service.

45. A method according to claim 44, wherein the mouth representation device is incorporated into the beacon device.

46. A method according to claim 44, wherein the mouth representation device is electro-mechanical in form with movable mouth parts operated by electrically-powered actuators in dependence on said control data.

47. A method according to claim 44, wherein the mouth representation device comprises an electronic display for displaying a mouth image.

48. A method according to claim 30, wherein the contact data is presented to the user'"'"'s equipment by the scanning into the equipment of markings that are located on or adjacent the entity and represent the contact data.

49. A method according to claim 30, wherein the contact data is visually or audibly presented to the user with the latter then inputting the contact data in their equipment.

50. A method according to claim 30, wherein both said voice input and voice output are carried across the wireless network between the voice service and sound input and output devices forming part of the user'"'"'s equipment.

51. A method according to claim 30, wherein both said voice input and voice output are exchanged with the user by local sound input and output devices that are associated with the locality of the entity rather than with the user and are connected with the voice service through a communications infrastructure.

52. A method according to claim 30, wherein said voice input is carried across the wireless network to the voice service from a sound input device forming part of the user'"'"'s equipment, and said voice output is effected through at least one local sound output device that is associated with the locality of the entity rather than with the user and is connected with the voice service through a communications infrastructure.

53. A method according to claim 30, wherein the voice service is effected by the serving of voice pages in the form of text with embedded voice markup tags to a voice browser, the voice browser interpreting these pages and carrying out speech recognition of user voice input, text to speech conversion to generate voice output, and dialog management;

the voice browser being disposed between a voice page server and the user.

54. A method according to claim 30, wherein the user equipment includes a mobile phone, (b) involving contacting the voice service using the mobile phone and (c) involving the mobile phone to transfer voice service input and output to and from the user.

55. A method according to claim 30, wherein the carrying out of (b) is subject to user approval at the time.

56. A method according to claim 30, wherein the user equipment ensures that the user is only connected to one voice service at a time regardless of how many local entities with beacon devices are within pickup range.

57. A method according to claim 30, wherein in (b) the identity of the user is sent to the voice service and used by the latter to look up user profile data which is then used to customise the voice service to the user.

58. A method according to claim 30, wherein the voice output provided from the service in (c) is dependent on the orientation of the local entity as perceived from the user'"'"'s current location.

59. A method according to claim 30, wherein the voice output provided from the service in (c) is dependent on the user'"'"'s orientation relative to the entity.

60. A method according to claim 30, wherein the voice output provided from the service in (c) is dependent on the user'"'"'s line of approach or departure relative to the entity.

31. A method according to claim wherein the contact data is presented to the user'"'"'s equipment by means of a beacon device located at or near the local entity and communicating with the user'"'"'s equipment over said short-range wireless link.

32. A method according to claim 31, wherein the beacon device includes parameter values relating to the state of said local entity in said contact data, these parameter values being passed in (b) over the wireless network to the voice service where they are used in conditioning the output of the voice service.

33. A method according to claim 31, wherein multiple beacon devices are associated with the entity each arranged to communicate different contact data, the voice service adapting its voice output in respect of said local entity in dependence on the contact data of the beacon device first or most-recently picked up by the user-equipment.