System and method for voice transmission over network protocols

• US 20040088168A1
• Filed: 07/21/2003
• Published: 05/06/2004
• Est. Priority Date: 11/05/1999
• Status: Active Grant

• Current Assignee: Open Invention Network LLC
• Original Assignee: Raindance Communications Incorporated (West Corporation)
• Inventors: Boyns, Mark Randle, Scherpbier, Andrew W.
• Granted Patent: US 7,236,926 B2
• Time in Patent Office: Days
• US Class Current: 704/270.1
• CPC Class Codes: G10L 19/012 Comfort noise or silence coding
• CPC Class Codes: H04L 49/90 Buffering arrangements
• CPC Class Codes: H04L 65/1101 Session protocols
• CPC Class Codes: H04L 65/4046 with distributed floor control
• CPC Class Codes: H04L 65/65 Network streaming protocols, e.g. real-time transport protocol [RTP] or real-time control protocol [RTCP]
• CPC Class Codes: H04L 65/70 Media network packetisation
• CPC Class Codes: H04L 65/80 Responding to QoS
• CPC Class Codes: H04L 67/02 based on web technology, e.g. hypertext transfer protocol [HTTP]
• CPC Class Codes: H04L 69/329 in the application layer [OSI layer 7]

Abstract

A system and method for voice transmission over high level network protocols. On the Internet and the World Wide Web, such high level protocols are HTTP/TCP. The restrictions imposed by firewalls and proxy servers are avoided by using HTTP level connections to transmit voice data. In addition, packet delivery guarantees are obtained by using TCP instead of UDP. Variable compression based on silence detection takes advantage of the natural silences and pauses in human speech, thus reducing the delays in transmission caused by using HTTP/TCP. The silence detection includes the ability to bookend the voice data sent with small portions of silence to insure that the voice sounds natural. Finally, the voice data is transmitted to each client computer, independently from a common circular list of voice data, thus insuring that all clients will stay current with the most recent voice data. The combination of these features enables simple, seamless, and interactive Internet conferencing.

Claims

1. A method for transmitting voice from a presenting computer to one or more client computers over a computer network, comprising the steps of:

inputting audio data into the presenting computer;

monitoring the audio data to distinguish between periods of sound and periods of silence;

storing the audio data associated with the periods of sound; and

transmitting the audio data from said storing step to the client computers.

2. The method of claim 1, wherein said storing step includes storing the audio data in a frame buffer having a predetermined size, and said transmitting step occurs whenever said monitoring step detects a period of silence or whenever the frame buffer is full.

3. The method of claim 2, the method further comprising the step of:

creating a voice buffer from the frame buffer before said transmitting step.

4. The method of claim 3, wherein said creating step is accomplished by compressing the contents of the frame buffer.

5. The method of claim 4, further comprising the step of:

saving the voice buffer in a data structure.

6. The method of claim 5, wherein said transmitting step further comprises the steps of:

observing the data structure to identify when one or more new voice buffers have been saved; and

sending each of the one or more new voice buffers to the client computers.

7. The method of claim 6, wherein said transmitting step is performed using a separate listening thread for each of the client computers.

8. The method of claim 7, wherein the data structure is a circular list having a predetermined size.

9. The method of claim 7, wherein said sending step is performed via an open HTTP connection to each of the client computers.

10. The method of claim 7, wherein said saving and transmitting steps are performed by a server, the method further comprising the step of:

transferring the voice buffer from the presenting computer to the server.

11. The method of claim 7, wherein said creating, saving, and transmitting steps are performed by a server, the method further comprising the step of:

transferring the frame buffer from the presenting computer to the server.

12. The method of claim 9, wherein said saving and transmitting steps are performed by a server, the method further comprising the step of:

transferring the voice buffer from the presenting computer to the server.

13. The method of claim 9, wherein said creating, saving, and transmitting steps are performed by a server, the method further comprising the step of:

transferring the frame buffer from the presenting computer to the server.

14. The method of claim 1, wherein said storing step further comprises the steps of:

maintaining a small portion of audio data associated with the periods of silence just before the periods of sound;

first-recording said small portion of audio data along with, and preceding, the audio data associated with the periods of sound, in a frame buffer having a predetermined size; and

second-recording a small portion of audio data associated with periods of silence that occur after the audio data associated with the periods of sound, at the end of said frame buffer.

15. The method of claim 14, wherein the transmitting step occurs whenever said monitoring step detects a period of silence longer than either of said small portions, or whenever said frame buffer is full.

16. The method of claim 15, the method further comprising the step of:

creating a voice buffer from the frame buffer before said transmitting step.

17. The method of claim 16, wherein said creating step is accomplished by compressing the contents of the frame buffer.

18. The method of claim 17, further comprising the step of:

saving the voice buffer in a data structure.

19. The method of claim 18, wherein said transmitting step further comprises the steps of:

observing the data structure to identify when one or more new voice buffers have been saved; and

sending each of the new voice buffers to the client computers.

20. The method of claim 19, wherein said transmitting step is performed by multiple listening threads, one for each of the client computers.

21. The method of claim 20, wherein the data structure is a circular list of predetermined size.

22. The method of claim 20, wherein said sending step is performed via an open HTTP connection to each of the client computers.

23. The method of claim 20, wherein said saving and transmitting steps are performed by a server, the method further comprising the step of:

transferring the voice buffer from the presenting computer to the server.

24. The method of claim 20, wherein said creating, saving, and transmitting steps are performed by a server, the method further comprising the step of:

transferring the frame buffer from the presenting computer to the server.

25. The method of claim 22, wherein said saving and transmitting steps are performed by a server, the method further comprising the step of:

transferring the voice buffer from the presenting computer to the server.

26. The method of claim 22, wherein said creating, saving, and transmitting steps are performed by a server, the method further comprising the step of:

transferring the frame buffer from the presenting computer to the server.

27. A system for allowing a presenting computer to transmit voice to one or more client computers via a computer network, comprising:

a data store on the presenting computer for storing audio data;

a microphone coupled to said data store for inputting audio data; and

a recorder coupled to said data store and to said computer network, for monitoring said audio data to distinguish between periods of sound and periods of silence, and for transmitting said audio data associated with periods of sound to said client computers.

28. The system of claim 27, wherein the recorder stores the audio data associated with periods of sound on the data store in a frame buffer, and transmits the frame buffer when the recorder detects a period of silence or the frame buffer is full.

29. The system of claim 28, wherein the recorder compresses the frame buffer, to thereby create a voice buffer, before transmission to the client computers.

30. The system of claim 29, further comprising:

a data structure resident on said data store, wherein said recorder stores one or more voice buffers; and

a voice position pointer resident on said data store, which points to a location in said data structure in which to store a next voice buffer of said one or more voice buffers.

31. The system of claim 30, wherein the recorder includes one or more separate listening threads, one for each of said client computers, and each of the listening threads transmits to an associated one of said client computers, the system further comprising:

one or more separate listening position pointers, resident on said data store, one for each of said separate listening threads, wherein said listening pointers indicate which of said one or more voice buffers is to be transmitted to said associated client computer.

32. The system of claim 31, wherein the data structure is a circular list.

33. The system of claim 32, wherein the recorder uses one or more open HTTP connections for transmission to each of said one or more client computers.

34. The system of claim 33, wherein the recorder includes a small portion of preceding and terminating silence when it stores the audio data associated with periods of sound.

35. A system for allowing a presenting computer to transmit voice to one or more client computers via a computer network, comprising:

a data store on said presenting computer for storing audio data;

a microphone coupled to said data store for inputting audio data;

a recorder coupled to said data store and to said computer network, for monitoring said audio data to distinguish between periods of sound and periods of silence, and for transmitting audio data associated with periods of sound to a server, and

said server is coupled to said computer network for storing said audio data received from said recorder, and for transmitting said received audio data to said client computers.

36. The system of claim 35, wherein the recorder stores the audio data associated with periods of sound on the data store in a frame buffer, and transmits the frame buffer when the recorder detects a period of silence or the frame buffer is full.

37. The system of claim 36, wherein the recorder compresses the frame buffer to thereby create a voice buffer, before transmission to the client computers.

38. The system of claim 37, further comprising:

a data structure resident on said server, for storing one or more voice buffers received from said recorder; and

a voice position pointer resident on said server, which points to a location in said data structure in which to store a next voice buffer of said one or more voice buffers.

39. The system of claim 38, wherein the server maintains one or more separate listening threads, one for each of said client computers, and each of said listening threads transmits to an associated one of said client computers, the system further comprising:

one or more separate listening position pointers, resident on said server, one for each of said separate listening threads, wherein said listening pointers indicate which of said one or more voice buffers is to be transmitted to said associated client computer.

40. The system of claim 39, wherein said data structure is a circular list.

41. The system of claim 40, wherein said listening threads utilize one or more open HTTP connections for transmission to each of said client computers.

42. The system of claim 41, wherein said recorder includes a small portion of preceding and terminating silence when it stores the audio data associated with periods of sound.

43. A computer program product comprising a computer useable medium having computer program instructions stored therein for enabling a presenting computer to transmit voice to one or more client computers, wherein said computer program product comprises instructions for:

monitoring incoming audio data to distinguish between periods of sound and periods of silence; and

transmitting sound to the client computers.

44. The computer program product of claim 43, further comprising program instructions for setting a threshold for distinguishing between periods of sound and periods of silence in said monitoring step.

45. The computer program product of claim 44, further comprising program instructions for:

storing said incoming audio data in a frame buffer if said audio data exceeds said threshold; and

creating a voice buffer from the frame buffer when said incoming audio data falls below the threshold or the frame buffer is full.

46. The computer program product of claim 45, further comprising program instructions for storing said voice buffer in a data structure which holds one or more voice buffers.

47. The computer program product of claim 46, further comprising program instructions for transmitting the voice buffers in the data structure independently to each of the client computers.

48. The computer program product of claim, 47, wherein said transmitting program instructions utilize an open HTTP connection to each of the client computers.

49. The computer program product of claim 47, further comprising program instructions for compressing the frame buffer in said creating program instructions.

50. The computer program product of claim 47, further comprising program instructions for adding small bookends of silence to both ends of the frame buffer in said creating program instructions.

58. The computer program product of claim 47, wherein said data structure is a circular list, the program product further comprising instructions for maintaining said circular list.

51. A computer program product comprising a computer useable medium having computer program instructions stored therein for enabling a presenting computer to transmit voice, via a server, to one or more client computers, wherein said computer program product comprises instructions for:

at the presenting computer, monitoring incoming audio data to distinguish between periods of sound and periods of silence;

at the presenting computer, creating one or more voice, buffers from the incoming audio data;

at the presenting computer, transmitting the voice buffers to the server;

at the server, maintaining the voice buffers in a data structure; and

at the server, transmitting the voice buffers to the client computers.

52. The computer program product of claim 51, further comprising instructions for setting a threshold for distinguishing between periods of sound and periods of silence.

53. The computer program product of claim 52, further comprising instructions for:

storing the incoming audio data in a frame buffer if the audio data exceeds the threshold; and

creating a voice buffer from the frame buffer when the incoming audio data falls below the threshold or the frame buffer is full.

54. The computer program product of claim 53, further comprising instructions for independently transmitting the voice buffers stored in the data structure to each of the client computers.

55. The computer program product of claim 54, wherein said transmitting program instructions utilize an open HTTP connection to each of the client computers.

56. The computer program product of claim 54, further comprising program instructions for compressing the voice buffers received by the server before storing them in the data structure.

57. The computer program product of claim 54, further comprising program instructions for compressing the frame buffer in said creating program instructions.

59. The computer program product of claim 54, wherein said data structure is a circular list, the program product further comprising instructions for maintaining said circular list.

60. A method for transmitting voice as part of a collaborative web browsing session from a presenting computer to one or more client computers over a computer network, comprising the steps of:

inputting audio data into the presenting computer;

monitoring the audio data to distinguish between periods of sound and periods of silence;

storing the audio data associated with the periods of sound; and

transmitting the audio data from said storing step to the client computers.

61. The method of claim 60, wherein said storing step includes storing the audio data in a frame buffer having a predetermined size, and said transmitting step occurs whenever said monitoring step detects a period of silence or whenever the frame buffer is full.

62. The method of claim 61, the method further comprising the step of:

creating a voice buffer from the frame buffer before said transmitting step.

63. The method of claim 62, wherein said creating step is accomplished by compressing the contents of the frame buffer.

64. The method of claim 63, further comprising the step of:

saving the voice buffer in a data structure.

65. The method of claim 64, wherein said transmitting step further comprises the steps of:

observing the data structure to identify when one or more new voice buffers have been saved; and

sending each of the one or more new voice buffers to the client computers.

66. The method of claim 65, wherein said transmitting step is performed using a separate listening thread for each of the client computers.