Method and system of implementing real-time video-audio interaction by data synchronization

• US 20040179554A1
• Filed: 03/11/2004
• Published: 09/16/2004
• Est. Priority Date: 03/12/2003
• Status: Abandoned Application

• Current Assignee: ASUSTek Computer, Inc.
• Original Assignee: ASUSTek Computer, Inc.
• Inventors: Tsao, Hsi-Kang
• Application Number:
  US10/797,006
• Publication Number:
  US 20040179554A1
• Time in Patent Office: Days
• US Class Current: 370/486
• CPC Class Codes: H04N 21/43072 of multiple content streams on the same device
• CPC Class Codes: H04N 21/4781 Games
• CPC Class Codes: H04N 21/4788 communicating with other users, e.g. chatting arrangements for providing for computer conferences, e.g. chat rooms, to substation in data switching networks H04L12/1813; distributed application using peer-to-peer [P2P] networks H04L67/104
• CPC Class Codes: H04N 21/6125 involving transmission via Internet transmission by internet of broadcast information H04H60/82
• CPC Class Codes: H04N 21/6175 involving transmission via Internet broadcast-related systems characterised by the transmission system being the Internet H04H60/82
• CPC Class Codes: H04N 21/8547 involving timestamps for synchronizing content
• CPC Class Codes: H04N 7/173 with two-way working, e.g. subscriber sending a programme selection signal
• CPC Class Codes: H04N 7/52 Systems for transmission of a pulse code modulated video signal with one or more other pulse code modulated signals, e.g. an audio signal or a synchronizing signal assembling of a multiplex stream by combining a video stream with other content or additional data, remultiplexing of multiplex streams, insertion of stuffing bits into the multiplex stream, assembling of a packetised elementary stream at server side H04N21/236; disassembling of a multiplex stream, remultiplexing of multiplex streams, extraction or processing of Service Information, disassembling of packetised elementary stream at client side H04N21/434

Abstract

A method of implementing real-time video-audio interaction by data synchronization in an Internet game. The inventive method establishes an Internet transmission channel between Internet game clients without connecting to the Internet game sever. First, the method retrieves real-time video and audio data and compresses it into video and audio data packets. Next, the video and audio data packets are packaged into a transmission package. A time stamp is attached to the transmission package for data synchronization. The transmission package is transmitted to other game client through the Internet transmission channel. Finally, the Internet game client receives the transmission package, decompresses the received transmission package into video and audio data, and executes data synchronization according to the time stamp and output thereof.

Claims

1. A method of implementing real-time video-audio interaction by data synchronization in an Internet game, comprising the steps of:

establishing an Internet transmission channel between a first internet game client and a second internet game client, wherein the Internet transmission channel is not connected to an Internet game server;

executing an internet game in the first Internet game client and the second internet game client and connecting the first and second Internet game clients to the internet game server;

retrieving first real-time video data and first real-time audio data in the first internet game client in the Internet game;

compressing/encoding the first real-time video data into a plurality of first video data frames, and compressing/encoding the first real-time audio data into a plurality of first audio data packets in the first Internet game client;

packaging the first video data frames and the first audio data packets into a transmission package in the first Internet game client and attaching a time stamp to transmission package, wherein the time stamp expresses the synchronous relationship between the first real-time video and audio data;

transmitting the transmission package to the second Internet game client through the Internet transmission channel;

decoding the transmission package into second real-time video data and second real-time audio data in the second Internet game client; and

synchronizing the second real-time video and audio data according to the time stamp, and outputting the second real-time audio and video data in the second Internet game client in the Internet game.

2. The method as claimed in claim 1, wherein the establishment of the Internet transmission channel further comprises the steps of:

designating an Internet address of the second Internet game client directly or according to a directory by the first Internet game client, wherein the directory includes the Internet address of the second Internet game client;

transmitting a connection request from the first Internet game client to the second Internet game client; and

establishing the Internet transmission channel by the second Internet game client in response to the connection request.

3. The method as claimed in claim 1, wherein it the bandwidth of the Internet transmission channel cannot transmit the first real-time audio data and the first real-time video data simultaneously, the first real-time audio data takes priority over first real-time video data.

4. The method as claimed in claim 1, wherein the time stamp provides is time information required to produce the first real-time video data and the first real-time audio data.

5. The method as claimed in claim 1, wherein the synchronization is achieved by adding the system time of the second internet game client to the time stamp to generate the display time of the second real-time video and audio data.

6. The method as claimed in claim 1, wherein synchronization is achieved by comparing the time stamp the amount of the frames dropped by the second real-time video data.

7. The method as claimed in claim 1, wherein playback of the second real-time video data is accomplished by integrating the second real-time video data into the game environments of the Internet game as texture mapping.

8. A system of implementing real-time video-audio interaction by data synchronization in an Internet game, comprising:

an Internet game server, executing an Internet game; and

a plurality of Internet game clients, comprising a first Internet game client, a second Internet game client, and an Internet transmission channel, the first and the second Internet game clients connecting to the Internet game server, the Internet transmission channel coupled to the first Internet game client and the second Internet game client

9. The system as claimed in claim 8, wherein the first internet game client further comprises:

a real-time data retriever, retrieving first real-time video data and first real-time audio data from the first Internet game client;

a data encoder, coupled to the real-time data retriever, compressing/encoding the first real-time video data into a plurality of first video data frames, and compressing/encoding the first audio data into a plurality of first audio data packets;

a transmission packager, coupled to the data encoder, packaging the first video data frames and the first audio data packets into a transmission package and attaching a time stamp into the transmission package, wherein the time stamp expresses the synchronous relationship between the first real-time video data and the first real-time audio data; and

an Internet sender, coupled to the transmission packager, transmitting the transmission package to the second Internet game client through the Internet transmission channel.

10. The system as claimed in claim 9, wherein if the bandwidth of the Internet transmission channel cannot transmit the first real-time audio data and the first real-time video data simultaneously, the first real-time audio data takes priority over first real-time video data.

11. The system as claimed in claim 9, wherein the time stamp provides the time information required to produce the first real-time video data and the first real-time audio data.

12. The system as claimed in claim 8, wherein the second Internet game client further comprises:

a data decoder, coupled to the Internet transmission channel, decoding the transmission package into second video data and second audio data;

a video-audio playback system, coupled to the data decoder, synchronizing the second real-time video and the second real-time audio data according to the time stamp and outputting the second video data and the second audio data.

13. The system as claimed in claim 12, wherein synchronization is achieved by adding the system time of the second internet game client to the time stamp to generate the display time of the second real-time video and audio data.

14. The system as claimed in claim 12, wherein synchronization is achieved by comparing the time stamp the amount of the frames dropped by the second real-time video data.

15. The system as claimed in claim 12, wherein the video-audio playback system integrates the second real-time video data into the game environments of the Internet game as texture mapping.

16. The system as claimed in claim 8, wherein the internet transmission channel is established by assigning an Internet address to the second Internet game client directly or according to a directory by the first Internet game client, transmitting a connecting request from the first Internet game client to the second Internet game client, and the second Internet game client establishing the Internet transmission channel according to the connecting request, wherein the directory includes the Internet address of the second Internet game client.

17. A method of implementing real-time interaction by video-audio synchronization between Internet game clients, wherein the Internet game client connects to an Internet game server, and executes an Internet game, comprising the steps of:

establishing an Internet transmission channel to an external Internet game client, wherein the Internet transmission channel is not connected to the Internet game server;

a real-time data retriever retrieving first real-time video data and first real-time audio data;

compressing/decoding the first real-time video data and the first real-time audio data into a first transmission package and attaching a time stamp to the transmission package, wherein the time stamp expresses the synchronous relationship between the video and audio data;

transmitting the first transmission package through the Internet transmission channel;

receiving a second transmission package through the Internet transmission channel;

decompressing/decoding the second transmission package into second real-time video data and second real-time audio data; and

synchronizing the second real-time video and the second real-time audio data according to the time stamp, and outputting the second real-time audio data and video data in the game environment.

18. The method as claimed in claim 17, wherein the establishment of the Internet transmission channel further comprises the steps of:

designating an Internet address of a third external Internet game client by the Internet game client or the external Internet game client;

transmitting a connecting request to the third Internet game client by the Internet game client or the external Internet game client according to the Internet address; and

establishing the Internet transmission channel between the Internet game client and the third Internet game client.

19. The method as claimed in claim 17, wherein if the bandwidth of the internet transmission channel cannot transmit the first real-time audio data and the first real-time video data simultaneously, the first real-time audio data takes priority over first real-time video data.

20. The method as claimed in claim 17, wherein in the establishing step, the Internet transmission channel is established according to a directory, having an Internet address of the third external Internet game client.

21. The method as claimed in claim 17, wherein synchronization is achieved by adding the system time of the second internet game client to the time stamp to generate the display time of the second real-time video and audio data.

22. The method as claimed in claim 17, wherein synchronization is achieved by comparing the time stamp the amount of the frames dropped by the second real-time video data.

23. The method as claimed in claim 17, wherein playback of the second video data is accomplished by integrating the second real-time video data into the game environment as texture mapping.

24. A storage medium for storing a computer program providing a method of implementing real-time video-audio interaction by data synchronization between Internet game clients, wherein the Internet game client connects to an Internet game server, and executes an Internet game, the computer program comprising using a computer to perform the steps of:

establishing an Internet transmission channel to an external Internet game client, wherein the Internet transmission channel is not connected to the Internet game server;

a real-time data retriever retrieving first real-time video data and first real-time audio data;

compressing/decoding the first real-time video data and the first real-time audio data into a first transmission package, and attaching a time stamp into the transmission package, wherein the time stamp expresses the synchronous relationship between the video and audio data;

transmitting the first transmission package through the Internet transmission channel;

receiving a second transmission package through the Internet transmission channel;

decompressing/decoding the second transmission package into second real-time video data and second real-time audio data; and

synchronizing the second real-time video and the second real-time audio data according to the time stamp, and outputting the second real-time audio data and video data in the game environment.

25. The method as claimed in claim 24, wherein the establishment of the Internet transmission channel further comprises the steps of:

designating an Internet address of a third external Internet game client by the Internet game client or the external Internet game client;

a transmitting a connecting request to the third Internet game client by the Internet game client or the external Internet game client according to the Internet address; and

establishing the Internet transmission channel between the Internet game client and the third Internet game client.

26. The method as claimed in claim 24, wherein if the bandwidth of the internet transmission channel cannot transmit the first real-time audio data and the first real-time video data simultaneously, the first real-time audio data takes priority over first real-time video data.

27. The method as claimed in claim 24, wherein in the establishing step, the Internet transmission channel is established according to a directory, having an Internet address of the third external Internet game client.

28. The method as claimed in claim 24, wherein synchronization is achieved by adding the system time of the second internet game client to the time stamp to generate the display time of the second real-time video and audio data.

29. The method as claimed in claim 24, wherein synchronization is achieved by comparing the time stamp the amount of the frames dropped by the second real-time video data.

30. The method as claimed in claim 24, wherein playback of the second video data is accomplished by integrating the second real-time video data into the game environment as texture mapping.

31. A computer system of an Internet game, executing an Internet game and having a storage medium for storing a computer program, wherein the computer program is applied to a computer system and executes the method of real-time video-audio interaction between Internet game clients, the Internet game client connecting to an Internet game server, executing an Internet game, and outputting a game environment, the method comprising the steps of:

establishing an Internet transmission channel to an external Internet game client, wherein the Internet transmission channel is not connected to the Internet game server;

a real-time data retriever retrieving first real-time video data and first real-time audio data;

compressing/decoding the first real-time video data and the first real-time audio data into a first transmission package, and attaching a time stamp into the transmission package, wherein the time stamp expresses the synchronous relationship between the video and audio data;

transmitting the first transmission package through the Internet transmission channel;

receiving a second transmission package through the Internet transmission channel;

decompressing/decoding the second transmission package into second real-time video data and second real-time audio data; and

synchronizing the second real-time video and the second real-time audio data according to the time stamp, and outputting the second real-time audio data and video data in the game environment,

32. The method as claimed in claim 31, wherein the establishment of the Internet transmission channel further comprises the steps of:

designating an Internet address of a third external Internet game client by the Internet game client or the external Internet game client;

transmitting a connecting request to the third Internet game client by the Internet game client or the external Internet game client according to the Internet address; and

establishing the Internet transmission channel between the Internet game client and the third Internet game client.

33. The method as claimed in claim 31, wherein if the bandwidth of the internet transmission channel cannot transmit the first real-time audio data and the first real-time video data simultaneously, the first real-time audio data takes priority over first real-time video data.

34. The method as claimed in claim 31, wherein in the establishing step, the Internet transmission channel is established according to a directory, having an Internet address of the third external Internet game client.

35. The method as claimed in claim 31, wherein synchronization is achieved by adding the system time of the second internet game client to the time stamp to generate the display time of the second real-time video and audio data.

36. The method as claimed in claim 31, wherein synchronization is achieved by comparing the time stamp the amount of the frames dropped by the second real-time video data.

37. The method as claimed in claim 31, wherein playback of the second video data is accomplished by integrating the second real-time video data into the game environment as texture mapping.

38. A method of implementing real-time video-audio interaction by data synchronization in an internet game for applying in a first Internet game client and a second Internet game client, wherein the first and second Internet game client execute an Internet game and connect to an Internet game server, comprising the steps of:

establishing an Internet transmission channel between the first Internet game client and the second Internet game client, wherein the Internet transmission channel is not connected to the Internet game server;

retrieving first real-time video data and first real-time audio data in the first Internet game client;

producing a plurality of first video data frames and a plurality of first audio data packets;

packaging the first video data frames and the first audio data packets into a transmission package and attaching a time stamp into the transmission package, wherein the time stamp expresses the synchronous relationship between the first real-time video and audio data;

transmitting the transmission package to the second Internet game client;

decoding the transmission package into second real-time video data and second real-time audio data; and

synchronizing the second real-time audio and video data according to the time stamp, and outputting the second real-time audio data and video data in the Internet game in the second Internet game client.

39. The method as claimed in claim 38, wherein the establishing step further comprises the steps of:

designating an Internet address of the second Internet game client directly or according to a directory by the first Internet game client, wherein the directory includes the Internet address of the second Internet game client;

transmitting a connection request from the first Internet game client to the second Internet game client; and

establishing the Internet transmission channel by the second Internet game client in response to the connection request.

40. The method as claimed in claim 38, wherein the first real-time audio data is primarily packaged in the first transmission package, and the remaining bandwidth is used for packaging the first real-time video data.

41. The method as claimed in claim 38, wherein the first video data frames and the first audio data frames are produced by compressing/encoding.

42. The method as claimed in claim 38, wherein the transmission package is transmitted to the second Internet game client through the Internet transmission channel.

43. The method as claimed in claim 38, wherein the synchronization is based on system time of the second Internet game client adding the time stamp as display time of the second real-time video and audio data.

44. The method as claimed in claim 38, wherein synchronization is achieved by comparing the time stamp the amount of the frames dropped by the second real-time video data.

45. A system of implementing real-time video-audio interaction by data synchronization in an Internet game for application to a first Internet game client, a second Internet game client, and an internet game server, wherein the Internet game server executes an Internet game, the system comprising:

an Internet transmission channel, the first Internet game client and the second Internet game client connecting to the Internet game server to execute the Internet game, the Internet transmission channel coupled to the first and second Internet game clients to execute real-time video-audio interaction.