Multimedia streaming system for wireless handheld devices

• US 20040181611A1
• Filed: 03/14/2003
• Published: 09/16/2004
• Est. Priority Date: 03/14/2003
• Status: Abandoned Application

• Current Assignee: Seiko Epson Corporation (Seiko Group)
• Original Assignee: Seiko Epson Corporation (Seiko Group)
• Inventors: Ratnakar, Viresh
• Application Number:
  US10/390,130
• Publication Number:
  US 20040181611A1
• Time in Patent Office: Days
• US Class Current: 709/250
• CPC Class Codes: H04L 65/1101 Session protocols
• CPC Class Codes: H04L 65/612 for unicast

Abstract

A client capable of displaying received video data is provided. The client includes a network input/output (IO) component configured to receive video data and transmit requests for video data to a server. The scheduled-thread component is configured to decode and display the video data. The reader-thread component is configured to obtain the video data through the network IO. The reader-thread component further configured to supply the scheduled-thread component with the received video data from a buffer associated with the reader-thread component. A system for streaming video/audio data, a method for communicating video/audio data between a server and a client and a computer readable medium are also included.

Claims

1. A system configured to support multimedia streaming, comprising:

a client, the client associated with a reader-thread component and a scheduled-thread component, the reader-thread component associated with a buffer for storing data received by the client, the reader-thread component further configured to perform network input/output functionality, the scheduled-thread component enabling decoding and displaying a multimedia stream of data, the scheduled-thread component further configured to obtain access units from the buffer; and

a server in communication with the client through a network, the server configured to transmit access units specified by the client to the client.

2. The system of claim 1, wherein the reader-thread component is configured to initiate data transfer to the client by transmitting a send packet to the server.

3. The system of claim 2, wherein the send packet includes a start sequence number indicating a frame of image data, a requested number of frames of image data subsequent to the start sequence number to be sent to the client, and a skip-list indicated frames of image data within the requested number of frames that the client has stored.

4. The system of claim 1, wherein the client is a wireless handheld device enabled for power scalability, the power scalability causing a video quality to degrade as power available for the wireless handheld device decreases.

5. The system of claim 1, wherein the access units are contained within packets configured to be transmitted over the network through a user datagram protocol (UDP).

6. A system for streaming digital video/audio data, comprising:

a client configured to maintain a state of the client;

a server in communication with the client through a network,

a User Datagram Protocol (UDP) enabling communication between the client and the server through the network, the UDP protocol defining a packet configured to hold at least one access unit (AU), each of the at least one AU representing one frame of video/audio data, the one frame of video/audio data being transmitted to the client from the server, the transmission of the one frame of data being initiated by transmission of a send packet from the client to the server based upon the state maintained by the client, wherein in response to receiving the send packet, the server transmits the one frame of video data specified in the send packet.

7. The system of claim 6, wherein the state of the client includes data associated with a starting frame of video/audio data needed by the client, data associated with a sequence of video/audio data from the starting frame, and data associated with portions of the sequence of video/audio data that the client has stored.

8. The system of claim 6, wherein the client and the server communicate through a wireless network.

9. The system of claim 6, wherein the packet is formatted with null terminated lines.

10. A client capable of displaying received video data, comprising:

a network input/output (IO) component configured to receive video data and transmit requests for video data to a server;

a scheduled-thread component configured to decode and display the video data; and

a reader-thread component configured to obtain the video data through the network IO, the reader-thread component further configured to supply the scheduled-thread component with the received video data from a buffer associated with the reader-thread component.

11. The client of claim 10, wherein the reader-thread component invokes a process function to obtain the video data.

12. The client of claim 11, wherein the process function determines whether the video data is in the buffer.

13. The client of claim 10, wherein the reader-thread component calls a helper function to request data, the helper function specifying a starting point and an amount of data requested for the client.

14. The client of claim 10, wherein the reader-thread component calls a helper function to read data stored in the buffer.

15. A method for communicating video/audio data from a server to a client, comprising:

transmitting a call from the client to the server to identify a streaming session;

in response to receiving the call from the client, transmitting an initial object descriptor (IOD) from the server to the client;

initiating data transfer in response to receiving the IOD, the initiating including,

communicating data indicating both a start sequence of streaming session data and an amount of the streaming session data to the server, the start sequence and the amount being determined by the client; and

responding to the receipt of the data indicating the start sequence and the amount of the streaming session data by supplying the client with access units containing the video/audio data.

16. The method of claim 15, wherein the method operation of initiating data transfer in response to receiving the IOD includes,

invoking a process function;

determining if needed streaming session data is available in a buffer associated with the client;

if the needed streaming session data is not available, then the method includes sending a request for the needed streaming session data.

17. The method of claim 15, wherein the call is an OPEN packet indicating a session identification and a streaming session name.

18. The method of claim 15 wherein, the IOD specifies a set of profile and level information needed by the client to assess the processing resources needed for the streaming session.

19. The method of claim 15, wherein a SEND packet is used by the client to communicate the start sequence and the amount of streaming session data to the server.

20. The method of claim 19, wherein the SEND packet includes a skip list, the skip list indicating a portion of the streaming session data the client has access to.

21. The method of claim 15, wherein the access units are transmitted to the client from the server in packets each packet containing one or more access units, each access unit corresponding to a single frame of video/audio data of the streaming session.

22. The method of claim 15 wherein, the system is built over a User Datagram Protocol.

23. The method of claim 16, wherein the process function is invoked periodically by a reader-thread associated with the client.

24. The method of claim 16, wherein the process function determines if the access units are available within a buffer associated with the client;

and

if the access units are available, then the method includes reading the access units from one of a buffer associated with the client and a network connection;

if the access units are not available, then the method includes determining if a buffer occupancy threshold has been exceeded, wherein if the buffer occupancy threshold has been exceeded a request is sent to the server from the client for additional access units.

25. The method of claim 24, wherein if the buffer occupancy threshold has not been exceeded, then the client waits for the access units to become available.

26. A computer readable medium having program instructions for communicating video/audio data from a server to a client, comprising:

program instructions for transmitting a call from the client to the server to identify a streaming session;

program instructions for transmitting an initial object descriptor (IOD) from the server to the client in response to receiving the call from the client;

program instructions for initiating data transfer in response to receiving the IOD, the program instructions for initiating including,

program instructions for communicating data indicating both a start sequence of streaming session data and an amount of the streaming session data to the server, the start sequence and the amount being determined by the client; and

program instructions for responding to the receipt of the data indicating the start sequence and the amount of the streaming session data by supplying the client with access units containing the video/audio data.

27. The computer readable medium of claim 26, wherein the program instructions for initiating data transfer in response to receiving the IOD includes,

program instructions for invoking a process function;

program instructions for determining if needed streaming session data is available in a buffer associated with the client; and

program instructions for sending a request for the needed streaming session data if the needed streaming session data is not available.

28. The computer readable medium of claim 27, wherein the program instructions for invoking the process function are invoked periodically by a reader-thread associated with the client.

29. The computer readable medium of claim 27, wherein the program instructions for the process function determines if the access units are available within a buffer associated with the client.

30. The computer readable medium of claim 29, further comprising:

program instructions for reading the access units from one of a buffer associated with the client and a network connection if the access units are available; and

program instructions for determining if a buffer occupancy threshold has been exceeded if the access units are not available, wherein if the buffer occupancy threshold has been exceeded, program instructions for requesting additional access units from the server are executed.