Displaying image data

• US 20020194354A1
• Filed: 08/13/2001
• Published: 12/19/2002
• Est. Priority Date: 04/19/2001
• Status: Abandoned Application

• Current Assignee: Autodesk, Inc.
• Original Assignee: Discreet Logic, Inc. (Autodesk, Inc.)
• Inventors: Bolduc, Marc, Duchesne, Stephane
• Application Number:
  US09/928,598
• Publication Number:
  US 20020194354A1
• Time in Patent Office: Days
• US Class Current: 709/231
• CPC Class Codes: H04N 19/132 Sampling, masking or truncation of coding units, e.g. adaptive resampling, frame skipping, frame interpolation or high-frequency transform coefficient masking
• CPC Class Codes: H04N 19/162 User input
• CPC Class Codes: H04N 19/164 Feedback from the receiver or from the transmission channel
• CPC Class Codes: H04N 19/172 the region being a picture, frame or field
• CPC Class Codes: H04N 19/587 involving temporal sub-sampling or interpolation, e.g. decimation or subsequent interpolation of pictures in a video sequence
• CPC Class Codes: H04N 19/61 in combination with predictive coding
• CPC Class Codes: H04N 21/44004 involving video buffer management, e.g. video decoder buffer or video display buffer
• CPC Class Codes: H04N 21/44209 Monitoring of downstream path of the transmission network originating from a server, e.g. bandwidth variations of a wireless network arrangements for maintenance or administration in data switching networks involving bandwidth and capacity management H04L41/0896
• CPC Class Codes: H04N 21/4621 Controlling the complexity of the content stream or additional data, e.g. lowering the resolution or bit-rate of the video stream for a mobile client with a small screen arrangements for using the results of monitoring on user's side in broadcast systems H04H60/65; flow control in packet networks H04L47/10
• CPC Class Codes: H04N 21/47217 for controlling playback functions for recorded or on-demand content, e.g. using progress bars, mode or play-point indicators or bookmarks specific graphical features in visual interfaces H04N21/4312

Abstract

A method of viewing a clip of image data stored (109) remotely on a network (106). The viewing is performed by an image processing station (101) connected to the network. Frames of a clip are prefetched (701) and certain of the frames in a frame sequence are skipped, in alternation with frames that are fetched. Frames are skipped to compensate for network conditions. Display (702) of the prefetched frames is performed by selecting (1001) a prefetched frame for display appropriate to the elapsed real time since playback started. The clip is viewed in real time, even though the network (106) does not necessarily support the data transfer rate required for full playback of the clip.

Claims

1. Apparatus for viewing image data, comprising:

(a) display means;

(b) network connecting means for transferring frames of said image data over a network from a remotely connected frame source, wherein;

(i) said image data comprises a plurality of image frames and has a frame rate from which may be inferred a due time for display of each frame in a sequence of frames in said image data;

(ii) said frame source returns a frame in response to a frame request issued over said network; and

(c) processing means configured to play a clip by;

(i) displaying selected frames from said frame source, on said display means, at their due time; and

(ii) skipping frames in said frame sequence in response to an indication of the data transfer rate of said network.

2. Apparatus according to claim 1, wherein said indication of the data transfer rate is provided by a comparison of the relative position of an input and an output pointer in a queue of frames that have been selected for display.

3. Apparatus according to claim 1, wherein said frame source includes means for storing pre-rendered image frames.

4. Apparatus according to claim 1, wherein said frames are skipped in response to a prediction of a network data transfer rate.

5. Apparatus according to claim 1, wherein frames are prefetched into a frame queue prior to their due time.

6. Apparatus according to claim 1, wherein a frame skip rate is defined by a user.

7. Apparatus according to claim 1, wherein a frame is selected for display by processing its due time with elapsed real time since playback started.

8. Apparatus for displaying image data, comprising:

(a) image data comprising a plurality of image frames, sequences of said frames being organised into clips, each clip having a frame rate, and each frame in a clip thereby having a due time for display with respect to a start time for playing the clip;

(b) display means;

(c) memory means;

(d) network connecting means for enabling transfer of image data over a network from a frame source remotely connected to said network; and

(e) processing means configured to perform operations to play a clip from said frame source by;

(i) selecting a next frame for preloading by skipping at least one frame in the clip'"'"'s frame sequence;

(ii) preloading a frame from said frame source into a frame queue in said memory means;

(iii) displaying a preloaded frame at its due time;

(iv) processing elapsed real time since the clip started playing with a frame timing parameter; and

(v) updating the number of frames to skip in response to said processing of elapsed real time.

9. Apparatus according to claim 8, wherein said frame timing parameter is the due time for a frame.

10. Apparatus according to claim 8, wherein instructions for the processing means are executed as multiple threads.

11. A method of displaying image data on an image viewing station, wherein:

(a) the image viewing station comprises display means, processing means, and network connecting means for transferring frames of said image data over a network from a remotely connected frame source;

(b) said image data comprises a plurality of image frames, and has a frame rate from which may be inferred a due time for display of each frame in a sequence of frames in said image data;

(c) said frame source returns a frame in response to a frame request issued over said network; and

(d) said processing means is configured to play a clip in which said method comprises;

(i) displaying selected frames from said frame source, on said display means, at their due time; and

(ii) skipping frames in said frame sequence in response to an indication of the data transfer rate of said network.

12. A method according to claim 11, wherein said indication of the data transfer rate is provided by a comparison of the relative position of an input and an output pointer in a queue of frames that have been selected for display.

13. A method according to claim 11, wherein said frame source includes means for storing pre-rendered image frames.

14. A method according to claim 11, wherein said frames are skipped in response to a prediction of a network data transfer rate.

15. A method according to claim 11, wherein frames are prefetched into a frame queue prior to their due time.

16. A method according to claim 11, wherein a frame skip rate is defined by a user.

17. A method according to claim 11, wherein a frame is selected for display by processing its due time with elapsed real time since playback started.

19. A method according to claim 18, wherein said frame timing parameter is the due time for a frame.

20. A method according to claim 18, wherein instructions for the processing means are executed as multiple threads.

18. A method for displaying image data on an image viewing station that comprises display means, processing means, memory means and network connecting means for enabling transfer of image data over a network from a frame source remotely connected to said network, wherein:

said image data comprises a plurality of image frames, sequences of said frames being organised into clips, each clip having a frame rate, and each frame in a clip thereby having a due time for display with respect to a start time for playing the clip;

said processing means is configured to perform operations to play a clip from said frame source by a method comprising;

(a) selecting a next frame for preloading by skipping at least one frame in the clip'"'"'s frame sequence;

(b) preloading a frame from said frame source into a frame queue in said memory means;

(c) displaying a preloaded frame at its due time;

(d) processing elapsed real time since the clip started playing with a frame timing parameter; and

(e) updating the number of frames to skip in response to said processing of elapsed real time.

21. A data structure upon a machine readable medium, comprising instructions for controlling an image viewing system to perform a method for viewing image data, said viewing system comprising:

display means, processing means and network connecting means for transferring frames of said image data over a network from a remotely connected frame source;

said image data comprising a plurality of image frames, and has a frame rate from which may be inferred a due time for display of each frame in a sequence of frames in said image data;

said frame source returns a frame in response to a frame request issued over said network;

wherein

said processing means being configurable by said instructions to play a clip in which said method includes;

displaying selected frames from said frame source, on said display means, at their due time; and

skipping frames in said frame sequence in response to an indication of the data transfer rate of said network.

22. A data structure according to claim 21, wherein said indication of the data transfer rate is provided by a comparison of the relative position of an input and an output pointer in a queue of frames that have been selected for display.

23. A data structure according to claim 21, wherein said frame source includes means f or storing pre-rendered image frames.

24. A data structure according to claim 21, wherein said frames are skipped in response to a prediction of a network data transfer rate.

25. A data structure according to claim 21, wherein frames are prefetched into a frame queue prior to their due time.

26. A data structure according to claim 21, wherein a frame skip rate is defined by a user.

27. A data structure according to claim 21, wherein a frame is selected for display by processing its due time with elapsed real time since playback started.

29. A data structure according to claim 28, wherein said frame timing parameter is the due time for a frame.

30. A data structure according to claim 28, wherein instructions for steps (a) to (e) will be executed as multiple threads.

28. A data structure upon a machine readable medium, comprising instructions for controlling an image viewing system to perform a method for viewing image data, said viewing system comprising:

display means, processing means, memory means and network connecting means for enabling transfer of image data over a network from a frame source remotely connected to said network, in which;

said image data comprises a plurality of image frames, sequences of said frames being organised into clips, each clip having a frame rate, and each frame in a clip thereby having a due time for display with respect to a start time for playing the clip;

wherein

said processing means is configured to perform operations to play a clip from said frame source by a method comprising;

(a) selecting a next frame for preloading by skipping at least one frame in the clip'"'"'s frame sequence;

(b) preloading a frame from said frame source into a frame queue in said memory means;

(c) displaying a preloaded frame at its due time;

(d) processing elapsed real time since the clip started playing with a frame timing parameter; and

(e) updating the number of frames to skip in response to said processing of elapsed real time.