System and method for video navigation and client side indexing

• US 7,401,351 B2
• Filed: 12/14/2000
• Issued: 07/15/2008
• Est. Priority Date: 12/14/2000
• Status: Expired due to Fees

• Current Assignee: Fuji Xerox Company Limited (Xerox Holdings Corp.)
• Original Assignee: Fuji Xerox Company Limited (Xerox Holdings Corp.)
• Inventors: Girgensohn, Andreas, Wilcox, Lynn, Schilit, William N., Boreczky, John
• Primary Examiner(s): Lonsberry; Hunter B.
• Application Number:
  US09/738,905
• Publication Number:
  US 20020075572A1
• Time in Patent Office: 2,770 Days
• Field of Search: 725 86- 95, 345/721, 345723-725
• US Class Current: 725/88
• CPC Class Codes: H04N 21/234363 by altering the spatial resolution, e.g. for clients with a lower screen resolution
• CPC Class Codes: H04N 21/234381 by altering the temporal resolution, e.g. decreasing the frame rate by frame skipping
• CPC Class Codes: H04N 21/431 Generation of visual interfaces for content selection or interaction; Content or additional data rendering
• CPC Class Codes: H04N 21/6587 Control parameters, e.g. trick play commands, viewpoint selection
• CPC Class Codes: H04N 21/8455 involving pointers to the content, e.g. pointers to the I-frames of the video stream
• CPC Class Codes: H04N 7/17336 Handling of requests in head-ends

Abstract

A system is provided where a number of representative video snapshots are displayed on a timeline indicating a position to jump to in a video using control buttons, such as fast forward, reverse, or a timeline scroll bar. The video snapshots are obtained by opening a low resolution connection to a video server to receive forward looking video segments. The forward looking video segments are summarized and the representative video snapshots are selected and displayed to provide a reference frame for the user to select a timeline position, or snapshots directly linked to positions in the higher resolution video. Backward looking video segments may be similarly summarized from the low resolution connection, or alternatively from video previously played. As the video progresses, new forward looking video segments are received and new representative video snapshots are summarized and displayed in close proximity with the video being played.

Claims

1. A method for providing on-the-fly client-side indexing and navigation of video data, comprising the steps of:

opening a main connection for a client-side device to receive transmissions of a data flow, wherein said data flow is not indexed;

opening a second connection for the client-side device to receive at least one look-x data stream comprising a plurality of data from said data flow, wherein said plurality of data is not indexed such that no frames in said data flow have been associated with any points in the look-x data stream;

indexing with the client-side device at least one point of the look-x data stream to at least one corresponding point in said data flow, wherein said indexing step with the client-side device further comprises determining a particular timeframe in said data flow and selecting on-the-fly at least one look-x point for display to represent the at least one corresponding point in said data flow at said particular timeframe and wherein selecting on-the-fly step includes automatically and without user intervention associating the look-x point with the point in said data flow while said data flow and said look-x data stream are being received at the client-side device; and

providing control of a playback position of said data flow based on the indexed points in the look-x data stream.

2. The method of claim 1, further comprising the step of:

displaying a timeline corresponding to the indexed look-x points, the timeline having at least one of said indexed look-x points displayed so as to reference a position on said timeline.

3. The method of claim 1, wherein said step of providing control includes the step of:

displaying at least one of a skip forward and a skip back button configured to step a play position of said data flow to a position corresponding to a respective one of a next and a previous of said look-x data points relative to the current play position of said data flow.

4. The method of claim 1, further comprising the steps of:

displaying a timeline having representations of the indexed points;

selecting at least one of the indexed points; and

displaying said data flow at a point beginning with the selected indexed point.

5. The method of claim 1, wherein said data flow is a video and said look-x points are frames of said data flow retrieved from one of said main connection and said second connection.

6. The method of claim 1, wherein said second connection is a low resolution connection relative to the main connection.

7. The method of claim 1, further comprising the step of:

selecting a predetermined number of said indexed look-x points;

displaying the predetermined number of indexed points to provide reference for a playback control mechanism; and

updating the selected predetermined number of indexed look-x points based on an update criteria.

8. The method of claim 7, wherein said step of selecting includes the step of:

selecting said predetermined number of look-x points such that each of the look-x points is within a predetermined distance of a first play position of said data flow.

9. The method of claim 7, wherein said update criteria comprises a change of the playback position a predetermined amount from the first play position during the selection step.

19. The method of claim 1, further comprising:

opening a third connection for a client-side device to receive information before said playback position in said data flow.

20. The method of claim 1, further comprising:

reaching a segment boundary in said data flow; and

updating the control of said playback position in said data flow upon reaching said segment boundary.

10. A device for client-side video indexing, comprising:

a video player comprising;

a client-side main data stream connection for receiving transmissions of a non-indexed data flow;

a client-side look-x data stream connection for receiving at least one non-indexed look-x data transmission of the data flow wherein no frames in said data flow have been associated with any points in the look-x data stream;

a client-side controller adapted to index on-the-fly at least one look-x point of the look-x data stream to a corresponding at least one point in said data flow by summarizing the look-x data stream and generating for display the at least one look-x point to the corresponding at least one point in said data flow wherein indexing on-the-fly step includes automatically, and without requiring user input, associating the look-x point with the point in said data flow while said data flow and said look-x data stream are being received at the client-side device; and

a display for displaying at least one of the indexed look-x points.

11. The device of claim 10, wherein the video player further comprises:

a skip forward button and a skip back button that each step a play position of said data stream to a respective one of a next and a previous of said look-x points relative to the current play position of said data stream.

12. The device of claim 10, wherein the display further comprises display of a timeline referenced to the data flow and at least one of said indexed look-x points, the indexed look-x points each displayed so as to reference a position on said timeline.

13. The device of claim 10, wherein the video player device further comprises:

a select button for allowing a user to select at least one of the indexed look-x points enabling display of the data flow to begin at the selected indexed point.

14. The device of claim 10, wherein said look-x data stream connection is a low resolution data stream relative to the main data stream connection.

15. A method for client-side navigating and indexing of video data, comprising the steps of:

opening a client-side main connection for receiving a video stream without an existing index;

opening a client-side second connection for receiving transmission of at least one look-x data stream without an existing index, such that no frames in said video stream have been associated with any points in the look-x data stream;

generating on-the-fly on the client-side an index, the index comprising at least one look-x data point and relating said at least one look-x data point to at least one point corresponding in time within said data flow wherein the generating on-the-fly step includes automatically, and without requiring user intervention, associating the look-x data point with the point in time within said data flow while said data flow and said look-x data stream are being received at the client-side; and

providing control of a playback position of said data flow based on at least one user-selected look-x data point corresponding to at least one of the indexed points in the look-x data stream.

16. A method for indexing and navigating a video stream, the method comprising:

opening a client-side connection to receive a main video stream;

opening at least another client-side connection to receive at least one look-x data stream of the main video stream such that no frames in said video stream have been associated with any points in the look-x data stream;

generating at least one keyframe at the client side that references at least one corresponding timeframe point in the main video stream, wherein the generating step results in an index created on-the-fly on the client side and wherein the generating step includes automatically, and without requiring user input, associating the timeframe point with the keyframe while said data flow and said look-x data stream are being received at the client-side;

displaying the at least one keyframe to a user;

providing control of a playback position of said main video stream based on the at least one keyframe that references the at least one corresponding point in the main video stream; and

updating the generating and displaying steps to keep pace with a general speed of playback of the main video stream.

17. The method as in claim 16 wherein the updating step is performed in at least one of real time and continuously.

18. The method as in claim 16 further comprising using the look-x data stream to feed playback positions continuously at pre-determined intervals forward and reverse of a current playback position of the main video stream;

and the generating step further comprises generating low resolution moving snaps near the current playback position of the main video stream.