Real-time receipt, decompression and play of compressed streaming video/hypervideo; with thumbnail display of past scenes and with replay, hyperlinking and/or recording permissively intiated retrospectively

• US 6,154,771 A
• Filed: 06/01/1998
• Issued: 11/28/2000
• Est. Priority Date: 06/01/1998
• Status: Expired due to Term

• Current Assignee: Tata America International Corporation (Tata Sons Pvt Ltd.)
• Original Assignee: Mediastra Incorporated
• Inventors: Arole, Parag, Ashwani, Arya, Shastri, Vijnan, Rangan, P. Venkat
• Primary Examiner(s): Maung, Zarni
• Assistant Examiner(s): Jaroenchonwanit, Bunjob
• Application Number:
  US09/088,513
• Time in Patent Office: 911 Days
• Field of Search: 345/327, 345/328, 345/302, 345/433, 345/335, 345/2, 345/10, 345/104, 386/109, 386/52, 709/217, 709/231, 709/203, 709/247, 709/226, 709/229, 709/305, 348/552, 707/501, 707/502, 707/513
• US Class Current: 709/217
• CPC Class Codes: G06F 16/748 Hypervideo linking data to content, e.g. by linking an URL to a video object in the context of video distribution systems H04N21/858
• CPC Class Codes: G06F 3/04842 Selection of displayed objects or displayed text elements G06F3/0482 takes precedence
• CPC Class Codes: G11B 27/10 Indexing; Addressing; Timing or synchronising; Measuring tape travel
• CPC Class Codes: H04N 21/426 Internal components of the client ; Characteristics thereof H04N5/44 takes precedence
• CPC Class Codes: H04N 21/4331 Caching operations, e.g. of an advertisement for later insertion during playback
• CPC Class Codes: H04N 21/4622 Retrieving content or additional data from different sources, e.g. from a broadcast channel and the Internet web site content organization and management for information retrieval from the Internet G06F16/958; transmission by internet of broadcast information H04H60/82; stock exchange data over packet-switching network H04L12/1804; push services including data channel over packet-switching network H04L12/1859
• CPC Class Codes: H04N 21/4725 using interactive regions of the image, e.g. hot spots details of information retrieval from the Internet by using URLs G06F16/955; processing chained hypermedia data for information retrieval G06F16/94
• CPC Class Codes: H04N 21/4782 Web browsing , e.g. WebTV
• CPC Class Codes: H04N 21/812 involving advertisement data advertising per se G06Q30/02
• CPC Class Codes: H04N 21/8193 dedicated tools, e.g. video decoder software or IPMP tool
• CPC Class Codes: H04N 21/858 Linking data to content, e.g. by linking an URL to a video object, by creating a hotspot
• CPC Class Codes: H04N 5/76 Television signal recording

Abstract

Streaming compressed digital hypervideo received upon a digital communications network is decoded (decompressed) and played in a client-computer-based "video on web VCR" software system. Scene changes, if not previously marked upstream, are automatically detected, and typically twenty-one past scenes are displayed as thumbnail images. Hyperlinks within the main video scene, and/or any thumbnail image, show as hotspots, with text annotations typically appearing upon a cursor "mouse over". All hyperlinks--as are provided and inserted by, inter alia, the upstream network service provider (the "ISP")--may be, and preferably are, full-custom dynamically-resolved to each subscriber/user/viewer ("SUV") upon volitional "click throughs" by the SUV, including retrospectively on past hypervideo scenes as appear within the thumbnail images. Hyperlinking permits (i) retrieving information and commercials, including streaming video/hypervideo, from any of local storage, a network (or Internet) service provider ("ISP"), a network content provider, and/or an advertiser network site, (ii) entering a contest of skill or a lottery of chance, (iii) gambling, (iv) buying (and less often, selling), (v) responding to a survey, and expressing an opinion, and/or (vi) sounding an alert.

Claims

1. A method of displaying hyperlinks in streaming digital hypervideo;

receiving streaming digital hypervideo containing both (i) hyperlinks and associated (ii) hotspot data;

extracting hotspot data from hypervideo data that is within the received hypervideo; and

displaying (i) the decompressed hypervideo with (ii) accompanying hotspots overlying in selected locations the displayed hypervideo in accordance with the extracted hotspot data;

wherein the location of hyperlinks within the streaming digital hypervideo may be visually discerned by a viewer of the displaying as the selected locations of the associated hotspots.

2. The method of displaying hyperlinks according to claim 1 wherein the receiving is of streaming digital hypervideo across a digital communications network.

3. The method of displaying hyperlinks according to claim 1

wherein the receiving is of compressed streaming digital hypervideo;

and wherein the method further comprises;

decompressing the received hypervideo;

wherein the extracting of hotspot data is from the decompressed received hypervideo.

4. The method of displaying hyperlinks according to claim 1 wherein the displaying of the hypervideo with accompanying hotspots comprises:

overlaying the hotspots for individual hypervideo frames, frame by frame, with a rendering filter in accordance with the hotspot data; and

displaying the hypervideo frame with the overlaid hotspot.

5. The method of displaying hyperlinks according to claim 4 wherein the overlaying of the hotspots of the individual hypervideo frames is with a rendering filter supplied with (i) shapes and (ii) sizes of the hotspots in accordance with the hotspot data.

6. A system for displaying hyperlinks in streaming digital hypervideo;

a receiver receiving compressed streaming digital hypervideo containing both (i) hyperlinks and associated (ii) hotspot data across a digital communications network;

a computer

decompressing the received hypervideo, and

extracting hotspot data from the decompressed hypervideo; and

a display displaying (i) the decompressed hypervideo with (ii) accompanying hotspots overlying in selected locations the displayed hypervideo in accordance with the extracted hotspot data;

wherein the location of hyperlinks within the streaming digital hypervideo may be discerned by a viewer of the display as the selected locations of the associated hotspots.

7. The system for displaying hyperlinks according to claim 6 further comprising:

a rendering filter overlaying the hotspots for individual hypervideo frames frame by frame in accordance with the extracted hotspot data;

wherein the display displays the hypervideo fame with the overlaid hotspot.

8. The method of displaying hyperlinks according to claim 7 wherein the rendering filter is supplied by the computer with (i) shapes and (ii) sizes of the hotspots in accordance with the extracted hotspot data.

9. A method of parsing and displaying streaming digital hypervideo so that not only may hyperlinks associated with selected frames of the hypervideo be transitorily seen by a subscriber/user/viewer during a video display as the selected frames are viewed, but so that, regardless that the streaming digital hypervideo unfolds in real time and the selected frames recede into the past, the hyperlinks may be seen and held available sufficiently long so as to reasonably be acted upon by the subscriber/user/viewer, the method comprising:

detecting as the streaming digital hypervideo unfolds scene changes in the streaming digital hypervideo;

generating a multiplicity of static keyframes each of which keyframes is indicative of the hypervideo that is within a corresponding one of a multiplicity of successive scenes of the hypervideo as the hypervideo unfolds, in each which keyframe any hyperlink of the hypervideo scene is preserved;

caching in a digital data cache recently unfolded digital hypervideo and any ones of the multiplicity of keyframes corresponding thereto;

displaying from the cache both (i) the streaming digital hypervideo where both the hypervideo and its display are innately transitory, and also (ii) those keyframes, part of the multiplicity ultimately generated, that are indicative of the streaming digital hypervideo most recently previously displayed;

wherein the displayed keyframes are not only static, but are not so quickly transitory as are the scenes of the streaming hypervideo with which the keyframes are associated, the static keyframes rather being displayed much longer than are the hypervideo scenes with which the keyframes are associated;

wherein a subscriber/user/viewer of the displaying may click through a hyperlink that is upon a displayed keyframe, and therein branch the receiving or the playing of digital hypervideo or both the receiving and the playing, even though the exact scene in which the hyperlink was present has already been displayed in real time and is now in the past;

wherein the subscriber/user/viewer need not pounce upon a hyperlink as it moves spatially rapidly or temporally transitorily, or both spatially rapidly and temporally transitorily, in the real time display of the received streaming digital hypervideo, but can click through upon a hyperlink within a longer-displayed static keyframe as and when desired.

10. The real-time method of parsing and displaying streaming digital hypervideo according to claim 9

wherein the receiving of streaming digital hypervideo is across a digital communications network.

11. The real-time method of parsing and displaying streaming digital hypervideo according to claim 9

wherein the receiving is of compressed streaming digital hypervideo;

and wherein the method further comprises before the generating;

decompressing the received compressed streaming digital hypervideo.

12. The real-time method of parsing and displaying streaming digital hypervideo according to claim 11

wherein the receiving is of MPEG-compressed streaming digital hypervideo.

13. The real-time method of parsing and displaying streaming digital hypervideo according to claim 11

wherein the detecting of scene changes is in respect of the bounding frames that are within the compressed streaming digital hypervideo.

14. The real-time method of parsing and displaying streaming digital hypervideo according to claim 9

wherein the generating is of a multiplicity of key frames each of which is a reduced image of a first frame that is within a corresponding multiplicity of successive scenes.

15. The real-time method of parsing and displaying streaming digital hypervideo according to claim 9

wherein the caching is of a predetermined number of keyframes and of the hypervideo corresponding thereto and associated therewith.

16. The real-time method of parsing and displaying streaming digital hypervideo according to claim 15

wherein the oldest cached keyframes are discarded in favor of newest cached keyframes in a first-in, first-out cache maintenance scheme.

17. The real-time method of parsing and displaying streaming digital hypervideo according to claim 15

wherein the caching is of from 3 to 42 keyframes.

18. The real-time method of parsing and displaying streaming digital hypervideo according to claim 9

wherein the displaying is of (i) the received streaming digital hypervideo substantially in normal time, proximately to (ii) the plurality of keyframes, in order to facilitate visual association between the (i) hypervideo and the (ii) keyframes by the subscriber/user/viewer.

19. A system for parsing and displaying streaming digital hypervideo so that not only may hyperlinks embedded therein be seen by a subscriber/user/viewer, but so that, regardless that the hypervideo unfolds in real time, the hyperlinks may be seen and held available sufficiently long so as to reasonably be acted upon by the subscriber/user/viewer, the system comprising:

a detector of scene changes occurring in the streaming digital hypervideo as the hypervideo unfolds;

a generator using the unfolding received streaming hypervideo and the detected scene changes to generate a multiplicity of keyframes each of which keyframes is associated with a corresponding one of a multiplicity of successive scenes as are defined by the detected scenes changes, in each which keyframe any hypervideo link of the associated hypervideo scene is preserved;

a digital data cache in which recent digital hypervideo and the keyframes corresponding thereto are cached; and

a display, obtaining recent digital hypervideo and corresponding keyframes from the cache, displaying both (i) the streaming digital hypervideo substantially in real time, and, in non-real time over a protracted period, (ii) a plurality of keyframes that are indicative of scenes of the streaming digital hypervideo most recently previously displayed;

wherein a subscriber/user/viewer of the display may click through a hyperlink that is upon a displayed keyframe, and therein branch the display even though the exact scene in which the hyperlink was present has already been displayed in normal time;

wherein the subscriber/user/viewer need not pounce upon a hyperlink as it moves spatially rapidly or temporally transitorily, or both spatially rapidly and temporally transitorily, within the streaming hypervideo during the real-time unfolding display of the streaming digital hypervideo, but can click through upon a hyperlink within a non-real-time protractedly displayed static keyframe as and when desired.

20. A method of parsing and displaying streaming digital hypervideo so that not only may hyperlinks embedded therein be transitorily seen by a subscriber/user/viewer during an unfolding video display, but so that, regardless that the hypervideo unfolds in real time, the hyperlinks may be seen and held available sufficiently long so as to reasonably be acted upon by the subscriber/user/viewer, the method comprising:

receiving streaming digital hypervideo in which hypervideo are contained marks defining scene changes in the streaming digital hypervideo;

generating from the received hypervideo and contained marks a multiplicity of keyframes each of which keyframes is indicative of the hypervideo that is within a corresponding one of a multiplicity of successive scenes as are defined by the scene change marks, in each which keyframe any hypervideo link of the hypervideo scene is preserved;

caching in a digital data cache recently received digital hypervideo and the multiplicity of keyframes corresponding thereto;

displaying both (i) the received streaming digital hypervideo substantially in real time, and, protractedly in non-real time, (ii) a plurality of keyframes, part of the multiplicity ultimately generated, that are indicative of the received streaming digital hypervideo most recently previously displayed;

wherein a subscriber/user/viewer of the displaying may click through a hyperlink that is upon a displayed keyframe, and therein branch the receiving or the playing of digital hypervideo or both the receiving and the playing, even though the exact scene in which the hyperlink was present has already been displayed in normal time;

wherein the subscriber/user/viewer need not pounce upon a hyperlink as it moves spatially rapidly or temporally transitorily, or both spatially rapidly and temporally transitorily, in the normal time display of the received streaming digital hypervideo, but can click through upon a hyperlink within a displayed keyframe as and when desired.

21. A computerized method of receiving and displaying streaming video data while caching, indexing and displaying thumb-nails of the same streaming video data to the purpose that a viewer of the streaming video data may arbitrarily regress and re-regress in time as desired, commencing and re-commencing and re-re-commencing the displaying of the video data from any arbitrary past time or times for howsoever many times as are desired, the infinitely flexible video instant replay method comprising:

receiving streaming digital video data over time;

first-displaying the received streaming digital video data substantially in real time;

detecting scene changes in the streaming digital video data;

caching in a digital data cache a multiplicity of scenes, as are defined by the detected scenes changes, of the received digital streaming video, oldest cached digital video scenes being discarded in favor of newest cached digital video scenes in a first-in, first-out cache maintenance scheme;

second-displaying as a multiplicity of still thumbnail images the multiplicity of scenes, oldest thumbnail images being discarded in favor of newest thumbnail images in a first-displayed, first-discarded scheme; and

,

in response to a viewer selection of some particular second-displayed thumbnail image, re-commencing the first-displaying from the time of this viewer-selected one of the multiplicity of thumbnail images;

wherein the viewer may arbitrarily regress and re-regress in time as desired, commencing and re-commencing and re-re-commencing the displaying of the digital streaming video data from any arbitrary past time or times for howsoever many times as are desired;

wherein the viewer is accorded infinitely flexible video instant replay.

22. A digital video data cache indexing and maintenance method in support of viewer-controlled instant video replay, the method comprising:

receiving digital streaming video data continuously over time substantially in real time;

first-displaying the received digital streaming video data substantially in real time as and when received;

detecting scenes transitions in the received digital streaming video data;

caching in a digital data cache a multiplicity of scenes, as are defined by the detected scenes transitions, of the received digital streaming video, oldest cached digital video scenes being discarded in favor of newest cached digital video scenes in a first-in, first-out cache maintenance scheme;

second-displaying as a multiplicity of still images the multiplicity of scenes, oldest images being discarded in favor of newest images in a first-displayed, first-discarded scheme; and

,

in response to a present selection of a viewer of some second-displayed scene image, re-initializing the first-displaying to commence from the time of this selected scene image, the receiving and the detecting and the caching continuing apace only that these operations will henceforth be dealing with digital video data that is in the future to that digital video data that is first-displayed.

23. A method of displaying and using digital streaming video data containing video hyperlinks, called interactive video, where, nonetheless to inexorable progress in displaying the video, the video hyperlinks are presented to a viewer user long enough, and slow enough, so that the user viewer can click on a hyperlink without tracking its occasionally rapid movement, and its occasionally transit appearance, on a display screen, the method of presenting streaming interactive video so that hyperlinks contained therein may reasonably be acted upon by mere humans comprising:

first-displaying streaming digital video data containing video hyperlinks, thus interactive video, substantially in real time;

detecting scenes transitions in the digital streaming video data;

caching in a digital data cache a multiplicity of scenes, as are defined by the detected scenes transitions, of the received digital streaming video, the hyperlinks being preserved in the cache;

second-displaying as a multiplicity of still images the multiplicity of scenes, oldest images being discarded in favor of newest images in a first-displayed, first-discarded scheme; and

,

responding to a click on hyperlink in a still image by a user viewer to branch the first-displaying identically that this first displaying would have been branched should the subscriber/user/viewer have been so fast, and so nimble, so as to click on the same video hyperlink in the first-displayed image;

wherein the subscriber/user/viewer is accorded a reasonable time to think and to act in clicking on hyperlinks in the second display, and need not remain poised to spring upon a hyperlink during its potentially rapidly moving, and its potentially transitory, first display.