Method and apparatus for hyperlinking in a television broadcast

• US 7,367,042 B1
• Filed: 10/20/2000
• Issued: 04/29/2008
• Est. Priority Date: 02/29/2000
• Status: Expired due to Fees

• Current Assignee: GoldPocket Interactive, Inc. (Telefonaktiebolaget LM Ericsson)
• Original Assignee: GoldPocket Interactive, Inc. (Telefonaktiebolaget LM Ericsson)
• Inventors: Dakss, Jon, Miller, Andrew, Sarachik, Karen, Milazzo, Paul, Bove, V. Michael Jr., Katcher, Dan, Wachman, Josh
• Primary Examiner(s): Koenig; Andrew Y
• Application Number:
  US09/694,079
• Time in Patent Office: 2,748 Days
• Field of Search: 725/60, 725/109, 725/110, 725/136, 725/112
• US Class Current: 725/60
• CPC Class Codes: G06F 16/748 Hypervideo linking data to content, e.g. by linking an URL to a video object in the context of video distribution systems H04N21/858
• CPC Class Codes: G06F 16/954 Navigation, e.g. using categorised browsing
• CPC Class Codes: G06F 16/9558 Details of hyperlinks; Management of linked annotations
• CPC Class Codes: H04N 1/64 Systems for the transmission or the storage of the colour picture signal; Details therefor, e.g. coding or decoding means therefor H04N19/00 takes precedence
• CPC Class Codes: H04N 19/93 Run-length coding
• CPC Class Codes: H04N 21/234318 by decomposing into objects, e.g. MPEG-4 objects
• CPC Class Codes: H04N 21/235 Processing of additional data, e.g. scrambling of additional data or processing content descriptors
• CPC Class Codes: H04N 21/435 Processing of additional data, e.g. decrypting of additional data, reconstructing software from modules extracted from the transport stream
• CPC Class Codes: H04N 21/4622 Retrieving content or additional data from different sources, e.g. from a broadcast channel and the Internet web site content organization and management for information retrieval from the Internet G06F16/958; transmission by internet of broadcast information H04H60/82; stock exchange data over packet-switching network H04L12/1804; push services including data channel over packet-switching network H04L12/1859
• CPC Class Codes: H04N 21/4722 for requesting additional data associated with the content
• CPC Class Codes: H04N 21/4725 using interactive regions of the image, e.g. hot spots details of information retrieval from the Internet by using URLs G06F16/955; processing chained hypermedia data for information retrieval G06F16/94
• CPC Class Codes: H04N 21/4728 for selecting a Region Of Interest [ROI], e.g. for requesting a higher resolution version of a selected region
• CPC Class Codes: H04N 21/4782 Web browsing , e.g. WebTV
• CPC Class Codes: H04N 21/812 involving advertisement data advertising per se G06Q30/02
• CPC Class Codes: H04N 21/854 Content authoring
• CPC Class Codes: H04N 21/8543 using a description language, e.g. Multimedia and Hypermedia information coding Expert Group [MHEG], eXtensible Markup Language [XML] information retrieval of semistructured data, the underlying structure being taken into account, e.g. mark-up language structure data G06F16/80
• CPC Class Codes: H04N 21/858 Linking data to content, e.g. by linking an URL to a video object, by creating a hotspot
• CPC Class Codes: H04N 21/8583 by creating hot-spots
• CPC Class Codes: H04N 21/8586 by using a URL processing chained hypermedia data for information retrieval G06F16/94; information retrieval from the Internet by using URLs G06F16/955; URL in broadcast information H04H20/93; Web-based protocols H04L67/02

Abstract

A system and method of adding hyperlinked information to a television broadcast. The broadcast material is analyzed and one or more regions within a frame are identified. Additional information can be associated with a region, and can be transmitted in encoded form, using timing information to identify the frame with which the information is associated. The system comprising a video source and an encoder that produces a transport stream in communication with the video source, an annotation source, a data packet stream generator that produces encoded annotation data packets in communication with the annotation source and the encoder, and a multiplexer system in communication with the encoder and the data packet stream generator. The encoder provides timestamp information to the data packet stream generator and the data packet stream generator synchronizes annotation data from the annotation source with a video signal from the video source in response to the timestamp information. The multiplexer generates a digital broadcast signal that includes an augmented transport stream from the transport stream and the encoded data packets. A receiver displays the annotation information associated with the video signal in response to a viewer request on a frame by frame basis. A viewer can respond interactively to the material, including performing commercial transactions, by using a backchannel that is provided for interactive communication.

Claims

1. A hyperlinked reception system comprising:

a receiver in communication with a broadcast channel, the receiver decoding a digital broadcast signal transmitted over the broadcast channel to recover a video signal, a plurality of masks, and a plurality of object data packets included in the broadcast signal, each mask corresponding to a particular video frame of a video program and including graphics data associated with one or more video objects in the particular video frame, each mask including an identifier to an object mapping table included in at least a particular one of the plurality of object data packets, the object mapping table including an entry associated with each of the one or more video objects in the particular video frame, each entry in the object mapping table referencing one or more information data structures included in one or more of the plurality of object data packets, the information data structures including information associated with the corresponding video object; and

a display device in communication with said receiver, the display device overlaying one or more graphics images on a particular video frame based on the graphics data in the mask for the particular video frame, and

a processor coupled to the receiver, the processor being configured to;

receive a user selection associated with one of the graphics images overlaid on the particular video frame;

retrieve the identifier of the object mapping table from the mask corresponding to the particular video frame responsive to the user selection;

retrieve the object mapping table based on the retrieved identifier;

locate the entry in the object mapping table for the video object associated with the one of the graphics images;

identify the one or more information data structures referenced in the located entry;

retrieve information in the identified one or more information data structures; and

display the retrieved information on the display device.

2. The system of claim 1, wherein said information data structures include information regarding goods and services for sale.

3. The system of claim 1, wherein said information data structures include non-commercial information.

4. The system of claim 1 wherein said information data structures include at least one of textual data and graphics data.

5. The system of claim 4 wherein each of the masks includes location information of the one or more video objects in the corresponding video frame.

6. The system of claim 5 wherein said location information includes a graphics location reference that represents a fixed relation to a set of pixels associated with said one or more video objects.

7. The system of claim 6 wherein said graphics location reference includes an upper left most pixel in said associated pixel set.

8. The system of claim 6 wherein said graphics location reference includes a centroid pixel of said associated pixel set.

9. The system of claim 4 wherein each of the masks includes location and shape information of the one or more video objects in the corresponding video frame.

10. The system of claim 9 wherein said shape information is represented by the overlaid one or more graphics images of said one or more video objects.

11. The system of claim 9 wherein said shape information is represented by an outline of said one or more video objects.

12. The system of claim 9 wherein said shape information is represented by a mathematical representation of a set of pixels associated with said one or more video objects.

13. The system of claim 1, further comprising:

means for transmitting information related to a viewer request of the one or more information data structures to a remote location over a backchannel communications channel.

14. The system of claim 1, further comprising:

means for accepting from a viewer information regarding a commercial transaction, and completing said transaction.

15. The system of claim 14, wherein the information regarding a commercial transaction accepted from a viewer comprises at least one of a viewer'"'"'s identifier, a viewers address, an identifier of a viewer'"'"'s financial account, a viewer'"'"'s password, a viewer'"'"'s personal identification number, a quantity, a model, a size, a color, a descriptor of a good or service to be vended, and a price.

16. The system of claim 1, wherein two or more of the plurality of object data packets have equal timing information, and the processor processes a viewer request including an indication as to which of the two or more of the plurality of object data packets is to be processed for displaying information contained therein.

17. The system of claim 1, wherein the mask further includes timing data, and the display device draws the one or more graphics images on a frame-by-frame basis based on the graphics data in the mask for the corresponding video frame, the drawing of the one or more graphics images being synchronized to the corresponding video frame based on the timing data in the corresponding mask.

18. The system of claim 17 wherein said timing data comprises one of timestamp information, timecode information, frame numbering information and global time of day.

19. The system of claim 1, wherein each of the plurality of masks is compressed based on a compression algorithm.

20. A hyperlinked broadcast system comprising:

a video source providing a video program having a plurality of video frames;

a video encoder in communication with said video source, said video encoder producing a transport stream for transporting the video program;

an authoring tool providing mask data and object data for the video program;

a data packet stream generator in communication with said authoring tool and said video encoder, said data packet stream generator producing encoded data packets including a plurality of masks and a plurality of object data packets, each mask corresponding to a particular video frame of the video program and including graphics data associated with one or more video objects in the particular video frame, each mask including an identifier to an object mapping table included in at least a particular one of the plurality of object data packets, the object mapping table including an entry associated with each of the one or more video objects in the particular video frame, each entry in the object mapping table referencing one or more information data structures included in one or more of the plurality of object data packets, the information data structures including information associated with the corresponding video object;

a multiplexer system in communication with said video encoder and said data packet stream generator, said multiplexer generating a digital broadcast signal comprising an augmented transport stream from said transport stream and said encoded data packets; and

means for transmitting the digital broadcast signal to a receiver.

21. The system of claim 20 wherein said information data structures include at least one of textual data and graphics data.

22. The system of claim 20 wherein each of the masks includes location information of the one or more video objects in the corresponding video frame.

26. The system of claim 20 wherein each of the masks include location and shape information of the one or more video objects in the corresponding video frame.

27. The system of claim 26 wherein said shape information is represented by the overlaid one or more graphics images of said one or more video objects.

28. The system of claim 26 wherein said shape information is represented by an outline of said one or more video objects.

29. The system of claim 26 wherein said shape information is represented by a mathematical representation of a set pixels associated with said one or more video objects.

30. The system of claim 20, wherein said video encoder provides timing information for each of the plurality of video frames of the video program provided by the video source to said data packet stream generator, and said data packet stream generator steps through the plurality of video frames and associates the timing information of each video frame to a corresponding mask.

31. The system of claim 30 wherein said timing information comprises one of timestamp information, timecode information, frame numbering information and global time of day.

32. The system of claim 20, wherein each of the plurality of masks is compressed based on a compression algorithm.

33. The system of claim 20, wherein the receiver is configured to:

decode the digital broadcast signal to recover the plurality of masks and the plurality of object data packets;

overlay one or more graphics images on a particular video frame based on the graphics data in the mask for the particular video frame;

receive a user selection associated with one of the graphics images overlaid on the particular video frame;

retrieve the identifier of the object mapping table from the mask corresponding to the particular video frame responsive to the user selection;

retrieve the object mapping table based on the retrieved identifier;

locate the entry in the object mapping table for the video object associated with the one of the graphics images;

identify the one or more information data structures referenced in the located entry;

retrieve information in the identified one or more information data structures; and

display the retrieved information on a display device.

34. The system of claim 33, wherein the mask further includes timing data, and the display device draws the one or more graphics images on a frame-by-frame basis based on the graphics data in the mask for the corresponding video frame, the drawing of the one or more graphics images being synchronized to the corresponding video frame based on the timing data in the corresponding mask.

23. The system of 22 wherein said location information includes a graphics location reference that represents a fixed relation to a set of pixels associated with said one or more video objects.

24. The system of claim 23 wherein said graphics location reference includes an upper left most pixel in said associated pixel set.

25. The system of claim 23 wherein said graphics location reference includes a centroid pixel of said associated pixel set.

35. A hyperlinked reception method comprising:

decoding a digital broadcast signal transmitted over the broadcast channel to recover a video signal, a plurality of masks, and a plurality of object data packets included in the broadcast signal, each mask corresponding to a particular video frame of a video program and including graphics data associated with one or more video objects in the particular video frame, each mask including an identifier to an object mapping table included in at least a particular one of the plurality of object data packets, the object mapping table including an entry associated with each of the one or more video objects in the particular video frame, each entry in the object mapping table referencing one or more information data structures included in one or more of the plurality of object data packets, the information data structures including information associated with the corresponding video object;

overlaying one or more graphics images on a particular video frame based on the graphics data in the mask for the particular video frame;

receiving a user selection associated with one of the graphics images overlaid on the particular video frame;

retrieving the identifier of the object mapping table from the mask corresponding to the particular video frame responsive to the user selection;

retrieving the object mapping table based on the retrieved identifier;

locating the entry in the object mapping table for the video object associated with the one of the graphics images;

identifying the one or more information data structures referenced in the located entry;

retrieving information in the identified one or more information data structures; and

displaying the retrieved information on a display device.

36. The method of claim 35, wherein the mask further includes timing data, the method further comprising:

drawing the one or more graphics images on a frame-by-frame basis based on the graphics data in the mask for the corresponding video frame, the drawing of the one or more graphics images being synchronized to the corresponding video frame based on the timing data in the corresponding mask.

37. The method of claim 35, wherein each of the plurality of masks is compressed based on a compression algorithm.

38. A hyperlinked broadcast method comprising:

providing a video program having a plurality of video frames;

generating, a transport stream for transporting the video program;

generating mask data and annotation data for the video program;

generating encoded data packets including a plurality of masks and a plurality of object data packets, each mask corresponding to a particular video frame of the video program and including graphics data associated with one or more video objects in the particular video frame, each mask including an identifier to an object mapping table included in at least a particular one of the plurality of object data packets, the object mapping table including an entry associated with each of the one or more video objects in the particular video frame, each entry in the object mapping table referencing one or more information data structures included in one or more of the plurality of object data packets, the information data structures including information associated with the corresponding video object;

generating a digital broadcast signal comprising an augmented transport stream from said transport stream and said encoded data packets; and

transmitting the digital broadcast signal to a receiver.

39. The method of claim 38 further comprising:

providing timing information for each of the plurality of video frames of the video program; and

stepping through the plurality of video frames and associating the timing information of each video frame to a corresponding mask.

40. The method of claim 38 further comprising:

compressing each of the plurality of masks based on a compression algorithm.

41. A hyperlinked reception system comprising:

a receiver receiving over one or more television broadcast signals a plurality of object data packets and a video program having a plurality of video frames, the object data packets including one or more object mapping tables including an entry associated with each of one or more video objects in a particular video frame, each entry in the object mapping table referencing one or more information data structures included in one or more of the plurality of object data packets, the information data structures including information associated with the corresponding video object; and

a display device in communication with said receiver, the display device overlaying one or more graphics images on a particular video frame based on associated mask data, the mask data including an identifier to a corresponding one of the one or more object mapping tables; and

a processor coupled to the receiver, the processor being configured to;

receive a user selection associated with one of the graphics images overlaid on the particular video frame;

retrieve the identifier of the object mapping table from the mask data corresponding to the user selection;

retrieve the object mapping table based on the retrieved identifier;

locate the entry in the object mapping table for the video object associated with the one of the graphics images;

identify the one or more information data structures referenced in the located entry;

retrieve information in the identified one or more information data structures; and

display the retrieved information on the display device.