Method and apparatus for switching between multiple programs by interacting with a hyperlinked television broadcast

• US 20020049983A1
• Filed: 12/22/2000
• Published: 04/25/2002
• Est. Priority Date: 02/29/2000
• Status: Active Grant

• Current Assignee: MK Systems USA Inc.
• Original Assignee: GoldPocket Interactive, Inc. (Telefonaktiebolaget LM Ericsson)
• Inventors: Dakss, Jon, Milazzo, Paul, Bove, V. Michael Jr., Katcher, Dan
• Granted Patent: US 7,249,367 B2
• Time in Patent Office: Days
• US Class Current: 725/135
• CPC Class Codes: G06F 16/748 Hypervideo linking data to content, e.g. by linking an URL to a video object in the context of video distribution systems H04N21/858
• CPC Class Codes: G06F 16/954 Navigation, e.g. using categorised browsing
• CPC Class Codes: G06F 16/9558 Details of hyperlinks; Management of linked annotations
• CPC Class Codes: H04N 1/64 Systems for the transmission or the storage of the colour picture signal; Details therefor, e.g. coding or decoding means therefor H04N19/00 takes precedence
• CPC Class Codes: H04N 19/93 Run-length coding
• CPC Class Codes: H04N 21/234318 by decomposing into objects, e.g. MPEG-4 objects
• CPC Class Codes: H04N 21/235 Processing of additional data, e.g. scrambling of additional data or processing content descriptors
• CPC Class Codes: H04N 21/435 Processing of additional data, e.g. decrypting of additional data, reconstructing software from modules extracted from the transport stream
• CPC Class Codes: H04N 21/4622 Retrieving content or additional data from different sources, e.g. from a broadcast channel and the Internet web site content organization and management for information retrieval from the Internet G06F16/958; transmission by internet of broadcast information H04H60/82; stock exchange data over packet-switching network H04L12/1804; push services including data channel over packet-switching network H04L12/1859
• CPC Class Codes: H04N 21/4722 for requesting additional data associated with the content
• CPC Class Codes: H04N 21/4725 using interactive regions of the image, e.g. hot spots details of information retrieval from the Internet by using URLs G06F16/955; processing chained hypermedia data for information retrieval G06F16/94
• CPC Class Codes: H04N 21/4728 for selecting a Region Of Interest [ROI], e.g. for requesting a higher resolution version of a selected region
• CPC Class Codes: H04N 21/4782 Web browsing , e.g. WebTV
• CPC Class Codes: H04N 21/812 involving advertisement data advertising per se G06Q30/02
• CPC Class Codes: H04N 21/854 Content authoring
• CPC Class Codes: H04N 21/8543 using a description language, e.g. Multimedia and Hypermedia information coding Expert Group [MHEG], eXtensible Markup Language [XML] information retrieval of semistructured data, the underlying structure being taken into account, e.g. mark-up language structure data G06F16/80
• CPC Class Codes: H04N 21/858 Linking data to content, e.g. by linking an URL to a video object, by creating a hotspot
• CPC Class Codes: H04N 21/8583 by creating hot-spots
• CPC Class Codes: H04N 21/8586 by using a URL processing chained hypermedia data for information retrieval G06F16/94; information retrieval from the Internet by using URLs G06F16/955; URL in broadcast information H04H20/93; Web-based protocols H04L67/02

Abstract

A system and method of adding hyperlinked information to a television broadcast. The broadcast material is analyzed and one or more regions within a frame are identified. Additional information can be associated with a region, and can be transmitted in encoded form, using timing information to identify the frame with which the information is associated. The system comprising a video source and an encoder that produces a transport stream in communication with the video source, an annotation source, a data packet stream generator that produces encoded annotation data packets in communication with the annotation source and the encoder, and a multiplexer system in communication with the encoder and the data packet stream generator. The encoder provides timestamp information to the data packet stream generator and the data packet stream generator synchronizes annotation data from the annotation source with a video signal from the video source in response to the timestamp information. The multiplexer generates a digital broadcast signal that includes an augmented transport stream from the transport stream and the encoded data packets. A receiver displays the annotation information associated with the video signal in response to a viewer request on a frame by frame basis. A viewer can respond interactively to the material, including performing commercial transactions, by using a backchannel that is provided for interactive communication.

Claims

1. A hyperlinked broadcast system comprising:

a video source;

a video encoder in communication with said video source, said video encoder producing a transport stream;

an annotation source;

a data packet stream generator in communication with said annotation source and said video encoder, said data packet stream generator producing encoded data packets; and

a multiplexer system in communication with said video encoder and said data packet stream generator, said multiplexer generating a digital broadcast signal comprising an augmented transport stream from said transport stream and said encoded data packets,

wherein said video encoder provides timing information to said data packet stream generator and said data packet stream generator synchronizes annotation data from said annotation source with a video signal from said video source in response to said timing information.

2. The system of claim 1 wherein said timing information comprises one of timestamp information, timecode information, frame numbering information and global time of day.

3. The system of claim 1 wherein said annotation data comprises mask data and at least one of textual data and graphics data.

4. The system of claim 3 wherein said mask data includes location information of an object in an annotated video frame.

5. The system of claim 4 wherein said location information includes a graphics location reference that represents a fixed relation to a set of pixels associated with said object.

6. The system of claim 5 wherein said graphics location reference includes an upper left most pixel in said associated pixel set.

7. The system of claim 5 wherein said graphics location reference includes a centroid pixel of said associated pixel set.

8. The system of claim 3 wherein said mask data comprises location and shape information of an object in a video fame to be annotated.

9. The system of claim 8 wherein said shape information is represented by a graphical overlay of said object.

10. The system of claim 8 wherein said shape information is represented by an outline of said object.

11. The system of claim 8 wherein said shape information is represented by a mathematical representation of a set of pixels associated with said object.

13. The system of claim 12 wherein said timing information comprises one of timestamp information, timecode information, frame numbering information and global time of day.

14. The system of claim 12, wherein said annotation data comprises mask data and at least one of textual data and graphics data.

15. The system of claim 14, wherein said mask data and said at least one of textual data and graphics data are transmitted over separate streams in said broadcast channel.

16. The system of claim 14 wherein said mask data comprises location information of an object in an annotated video frame.

17. The system of claim 16 wherein said mask data further comprises shape information of an object in an annotated video frame.

18. The system of claim 12 wherein said timing information is employed to synchronize the display of annotation data from said annotation source with said video signal from said video source.

19. The system of claim 18 wherein said timing information comprises one of timestamp information, timecode information, frame numbering information and global time of day.

20. The system of claim 12 wherein said system transmits information related to said viewer request to a remote location over a backchannel communications channel.

21. The system of claim 20 wherein said backchannel communications channel is a store-and-forward channel.

22. The system of claim 12, wherein said annotation data comprises information regarding goods and services for sale.

23. The system of claim 12, wherein said annotation data comprises non-commercial information.

25. The system of claim 24 wherein timing information is employed to synchronize a display of said annotation data with said video signal.

26. The system of claim 25 wherein said timing information comprises one of timestamp information, timecode information, frame numbering information and global time of day.

27. The system of claim 25 wherein said receiver synchronizes said annotation data with said video signal in response to said timing information.

28. The system of claim 24 wherein said annotation data comprises a plurality of annotations having equal timing information, and said viewer request comprises an indication as to which annotation of said plurality of annotations is to be displayed.

30. The system of claim 29 wherein said timing information comprises one of timestamp information, timecode information, frame numbering information and global time of day.

12. A hyperlinked broadcast and reception system comprising:

a video source;

a video encoder in communication with said video source, said video encoder producing a transport stream;

an annotation source;

a data packet stream generator in communication with said annotation source and said video encoder, said data packet stream generator producing encoded data packets;

a multiplexer system in communication with said video encoder and said data packet stream generator, said multiplexer generating a digital broadcast signal comprising an augmented transport stream from said transport stream and said encoded data packets;

a broadcast channel in communication with said multiplexer system;

a receiver in communication with said broadcast channel; and

a display device in communication with said receiver,

wherein said video encoder provides timing information to said data packet stream generator and said data packet stream generator synchronizes annotation data from said annotation source with a video signal from said video source in response to said timing information, and

wherein said display device displays said annotation data associated with said video signal in response to a viewer request on a frame by frame basis.

24. A hyperlinked reception system comprising:

a receiver in communication with a broadcast channel; and

a display device in communication with said receiver, wherein said receiver decodes a digital broadcast signal to recover a video signal and annotation data; and

wherein, in response to a viewer request, said display device displays said annotation data associated with said video signal on a frame by frame basis.

29. A hyperlinked reception system comprising:

a receiver in communication with a broadcast channel; and

a display device in communication with said receiver, wherein said display device displays annotation data associated with a video signal, in response to a viewer request, on a frame by frame basis, said annotation data being associated with said video signal in response to timing information.

31. A method of generating a hyperlinked video signal, comprising:

providing a source video;

encoding said source video with a video encoder to produce a transport stream and timing information;

providing a source of annotation data;

communicating said transport stream and timing information from said video encoder and said annotation data from said source of annotation data to a data packet stream generator, said data packet stream generator synchronizing said annotation data with a video signal from said video source in response to said timing information and producing encoded data packets.

32. The method of claim 31 wherein said timing information comprises one of timestamp information, timecode information, frame numbering information and global time of day.

33. The method of claim 31, wherein said annotation data comprises information regarding goods and services for sale.

34. The method of claim 31, wherein said annotation data comprises non-commercial information.

35. The method of claim 31 wherein said annotation data comprises mask data and at least one of textual data and graphics data.

36. The method of claim 35 wherein said mask data includes location information of an object in an annotated video frame.

37. The method of claim 36 wherein said location information includes a graphics location reference that represents a fixed relation to a set of pixels associated with said object.

38. The method of claim 37 wherein said graphics location reference includes an upper left most pixel in said associated pixel set.

39. The method of claim 38 wherein said graphics location reference includes a centroid pixel of said associated pixel set.

40. The method of claim 35 wherein said mask data comprises location and shape information of an object in an annotated video frame.

41. The method of claim 40 wherein said shape information is represented by a graphical overlay of said object.

42. The method of claim 40 wherein said shape information is represented by an outline of said object.

43. The method of claim 40 wherein said shape information is represented by a mathematical representation of a set of pixels associated with said object.

44. The method of claim 31, further comprising:

communicating said transport stream and said encoded data packets to a multiplexer system, said multiplexer system generating a hyperlinked digital broadcast signal comprising an augmented transport stream from said transport stream and said encoded data packets.

45. The method of claim 44 wherein said annotation data comprises mask data and at least one of textual data and graphics data and wherein said mask data and said at least one of textual data and graphics data represent separate streams contained in said hyperlinked digital broadcast signal.

46. The method of claim 44, further comprising:

broadcasting said hyperlinked digital broadcast signal.

48. The method of claim 47 wherein said timing information comprises one of timestamp information, timecode information, frame numbering information and global time of day.

49. The method of claim 47, further comprising

transmitting information related to said viewer request to a remote location over a backchannel communications channel.

50. The method of claim 47, further comprising:

in response to a viewer prompt, accepting from a viewer information regarding a commercial transaction, and completing said transaction.

51. The method of claim 50, wherein the information regarding a commercial transaction accepted from a viewer comprises at least one of a viewer'"'"'s identifier, a viewer'"'"'s address, an identifier of a viewer'"'"'s financial account, a viewer'"'"'s password, a viewer'"'"'s personal identification number, a quantity, a model, a size, a color, a descriptor of a good or service to be vended, and a price.

47. A method of viewing a hyperlinked video signal, comprising:

receiving a hyperlinked video signal comprising an augmented transport stream;

demultiplexing said augmented transport stream to recover a transport stream and encoded data packets;

displaying said transport stream framewise as a video on a display;

decoding said data packets to recover annotation data and timing information;

synchronizing said annotation data with said video using said timing information; and

in response to a viewer request, displaying said annotation data on said display on a frame by frame basis.