Method and apparatus for generating data structures for a hyperlinked television broadcast

• US 7,117,517 B1
• Filed: 10/26/2000
• Issued: 10/03/2006
• Est. Priority Date: 02/29/2000
• Status: Expired due to Term

• Current Assignee: MK Systems USA Inc. (JP Morgan Chase & Co)
• Original Assignee: GoldPocket Interactive, Inc. (Telefonaktiebolaget LM Ericsson)
• Inventors: Dakss, Jon, Sarachik, Karen, Bove, V. Michael Jr., Katcher, Dan, Milazzo, Paul G.
• Primary Examiner(s): Koenig, Andrew Y.
• Application Number:
  US09/697,775
• Time in Patent Office: 2,168 Days
• Field of Search: 725/60, 725/112, 725/113, 725/135
• US Class Current: 725/60
• CPC Class Codes: G06F 16/748 Hypervideo linking data to content, e.g. by linking an URL to a video object in the context of video distribution systems H04N21/858
• CPC Class Codes: G06F 16/954 Navigation, e.g. using categorised browsing
• CPC Class Codes: G06F 16/9558 Details of hyperlinks; Management of linked annotations
• CPC Class Codes: H04N 1/64 Systems for the transmission or the storage of the colour picture signal; Details therefor, e.g. coding or decoding means therefor H04N19/00 takes precedence
• CPC Class Codes: H04N 19/93 Run-length coding
• CPC Class Codes: H04N 21/234318 by decomposing into objects, e.g. MPEG-4 objects
• CPC Class Codes: H04N 21/235 Processing of additional data, e.g. scrambling of additional data or processing content descriptors
• CPC Class Codes: H04N 21/435 Processing of additional data, e.g. decrypting of additional data, reconstructing software from modules extracted from the transport stream
• CPC Class Codes: H04N 21/4622 Retrieving content or additional data from different sources, e.g. from a broadcast channel and the Internet web site content organization and management for information retrieval from the Internet G06F16/958; transmission by internet of broadcast information H04H60/82; stock exchange data over packet-switching network H04L12/1804; push services including data channel over packet-switching network H04L12/1859
• CPC Class Codes: H04N 21/4722 for requesting additional data associated with the content
• CPC Class Codes: H04N 21/4725 using interactive regions of the image, e.g. hot spots details of information retrieval from the Internet by using URLs G06F16/955; processing chained hypermedia data for information retrieval G06F16/94
• CPC Class Codes: H04N 21/4728 for selecting a Region Of Interest [ROI], e.g. for requesting a higher resolution version of a selected region
• CPC Class Codes: H04N 21/4782 Web browsing , e.g. WebTV
• CPC Class Codes: H04N 21/812 involving advertisement data advertising per se G06Q30/02
• CPC Class Codes: H04N 21/854 Content authoring
• CPC Class Codes: H04N 21/8543 using a description language, e.g. Multimedia and Hypermedia information coding Expert Group [MHEG], eXtensible Markup Language [XML] information retrieval of semistructured data, the underlying structure being taken into account, e.g. mark-up language structure data G06F16/80
• CPC Class Codes: H04N 21/858 Linking data to content, e.g. by linking an URL to a video object, by creating a hotspot
• CPC Class Codes: H04N 21/8583 by creating hot-spots
• CPC Class Codes: H04N 21/8586 by using a URL processing chained hypermedia data for information retrieval G06F16/94; information retrieval from the Internet by using URLs G06F16/955; URL in broadcast information H04H20/93; Web-based protocols H04L67/02

Abstract

A system and method of adding hyperlinked information to a television broadcast. The broadcast material is analyzed and one or more regions within a frame are identified. Additional information can be associated with a region, and can be transmitted in encoded form, using timing information to identify the frame with which the information is associated. The system comprising a video source and an encoder that produces a transport stream in communication with the video source, an annotation source, a data packet stream generator that produces encoded annotation data packets in communication with the annotation source and the encoder, and a multiplexer system in communication with the encoder and the data packet stream generator. The encoder provides timestamp information to the data packet stream generator and the data packet stream generator synchronizes annotation data from the annotation source with a video signal from the video source in response to the timestamp information. The multiplexer generates a digital broadcast signal that includes an augmented transport stream from the transport stream and the encoded data packets. A receiver displays the annotation information associated with the video signal in response to a viewer request on a frame by frame basis. A viewer can respond interactively to the material, including performing commercial transactions, by using a backchannel that is provided for interactive communication.

Claims

1. In a video hyperlinked annotation data system, a computer-readable medium storing data for access by a computer program executed by a processor in the hyperlinked annotation data system, the computer-readable medium comprising:

a sixth data structure element storing image overlay data for a plurality of objects in a video frame, the sixth data structure element being associated with a visibility indicia indicative of whether the plurality of objects in the video frame are enabled for being visually identified for a particular video shot, the sixth data structure element further being associated with a sixth identifier;

a first annotation data structure element identified by said sixth identifier, the first annotation data structure element including an object reference for each of the plurality of objects in the video frame, each of the objects in the video frame having a corresponding first identifier; and

a second annotation data structure element identified by said first identifier, said second annotation data structure element including a first set of annotation data references.

2. The computer-readable medium of claim 1 further comprising at least one timing data indicator associated with at least one of said plurality of data structure elements, wherein said timing data indicator indicates an expiration time.

3. The computer-readable medium of claim 1 further comprising at least one timing data indicator associated with at least one of said plurality of data structure elements, wherein said timing data indicator indicates an activation time.

4. The computer-readable medium of claim 1 wherein said first annotation data structure element is associated with a set of video frames of a video program and wherein said second annotation data structure element is associated with said video program.

5. The computer-readable medium of claim 1 wherein said first data structure element and said second data structure element are transmitted separately.

6. The computer-readable medium of claim 1 wherein said first set of annotation data references includes an annotation data field and a second identifier referencing a third annotation data structure element.

7. The computer-readable medium according to claim 6 wherein said annotation data field is a title data field and said third annotation data structure element is a string including a title of said object.

8. The computer-readable medium according to claim 6 wherein said third annotation data structure element includes at least one display identifier for referencing a fourth data structure element to be displayed to a viewer and at least one action identifier referencing a fifth data structure element providing instructions so said system for action to be taken by said system.

9. The computer-readable medium according to claim 6 wherein said annotation data field is a variable parameter field.

10. The computer-readable medium according to claim 6 wherein said second identifier is a variable value.

11. The computer-readable medium according to claim 6 wherein said first and second identifiers are never duplicated by the system.

12. The computer-readable medium according to claim 1 wherein the sixth data structure element includes location and shape information about said object.

13. The computer-readable medium according to claim 12, wherein said sixth data structure element is associated with a video frame.

14. The computer-readable medium of claim 1 further comprising at least one timing data indicator associated with at least one of said plurality of data structure elements, wherein presentation of the image overlay is synchronized with the video frame based on the timing data indicator.

15. The computer-readable medium of claim 14, wherein the application program compares the timing data indicator with second timing data associated with a current video frame and sleeps for a period of time equivalent to a difference in times indicated by the first and second timing data, the computer program code being awakened for visually identifying the object based on the image overlay data in response to an expiration of the time period.

16. The computer-readable medium of claim 14, wherein at least one of the data structures is associated with second timing data, the second timing data being indicative of a last instance the data structure is used in a video program associated with the video frame, wherein the data structure is removed from the computer-readable medium in response to a determination based on the second timing data that the data structure is no longer used in the video program.

17. The computer-readable medium of claim 1 further comprising:

at least one timing data indicator in association with at least one of the plurality of data structure elements, wherein the application program retrieves the timing data indicator and synchronizes presentation of at least one of the plurality of data structure elements with a video frame based on the retrieved timing data indicator.

18. The computer-readable medium of claim 1, wherein each of the plurality of objects are visually identified in sequence in response to one or more user commands.

19. A method for generating one or more data structures via a first processor in a hyperlinked video signal annotation data system, the one or more data structures being stored in a computer-readable medium for access by a computer program executed by a second processor in the hyperlinked video signal annotation data system, the method comprising:

creating a sixth data structure element storing image overlay data representing a plurality of objects in a video frame, the sixth data structure element being associated with a visibility indicia indicative of whether the plurality of objects in the video frame are enabled for being visually identified for a particular video shot, the sixth data structure element further being associated with a sixth identifier;

creating a first annotation data structure element identified by said sixth identifier, the first annotation data structure element including an object reference for each of the plurality of objects in the video frame, each of the objects in the video frame having a corresponding first identifier; and

creating a second annotation data structure element identified by said first identifier, said second annotation data structure element including a first set of annotation data references.

20. The method of claim 19 further comprising at least one timing data indicator associated with at least one of said plurality of data structure elements, wherein said timing data indicator indicates an expiration time.

21. The method of claim 19 further comprising at least one timing data indicator associated with at least one of said plurality of data structure elements, wherein said timing data indicator indicates an activation time.

22. The method of claim 19 wherein said first annotation data structure element is associated with a set of video frames of a video program and wherein said second annotation data structure element is associated with said video program.

23. The method of claim 19 wherein said first data structure element and said second data structure element are transmitted separately.

24. The method of claim 19 wherein said first set of annotation data references includes an annotation data field and a second identifier referencing a third annotation data structure element.

25. The method according to claim 24 wherein said annotation data field is a title data field and said third annotation data structure element is a string including a title of said object.

26. The method according to claim 24 wherein said third annotation data structure element includes at least one display identifier for referencing a fourth data structure element to be displayed to a viewer and at least one action identifier referencing a fifth data structure element providing instructions to said system for actions to be taken by said system.

27. The method according to claim 24 wherein said annotation data field is a variable parameter field.

28. The method according to claim 24 wherein said second identifier is a variable value.

29. The method according to claim 24 wherein said first and second identifiers are never duplicated by the system.

30. The method according to claim 19 wherein the sixth data structure element includes location and shape information about said object.

31. The method according to claim 30, wherein said sixth data structure element is associated with a video frame.

32. The method of claim 19 further comprising:

creating at least one timing data indicator in association with at least one of the plurality of data structure elements, wherein the application program retrieves the timing data indicator and synchronizes presentation of at least one of the plurality of data structure elements with a video frame based on the retrieved timing data indicator.

33. The method of claim 19, wherein each of the plurality of objects are visually identified in sequence in response to one or more user commands.

34. The method of claim 19 further comprising:

creating at least one timing data indicator in association with at least one of the data structures, wherein the application program retrieves the timing data indicator and synchronizes presentation of at least one of the data structures with a video frame based on the retrieved timing data indicator.