Production apparatus for index information with link information, production apparatus for image data with tag information, production method for index information with link information, production method for image data with tag information and recording medium

• US 8,019,183 B2
• Filed: 07/05/2006
• Issued: 09/13/2011
• Est. Priority Date: 07/06/2005
• Status: Expired due to Fees

• Current Assignee: Sony Corporation (Sony Group Corp.)
• Original Assignee: Sony Corporation (Sony Group Corp.)
• Inventors: Kikuchi, Hiroaki, Imaida, Hideyuki
• Primary Examiner(s): Le; Brian Q
• Assistant Examiner(s): BEKELE, MEKONEN T
• Application Number:
  US11/428,711
• Publication Number:
  US 20070009100A1
• Time in Patent Office: 1,896 Days
• Field of Search: 382/203, 382/223, 382/226, 382/228, 382/305, 382/103, 382/107, 345/156, 348/416.6, 348/169, 348/578, 348/586, 348/584, 348/626
• US Class Current: 382/305
• CPC Class Codes: G06F 16/748 Hypervideo linking data to content, e.g. by linking an URL to a video object in the context of video distribution systems H04N21/858
• CPC Class Codes: G11B 27/034 on discs G11B27/036, G11B27/038 take precedence
• CPC Class Codes: G11B 27/105 of operating discs
• CPC Class Codes: G11B 27/322 used signal is digitally coded
• CPC Class Codes: H04N 21/235 Processing of additional data, e.g. scrambling of additional data or processing content descriptors
• CPC Class Codes: H04N 21/435 Processing of additional data, e.g. decrypting of additional data, reconstructing software from modules extracted from the transport stream
• CPC Class Codes: H04N 21/47815 Electronic shopping payment schemes, payment architectures or payment protocols for electronic shopping systems G06Q20/12
• CPC Class Codes: H04N 21/8583 by creating hot-spots
• CPC Class Codes: H04N 21/8586 by using a URL processing chained hypermedia data for information retrieval G06F16/94; information retrieval from the Internet by using URLs G06F16/955; URL in broadcast information H04H20/93; Web-based protocols H04L67/02
• CPC Class Codes: H04N 7/17318 Direct or substantially direct transmission and handling of requests

Abstract

Disclosed herein is a production apparatus for index information with link information. The apparatus includes a database in which a corresponding relationship between region identification information applied to a key region which provides a partial region in a main image specified arbitrarily by an operator and link information for accessing to related information of the partial region is listed. The apparatus further includes an information embedding section configured to refer to said database to embed, into index information in which a corresponding relationship between the region identification information applied to the key region and time information is listed, the corresponding link information.

Claims

1. A production apparatus for index information with link information, comprising:

a key information unit which determines an initial key region in a first frame of a plurality of frames, which is a partial image of an entire image in the first frame, based on an input from an operator and applies region identification information to the initial key region in the first frame, wherein the region identification information corresponds to at least one pixel value of the initial key region in the first frame and the plurality of frames are each associated with a respective time code, and determines a final key region in a final frame of the plurality of frames, which is a partial image of an entire image in the final frame based on an input from the operator, the time code associated with the final frame being subsequent to the time code associated with the first frame;

a follow-up unit which automatically determines whether a key region in a second frame of the plurality of frames includes at least one pixel value similar to the initial key region in the first frame, determines a position of the key region in the second frame based on an initial position of the initial key region in the first frame and a final position of the final key region in the final frame, and applies region identification information to the key region in the second frame when the key region in the second frame is determined to include the at least one pixel value similar to the initial key region in the first frame, wherein the time code associated with the second frame is subsequent to the time code associated with the first frame and prior to the time code associated with the final frame;

a database which stores a corresponding relationship between region identification information applied to at least one of the key regions of the plurality of frames received from the key information unit or the follow-up unit and link information which provides access to information related to the at least one of the key regions; and

an information embedding section configured to refer to said database to embed the link information corresponding to the at least one of the key regions into index information which associates the region identification information applied to the at least one of the key regions and time information.

2. The production apparatus for index information with link information according to claim 1, wherein said database records the index information.

39. The production apparatus for index information with link information according to claim 1, wherein the link information includes one of a Uniform Resource Locator (URL) and a commercial message (CM) identifier (local address).

40. The production apparatus for index information with link information according to claim 1, wherein a numerical value representing the region identification information is a gray scale value of the key region.

41. The production apparatus for index information with link information according to claim 1, wherein the key information unit determines the initial key region as a start point and the final key region as an end point, and

the follow-up unit estimates a plurality of estimated key regions between the initial key region and the final key region, based on time information associated with the initial key region and time information associated with the final key region.

43. The production apparatus for index information with link information according to claim 1, further comprising:

a key image production unit that generates and stores a key image for each key region determined by the key information unit or the follow-up unit, wherein the key image includes the determined key region, the applied region identification information, and time information indicating the time code of the respective frame.

44. The production apparatus for index information with link information according to claim 1, wherein the time code associated with the second frame is sequential to the time code associated with the first frame.

3. A production apparatus for index information with link information, comprising:

a key information unit which determines an initial key region in a first frame of a plurality of frames, which is a partial image of an entire image in the first frame, based on an input from an operator, applies region identification information to the initial key region in the first frame, wherein the region identification information corresponds to at least one pixel value of the initial key region in the first frame, applies object identification information to the key region in the first frame, and determines a final key region in a final frame of the plurality of frames, which is a partial image of an entire image in the final frame based on an input from the operator,

wherein the plurality of frames are each associated with a respective time code and the time code associated with the final frame is subsequent to the time code associated with the first frame;

a follow-up unit which automatically determines whether a key region in a second frame of the plurality of frames includes at least one pixel value similar to the key region in the first frame, determines a position of the key region in the second frame based on an initial position of the initial key region in the first frame and a final position of the final key region in the final frame, and applies region identification information to the key region in the second frame when the key region in the second frame is determined to include the at least one pixel value similar to the initial key region in the first frame, wherein the time code associated with the second frame is subsequent to the time code associated with the first frame and prior to the time code associated with the final frame;

a database which stores a corresponding relationship between region identification information received from the key information unit or the follow-up unit, object identification information applied to at least one of the key regions of the plurality of frames, and link information which provides access to information related to the at least one of the key regions; and

an information embedding section configured to refer to said database to embed the link information corresponding to the at least one of the key regions into index information which associates the region identification information applied to the at least one of the key regions, the object identification information applied to the at least one of the key regions, and time information.

4. The production apparatus for index information with link information according to claim 3, wherein said database records the index information.

42. The production apparatus for index information with link information according to claim 3, wherein the object identification information is at least one character which describes a feature of the key region.

5. A production apparatus for image data with tag information, comprising:

a key information unit which determines an initial key region in a first frame of a plurality of frames, which is a partial image of an entire image in the first frame, based on an input from an operator and applies region identification information to the initial key region in the first frame, wherein the region identification information corresponds to at least one pixel value of the initial key region in the first frame and the plurality of frames are each associated with a respective time code, and determines a final key region in a final frame of the plurality of frames, which is a partial image of an entire image in the final frame based on an input from the operator, the time code associated with the final frame being subsequent to the time code associated with the first frame;

a follow-up unit which automatically determines whether a key region in a second frame of the plurality of frames includes at least one pixel value similar to the key region in the first frame, determines a position of the key region in the second frame based on an initial position of the initial key region in the first frame and a final position of the final key region in the final frame, and applies region identification information to the key region in the second frame when the key region in the second frame is determined to include the at least one pixel value similar to the initial key region in the first frame, wherein the time code associated with the second frame is subsequent to the time code associated with the first frame and prior to the time code associated with the final frame;

a database which stores a corresponding relationship between region identification information applied to at least one of the key regions of the plurality of frames received from the key information unit or the follow-up unit and link information which provides access to information related to the at least one of the key regions;

an information embedding section configured to refer to said database to embed the link information corresponding to the at least one of the key regions into index information which associates the region identification information applied to the at least one of the key regions and time information; and

a data outputting section configured to output the index information with link information in which the link information is embedded, the plurality of frames, and a key image to which the region identification information which specifies the key region in combination with the time information is applied in a mutually associated relationship.

6. The production apparatus for image data with tag information according to claim 5, wherein the outputting destination of said data outputting section is a transmission line.

7. The production apparatus for image data with tag information according to claim 5, wherein said data outputting section outputs the index information with link information, the plurality of frames, and the key image in a broadcasting form.

8. The production apparatus for image data with tag information according to claim 5, wherein said data outputting section outputs the index information with link information, the plurality of frames, and the key image in a distribution form.

9. The production apparatus for image data with tag information according to claim 5, wherein the outputting destination of said data outputting section is a recording medium.

10. The production apparatus for image data with tag information according to claim 5, wherein the plurality of frames is a live image.

11. The production apparatus for image data with tag information according to claim 5, wherein said data outputting section outputs the index information with link information, the plurality of frames, and the key image collectively as a file.

12. The production apparatus for image data with tag information according to claim 5, wherein said data outputting section applies the index information with link information to two compression coded data obtained by compression coding the plurality of frames and the key image to produce a file.

13. The production apparatus for image data with tag information according to claim 5, wherein said data outputting section applies the index information with link information to single compression coded data obtained by compression coding image data wherein three of four data channels are allocated to the plurality of frames and the remaining one data channel is allocated to the key image to produce a file.

14. The production apparatus for image data with tag information according to claim 5, wherein said data outputting section applies the index information with link information to compression coded data obtained by compression coding image data wherein three of four data channels are allocated to the plurality of frames and the remaining one data channel is allocated to the key image in which the index information with link information is embedded as incidental information to produce a file.

15. The production apparatus for image data with tag information according to claim 5, wherein said data outputting section outputs the plurality of frames to a main channel and simultaneously outputs the key image to a sub channel while the index information with link information image is placed in a meta data area of the plurality of frames or the key image.

16. The production apparatus for image data with tag information according to claim 5, wherein said data outputting section outputs the plurality of frames to a main channel while the key image and the index information with link information are placed in a meta data area of the plurality of frames.

17. The production apparatus for image data with tag information according to claim 5, wherein said data outputting section outputs a body of related information which makes a destination of linking of the link information embedded in the index information with link information in an associated relationship with the index information with link information, the plurality of frames, and the key image.

18. A production apparatus for image data with tag information, comprising:

a key information unit which determines an initial key region in a first frame of a plurality of frames, which is a partial image of an entire image in the first frame, based on an input from an operator, applies region identification information to the initial key region in the first frame, wherein the region identification information corresponds to at least one pixel value of the initial key region in the first frame, applies object identification information to the initial key region in the first frame, and determines a final key region in a final frame of the plurality of frames, which is a partial image of an entire image in the final frame based on an input from the operator,

wherein the plurality of frames are each associated with a respective time code and the time code associated with the final frame is subsequent to the time code associated with the first frame;

a follow-up unit which automatically determines whether a key region in a second frame of the plurality of frames includes at least one pixel value similar to the key region in the first frame, determines a position of the key region in the second frame based on an initial position of the initial key region in the first frame and a final position of the final key region in the final frame, and applies region identification information to the key region in the second frame when the key region in the second frame is determined to include the at least one pixel value similar to the initial key region in the first frame, wherein the time code associated with the second frame is subsequent to the time code associated with the first frame and prior to the time code associated with the final frame;

a database which stores a corresponding relationship between region identification information received from the key information unit or the follow-up unit, object identification information applied to at least one of the key regions of the plurality of frames, and link information which provides access to information related to the at least one of the key regions;

an information embedding section configured to refer to said database to embed the link information corresponding to the at least one of the key regions into index information which associates the region identification information applied to the at least one of the key regions, the object identification information applied to the at least one of the key regions, and time information; and

a data outputting section configured to output the index information with link information in which the link information is embedded, the plurality of frames, and a key image to which the region identification information which specifies the key region in combination with the time information is applied in a mutually associated relationship.

19. The production apparatus for image data with tag information according to claim 18, wherein the outputting destination of said data outputting section is a transmission line.

20. The production apparatus for image data with tag information according to claim 18, wherein said data outputting section outputs the index information with link information, the plurality of frames, and the key image in a broadcasting form.

21. The production apparatus for image data with tag information according to claim 18, wherein said data outputting section outputs the index information with link information, the plurality of frames, and the key image in a distribution form.

22. The production apparatus for image data with tag information according to claim 18, wherein the outputting destination of said data outputting section is a recording medium.

23. The production apparatus for image data with tag information according to claim 18, wherein the plurality of frames is a live image.

24. The production apparatus for image data with tag information according to claim 18, wherein said data outputting section outputs the index information with link information, the plurality of frames, and the key image collectively as a file.

25. The production apparatus for image data with tag information according to claim 18, wherein said data outputting section applies the index information with link information to two compression coded data obtained by compression coding the plurality of frames and the key image to produce a file.

26. The production apparatus for image data with tag information according to claim 18, wherein said data outputting section applies the index information with link information to single compression coded data obtained by compression coding image data wherein three of four data channels are allocated to the plurality of frames and the remaining one data channel is allocated to the key image to produce a file.

27. The production apparatus for image data with tag information according to claim 18, wherein said data outputting section applies the index information with link information to compression coded data obtained by compression coding image data wherein three of four data channels are allocated to the plurality of frames and the remaining one data channel is allocated to the key image in which the index information with link information is embedded as incidental information to produce a file.

28. The production apparatus for image data with tag information according to claim 18, wherein said data outputting section outputs the plurality of frames to a main channel and simultaneously outputs the key image to a sub channel while the index information with link information image is placed in a meta data area of the plurality of frames or the key image.

29. The production apparatus for image data with tag information according to claim 18, wherein said data outputting section outputs the plurality of frames to a main channel while the key image and the index information with link information are placed in a meta data area of the plurality of frames.

30. The production apparatus for image data with tag information according to claim 18, wherein said data outputting section outputs a body of related information which makes a destination of linking of the link information embedded in the index information with link information in an associated relationship with the index information with link information, the plurality of frames, and the key image.

31. A production method, implemented on a production apparatus, for index information with link information, comprising:

determining, at a key information unit of the production apparatus, an initial key region in a first frame of a plurality of frames, which is a partial image of an entire image in the frame, based on an input from an operator, determining a final key region in a final frame of the plurality of frames, which is a partial image of an entire image in the final frame based on an input from the operator, and applying region identification information to the initial key region in the first frame, wherein the region identification information corresponds to at least one pixel value of the initial key region in the first frame the plurality of frames are each associated with a respective time code, and the time code associated with the final frame is subsequent to the time code associated with the first frame;

determining automatically, at a follow-up unit, whether a key region in a second frame of the plurality of frames includes at least one pixel value similar to the key region in the first frame, determining a position of the key region in the second frame based on a position of the initial key region in the first frame and a position of the final key region in the final frame, and applying region identification information to the key region in the second frame when the key region in the second frame is determined to include the at least one pixel value similar to the key region in the first frame, wherein the time code associated with the second frame is subsequent to the time code associated with the first frame and prior to the time code associated with the final frame; and

referring to a database which stores a corresponding relationship between region identification information applied to at least one of the key regions of the plurality of frames received from the key information unit or the follow-up unit and link information which provides access to information related to the at least one of the key regions to embed the link information corresponding to the at least one of the key regions into index information which associates the region identification information applied to the at least one of the key regions and time information.

32. A production method, implemented on a production apparatus, for index information with link information, comprising:

determining, at a key information unit of the production apparatus, an initial key region in a first frame of a plurality of frames, which is a partial image of an entire image in the frame, based on an input from an operator, applying region identification information to the initial key region in the first frame, wherein the region identification information corresponds to at least one pixel value of the initial key region in the first frame, applying object identification information to the initial key region in the first frame, and determining a final key region in a final frame of the plurality of frames, which is a partial image of an entire image in the final frame based on an input from the operator,

wherein the plurality of frames are each associated with a respective time code and the time code associated with the final frame is subsequent to the time code associated with the first frame;

determining automatically, at a follow-up unit, whether a key region in a second frame of the plurality of frames includes at least one pixel value similar to the key region in the first frame, determining a position of the key region in the second frame based on an initial position of the initial key region in the first frame and a final position of the final key region in the final frame, and applying region identification information to the key region in the second frame when the key region in the second frame is determined to include the at least one pixel value similar to the key region in the first frame, wherein the time code associated with the second frame is subsequent to the time code associated with the first frame and prior to the time code associated with the final frame; and

referring to a database which stores a corresponding relationship between region identification information received from the key information unit or the follow-up unit, object identification information applied to at least one of the key regions of the plurality of frames, and link information which provides access to information related to the at least one of the key regions and to embed the link information corresponding to the at least one of the key regions into index information which associates the region identification information applied to the at least one of the key regions, the object identification information applied to the at least one of the key regions, and time information.

33. A production method, implemented on a production apparatus, for image data with tag information, comprising:

determining, at a key information unit of the production apparatus, an initial key region in a first frame of a plurality of frames, which is a partial image of an entire image in the frame, based on an input from an operator, applying region identification information to the initial key region in the first frame, and determining a final key region in a final frame of the plurality of frames, which is a partial image of an entire image in the final frame based on an input from the operator, wherein the region identification information corresponds to at least one pixel value of the initial key region in the first frame, the plurality of frames are each associated with a respective time code, and the time code associated with the final frame is subsequent to the time code associated with the first frame;

determining automatically, at a follow-up unit, whether a key region in a second frame of the plurality of frames includes at least one pixel value similar to the initial key region in the first frame, determining a position of the key region in the second frame based on an initial position of the initial key region in the first frame and a final position of the final key region in the final frame, and applying region identification information to the key region in the second frame when the key region in the second frame is determined to include the at least one pixel value similar to the initial key region in the first frame, wherein the time code associated with the second frame is subsequent to the time code associated with the first frame and prior to the time code associated with the final frame;

referring to a database which stores a corresponding relationship between region identification information applied to at least one of the key regions of the plurality of frames received from the key information unit or the follow-up unit and link information which provides access to information related to the at least one of the key regions to embed the link information corresponding to the at least one of the key regions into index information which associates the region identification information applied to the at least one of the key regions and time information; and

outputting, at an outputting section, the index information with link information in which the link information is embedded, the plurality of frames and a key image to which the region identification information which specifies the key region in combination with the time information is applied in a mutually associated relationship.

34. A production method, implemented on a production apparatus, for image data with tag information, comprising:

determining, at a key information unit of the production apparatus, an initial key region in a first frame of a plurality of frames, which is a partial image of an entire image in the frame, based on an input from an operator, applying region identification information to the initial key region in the first frame, wherein the region identification information corresponds to at least one pixel value of the initial key region in the first frame, applying object identification information to the initial key region in the first frame, and determining a final key region in a final frame of the plurality of frames, which is a partial image of an entire image in the final frame based on an input from the operator,

wherein the plurality of frames are each associated with a respective time code and the time code associated with the final frame is subsequent to the time code associated with the first frame;

determining automatically, at a follow-up unit, whether a key region in a second frame of the plurality of frames includes at least one pixel value similar to the key region in the first frame, determining a position of the key region in the second frame based on a position of the initial key region in the first frame and a position of the final key region in the final frame, and applying region identification information to the key region in the second frame when the key region in the second frame is determined to include the at least one pixel value similar to the initial key region in the first frame object identification information to the key region in the first frame, wherein the time code associated with the second frame is subsequent to the time code associated with the first frame and prior to the time code associated with the final frame;

referring to a database which stores a corresponding relationship between region identification information received from the key information unit or the follow-up unit, object identification information applied to at least one of the key regions of the plurality of frames, and link information which provides access to information related to the at least one of the key regions to embed the link information corresponding to the at least one of the key regions into index information which associates the region identification information applied to the at least one of the key regions, the object identification information applied to the at least one of the key regions, and time information; and

outputting, at an outputting section, the index information with link information in which the link information is embedded, the plurality of frames, and a key image to which the region identification information which specifies the key region in combination with the time information is applied in a mutually associated relationship.

35. A non-transitory computer-readable recording medium for storing therein a computer program that includes instructions which when executed on a computer causes the computer to execute a method comprising:

determining, at a key information unit, an initial key region in a first frame of a plurality of frames, which is a partial image of an entire image in the frame, based on an input from an operator, applying region identification information to the initial key region in the first frame, and determining a final key region in a final frame of the plurality of frames, which is a partial image of an entire image in the final frame based on an input from the operator, wherein the region identification information corresponds to at least one pixel value of the initial key region in the first frame, the plurality of frames are each associated with a respective time code, and the time code associated with the final frame is subsequent to the time code associated with the first frame;

determining automatically, at a follow-up unit, whether a key region in a second frame of the plurality of frames includes at least one pixel value similar to the initial key region in the first frame, determining a position of the key region in the second frame based on a position of the initial key region in the first frame and a position of the final key region in the final frame, and applying region identification information to the key region in the second frame when the key region in the second frame is determined to include the at least one pixel value similar to the initial key region in the first frame, wherein the time code associated with the second frame is subsequent to the time code associated with the first frame and prior to the time code associated with the final frame; and

referring to a database which stores a corresponding relationship between region identification information applied to at least one of the key regions of the plurality of frames received from the key information unit or the follow-up unit and link information which provides a link to information related to the at least one of the key regions to embed the link information corresponding to the at least one of the key regions into index information which associates the region identification information applied to the at least one of the key regions and time information.

36. A non-transitory computer-readable recording medium for storing therein a computer program that includes instructions which when executed on a computer causes the computer to execute a method comprising:

determining, at a key information unit, an initial key region in a first frame of a plurality of frames, which is a partial image of an entire image in the frame, based on an input from an operator, applying region identification information to the initial key region in the first frame, wherein the region identification information corresponds to at least one pixel value of the initial key region in the first frame, applying object identification information to the initial key region in the first frame, and determining a final key region in a final frame of the plurality of frames, which is a partial image of an entire image in the final frame based on an input from the operator,

wherein the plurality of frames are each associated with a respective time code and the time code associated with the final frame is subsequent to the time code associated with the first frame;

determining automatically, at a follow-up unit, whether a key region in a second frame of the plurality of frames includes at least one pixel value similar to the initial key region in the first frame, determining a position of the key region in the second frame based on a position of the initial key region in the first frame and a position of the final key region in the final frame, and applying region identification information to the key region in the second frame when the key region in the second frame is determined to include the at least one pixel value similar to the initial key region in the first frame, wherein the time code associated with the second frame is subsequent to the time code associated with the first frame and prior to the time code associated with the final frame; and

referring to a database which stores a corresponding relationship between region identification information received from the key information unit or the follow-up unit, object identification information applied to at least one of the key regions of the plurality of frames, and link information which provides access to information related to the at least one of the key regions to embed the link information corresponding to the at least one of the key regions into index information which associates the region identification information applied to the at least one of the key regions, the object identification information applied to the at least one of the key regions, and time information.

37. A non-transitory computer-readable recording medium for storing therein a computer program that includes instructions which when executed on a computer causes the computer to execute a method comprising:

determining, at a key information unit, an initial key region in a first frame of a plurality of frames, which is a partial image of an entire image in the frame, based on an input from an operator, applying region identification information to the initial key region in the first frame, and determining a final key region in a final frame of the plurality of frames, which is a partial image of an entire image in the final frame based on an input from the operator, wherein the region identification information corresponds to at least one pixel value of the initial key region in the first frame, the plurality of frames are each associated with a respective time code, and the time code associated with the final frame is subsequent to the time code associated with the first frame;

determining automatically, at a follow-up unit, whether a key region in a second frame of the plurality of frames includes at least one pixel value similar to the initial key region in the first frame, determining a position of the key region in the second frame based on a position of the initial key region in the first frame and a position of the final key region in the final frame, and applying region identification information to the key region in the second frame when the key region in the second frame is determined to include the at least one pixel value similar to the initial key region in the first frame, wherein the time code associated with the second frame is subsequent to the time code associated with the first frame and prior to the time code associated with the final frame;

referring to a database which stores a corresponding relationship between region identification information applied to at least one of the key regions of the plurality of frames received from the key information unit or the follow-up unit and link information which provides access to information related to the at least one of the key regions to embed the link information corresponding to the at least one of the key regions into index information which associates the region identification information applied to the at least one of the key regions and time information; and

outputting the index information with link information in which the link information is embedded, the plurality of frames, and a key image to which the region identification information which specifies the key region in combination with the time information is applied in a mutually associated relationship.

38. A non-transitory computer-readable recording medium for storing therein a computer program that includes instructions which when executed on a computer causes the computer to execute a method comprising:

determining, at a key information unit, an initial key region in a first frame of a plurality of frames, which is a partial image of an entire image in the frame, based on an input from an operator, applying region identification information to the initial key region in the first frame, wherein the region identification information corresponds to at least one pixel value of the initial key region in the first frame, applying object identification information to the initial key region in the first frame, and determining a final key region in a final frame of the plurality of frames, which is a partial image of an entire image in the final frame based on an input from the operator,

wherein the plurality of frames are each associated with a respective time code and the time code associated with the final frame is subsequent to the time code associated with the first frame;

determining automatically, at a follow-up unit, whether a key region in a second frame of the plurality of frames includes at least one pixel value similar to the key region in the first frame, determining a position of the key region in the second frame based on a position of the initial key region in the first frame and a position of the final key region in the final frame, and applying region identification information to the key region in the second frame when the key region in the second frame is determined to include the at least one pixel value similar to the key region in the first frame, wherein the time code associated with the second frame is subsequent to the time code associated with the first frame and prior to the time code associated with the final frame;

referring to a database which stores a corresponding relationship between region identification information received from the key information unit or the follow-up unit, object identification information applied to at least one of the key regions of the plurality of frames, and link information which provides a link to information related to the at least one of the key regions to embed the link information corresponding to the at least one of the key regions into index information which associates the region identification information applied to the at least one of the key regions, the object identification information applied to the at least one of the key regions, and time information; and

outputting the index information with link information in which the link information is embedded, the plurality of frames, and a key image to which the region identification information which specifies the key region in combination with the time information is applied in a mutually associated relationship.