Hierarchical method and system for object-based audiovisual descriptive tagging of images for information retrieval, editing, and manipulation

• US 6,070,167 A
• Filed: 03/02/1998
• Issued: 05/30/2000
• Est. Priority Date: 09/29/1997
• Status: Expired due to Term

• Current Assignee: Sharp Kabushiki Kaisha (Hon Hai Precision Industry Co., Ltd.)
• Original Assignee: Sharp Laboratories of America Incorporated (Hon Hai Precision Industry Co., Ltd.)
• Inventors: Qian, Richard J., Sezan, M. Ibrahim
• Primary Examiner(s): AMSBURY, WAYNE P
• Application Number:
  US09/033,568
• Time in Patent Office: 820 Days
• Field of Search: 707/102
• US Class Current: 1/1
• CPC Class Codes: G06F 16/71 Indexing; Data structures therefor; Storage structures
• CPC Class Codes: G06F 16/748 Hypervideo linking data to content, e.g. by linking an URL to a video object in the context of video distribution systems H04N21/858
• CPC Class Codes: G11B 27/034 on discs G11B27/036, G11B27/038 take precedence
• CPC Class Codes: H04N 21/234318 by decomposing into objects, e.g. MPEG-4 objects
• CPC Class Codes: H04N 21/235 Processing of additional data, e.g. scrambling of additional data or processing content descriptors
• CPC Class Codes: H04N 21/435 Processing of additional data, e.g. decrypting of additional data, reconstructing software from modules extracted from the transport stream
• CPC Class Codes: H04N 21/47205 for manipulating displayed content, e.g. interacting with MPEG-4 objects, editing locally
• CPC Class Codes: Y10S 707/99943 Generating database or data structure, e.g. via user interface

Abstract

A hierarchical system for object-based audiovisual descriptive tagging of images for information retrieval, editing, and manipulation, includes: an object-based selection mechanism for selecting an object of interest in said image; hierarchical data structure generation means for generating a hierarchical data structure for said image and for associating auxiliary information with said image; and a transmission/storage mechanism for storing the image and the hierarchical data structure. A hierarchical method for object-based audiovisual descriptive tagging of images for information retrieval, editing, and manipulation, includes: selecting an object of interest in said image with an object-based selection mechanism; generating a hierarchical data structure for said image and for associating auxiliary information with said image; and transmitting/storing the image and the hierarchical data structure.

Claims

1. A hierarchical system for associating object-based information with images for information retrieval, editing, and manipulation, comprising:

an object-based selection mechanism for selecting an object of interest in an image;

hierarchical data structure generation means for generating a hierarchical data structure for said image and for associating object-based information with said image;

a transmission/storage mechanism for storing the image and the hierarchical data structure;

a retrieval and manipulation mechanism for allowing a user selectively to retrieve and manipulate the image and the object-based information associated therewith; and

a generation mechanism for generating an overlay associated with said image, wherein said overlay includes at least one hot spot which is visually distinguishable from the remainder of the image when highlighted by the user.

2. The system of claim 1 which includes an image acquisition mechanism for acquiring an image.

3. The system of claim 1 which includes a display mechanism for displaying the image to a user.

4. The system of claim 3 wherein said display mechanism is constructed and arranged to display said hierarchical data structure to a user.

5. The system of claim 1 which includes a storage mechanism for storing an image.

6. The system of claim 1 which includes a database containing a collection of digital images therein.

7. The system of claim 1 wherein said image and said hierarchical data structure for said image are stored in a single file.

8. The system of claim 1 wherein said image and said hierarchical data structure for said image are stored in separate files.

9. The system of claim 1 wherein said generation mechanism generates boundary information for identifying a boundary about an object of interest, and wherein said boundary groups all of the information within said boundary for manipulation by the user.

10. The system of claim 1 which includes an audiovisual realization mechanism wherein object-based information is visually displayed to the user, and audibly played to the user, upon the user'"'"'s request.

11. The system of claim 1 which includes an audiovisual realization mechanism wherein object-based information is used for object-based image editing.

12. The system of claim 1 wherein said hierarchical data structure includes a base layer which includes only content-related information indicators, and a second layer which includes content-related information.

13. A hierarchical system for associating object-based information with images for information retrieval, editing, and manipulation, comprising:

an image acquisition mechanism for acquiring an image;

an object-based selection mechanism for selecting an object of interest in said image;

hierarchical data structure generation means for generating a hierarchical data structure for said image and for associating object-based information with said image, thereby forming an information-enhanced image;

a transmission/storage mechanism for storing the information-enhanced image;

a display mechanism for displaying the information-enhanced image to a user;

a retrieval and manipulation mechanism for allowing a user selectively to retrieve and manipulate the image and the object-based information associated therewith; and

a generation mechanism for generating boundary information for identifying a boundary about an object of interest, and wherein said boundary groups all of the information within said boundary for manipulation by the user.

14. The system of claim 13 wherein said display mechanism is constructed and arranged to display said hierarchical data structure of the information-enhanced image to a user.

15. The system of claim 13 which includes a database containing a collection of digital images therein.

16. The system of claim 13 wherein said image and said hierarchical data structure for said image are stored in a single file.

17. The system of claim 13 wherein said image and said hierarchical data structure for said image are stored in separate files.

18. The system of claim 13 wherein said generation mechanism generates an overlay associated with said image, and wherein said overlay includes at least one hot spot which is visually distinguishable from the remainder of the image when highlighted by the user.

19. The system of claim 13 which includes an audiovisual realization mechanism wherein object-based information is visually displayed to the user, and audibly played to the user, upon the user'"'"'s request.

20. The system of claim 13 which includes an audiovisual realization mechanism wherein object-based information is used for object-based image editing.

21. The system of claim 13 wherein said hierarchical data structure includes a base layer which includes only content-related information indicators, and a second layer which includes content-related information.

22. A hierarchical method for associating object-based information with images for information retrieval, editing, and manipulation, comprising:

selecting an object of interest in an image with an object-based selection mechanism;

generating a hierarchical data structure for said image and for associating object-based information with the image, including generating a base layer which includes only content-related information indicators, and generating a second layer which includes content-related information;

selectively retrieving and manipulating the image and the information associated therewith;

including;

(a) retrieving the image data;

(b) reading the base layer information;

(c) displaying the image;

(d) generating an overlay to visually indicate the regions that contain object-based information in terms of "hot spots", according to the region information contained in the base layer;

(e) displaying pop-up menus as the user points and clicks on the hot spots, wherein the types of available object-based information are featured in the menus; and

(f) retrieving and rendering the object-based information selected by the user when the user clicks on the appropriate entry in the menu; and

transmitting/storing the image and the hierarchical data structure.

23. The method of claim 22 which includes acquiring an image with an image acquisition mechanism.

24. The method of claim 22 which includes displaying the transmitted/stored image to a user.

25. The method of claim 22 which further includes displaying visually object-based information and playing, audibly object-based information to the user, upon the user'"'"'s request.

26. The method of claim 22 which includes using object-based information for object-based image editing.

27. The method of claim 22 wherein said generating an overlay includes highlighting a hot spot when user'"'"'s pointing device points at a location within the area of that region.

28. The method of claim 22 wherein said generating an overlay includes identifying a boundary about an object of interest.