Indexing and database apparatus and method for automatic description of content, archiving, searching and retrieving of images and other data

• US 20020071677A1
• Filed: 12/11/2000
• Published: 06/13/2002
• Est. Priority Date: 12/11/2000
• Status: Abandoned Application

• Current Assignee: Sumanaweera, Thilaka S.
• Original Assignee: Sumanaweera, Thilaka S.
• Inventors: Sumanaweera, Thilaka S.
• Application Number:
  US09/734,356
• Publication Number:
  US 20020071677A1
• Time in Patent Office: Days
• US Class Current: 396/429
• CPC Class Codes: G06F 16/58 Retrieval characterised by using metadata, e.g. metadata not derived from the content or metadata generated manually
• CPC Class Codes: H04N 1/00127 Connection or combination of a still picture apparatus with another apparatus, e.g. for storage, processing or transmission of still picture signals or of information associated with a still picture
• CPC Class Codes: H04N 1/00204 with a digital computer or a digital computer system, e.g. an internet server programmed control between transmitter and receiver or between image input and image output device H04N1/32561
• CPC Class Codes: H04N 1/00323 with a measuring, monitoring or signaling apparatus, e.g. for transmitting measured information to a central location
• CPC Class Codes: H04N 2101/00 Still video cameras
• CPC Class Codes: H04N 2201/0084 Digital still camera
• CPC Class Codes: H04N 2201/3215 of a time or duration
• CPC Class Codes: H04N 2201/3225 of data relating to an image, a page or a document
• CPC Class Codes: H04N 2201/3253 Position information, e.g. geographical position at time of capture, GPS data
• CPC Class Codes: H04N 2201/3254 Orientation, e.g. landscape or portrait; Location or order of the image data, e.g. in memory
• CPC Class Codes: H04N 2201/3277 The additional information being stored in the same storage device as the image data

Abstract

An indexing database apparatus and methods provide a way to automatically label, catalog and describe the content of images and other data using position, orientation, field of view and time of capture using a camera or a sensor. The present invention automatically generates a description of the content of an image or other data by querying a description database indexed to parameters such as location, orientation, field of view and time or a subset thereof recorded with images and other data. An exemplary apparatus comprises a GPS receiver for sensing the location of a camera or sensor, an angular position sensor for sensing the orientation of the axis of the camera or sensor, a field of view sensor for sensing the field of view of the camera or sensor and a clock for indicating the time of image or data capture. A recording medium records the image or other data and the above parameters. A display subsystem including a computer, a monitor, and at least one computer program, displays the image or other data along with the description of the content of the images or other data on a monitor. An indexing database apparatus and methods also provide means for archiving images or other data, its parameters and a description of its content in a database. The present invention also provides a way to search and retrieve images or other data from a database based on the content of the images or other data.

Claims

1. A method for generating a description of the content of an image comprising the steps of:

(a) reading in parameters associated with the image, wherein the parameters include the parameters describing the location of the sensor when the image is captured,

(b) querying a description database indexed to the parameters in (a),

(c) collecting the response corresponding to the description of the content of the image from the description database in response to (b).

2. The method as recited in claim 1, wherein the parameters include the parameters describing the orientation of the axis of the sensor when the image is captured.

3. The method as recited in claim 1, wherein the parameters include the parameters describing the field of view of the sensor when the image is captured.

4. The method as recited in claim 1, wherein the parameters include the parameters describing the time of image capture.

5. The method as recited in claim 1, wherein the location of the sensor when the image is captured is sensed using a GPS receiver.

6. The method as recited in claim 1, wherein the location of the sensor when the image is captured is sensed using a triangulation scheme of radio transmitting and receiving antenna towers.

7. The method as recited in claim 6 wherein the triangulation scheme is selected from a group consisting of time difference of arrival and angle of arrival.

8. The method as recited in claim 1, wherein the location of the sensor when the image is captured is sensed using a pattern-matching scheme.

9. The method as recited in claim 1, wherein the description contained in the database includes textual, diagrammatic, pictorial, audio, video and multimedia descriptions of images indexed to the parameters.

10. The method as recited in claim 1, wherein the sensor includes a camera.

11. The method as recited in claim 1 wherein the sensor is selected from a group consisting of sound sensors, ultrasonic sensors, diagnostic medical ultrasound scanners, X-ray detectors, magnetic resonance imaging scanners, computed tomography scanners, radar imagers, sonar imagers, motion pictures cameras, and sensors for producing electronic representations of smell and taste.

12. A method for archiving an image in a database of images comprising the steps of:

(a) generating a description of the content of the image using the method claimed in claims 1-11; and

(b) saving the description in the database.

13. A method for searching and retrieving images from a database of images based on their content comprising the steps of:

(a) generating a description of the content of an image in the database of images using the method claimed in claims 1-11; and

(b) matching a keyword to the description.

15. The apparatus as recited in claim 14 wherein the parameters include the parameters describing the orientation of the axis of the sensor when the image is captured.

16. The apparatus as recited in claim 14 wherein the parameters include the parameters describing the field of view of the sensor when the image is captured.

17. The apparatus as recited in claim 14 wherein the parameters include the parameters describing the time of image capture.

18. The apparatus as recited in claim 14 wherein the location of the sensor when the image is captured is sensed using a GPS receiver.

19. The apparatus as recited in claim 14 wherein the location of the sensor when the image is captured is sensed using a triangulation scheme of radio transmitting and receiving antenna towers.

20. The apparatus as recited in claim 19 wherein the triangulation scheme is selected from a group consisting of time difference of arrival and angle of arrival.

21. The apparatus as recited in claim 14 wherein the location of the sensor when the image is captured is sensed using a pattern-matching scheme.

22. The apparatus as recited in claim 14 wherein the description contained in the database includes textual, diagrammatic, pictorial, audio, video and multimedia descriptions of images 29 indexed to the parameters.

23. The apparatus as recited in claim 14 wherein the sensor includes a camera.

24. The apparatus as recited in claim 14 wherein the sensor is selected from a group consisting of sound sensors, ultrasonic sensors, diagnostic medical ultrasound scanners, X-ray detectors, magnetic resonance imaging scanners, computed tomography scanners, radar imagers, sonar imagers, motion pictures cameras, and sensors for producing electronic representations of smell and taste.

25. The apparatus as recited in claim 14 wherein the recording medium is selected from a group consisting of an analog recording medium, a photographic film, an analog video tape, a digital recording medium, a random access memory, a compact disc, a digital video disc, a digital video tape, a magnetic recording medium, an optical recording medium or magneto-optical recording medium.

26. The apparatus as recited in claim 14 further comprising a digitizer for digitizing the image and associated parameters.

27. The apparatus as recited in claim 14 wherein the description database is located on a local storage medium.

28. The apparatus as recited in claim 14 wherein the database is located on a remote computer connected to the computer 21.

14. Apparatus for generating the content of an image comprising the steps of:

(a) a sensor for capturing an image,

(b) a parameter sensor for sensing parameters associated with an image wherein the parameters include the parameters describing the location of the sensor when the image is captured,

(c) a recording medium for recording the image and associated parameters;

(d) a description database containing descriptions of the contents of images indexed to the parameters in (b),

(e) an image display system comprising a computer, a monitor and at least one computer program 23, for viewing the image, for reading in the parameters, for querying the description database in (d), for generating a description of the content of the image, and for presenting the description of the image to the user.