System and method of organizing images

• US 8,862,582 B2
• Filed: 11/15/2007
• Issued: 10/14/2014
• Est. Priority Date: 11/15/2007
• Status: Active Grant

• Current Assignee: AT&T Labs Incorporated (AT&T, Inc.)
• Original Assignee: AT&T Intellectual Property I LP (AT&T, Inc.)
• Inventors: Blewett, Charles, Bocchieri, Enrico, Di Fabbrizio, Giuseppe, Henderson, Donnie, Killian, Thomas, Kirk, Thomas, Kormann, David, Vesonder, Gregory T.
• Primary Examiner(s): Chbouki, Tarek
• Application Number:
  US11/940,703
• Publication Number:
  US 20090132467A1
• Time in Patent Office: 2,525 Days
• Field of Search: None
• US Class Current: 707/736
• CPC Class Codes: G06F 16/50 of still image data
• CPC Class Codes: G06F 16/58 Retrieval characterised by using metadata, e.g. metadata not derived from the content or metadata generated manually
• CPC Class Codes: G06F 16/587 using geographical or spatial information, e.g. location
• CPC Class Codes: G06F 16/951 Indexing; Web crawling techniques

Abstract

Disclosed are a system, method and computer-readable medium for organizing images. A method aspect relates to receiving an image into a device, receiving incidental information associated with the image, organizing the image and the incidental information into a data structure such as a sparse array, classifying the received image with an image classifier and storing the classified image in an image database, receiving a search query and responding to the search query by searching for and retrieving matching images in the image database based on a comparison of the image search query to the data structure.

Claims

1. A method comprising:

receiving scene description information associated with an image, the scene description information comprising semantic data for a plurality of objects within the image;

assigning a weight to each object in the plurality of objects based on the semantic data, the weight being independent of a geographic location, to yield weighted scene description information;

organizing, via a processor, the image and the weighted scene description information into a data structure, wherein the data structure comprises, for each object in the plurality of objects, a storage array as an infinite array of the each object;

classifying the image based on the data structure to, yield a classified image;

modifying the weight of a specific object in the plurality of objects based on user search preferences and repeated searching for the specific object, to yield a modified object weight;

storing the modified object weight for the specific object in the infinite array for the specific object; and

upon receiving a search query, responding to the search query by returning matching images in the image database based on a comparison of the search query to the data structure using the modified object weight.

2. The method of claim 1, wherein the scene description information comprises one of environmental data, chronologically relevant audio input to a device, network time difference of arrival data, and device orientation data.

3. The method of claim 2, wherein environmental data comprises one of time, color, object detection, temperature, received audio when the image was received by the device, and audio.

4. The method of claim 1, further comprising:

after storing the classified image in the image database, upon retrieval of the image from the image database, receiving second scene description information associated with the image and storing the second scene description information into the data structure.

5. The method of claim 1, further comprising during a later viewing by a user of the image stored in the image database, receiving second scene description information associated with the image and automatically assigning a modified weight to a portion of the second scene description information, when the second scene description information is stored in the data structure.

6. The method of claim 5, wherein the modified weight assigned to the second scene description information is based on a parameter that relates data associated with the scene description information to the second scene description information.

7. The method of claim 1, wherein the image classifier is trained using personal data of a user who is associated with a device that receives the image.

8. The method of claim 1, wherein the image is a video.

9. A system comprising:

a processor; and

a computer-readable storage medium having instructions stored which, when executed by the processor, cause the processor to perform operations comprising;

receiving scene description information associated with an image, the scene description information comprising semantic data for a plurality of objects within the images;

assigning a weight to each object in the plurality of objects based on the semantic data, the weight being independent of a geographic location, to yield weighted scene description information;

organizing the image and the weighted scene description information into a data structure, wherein the data structure comprises, for each object in the plurality of objects, a storage array as an infinite array of the each object;

classifying the image based on the data structure to, yield a classified image;

modifying the weight of a specific object in the plurality of objects based on user search preferences and repeated searching for the specific object, to yield a modified object weight;

storing the modified object weight for the specific object in the infinite array for the specific object; and

upon receiving a search query, responding to the search query by returning matching images in the image database based on a comparison of the search query to the data structure using the modified object weight.

10. The system of claim 9, wherein the scene description information comprises one of environmental data, chronologically relevant audio input to a network time difference of arrival data, and device orientation data.

11. The system of claim 10, wherein the environmental data comprises one of time, color, object detection, temperature, received audio when the image was received by a device, and audio.

12. The system of claim 9, the computer-readable storage medium having additional instructions stored which result in the operations further comprising:

after storing the classified image in the database and upon retrieval of the image from the image database, receiving second scene description information associated with the image and storing the second scene description information into the data structure.

13. The system of claim 9, the computer-readable storage medium having additional instructions stored which result in the operations further comprising:

during a later viewing by a user of the image stored in the image database, receiving second scene description information associated with the image and automatically assigning a reduced weight to a portion of the second scene description information wherein the second scene description information is stored in the data structure.

14. A computer-readable storage device having instructions stored therein, which, when executed by a computing device, cause the computing device to perform operations comprising:

receiving scene description information associated with an image, the scene description information comprising semantic data for a plurality of objects within the image;

assigning a weight to each object in the plurality of objects based on the semantic data, the weight being independent of a geographic location, to yield weighted scene description information;

organizing the image and the weighted scene description information into a data structure, wherein the data structure comprises, for each object in the plurality of objects, a storage array as an infinite array of the each object;

classifying the image based on the data structure to, yield a classified image;

modifying the weight a specific object in the plurality of objects based on user search preferences and repeated searching for the specific object, to yield a modified object weight;

storing the modified object weight for the specific object in the infinite array for the specific object; and

upon receiving a search query, responding to the search query by returning matching images in the image database based on a comparison of the search query to the data structure using the modified object weight.

15. The computer-readable storage device of claim 14, wherein the scene description information comprises one of environmental data, chronologically relevant audio input to a network time difference of arrival data, and device orientation data.

16. The computer-readable storage device of claim 15, wherein the environmental data comprises one of time, color, object detection, temperature, received audio when the image was received by a device, and audio.

17. The computer-readable storage device of claim 14, having additional instructions stored which result in the operations further comprising:

after storing the classified image in the image database and upon retrieval of the image from the image database, receiving second scene description information associated with the image and storing the second scene description information into the data structure.

18. The computer-readable storage device of claim 14, having additional instructions stored which result in the operations further comprising:

during a later viewing by a user of the image stored in the image database, receiving second scene description information associated with the image and automatically assigning a reduced weight to a portion of the second scene description information wherein the second scene description information is stored in the data structure.

19. The computer-readable storage device of claim 18, wherein the reduced weight assigned to the second scene description information is based on one parameter that relates data associated with the scene description information to the second scene description information.