Image and text searching techniques

• US 6,522,782 B2
• Filed: 01/31/2001
• Issued: 02/18/2003
• Est. Priority Date: 12/15/2000
• Status: Expired due to Term

• Current Assignee: Meta Platforms, Inc. (f/k/a Facebook, Inc.)
• Original Assignee: America Online Inc. (Warner Bros. Discovery, Inc.)
• Inventors: Pass, Gregory S., Wood, Frank
• Primary Examiner(s): Tran, Phuoc
• Assistant Examiner(s): SHERALI, ISHRAT I
• Application Number:
  US09/772,903
• Publication Number:
  US 20020090148A1
• Time in Patent Office: 748 Days
• Field of Search: 382/218, 382/209, 382/219, 382/220, 382/168, 382/170, 382/162, 382/165, 382/224, 382/305, 382/108, 358/403, 358/522, 358/518, 707/1-6, 707/104.1, 707/520, 707/522, 348/515-516, 348/403.1, 345/700, 345/738, 345/788, 345/835, 345/419, 345/589
• US Class Current: 382/218
• CPC Class Codes: G06F 16/5838 using colour
• CPC Class Codes: Y10S 707/99936 Pattern matching access

Abstract

Conducting a search for both text and images includes receiving input, which may include one or more search criteria for at least one of searchable image characteristics and text terms. The received input may be compared with searchable image characteristics and text terms that have been indexed in a single format for multiple stored images and text segments. A search result is produced based on the comparison. The searchable image characteristics and text terms may be indexed in a uniform data format that enables a search engine to use the same search methodology to search for both images and text. The uniform data format may be an inverted index data format. The produced search result may include identifiers for one or more images and text segments that have been retrieved for display in a ranked manner based on similarity metrics.

Claims

1. A method for conducting a search for both text and images, the method comprising:

receiving input including one or more search criteria for at least one of searchable image characteristics and searchable text terms, wherein the searchable image characteristics are represented by joint features of a joint histogram;

comparing the input with searchable image characteristics and searchable text terms that have been indexed in a single format for multiple stored images and text segments, wherein the indexed searchable image characteristics include individual joint features to which associated stored images correspond; and

producing a search result based on the comparison.

2. The method as in claim 1 wherein the searchable image characteristics and the searchable text terms are indexed in a uniform data format that enables a search engine to use the same search methodology to search for both images and text.

3. The method as in claim 2 wherein the uniform data format comprises an inverted index data format.

4. The method as in claim 1 wherein producing a search result includes retrieving identifiers for one or more images and text segments to be displayed in a ranked manner.

5. The method as in claim 4 wherein retrieving the identifiers includes determining a relative ranking among one or more of the images and the text segments based on similarity metrics.

6. The method as in claim 1 wherein comparing the input includes a single comparison of the input with searchable image characteristics and searchable text terms that have been indexed in the single format for multiple stored images and text segments.

7. The method as in claim 1 wherein comparing the input includes:

a first comparison of the input with searchable image characteristics that have been indexed in the single format for multiple stored images and text segments; and

a second comparison of the input with searchable text terms that have been indexed in the single format for multiple stored images and text segments.

8. The method as in claim 1 wherein the multiple stored images and text segments are stored in a single electronic data store.

9. The method as in claim 1 wherein the multiple stored images are stored in a first electronic data store and the multiple text segments are stored in a second electronic data store that differs from the first electronic data store.

28. The method as in claim 1 wherein:

receiving the input includes receiving input including one or more search criteria for at least one of searchable image characteristics and searchable text terms, wherein the searchable image characteristics are represented by posterized joint features of a posterized joint histogram; and

comparing the input includes comparing the input with searchable image characteristics and searchable text terms that have been indexed in a single format for multiple stored images and text segments, wherein the indexed searchable image characteristics include individual posterized joint features to which associated stored images correspond.

10. A computer program for conducting a search for both text and images, the computer program being stored on a computer readable medium or a propagated signal and comprising:

a receiving code segment that causes the computer to receive input including one or more search criteria for at least one of searchable image characteristics and text terms, wherein the searchable image characteristics are represented by joint features of a joint histogram;

a comparing code segment that causes the computer to compare the input with searchable image characteristics and searchable text terms that have been indexed in a single format for multiple stored images and text segments, wherein the indexed searchable image characteristics include individual joint features to which associated stored images correspond; and

a producing code segment that causes the computer to produce a search result based on the comparison.

11. The computer program of claim 10 wherein the searchable image characteristics and the searchable text terms are indexed in a uniform data format that enables a search engine to use the same search methodology to search for both images and text.

12. The computer program of claim 11 wherein the uniform data format comprises an inverted index data format.

13. The computer program of claim 10 wherein the producing code segment includes a retrieving code segment that causes the computer to retrieve identifiers for one or more images and text segments to be displayed in a ranked manner.

14. The computer program of claim 13 wherein the retrieving code segment includes a determining code segment that causes the computer to determine a relative ranking among one or more of the images and the text segments based on similarity metrics.

15. The computer program of claim 10 wherein the comparing code segment causes the computer to perform a single comparison of the input with searchable image characteristics and searchable text terms that have been indexed in the single format for multiple stored images and text segments.

16. The computer program of claim 10 wherein the comparing code segment causes the computer to:

perform a first comparison of the input with searchable image characteristics that have been indexed in the single format for multiple stored images and text segments; and

perform a second comparison of the input with searchable text terms that have been indexed in the single format for multiple stored images and text segments.

17. The computer program of claim 10 wherein the multiple stored images and text segments are stored in a single electronic data store.

18. The computer program of claim 10 wherein the multiple stored images are stored in a first electronic data store and the multiple text segments are stored in a second electronic data store that differs from the first electronic data store.

29. The computer program of claim 10 wherein:

the receiving code segment causes the computer to receive input including one or more search criteria for at least one of searchable image characteristics and searchable text terms, wherein the searchable image characteristics are represented by posterized joint features of a posterized joint histogram; and

the comparing code segment causes the computer to compare the input with searchable image characteristics and searchable text terms that have been indexed in a single format for multiple stored images and text segments, wherein the indexed searchable image characteristics include individual posterized joint features to which associated stored images correspond.

19. A system for conducting a search for both text and images, the system comprising:

a receiving component that is structured and arranged to receive input including one or more search criteria for at least one of searchable image characteristics and searchable text terms, wherein the searchable image characteristics are represented by joint features of a joint histogram;

a comparing component that is structured and arranged to compare the input with searchable image characteristics and searchable text terms that have been indexed in a single format for multiple stored images and text segments, wherein the indexed searchable image characteristics include individual joint features to which associated stored images correspond; and

a producing component that is structured and arranged to produce a search result based on the comparison.

20. The system of claim 19 wherein the searchable image characteristics and the searchable text terms are indexed in a uniform data format that enables a search engine to use the same search methodology to search for both images and text.

21. The system of claim 20 wherein the uniform data format comprises an inverted index data format.

22. The system of claim 19 wherein the producing component includes a retrieving component that is structured and arranged to retrieve identifiers for one or more images and the text segments to be displayed in a ranked manner.

23. The system of claim 22 wherein the retrieving component includes a determining component that is structured and arranged to determine a relative ranking among one or more of the images and the text segments based on similarity metrics.

24. The system of claim 19 wherein the comparing component performs a single comparison of the input with searchable image characteristics and searchable text terms that have been indexed in the single format for multiple stored images and text segments.

25. The system of claim 19 wherein the comparing component performs:

a first comparison of the input with searchable image characteristics that have been indexed in the single format for multiple stored images and text segments; and

a second comparison of the input with searchable text terms that have been indexed in the single format for multiple stored images and text segments.

26. The system of claim 19 wherein the multiple stored images and text segments are stored in a single electronic data store.

27. The system of claim 19 wherein the multiple stored images are stored in a first electronic data store and the multiple text segments are stored in a second electronic data store that differs from the first electronic data store.

30. The system of claim 19 wherein:

the receiving component is structured and arranged to receive input including one or more search criteria for at least one of searchable image characteristics and searchable text terms, wherein the searchable image characteristics are represented by posterized joint features of a posterized joint histogram; and

the comparing component is structured and arranged to compare the input with searchable image characteristics and searchable text terms that have been indexed in a single format for multiple stored images and text segments, wherein the indexed searchable image characteristics include individual posterized joint features to which associated stored images correspond.