Information search system

• US 20020019819A1
• Filed: 06/22/2001
• Published: 02/14/2002
• Est. Priority Date: 06/23/2000
• Status: Abandoned Application

• Current Assignee: NTT Docomo Incorporated (Nippon Telegraph and Telephone Corporation)
• Original Assignee: NTT Docomo Incorporated (Nippon Telegraph and Telephone Corporation)
• Inventors: Nakamura, Norio, Kawahara, Toshiro, Sekiguchi, Shunichi
• Application Number:
  US09/888,998
• Publication Number:
  US 20020019819A1
• Time in Patent Office: Days
• US Class Current: 707/3
• CPC Class Codes: G06F 16/5838 using colour
• CPC Class Codes: G06V 10/50 by performing operations within image blocks; by using histograms, e.g. histogram of oriented gradients [HoG]; by summing image-intensity values; Projection analysis
• CPC Class Codes: G06V 10/7515 Shifting the patterns to accommodate for positional errors

Abstract

A mobile terminal is provided with an imaging unit for acquiring a plurality of frames of images including a search object and a feature descriptor generating unit for generating a feature descriptor from an image of a search object included in the frames. The mobile terminal transmits the feature descriptor generated by the feature descriptor generating unit to a search server. The search server has a matching processing unit. The matching processing unit searches for a record matching with the feature descriptor received from the mobile terminal from a database of the content server.

Claims

1. An information search system comprising:

a database linking and storing records relating to various search objects and feature descriptors serving as conditions for selection of the same;

an image acquiring unit for acquiring an image including a search object;

a feature descriptor generating unit for generating a feature descriptor from an image of a search object included in an image acquired by said image acquiring unit; and

a matching processing unit for searching for a record relating to a search object matching a feature descriptor generated by said feature descriptor generating unit from said database.

2. An information search system as set forth in claim 1, further comprising:

a terminal having said image acquiring unit and said feature descriptor generating unit and transmitting a feature descriptor generated by said feature descriptor generating unit through a network; and

a search server provided with said matching processing unit, receiving said feature descriptor from said terminal through said network, searching for a record matching said feature descriptor from said database, and transmitting it to said terminal.

3. An information search system as set forth in claim 1, further comprising:

a terminal having said image acquiring unit and transmitting an image acquired by said image acquiring unit through a network; and

a search server provided with said feature descriptor generating unit and said matching processing unit, receiving an image from said terminal through the network, generating a feature descriptor from said image by said feature feature descriptor generating unit, searching for a record matching said feature descriptor from said database, and transmitting it to said terminal.

4. An information search system as set forth in any one of claims 1 to 3, wherein

said image acquiring unit acquires a plurality of frames of images comprising a moving image and

said feature descriptor generating unit generates a feature descriptor from an image of said search object included in each of the plurality of frames of images.

5. An information search system as set forth in any one of claims 1 to 4, wherein said image acquiring unit is an imaging unit for obtaining an image of the outside world.

6. An information search system as set forth in any one of claims 1 to 4, wherein said image acquiring unit is a receiving unit for receiving an image through a network.

7. An information search system as set forth in claim 2 or 3, wherein said terminal is a mobile terminal.

8. An information search system as set forth in any one of claims 1 to 7, wherein

said feature descriptor generating unit outputs search control data specifying a search condition and

said matching processing unit searches for information in accordance with said search control data.

9. An information search system as set forth in claim 2 or 3, wherein

said terminal is provided with a unit for transmitting search request data specifying a synchronous search or asynchronous search,

said search server searches for information using the feature descriptor as a search key and transmitting data of the result of the search to said terminal each time a feature descriptor corresponding to a frame is obtained when search request data specifying said synchronous search is received, and searches for information using the feature descriptor as a search key and transmitting data of the result of the search to said terminal when a feature descriptor corresponding to a predetermined number of frames is obtained when search request data specifying said asynchronous search is received.

10. An information search system as set forth in claim 2 or 3, wherein

said terminal is provided with a unit for transmitting search request data specifying a search range and

said search server searches for information in the search range specified by said search request data.

11. An information search system as set forth in any one of claims 1 to 10, wherein

said feature descriptor generating unit generates a plurality of types of feature descriptor sets comprised of feature descriptors from an image of a search object and

said matching processing unit searches for a record relating to a search object matching said feature descriptor set from said database.

12. An information search system as set forth in claim 11, wherein said feature descriptor generating unit selects one of the plurality of types of feature descriptor sets, generates the selected feature descriptor set from an image of a search object, and notifies said matching processing unit of feature descriptor set selection information indicating the selected feature descriptor set.

13. An information search system as set forth in any one of claims 1 to 12, wherein

said feature descriptor generating unit generates a plurality of types of feature descriptors and determines priorities of the feature descriptors and

said matching processing unit searches for information using said feature descriptors as search keys in accordance with their priorities.

14. An information search system as set forth in claim 13, wherein said feature descriptor generating unit determines priorities of said feature descriptors in accordance with description precisions of said feature descriptors.

15. An information search system as set forth in claim 13 or 14, wherein said feature descriptor generating unit determines priorities of said feature descriptors based on a specification by a user.

16. An information search system as set forth in any one of claims 1 to 15, wherein

said system is further provided with a sensor and

said matching processing unit searches for information using sensing data obtained from said sensor as a search key along with said feature descriptors.

17. An information search system as set forth in claim 16, wherein said sensor is a position sensor outputting sensing data indicating a position of location.

18. An information search system as set forth in any one of claims 1 to 17, wherein said feature descriptor generating unit negotiates with said matching processing unit to determine a feature descriptor generated from an image of the search object.

19. An information search system as set forth in any one of claims 1 to 18, wherein

said system is further provided with an image encoding unit for encoding an acquired image and

said feature descriptor generating unit generates a feature descriptor from data generated in the process of encoding an image in said image encoding unit.

20. An information search system as set forth in any one of claims 1 to 19, wherein

said system is further provided with a unit for specifying an object region which a search object occupies in an image acquired by said image acquiring unit,

said image encoding unit performs intraframe encoding on an initial frame and repeatedly performs interframe predictive encoding with motion compensation on subsequent frames when generating a feature descriptor, and

said feature descriptor generating unit generates a feature descriptor corresponding to a search object from data obtained in the process of intraframe encoding and estimates an object region of a search object from a motion vector obtained in the process of interframe predictive encoding and generates a feature descriptor when interframe predictive encoding is performed.

21. An information search system as set forth in any one of claims 1 to 19, wherein

said system is further provided with a unit for specifying an object region which a search object occupies in an image acquired by said image acquiring unit,

said image encoding unit performs intraframe encoding when generating a feature descriptor, and

said feature descriptor generating unit generates a feature descriptor corresponding to a search object from data obtained in the process of intraframe encoding.

22. A communication terminal comprising:

a communication unit for communicating with a search server in a network;

an image acquiring unit for acquiring an image including a search object;

a feature descriptor generating unit for generating a feature descriptor from an image of a search object included in the image acquired by the image acquiring unit;

a unit for transmitting the feature descriptor generated by said feature descriptor generating unit to the search server through the communication unit, and receiving a record relating to a search object matching the feature descriptor generated by the feature descriptor generating unit through the communication unit.

23. A communication terminal comprising:

a communication unit for communicating with a search server in a network;

an image acquiring unit for acquiring an image including a search object;

a feature descriptor generating unit for transmitting performance exchange information which includes types of feature descriptors able to be provided through the communication unit, receiving information relating to search results to be obtained by a search using the feature descriptors from the search server through the communication unit, determining a feature descriptor for information search based on the received information, generating a feature descriptor corresponding to the determined feature descriptor from an image of a search object included in the image acquired by the image acquiring unit; and

transmitting the feature descriptor to the search server through the communication unit; and

a unit for receiving a search result from the search server through the communication unit.

24. A search server comprising:

a communication unit for communicating with a communication terminal in a network;

a matching processing unit for receiving performance exchange information which includes types of feature descriptors able to be provided from the communication terminal through the communication unit, transmitting information relating to search results to be obtained by a search using the feature descriptors corresponding to the performance exchange information, and searching for a record relating to a search object matching a feature descriptor when receiving the feature descriptor from the communication terminal through the communication unit; and

a unit for transmitting the record searched to the communication terminal through the communication unit.