Digital pictorial book system, a pictorial book searching method, and a machine readable medium storing thereon a pictorial book searching program

• US 7,639,899 B2
• Filed: 02/22/2005
• Issued: 12/29/2009
• Est. Priority Date: 02/20/2004
• Status: Active Grant

• Current Assignee: Fujifilm Corporation (Fujifilm Holdings Corporation)
• Original Assignee: Fujifilm Corporation (Fujifilm Holdings Corporation)
• Inventors: Isomura, Ayumu, Igarashi, Takashi
• Primary Examiner(s): Werner; Brian P
• Assistant Examiner(s): Patel; Jayesh
• Application Number:
  US11/061,927
• Publication Number:
  US 20050185862A1
• Time in Patent Office: 1,771 Days
• Field of Search: 382/305, 705/1, 707/1
• US Class Current: 382/305
• CPC Class Codes: G06F 16/5838 using colour
• CPC Class Codes: G06F 16/5854 using shape and object relationship
• CPC Class Codes: G06V 30/242 Division of the character sequences into groups prior to recognition; Selection of dictionaries
• CPC Class Codes: Y10S 707/99931 Database or file accessing

Abstract

A digital pictorial book system for searching for and providing a user with an explanation of an object captured by an image capturing module includes an image capturing module for capturing an image; a main object selecting module for selecting a main object out of the image; a feature extracting module for extracting a feature of the main object; an explanation searching module for searching for the explanation of the main object in a plurality of image databases, which store explanations of the objects corresponded to a plurality of different kinds of features of the objects by using the feature extracted by the feature extracting module; and a distinguishing feature selecting module for selecting a distinguishing feature, of which an overlap of certainty distributions is the smallest for each object and each kind of the feature out of the different kinds of features stored in the image database corresponded to the plurality of objects in case the explanations of the plurality of objects are searched out by the explanation searching module, in order to narrow down the candidates even in case that a plurality of candidates is searched out in the image databases.

Claims

1. A digital pictorial book system for searching for and providing a user with an explanation of an object captured by an image capturing sensor, the digital pictorial book system comprising:

an image capturing sensor for capturing an image;

a display that displays the image captured by the image capturing sensor;

a main object selecting means for selecting a main object out of the image;

a feature extracting means for extracting a feature of the main object;

an explanation searching means for searching for an explanation of the main object in a plurality of image databases, which store explanations of objects corresponding to a plurality of different kinds of features of the objects by using the feature extracted by said feature extracting means; and

a distinguishing feature selecting means for selecting a distinguishing feature, of which an overlap of certainty distributions is the smallest for each object and each kind of the feature out of the different kinds of features stored in the plurality of image databases corresponding to the plurality of objects in case the explanations of the plurality of objects are searched out by said explanation searching means,

wherein the distinguishing feature is the feature having a largest probability difference between a probability of a quantity of the feature extracted by said feature extracting means existing in an object corresponding to one of the plurality of explanations searched out by said explanation searching means and a highest probability among probabilities of the quantity of the feature extracted by said feature extracting means existing in other objects corresponding to the plurality of explanations searched out by said explanation searching means.

2. The digital pictorial book system as claimed in claim 1, further comprising a second feature extracting module for extracting a feature of an object other than the main object out of the image,

wherein said explanation searching means searches for the explanation of the main object by using the feature extracted by said second feature extracting module in the plurality of image databases which store the explanation of the object further corresponded to the feature of a thing of high relevance with the object.

3. The digital pictorial book system as claimed in claim 1, further comprising annexed information acquiring module for acquiring annexed information annexed to the image,

wherein said explanation searching means searches for the explanation of the main object by using the annexed information acquired by said annexed information acquiring module in the plurality of image databases which store the explanation of the object further corresponded to the annexed information corresponded to the image of the object.

4. The digital pictorial book system as claimed in claim 1, further comprising an informing means for informing the user of said digital pictorial book system, of a content of the distinguishing feature selected by said distinguishing feature selecting means.

5. The digital pictorial book system as claimed in claim 4, wherein said distinguishing feature shows a part of the object, and

said informing means informs the user that the part of the main object should be captured.

6. The digital pictorial book system as claimed in claim 5,

wherein said image database comprises an image of each part of the object as a feature, and

said informing means displays the image of the part, which is the distinguishing feature, on said display.

7. The digital pictorial book system as claimed in claim 5 further comprising a featuring part searching means for searching for the image showing the part of the main object,

wherein said informing means displays a frame which surrounds the part on said display.

8. The digital pictorial book system as claimed in claim 7, wherein said informing means informs the user of information showing an image capturing method for capturing the image comprising the part by using said image capturing sensor in case the image showing the part is searched out in the main object by said featuring part searching means.

9. The digital pictorial book system as claimed in claim 8, wherein the information showing the image capturing method comprises information showing the image capturing direction of said image capturing sensor.

10. The digital pictorial book system as claimed in claim 8, wherein the information showing the image capturing method comprises information showing a position of said image capturing sensor.

11. The digital pictorial book system as claimed in claim 8, wherein the information showing the image capturing method comprises information showing image capturing magnification of said image capturing sensor.

12. The digital pictorial book system as claimed in claim 5, wherein said informing means informs the user of the digital pictorial book system of the content of the distinguishing feature selected by said distinguishing feature selecting means by using a voice.

13. The digital pictorial book system as claimed in claim 5,

wherein said image database further stores a dangerous thing information, which shows whether or not the object is a highly dangerous thing, corresponded to a plurality of different kinds of features of the objects, and

said informing means informs the user that the object is a highly dangerous thing in case it is shown by the dangerous thing information corresponded to the object of which the explanation is searched out by said explanation searching means.

14. The digital pictorial book system as claimed in claim 1, further comprising an image capturing control means for controlling the operation of said image capturing sensor on the basis of the content of the distinguishing feature selected by said distinguishing feature selecting means.

15. The digital pictorial book system as claimed in claim 14 further comprising a featuring part searching means for searching for an image showing a part of the main object,

wherein the distinguishing feature shows the part of the object, and

said image capturing control means controls the operation of said image capturing sensor on the basis of the image showing the part searched out by said featuring part searching means.

16. The digital pictorial book system as claimed in claim 15, wherein said image capturing control means controls the image capturing direction of said image capturing sensor in order for the part to be comprised in the image capturing range of said image capturing sensor in case the image showing the part is not searched out by said featuring part searching means in the main object.

17. The digital pictorial book system as claimed in claim 15, wherein said image capturing control means controls the position of said image capturing sensor in order for the part to be comprised in the image capturing range of said image capturing sensor in case the image showing the part is not searched out by said featuring part searching means in the main object.

18. The digital pictorial book system as claimed in claim 15, wherein said image capturing control means increases the image capturing magnification of said image capturing sensor in case a ratio of a size of the image showing the part searched out by the featuring part searching means to that of the whole image captured by said image capturing sensor is smaller than a predetermined reference value.

19. A digital pictorial book system for searching for and providing a user with an explanation of an object captured by an image capturing sensor, the digital pictorial book system comprising:

an image capturing sensor for capturing an image;

a display that displays the image captured by the image capturing sensor;

a main object selecting means for selecting a main object out of the image;

a feature extracting means for extracting a feature of the main object;

an explanation searching means for searching for an explanation of the main object in a plurality of image databases, which store explanations of the objects corresponding to a plurality of different kinds of features of the objects by using the feature extracted by said feature extracting means; and

a distinguishing feature selecting means for selecting a distinguishing feature, of which an overlap of certainty distributions is the smallest for each object and each kind of the feature out of the different kinds of features stored in the plurality of image databases corresponding to the plurality of objects in case the explanations of the plurality of objects are searched out by said explanation searching means,

wherein the certainty distributions indicate a relationship between a quantity of the feature extracted by said feature extracting means and a probability of the quantity of the feature extracted by said feature extracting means existing in the objects.