Digital pictorial book system, a pictorial book searching method, and a machine readable medium storing thereon a pictorial book searching program

• US 20050185862A1
• Filed: 02/22/2005
• Published: 08/25/2005
• Est. Priority Date: 02/20/2004
• Status: Active Grant

• Current Assignee: Fujifilm Corporation (Fujifilm Holdings Corporation)
• Original Assignee: Fuji Photo Film Co Limited (Fujifilm Holdings Corporation)
• Inventors: Isomura, Ayumu, Igarashi, Takashi
• Granted Patent: US 7,639,899 B2
• Time in Patent Office: Days
• US Class Current: 382/305
• CPC Class Codes: G06F 16/5838 using colour
• CPC Class Codes: G06F 16/5854 using shape and object relationship
• CPC Class Codes: G06V 30/242 Division of the character sequences into groups prior to recognition; Selection of dictionaries
• CPC Class Codes: Y10S 707/99931 Database or file accessing

Abstract

A digital pictorial book system for searching for and providing a user with an explanation of an object captured by an image capturing module includes an image capturing module for capturing an image; a main object selecting module for selecting a main object out of the image; a feature extracting module for extracting a feature of the main object; an explanation searching module for searching for the explanation of the main object in a plurality of image databases, which store explanations of the objects corresponded to a plurality of different kinds of features of the objects by using the feature extracted by the feature extracting module; and a distinguishing feature selecting module for selecting a distinguishing feature, of which an overlap of certainty distributions is the smallest for each object and each kind of the feature out of the different kinds of features stored in the image database corresponded to the plurality of objects in case the explanations of the plurality of objects are searched out by the explanation searching module, in order to narrow down the candidates even in case that a plurality of candidates is searched out in the image databases.

Claims

1. A digital pictorial book system for searching for and providing a user with an explanation of an object captured by an image capturing module comprising:

an image capturing module for capturing an image;

a main object selecting module for selecting a main object out of the image;

a feature extracting module for extracting a feature of the main object;

an explanation searching module for searching for the explanation of the main object in a plurality of image databases, which store explanations of the objects corresponded to a plurality of different kinds of features of the subjects by using the feature extracted by said feature extracting module; and

a distinguishing feature selecting module for selecting a distinguishing feature, of which an overlap of certainty distributions is the smallest for each object and each kind of the feature out of the different kinds of features stored in the image database corresponded to the plurality of objects in case the explanations of the plurality of objects are searched out by said explanation searching module.

2. A digital pictorial book system as claimed in claim 1 further comprising a second feature extracting module for extracting a feature of an object other than the main object out of the image,

wherein said explanation searching module searches for the explanation of the main object by using the feature extracted by said second feature extracting module in the plurality of image databases which store the explanation of the object further corresponded to the feature of a thing of high relevance with the object.

3. A digital pictorial book system as claimed in claim 1 further comprising annexed information acquiring module for acquiring annexed information annexed to the image,

wherein explanation searching module searches for the explanation of the main object by using the annexed information acquired by said annexed information acquiring module in the plurality of image databases which store the explanation of the object further corresponded to the annexed information corresponded to the image of the object.

4. A digital pictorial book system as claimed in claim 1 further comprising an informing module for informing the user of said digital pictorial book system, of a content of the distinguishing feature selected by said distinguishing feature selecting module.

5. A digital pictorial book system as claimed in claim 4, wherein said distinguishing feature shows a part of the object, and

said informing module informs the user that the part of the main object should be captured.

6. A digital pictorial book system as claimed in claim 5 further comprising a display module for displaying the image captured by said image capturing module,

wherein said image database comprises an image of each part of the object as a feature, and

said informing module displays the image of the part, which is the distinguishing feature, on said display module.

7. A digital pictorial book system as claimed in claim 5 further comprising a featuring part searching module for searching for the image showing the part of the main object, wherein said informing module displays a frame which surrounds the part on said displaying module.

8. A digital pictorial book system as claimed in claim 7, wherein said informing module informs the user of information showing an image capturing method for capturing the image comprising the part by using said image capturing module in case the image showing the part is searched out in the main object by said featuring part searching module.

9. A digital pictorial book system as claimed in claim 8, wherein the information showing the image capturing method comprises information showing the image capturing direction of said image capturing module.

10. A digital pictorial book system as claimed in claim 8, wherein the information showing the image capturing method comprises information showing a position of said image capturing module.

11. A digital pictorial book system as claimed in claim 8, wherein the information showing the image capturing method comprises information showing image capturing magnification of said image capturing module.

12. A digital pictorial book system as claimed in claim 5, wherein said informing module informs the user of the digital pictorial book system of the content of the distinguishing feature selected by said distinguishing feature selecting module by using a voice.

13. A digital pictorial book system as claimed in claim 1 further comprising an image capturing control module for controlling the operation of said image capturing module on the basis of the content of the distinguishing feature selected by said distinguishing feature selecting module.

14. A digital pictorial book system as claimed in claim 13 further comprising a featuring part searching module for searching for an image showing a part of the main object,

wherein the distinguishing feature shows the part of the object, and

said image capturing control module controls the operation of said image capturing module on the basis of the image showing the part searched out by said featuring part searching module.

15. A digital pictorial book system as claimed in claim 14, wherein said image capturing control module controls the image capturing direction of said image capturing module in order for the part to be comprised in the image capturing range of said image capturing module in case the image showing the part is not searched out by said featuring part searching module in the main object.

16. A digital pictorial book system as claimed in claim 14, wherein said image capturing control module controls the position of said image capturing module in order for the part to be comprised in the image capturing range of said image capturing module in case the image showing the part is not searched out by said featuring part searching module in the main object.

17. A digital pictorial book system as claimed in claim 14, wherein said image capturing control module increases the image capturing magnification of said image capturing module in case a ratio of a size of the image showing the part searched out by the featuring part searching module to that of the whole image captured by said image capturing module is smaller than a predetermined reference value.

18. A digital pictorial book system as claimed in claim 5,

wherein said image database further stores a dangerous thing information, which shows whether or not the object is a highly dangerous thing, corresponded to a plurality of different kinds of features of the objects, and

said informing module informs the user that the object is a highly dangerous thing in case it is shown by the dangerous thing information corresponded to the object of which the explanation is searched out by said explanation searching module.

19. A digital pictorial book searching method performed by a digital pictorial book system for searching for and providing a user with an explanation of an object captured by an image capturing module comprising:

an image capturing step of capturing an image;

a main object selecting step of selecting a main object out of the image;

a feature extracting step of extracting a feature of the main object;

an explanation searching step of searching for the explanation of the main object in a plurality of image databases, which store explanations of the objects corresponded to a plurality of different kinds of features of the objects by using the feature extracted by said feature extracting step;

a distinguishing feature selecting step of selecting a distinguishing feature, of which an overlap of certainty distributions is the smallest for each object and each kind of the feature out of the different kinds of features stored in the image database corresponded to the plurality of objects in case the explanations of the plurality of objects are searched out by said explanation searching step; and

an informing step of informing the user of said digital pictoria. book system, of a content of the distinguishing feature selected by said distinguishing feature selecting step.

20. A digital pictorial book searching method as claimed in claim 19 further comprising a featuring part searching step of searching for the image showing the part of the main object,

wherein, during said informing step, a frame which surrounds the part is displayed in said displaying step.

21. A machine readable medium storing thereon a computer program making a computer perform as a digital pictorial book system for searching for and providing a user with an explanation of an object captured by an image capturing module, said digital pictorial book system comprising:

an image capturing module for capturing an image;

a main object selecting module for selecting a main object out of the image;

a feature extracting module for extracting a feature of the main object;

an explanation searching module for searching for the explanation of the main object in a plurality of image databases, which store explanations of the objects corresponded to a plurality of different kinds of features of the objects by using the feature extracted by said feature extracting module;

a distinguishing feature selecting module for selecting a distinguishing feature, of which an overlap of certainty distributions is the smallest for each object and each kind of the feature out of the different kinds of features stored in the image database corresponded to the plurality of objects in case the explanations of the plurality of objects are searched out by said explanation searching module; and

an informing module for informing the user of said digital pictorial book system, of a content of the distinguishing feature selected by said distinguishing feature selecting module.

22. A machine readable medium as claimed in claim 21 further comprising a featuring part searching module for searching for the image showing the part of the main object,

wherein said informing module displays a frame which surrounds the part on said displaying module.