Information retrieval apparatus for enabling information retrieval with ambiguous retrieval key

• US 6,130,962 A
• Filed: 06/04/1998
• Issued: 10/10/2000
• Est. Priority Date: 06/06/1997
• Status: Expired due to Term

• Current Assignee: Panasonic Intellectual Property Corporation of America (Panasonic Holdings Corporation)
• Original Assignee: Matsushita Electric Industrial Company Limited (Panasonic Holdings Corporation)
• Inventors: Sakurai, Yasuhiro
• Primary Examiner(s): Tran, Phuoc
• Assistant Examiner(s): Mariam, Daniel G.
• Application Number:
  US09/090,703
• Time in Patent Office: 859 Days
• Field of Search: 382/186, 382/187, 382/190, 382/305, 382/306, 382/181, 358/403, 358/404, 707/3, 707/4, 707/6
• US Class Current: 382/190
• CPC Class Codes: G06V 30/347 Sampling; Contour coding; Stroke extraction
• CPC Class Codes: G06V 30/373 using a special pattern or subpattern alphabet
• CPC Class Codes: H04N 1/00127 Connection or combination of a still picture apparatus with another apparatus, e.g. for storage, processing or transmission of still picture signals or of information associated with a still picture
• CPC Class Codes: H04N 1/00392 Other manual input means, e.g. digitisers or writing tablets
• CPC Class Codes: H04N 1/00408 Display of information to the user, e.g. menus
• CPC Class Codes: H04N 1/2175 with local image input
• CPC Class Codes: H04N 2201/0087 Image storage device
• CPC Class Codes: Y10S 707/99933 Query processing, i.e. searching
• CPC Class Codes: Y10S 707/99934 Query formulation, input preparation, or translation

Abstract

An input unit outputs handwritten data as stroke data to a control unit. The control unit stores the outputted stroke data in a handwritten memo database without recognizing the handwritten data as character code data, and stores stroke data of a retrieval mark in a mark database. The control unit classifies stroke data sets in each page into categories, such as characters and figures, and stores the classification result as retrieval information in the handwritten memo database. On receiving an indication of a retrieval mark or an indication of a retrieval key related to page contents, such as character contents or figure contents, from the input unit, the control unit searches the mark database or the handwritten memo database, and makes a display unit display handwritten data in the pages matching the indication.

Claims

1. An information retrieval apparatus comprising:

an input unit for receiving handwritten input from a user;

a stroke data set extracting means for extracting stroke data sets, wherein each stroke data set is a collection of data representing strokes in the handwritten input made by the user;

an identifier assigning unit for grouping the stroke data sets into predetermined groups and assigning a group identifier to each predetermined group;

a storage means for storing each predetermined group and the corresponding group identifier;

a design input receiving unit for receiving at least one handwritten design input inputted by the user;

a design stroke data set extracting unit for extracting a design stroke data set from each handwritten design input, wherein the storage means includes a design stroke data storage unit for storing the extracted design stoke data sets;

a design characteristic extracting unit for extracting characteristics of each design stroke data set;

a design appending means for appending each design stroke data set to one of the predetermined groups, in accordance with an indication given by the user;

a design management information generating means for generating information that associates each design stroke data set with the characteristics of the design stroke data set, and the group identifier corresponding to the predetermined group to which each design stroke data set has been appended, wherein the design management information is stored in the storage means;

a retrieval information receiving means for receiving a handwritten retrieval design input inputted by the user;

a retrieval design stroke data set extracting unit for extracting a retrieval design stroke data set from the handwritten retrieval design input;

a retrieval design characteristic extracting unit for extracting characteristics of the retrieval design stroke data set;

a design selecting unit for comparing the characteristics of each design stroke data set in the storage means with the characteristics of the retrieval design stroke data set, and selecting at least one of the design stroke data sets according to the degree of similarity between the characteristics of each design stroke data set and the characteristics of the retrieval design stroke data set;

a selected identifier extracting unit for extracting the group identifiers corresponding to the predetermined groups to which the selected design stroke data sets have been appended; and

a display means for displaying the stroke data sets in the predetermined groups corresponding to the group identifiers extracted by the retrieval means.

2. The information retrieval apparatus defined in claim 1, wherein

data for each stroke in a handwritten design is expressed by coordinate points that are detected at a predetermined interval, and

the design characteristic extracting unit detects a direction of at least one straight line that passes a plurality of coordinate points in a stroke and a direction of a straight line which joins an end point of a stroke with a start point of a next stroke, and converts the detected directions into normalized codes.

3. The information retrieval apparatus defined in claim 2, wherein

data for each stroke in a retrieval handwritten design is expressed by coordinate points that are detected at a predetermined interval, and

the retrieval design characteristic extracting unit detects a direction of at least one straight line that passes a plurality of coordinate points in a stroke in the retrieval handwritten design and a direction of a straight line which joins an end point of a stroke with a start point of a next stroke in the retrieval handwritten design, and converts the detected directions into normalized codes, wherein

the design selection unit calculates a degree of similarity between the codes obtained by the design characteristic extracting unit with the codes obtained by the retrieval design extracting unit using a dynamic programming matching method.

4. The information retrieval apparatus defined in claim 3, further comprising:

a design display receiving means for receiving an instruction as to whether to have the display means display the design appended by the design appending means,

wherein when the design display receiving means receives an instruction to have the design displayed, the display means reads and displays, in addition to displaying a stroke data set, an appended design stroke data set stored in the design stroke data set storage unit.

5. The information retrieval apparatus defined in claim 3, wherein

the design selection unit selects at least one design stroke data set with a degree of similarity that exceeds a predetermined threshold value.

6. The information retrieval apparatus defined in claim 1, wherein

the design selection unit selects at least one design stroke data set with a degree of similarity that exceeds a predetermined threshold value.

7. An information retrieval apparatus comprising:

an input unit for receiving handwritten input from a user;

a stroke data set extracting means for extracting stroke data sets, wherein each stroke data set is a collection of data representing strokes in the handwritten input made by the user;

a classification means for classifying each stroke data set as one of a character and a figure;

an identifier assigning unit for grouping the stroke data sets into predetermined groups and assigning a group identifier to each predetermined group;

a storage means for storing the stroke data sets along with their classification and the group identifier for each predetermined group;

a calculating means for calculating a character ratio and a figure ratio for each predetermined group, wherein the character ratio for each predetermined group is calculated by dividing the total number of strokes in the stroke data sets classified as a character in the predetermined group by the total number of strokes in the predetermined group, the figure ratio for each predetermined group is calculated by diving the total number of strokes in the stroke data sets classified as a figure in the predetermined group by the total number of strokes in the predetermined group, and the character and figure ratio for each predetermined group is stored in the storage means;

a display means for reading and displaying the stroke data sets to the user;

a retrieval menu display controlling means for having the display unit display a character menu item and a figure menu item to the user;

a retrieval information receiving means for receiving retrieval information inputted by the user, wherein the retrieval information is a designation of one of the menu items displayed by the display unit;

a retrieval means for extracting each group identifier corresponding to a predetermined group having a character ratio equal to or greater than a predetermined threshold when the retrieval information receiving means receives the character menu item, and extracting each group identifier corresponding to a predetermined group having a figure ratio equal to or greater than a predetermined threshold when the information receiving means receives the figure menu item;

wherein the display means reads from the storage means and displays the predetermined groups corresponding to the group identifiers extracted from the retrieval means.

8. The information retrieval apparatus defined in claim 7, wherein

the classification means includes:

a character classification unit for classifying each stroke data set that has been classified as a character as one of a Chinese character and a numeral, according to whether a number of strokes that composes a character of the stroke data set exceeds a predetermined threshold value;

a first calculating unit for calculating a number of Chinese characters and a number of numerals in each predetermined group of stroke data sets; and

a first additional writing unit for writing the number of Chinese characters and the number of numerals calculated by the first calculating unit into the characteristic storage unit so as to correspond to an identifier of the predetermined group for which the number of Chinese characters and the number of numerals were calculated,

the retrieval menu display controlling means includes;

a first auxiliary menu display control unit for having the display means display a Chinese character auxiliary menu item for retrieving predetermined groups of the stroke data sets according to the number of Chinese characters and a numeral auxiliary menu item for retrieving predetermined groups of the stroke data sets according to the number of numerals,

the retrieval means includes;

a first extracting unit for extracting an identifier of each predetermined group whose number of Chinese characters is either of equal to and greater than a predetermined value when the retrieval information receiving means receives the Chinese character auxiliary menu, and extracting an identifier of each predetermined group whose number of numerals is either of equal to and greater than a predetermined value when the retrieval information receiving means receives the numeral auxiliary menu.

9. The information retrieval apparatus defined in claim 8, wherein

the classification means further includes:

a segment classification unit for classifying each stroke data set that has been classified as a character as one of a word and a single character, according to whether a number of segments that composes a character of the stroke data set exceeds a predetermined threshold value;

a second calculating unit for calculating a number of words and a number of single characters in each predetermined group of stroke data sets; and

a second additional writing unit for writing the number of words and the number of single characters calculated by the second calculating unit into the characteristic storage unit so as to correspond to an identifier of the predetermined group for which the number of words and the number of single characters were calculated,

the retrieval menu display controlling means further includes;

a second auxiliary menu display control unit for having the display means display a word auxiliary menu item for retrieving predetermined groups of the stroke data sets according to the number of words and a single character auxiliary menu item for retrieving predetermined groups of the stroke data sets according to the number of single characters,

the retrieval means further includes;

a second extracting unit for extracting an identifier of each predetermined group whose number of words is either of equal to and greater than a predetermined value when the retrieval information receiving means receives the word auxiliary menu, and extracting an identifier of each predetermined group whose number of single characters is either of equal to and greater than a predetermined value when the retrieval information receiving means receives the single character auxiliary menu.

10. The information retrieval apparatus defined in claim 9, wherein

the classification means further includes:

a figure classification unit for classifying each stroke data set that has been classified as a figure and conforms to a predetermined condition as one of an oval, a rectangle, and a straight line;

a third calculating unit for calculating a number of ovals, a number of rectangles, and a number of straight lines in each predetermined group of stroke data sets; and

a third additional writing unit for writing the number of ovals, the number of rectangles, and the number of straight lines calculated by the third calculating unit into the characteristic storage unit so as to correspond to an identifier of the predetermined group for which the number of ovals, the number of rectangles, and the number of straight lines were calculated,

the retrieval menu display controlling means further includes;

a third auxiliary menu display control unit for having the display means display an oval auxiliary menu item for retrieving predetermined groups of the stroke data sets according to the number of ovals, a rectangle auxiliary menu item for retrieving predetermined groups of the stroke data sets according to the number of rectangles, and a straight line auxiliary menu item for retrieving predetermined groups of the stroke data sets according to the number of straight lines,

the retrieval means further includes;

a third extracting unit for extracting an identifier of each predetermined group whose number of ovals is either of equal to and greater than a predetermined value when the retrieval information receiving means receives the oval auxiliary menu, extracting an identifier of each predetermined group whose number of rectangles is either of equal to and greater than a predetermined value when the retrieval information receiving means receives the rectangle auxiliary menu, and extracting an identifier of each predetermined group whose number of straight lines is either of equal to and greater than a predetermined value when the retrieval information receiving means receives the straight line auxiliary menu.

11. The information retrieval apparatus defined in claim 8, wherein

the classification means further includes:

a length-to-width ratio classification unit for classifying each stroke data set that has been classified as a character as one of a word and a single character, according to whether a length-to-width ratio of a character of the stroke data set exceeds a predetermined threshold value;

a fourth calculating unit for calculating a number of words and a number of single characters in each predetermined group of stroke data sets; and

a fourth additional writing unit for writing the number of words and the number of single characters calculated by the fourth calculating unit into the characteristic storage unit so as to correspond to an identifier of the predetermined group for which the number of words and the number of single characters were calculated,

the retrieval menu display controlling means further includes;

a fourth auxiliary menu display control unit for having the display means display a word auxiliary menu item for retrieving predetermined groups of the stroke data sets according to the number of words and a single character auxiliary menu item for retrieving predetermined groups of the stroke data sets according to the number of single characters,

the retrieval means further includes;

a fourth extracting unit for extracting an identifier of each predetermined group whose number of words is either of equal to and greater than a predetermined value when the retrieval information receiving means receives the word auxiliary menu, and extracting an identifier of each predetermined group whose number of single characters is either of equal to and greater than a predetermined value when the retrieval information receiving means receives the single character auxiliary menu.

12. The information retrieval apparatus defined in claim 7, wherein

the classification means further includes:

a segment classification unit for classifying each stroke data set that has been classified as a character as one of a word and a single character, according to whether a number of segments that composes a character of the stroke data set exceeds a predetermined threshold value;

a second calculating unit for calculating a number of words and a number of single characters in each predetermined group of stroke data sets; and

a second additional writing unit for writing the number of words and the number of single characters calculated by the second calculating unit into the characteristic storage unit so as to correspond to an identifier of the predetermined group for which the number of words and the number of single characters were calculated,

the retrieval menu display controlling means further includes;

a second auxiliary menu display control unit for having the display means display a word auxiliary menu item for retrieving predetermined groups of the stroke data sets according to the number of words and a single character auxiliary menu item for retrieving predetermined groups of the stroke data sets according to the number of single characters,

the retrieval means further includes;

a second extracting unit for extracting an identifier of each predetermined group whose number of words is either of equal to and greater than a predetermined value when the retrieval information receiving means receives the word auxiliary menu, and extracting an identifier of each predetermined group whose number of single characters is either of equal to and greater than a predetermined value when the retrieval information receiving means receives the single character auxiliary menu.

13. The information retrieval apparatus defined in claim 7, wherein

the classification means further includes:

a figure classification unit for classifying each stroke data set that has been classified as a figure and conforms to a predetermined condition as one of an oval, a rectangle, and a straight line;

a third calculating unit for calculating a number of ovals, a number of rectangles, and a number of straight lines in each predetermined group of stroke data sets; and

a third additional writing unit for writing the number of ovals, the number of rectangles, and the number of straight lines calculated by the third calculating unit into the characteristic storage unit so as to correspond to an identifier of the predetermined group for which the number of ovals, the number of rectangles, and the number of straight lines were calculated,

the retrieval menu display controlling means further includes;

a third auxiliary menu display control unit for having the display means display an oval auxiliary menu item for retrieving predetermined groups of the stroke data sets according to the number of ovals, a rectangle auxiliary menu item for retrieving predetermined groups of the stroke data sets according to the number of rectangles, and a straight line auxiliary menu item for retrieving predetermined groups of the stroke data sets according to the number of straight lines,

the retrieval means further includes;

a third extracting unit for extracting an identifier of each predetermined group whose number of ovals is either of equal to and greater than a predetermined value when the retrieval information receiving means receives the oval auxiliary menu, extracting an identifier of each predetermined group whose number of rectangles is either of equal to and greater than a predetermined value when the retrieval information receiving means receives the rectangle auxiliary menu, and extracting an identifier of each predetermined group whose number of straight lines is either of equal to and greater than a predetermined value when the retrieval information receiving means receives the straight line auxiliary menu.

14. The information retrieval apparatus defined in claim 7, wherein

the classification means further includes:

a length-to-width ratio classification unit for classifying each stroke data set that has been classified as a character as one of a word and a single character, according to whether a length-to-width ratio of a character of the stroke data set exceeds a predetermined threshold value;

a fourth calculating unit for calculating a number of words and a number of single characters in each predetermined group of stroke data sets; and

a fourth additional writing unit for writing the number of words and the number of single characters calculated by the fourth calculating unit into the characteristic storage unit so as to correspond to an identifier of the predetermined group for which the number of words and the number of single characters were calculated,

the retrieval menu display controlling means further includes;

a fourth auxiliary menu display control unit for having the display means display a word auxiliary menu item for retrieving predetermined groups of the stroke data sets according to the number of words and a single character auxiliary menu item for retrieving predetermined groups of the stroke data sets according to the number of single characters,

the retrieval means further includes;

a fourth extracting unit for extracting an identifier of each predetermined group whose number of words is either of equal to and greater than a predetermined value when the retrieval information receiving means receives the word auxiliary menu, and extracting an identifier of each predetermined group whose number of single characters is either of equal to and greater than a predetermined value when the retrieval information receiving means receives the single character auxiliary menu.

15. An information retrieval apparatus comprising:

an image data obtaining means for obtaining image data sets from an outside source;

an identifier means for assigning an identifier to each image data set;

a storage means for storing each image data set and the corresponding identifier;

a design input receiving means for receiving at least one handwritten design input from a user,

a design stroke data set extracting means for extracting a design stroke data set from each handwritten design input, wherein each design stroke data set is a collection of data representing the strokes in a handwritten input;

a design characteristic extracting means for extracting characteristics of each design stroke data set;

a design appending means for appending each design stroke data set to one of the image data sets;

a design management information generating means for generating information that associates each design stroke data set with the characteristics of the design stroke data set, and the identifier corresponding to the image data set to which the design stroke data set has been appended, wherein the design management information is stored in the storage means;

a retrieval information receiving means for receiving a handwritten retrieval design input from the user;

a retrieval design stroke data set extracting means for extracting a retrieval design stroke data set from the handwritten retrieval design input, wherein each retrieval design stroke data set is a collection of data representing strokes in a handwritten retrieval design input;

a retrieval design characteristic extracting means for extracting characteristics of the retrieval design stroke data set;

a design selecting means for comparing the characteristics of each design stroke data set in the storage means with the characteristics of the retrieval design stroke data set, and selecting at least one of the design stroke data sets according to the degree of similarity between the characteristics of each design stroke data set and the characteristics of the retrieval design stroke data set;

a selected identifier extracting means for extracting the identifiers corresponding to the image data sets to which the selected design stroke data sets have been appended; and

a display means for displaying the image data sets from the storage means corresponding to the identifiers extracted by the retrieval means.

16. An information retrieval method for an information retrieval apparatus having an input unit, a storage unit, and a display unit, comprising the steps of:

receiving handwritten input by a user from the input unit;

generating stroke data sets, wherein each stroke data set is a collection of data representing strokes in the handwritten input made by the user;

grouping the stroke data sets into predetermined groups;

assigning a group identifier to each predetermined group;

storing each predetermined group and the corresponding group identifier in the storage unit;

receiving at least one handwritten design input from the user;

generating a design stroke data set from each handwritten design input;

extracting characteristics of each design stroke data set;

appending each design stroke data set to one of the predetermined groups, in accordance with an indication given by the user;

generating design management information that associates each design stroke data set with the characteristics of the design stroke data set, and the group identifier corresponding to the predetermined group to which each design stroke data set has been appended;

storing the design management information in the storage means;

receiving a handwritten retrieval design input from the user;

generating a retrieval design stroke data set from the handwritten retrieval design input;

extracting characteristics of the retrieval design stroke data set;

comparing the characteristics of each design stroke data set in the storage means with the characteristics of the retrieval design stroke data set;

selecting at least one of the design stroke data sets according to the degree of similarity between the characteristics of each design stroke data set and the characteristics of the retrieval design stroke data set;

extracting the group identifiers corresponding to the predetermined groups to which the design stroke data sets selected in the selecting step have been appended; and

displaying the predetermined groups from the storage means corresponding to the group identifiers extracted in the step extracting the group identifiers on the display unit.

17. An information retrieval apparatus comprising:

an input unit for receiving handwritten input from a user;

a stroke data set extracting means for extracting stroke data sets, wherein each stroke data set is a collection of data representing strokes in the handwritten input made by the user;

a classification means for classifying each stroke data set as one of a character and a figure;

an identifier assigning means for grouping the stroke data sets into predetermined groups and assigning a group identifier to each predetermined group;

a storage means for storing the stroke data sets along with their classification and the group identifier for each predetermined group;

a calculating means for calculating a character ratio and a figure ratio for each predetermined group, wherein the character ratio for each predetermined group is calculated by dividing the total number of strokes in the stroke data sets classified as a character in the predetermined group by the total number of strokes in the predetermined group, the figure ratio for each predetermined group is calculated by diving the total number of strokes in the stroke data sets classified as a figure in the predetermined group by the total number of strokes in the predetermined group, and the character and figure ratio for each predetermined group is stored in the storage means;

a display unit for reading and displaying the stroke data sets to the user; and

a retrieval means for extracting group identifiers based on one of the character ratio of each predetermined group and the figure ratio of each predetermined group;

wherein the display unit reads from the storage means and displays the predetermined groups corresponding to the group identifiers extracted from the retrieval means.

18. The information retrieval apparatus of claim 17, wherein the classification means further classifies each stroke data set classified as a character as one of a Chinese character and a numeral, the calculation means calculates the number of stroke data sets classified as a Chinese character and a numeral in each predetermined group, and the retrieval means further extracts group identifiers based on one of the number of Chinese characters in each predetermined group and the number of numerals in each predetermined group.

19. The information retrieval apparatus of claim 17, wherein the classification means further classifies each stroke data set classified as a figure as one of an oval, a rectangle and a straight line, the calculation means calculates the number of stroke data sets classified as a oval, a rectangle and a straight line in each predetermined group, and the retrieval means further extracts group identifiers based on one of the number of ovals in each predetermined group, the number of rectangles in each predetermined group, and the number of straight lines in each predetermined group.

20. A computer-readable recording medium which records a program for having an information retrieval apparatus execute an information retrieval process comprising:

a handwritten input receiving step for receiving a handwritten input made by a user;

a stroke data set extracting steps for extracting stroke data sets, wherein each stroke data set is a collection of data representing strokes in the handwritten input made by the user;

a storage step for storing each extracted stroke data set;

an identifier assigning step for assigning a group identifier to the stroke data sets in predetermined groups, wherein each predetermined group of stroke data sets is stored together with a corresponding group identifier;

a design input receiving step for receiving at least one handwritten design input from the user;

a design stroke data set extracting step for extracting a design stroke data set from each handwritten design input, wherein each design stroke data set is a collection of data representing strokes in a handwritten deign input;

a design stroke data set storage step for storing design stroke data sets extracted in the design stroke data set extracting step;

a design characteristic extracting step for extracting characteristics of each design stroke data set stored in the design stroke data set storage step;

a design appending step for appending each design stroke data set to one of the predetermined groups, in accordance with an indication given by the user;

a design management information generating step for generating design management information that associates each design stroke data set with the characteristics of the design stroke data set, and the group identifier corresponding to the predetermined group to which each design stroke data set has been appended;

a design management information storage step for storing the design management information;

a retrieval design input receiving step for receiving a handwritten retrieval design input from the user;

a retrieval design stroke data set extracting step for extracting a retrieval design stroke data set from the handwritten retrieval design input, wherein the retrieval design stroke data set is a collection of data representing strokes in the retrieval design input;

a retrieval design characteristic extracting step for extracting characteristics of the retrieval design stroke data set;

a design selection step for comparing the characteristics of each design stroke data set stored in the design management information storage step with the characteristics of the retrieval design stroke data set, and for selecting at least one of the design stroke data sets according to the degree of similarity between the characteristics of each design stroke data set and the characteristics of the retrieval design stroke data set;

a selected identifier extracting step for extracting the group identifiers corresponding to the predetermined groups to which the design stroke data sets selected in the design selection step have been appended; and

a display step for displaying the stroke data sets in the predetermined groups corresponding to the group identifiers extracted in the selected identifier extracting step.