Method and system for locating position in printed texts and delivering multimedia information

• US 7,239,747 B2
• Filed: 01/24/2003
• Issued: 07/03/2007
• Est. Priority Date: 01/24/2002
• Status: Expired due to Fees

• Current Assignee: Chatterbox Systems, Inc.
• Original Assignee: Chatterbox Systems, Inc.
• Inventors: Raffel, Jack, Bresler, Joel
• Primary Examiner(s): Mehta; Bhavesh M.
• Assistant Examiner(s): CARTER, AARON W
• Application Number:
  US10/351,277
• Publication Number:
  US 20030152293A1
• Time in Patent Office: 1,621 Days
• Field of Search: 382173-189, 382/321
• US Class Current: 382/176
• CPC Class Codes: G06F 16/40 of multimedia data, e.g. slideshows comprising image and additional audio data retrieval of still image data G06F16/50; retrieval of audio data G06F16/60; retrieval of video data G06F16/70
• CPC Class Codes: G06V 30/10 Character recognition
• CPC Class Codes: G06V 30/1444 Selective acquisition, locating or processing of specific regions, e.g. highlighted text, fiducial marks or predetermined fields
• CPC Class Codes: G06V 30/414 Extracting the geometrical structure, e.g. layout tree; Block segmentation, e.g. bounding boxes for graphics or text

Abstract

Apparatus and methods are disclosed for processing printed material to provide an index of locations within the printed material that can be associated with external actions such as displaying a graphical image, providing an audio or video output, or providing a multimedia output. The associations are stored in a database. Apparatus and methods are also disclosed that allow a user to image the printed material and by indicating a desired location within the printed material can cause one of the associated actions to be executed.

Claims

1. An apparatus for processing printed material comprising:

a pre-processing module for optically scanning printed text and generating pre-processed patterns for sub-regions of the printed text, each pre-processed pattern including a unique signature;

an input module for pointing to a designated location in the printed text and generating a designated localized sub-image including a pattern with a unique signature corresponding to the designated location;

a comparison module for comparing the pattern of the designated localized sub-image to the pre-processed patterns for sub-regions of the printed text to locate the designated localized sub-image in the pre-processed patterns for sub-regions; and

an output module for associating the located designated localized sub-image with a predetermined external action.

2. An apparatus of claim 1 in which the pre-processed patterns for sub-regions and the pattern of the designated localized sub-image are indexed to the word spaces surrounding each word within a page of scanned printed text such that only a single pattern is stored for each word on the page.

3. The apparatus of claim 1 in which each pre-processed pattern of the pre-processed patterns for sub-regions includes a pattern chosen from the group consisting of:

a blank space pattern of the spaces between the characters in the pre-processed patterns for sub-regions, a blank space pattern of the spaces between the words in the pre-processed patterns for sub-regions, the position of blank lines within the pre-processed patterns for sub-regions, the spaces before and after each paragraph within pre-processed patterns for sub-regions, and any combination thereof.

4. The apparatus of claim 1 in which the pattern for the designated localized sub-image includes a pattern chosen from the group consisting of:

a blank space pattern of the spaces between the characters in the designated localized sub-image, a blank space pattern of the spaces between the words in the designated localized sub-image, the position of blank lines within the designated localized sub-image, the spaces before and after each paragraph within the designated localized sub-image, and any combination thereof.

5. The apparatus of claim 1 in which the pre-processing module generates one or more metrics from the pre-processed patterns for sub-regions of the printed text.

6. The apparatus of claim 5 in which the pre-processing module subdivides the pre-processed patterns for sub-regions into a plurality of bins based on the one or more metrics.

7. The apparatus of claim 5 in which the one or more metrics include the sum of all blank space widths in a given section of the pre-processed patterns for sub-regions.

8. The apparatus of claim 7 in which the sum of the blank space widths is taken over a plurality of lines of the pre-processed patterns for sub-regions.

9. The apparatus of claim 5 in which the one or more metrics include the percentage of white space in a given section of the pre-processed patterns for sub-regions.

10. The apparatus of claim 5 in which the one or more metrics include the length of the last line in at least one paragraph in the pre-processed patterns for sub-regions.

11. The apparatus of claim 6 in which the input module generates one or more metrics from the pattern of the designated localized sub-image.

12. The apparatus of claim 11 in which a metric associated with the pattern of the designated localized sub-image is coarsely compared with the one or more metrics subdivided in the plurality of bins to locate the bin having the best match therebetween.

13. The apparatus of claim 12 in which the matched bin is searched to locate a match between the pattern of the designated localized sub-image and the pre-processed patterns for sub-regions.

14. The apparatus of claim 5 in which the one or more metrics are stored in a database.

15. The apparatus of claim 1 further including a display for displaying a portion of the printed text.

16. The apparatus of claim 1 in which the pre-processed patterns for sub-regions and/or the pattern of the designated localized sub-image are stored on a medium accessible by the comparison module.

17. The apparatus of claim 16 in which the medium is a medium chosen from the group consisting of:

a CD ROM, a DVD, a memory device, and magnetic media.

18. The apparatus of claim 1 in which the pre-processed patterns for sub-regions and/or the pattern for the designated localized sub-regions are stored on a server and available to the comparison module via a network.

19. The apparatus of claim 18 in which the network is a network chosen from the group consisting of:

the Internet, a local area network, a wide area network, and a wireless network.

20. The apparatus of claim 1 in which the input module includes an imager for imaging a sub-image of the printed text and a pointing device such that an image of the pointing device is superimposed on the sub-image of the printed text.

21. The apparatus of claim 1 in which the input module includes an imager and a pointing device fixed to the imager, the imager for scanning a sub-image of the printed text and the pointing device.

22. The apparatus of claim 1 in which the pointing device is chosen from the group consisting of:

a stylus, a pencil, a finger, a pointed arrow, and a light beam placed at a user indicated position.

23. The apparatus of claim 1 in which the predetermined external action provides output chosen from the group consisting of:

a graphical image, a video image, a multimedia image, a link to an internet site, a trigger to program logic in software, and an audio output.

24. The apparatus of claim 1 in which the designated localized sub-image comprises between about 10 and 30 adjacent character spaces located on about 3 to 4 adjacent lines.

25. The apparatus of claim 1 in which the scanned text includes a character set chosen from the group consisting of:

Latin, Hebrew, Chinese, Japanese, Cyrillic, Devanagari, and Arabic.