Automatically capturing information, such as capturing information using a document-aware device

• US 8,638,363 B2
• Filed: 02/18/2010
• Issued: 01/28/2014
• Est. Priority Date: 02/18/2009
• Status: Active Grant

• Current Assignee: Kyocera Corporation
• Original Assignee: Google Inc. (Alphabet Inc.)
• Inventors: King, Martin T., Stephens, Redwood, Mannby, Claes-Fredrik, Peterson, Jesse, Sanvitale, Mark, Smith, Michael J.
• Primary Examiner(s): LIN, WEN TAI
• Application Number:
  US12/660,146
• Publication Number:
  US 20110025842A1
• Time in Patent Office: 1,440 Days
• Field of Search: None
• US Class Current: 348/135
• CPC Class Codes: G01B 11/00 Measuring arrangements characterised by the use of optical techniques
• CPC Class Codes: G01B 11/14 for measuring distance or clearance between spaced objects or spaced apertures G01B11/26 takes precedence; rangefinders G01C3/00
• CPC Class Codes: G06F 16/434 using image data, e.g. images, photos, pictures taken by a user
• CPC Class Codes: G06F 16/93 Document management systems
• CPC Class Codes: G06F 40/103 Formatting, i.e. changing of presentation of documents automatic justification G06F40/189; automatic line break hyphenation G06F40/191
• CPC Class Codes: G06F 40/169 Annotation, e.g. comment data or footnotes
• CPC Class Codes: G06F 40/174 Form filling; Merging
• CPC Class Codes: G06T 2207/30176 Document
• CPC Class Codes: G06T 7/20 Analysis of motion motion estimation for coding, decoding, compressing or decompressing digital video signals H04N19/43, H04N19/51
• CPC Class Codes: G06T 7/70 Determining position or orientation of objects or cameras camera calibration G06T7/80
• CPC Class Codes: G06V 30/224 of printed characters having additional code marks or containing code marks
• CPC Class Codes: G06V 30/40 Document-oriented image-based pattern recognition
• CPC Class Codes: H04N 1/00307 with a mobile telephone apparatus constructional features of portable telephone sets H04M1/0202; mobile radio systems H04W84/00
• CPC Class Codes: H04N 1/00331 with an apparatus performing optical character recognition character recognition G06V30/10
• CPC Class Codes: H04N 1/00355 Mark-sheet input sensing record carriers G06K7/00; character recognition G06V30/10; document-oriented image-based pattern recognition G06V30/40
• CPC Class Codes: H04N 1/0036 Alphanumeric symbols
• CPC Class Codes: H04N 1/00374 on the same page as at least a part of the image
• CPC Class Codes: H04N 1/00408 Display of information to the user, e.g. menus
• CPC Class Codes: H04N 1/00469 with enlargement of a selected area of the displayed information
• CPC Class Codes: H04N 1/00689 Presence
• CPC Class Codes: H04N 1/00737 : using the scanning elements as detectors
• CPC Class Codes: H04N 1/00758 : using a prescan
• CPC Class Codes: H04N 1/00782 : Initiating operations
• CPC Class Codes: H04N 1/19594 : using a television camera or a still video camera
• CPC Class Codes: H04N 1/32144 : embedded in the image data, i.e. enclosed or integrated in the image, e.g. watermark, super-imposed logo or stamp
• CPC Class Codes: H04N 2201/0084 : Digital still camera
• CPC Class Codes: H04N 2201/0096 : Portable devices
• CPC Class Codes: H04N 2201/3249 : data relating to a linked page or object, e.g. hyperlink
• CPC Class Codes: H04N 2201/3266 : of text or character information, e.g. text accompanying an image H04N2201/3226 takes precedence
• CPC Class Codes: H04N 23/64 : Computer-aided capture of images, e.g. transfer from script file into camera, check of taken image quality, advice or proposal for image composition or decision on when to take image
• CPC Class Codes: H04N 23/66 : Remote control of cameras or camera parts, e.g. by remote control devices

Abstract

A system and method for automatically changing the operation of a mobile device in response to a presence of information is described. In some examples, the system determines an information capture device is proximate to text, automatically changes operation of the capture device to a certain mode, captures the text, and performs an action associated with the captured text.

Claims

1. A system, comprising:

a detection component configured to make a prediction that text is within proximity of a capture component based on proximity measurements from a proximity sub-component of the detection component and duration measurements related to the proximity measurements, wherein the prediction is based on a proximity and a minimum duration related to the proximity, and wherein the detection component is further configured to modify at least one of the proximity and the minimum duration based on the proximity measurements and the duration measurements;

a device operation component, configured to generate information to modify a mode of operation in response to receiving the prediction that text is within proximity of the capture component based on the capture component being within the proximity to information to be captured for at least the minimum duration;

the capture component, configured to capture an image of the information to be captured in response to receiving the information to modify the mode of operation;

an identification component, wherein the identification component is configured to identify text within the captured image; and

an action component, wherein the action component is programmed to perform an action associated with the identified text.

2. The system of claim 1, wherein the capture component is configured to use the information to modify the mode of operation from a default mode of operation to a different mode of operation that enables capturing of the information to be captured.

3. The system of claim 1, wherein the detection component is configured to periodically take the proximity measurements and the duration measurements.

4. The system of claim 1, wherein the detection component is further configured to determine that the system is within the proximity to an object that contains the information to be captured.

5. The system of claim 1, wherein the identification component is configured to identify a rendered document that includes the information to be captured;

and

wherein the action component is configured to present an electronic counterpart of the rendered document.

6. The system of claim 1, further comprising:

a display component, wherein the display component is configured to present a view of the captured image;

wherein the action component is configured to overlay text within the presented view of the captured image with graphical elements that indicate text within the presented view is associated with additional information.

7. The system of claim 1, further comprising:

a display component, wherein the display component is configured to present a view of the captured image;

wherein the action component is configured to present one or more user selectable options associated with the presented view of the captured image.

8. The system of claim 1, wherein the identification component is configured to identify at least one feature of text within the captured image.

9. The system of claim 1, wherein the identification component is configured to recognize at least some of the text within the captured image.

10. A method, comprising:

performing one or more captures of data using a capture component of a mobile device, wherein the mobile device is configured to perform a capture of data based on a distance between the mobile device and a rendered document to be captured and a minimum duration of time that the mobile device is proximate to the rendered document;

when at least some of the captured data has associated actions, causing the mobile device to perform at least one of the associated actions associated with the captured data;

receiving capture-related measurements related to the one or more captures of data, wherein the capture-related measurements comprise, for each capture of data from a rendered document, a proximity measurement of a proximity between the rendered document and the mobile device and a duration measurement that the mobile device was proximate to the rendered document; and

modifying at least one of the distance and the minimum duration of time based on the capture-related measurements.

11. The method of claim 10, wherein the performed at least one associated action comprises displaying, using a display component of the mobile device, an indication that content is associated with at least some of the captured data.

12. The method of claim 10, wherein the performed at least one associated action comprises storing an indication that at least some of the captured data is text.

13. The method of claim 10, wherein the performed at least one associated action comprises storing the captured data, and wherein the method further comprises:

generating a captured-information timeline comprising at least some of the stored data; and

displaying the captured-information timeline.

14. The method of claim 13, wherein the captured-information timeline presents the at least some of the stored data captured during a second predetermined period of time.

15. The method of claim 10, wherein performing the one or more captures of data using the capture component comprises:

generating a prediction based on the capture-related measurements.

16. The method of claim 10, wherein performing the one or more captures of data using the capture component comprises capturing images and audio using the capture component.

17. An article of manufacture including a non-transitory computer-readable storage medium having software stored thereon that, in response to execution by a processor, causes the processor to perform functions comprising:

receiving a plurality of measurements for image captures, comprising a plurality of proximity measurements and a plurality of duration measurements related to the proximity measurements;

determining a first proximity for an image capture, based on the plurality of proximity measurements;

determining a first minimum duration related to the proximity for the image capture, based on the plurality of duration measurements;

making a first prediction that text is within proximity of a camera based on a determination that the camera is within the first proximity for the image capture for at least the first minimum duration;

in response to the first prediction, capturing an image of the text using the camera;

identifying text within the captured image;

performing an action associated with the identified text;

receiving a second plurality of measurements for image captures, comprising a second plurality of proximity measurements and a second plurality of duration measurements related to the second plurality of proximity measurements;

modifying at least one of the first proximity and the first minimum duration of time based on the second plurality of measurements; and

after modifying at least one of the first proximity and the first minimum duration of time, making a second prediction that text is within proximity of the camera based on a determination that the camera is within the first proximity for the image capture for at least the first minimum duration.

18. The article of manufacture of claim 17, wherein making the first prediction that text is within proximity of the camera comprises periodically sampling images captured by the camera.

19. The article of manufacture of claim 17, wherein making the first prediction that text is within proximity of the camera comprises determining a proximity to the text;

and

wherein capturing the image of the text comprises initiating capturing of images by the camera based on the determined proximity.

20. The article of manufacture of claim 17, wherein the functions further comprise:

identifying a rendered document that includes the text; and

presenting an electronic counterpart of the rendered document.

21. The article of manufacture of claim 17, wherein the functions further comprise:

presenting a view comprising the text and a graphical element that indicates the text is associated with additional information.

22. The article of manufacture of claim 17, wherein the functions further comprise:

presenting a view and one or more user selectable options associated with the view.

23. A method, comprising:

receiving a plurality of measurements for image captures comprising a plurality of proximity measurements and a plurality of duration measurements related to the proximity measurements;

determining a first proximity for a first image capture, based on the plurality of proximity measurements;

determining a first minimum duration of time related to the first proximity for the first image capture, based on the plurality of duration measurements;

making a first prediction that information is within proximity of a capture device based on a determination that the capture device is within the first proximity for the first image capture for at least the first minimum duration of time;

after making the first prediction, capturing a first image using the capture device;

receiving a second plurality of measurements for image captures comprising a second plurality of proximity measurements and a second plurality of duration measurements related to the second plurality of proximity measurements;

modifying at least one of the first proximity and the first minimum duration of time based on the second plurality of measurements; and

after modifying at least one of the first proximity and the first minimum duration of time;

making a second prediction that information is within proximity of the capture device based on a determination that the capture device is within the first proximity for a second image capture for at least the first minimum duration of time and

after making the second prediction, capturing a second image using the capture device.