Automatically capturing information such as capturing information using a document-aware device

• US 9,113,076 B2
• Filed: 01/22/2014
• Issued: 08/18/2015
• Est. Priority Date: 02/18/2009
• Status: Active Grant

• Current Assignee: Kyocera Corporation
• Original Assignee: Google Inc. (Alphabet Inc.)
• Inventors: King, Martin T., Stephens, Redwood, Mannby, Claes-Fredrik, Peterson, Jesse, Sanvitale, Mark, Smith, Michael J.
• Primary Examiner(s): LIN, WEN TAI
• Application Number:
  US14/161,181
• Publication Number:
  US 20140232889A1
• Time in Patent Office: 573 Days
• US Class Current: 1/1
• CPC Class Codes: G01B 11/00 Measuring arrangements characterised by the use of optical techniques
• CPC Class Codes: G01B 11/14 for measuring distance or clearance between spaced objects or spaced apertures G01B11/26 takes precedence; rangefinders G01C3/00
• CPC Class Codes: G06F 16/434 using image data, e.g. images, photos, pictures taken by a user
• CPC Class Codes: G06F 16/93 Document management systems
• CPC Class Codes: G06F 40/103 Formatting, i.e. changing of presentation of documents automatic justification G06F40/189; automatic line break hyphenation G06F40/191
• CPC Class Codes: G06F 40/169 Annotation, e.g. comment data or footnotes
• CPC Class Codes: G06F 40/174 Form filling; Merging
• CPC Class Codes: G06T 2207/30176 Document
• CPC Class Codes: G06T 7/20 Analysis of motion motion estimation for coding, decoding, compressing or decompressing digital video signals H04N19/43, H04N19/51
• CPC Class Codes: G06T 7/70 Determining position or orientation of objects or cameras camera calibration G06T7/80
• CPC Class Codes: G06V 30/224 of printed characters having additional code marks or containing code marks
• CPC Class Codes: G06V 30/40 Document-oriented image-based pattern recognition
• CPC Class Codes: H04N 1/00307 with a mobile telephone apparatus constructional features of portable telephone sets H04M1/0202; mobile radio systems H04W84/00
• CPC Class Codes: H04N 1/00331 with an apparatus performing optical character recognition character recognition G06V30/10
• CPC Class Codes: H04N 1/00355 Mark-sheet input sensing record carriers G06K7/00; character recognition G06V30/10; document-oriented image-based pattern recognition G06V30/40
• CPC Class Codes: H04N 1/0036 Alphanumeric symbols
• CPC Class Codes: H04N 1/00374 on the same page as at least a part of the image
• CPC Class Codes: H04N 1/00408 Display of information to the user, e.g. menus
• CPC Class Codes: H04N 1/00469 with enlargement of a selected area of the displayed information
• CPC Class Codes: H04N 1/00689 Presence
• CPC Class Codes: H04N 1/00737 : using the scanning elements as detectors
• CPC Class Codes: H04N 1/00758 : using a prescan
• CPC Class Codes: H04N 1/00782 : Initiating operations
• CPC Class Codes: H04N 1/19594 : using a television camera or a still video camera
• CPC Class Codes: H04N 1/32144 : embedded in the image data, i.e. enclosed or integrated in the image, e.g. watermark, super-imposed logo or stamp
• CPC Class Codes: H04N 2201/0084 : Digital still camera
• CPC Class Codes: H04N 2201/0096 : Portable devices
• CPC Class Codes: H04N 2201/3249 : data relating to a linked page or object, e.g. hyperlink
• CPC Class Codes: H04N 2201/3266 : of text or character information, e.g. text accompanying an image H04N2201/3226 takes precedence
• CPC Class Codes: H04N 23/64 : Computer-aided capture of images, e.g. transfer from script file into camera, check of taken image quality, advice or proposal for image composition or decision on when to take image
• CPC Class Codes: H04N 23/66 : Remote control of cameras or camera parts, e.g. by remote control devices

Abstract

A system and method for automatically changing the operation of a mobile device in response to a presence of information is described. In some examples, the system determines an information capture device is proximate to text, automatically changes operation of the capture device to a certain mode, captures the text, and performs an action associated with the captured text.

Claims

1. A mobile device comprising:

one or more data processors that execute instructions that cause the one or more data processors to perform operations comprising;

measuring a relative orientation between a document and the mobile device; and

comparing the measured relative orientation to a threshold relative orientation associated with a document capture mode of the mobile device; and

an action component that performs operations comprising;

automatically transitioning the mobile device to the document capture mode when the measured relative orientation is within the threshold relative orientation associated with the document capture mode, wherein the threshold relative orientation is based on an analysis that includes;

analyzing occurrences when a user of the mobile device rejected automatic launching of an application that performs actions in response to text captures; and

analyzing occurrences when a user of the mobile device manually launched the application that performs actions in response to text captures.

2. The mobile device of claim 1, wherein the one or more data processors are programmed to determine that the mobile device is within a certain proximity to the document and capture an image of the document based on the determined proximity.

3. The mobile device of claim 2, wherein the one or more data processors are programmed to identify a rendered document based on text in the image and present an electronic counterpart of the rendered document at the mobile device.

4. The mobile device of claim 1, further comprising:

a display component that presents a view of a camera at the mobile device, wherein the action component is programmed to overlay text within the presented view of the camera with graphical elements that indicate the text is associated with additional information.

5. The mobile device of claim 1, further comprising:

a display component that presents a view of a camera at the mobile device, wherein the action component is programmed to present one or more user-selectable options associated with the text or a display of text with the presented view of the camera.

6. A non-transitory computer-readable medium whose contents, when executed by a mobile device, cause the mobile device to perform a method for performing an action associated with a rendered document, the method comprising:

measuring a relative orientation between a rendered document and the mobile device;

comparing the measured relative orientation to a threshold relative orientation associated with a document capture mode of the mobile device; and

automatically transitioning the mobile device to the document capture mode when the measured relative orientation is within the threshold relative orientation associated with the document capture mode, wherein the threshold relative orientation is determined by performing an analysis of previous occurrences of the mobile device in document capture mode, wherein the analysis includes;

analyzing occurrences when a user of the mobile device rejected automatic launching of an application that performs actions in response to text captures; and

analyzing occurrences when a user of the mobile device manually launched the application that performs actions in response to text captures.

7. The non-transitory computer-readable medium of claim 6, wherein measuring the relative orientation between the rendered document and the mobile device includes measuring a distance between an imaging component of the mobile device and the rendered document.

8. The non-transitory computer-readable medium of claim 6, wherein measuring the relative orientation between the rendered document and the mobile device includes measuring a distance between a center point of a display associated with the mobile device and the rendered document.

9. The non-transitory computer-readable medium of claim 6, wherein the threshold relative orientation is determined by a user of the mobile device.

10. A method comprising:

measuring a relative orientation between a rendered document and a mobile device;

comparing the measured relative orientation to a threshold relative orientation associated with a document capture mode of the mobile device; and

automatically transitioning the mobile device to the document capture mode when the measured relative orientation is within the threshold relative orientation associated with the document capture mode, wherein the threshold relative orientation is determined by performing an analysis of previous occurrences of the mobile device in document capture mode, wherein the threshold relative orientation is based on an analysis that includes;

analyzing occurrences when a user of the mobile device rejected automatic launching of an application that performs actions in response to text captures; and

analyzing occurrences when a user of the mobile device manually launched the application that performs actions in response to text captures.

11. The method of claim 10, wherein measuring the relative orientation between the rendered document and the mobile device includes measuring a distance between an imaging component of the mobile device and the rendered document.

12. The method of claim 10, wherein measuring the relative orientation between the rendered document and the mobile device includes measuring a distance between a center point of a display associated with the mobile device and the rendered document.

13. The method of claim 10, wherein the threshold relative orientation is determined by a user of the mobile device.