HANDHELD DEVICE FOR CAPTURING TEXT FROM BOTH A DOCUMENT PRINTED ON PAPER AND A DOCUMENT DISPLAYED ON A DYNAMIC DISPLAY DEVICE

• US 20140168716A1
• Filed: 11/14/2013
• Published: 06/19/2014
• Est. Priority Date: 04/19/2004
• Status: Active Grant

• Current Assignee: Google Inc. (Alphabet Inc.)
• Original Assignee: Google Inc. (Alphabet Inc.)
• Inventors: King, Martin T., Grover, Dale L., Kushler, Clifford A., Stafford-Fraser, James Q.
• Granted Patent: US 9,460,346 B2
• Time in Patent Office: Days
• US Class Current: 358/473
• CPC Class Codes: G06F 16/5846 using extracted text
• CPC Class Codes: G06Q 10/10 Office automation; Time management
• CPC Class Codes: G06Q 30/00 Commerce
• CPC Class Codes: G06V 10/95 structured as a network, e.g. client-server architectures
• CPC Class Codes: G06V 20/66 Trinkets, e.g. shirt buttons or jewellery items recognising microscopic objects G06V20/69
• CPC Class Codes: G06V 30/142 using hand-held instruments; Constructional details of the instruments
• CPC Class Codes: G06V 30/36 Matching; Classification
• CPC Class Codes: G06V 30/416 Extracting the logical structure, e.g. chapters, sections or page numbers; Identifying elements of the document, e.g. authors
• CPC Class Codes: G06V 30/418 Document matching, e.g. of document images
• CPC Class Codes: H04N 1/00241 using an image reading device as a local input to a computer input via network, e.g. network scanners H04N1/00225
• CPC Class Codes: H04N 1/00307 with a mobile telephone apparatus constructional features of portable telephone sets H04M1/0202; mobile radio systems H04W84/00
• CPC Class Codes: H04N 1/00488 providing an audible output to the user
• CPC Class Codes: H04N 1/0049 providing a visual indication to the user, e.g. using a lamp H04N1/00408 takes precedence
• CPC Class Codes: H04N 1/107 with manual scanning
• CPC Class Codes: H04N 1/32101 Display, printing, storage or transmission of additional information, e.g. ID code, date and time or title
• CPC Class Codes: H04N 2201/0041 Point to point H04N2201/0039 takes precedence
• CPC Class Codes: H04N 2201/0055 By radio
• CPC Class Codes: H04N 2201/0058 Docking-station, cradle or the like
• CPC Class Codes: H04N 2201/0081 Image reader H04N2201/0091 - H04N2201/0094 take precedence
• CPC Class Codes: H04N 2201/0084 Digital still camera
• CPC Class Codes: H04N 2201/0096 : Portable devices
• CPC Class Codes: H04N 2201/3249 : data relating to a linked page or object, e.g. hyperlink
• CPC Class Codes: H04N 2201/3264 : of sound signals
• CPC Class Codes: H04N 2201/3269 : of machine readable codes or marks, e.g. bar codes or glyphs

Abstract

A device for capturing rendered text is described. The device incorporates one or more visual sensors that receive visual information as a part of capturing rendered text. The visual sensors are collectively capable of capturing both text that is permanently printed on a page, and text that is displayed transitorily on a dynamic device. The device further incorporates a visual information disposition subsystem for disposing of visual information received by the visual sensors. The device further incorporates a package that bears the visual sensors and the visual information disposition subsystem, and is suitable to be held in a human hand.

Claims

1-20. -20. (canceled)

21. A method comprising:

capturing text of a document displayed transitorily on a dynamic display using a visual sensor;

using textual content of the captured text to identify the document and a position in the document at which a capture is performed;

determining a set of actions appropriate in connection with the capture from the identified document at the identified position; and

presenting at least a portion of the determined set of actions to a user.

22. The method of claim 21, further comprising:

receiving user input selecting one of the presented actions; and

in response to the received user input, performing the selected action in connection with the capture.

23. The method of claim 22, wherein presenting at least a portion of the determined set of actions comprises presenting, by the dynamic display, one or more actions as menu items in a menu, and wherein the user input comprises a user selection of one of the menu items.

24. The method of claim 21, wherein the set of actions comprises an action to be performed in accordance with a keyword or key phrase that occurs in the identified document in proximity to the identified position.

25. The method of claim 24, wherein the action to be performed in accordance with the keyword or key phrase comprises providing information associated with a seller of a product related to the keyword or key phrase.

26. The method of claim 21, wherein the set of actions comprises providing help information for the identified document.

27. Computer-readable media having contents capable of causing a computing system including one or more computers and a capture device to perform a method, the method comprising:

capturing text of a document displayed transitorily on a dynamic display using a visual sensor;

using textual content of the captured text to identify the document and a position in the document at which a capture is performed;

determining a set of actions appropriate in connection with the capture from the identified document at the identified position; and

presenting at least a portion of the determined set of actions to a user.

28. The computer-readable media of claim 27, the method further comprising:

receiving user input selecting one of the presented actions; and

in response to the received user input, performing the selected action in connection with the capture.

29. The computer-readable media of claim 28, wherein presenting at least a portion of the determined set of actions comprises presenting, by the dynamic display, one or more actions as menu items in a menu, and wherein the user input comprises a user selection of one of the menu items.

30. The computer-readable media of claim 27, wherein the set of actions comprises an action to be performed in accordance with a keyword or key phrase that occurs in the identified document in proximity to the identified position.

31. The computer-readable media of claim 30, wherein the action to be performed in accordance with the keyword or key phrase comprises providing information associated with a seller of a product related to the keyword or key phrase.

32. The computer-readable media of claim 27, wherein the set of actions comprises providing help information for the identified document.

33. A system, comprising:

a capture device configured to capture text of a document displayed transitorily on a dynamic display using a visual sensor; and

one or more computers having one or more memory devices on which are stored computer executable instructions operable to cause the one or more computers to perform operations comprising;

receiving captured text from the capture device,

using textual content of the captured text to identify the document and a position in the document at which a capture is performed;

determining a set of actions appropriate in connection with the capture from the identified document at the identified position; and

providing at least a portion of the determined set of actions for presentation to a user.

34. The system of claim 33, the operations further comprising:

receiving user input selecting one of the presented actions; and

in response to the received user input, performing the selected action in connection with the capture.

35. The system of claim 34, wherein providing at least a portion of the determined set of actions comprises providing one or more actions for presentation by the capture device as menu items in a menu, and wherein the user input comprises a user selection of one of the menu items.

36. The system of claim 33, wherein the set of actions comprises an action to be performed in accordance with a keyword or key phrase that occurs in the identified document in proximity to the identified position.

37. The system of claim 36, wherein the action to be performed in accordance with the keyword or key phrase comprises providing information associated with a seller of a product related to the keyword or key phrase.

38. The system of claim 33, wherein the set of actions comprises providing help information for the identified document.