Method and system for navigating a hard copy of a web page

• US 9,697,182 B2
• Filed: 12/11/2012
• Issued: 07/04/2017
• Est. Priority Date: 12/11/2012
• Status: Active Grant

• Current Assignee: Xerox Corporation (Xerox Holdings Corp.)
• Original Assignee: Xerox Corporation (Xerox Holdings Corp.)
• Inventors: Fan, Zhigang, Moore, Lee Coy, Emmett, Phillip J., Zhao, Yonghui
• Primary Examiner(s): Hong, Stephen
• Assistant Examiner(s): Sheffield, Hope C
• Application Number:
  US13/710,804
• Publication Number:
  US 20140164891A1
• Time in Patent Office: 1,666 Days
• Field of Search: None
• CPC Class Codes: G06F 16/951 Indexing; Web crawling techniques
• CPC Class Codes: G06F 40/134 Hyperlinking
• CPC Class Codes: H04N 1/00204 with a digital computer or a digital computer system, e.g. an internet server programmed control between transmitter and receiver or between image input and image output device H04N1/32561
• CPC Class Codes: H04N 1/00968 by scanning marks on a sheet mark-sheet input means H04N1/00355; sensing record carriers G06K7/00
• CPC Class Codes: H04N 1/32133 on the same paper sheet, e.g. a facsimile page header
• CPC Class Codes: H04N 2201/0084 Digital still camera
• CPC Class Codes: H04N 2201/3249 data relating to a linked page or object, e.g. hyperlink

Abstract

This disclosure provides a method and system for navigating hard copies of web pages. According to one exemplary method, a hard copy of a web page is navigated by capturing with a mobile device an image of the hard copy, performing OCR on the captured image to identify the captured web page, accessing a database to retrieve a copy of the identified web page and displaying the retrieved copy of the identified web page along with active web page links on the mobile device.

Claims

1. A computer-implemented method of navigating a hard copy of a web page comprising:

A) capturing an image of a physically displayed web page hard copy using a mobile hand held image capturing device including a camera, a display and a user interface, an alignment of the mobile hand held image capturing device and the captured image horizontally and/or vertically skewed relative to a displayed angle of view associated with the physically displayed web page, and the captured image including one or more text strings which provide dormant links to other web pages;

B) performing OCR (optical character recognition) on the captured image to produce a text searchable version of the captured image;

C) searching a database for an electronic copy of a web page matching the web page hardcopy by searching the database for one or more portions of text associated with the text searchable version of the captured image, the database including text searchable versions of a plurality of electronic copies of web pages retrieved from one or more web servers including the electronic a copy of the web page hard copy, each text searchable version of the plurality of electronic copies of web pages including control information which links one or more text strings contained in the plurality of electronic copies of web pages to other linked copies of web pages;

D) identifying the matching electronic copy of the web page hard copy;

E) retrieving the control information associated with the captured web page hard copy from the matching electronic copy of the web page hard copy; and

F) generating and displaying on the mobile handheld image capturing device display a processed image of the matching electronic copy of the web page hard copy horizontally and vertically aligned to the display substantially equivalent to the captured image alignment horizontally and/or vertically skewed relative to the displayed angle of view associated with the physically displayed web page, the processed image including the retrieved control information associated with the matching electronic copy of the web page hard copy and the processed image including a modified appearance of the one or more text strings associated with the web page hard copy which are linked to one or more other electronic copies of web pages retrieved from the one or more web servers, the modified appearance indicating the one or more text strings are active and the modified appearance of the text strings at substantially the same angle as the captured image horizontally and/or vertically skewed relative to the displayed angle of view associated with the physically displayed web page.

2. The computer-implemented method of navigating a hard copy of a web page according to claim 1, further comprising:

a user selecting one of the active text strings and the mobile handheld image capturing device displaying the electronic copy of the linked web page associated with the active text string.

3. The computer-implemented method of navigating a web page hard copy according to claim 1, wherein the mobile hand held image capturing device is one of a smart phone, mobile computing device, and laptop computer.

4. The computer-implemented method of navigating a web page hard copy according to claim 1, wherein step E) displays the processed image of the web page hard copy using a native browser associated with the mobile hand held image capturing device.

5. The computer-implemented method of navigating a web page hard copy according to claim 1, wherein step E) displays an augmented image including the retrieved control information associated with the matching electronic copy of web page hard copy, and the augmented image includes an overlay on the captured image which modifies the appearance of one or more text strings associated with the web page hard copy which are linked to one or more other electronic copies of web pages retrieved from one or more web servers, the modified appearance indicating the one or more text strings are active.

6. The computer-implemented method of navigating a web page hard copy according to claim 1, further comprising:

a user selecting one of the active text strings and the mobile hand held image capturing device display displaying the electronic copy of the linked web page associated with the active text string; and

the user selecting a second active text string associated with the electronic copy of the linked web page and the mobile hand held image capturing device display displaying an electronic copy of the linked web page associated with the second active text string.

7. A web page hard copy navigation system comprising:

a mobile hand held image capturing device including a camera, a display and a user interface, the mobile hand held image capturing device configured to capture an image of a physically displayed web page hard copy containing one or more dormant links to other web pages, an alignment of the mobile hand held image capturing device and the captured image horizontally and/or vertically skewed relative to a displayed angle of view associated with the physically displayed web page;

a database operatively connected to the mobile hand held image capturing device, the database configured to store a plurality of web page electronic copies retrieved from one or more web servers, the plurality of web page electronic copies including control information which links text strings contained in the web page electronic copies to other web page electronic copies;

a server operatively connected to the mobile hand held image capturing device and the database, the server configured to search the database for a matching electronic copy of the captured image of the web page hard copy, retrieve the electronic copy of the web page hard copy associated with the captured image including any control information, and communicating the retrieved electronic copy of the web page hard copy associated with the captured image to the mobile hand held image capturing device, the communicated retrieved electronic copy including a processed image of the web page hard copy aligned substantially equivalent to the captured image alignment which is horizontally and/or vertically skewed relative to the displayed angle of view associated with the physically displayed web page, the processed image including the retrieved control information associated with the retrieved electronic copy of the web page hard copy, and the processed image including an overlay on the captured image which modifies an appearance of one or more text strings associated with the web page hard copy which are linked to one or more other electronic copies of web pages indicating the one or more text strings are active.

8. The web page hard copy navigation system according to claim 7, wherein the system is configured to receive a user selection via the UI indicating a selection of one of the active text strings and display the electronic copy of the linked web page associated with the active text string.

9. The web page hard copy navigation system according to claim 7, wherein the mobile hand held image capturing device is one of a smart phone, mobile computing device, and laptop computer.

10. The web page hard copy navigation system according to claim 7, wherein the system is configured to display the processed image of the web page hard copy using a native browser associated with the mobile hand held image capturing device.

11. The web page hard copy navigation system according to claim 7, wherein the system is configured to display an augmented image including the retrieved control information associated with the web page hard copy, and the augmented image includes an overlay on the captured image which modifies an appearance of one or more text strings associated with the web page hard copy which are linked to one or more other electronic copies of web pages retrieved from one or more web servers, the modified appearance indicating the one or more text strings are active.

12. The web page hard copy navigation system according to claim 7, wherein the system is configured to receive a user selection via the user interface indicating a selection of one of the active text strings, display the electronic copy of the linked web page associated with the active text string, receive a second user selection of a second active text string associated with the electronic copy of the linked web page, and display an electronic copy of the linked web page associated with the second active text string.

13. A computer program product comprising:

a non-transitory computer-usable data carrier storing instructions that, when executed by a computer, cause the computer to perform a method comprising;

A) capturing an image of a physically displayed web page hard copy using a mobile hand held image capturing device including a camera, a display and a user interface, an alignment of the mobile hand held image capturing device and the captured image horizontally and/or vertically skewed relative to a displayed angle of view associated with the physically displayed web page, and the captured image including one or more text strings which provide dormant links to other web pages;

B) performing OCR (optical character recognition) on the captured image to produce a text searchable version of the captured image;

C) searching a database for an electronic copy of a web page matching the web page hardcopy by searching the database for one or more portions of text associated with the text searchable version of the captured image, the database including text searchable versions of a plurality of electronic copies of web pages retrieved from one or more web servers including the electronic a copy of the web page hard copy, each text searchable version of the plurality of electronic copies of web pages including control information which links one or more text strings contained in the plurality of electronic copies of web pages to other linked copies of web pages;

identifying the matching electronic copy of the web page hard copy;

E) retrieving the control information associated with the captured web page hard copy from the matching electronic copy of the web page hard copy; and

F) generating and displaying on the mobile handheld image capturing device display a processed image of the matching electronic copy of the web page hard copy horizontally and vertically aligned to the display substantially equivalent to the captured image alignment horizontally and/or vertically skewed relative to the displayed angle of view associated with the physically displayed web page, the processed image including the retrieved control information associated with the matching electronic copy of the web page hard copy and the processed image including a modified appearance of the one or more text strings associated with the web page hard copy which are linked to one or more other electronic copies of web pages retrieved from the one or more web servers, the modified appearance indicating the one or more text strings are active and the modified appearance of the text strings at substantially the same angle as the captured image horizontally and/or vertically skewed relative to the displayed angle of view associated with the physically displayed web page.

14. The computer program product according to claim 13, the method further comprising:

receiving a user selection of one of the active text strings and displaying the electronic copy of the linked web page associated with the active text string.

15. The computer program product according to claim 13, wherein the mobile hand held image capturing device is one of a smart phone, mobile computing device, and laptop computer.

16. The computer program product according to claim 13, wherein step E) displays the processed image of the web page hard copy using a native browser associated with the mobile hand held image capturing device.

17. The computer program product according to claim 13, wherein step E) displays an augmented image including the retrieved control information associated with the matching electronic copy of the web page hard copy, and the augmented image includes an overlay on the captured image which modifies the appearance of one or more text strings associated with the web page hard copy which are linked to one or more other electronic copies of web pages retrieved from one or more web servers, the modified appearance indicating the one or more text strings are active.

18. The computer program product according to claim 13, the method further comprising:

a user selecting one of the active text strings and the mobile hand held image capturing device display displaying the electronic copy of the linked web page associated with the active text string; and

the user selecting a second active text string associated with the electronic copy of the linked web page and the mobile hand held image capturing device display displaying an electronic copy of the linked web page associated with the second active text string.