Apparatus and method to detect a paper document using one or more sensors

• US 9,218,526 B2
• Filed: 05/24/2012
• Issued: 12/22/2015
• Est. Priority Date: 05/24/2012
• Status: Expired due to Fees

• Current Assignee: IP3 2017, Series 200 of Allied Security Trust I (Allied Security Trust)
• Original Assignee: HJ Laboratories, LLC
• Inventors: Vartanian, Harry, Jurikson-Rhodes, Jaron
• Primary Examiner(s): HOGUE, DENNIS A
• Application Number:
  US13/480,107
• Publication Number:
  US 20130314555A1
• Time in Patent Office: 1,307 Days
• Field of Search: 348/36, 348/38, 348/61, 34820799-2072, 348/218.1, 348/159, 348/169, 382/112, 382/113, 382/135, 382/137, 382/100, 358/302, 358/462, 358/482, 358/483, 358/473, 707/3, 707/6, 707/765, 455/556.1, 340/825.49
• US Class Current: 1/1
• CPC Class Codes: G06F 16/955 using information identifiers, e.g. uniform resource locators [URL]
• CPC Class Codes: G06F 3/017 Gesture based interaction, e.g. based on a set of recognized hand gestures interaction based on gestures traced on a digitiser G06F3/04883
• CPC Class Codes: G06T 2207/30176 Document
• CPC Class Codes: G06V 30/40 Document-oriented image-based pattern recognition
• CPC Class Codes: H04N 1/00827 Arrangements for reading an image from an unusual original, e.g. 3-dimensional objects
• CPC Class Codes: H04N 1/19594 using a television camera or a still video camera
• CPC Class Codes: H04N 13/25 using two or more image sensors with different characteristics other than in their location or field of view, e.g. having different resolutions or colour pickup characteristics; using image signals from one sensor to control the characteristics of another sensor
• CPC Class Codes: H04N 2013/0081 Depth or disparity estimation from stereoscopic image signals
• CPC Class Codes: H04N 2201/0084 Digital still camera
• CPC Class Codes: H04N 23/61 based on recognised objects

Abstract

An apparatus and method for detecting paper documents, books, or other objects using one or more sensors on one or more computing devices is disclosed. The one or more computing devices communicate and share information such that a paper document, book, or other object is detected and identified. Once identified, information relevant to the paper document, book, or other object is retrieved. Such an apparatus and method is disclosed to help to create a smart or interactive paper environment.

Claims

1. A mobile device comprising:

a multi-touch screen configured to display an image;

a rear facing camera and a depth sensor configured to capture a portion of a paper document or a book in an environment;

two or more cameras configured to capture part of the environment and to capture another part of the environment;

a processor configured to identify the paper document or the book based on the captured portion, the captured part of the environment, the captured another part of the environment, and information received via a network adapter; and

wherein tracking information of a desk the identified paper document or the book has been is retrieved by the mobile device.

2. The mobile device of claim 1 further comprising:

the processor is configured to utilize assistance information for the identification of the paper document or the book; and

wherein the assistance information is obtained, by the mobile device, from the captured part of the environment or the captured another part of the environment.

3. The mobile device of claim 1, wherein additional information related to the identified paper document or the book is received, by the mobile device via the network adapter, from a cloud based server.

4. A method performed by a mobile device, the method comprising:

displaying, by the mobile device on a multi-touch screen, an image;

capturing, by a rear facing camera and a depth sensor of the mobile device, a portion of a paper document or a book in an environment;

capturing, by two or more cameras of the mobile device, part of the environment and another part of the environment;

identifying, by the mobile device, the paper document or the book based on the captured portion, the captured part of the environment, the captured another part of the environment, and information received via a network adapter; and

wherein tracking information of a desk the identified paper document or the book has been is retrieved by the mobile device.

5. The method of claim 4, further comprising:

utilizing, by the mobile device, assistance information for the identification of the paper document or the book; and

wherein the assistance information is obtained, by the mobile device, from the captured part of the environment or the captured another part of the environment.

6. The method of claim 4, wherein additional information related to the identified paper document or the book is received, by the mobile device via the network adapter, from a cloud based server.

7. A mobile device comprising:

a multi-touch screen configured to display an image;

a rear facing camera and a depth sensor configured to capture a portion of a paper document in an environment;

two or more cameras configured to capture part of the environment and to capture another part of the environment;

a processor configured to identify the paper document based on the captured portion, the captured part of the environment, the captured another part of the environment, and information received via a network adapter; and

wherein tracking information of a desk the identified paper document has been is retrieved by the mobile device.

8. The mobile device of claim 7 further comprising:

the processor is configured to utilize assistance information for the identification of the paper document; and

wherein the assistance information is obtained, by the mobile device, from the captured part of the environment or the captured another part of the environment.

9. The mobile device of claim 7, wherein additional information related to the identified paper document is received, by the mobile device via the network adapter, from a cloud based server.