Key input using an active pixel camera

• US 9,176,539 B2
• Filed: 11/10/2012
• Issued: 11/03/2015
• Est. Priority Date: 11/10/2012
• Status: Active Grant

• Current Assignee: eBay Inc.
• Original Assignee: eBay Inc.
• Inventors: Tapley, John, Farraro, Eric J., Lyman, Nate
• Primary Examiner(s): GIESY, ADAM
• Application Number:
  US13/674,004
• Publication Number:
  US 20140132564A1
• Time in Patent Office: 1,088 Days
• Field of Search: None
• US Class Current: 1/1
• CPC Class Codes: G06F 1/1686 the I/O peripheral being an integrated camera
• CPC Class Codes: G06F 3/042 by opto-electronic means
• CPC Class Codes: G06F 3/0425 using a single imaging device like a video camera for tracking the absolute position of a single or a plurality of objects with respect to an imaged reference surface, e.g. video camera imaging a display or a projection screen, a table or a wall surface, on which a computer generated image is displayed or projected tracking a projected light spot to determine a position on a display surface G06F3/0386

Abstract

In an example embodiment, an active pixel sensor on a user device is utilized to capture graphical user interface navigation related movements by a user. Areas of low luminance can be identified and movements or alterations in the areas of low luminance can be translated into navigation commands fed to an application running on the user device.

Claims

1. A method for performing graphical user interface navigating on a user device using an active pixel sensor located on the user device, the method comprising:

capturing a first image using an active pixel sensor, the active pixel sensor comprising a plurality of photosensors, each photosensor corresponding to a different pixel in a two-dimensional array of pixels;

detecting a low luminance area of the first image, the low luminance area caused by an object blocking light projected by a light source outside of the user device from reaching the active pixel sensor;

capturing a second image using the active pixel sensor;

detecting a movement of the low luminance area between the first image and the second image;

determining a direction based at least in part on the movement of the low luminance area; and

translating the direction into an input for an application running on the user device.

2. The method of claim 1, further comprising:

determining a distance based at least in part on the movement of the low luminance area; and

using the distance when translating the direction into the input for the application running on the user device.

3. The method of claim 1, wherein the application running on the user device is not an application designed to capture images from the active pixel sensor.

4. The method of claim 1, wherein the active pixel sensor is located on a front side of the user device.

5. The method of claim 1, wherein the active pixel sensor is located on a back side of the user device.

6. The method of claim 1, wherein the detecting the low luminance area includes locating an area having luminance values less than a predetermined luminance threshold and having a size greater than or equal to a size threshold.

7. A method for performing graphical user interface navigating on a user device using an active pixel sensor located on the user device, comprising:

capturing a first image using the active pixel sensor, the active pixel sensor comprising a plurality of photosensors, each photosensor corresponding to a different pixel in a two-dimensional array of pixels;

detecting an area of the first image having low luminance, the low luminance area caused by an object blocking light projected by a light source outside of the user device from reaching the active pixel sensor;

capturing a second image using the active pixel sensor;

capturing a third image using the active pixel sensor;

detecting an alteration in the detected area among the first image, the second image, and the third image such that the detected area changes from low luminance in the first image to high luminance in the second image and then back to low luminance in the third image and the first, second, and third images are captured within a predetermined time period of each other; and

generating navigation input to an application running on the user device based at least in part on detecting the alteration in the detected area, the navigation input indicating detection of a tapping motion.

8. The method of claim 7, further comprising:

further detecting an additional alteration in a detected area such that the detected area changes again from low luminance to high luminance and then back to low luminance within the predetermined time period; and

generating navigation input to the application, the navigation input indicating the detection of a double tapping motion.

9. The method of claim 7, wherein the user device has no physical buttons capable of providing navigation input to the application.

10. A user device comprising:

a processor;

memory;

a touchscreen;

an active pixel sensor located opposite the touchscreen and configured to act as a camera while a camera application is executing by the processor, and configured to act as an input device while a non-camera application is executing by the process, the active pixel sensor comprising a plurality of photosensors, each photosensor corresponding to a different pixel in a two-dimensional array of pixels and further configured to capture a first image and a second image; and

an active pixel sensory monitor configured to;

detect a low luminance area of the first image, the low luminance area caused by an object blocking light projected by a light source outside of the user device from reaching the active pixel sensor;

detect movement of the low luminance area between the first image and the second image;

determine a direction for the movement of the low luminance area; and

translate the direction into navigation input for an application running on the user device.

11. The user device of claim 10, wherein the active pixel sensor is located on a front of the user device.

12. The user device of claim 10, wherein the active pixel sensor is located on a back of the user device.

13. The user device of claim 12, wherein translating the direction includes generating a navigation input in a direction opposite from the direction determined for the movement of the low luminance area.

14. The user device of claim 10, wherein the user device is a smartphone.

15. The user device of claim 10, wherein the user device is a tablet computer.

16. A user device having a front and a back, comprising:

a processor;

memory;

a touchscreen located on the front of the user device;

an active pixel sensor configured to act as a camera while a camera application is executing by the processor, and configured to act as an input device while a non-camera application is executing by the process, the active pixel sensor comprising a plurality of photosensors, each photosensor corresponding to a different pixel in a two-dimensional array of pixels and further configured to capture a first image, a second image, and a third image; and

an active pixel sensor monitor configured to;

detect an area of the first image having low luminance, the low luminance area caused by an object blocking light projected by a light source outside of the user device from reaching the active pixel sensor;

detect an alteration in the detected area among the first image, the second image, and the third image such that the detected area changes from low luminance in the first image to high luminance in the second image and then back to low luminance in the third image, wherein the first image, the second image, and the third image were captured within a predetermined time period of each other; and

generate navigation input to an application running on the user device, the navigation input indicating detection of a tapping motion.

17. The user device of claim 16, wherein the user device lacks a button dedicated to receiving a user indication of a selection of an item displayed on the touchscreen.

18. The user device of claim 16, wherein the active pixel sensors is one of two identical active pixel sensors on the user device.

19. A non-transitory machine-readable storage medium including instructions that, when executed by a machine, cause the machine to perform operations comprising:

selecting a user input mode for data received from an active pixel sensor;

receiving sensor data from the active pixel sensor, the active pixel sensor comprising a plurality of photosensors, each photosensor corresponding to a different pixel in a two-dimensional array of pixels;

parsing, over a period of time, received sensor data into a plurality of input events; and

generating input to an application running on the machine based at least in part on the plurality of input events, wherein the plurality of input events include movement of a low luminance area, the low luminance area caused by an object blocking light projected by a light source outside of the user device from reaching the active pixel sensor.

20. The non-transitory machine-readable storage medium of claim 19, wherein the plurality of input events include a series of light to dark transitions indicating a tapping motion.