Portable device using passive sensor for initiating touchless gesture control

• US 9,733,763 B2
• Filed: 04/03/2014
• Issued: 08/15/2017
• Est. Priority Date: 04/11/2013
• Status: Active Grant

• Current Assignee: Crunchfish Gesture Interaction AB
• Original Assignee: Crunchfish AB
• Inventors: Cronholm, Paul, Henriz, Martin
• Primary Examiner(s): Cheng, Joe H
• Application Number:
  US14/780,292
• Publication Number:
  US 20160054858A1
• Time in Patent Office: 1,230 Days
• CPC Class Codes: G06F 1/3231 Monitoring the presence, absence or movement of users
• CPC Class Codes: G06F 1/325 Power saving in peripheral device
• CPC Class Codes: G06F 2203/0381 Multimodal input, i.e. interface arrangements enabling the user to issue commands by simultaneous use of input devices of different nature, e.g. voice plus gesture on digitizer
• CPC Class Codes: G06F 2203/04108 Touchless 2D- digitiser, i.e. digitiser detecting the X/Y position of the input means, finger or stylus, also when it does not touch, but is proximate to the digitiser's interaction surface without distance measurement in the Z direction
• CPC Class Codes: G06F 3/005 Input arrangements through a video camera
• CPC Class Codes: G06F 3/011 Arrangements for interaction with the human body, e.g. for user immersion in virtual reality blind teaching G09B21/00
• CPC Class Codes: G06F 3/017 Gesture based interaction, e.g. based on a set of recognized hand gestures interaction based on gestures traced on a digitiser G06F3/04883
• CPC Class Codes: G06F 3/042 by opto-electronic means
• CPC Class Codes: G06F 3/0425 using a single imaging device like a video camera for tracking the absolute position of a single or a plurality of objects with respect to an imaged reference surface, e.g. video camera imaging a display or a projection screen, a table or a wall surface, on which a computer generated image is displayed or projected tracking a projected light spot to determine a position on a display surface G06F3/0386
• CPC Class Codes: G06F 3/04883 for inputting data by handwriting, e.g. gesture or text
• CPC Class Codes: Y02D 10/00 Energy efficient computing, e.g. low power processors, power management or thermal management

Abstract

A portable device (100) comprising a controller (210), a memory (240), a camera (160) and a passive proximity sensor (170). The controller being configured for receiving input from said passive proximity sensor (170) indicating a light environment in front of the passive proximity sensor (170) detecting a change in light environment, determining a light pattern (P) of changes in light environment, determining if the light pattern (P) matches a stored initiating pattern, which is stored in the memory (240), and if so, accepting the light pattern (P) and in response thereto activating the camera (160), identifying an object (H) in front of the camera (160) and tracking the object (H) thereby enabling touchless gesture control of the portable device (100), characterized in that the passive proximity sensor is an ambient light sensor and the light pattern consists of a series of changes between dark and light (dark is light below a threshold, light is light above a second threshold).

Claims

1. A portable device comprising:

a controller;

a memory;

a camera; and

a passive proximity sensor, wherein

the controller is configured to;

receive input from said passive proximity sensor indicating a light environment in front of the passive proximity sensor;

detect a change in the light environment;

determine a light pattern of changes in the light environment;

determine if the light pattern matches a stored initiating pattern, which is stored in the memory, and if so, accept the light pattern; and

in response to accepting the light pattern, being configured to

activate the camera,

identify an object in front of the camera, and

track the object, thereby enabling touchless gesture control of the portable device;

the passive proximity sensor is an ambient light sensor; and

the light pattern consists of a series of changes between dark and light, where dark is a light intensity below a threshold, and light is a light intensity above a second threshold.

2. The portable device of claim 1, wherein the passive proximity sensor is configured to detect a presence of an object and the light pattern is a proximity pattern.

3. The portable device of claim 2, wherein

the controller is further configured to determine a direction in which a change in light occurs, and

the light pattern further consists of the directions of the light changes.

4. The portable device of claim 2, wherein the controller is further configured to:

prompt for a confirmation,

detect a confirmation light pattern, and

determine if the confirmation light pattern indicates a confirmation and if so accept the light pattern.

5. The portable device of claim 1, wherein the changes are in a time relation with each other.

6. The portable device of claim 5, wherein

the controller is further configured to determine a direction in which a change in light occurs, and

the light pattern further consists of the directions of the light changes.

7. The portable device of claim 5, wherein the controller is further configured to:

prompt for a confirmation, and

detect a confirmation light pattern, and

determine if the confirmation light pattern indicates a confirmation and if so accept the light pattern.

8. The portable device of claim 1, wherein the changes are in a time-relation to a full pattern, and

wherein the controller is further configured to buffer the detected light changes and detect the light pattern by retro-checking the detected light changes.

9. The portable device of claim 8, wherein

the controller is further configured to determine a direction in which a change in light occurs, and

the light pattern further consists of the directions of the light changes.

10. The portable device of claim 1, wherein

the controller is configured to determine a direction in which a change in light occurs;

and

the light pattern further consists of the directions of the light changes.

11. The portable device of claim 1, wherein the controller is further configured to

prompt for a confirmation;

detect a confirmation light pattern; and

determine if the confirmation light pattern indicates a confirmation and if so accept the light pattern.

12. The portable device of claim 1, further comprising:

a movement sensor; and

wherein the controller is further configured to

receive movement input from the movement sensor,

determine if the portable device is moving or not while detecting the light pattern, and

accept the pattern if the portable device is not moving.

13. The portable device of claim 12, wherein the controller is further configured to prompt for a confirmation if it is detected that the portable device is moving while detecting the light pattern.

14. The portable device of claim 12, wherein the controller is further configured to

determine if the detected movement constitutes a moving pattern,

determine if the moving pattern matches the light pattern, and

if there is a match, discard the light pattern or prompt for confirmation.

15. The portable device of claim 1, further comprising:

a second passive proximity sensor; and

wherein the pattern includes at least two sub-patterns, one sub-pattern for each passive proximity sensor.

16. The portable device of claim 15, wherein the at least two sub-patterns are timed with each other so that the pattern changes occur at approximately the same time or are overlapping.

17. The portable device of claim 1, wherein the portable device is a mobile communications terminal.

18. The portable device of claim 17, wherein the mobile communications terminal is a mobile phone.

19. A method for use in a portable device, the portable device including a controller, a memory, a camera and a passive proximity sensor, the method comprising:

receiving input from said passive proximity sensor indicating a light environment in front of the passive proximity sensor;

detecting a change in the light environment;

determining a light pattern of changes in the light environment;

determining if the light pattern matches a stored initiating pattern, which is stored in the memory, and if so, accepting the light pattern; and

in response to accepting the light pattern;

activating the camera,

identifying an object in front of the camera, and

tracking the object, thereby enabling touchless gesture control of the portable device, wherein

the passive proximity sensor is an ambient light sensor, and

the light pattern consists of a series of changes between dark and light, where dark is a light intensity below a threshold, and light is a light intensity above a second threshold.

20. A non-transitory computer readable storage medium encoded with instructions that, when loaded and executed on a controller of a portable device, causes the controller to perform the method according to claim 19.