Method and apparatus for applying free space input for surface constrained control

• US 10,126,881 B2
• Filed: 04/24/2017
• Issued: 11/13/2018
• Est. Priority Date: 05/15/2015
• Status: Active Grant

• Current Assignee: West Texas Technology Partners LLC
• Original Assignee: Atheer, Inc.
• Inventors: Kandadai, Shashwat, Abercrombie, Nathan, Chen, Yu-Hsiang, Itani, Sleiman
• Primary Examiner(s): Johnson, Gerald
• Application Number:
  US15/495,864
• Publication Number:
  US 20180101278A1
• Time in Patent Office: 568 Days
• CPC Class Codes: G06F 3/017 Gesture based interaction, e.g. based on a set of recognized hand gestures interaction based on gestures traced on a digitiser G06F3/04883
• CPC Class Codes: G06F 3/0346 with detection of the device orientation or free movement in a 3D space, e.g. 3D mice, 6-DOF [six degrees of freedom] pointers using gyroscopes, accelerometers or tilt-sensors
• CPC Class Codes: G06F 3/0416 Control or interface arrangements specially adapted for digitisers
• CPC Class Codes: G06F 3/0425 using a single imaging device like a video camera for tracking the absolute position of a single or a plurality of objects with respect to an imaged reference surface, e.g. video camera imaging a display or a projection screen, a table or a wall surface, on which a computer generated image is displayed or projected tracking a projected light spot to determine a position on a display surface G06F3/0386
• CPC Class Codes: G06F 3/04815 Interaction with a metaphor-based environment or interaction object displayed as three-dimensional, e.g. changing the user viewpoint with respect to the environment or object
• CPC Class Codes: G06F 3/0488 using a touch-screen or digitiser, e.g. input of commands through traced gestures
• CPC Class Codes: G06F 3/04883 for inputting data by handwriting, e.g. gesture or text
• CPC Class Codes: G06T 19/00 Manipulating 3D models or images for computer graphics

Abstract

A free space input standard is instantiated on a processor. Free space input is sensed and communicated to the processor. If the free space input satisfies the free space input standard, a touch screen input response is invoked in an operating system. The free space input may be sensed using continuous implicit, discrete implicit, active explicit, or passive explicit approaches. The touch screen input response may be invoked through communicating virtual touch screen input, a virtual input event, or a virtual command to or within the operating system. In this manner free space gestures may control existing touch screen interfaces and devices, without modifying those interfaces and devices directly to accept free space gestures.

Claims

1. A method, comprising:

determining, by a processor, a surface constrained input standard for a surface constrained input, wherein the surface constrained input is an input generated approximate to a physical surface;

determining a free space input boundary to receive a free space input, wherein the free space input boundary is a virtual boundary within an augmented reality construct that is bounded by a virtual space with a dimension substantially similar to the physical surface;

determining, by the processor, a free space input standard for a free space input, wherein the free space input standard comprises;

a first substandard with a first input parameter;

a second substandard with a second input parameter;

sensing, by a sensor, a free space input within the free space input boundary;

determining that a first portion of the free space input meets the first input parameter;

determining that a second portion of the free space input meets the second input parameter;

identifying the surface constrained input that is associated with the free space input;

generating a device input that corresponds to the surface constrained input; and

sending the device input to a device not configured to receive the free space input.

2. The method of claim 1, wherein:

the device is configured to receive an analog touch input; and

the device input conforms to an analog touch input standard.

3. The method of claim 1, wherein the free space input comprises a three dimensional input.

4. The method of claim 1, wherein the free space input comprises a hand input.

5. The method of claim 1, wherein:

the surface constrained input standard comprises a touch screen input standard;

the surface constrained input comprises a touch screen input;

the free space input comprises a virtual touch screen input;

or

the device input comprises an analog touch screen command.

6. The method of claim 1, wherein the device input comprises a control function adapted to control a system in communication with the processor.

7. The method of claim 1, wherein:

the first input parameter include at least one of a finger movement, a shape parameter, a velocity parameter, an acceleration parameter, or a joint position; and

the second input parameter excludes at least one of the finger movement, the shape parameter, the velocity parameter, the acceleration parameter, or the joint position.

8. The method of claim 1, comprising excluding a movement of the free space input that is outside the free space input boundary.

9. A method, comprising:

determining, by a processor, a surface constrained input standard for a surface constrained input, wherein the surface constrained input is an input generated approximate to a physical surface;

determining, by the processor, a free space input standard for a free space input, wherein the free space input standard comprises;

a first substandard with a first input parameter;

a second substandard with a second input parameter;

sensing, by a sensor the free space input within the free space input boundary;

determining that a first portion of the free space input meets the first input parameter;

determining that a second portion of the free space input meets the second input parameter; and

sending the free space input to a device.

10. The method of claim 9, wherein:

the device is configured to receive an analog touch input; and

the free space input conforms to an analog touch input standard.

11. The method of claim 9, wherein the free space input comprises a three dimensional input.

12. The method of claim 9, wherein the free space input comprises a hand input.

13. The method of claim 9, wherein:

the surface constrained input standard comprises a touch screen input standard;

the surface constrained input comprises touch screen input;

the free space input comprises virtual touch screen input;

or

the device input comprises an analog touch screen command.

14. The method of claim 9, wherein the free space input comprises a control function adapted to control a system in communication with the processor.

15. The method of claim 9, wherein:

the first portion of the free space input is sensed by the sensor at a first point in time; and

the second portion of the free space input is sensed by the sensor at a second point in time.

16. The method of claim 9, wherein the device is configured to receive an analog touch input and not the free space input.

17. The method of claim 9, further comprising:

determining a drift threshold indicative of a movement precision range of the free space input; and

determining that the free space input is within the drift threshold.

20. The method of claim 15, wherein the device is configured to receive an analog touch input and not the free space input.

21. The method of claim 15, further comprising excluding movements for the free space input that are outside an input boundary.

22. The method of claim 21, wherein the input boundary is an augmented reality construct that is bounded by a virtual space with a dimension substantially similar to the physical surface.

23. The method of claim 15, wherein the first sensor is a first type of sensor and the second sensor is a second type of sensor that is different than the first type of sensor.

24. The method of claim 15, wherein the free space input changes over a period of time.

18. A method, comprising:

determining, by a processor, a surface constrained input standard for a surface constrained input, wherein the surface constrained input is an input generated approximate to a physical surface;

determining, by the processor, a free space input standard for a free space input, wherein the free space input comprises;

a first substandard with a first input parameter;

a second substandard with a second input parameter;

sensing, by a first sensor, a first portion of the free space input;

determining that the first portion of the free space input meets the first input parameter;

sensing, by a second sensor, a second portion of the free space input;

determining that the second portion of the free space input meets the second input parameter;

generating a device input that corresponds to the surface constrained input; and

sending the device input to a device.

19. The method of claim 18, wherein:

the device is configured to receive an analog touch input; and

the device input conform to an analog touch input standard.