Methods and apparatuses for applying free space inputs for surface constrained controls

• US 10,620,753 B2
• Filed: 10/09/2018
• Issued: 04/14/2020
• Est. Priority Date: 05/15/2015
• Status: Active Grant

• Current Assignee: West Texas Technology Partners LLC
• Original Assignee: Atheer, Inc.
• Inventors: Kandadai, Shashwat, Abercrombie, Nathan, Chen, Yu-Hsiang, Itani, Sleiman
• Primary Examiner(s): Johnson, Gerald
• Application Number:
  US16/155,678
• Publication Number:
  US 20190050112A1
• Time in Patent Office: 553 Days
• CPC Class Codes: G06F 3/017 Gesture based interaction, e.g. based on a set of recognized hand gestures interaction based on gestures traced on a digitiser G06F3/04883
• CPC Class Codes: G06F 3/0346 with detection of the device orientation or free movement in a 3D space, e.g. 3D mice, 6-DOF [six degrees of freedom] pointers using gyroscopes, accelerometers or tilt-sensors
• CPC Class Codes: G06F 3/0416 Control or interface arrangements specially adapted for digitisers
• CPC Class Codes: G06F 3/0425 using a single imaging device like a video camera for tracking the absolute position of a single or a plurality of objects with respect to an imaged reference surface, e.g. video camera imaging a display or a projection screen, a table or a wall surface, on which a computer generated image is displayed or projected tracking a projected light spot to determine a position on a display surface G06F3/0386
• CPC Class Codes: G06F 3/04815 Interaction with a metaphor-based environment or interaction object displayed as three-dimensional, e.g. changing the user viewpoint with respect to the environment or object
• CPC Class Codes: G06F 3/0488 using a touch-screen or digitiser, e.g. input of commands through traced gestures
• CPC Class Codes: G06F 3/04883 for inputting data by handwriting, e.g. gesture or text
• CPC Class Codes: G06T 19/00 Manipulating 3D models or images for computer graphics

Abstract

A method, system, apparatus, and/or device for sensing an input in an augmented reality construct. The method, system, apparatus, and/or device may include a processing device and a sensor. The processing device may be configured to determine a virtual boundary within an augmented reality construct, define a first portion of the virtual boundary within the augmented reality construct, and define a second portion of the virtual boundary within the augmented reality construct. The sensor may be sense a first input from an end-effector within the first portion of the virtual boundary and sense a second input from the end-effector within the second portion of the virtual boundary, where the processing device may further generate a device input that corresponds to the first input and the second input and send the device input to another device.

Claims

1. A method, comprising:

determining, by a processing device, a surface-constrained input standard for a surface-constrained input generated approximate to a physical surface;

determining, by the processing device, is a virtual boundary within an augmented reality construct;

defining, by the processing device, a first portion of the virtual boundary to receive a first input;

defining, by the processing device, a second portion of the virtual boundary to receive a second input;

defining, by the processing device, a third portion of the virtual boundary to receive a third input;

sensing, by a sensor, a first input within the first portion of the virtual boundary;

sensing, by the sensor, a second input within the second portion of the virtual boundary;

sensing, by the sensor, a third input within the third portion of the virtual boundary;

generating, by the processing device, a device input that corresponds to the first input, the second input, and the third input; and

sending the device input to a device.

2. The method of claim 1, further comprising:

determining, by the processing device, a first input standard for the first input;

determining, by the processing device, a second input standard for the second input; and

determining, by the processing device, a third input standard for the third input.

3. The method of claim 2, wherein:

the first input standard defines a touch down motion where an end-effector extends downwardly from a first plane to a second plane;

the second input standard defines a swipe motion where the end-effector extends along the second plane from a first point to a second point; and

the third input standard defines a liftoff motion where the end-effector extends upwardly from the second plane to a third plane.

4. The method of claim 3, wherein the end-effector comprises one or more fingers of an individual.

5. The method of claim 3, wherein:

the touch down motion extends downwardly at a first angle; and

the liftoff motion extends upwardly at a second angle.

6. The method of claim 3, further comprising:

determining that the first input meets the first input standard;

determining that the second input meets the second input standard; and

determining that the third input meets the third input standard.

7. The method of claim 6, further comprising generating the device input when the first input meets the first input standard, the second input meets the second input standard, the third input meets the third input standard.

8. The method of claim 1, wherein a width of the second portion of the virtual boundary is substantially similar to a width of the physical surface.

9. The method of claim 1, wherein the device is only configured to receive the surface-constrained input.

10. The method of claim 8, wherein the device is configured to only receive an analog touch input that conforms to an analog touch input standard.

11. The method of claim 1, further comprising identifying the surface-constrained input that is associated with the first input, the second input, and the third input, wherein the device input corresponds to the surface-constrained input.

12. The method of claim 1, wherein the surface-constrained input is an input received on the surface of a touchscreen device.

13. A device, comprising:

a processing device configured to;

determine a virtual boundary within an augmented reality construct;

define a first portion of the virtual boundary within the augmented reality construct;

define a second portion of the virtual boundary within the augmented reality construct; and

a sensor coupled to the processing device, wherein the sensor is configured to;

sense a first input from an end-effector within the first portion of the virtual boundary; and

sense a second input from the end-effector within the second portion of the virtual boundary, wherein the processing device is further configured to;

generate a device input that corresponds to the first input and the second input; and

send the device input to another device.

14. The device of claim 13, wherein the processing device is further configured to associate a surface-constrained input with the device input, wherein the surface-constrained input is an input generated approximate to a physical surface.

15. The device of claim 13, wherein the processing device is further configured to limit a bounce acceleration of at least one of the first input or the second input, wherein the bounce acceleration is a motion indicative of a downward motion that is rapidly reversed to an upward motion.

16. The device of claim 13, wherein:

the first input comprises at least one of;

a touch down motion where the end-effector extends downwardly from a first plane to a second plane;

or

a liftoff motion where the end-effector extends upwardly from the second plane to a third plane; and

the second input comprises a swipe motion where the end-effector extends along the second plane from a first point to a second point.

17. The device of claim 16, wherein the processing device is further configured to:

determine a threshold distance range for the end-effector to extend along the second plane; and

generate the device input when the end-effector extends along the second plane within the threshold distance range.

18. The device of claim 16, wherein:

the end-effector is a hand of a user; and

the processing device is further configured to;

determine a threshold distance range for a knuckle of the hand to extend along the second plane; and

generate the device input when the knuckle extends along the second plane within the threshold distance range.

19. An apparatus, comprising:

a processing device configured to;

determine a virtual boundary within an augmented reality construct;

define a first portion of the virtual boundary within the augmented reality construct;

define a second portion of the virtual boundary within the augmented reality construct; and

a sensor coupled to the processing device, wherein the sensor is configured to;

sense a first input from an end-effector within the first portion of the virtual boundary, the first input being a first motion where an end-effector moves from a first plane to a second plane; and

sense a second input from the end-effector within the second portion of the virtual boundary, the second input being a second motion where the end-effector moves from along the a second plane, wherein the processing device is further configured to;

generate a device input that corresponds to the first input and the second input; and

send the device input to a device.

20. The apparatus of claim 19, wherein the processing device is further configured to:

determine a threshold distance range for the end-effector to move from the first plane to the second plane; and

generate the device input when the end-effector extends moves from the first plane to the second plane within the threshold distance range.