Method and apparatus for determining user input in a three-dimensional field

• US 9,367,204 B2
• Filed: 09/24/2014
• Issued: 06/14/2016
• Est. Priority Date: 01/14/2013
• Status: Expired due to Fees

• Current Assignee: Darren Lim, Lai Xue
• Original Assignee: Darren Lim, Lai Xue
• Inventors: Xue, Lai, Lim, Darren
• Primary Examiner(s): HALEY, JOSEPH R
• Application Number:
  US14/495,611
• Publication Number:
  US 20150012891A1
• Time in Patent Office: 629 Days
• Field of Search: None
• US Class Current: 1/1
• CPC Class Codes: G06F 1/1662 Details related to the integrated keyboard
• CPC Class Codes: G06F 1/1686 the I/O peripheral being an integrated camera
• CPC Class Codes: G06F 1/169 the I/O peripheral being an integrated pointing device, e.g. trackball in the palm rest area, mini-joystick integrated between keyboard keys, touch pads or touch stripes G06F1/1643 takes precedence; constructional details of pointing devices G06F3/033
• CPC Class Codes: G06F 2203/04808 Several contacts: gestures triggering a specific function, e.g. scrolling, zooming, right-click, when the user establishes several contacts with the surface simultaneously; e.g. using several fingers or a combination of fingers and pen
• CPC Class Codes: G06F 3/017 Gesture based interaction, e.g. based on a set of recognized hand gestures interaction based on gestures traced on a digitiser G06F3/04883
• CPC Class Codes: G06F 3/0233 Character input methods
• CPC Class Codes: G06F 3/0425 using a single imaging device like a video camera for tracking the absolute position of a single or a plurality of objects with respect to an imaged reference surface, e.g. video camera imaging a display or a projection screen, a table or a wall surface, on which a computer generated image is displayed or projected tracking a projected light spot to determine a position on a display surface G06F3/0386
• CPC Class Codes: G06F 3/0426 tracking fingers with respect to a virtual keyboard projected or printed on the surface virtual keyboards on touch screens G06F3/04886
• CPC Class Codes: G06F 3/04815 Interaction with a metaphor-based environment or interaction object displayed as three-dimensional, e.g. changing the user viewpoint with respect to the environment or object
• CPC Class Codes: G06F 3/0488 using a touch-screen or digitiser, e.g. input of commands through traced gestures
• CPC Class Codes: G06F 3/04883 for inputting data by handwriting, e.g. gesture or text
• CPC Class Codes: G06F 3/04886 by partitioning the display area of the touch-screen or the surface of the digitising tablet into independently controllable areas, e.g. virtual keyboards or menus

Abstract

In accordance with one implementation, a method is illustrated that allows a computing device to determine a user input. The method includes detecting one or more user input objects in a 3-dimensional field relative to a 2-dimensional surface. The method also includes determining coordinates for the one or more user input objects relative to the 2-dimensional surface. And, the method further includes determining a user input based on the coordinates.

Claims

1. A method of determining user input in a 3D field relative to a 2D surface, the method comprising:

defining an inner perimeter;

defining an outer perimeter;

defining a lower perimeter;

defining an upper perimeter;

defining a first side perimeter;

defining a second side perimeter;

wherein said inner perimeter intersects said lower perimeter, said upper perimeter, said first side perimeter, and said second side perimeter;

wherein said outer perimeter intersects said lower perimeter, said upper perimeter, said first side perimeter, and said second side perimeter;

wherein said upper perimeter intersects said inner perimeter, said outer perimeter, said first side perimeter, and said second side perimeter;

wherein said lower perimeter intersects said inner perimeter, said outer perimeter, said first side perimeter, and said second side perimeter;

wherein said first side perimeter intersects said inner perimeter, said outer perimeter, said upper perimeter, and said lower perimeter;

wherein said second side perimeter intersects said inner perimeter, said outer perimeter, said upper perimeter, and said lower perimeter;

wherein said upper perimeter, lower perimeter, inner perimeter, outer perimeter, first side perimeter, and second side perimeter define a 3-dimensional detection field;

detecting one or more user input objects in said 3-dimensional detection field relative to a 2-dimensional surface;

determining 3-dimensional coordinates for the one or more user input objects within said 3-dimensional detection field relative to the 2-dimensional surface;

determining a touch user input based on the coordinates;

determining a touchless user input based on a second set of 3-dimensional coordinates within said 3-dimensional field associated with the one or more user input objects.

2. The method of claim 1 and further comprising:

mapping coordinates corresponding to the position of at least one of the input objects relative to the 2-dimensional surface to a display surface which is separate and distinct from the 2-dimensional surface.

3. The method of claim 1 wherein the detecting one or more user input objects comprises:

utilizing capacitive sensors in keys of a keyboard to detect the one or more user input objects.

4. The method of claim 3 wherein the keys of the keyboard delineate the 2 dimensional surface.

5. The method of claim 1 and further comprising:

utilizing at least one camera to detect the one or more user input objects.

6. The method of claim 1 and further comprising:

utilizing optical sensors in keys of a keyboard to detect the one or more user input objects.

7. The method of claim 6 wherein the keys of the keyboard delineate the 2 dimensional surface.

8. The method of claim 1 wherein determining a user input comprises determining a hover event or determining a touch event, said method further comprising turning off a hover mode when a touch event is detected and displaying a user input on the display surface.

9. One or more non-transitory computer-readable storage media encoding computer-executable instructions for executing on a computer system a computer process, the computer process comprising:

defining an inner perimeter;

defining an outer perimeter;

defining a lower perimeter;

defining an upper perimeter;

defining a first side perimeter;

defining a second side perimeter;

wherein said inner perimeter intersects said lower perimeter, said upper perimeter, said first side perimeter, and said second side perimeter;

wherein said outer perimeter intersects said lower perimeter, said upper perimeter, said first side perimeter, and said second side perimeter;

wherein said upper perimeter intersects said inner perimeter, said outer perimeter, said first side perimeter, and said second side perimeter;

wherein said lower perimeter intersects said inner perimeter, said outer perimeter, said first side perimeter, and said second side perimeter;

wherein said first side perimeter intersects said inner perimeter, said outer perimeter, said upper perimeter, and said lower perimeter;

wherein said second side perimeter intersects said inner perimeter, said outer perimeter, said upper perimeter, and said lower perimeter;

wherein said upper perimeter, lower perimeter, inner perimeter, outer perimeter, first side perimeter, and second side perimeter define a 3-dimensional detection field;

detecting one or more user input objects in said 3-dimensional detection field relative to a 2-dimensional surface;

determining 3-dimensional coordinates for the one or more user input objects relative to the 2-dimensional surface;

determining a touch user input based on the coordinates;

determining a touchless user input based on a second set of 3-dimensional coordinates within said 3-dimensional field associated with the one or more user input objects.

10. The one or more non-transitory computer-readable storage media of claim 9 wherein the computer process further comprises:

mapping coordinates corresponding to the position of at least one of the input objects relative to the 2-dimensional surface to a display surface.

11. The one or more non-transitory computer-readable storage media of claim 9 wherein the detecting one or more user input objects comprises:

utilizing capacitive sensors in keys of a keyboard to detect the one or more user input objects.

12. The one or more non-transitory computer-readable storage media of claim 9 wherein the computer process further comprises:

utilizing at least one camera to detect the one or more user input objects.

13. The one or more non-transitory computer-readable storage media of claim 9 wherein the computer process further comprises:

utilizing optical sensors in keys of a keyboard to detect the one or more user input objects.

14. The one or more non-transitory computer-readable storage media of claim 9 wherein determining a user input comprises determining a hover event.

15. The one or more non-transitory computer-readable storage media of claim 9 wherein determining a user input comprises determining a touch event.

16. The one or more non-transitory computer-readable storage media of claim 9 wherein the computer process further comprises:

turning off a hover mode when a touch event is detected.

17. The one or more non-transitory computer-readable storage media of claim 9 wherein the computer process further comprises:

displaying a user input on the display surface.

18. An apparatus comprising:

a 2-dimensional surface;

an object detection circuit configured to

define an inner perimeter, define an outer perimeter, define a lower perimeter, define an upper perimeter, define a first side perimeter, define a second side perimeter;

wherein said inner perimeter intersects said lower perimeter, said upper perimeter, said first side perimeter, and said second side perimeter;

wherein said outer perimeter intersects said lower perimeter, said upper perimeter, said first side perimeter, and said second side perimeter;

wherein said upper perimeter intersects said inner perimeter, said outer perimeter, said first side perimeter, and said second side perimeter;

wherein said lower perimeter intersects said inner perimeter, said outer perimeter, said first side perimeter, and said second side perimeter;

wherein said first side perimeter intersects said inner perimeter, said outer perimeter, said upper perimeter, and said lower perimeter;

wherein said second side perimeter intersects said inner perimeter, said outer perimeter, said upper perimeter, and said lower perimeter;

wherein said upper perimeter, lower perimeter, inner perimeter, outer perimeter, first side perimeter, and second side perimeter define a 3-dimensional detection field;

detect one or more user input objects in said 3-dimensional detection field relative to the 2-dimensional surface, and

configured to determine 3-dimensional coordinates of one or more input objects relative to the 2-dimensional surface; and

a user input detector configured to determine a touch user input based on the coordinates and configured to determine a touchless user input based on a second set of 3-dimensional coordinates.

19. The apparatus as claimed in claim 18 and further comprising:

a display surface; and

a mapper configured to map coordinates corresponding to the position of at least one of the input objects relative to the 2-dimensional surface to the display surface.

20. The apparatus as claimed in claim 18 wherein the 2-dimensional surface comprises a keyboard comprising keys having capacitive sensors.

21. The apparatus as claimed in claim 18 wherein the object detection circuit comprises one or more cameras.

22. The apparatus as claimed in claim 18 wherein the object detection circuit comprises a keyboard comprising keys having optical sensors.

23. The apparatus as claimed in claim 18 wherein the user input detector is configured to determine a hover event.

24. The apparatus as claimed in claim 18 wherein the user input detector is configured to determine a touch event.

25. The apparatus as claimed in claim 18 and further comprising a display controller configured to display user input on the display based on the coordinates.

26. The apparatus as claimed in claim 25 wherein the display controller is further configured to vary a display element characteristic based on a change in a proximity of at least one input object to the 2-dimensional surface.

27. The apparatus as claimed in claim 26 wherein the display controller is configured to vary a diameter of a circle used for the display element.