Adaptive ergonomic keyboard

• US 9,548,012 B1
• Filed: 08/29/2012
• Issued: 01/17/2017
• Est. Priority Date: 08/29/2012
• Status: Expired due to Fees

• Current Assignee: Amazon Technologies, Inc. (Amazon.com, Inc.)
• Original Assignee: Amazon Technologies, Inc. (Amazon.com, Inc.)
• Inventors: Sjoberg, Robert W., Coley, Christopher D.
• Primary Examiner(s): To, Jennifer
• Assistant Examiner(s): SPRATT, BEAU D
• Application Number:
  US13/597,932
• Time in Patent Office: 1,602 Days
• Field of Search: 715/773
• US Class Current: 1/1
• CPC Class Codes: G06F 3/017 Gesture based interaction, e.g. based on a set of recognized hand gestures interaction based on gestures traced on a digitiser G06F3/04883
• CPC Class Codes: G06F 3/0425 using a single imaging device like a video camera for tracking the absolute position of a single or a plurality of objects with respect to an imaged reference surface, e.g. video camera imaging a display or a projection screen, a table or a wall surface, on which a computer generated image is displayed or projected tracking a projected light spot to determine a position on a display surface G06F3/0386
• CPC Class Codes: G06F 3/04886 by partitioning the display area of the touch-screen or the surface of the digitising tablet into independently controllable areas, e.g. virtual keyboards or menus
• CPC Class Codes: G09G 3/002 to project the image of a two-dimensional display, such as an array of light emitting or modulating elements or a CRT

Abstract

Techniques are described for adapting a keyboard arrangement to be personalized for a user, based on detected gestures made by the user while employing the keyboard arrangement. An adaptable keyboard arrangement may be provided for the user as part of a device, projected onto a surface near the user, or rendered abstractly into a space near the user. Sensor and/or contextual data may be employed to identify mistypes or near-mistypes of the user, and a statistical model may be developed of the user'"'"'s typing habits. Keyboard modifications may be determined that adapt the keyboard to minimize potential typing errors and/or provide a more ergonomic keyboard arrangement for the user.

Claims

1. A computer-implemented method comprising:

detecting one or more gestures made by a user while using a first keyboard arrangement during a first session within a first environment, the first keyboard arrangement being a virtual keyboard arrangement that includes one or more elements that are customizable;

analyzing the one or more gestures to develop a statistical model based at least in part on a location of one or more gestures made by the user in association with a selection of one or more elements of the first keyboard arrangement;

based at least in part on the statistical model, determining a second keyboard arrangement that is customized with at least one or more altered elements for the user based at least in part on the statistical model, the second keyboard arrangement to minimize one or more potential errors made by the user in the selection of the one or more elements;

identifying the user in a second environment that is different from the first environment; and

presenting the second keyboard arrangement during a second session subsequent to the first session within the second environment.

2. The method of claim 1 wherein the one or more elements are customizable in at least one of a size, a caption, a response profile, a location, or an orientation and wherein the second keyboard arrangement includes at least one alteration includes an alteration of at least one of the size, the caption, the response profile, the location, the caption, or the orientation of at least one of the one or more elements of the first keyboard arrangement.

3. The method of claim 1 wherein the analyzing of the one or more gestures is performed dynamically during the first session, and wherein the second keyboard arrangement is provided for the user during the first session within the first environment prior to presenting the second keyboard arrangement during the second session within the second environment.

4. The method of claim 1 wherein developing the statistical model further includes detecting a miss of at least one of the one or more elements by the user.

5. The method of claim 4 wherein detecting the miss is based at least in part on a subsequent gesture of the user indicating the miss.

6. The method of claim 4 wherein detecting the miss is based at least in part on a contextual analysis indicating at least one misspelling.

25. The method of claim 1 wherein the second keyboard arrangement is presented as an abstract arrangement within the second environment in a space proximal to the user located in the second environment.

26. The method of claim 1 wherein the identifying the user comprises identifying the user using at least facial recognition.

7. A computer-implemented method comprising:

presenting a keyboard arrangement in a first environment;

detecting one or more gestures made by a user while using the keyboard arrangement in the first environment, the one or more gestures includes at least a miss-event of at least one of the one or more elements of the keyboard arrangement;

analyzing the one or more gestures to develop a usage model based at least in part on the one or more gestures made by the user in association with selection of one or more elements of the keyboard arrangement;

based at least in part on the usage model, determining a modified keyboard arrangement that is different than the keyboard arrangement, the modified keyboard arrangement includes a change of one or more of a size, a label, a location, or an orientation of at least one of the one or more elements of the keyboard arrangement;

identifying the user in a second environment different than the first environment; and

presenting the modified keyboard arrangement in the second environment.

8. The method of claim 7 wherein the usage model is a statistical usage model.

9. The method of claim 7 wherein the modified keyboard arrangement is further projected by a projector onto a surface in the second environment.

10. The method of claim 7 wherein the modified keyboard arrangement further comprises altering a response profile for the keyboard arrangement without changing an appearance of the one or more elements of the keyboard arrangement.

11. The method of claim 7 wherein detecting the miss-event is based at least in part on at least one of:

a subsequent gesture of the user that is indicative of the miss-event;

or

detecting an error in an entry made by the user based on a contextual analysis of the entry.

12. The method of claim 7 further comprising monitoring a position of at least one body part of the user, and wherein the modified keyboard arrangement includes changing a location of one or more portions of the keyboard arrangement to provide at least one ergonomic benefit for the user, further based at least in part on the monitored position.

13. A system comprising:

a memory;

at least one processor in communication with the memory;

a projector to project a first keyboard arrangement in a first environment;

a sensor data receiving component stored in the memory and executed by the at least one processor to receive data associated with one or more gestures made by a user while using the first keyboard arrangement in the first environment;

a keyboard modification component stored in the memory and executed by the at least one processor to;

analyze the one or more gestures to develop a statistical model based at least in part on a location of gestures made by the user in association with selection of one or more elements of the first keyboard arrangement during a first session; and

create a second keyboard arrangement different from the first keyboard arrangement, the second keyboard arrangement is customized with at least one or more altered elements based at least in part on the statistical model to minimize one or more potential errors in the selection of the one or more elements;

a keyboard storage component stored in the memory and executed by the at least one processor to store the second keyboard arrangement for the user;

a user identification component stored in the memory and executed by the at least one processor to identify the user in a second environment during a second session subsequent to the first session; and

a keyboard rendering component stored in the memory and executed by the at least one processor to provide the second keyboard arrangement in response to the identification of the user.

14. The system of claim 13 wherein second keyboard arrangement is further created based at least in part on an application in use by the user.

15. The system of claim 13 wherein the second keyboard arrangement is further created based at least in part on a current application context of an application in use by the user.

16. The system of claim 13 wherein the user identification component identifies the user by at least facial recognition.

17. The system of claim 13 wherein the analyzing of the one or more gestures is performed dynamically during a session, and wherein the second keyboard arrangement is provided for use by the user during the session in the first environment.

27. The system of claim 13 wherein the first environment is a first room in a house and the second environment is a second room in the house.

18. One or more computer-readable storage media storing instructions that, when executed, instruct at least one processor to perform actions comprising:

presenting a first image of a first keyboard arrangement at a first time in a first environment;

detecting one or more gestures made by a user interacting with the first image of the first keyboard arrangement in the first environment;

analyzing the one or more gestures to develop a statistical model based at least in part on a location of the one or more gestures made by the user in association with a selection of one or more elements of the first keyboard arrangement;

creating a second keyboard arrangement different than the first keyboard arrangement, the second keyboard arrangement is customized with at least one or more altered elements based at least in part on the statistical model to minimize one or more potential errors in the selection of the one or more elements;

storing the second keyboard arrangement in association with the user;

identifying the user in a second environment at a second time subsequent the first time; and

presenting a second image of the second keyboard arrangement in the second environment.

19. The one or more computer-readable storage media of claim 18 wherein the actions further comprise:

identifying the user in the second environment by at least facial recognition; and

presenting the second keyboard arrangement for the user based at least in part on the identification.

20. The one or more computer-readable storage media of claim 18 wherein identifying the user is based at least in part on recognizing at least one of a hand position or a hand gesture of the user.

21. The one or more computer-readable storage media of claim 18 wherein the actions further comprise presenting the second keyboard arrangement onto a surface in the second environment.

22. The one or more computer-readable storage media of claim 18 wherein providing the first image of the first keyboard arrangement or the second image of the second keyboard arrangement as an abstract representation away from a physical object in a space that is proximal to the user.

23. The one or more computer-readable storage media of claim 18 wherein the second keyboard arrangement is created by altering at least one of the one or more elements of the first keyboard arrangement, the altering includes altering one or more of a size, a caption, a location, or an orientation of the at least one of the one or more elements of the first keyboard arrangement.

24. The one or more computer-readable storage media of claim 18 wherein detecting the one or more gestures includes tracking one or more motions of one or more fingers of the user to perform the one or more gestures.