Virtual keyboard for a non-tactile three dimensional user interface
First Claim
Patent Images
1. A method, comprising:
- presenting, by a computer system executing a non-tactile three dimensional user interface, a virtual keyboard on a display, the virtual keyboard comprising multiple virtual keys;
capturing a sequence of depth maps, via a 3D capturing device, over time of a hand of a human subject while the human subject moves the hand in a plane;
presenting, on the display, a cursor at positions indicated by the hand in the captured sequence of depth maps such that the cursor moves over the virtual keys on the display in response to movement of the hand in the plane; and
selecting one of the multiple virtual keys if a change in direction of trajectory is determined by using the captured sequence of depth maps and calculating points along the presented cursor'"'"'s trajectory path segment over the virtual keys, wherein the trajectory of the user'"'"'s hand tracks a plurality of keys for which a cursor crosses spatially from a previously determined key point to a newly desired key point, and wherein the keys between the previously determined key point to the newly desired key point are used in combination with previously determined key inputs to configure a language model for estimating probable words and estimating a most likely key from the keys based on the language model and displaying the most likely key appended with the previously determined key inputs; and
wherein if the change of trajectory is not detected then determining whether the cursor is in proximity to a plurality of keys for a standard time period and selecting keys to be used by the language model for estimating a most likely key from the plurality of keys and displaying the most likely key appended with the previously determined key inputs.
3 Assignments
0 Petitions
Accused Products
Abstract
A method, including presenting, by a computer system executing a non-tactile three dimensional user interface, a virtual keyboard on a display, the virtual keyboard including multiple virtual keys, and capturing a sequence of depth maps over time of a body part of a human subject. On the display, a cursor is presented at positions indicated by the body part in the captured sequence of depth maps, and one of the multiple virtual keys is selected in response to an interruption of a motion of the presented cursor in proximity to the one of the multiple virtual keys.
-
Citations
27 Claims
-
1. A method, comprising:
-
presenting, by a computer system executing a non-tactile three dimensional user interface, a virtual keyboard on a display, the virtual keyboard comprising multiple virtual keys; capturing a sequence of depth maps, via a 3D capturing device, over time of a hand of a human subject while the human subject moves the hand in a plane; presenting, on the display, a cursor at positions indicated by the hand in the captured sequence of depth maps such that the cursor moves over the virtual keys on the display in response to movement of the hand in the plane; and selecting one of the multiple virtual keys if a change in direction of trajectory is determined by using the captured sequence of depth maps and calculating points along the presented cursor'"'"'s trajectory path segment over the virtual keys, wherein the trajectory of the user'"'"'s hand tracks a plurality of keys for which a cursor crosses spatially from a previously determined key point to a newly desired key point, and wherein the keys between the previously determined key point to the newly desired key point are used in combination with previously determined key inputs to configure a language model for estimating probable words and estimating a most likely key from the keys based on the language model and displaying the most likely key appended with the previously determined key inputs; and
wherein if the change of trajectory is not detected then determining whether the cursor is in proximity to a plurality of keys for a standard time period and selecting keys to be used by the language model for estimating a most likely key from the plurality of keys and displaying the most likely key appended with the previously determined key inputs. - View Dependent Claims (2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13)
-
-
14. An apparatus, comprising:
-
a display; and a computer executing a non-tactile three dimensional user interface and configured to present a virtual keyboard on a display, the virtual keyboard comprising multiple virtual keys, to capture a sequence of depth maps, via a 3D capturing device, over time of a hand of a human subject while the human subject moves the hand in a plane, to present, on the display, a cursor at positions indicated by the hand in the captured sequence of depth maps such that the cursor moves over the virtual keys on the display in response to movement of the hand in the plane, and to select one of the multiple virtual keys if a change in direction of trajectory is determined by using the captured sequence of depth maps and calculating points along the presented cursor'"'"'s trajectory path segment over the virtual keys, wherein the trajectory of the user'"'"'s hand tracks a plurality of keys for which a cursor crosses spatially from a previously determined key point to a newly desired key point, and wherein the keys between the previously determined key point to the newly desired key point are used in combination with previously determined key inputs to configure a language model for estimating probable words and estimating a most likely key from the keys based on the language model and displaying the most likely key appended with the previously determined key inputs; and
wherein if the change of trajectory is not detected then determining whether the cursor is in proximity to a plurality of keys for a standard time period and selecting keys to be used by the language model for estimating a most likely key from the plurality of keys and displaying the most likely key appended with the previously determined key inputs. - View Dependent Claims (15, 16, 17, 18, 19, 20, 21, 22, 23, 24, 25, 26)
-
-
27. A computer software product comprising a non-transitory computer-readable medium, in which program instructions are stored, which instructions, when read by a computer executing a non-tactile three dimensional user interface, cause the computer:
to present a virtual keyboard on a display, the virtual keyboard comprising multiple virtual keys, to capture a sequence of depth maps, via a 3D capturing device, over time of a hand of a human subject while the human subject moves the hand in a plane, to present on the display, a cursor at positions indicated by the hand in the captured sequence of depth maps such that the cursor moves over the virtual keys on the display in response to movement of the hand in the plane, and to select one of the multiple virtual keys if a change in direction of trajectory is determined by using the captured sequence of depth maps and calculating points along the presented cursor'"'"'s trajectory path segment over the virtual keys, wherein the trajectory of the user'"'"'s hand tracks a plurality of keys for which a cursor crosses spatially from a previously determined key point to a newly desired key point, and wherein the keys between the previously determined key point to the newly desired key point are used in combination with previously determined key inputs to configure a language model for estimating probable words and estimating a most likely key from the keys based on the language model and displaying the most likely key appended with the previously determined key inputs; and
wherein if the change of trajectory is not detected then determining whether the cursor is in proximity to a plurality of keys for a standard time period and selecting keys to be used by the language model for estimating a most likely key from the plurality of keys and displaying the most likely key appended with the previously determined key inputs.
Specification