Method and system for determining user input based on gesture
First Claim
1. A method for determining a user input, comprising:
- capturing, at one or more image capturing sensors, an image of a field of view of a user, the image comprising a gesture created by the user;
determining a sequence for a plurality of gesture analysis processes based in part or in whole upon computational resource utilization of the plurality of gesture analysis processes;
analyzing, at least by a microprocessor, the image to determine a set of candidates and to identify a set of points associated with the gesture;
removing at least one candidate from gesture recognition with at least a first gesture analysis process of the plurality of gesture analysis processes to reduce the set of candidates to a remaining set of one or more remaining candidates while skipping one or more remaining gesture analysis processes of the plurality of gesture analysis processes for the at least one candidate;
generating respective scoring values for the one or more remaining candidates based in part or in whole on matching results of the one or more remaining candidates with predetermined gestures in a database; and
determining a user input based at least in part on a recognized gesture that is recognized by at least a second gesture analysis process.
3 Assignments
0 Petitions
Accused Products
Abstract
A waveguide apparatus includes a planar waveguide and at least one optical diffraction element (DOE) that provides a plurality of optical paths between an exterior and interior of the planar waveguide. A phase profile of the DOE may combine a linear diffraction grating with a circular lens, to shape a wave front and produce beams with desired focus. Waveguide apparati may be assembled to create multiple focal planes. The DOE may have a low diffraction efficiency, and planar waveguides may be transparent when viewed normally, allowing passage of light from an ambient environment (e.g., real world) useful in AR systems. Light may be returned for temporally sequentially passes through the planar waveguide. The DOE(s) may be fixed or may have dynamically adjustable characteristics. An optical coupler system may couple images to the waveguide apparatus from a projector, for instance a biaxially scanning cantilevered optical fiber tip.
285 Citations
20 Claims
-
1. A method for determining a user input, comprising:
-
capturing, at one or more image capturing sensors, an image of a field of view of a user, the image comprising a gesture created by the user; determining a sequence for a plurality of gesture analysis processes based in part or in whole upon computational resource utilization of the plurality of gesture analysis processes; analyzing, at least by a microprocessor, the image to determine a set of candidates and to identify a set of points associated with the gesture; removing at least one candidate from gesture recognition with at least a first gesture analysis process of the plurality of gesture analysis processes to reduce the set of candidates to a remaining set of one or more remaining candidates while skipping one or more remaining gesture analysis processes of the plurality of gesture analysis processes for the at least one candidate; generating respective scoring values for the one or more remaining candidates based in part or in whole on matching results of the one or more remaining candidates with predetermined gestures in a database; and determining a user input based at least in part on a recognized gesture that is recognized by at least a second gesture analysis process. - View Dependent Claims (2, 3, 4)
-
-
5. A system for determining a user input, comprising:
-
one or more image capturing sensors configured to capture an image of a field of view of a user, the image comprising a gesture created by the user; the at least one microprocessor further configured to determine a sequence for a plurality of gesture analysis processes based in part or in whole upon computational resource utilization of the plurality of gesture analysis processes; at least one microprocessor further configured to analyze the image with at least one of the plurality of gesture analysis processes according to the sequence to determine a set of candidates and to identify a set of points associated with the gesture; the at least one microprocessor further configured to remove at least one candidate from gesture recognition to reduce the set of candidates to a remaining set of one or more remaining candidates while skipping one or more remaining gesture analysis processes of the plurality of gesture analysis processes for the at least one candidate; the at least one microprocessor further configured to generate respective scoring values for the one or more remaining candidates based in part or in whole on matching results of the one or more remaining candidates with predetermined gestures in a database; and the at least one microprocessor further configured to determine a user input based at least in part on a recognized gesture of the one or more remaining gestures.
-
-
6. A computer program product comprising a non-transitory computer-usable storage medium storing thereupon executable code which, when executed by at least one microprocessor, causes the at least one microprocessor to perform a set of acts for determining a user input, the set of acts comprising:
-
capturing, at one or more image capturing sensors, an image of a field of view of a user, the image comprising a gesture created by the user; determining a sequence for a plurality of gesture analysis processes based in part or in whole upon computational resource utilization of the plurality of gesture analysis processes; analyzing, at least by a microprocessor, the image with at least one of the plurality of gesture analysis processes according to the sequence to determine a set of candidates and to identify a set of points associated with the gesture; removing at least one candidate from gesture recognition to reduce the set of candidates to a remaining set of one or more remaining candidates while skipping one or more remaining gesture analysis processes of the plurality of gesture analysis processes for the at least one candidate; generating respective scoring values for the one or more remaining candidates based in part or in whole on matching results of the one or more remaining candidates with predetermined gestures in a database; and determining a user input based at least in part on a recognized gesture of the one or more remaining gestures.
-
-
7. A method of identifying a gesture, comprising:
-
capturing, at one or more image capturing sensors, a plurality of images of respective fields of view of a user; determining a predetermined processing order for a plurality of gesture analysis processes based in part or in whole upon computational resource utilization of the plurality of gesture analysis processes; analyzing, with at least one microprocessor, the plurality of images with at least one of the plurality of gesture analysis processes according to the sequence at least by performing a rejection cascade processing on a set of candidates to remove at least one candidate from a set of candidates for the plurality of images to generate a reduced set of one or more remaining candidates while skipping one or more gesture analysis processes based in part or in whole upon the predetermined processing order, the rejection cascade processing comprising; a relatively less computational intensive stage using relatively less expensive computations and configured to remove one or more candidates to transform the set of candidates into a reduced set of candidates; and a later, more computational intensive stage using relatively more expensive computations and configured to analyze the reduced set of candidates to determine one or more gestures from the plurality of images; and identifying at least one gesture by performing at least a second gesture analysis process of the plurality of gesture analysis processes on the plurality of images.
-
-
8. A method of identifying a gesture, comprising:
-
capturing, at one or more image capturing sensors, a plurality of images of respective fields of view of a user; generating a plurality of gesture candidates from the plurality of images at least by performing a depth segmentation analysis based in part or in whole upon depth data provided by the one or more one or more image capturing sensors; determining a sequence for a plurality of gesture analysis processes based in part or in whole upon computational resource utilization of the plurality of gesture analysis processes; generating analysis data values corresponding to each of the plurality of gesture candidates; sorting the plurality of gesture candidates based on the analysis data values; eliminating, with at least a first gesture analysis process, one or more gesture candidates with analysis data values less than a threshold to generate a reduced set of gesture candidates while skipping one or more remaining gesture analysis processes of the plurality of gesture analysis processes; and identifying at least one gesture candidate from the reduced set of gesture candidates as the gesture for interaction with at least a second gesture analysis process executing on a computing system.
-
-
9. A method for classifying a gesture, comprising:
-
capturing, at one or more image capturing sensors, an image of a field of view of a user; determining a sequence for a plurality of gesture analysis processes based in part or in whole upon computational resource utilization of the plurality of gesture analysis processes; reducing a set of gesture candidates into a reduced set of gesture candidates at least by removing one or more gesture candidates with at least a first gesture analysis process of the plurality of gesture analysis processes while skipping one or more remaining gesture analysis processes of the plurality of gesture analysis processes for the image; performing, at least by a microprocessor operatively coupled to the one or more image capturing sensors, depth segmentation on the image at least by performing a line search with a series of lines on data in the image to generate a depth map; analyzing the depth map using a classifier mechanism to identify a part of a hand corresponding to a point in the depth map; skeletonizing the depth map into a skeletonized depth map based at least in part on an identification of the part of the hand; classifying the image as a gesture in the reduced set of gesture candidates with at least a second gesture analysis process of the plurality of gesture analysis processes based in part or in whole on the skeletonized depth map. - View Dependent Claims (10, 11, 12, 13, 14, 15, 16, 17, 18, 19, 20)
-
Specification