Hand and hand pose detection
First Claim
1. A system comprising:
- one or more processors;
an imaging sensor; and
one or more computer-readable media storing computer-executable instructions that, when executed by the one or more processors, cause the one or more processors to perform acts comprising;
obtaining a point set using the imaging sensor;
identifying a first local extremum point from the point set along a first portion of an outer boundary of the point set, the first portion associated with a first curvature value that satisfies one or more curvature criteria;
identifying a second local extremum point from the point set along a second portion of the outer boundary of the point set, the second portion associated with a second curvature value that satisfies the one or more curvature criteria;
identifying a potential hand segment within the point set;
identifying a first line segment from the point set, the first line segment extending from the first local extremum point and terminating at a first endpoint, the first endpoint different than the second local extremum point;
identifying a second line segment from the point set, the second line segment extending from the second local extremum point and terminating at a second endpoint, the first endpoint different than the second endpoint, and the second endpoint different than the first local extremum point;
identifying a centroid of a shape within the point set, the centroid based at least in part on the first endpoint and the second endpoint;
applying a shape model to the potential hand segment based at least in part on a location of the centroid;
determining that the potential hand segment is a potential palm segment based at least in part on the shape model and the location of the centroid;
designating the point set as corresponding to a hand of a user; and
comparing the potential hand segment to one or more hand pose models to identify a pose of the hand of the user.
2 Assignments
0 Petitions
Accused Products
Abstract
Techniques for detecting a hand of a user and a pose of the hand are described herein. In some instances, as a user interacts with a computing system through various hand signals, the hand of the user and pose of the hand (e.g., position and orientation) may be detected. To detect a hand, a point set may be analyzed to identify local extremum points (e.g., peaks and valleys) along an outer boundary of the point set. The local extremum points may then be used to identify a potential hand segment including one or more potential finger segments and/or a potential palm segment. When the potential hand segment satisfies one or more criteria, a hand may be detected. To detect a hand pose, an observed pose of the hand segment may be compared to one or more hand pose models.
32 Citations
25 Claims
-
1. A system comprising:
-
one or more processors; an imaging sensor; and one or more computer-readable media storing computer-executable instructions that, when executed by the one or more processors, cause the one or more processors to perform acts comprising; obtaining a point set using the imaging sensor; identifying a first local extremum point from the point set along a first portion of an outer boundary of the point set, the first portion associated with a first curvature value that satisfies one or more curvature criteria; identifying a second local extremum point from the point set along a second portion of the outer boundary of the point set, the second portion associated with a second curvature value that satisfies the one or more curvature criteria; identifying a potential hand segment within the point set; identifying a first line segment from the point set, the first line segment extending from the first local extremum point and terminating at a first endpoint, the first endpoint different than the second local extremum point; identifying a second line segment from the point set, the second line segment extending from the second local extremum point and terminating at a second endpoint, the first endpoint different than the second endpoint, and the second endpoint different than the first local extremum point; identifying a centroid of a shape within the point set, the centroid based at least in part on the first endpoint and the second endpoint; applying a shape model to the potential hand segment based at least in part on a location of the centroid; determining that the potential hand segment is a potential palm segment based at least in part on the shape model and the location of the centroid; designating the point set as corresponding to a hand of a user; and comparing the potential hand segment to one or more hand pose models to identify a pose of the hand of the user. - View Dependent Claims (2, 3, 4)
-
-
5. A system comprising:
-
one or more processors; and one or more computer-readable media storing computer-executable instructions that, when executed, cause the one or more processors to perform acts comprising; obtaining a point set; identifying a local extremum point along an outer boundary of the point set that has a curvature value that satisfies one or more curvature criteria; identifying a potential finger segment of one or more potential finger segments comprising points from the point set that are based at least in part on the local extremum point, the local extremum point representing a distal portion of the potential finger segment; determining that the potential finger segment has a length that satisfies a predetermined finger length; determining that the potential finger segment has a width that satisfies a predetermined finger width; determining that the potential finger segment has a centerline; identifying a line segment from the point set based at least in part on the centerline, the line segment extending from the local extremum point of the potential finger segment and terminating at an endpoint; identifying a centroid of a shape within the point set, the centroid based at least in part on the endpoint of the line segment, wherein the centroid is different than the endpoint; identifying a potential palm segment based at least in part on a location of the centroid; and determining that the point set represents, at least in part, a hand of a user. - View Dependent Claims (6, 7, 8, 9, 10, 11, 12, 13, 14)
-
-
15. One or more non-transitory computer-readable storage media storing computer-readable instructions that, when executed, instruct one or more processors to perform operations comprising:
-
obtaining a point set; identifying a local extremum point along an outer boundary of the point set that has a curvature value that satisfies one or more curvature characteristics; identifying a potential finger segment comprising points from the point set that are based at least in part on the local extremum point; identifying a centerline of the potential finger segment; identifying a line segment from the point set based at least in part on the centerline, the line segment extending from the local extremum point of the potential finger segment and terminating at an endpoint; identifying a centroid of a shape within the point set, the centroid based at least in part on the endpoint of the line segment, the centroid different than the endpoint; identifying a potential palm segment comprising points from the point set that are at least partly bordered by the potential finger segment, the potential palm segment based at least in part on a location of the centroid; determining that the potential palm segment satisfies one or more palm criteria; and determining that the point set represents, at least in part, a hand of a user. - View Dependent Claims (16, 17, 18, 19, 20, 21)
-
-
22. A method comprising:
under control of one or more computing systems configured with executable instructions, obtaining a point set; determining that the point set describes, at least in part, a hand of a user, by; determining that at least some of the points of the point set correspond to a finger segment based at least in part on a curvature value of a local extremum point from the point set, the local extremum point representing a distal portion of the finger segment; determining a centerline of the finger segment; determining a line segment from the point set based at least in part on the centerline, the line segment extending from the local extremum point of the finger segment and terminating at an endpoint; determining a centroid of a shape within the point set, the centroid based at least in part on the endpoint of the line segment, wherein the centroid of the shape is different than the endpoint; determining that at least some of the points of the point set correspond to a palm segment based at least in part on the centroid; and determining a wrist line associated with the point set; identifying an observed pose of the hand by; identifying a pose for the finger segment, the pose of the finger segment comprising a pose from a group of poses including an extended pose, a half-folded pose, and a folded pose; and identifying a pose of the palm segment; comparing the observed pose of the hand to one or more hand pose models; and determining a pose of the hand based at least in part on the comparing. - View Dependent Claims (23, 24, 25)
Specification