Child-appropriate interface selection using hand recognition

• US 9,323,352 B1
• Filed: 10/23/2012
• Issued: 04/26/2016
• Est. Priority Date: 10/23/2012
• Status: Active Grant

• Current Assignee: Amazon Technologies, Inc. (Amazon.com, Inc.)
• Original Assignee: Amazon Technologies, Inc. (Amazon.com, Inc.)
• Inventors: Haskin, Menashe, Shepard, Deborah May
• Primary Examiner(s): Olson, Jason
• Assistant Examiner(s): Abebe, Sosina
• Application Number:
  US13/658,718
• Time in Patent Office: 1,281 Days
• Field of Search: 345/158, 382/103, 715/700
• US Class Current: 1/1
• CPC Class Codes: G03B 21/142 Adjusting of projection optics
• CPC Class Codes: G06F 3/005 Input arrangements through a video camera
• CPC Class Codes: G06F 3/017 Gesture based interaction, e.g. based on a set of recognized hand gestures interaction based on gestures traced on a digitiser G06F3/04883
• CPC Class Codes: G06F 3/0304 Detection arrangements using opto-electronic means constructional details of pointing devices not related to the detection arrangement using opto-electronic means G06F3/033; optical digitisers G06F3/042
• CPC Class Codes: G06F 3/0346 with detection of the device orientation or free movement in a 3D space, e.g. 3D mice, 6-DOF [six degrees of freedom] pointers using gyroscopes, accelerometers or tilt-sensors
• CPC Class Codes: G06F 3/03547 Touch pads, in which fingers can move on a surface
• CPC Class Codes: G06F 3/048 Interaction techniques based on graphical user interfaces [GUI]
• CPC Class Codes: G06F 3/0481 based on specific properties of the displayed interaction object or a metaphor-based environment, e.g. interaction with desktop elements like windows or icons, or assisted by a cursor's changing behaviour or appearance
• CPC Class Codes: G06F 3/0482 Interaction with lists of selectable items, e.g. menus
• CPC Class Codes: G06F 3/0484 for the control of specific functions or operations, e.g. selecting or manipulating an object, an image or a displayed text element, setting a parameter value or selecting a range
• CPC Class Codes: G06F 3/0488 using a touch-screen or digitiser, e.g. input of commands through traced gestures
• CPC Class Codes: G06V 10/145 Illumination specially adapted for pattern recognition, e.g. using gratings
• CPC Class Codes: G06V 40/11 Hand-related biometrics; Hand pose recognition

Abstract

Hand recognition is used to distinguish among users of gesture-based computer systems, and particularly between adults and children. Hands may be classified as adults or children based on certain physical characteristics, such as hand size, shape, or curvature. Depending upon whether the hand belongs to an adult or a child, user-appropriate interfaces are provided. For instance, in response to detecting a presence of a child, the interface may include more child-appropriate features such as simpler graphical elements, use of basic audio interfaces, and so forth.

Claims

1. A system comprising:

one or more processors;

one or more projectors to project a visual portion of an interface into an environment;

one or more cameras to capture an image of the environment, the image showing at least a hand of a person;

one or more computer-readable media storing computer-executable instructions that, when executed by the one or more processors, cause the one or more processors to perform acts comprising;

detecting the hand in the image;

determining a physical characteristic of the hand in the image by translating points along a periphery of the hand in the image into a set of values on a curvature map and by determining a distance between a first point and a second point of the points on a distance map that is different from the curvature map;

comparing the physical characteristic of the hand with one or more model characteristics associated with a hand model representative of a child'"'"'s hand; and

based on the physical characteristic of the hand matching, at least in part, the one or more model characteristics associated with the hand model representative of the child'"'"'s hand, selecting child-appropriate features for the interface for interacting with the child.

2. The system of claim 1, wherein the physical characteristic comprises one of a size of the hand, a curvature of the hand or a shape of the hand.

3. The system of claim 1, wherein the physical characteristic is determined by deriving an approximate size of a palm portion of the hand in the image.

4. The system of claim 1, wherein the physical characteristic is determined by control points marked along features of the hand in the image.

5. The system of claim 1, wherein the one or more computer-readable media store additional computer-executable instructions that, when executed by the one or more processors, cause the one or more processors to perform acts comprising:

evaluating secondary indicia captured from the environment that differentiates between an adult and child, wherein the secondary indicia is used to confirm presence of a child in the environment.

6. The system of claim 1, wherein the one or more computer-readable media store additional computer-executable instructions that, when executed by the one or more processors, cause the one or more processors to perform acts comprising selecting an adult interface for interacting with an adult based at least partly on a determination that the physical characteristic of the hand fails to acceptably match the hand model representative of the child.

7. The system of claim 1, wherein the one or more projectors project a pattern into the environment and onto the hand, wherein the one or more cameras capture the image based at least in part on analyzing a reflection of the pattern.

8. A system comprising:

a camera to capture an image of a hand within an environment; and

a computing device configured to;

determine a physical characteristic of the hand from the image by translating points along a periphery of the hand in the image into a set of values on a curvature map and by determining a distance between a first point and a second point of the points on a distance map that is different from the curvature map;

determine whether the hand is associated with an adult or a child; and

select a first interface for use in the environment to facilitate human interaction with the computing device based at least partly on a first determination that the hand is associated with a child and a second interface for use in the environment to facilitate human interaction with the computing device based at least partly on a second determination that the hand is associated with an adult.

9. The system of claim 8, wherein the computing device is configured to determine, from the image, at least one of a size of the hand, a shape of the hand, or a curvature of the hand.

10. The system of claim 8, wherein the computing device is configured to compare the physical characteristic of the hand to one or more model characteristics associated with a hand model, the hand model being representative of a child'"'"'s hand.

11. The system of claim 8, wherein the camera captures other physical features of a person in the image, and the computing device is further configured to evaluate the other physical features of the person to confirm a determination that the hand is associated with one of the adult or the child.

12. The system of claim 8, further comprising at least one projector to project a visual portion of the interface onto a surface in the environment.

13. The system of claim 8, further comprising at least one microphone to receive audio input from a person associated with the hand, wherein the computing system is further configured to analyze the audio input from the person to confirm a determination that the hand is associated with one of the adult or the child.

14. A system comprising:

a projector to project visual portions of a human-to-computer interface onto a surface in an environment;

a camera to capture an image of the environment, the image including at least a portion of a human; and

a computing device configured to;

determine whether the portion of the human is associated with an adult or a child based, at least in part, on a size of the portion of the human;

determine that the size of the portion of the human is insufficient to indicate that the human is an adult or a child;

determine, from the image and based at least partly on a determination that the size of the portion of the human is insufficient to indicate that the human is an adult or a child, secondary indicia associated with the human; and

select a first interface for use in the environment to facilitate human interaction with the computing device based at least partly on a first determination that the portion of the human is associated with a child and a second interface for use in the environment to facilitate human interaction with the computing device based at least partly on a second determination that the portion of the human is associated with an adult.

15. The system of claim 14, wherein the portion of the human comprises at least one of a head, an arm, a hand, a torso, or a leg.

16. One or more non-transitory computer-readable media storing computer-executable instructions that, when executed by one or more processors, cause the one or more processors to perform acts comprising:

detecting a hand of a user in an image captured from an environment;

determining a physical characteristic associated with the hand by translating points along a periphery of the hand in the image into a set of values on a curvature map and by determining a distance between a first point and a second point of the points on a distance map that is different from the curvature map;

ascertaining whether the hand is associated with a child or an adult based, at least in part, on the physical characteristic associated with the hand; and

selecting among multiple features to use in an interface for interacting with the user depending, at least in part, on whether the hand is associated with a child or an adult.

17. The one or more non-transitory computer-readable media of claim 16, wherein the physical characteristic comprises at least one of a size of the hand, a shape of the hand, or a curvature of the hand.

18. The one or more non-transitory computer-readable media of claim 16, further comprising instructions that, when executed by one or more processors, cause the one or more processors to perform acts comprising evaluating secondary indicia collected from the environment to confirm whether the hand is associated with a child or an adult.

19. The one or more non-transitory computer-readable media of claim 16, further comprising instructions that, when executed by one or more processors, cause the one or more processors to perform acts comprising projecting a first graphical elements as part of the interface based at least partly on a first determination that the hand is associated with a child and a second graphical elements as part of the interface based at least partly on a second determination that the hand is associated with an adult.

20. A method comprising:

determining a physical characteristic associated with a hand as observed in an environment by a computing system, the physical characteristic being determined by translating points along a periphery of the hand into a set of values on a curvature map and by determining a distance between a first point and a second point of the points on a distance map that is different from the curvature map;

ascertaining, by the computing system, whether the hand is associated with a child or an adult based, at least in part, on the physical characteristic; and

based at least partly on a determination that the hand is associated with the child, selecting one or more child-appropriate features for an interface to facilitate interaction between the child and the computing system.

21. The method of claim 20, wherein the one or more child-appropriate features comprises a combination of graphical elements and audio dialog.

22. The method of claim 20, wherein the physical characteristic comprises at least one of a size of the hand, a shape of the hand, or a curvature of the hand.

23. The method of claim 20, wherein the determination is a first determination, and further comprising selecting one or more adult-suitable features for the interface based at least partly on a second determination that the hand is associated with the adult.