Information processing device, information processing method, and recording medium

• US 9,367,732 B2
• Filed: 04/27/2012
• Issued: 06/14/2016
• Est. Priority Date: 04/28/2011
• Status: Expired due to Fees

• Current Assignee: NEC Solution Innovators, Ltd (NEC Corporation)
• Original Assignee: NEC Solution Innovators, Ltd (NEC Corporation)
• Inventors: Tanimura, Ryohtaroh, Kurokawa, Takafumi
• Primary Examiner(s): Koziol, Stephen R
• Assistant Examiner(s): Patel, Pinalben
• Application Number:
  US14/112,294
• Publication Number:
  US 20140247964A1
• Time in Patent Office: 1,509 Days
• US Class Current: 1/1
• CPC Class Codes: G06F 3/005 Input arrangements through a video camera
• CPC Class Codes: G06F 3/017 Gesture based interaction, e.g. based on a set of recognized hand gestures interaction based on gestures traced on a digitiser G06F3/04883
• CPC Class Codes: G06F 3/0304 Detection arrangements using opto-electronic means constructional details of pointing devices not related to the detection arrangement using opto-electronic means G06F3/033; optical digitisers G06F3/042
• CPC Class Codes: G06T 2207/10016 Video; Image sequence
• CPC Class Codes: G06T 2207/10024 Color image
• CPC Class Codes: G06T 2207/30196 Human being; Person
• CPC Class Codes: G06T 7/254 involving subtraction of images
• CPC Class Codes: G06V 10/34 Smoothing or thinning of the pattern; Morphological operations; Skeletonisation
• CPC Class Codes: G06V 10/435 Computation of moments
• CPC Class Codes: G06V 40/113 Recognition of static hand signs
• CPC Class Codes: G06V 40/28 Recognition of hand or arm movements, e.g. recognition of deaf sign language static hand signs G06V40/113

Abstract

An image inputter inputs image data in which a user has been captured. An extractor extracts a region corresponding to a hand of the user included in the image data input by the image inputter. A determinator determines whether or not the region corresponding to the hand of the user extracted by the extractor is in a stationary state. If it is determined by the determinator that the region corresponding to the hand of the user is in a stationary state, a recognizer recognizes the shape of the region corresponding to the hand of the user extracted by the extractor. Otherwise, if it is determined by the determinator that the region corresponding to the hand of the user is not in a stationary state, the recognizer recognizes the movement of the region corresponding to the hand of the user extracted by the extractor. An inputter inputs data associated with the shape or the movement recognized by the recognizer.

Claims

1. An information processing device comprising:

an image inputter configured to input image data in which a user has been captured;

an extractor configured to extract a region corresponding to a hand of the user included in the image data input by the image inputter;

a determinator configured to determine whether or not the region corresponding to the hand of the user extracted by the extractor is in a stationary state;

a recognizer configured to, if the region corresponding to the hand of the user is determined by the determinator as being in a stationary state, recognize the shape of the region corresponding to the hand of the user extracted by the extractor; and

an inputter configured to input data associated with the shape recognized by the recognizer,

wherein the extractor is further configured to set a rectangle that circumscribes a movement region obtained from difference data between frames in the image data input by the image inputter, specify a region corresponding to a wrist of the user, on the basis of a width of the hand included in the movement region obtained by scanning the inside of the rectangle, and extract the region corresponding to the hand of the user, on the basis of the specified region corresponding to the wrist of the user.

2. The information processing device according to claim 1, wherein

the recognizer is further configured to specify the position of a fingertip in the region corresponding to the hand of the user extracted by the extractor, and on the basis of the specified position of the fingertip, to recognize the shape of the region corresponding to the hand of the user.

3. The information processing device according to claim 2, wherein

the recognizer is further configured to specify the position of the fingertip by carrying out thinning processing on the region corresponding to the hand of the user extracted by the extractor.

4. The information processing device according to claim 2, wherein,

the recognizer is further configured to specify, on the basis of the width of the hand within the region corresponding to the hand of the user extracted by the extractor, the position of the fingertip by estimating the distance between the fingertip and a center of gravity calculated from the region corresponding to the hand of the user extracted by the extractor.

5. The information processing device according to claim 1, wherein

the determinator is further configured to determine whether or not the hand of the user is in a stationary state, on the basis of the amount of movement of the center of gravity of the region corresponding to the hand of the user extracted by the extractor.

6. The information processing device according to claim 1, wherein,

the recognizer is further configured to, if it is determined by the determinator that the hand of the user is not in a stationary state, movement of the region corresponding to the hand of the user extracted by the extractor, and

the inputter is further configured to input data associated with the movement recognized by the recognizer.

7. The information processing device according to claim 1, further comprising

a display configured to display an image corresponding to the image data input by the image inputter.

8. An information processing method including:

an image input step for inputting image data in which a subject has been captured;

an extraction step for extracting a region corresponding to a hand of the user included in the image data input in the image input step;

a determination step for determining whether or not the region corresponding to the hand of the user extracted in the extraction step is in a stationary state;

a recognizing step for, if the region corresponding to the hand of the user is determined in the determination step as being in a stationary state, recognizing the shape of the region corresponding to the hand of the user extracted in the extraction step; and

an input step for inputting data associated with the shape recognized in the recognizing step,

wherein the extraction step further includes setting a rectangle that circumscribes a movement region obtained from difference data between frames in the image data input in the image input step, specifying a region corresponding to a wrist of the user, on the basis a width of the hand included in the movement region obtained by scanning the inside of the rectangle, and extracting the region corresponding to the hand of the user, on the basis of the specified region corresponding to the wrist of the user.

9. A non-transitory recording medium on which a program is recorded, the program causing a computer to function as:

an image inputter configured to input image data in which a subject has been captured;

an extractor configured to extract a region corresponding to a hand of the user included in the image data input by the image inputter;

a determinator configured to determine whether or not the region corresponding to the hand of the user extracted by the extractor is in a stationary state;

a recognizer configured to, if the region corresponding to the hand of the user is determined by the determinator as being in a stationary state, recognize the shape of the region corresponding to the hand of the user extracted by the extractor; and

an inputter configured to input data associated with the shape recognized by the recognizer,

wherein the extractor is further configured to set a rectangle that circumscribes a movement region obtained from difference data between frames in the image data input by the image inputter, specify a region corresponding to a wrist of the user, on the basis of a width of the hand included in the movement region obtained by scanning the inside of the rectangle, and extract the region corresponding to the hand of the user, on the basis of the specified region corresponding to the wrist of the user.