Data input device, data input method, and non-transitory computer readable recording medium storing data input program

• US 10,186,057 B2
• Filed: 11/14/2013
• Issued: 01/22/2019
• Est. Priority Date: 11/22/2012
• Status: Active Grant

• Current Assignee: Sharp Kabushiki Kaisha (Hon Hai Precision Industry Co., Ltd.)
• Original Assignee: Sharp Kabushiki Kaisha (Hon Hai Precision Industry Co., Ltd.)
• Inventors: Shimura, Tomoya, Saito, Sakae
• Primary Examiner(s): Xiao, Ke
• Assistant Examiner(s): Shin, Andrew
• Application Number:
  US14/646,652
• Publication Number:
  US 20150302617A1
• Time in Patent Office: 1,895 Days
• Field of Search: 345619, 345642, 345156, 345157, 382103, 382190, 382195, 382202, 382203, 382286, 382291
• CPC Class Codes: G06F 3/011 Arrangements for interaction with the human body, e.g. for user immersion in virtual reality blind teaching G09B21/00
• CPC Class Codes: G06T 11/20 Drawing from basic elements, e.g. lines or circles
• CPC Class Codes: G06T 11/203 Drawing of straight lines or curves
• CPC Class Codes: G06V 40/28 Recognition of hand or arm movements, e.g. recognition of deaf sign language static hand signs G06V40/113

Abstract

In a data input device according to one aspect of the present invention, a position detection unit detects a first position based on a position of one part of a body of a user represented in a video captured by an image capturing device and a second position based on a position of another part of the body. A drawing input detection unit establishes, based on the first position, an input detection region in which a drawing input is detected, and detects, based on whether or not the second position is included in the input detection region, the drawing input. A position-of-interest detection unit detects a position-of-interest corresponding to the second position on an image display plane. An image forming unit forms an image indicating a position-of-interest in a case that the drawing input detection unit detects the drawing input.

Claims

1. A data input device comprising:

one or more processing devices and one or more memory devices storing instructions to;

detect a first position, a second position, and a third position, the first position being based on a position of one part of a body of a user represented in a video captured by an image capturing device, the second position being based on a position of another part of the body, the third position being based on a position of yet another part of the body;

establish a three-dimensional operation detection space based on the first detected position;

establish one boundary of a three-dimensional input detection space in which a drawing input is detected, the establishment of the one boundary being performed by using, as a reference, the third detected position, the three-dimensional input detection space being included in the three-dimensional operation detection space;

detect the drawing input, the detection of the drawing input being performed in a case that the second position is included in the three-dimensional input detection space, and whether or not the second position is farther rearward than the third position;

detect a position-of-interest on an image display plane, the detection of the position-of-interest being performed based on the first and second detected positions; and

form an image indicating the detected position-of-interest in a case that the drawing input is detected.

2. The data input device according to claim 1, wherein in the case of forming the image, an information related to a processing of the image is established, the establishment of the information being performed in accordance with a coordinate value of the second position in a rearward direction.

3. The data input device according to claim 1,

wherein the one or more memory devices further storing instructions to:

detect a first shape of the another part of the body,

wherein in a case of forming the image, an information related to a processing the image is established, the establishment of the information being performed based on the first shape detected.

4. The data input device according to claim 1,

wherein the one or more memory devices further storing instructions to:

detect a second shape of the yet another part of the body,

wherein in a case of forming the image, an information related to a processing of the image is established, the establishment of the information being performed based on the second shape detected.

5. The data input device according to claim 1, wherein the one part of the body of the user is a face, and the another part of the body is either one hand of a left hand and a right hand.

6. The data input device according to claim 1, wherein the one part of the body of the user is a face, the another part of the body is either one hand of a left hand and a right hand, and the yet another part of the body is a hand that is opposite from the one hand.

9. The data input device according to claim 1, wherein in a case of the position-of-interest, as the position-of-interest, an intersection point between a straight line and the image display plane is used, the straight line passing through the first position and the second position.

10. The data input device according to claim 1,

wherein in a case of detecting the drawing input a three-dimensional operation detection space and the three-dimensional input detection space are established, the three-dimensional operation detection space being for detecting the another part, the three-dimensional input detection space being part of the three-dimensional operation detection space, and

wherein in a case of forming the image:

an input detection image is formed as the image, and

the input detection image in a first form which does not stand out compared to a second form is displayed, the first form being used in a case that the drawing input is not detected, the second form being used in a case that the drawing input is detected.

11. The data input device according to claim 1, wherein the three-dimensional input detection space is established such that the three-dimensional input detection space is rearward than the one boundary.

12. The data input device according to claim 11, wherein the three-dimensional input detection space is established such that the three-dimensional input detection space is deeper than a rearward value of the third position.

13. The data input device according to claim 1, wherein the one boundary of the three-dimensional input detection space is flexibly changed by using the third detected position.

14. The data input device according to claim 1, wherein the detection of the drawing input is performed based on whether the second position is rearward or forward from the boundary.

15. The data input device according to claim 1,

wherein the three-dimensional operation detection space includes the three-dimensional input detection space and a three-dimensional input non-detection space, and

the boundary is between the three-dimensional input detection space and the three-dimensional input non-detection space.

7. A data input method of a data input device, the data input method comprising:

detecting a first position, a second position, and a third position, the first position being based on a position of one part of a body of a user represented in a video captured by an image capturing device, the second position being based on a position of another part of the body, the third position being based on a position of yet another part of the body;

establishing a three-dimensional operation detection space based on the first detected position;

establishing one boundary of a three-dimensional input detection space in which a drawing input is detected, the establishment of the one boundary being performed by using, as a reference, the third detected position, the three-dimensional input detection space being included in the three-dimensional operation detection space;

detecting the drawing input, the detection of the drawing input being performed in a case that the second position is included in the three-dimensional input detection space, and whether or not the second position is farther rearward than the third position;

detecting a position-of-interest on an image display plane, the detection of the position-of-interest being performed based on the first and second detected positions; and

forming an image indicating the detected position-of-interest in a case that the drawing input is detected.

8. A non-transitory computer readable recording medium storing a data input program that causes a computer of a data input device to execute:

detecting a first position, a second position, and a third position, the first position being based on a position of one part of a body of a user represented in a video captured by an image capturing device, the second position being based on a position of another part of the body, the third position being based on a position of yet another part of the body;

establishing a three-dimensional operation detection space based on the first position detected;

establishing one boundary of a three-dimensional input detection space in which a drawing input is detected, the establishment of the one boundary being performed by using, as a reference, the third detected position, the three-dimensional input detection space being included in the three-dimensional operation detection space;

detecting the drawing input, the detection of the drawing input being performed in a case that the second position is included in the three-dimensional input detection space, and whether or not the second position is farther rearward than the third position;

detecting a position-of-interest on an image display plane, the detection of the position-of-interest being performed based on the first and second detected positions; and

forming an image indicating the detected position-of-interest in a case that the drawing input is detected.