MANIPULATION DETECTION APPARATUS, MANIPULATION DETECTION METHOD, AND PROJECTOR

• US 20140253512A1
• Filed: 02/04/2014
• Published: 09/11/2014
• Est. Priority Date: 03/11/2013
• Status: Active Grant

• Current Assignee: Maxell, Ltd.
• Original Assignee: Hitachi Maxell Limited (Maxell, Ltd.)
• Inventors: MORI, Naoki, NARIKAWA, Sakiko, MATSUBARA, Takashi
• Granted Patent: US 10,152,177 B2
• Time in Patent Office: Days
• US Class Current: 345/175
• CPC Class Codes: G06F 3/0304 Detection arrangements using opto-electronic means constructional details of pointing devices not related to the detection arrangement using opto-electronic means G06F3/033; optical digitisers G06F3/042
• CPC Class Codes: G06F 3/0425 using a single imaging device like a video camera for tracking the absolute position of a single or a plurality of objects with respect to an imaged reference surface, e.g. video camera imaging a display or a projection screen, a table or a wall surface, on which a computer generated image is displayed or projected tracking a projected light spot to determine a position on a display surface G06F3/0386

Abstract

A shadow region extraction portion compares the brightness of an image captured by a camera with a threshold value and extracts first and second shadows caused by first and second light sources for a user'"'"'s finger. An approach degree detection portion detects an approach degree for the finger with reference to a manipulation surface from distance d between two feature points in the first and second shadows. A contact point detection portion detects contact point P for the finger with reference to the manipulation surface from positions of the two feature points. A direction detection portion detects a direction pointed by the finger from directions of contours corresponding to the first and second shadows. A display control portion generates display control data for a manipulation target apparatus based on detection result data.

Claims

1. A manipulation detection apparatus to detect manipulation by a user'"'"'s finger with reference to a manipulation surface, comprising:

first and second light sources to radiate illumination light to the manipulation surface from different positions;

a camera to capture the manipulation surface radiated by the illumination light as well as the user'"'"'s finger;

a shadow region extraction portion that extracts first and second shadows caused by the first and second light sources corresponding to the user'"'"'s finger based on an image captured by the camera; and

a manipulation state detection portion that detects at least one of manipulation states such as an approach degree, a contact point, and a direction indicated by the user'"'"'s finger with reference to the manipulation surface from shapes of the extracted first and second shadows.

2. The manipulation detection apparatus according to claim 1,

wherein the manipulation state detection portion includes:

a feature point detection portion to detect a feature point from each of the extracted first and second shadows;

a contour detection portion to detect an approximately linear line segment as each contour of the extracted first and second shadows from contours of the extracted first and second shadows;

an approach degree detection portion to detect an approach degree for the user'"'"'s finger toward the manipulation surface from a distance between the two detected feature points;

a contact point detection portion to detect a contact point for the user'"'"'s finger on the manipulation surface from positions of the two detected feature points; and

a direction detection portion to detect a pointing direction of the user'"'"'s finger from a direction of the detected contour,

wherein the approach degree detection portion determines contact of the user'"'"'s finger with the manipulation surface based on a distance between the two detected feature points.

3. The manipulation detection apparatus according to claim 2,

wherein the feature point detection portion detects a tip position of the finger in the detected first and second shadows as the feature point, and

wherein the contact point detection portion detects a center position of the two detected feature points as the contact point.

4. The manipulation detection apparatus according to claim 2,

wherein, when the extracted first and second shadows are extracted while overlapping with each other, the shadow region extraction portion adds a weak-shadow region and an adjacent strong-shadow region to separate the overlapping shadows into the first and second shadows resulting from the first and second light sources.

5. The manipulation detection apparatus according to claim 1, comprising:

a display control portion to generate display control data for a manipulation target apparatus connected to the manipulation detection apparatus based on a detection result including an approach degree, a contact point, and a direction pointed by a finger detected by the manipulation state detection portion.

6. The manipulation detection apparatus according to claim 5,

wherein the display control data generated by the display control portion contains at least one of manipulation mode changeover, pointer display position correction, pointer display direction correction, and contact point position correction.

7. A manipulation detection method of detecting user manipulation with reference to a manipulation surface comprising:

using first and second light sources to radiate illumination light to the manipulation surface from different positions;

using a camera to capture the manipulation surface radiated by the illumination light as well as the user'"'"'s finger;

extracting first and second shadows caused by the first and second light sources corresponding to the user'"'"'s finger based on an image captured by the camera; and

detecting at least one of manipulation states such as an approach degree, a contact point, and a direction indicated by the user'"'"'s finger with reference to the manipulation surface from shapes of the extracted first and second shadows.

8. The manipulation detection method according to claim 7,

wherein detection of the manipulation state requires:

detecting a feature point from each of the extracted first and second shadows;

detecting an approximately linear line segment as each contour of the extracted first and second shadows from the extracted first and second shadows;

detecting an approach degree for the user'"'"'s finger toward the manipulation surface from a distance between the two detected feature points;

detecting a contact point for the user'"'"'s finger on the manipulation surface from positions of the two detected feature points; and

detecting a pointing direction of the user'"'"'s finger from a direction of the detected contour,

wherein the method determines contact of the user'"'"'s finger with the manipulation surface based on a distance between the two detected feature points.

9. The manipulation detection method according to claim 8, comprising:

detecting a tip position of the finger in the detected first and second shadows as the feature point; and

detecting a center position of the two detected feature points as the contact point.

10. The manipulation detection method according to claim 8,

wherein, when the extracted first and second shadows are extracted while overlapping with each other, extraction of a shadow region adds a weak-shadow region and an adjacent strong-shadow region to separate the overlapping shadows into the first and second shadows resulting from the first and second light sources.

11. The manipulation detection method according to claim 7, comprising:

generating display control data for a manipulation target apparatus based on a detection result including an approach degree, a contact point, and a direction pointed by a finger detected as the manipulation state.

12. The manipulation detection method according to claim 11,

wherein display control data contains at least one of manipulation mode changeover, pointer display position correction, pointer display direction correction, and contact point position correction.

13. A projector comprising:

a projection portion to project video;

first and second light sources to radiate illumination light from different positions to a manipulation surface that at least partially overlaps with video projected by the projection portion;

a camera to capture the manipulation surface radiated by the illumination light as well as a user'"'"'s finger;

a shadow region extraction portion that extracts first and second shadows caused by the first and second light sources corresponding to the user'"'"'s finger based on an image captured by the camera; and

a manipulation state detection portion that detects at least one of manipulation states such as an approach degree, a contact point, and a direction indicated by the user'"'"'s finger with reference to the manipulation surface from shapes of the extracted first and second shadows.

14. The projector according to claim 13,

wherein the manipulation state detection portion includes:

a feature point detection portion to detect a feature point from each of the extracted first and second shadows;

a contour detection portion to detect an approximately linear line segment as each contour of the extracted first and second shadows from contours of the extracted first and second shadows;

an approach degree detection portion to detect an approach degree for the user'"'"'s finger toward the manipulation surface from a distance between the two detected feature points;

a contact point detection portion to detect a contact point for the user'"'"'s finger on the manipulation surface from positions of the two detected feature points; and

a direction detection portion to detect a pointing direction of the user'"'"'s finger from a direction of the detected contour,

wherein the approach degree detection portion determines contact of the user'"'"'s finger with the manipulation surface based on a distance between the two detected feature points.

15. The projector according to claim 14,

wherein the feature point detection portion detects a tip position of the finger in the detected first and second shadows as the feature point;

and

wherein the contact point detection portion detects a center position of the two detected feature points as the contact point.

16. The projector according to claim 14,

wherein, when the extracted first and second shadows are extracted while overlapping with each other, the shadow region extraction portion adds a weak-shadow region and an adjacent strong-shadow region to separate the overlapping shadows into the first and second shadows resulting from the first and second light sources.

17. The projector according to claim 13, comprising:

a display control portion to generate display control data for a manipulation target apparatus connected to the manipulation detection apparatus based on a detection result including an approach degree, a contact point, and a direction pointed by a finger detected by the manipulation state detection portion.

18. The projector according to claim 17,

wherein the display control data generated by the display control portion contains at least one of manipulation mode changeover, pointer display position correction, pointer display direction correction, and contact point position correction.