OPERATION DETECTION DEVICE, OPERATION DETECTION METHOD AND PROJECTOR

• US 20140253513A1
• Filed: 02/24/2014
• Published: 09/11/2014
• Est. Priority Date: 03/11/2013
• Status: Active Grant

• Current Assignee: Maxell, Ltd.
• Original Assignee: Hitachi Maxell Limited (Hitachi, Ltd.)
• Inventors: MATSUBARA, Takashi, NARIKAWA, Sakiko, MORI, Naoki
• Granted Patent: US 9,367,176 B2
• Time in Patent Office: Days
• US Class Current: 345/175
• CPC Class Codes: G06F 3/017 Gesture based interaction, e.g. based on a set of recognized hand gestures interaction based on gestures traced on a digitiser G06F3/04883
• CPC Class Codes: G06F 3/042 by opto-electronic means
• CPC Class Codes: G06F 3/0425 using a single imaging device like a video camera for tracking the absolute position of a single or a plurality of objects with respect to an imaged reference surface, e.g. video camera imaging a display or a projection screen, a table or a wall surface, on which a computer generated image is displayed or projected tracking a projected light spot to determine a position on a display surface G06F3/0386
• CPC Class Codes: G06V 40/107 Static hand or arm
• CPC Class Codes: G06V 40/28 Recognition of hand or arm movements, e.g. recognition of deaf sign language static hand signs G06V40/113
• CPC Class Codes: H04N 9/31 Projection devices for colour picture display , e.g. using electronic spatial light modulators [ESLM]

Abstract

An operation detection device includes: first and second illuminations that irradiate illumination light from different positions onto an operation surface on which a user performs an operation; a camera that captures the operation surface together with an operation part (finger) of the user; a shadow region extraction unit that extracts first and second shadows of the operation part of the user from a captured image obtained by the camera; a contour detection unit that detects contours of each of the first and second shadows extracted; and a touch point detection unit that detects a touch point of the operation part of the user on the operation surface from the distance between the contours.

Claims

1. An operation detection device that detects an operation performed by a finger of a user with respect to an operation surface, the operation detection device comprising:

a first illumination and a second illumination that irradiate illumination light from different positions onto the operation surface;

a camera that captures, together with the finger of the user, the operation surface onto which the illumination light has been irradiated;

a shadow region extraction unit that, on the basis of a captured image obtained by the camera, extracts a first shadow of the finger of the user produced by the first illumination and a second shadow of the finger of the user produced by the second illumination; and

a touch point detection unit that, on the basis of the first shadow and the second shadow extracted, detects a touch point of the finger of the user on the operation surface.

2. The operation detection device according to claim 1, the operation detection device comprising:

a contour detection unit that detects a contour of the first shadow and a contour of the second shadow extracted, wherein

the touch point detection unit detects a touch point of the finger of the user on the operation surface from the distance between the contour of the first shadow and the contour of the second shadow.

3. The operation detection device according to claim 2, wherein

the shadow region extraction unit compares the brightness of the captured image with a predetermined threshold value, and discerns and extracts the first shadow of the finger of the user and the second shadow of the finger of the user,

the contour detection unit extracts, as contours, corresponding substantially linear line segments from within the contour of the first shadow and the contour of the second shadow, and

the touch point detection unit determines that the finger of the user has touched the operation surface when the distance between the two contours extracted has become equal to or less than a predetermined threshold value.

4. The operation detection device according to claim 1, wherein

the first illumination and the second illumination are made to irradiate in a temporally alternating manner,

the camera captures the operation surface in accordance with irradiation timings of each of the irradiation of the first illumination and the irradiation of the second illumination, and

the shadow region extraction unit extracts the first shadow from an image captured by the first illumination, and extracts the second shadow from an image captured by the second illumination temporally separated from the image captured by the first illumination.

5. The operation detection device according to claim 1, wherein

the first illumination and the second illumination are installed in such a way that the illumination directions thereof are oriented toward substantially the same side as the imaging direction of the camera.

6. The operation detection device according to claim 2, wherein

the contour detection unit extracts, from within the contour of the first shadow and the contour of the second shadow, a contour in a position far from the finger of the user, as a contour for each of the first shadow and the second shadow.

7. An operation detection device that detects an operation performed by a finger of a user with respect to an operation surface, the operation detection device comprising:

a plurality of illuminations that irradiate illumination light from different positions onto the operation surface;

a camera that captures, together with the finger of the user, the operation surface onto which the illumination light has been irradiated;

a shadow region extraction unit that, from a captured image obtained by the camera, extracts a region in which a plurality of shadows of the finger of the user overlap; and

a touch point detection unit that, on the basis of the region in which the plurality of shadows overlap, detects a touch point of the finger of the user on the operation surface, wherein

the shadow region extraction unit extracts a portion in which the brightness of a shadow is equal to or less than a predetermined threshold value, as the region in which the plurality of shadows overlap.

8. An operation detection method that detects an operation performed by a finger of a user with respect to an operation surface, wherein

illumination light is irradiated from different positions onto the operation surface by a first illumination and a second illumination,

together with the finger of the user, the operation surface onto which the illumination light has been irradiated is captured by a camera,

a first shadow of the finger of the user produced by the first illumination and a second shadow of the finger of the user produced by the second illumination are extracted on the basis of a captured image obtained by the camera, and

a touch point of the finger of the user on the operation surface is detected on the basis of the first shadow and the second shadow extracted.

9. The operation detection method according to claim 8, wherein

corresponding substantially linear line segments are extracted as contours from within contours of the first shadow and the second shadow extracted, and

it is determined that the finger of the user has touched the operation surface when the distance between the two contours extracted has become equal to or less than a predetermined threshold value.

10. The operation detection method according to claim 8, wherein

the first illumination and the second illumination are made to irradiate in a temporally alternating manner,

the operation surface is captured by the camera in accordance with irradiation timings of each of the irradiation of the first illumination and the irradiation of the second illumination, and

the first shadow is extracted from an image captured by the first illumination, and the second shadow is extracted from an image captured by the second illumination temporally separated from the image captured by the first illumination.

11. An operation detection method that detects an operation performed by a finger of a user with respect to an operation surface, wherein

illumination light is irradiated from different positions onto the operation surface by a plurality of illuminations,

together with the finger of the user, the operation surface onto which the illumination light has been irradiated is captured by a camera,

a maximum shadow density region in which a plurality of shadows of the finger of the user overlap and the brightness of the shadows is equal to or less than a predetermined threshold value is extracted on the basis of a captured image obtained by the camera, and

a touch point of the finger of the user on the operation surface is detected from the maximum shadow density region.

12. A projector comprising:

a video projection unit that projects video;

a first illumination and a second illumination that irradiate illumination light from different positions onto an operation surface at least part of which overlaps a video surface projected by the video projection unit;

a camera that captures, together with the finger of the user, the operation surface onto which the illumination light has been irradiated;

a shadow region extraction unit that, on the basis of a captured image obtained by the camera, extracts a first shadow of the finger of the user produced by the first illumination and a second shadow of the finger of the user produced by the second illumination; and

a touch point detection unit that, on the basis of the first shadow and the second shadow extracted, detects a touch point of the finger of the user on the operation surface.

13. The projector according to claim 12, the projector comprising:

a contour detection unit that detects a contour of the first shadow and a contour of the second shadow extracted, wherein

the touch point detection unit detects a touch point of the finger of the user on the operation surface from the distance between the contour of the first shadow and the contour of the second shadow.

14. The projector according to claim 13, wherein

the shadow region extraction unit compares the brightness of the captured image with a predetermined threshold value, and discerns and extracts the first shadow of the finger of the user and the second shadow of the finger of the user,

the contour detection unit extracts, as contours, corresponding substantially linear line segments from within the contour of the first shadow and the contour of the second shadow, and

the touch point detection unit determines that the finger of the user has touched the operation surface when the distance between the two contours extracted has become equal to or less than a predetermined threshold value.

15. The projector according to claim 12, wherein

the first illumination and the second illumination are made to irradiate in a temporally alternating manner,

the camera captures the operation surface in accordance with irradiation timings of each of the irradiation of the first illumination and the irradiation of the second illumination, and

the shadow region extraction unit extracts the first shadow from an image captured by the first illumination, and extracts the second shadow from an image captured by the second illumination temporally separated from the image captured by the first illumination.

16. The projector according to claim 12, wherein

the first illumination and the second illumination are installed in such a way that the illumination directions thereof are oriented toward substantially the same side as the imaging direction of the camera.

17. The projector according to claim 13, wherein

the contour detection unit extracts, from within the contour of the first shadow and the contour of the second shadow, a contour in a position far from the finger of the user, as a contour for each of the first shadow and the second shadow.

18. A projector that detects an operation performed by a finger of a user with respect to an operation surface, the projector comprising:

a plurality of illuminations that irradiate illumination light from different positions onto the operation surface;

a camera that captures, together with the finger of the user, the operation surface onto which the illumination light has been irradiated;

a shadow region extraction unit that, from a captured image obtained by the camera, extracts a region in which a plurality of shadows of the finger of the user overlap; and

a touch point detection unit that, on the basis of the region in which the plurality of shadows overlap, detects a touch point of the finger of the user on the operation surface, wherein

the shadow region extraction unit extracts a portion in which the brightness of a shadow is equal to or less than a predetermined threshold value, as the region in which the plurality of shadows overlap.