INTERFACE DEVICE, PORTABLE DEVICE, CONTROL DEVICE AND MODULE

• US 20160349926A1
• Filed: 01/07/2015
• Published: 12/01/2016
• Est. Priority Date: 01/10/2014
• Status: Abandoned Application

• Current Assignee: NEC Corporation
• Original Assignee: NEC Corporation
• Inventors: OKUMURA, Fujio
• Application Number:
  US15/110,486
• Publication Number:
  US 20160349926A1
• Time in Patent Office: Days
• US Class Current: 1/1
• CPC Class Codes: G03B 21/00 Projectors or projection-type viewers; Accessories therefor devices for changing pictures G03B23/00
• CPC Class Codes: G03B 21/26 Projecting separately subsidiary matter simultaneously with main image
• CPC Class Codes: G06F 1/163 Wearable computers, e.g. on a belt
• CPC Class Codes: G06F 1/1673 Arrangements for projecting a virtual keyboard
• CPC Class Codes: G06F 3/017 Gesture based interaction, e.g. based on a set of recognized hand gestures interaction based on gestures traced on a digitiser G06F3/04883
• CPC Class Codes: G06F 3/0304 Detection arrangements using opto-electronic means constructional details of pointing devices not related to the detection arrangement using opto-electronic means G06F3/033; optical digitisers G06F3/042
• CPC Class Codes: G06F 3/0346 with detection of the device orientation or free movement in a 3D space, e.g. 3D mice, 6-DOF [six degrees of freedom] pointers using gyroscopes, accelerometers or tilt-sensors
• CPC Class Codes: G06F 3/0425 using a single imaging device like a video camera for tracking the absolute position of a single or a plurality of objects with respect to an imaged reference surface, e.g. video camera imaging a display or a projection screen, a table or a wall surface, on which a computer generated image is displayed or projected tracking a projected light spot to determine a position on a display surface G06F3/0386
• CPC Class Codes: G06F 3/0426 tracking fingers with respect to a virtual keyboard projected or printed on the surface virtual keyboards on touch screens G06F3/04886
• CPC Class Codes: G06F 3/0482 Interaction with lists of selectable items, e.g. menus
• CPC Class Codes: G06F 3/04845 for image manipulation, e.g. dragging, rotation, expansion or change of colour
• CPC Class Codes: G06F 3/04886 by partitioning the display area of the touch-screen or the surface of the digitising tablet into independently controllable areas, e.g. virtual keyboards or menus
• CPC Class Codes: H04N 9/3194 including sensor feedback

Abstract

In order to provide an interface that has input functionality with high recognition precision, this interface device (100) is provided with an imaging unit (110), a control unit (120), and a projection unit (130). The protection unit (130) projects projected video (300). The imaging unit (110) captures video of the area in which the projected video (300) is being projected. If captured video captured by the imaging unit (110) contains both the projected video (300) and a user-input object (400), then on the basis of the relationship between the position of the projected video (300) in the captured video and the position of the user-input object (400) in the captured video, the control unit (120) generates user-input information by recognizing user input being performed using the user-input object (400).

Claims

1. An interface device comprising:

a projection unit that projects a first projected image;

an imaging unit that captures an image of a region on which the first projected image is projected; and

a control unit that, when the first projected image is captured and also an operating object is captured in a captured image that is an image captured by the imaging unit, recognizes operation information on operation of the operating object based on a positional relation between a captured position in which the first projected image is captured and a captured position in which the operating object is captured in the captured image.

2. The interface device according to claim 1, wherein

the control unit controls the projection unit to project a second projected image in response to the operation information, and

the control unit further recognizes operation information on next operation of the operating object based on a positional relation between a captured position of the operating object and a captured position of the second projected image provided by the projection unit in the captured image.

3. The interface device according to claim 2, wherein the first projected image is an image in which a plurality of options are displayed,

the control unit recognizes information of an operation which selects a selected option from the plurality of options based on a positional relation between a captured position of the operating object and a captured position of the first projected image in the captured image, and

the second projected image is an image in which an option related to the selected option is displayed.

4. The interface device according to claim 3, wherein a criterion used by the control unit when recognizing operation information of the operating object based on a captured position of the first projected image in the captured image, is different from a criterion used by the control unit when recognizing operation information of the operating object based on a captured position of the second projected image in the captured image.

5. The interface device according to claim 2, wherein the projection unit projects the second projected image with superposing on the first projected image.

6. The interface device according to claim 1, wherein the control unit recognizes operation information on the operating object based on information about a motion of the operating object in addition to the positional relation.

7. The interface device according to claim 6, wherein,

when detecting an action of the operating object staying for a predetermined time or longer, the control unit recognizes operation information on the operating object based on the projected image projected on a position at which the operating object stays, or,

when detecting an action of the operating object moving linearly at a predetermined speed or greater, or at a predetermined acceleration or greater, the control unit recognizes operation information on the operating object based on the projected image positioned in a direction in which the operating object moves, or,

when detecting an action of the operating object moving linearly at a predetermined speed or greater, or at a predetermined acceleration or greater, the control unit recognizes operation information on the operating object based on the projected image projected on a position from which the operating object starts the motion.

8. The interface device according to claim 1, wherein the control unit has a function of distinguishing a plurality of types of the operating objects, and,

when the plurality of types of the operating objects are displayed in the captured image, the control unit controls the projection unit in such a way mutually distinct projected images are projected in proximity to the respective operating objects.

9. The interface device according to claim 8, wherein one of the operating objects is a thumb of a user, and another of the operating objects is a forefinger of the user,

the control unit controls the projection unit such that the first projected image is projected on a region including the thumb and the forefinger of the user, and further controls the projection unit such that mutually distinct projected images are projected in proximity to the thumb and in proximity to the forefinger, respectively, and

the control unit further recognizes operation information on the thumb based on a positional relation between a captured position of the thumb and a captured position of the projected image projected in proximity to the thumb, and also recognizes operation information on the forefinger based on a positional relation between a captured position of the forefinger and a captured position of the projected image projected in proximity to the forefinger.

12. The interface device according to claim 1, wherein the projection unit includes a laser light source emitting laser light and an element modulating a phase of the laser light entered from the laser light source and emitting the laser light after the modulation, and

the control unit determines an image formed by light emitted by the element in response to the operation information recognizing, and controls the element such that the determined image is formed.

13. The interface device according to claim 12, wherein the element includes a plurality of light-receiving regions and each of the light-receiving regions modulates a phase of laser light incident on the light-receiving region and emits the modulated light and,

the control unit controls the element such that a parameter of the light-receiving region is changed for each of the light-receiving regions, the parameter determining a difference between a phase of light incident on the light-receiving region and a phase of light emitting from the light-receiving region.

14. A portable device comprising the interface device according to claim 1.

10. An interface device comprising:

a projection unit that projects a projected image;

an imaging unit that captures an image of a region on which the projected image is projected; and

a control unit that calculates a positional relation between a surface on which the projected image is projected and the imaging unit based on information about a captured position in which the projected image is captured in a captured image being an image captured by the imaging unit.

11. The interface device according to claim 10, wherein, when the projected image is captured and also an operating object is captured in a captured image being an image captured by the imaging unit, the control unit recognizes operation information on the operating object based on a positional relation between a captured position in which the projected image is captured and a captured position in which the operating object is captured in the captured image, and,

the projected image provided by the projection unit further includes an image region used in processing of recognizing the operation information of the operating object, and an image region used in processing of calculating the positional relation between a surface on which the projected image is projected and the imaging unit.

15. A control device, comprising

a unit that receives a captured image displaying a projected image projected by a projection unit being a control target;

when the projected image and also an operating object are displayed in the captured image, recognizing operation information on the operating object based on a relation between a captured position in which the projected image is displayed and a captured position in which the operating object is displayed in the captured image; and

controlling the projection unit in response to the operation information.

16. A module comprising:

the control device according to claim 15 and

a projection unit controlled by the control device.

17. (canceled)

18. (canceled)