Information device operation apparatus

• US 20080240507A1
• Filed: 03/25/2008
• Published: 10/02/2008
• Est. Priority Date: 03/30/2007
• Status: Active Grant

• Current Assignee: DENSO Corporation
• Original Assignee: DENSO Corporation
• Inventors: Niwa, Shinji, Akahori, Ichiro, Kawashima, Takeshi, Morishita, Toshiyuki
• Granted Patent: US 8,023,698 B2
• Time in Patent Office: Days
• US Class Current: 382/104
• CPC Class Codes: G06F 3/017 Gesture based interaction, e.g. based on a set of recognized hand gestures interaction based on gestures traced on a digitiser G06F3/04883
• CPC Class Codes: G06V 40/28 Recognition of hand or arm movements, e.g. recognition of deaf sign language static hand signs G06V40/113

Abstract

An information device operation apparatus includes an image capturing device, an illumination device, a controller, image storage device, operation object extracting device, detection device, and signal output device. The image storage device stores a plurality of images that are acquired by the image capturing device under a plurality of illumination intensity conditions. The operation object extracting device extracts the operation object by comparing the plurality of images. The detection device detects at least one of a predetermined shape and a movement of the operation object that is extracted by the operation object extracting device. The signal output device outputs a signal to the target device, the signal corresponding to the at least one of the predetermined shape and the movement of the operation object detected by the detection device.

Claims

1. An information device operation apparatus for extracting an operation object by image processing, wherein the operation object is used for an operation of a target device, the information device operation apparatus comprising:

an image capturing device that is disposed at a position for capturing an image of an opposite surface of the operation object opposite to a reception surface of the operation object, the reception surface receiving extraneous light.

an illumination device that is disposed at a position for generating light that is applied to the opposite surface of the operation object;

control means for controlling the illumination device to generate the light under a plurality of luminance conditions, the control means controlling the image capturing device to capture the image;

image storage means for storing a plurality of images that are acquired by the image capturing device under a plurality of illumination intensity conditions, the plurality of illumination intensity conditions being caused by the illumination device that is controlled by the control means to generate the plurality of luminance conditions;

operation object extracting means for extracting the operation object by comparing the plurality of images;

detection means for detecting at least one of a predetermined shape and a movement of the operation object that is extracted by the operation object extracting means; and

signal output means for outputting a signal to the target device, the signal corresponding to the at least one of the predetermined shape and the movement of the operation object detected by the detection means.

2. The information device operation apparatus according to claim 1, wherein the detection means detects the movement of the operation object on an imaginary plane that is perpendicular to an optical axis of the image capturing device.

3. The information device operation apparatus according to claim 1, wherein:

the detection means includes rotation angle detection means for detecting a rotation angle of the operation object with respect to an imaginary plane that is perpendicular to an optical axis of the image capturing device; and

the signal output means outputs the rotation angle of the operation object detected by the rotation angle detection means.

4. The information device operation apparatus according to claim 3, wherein:

the operation object extracting means causes the image storage means to store an extracted image of the operation object indicating an extract result; and

the rotation angle detection means detects an edge direction of each pixel of the extracted image stored in the image storage means, the rotation angle detection means detecting an average edge direction based on the detected edge directions, the rotation angle detection means determining the detected average edge direction as the rotation angle of the operation object.

5. The information device operation apparatus according to claim 1, wherein:

the detection means includes movement detection means for detecting a movement direction and a movement amount of the operation object in two directions that are orthogonal to each other on an imaginary plane, the imaginary plane being perpendicular to an optical axis of the image capturing device; and

the signal output means outputs the movement direction and the movement amount of the operation object detected by the movement detection means.

6. The information device operation apparatus according to claim 5, wherein:

the movement detection means detects a point-of-balance position of the operation object extracted by the operation object extracting means; and

the movement detection means detects the movement direction and the movement amount of the operation object based on the detected point-of-balance position of the operation object.

7. The information device operation apparatus according to claim 1, wherein the signal output means is configured to output at least one of a rotation angle, a movement direction, and a movement amount of the operation object detected by the detection means.

8. The information device operation apparatus according to claim 1, wherein:

the detection means detects the movement of the operation object in a three dimensional space that is defined by an optical axis of the image capturing device and two axes that are orthogonal to each other on an imaginary plane, the imaginary plane being perpendicular to an optical axis of the image capturing device.

9. The information device operation apparatus according to claim 8, wherein:

the detection means includes space rotation angle detection means for detecting a three dimensional rotation angle of the operation object in the three dimensional space, the three dimensional rotation angle indicating an angle measured about a rotation axis associated with the operation object in the three dimensional space; and

the signal output means outputs the three dimensional rotation angle of the operation object detected by the space rotation angle detection means.

10. The information device operation apparatus according to claim 9, wherein:

the space rotation angle detection means divides the extracted operation object into a plurality of regions, the extracted operation object being extracted by the operation object extracting means;

the space rotation angle detection means detects an average luminance and a point-of-balance position for each of the plurality of regions; and

the space rotation angle detection means detects the three dimensional rotation angle of the operation object based on the detected average luminance and the detected point-of-balance position for the each of the plurality of regions.

11. The information device operation apparatus according to claim 10, wherein:

the space rotation angle detection means acquires a point-of-balance position of the extracted operation object; and

the space rotation angle detection means divides the extracted operation object along one of the followings;

a plurality of lines that passes through the acquired point-of-balance position; and

a parallel line, which is in parallel with a line passing through the point-of-balance position, and a crossing line, which intersects the parallel line.

12. The information device operation apparatus according to claim 9, further comprising at least three distance sensors, each of which measures a distance to the operation object, wherein:

the space rotation angle detection means detects the three dimensional rotation angle of the operation object based on the distances measured by the at least three distance sensor.

13. The information device operation apparatus according to claim 9, wherein:

the illumination device includes a first illumination device generating a first light and a second illumination device generating a second light;

the second illumination device is disposed at a position different from a position of the first illumination device;

the second illumination device is configured to generate the second light that intersects the first light generated by the first illumination device at a position that is a first predetermined distance away from the image capturing device;

the space rotation angle detection means detects the first light and second light projected on the operation object based on the image inputted by the image capturing device; and

the space rotation angle detection means detects the three dimensional rotation angle of the operation object based on a distance between the first light and the second light projected on the operation object, the operation object being positioned on an imaginary plane, which is a second predetermined distance away from the image capturing device, and which is perpendicular to the optical axis of the image capturing device.

14. The information device operation apparatus according to claim 13, wherein:

the first illumination device is configured to generate the first light that has directivity; and

the second illumination device is configured to generate the second light that has directivity.

15. The information device operation apparatus according to claim 13, further comprising:

a first baffle device that is disposed between the first illumination device and the operation object, the first baffle device having an opening, the first baffle device allowing the first illumination device to project part of the first light on the operation object through the opening of the first baffle device; and

a second baffle device that is disposed between the second illumination device and the operation object, the second baffle device having an opening, the second baffle device allowing the second illumination device to project part of the second light on the operation object through the opening of the second baffle device, wherein;

the first baffle device and the second baffle device are arranged such that the part of the first light intersects the part of the second light at a position that is the first predetermined distance away from the image capturing device;

the space rotation angle detection means detects the part of the first light and the part of the second light projected on the operation object based on the image inputted by the image capturing device; and

the space rotation angle detection means detects the three dimensional rotation angle of the operation object based on a distance between the part of the first light and the part of the second light projected on the operation object, the operation object being positioned on the imaginary plane, which is the second predetermined distance away from the image capturing device, and which is perpendicular to the optical axis.

16. The information device operation apparatus according to claim 8, wherein:

the detection means includes space movement detection means for detecting a movement direction and a movement amount of the operation object in the three dimensional space; and

the signal output means outputs the detected movement direction and the detected movement amount of the operation object in the three dimensional space.

17. The information device operation apparatus according to claim 16, wherein:

the space movement detection means detects an average luminance value of the extracted operation object at predetermined time intervals, the extracted operation object being extracted by the operation object extracting means; and

the space movement detection means detects the movement of the operation object in a direction of the optical axis of the image capturing device by comparing the detected average luminance values.

18. The information device operation apparatus according to claim 16, further comprising:

a distance sensor that measures a distance to the operation object, wherein;

the space movement detection means detects the movement direction and the movement amount of the operation object in the three dimensional space based on the distance detected by the distance sensor.

19. The information device operation apparatus according to claim 16, wherein:

the illumination device includes a first illumination device generating a first light and a second illumination device generating a second light;

the second illumination device is disposed at a position different from a position of the first illumination device;

the second illumination device is configured to generate the second light that intersects the first light generated by the first illumination device at a position that is a predetermined distance away from the image capturing device;

the space movement detection means detects a first light spot and a second light spot, which are formed on the operation object, based on the image inputted by the image capturing device, the first light spot being formed by projecting the first light on the operation object, the second light spot being formed by projecting the second light on the operation object;

the space movement detection means detects a size of the detected first light spot and a size of the detected second light spot based on the image acquired from the image capturing device; and

the space movement detection means detects a distance to the operation object based on the size of the first light spot and the size of the second light spot.

20. The information device operation apparatus according to claim 19, wherein:

the first illumination device is configured to generate the first light that has directivity; and

the second illumination device is configured to generate the second light that has directivity.

21. The information device operation apparatus according to claim 19, further comprising:

a first baffle device that is disposed between the first illumination device and the operation object, the first baffle device having an opening, the first baffle device allowing the first illumination device to project part of the first light on the operation object through the opening of the first baffle device; and

a second baffle device that is disposed between the second illumination device and the operation object, the second baffle device having an opening, the second baffle device allowing the second illumination device to project part of the second light to the operation object through the opening of the second baffle device, wherein;

the first baffle device and the second baffle device are arranged such that the part of the first light intersects the part of the second light at a position that is a predetermined distance away from the image capturing device;

the space movement detection means detects the part of the first light and the part of the second light, which are projected on the operation object, based on the image inputted by the image capturing device;

the space movement detection means detects a first light spot and a second light spot, which are formed on the operation object, based on the image inputted by the image capturing device, the first light spot being formed by projecting the first light on the operation object, the second light spot being formed by projecting the second light on the operation object;

the space movement detection means detects a size of the detected first light spot and a size of the detected second light spot based on the image acquired from the image capturing device; and

the space movement detection means detects a distance to the operation object based on the size of the first light spot and the size of the second light spot.

22. The information device operation apparatus according to claim 8, wherein the signal output means is configured to output at least one of a rotation angle, a movement direction, a the movement amount of the operation object in the space detected by the detection means.

23. The information device operation apparatus according to claim 1, wherein the signal output means is configured to output the signal that corresponds to the predetermined shape of the operation object detected by the detection means.

24. The information device operation apparatus according to claim 23, wherein:

the image storage means stores a predetermined pattern image; and

the detection means determines that the operation object has the predetermined shape by comparing the predetermined pattern image stored in the image storage means and the image of the operation object extracted by the operation object extracting means.

25. The information device operation apparatus according to claim 1, further comprising output switch means for switching the signal, which is outputted by the signal output means, based on an operation by an operator, wherein:

the signal output means is configured to output the signal in accordance with a state of the output switch means when the operation object indicates the predetermined shape and movement.

26. The information device operation apparatus according to claim 25, wherein:

the output switch means is operated when the operator pushes the output switch means;

the signal output means outputs a first signal when the output switch means is pushed; and

the signal output means outputs a second signal when the output switch means is not pushed.