Human interface system using a plurality of sensors

• US 20020118232A1
• Filed: 08/29/2001
• Published: 08/29/2002
• Est. Priority Date: 02/23/2001
• Status: Active Grant

• Current Assignee: Fujitsu Limited
• Original Assignee: Fujitsu Limited
• Inventors: Watanabe, Kazuhiro, Watanabe, Masaki, Murase, Kentaro, Noda, Takuya
• Granted Patent: US 6,583,723 B2
• Time in Patent Office: Days
• US Class Current: 345/863
• CPC Class Codes: G06F 3/017 Gesture based interaction, e.g. based on a set of recognized hand gestures interaction based on gestures traced on a digitiser G06F3/04883
• CPC Class Codes: G07C 9/37 using biometric data, e.g. fingerprints, iris scans or voice recognition

Abstract

An image sensor, a voice sensor, an auxiliary sensor part (infrared sensor, etc.), a total analyzing part, and an application communicate with each other through data/control signal communication units. Each sensor provides feedback on its signal detection results and control information used by the other sensors for determining a range of a detection target and a detection sensitivity at a time of subsequent signal acquisition, to the other sensors through the communication units. The total analyzing part investigates whether or not there is inconsistency among the results detected by the respective sensors, and provide control information to each sensor. Each sensor determines a range of a signal detection target and a detection sensitivity based on the obtained information, and acquires a signal in accordance with the determination.

Claims

1. A human interface system using a plurality of sensors, comprising:

at least two kinds of sensors, each determining a range of a detection target and a detection sensitivity and acquiring a particular detection signal from the detection target at the detection sensitivity, the detection signals acquired by the sensors being of different types;

a total analyzing part for investigating whether or not there is inconsistency among signal detection results obtained by the respective sensors, and generating control information to the respective sensors;

an application utilizing the signal detection results obtained by the respective sensors; and

communication units for communicating data and control information between the respective sensors, between the respective sensors and the total analyzing part, and between the total analyzing part and the application,

wherein each of the sensors uses either of or a combination of

the signal detection results or control information obtained from the other sensors, and

the control information obtained from the total analyzing part,

thereby determining a range of a detection target and a detection sensitivity at a time of subsequent signal acquisition,

each of the sensors outputs its signal detection results and control information used by the other sensors for determining a range of a detection target and a detection sensitivity at a time of subsequent signal acquisition, to the other sensors through the communication units, and

the total analyzing part outputs control information used by each of the sensors for determining a range of a detection target and a detection sensitivity at a time of subsequent signal acquisition through the communication units.

2. A human interface system using a plurality of sensors according to claim 1, wherein the detection target is a human being, and the sensors include at least an image sensor and a voice sensor,

a detection signal of the image sensor is human picture recognition information,

the image sensor includes an action recognizing part for interpreting an action of the detection target based on picture recognition results and recognizing a command inputted through a gesture,

a detection signal of the voice sensor is human voice recognition information, and

the voice sensor includes a voice recognizing part for interpreting a voice of the detection target based on voice recognition results, and recognizing a command inputted through a voice.

3. A human interface system using a plurality of sensors according to claim 1, wherein the detection target is a human being, and the sensors include at least an image sensor and an auxiliary sensor,

a detection signal of the image sensor is human picture recognition information,

the image sensor includes an action recognizing part for interpreting an action of the detection target based on picture recognition results and recognizing a command inputted through a gesture, and

a detection signal of the auxiliary sensor is information useful for detecting human position information.

4. A human interface system using a plurality of sensors according to claim 3, wherein either of or a combination of an ultrasonic sensor, an infrared sensor, and a range sensor is used as the auxiliary sensor.

5. A human interface system using a plurality of sensors according to claim 1, wherein the detection target is a human being, and the sensors include at least a voice sensor and an auxiliary sensor,

a detection signal of the voice sensor is human voice recognition information,

the voice sensor includes a voice recognizing part for interpreting a voice of the detection target based on voice recognition results and recognizing a command inputted through a voice, and

a detection signal of the auxiliary sensor is information useful for detecting human position information.

6. A human interface system using a plurality of sensors according to claim 5, wherein either of or a combination of an ultrasonic sensor, an infrared sensor, and a range sensor is used as the auxiliary sensor.

7. A human interface system using a plurality of sensors according to claim 1, wherein the detection target is a human being, and the sensors include at least an image sensor, a voice sensor, and an auxiliary sensor,

a detection signal of the image sensor is human picture recognition information,

the image sensor includes an action recognizing part for interpreting an action of the detection target based on picture recognition results, and recognizing a command inputted through a gesture,

a detection signal of the voice sensor is voice recognition information,

the voice sensor includes a voice recognizing part for interpreting a voice of the detection target based on voice recognition results and recognizing a command inputted through a voice, and

a detection signal of the auxiliary sensor is information useful for detecting human position information.

8. A human interface system using a plurality of sensors according to claim 7, wherein either of or a combination of an ultrasonic sensor, an infrared sensor, and a range sensor is used as the auxiliary sensor.

9. A human interface processing method using a plurality of sensors, comprising:

at least two kinds of sensor processings, each determining a range of a detection target and a detection sensitivity and acquiring a particular detection signal from the detection target at the detection sensitivity, the detection signals acquired by the sensor processings being of different types;

total analyzing processing for investigating whether or not there is inconsistency among signal detection results detected by the respective sensor processings, and generating control information to the respective sensor processings;

application processing utilizing the signal detection results obtained by the respective sensor processings; and

communication processing for communicating data and control information between the respective sensor processings, between the respective sensor processings and the total analyzing processing, and between the total analyzing processing and the application processing,

wherein each of the sensor processings uses either of or a combination of the signal detection results or control information obtained from the other sensor processings, and the control information obtained from the total analyzing processing, thereby determining a range of a detection target and a detection sensitivity at a time of subsequent signal acquisition,

each of the sensor processings outputs its signal detection results to the other sensor processings through the communication processing, and the other sensor processings output control information used by the other sensor processings for determining a range of a detection target and a detection sensitivity at a time of subsequent signal acquisition, and

the total analyzing processing outputs control information used by each of the sensor processings for determining a range of a detection target and a detection sensitivity at a time of subsequent signal acquisition through the communication processings.

10. A human interface processing method using a plurality of sensors according to claim 9, wherein the detection target is a human being, and the sensor processings include at least image sensor processing and voice sensor processing,

a detection signal of the image sensor processing is human recognition information,

the image sensor processing includes action recognition processing for interpreting an action of the detection target based on picture recognition results and recognizing a command inputted through a gesture,

a detection signal of the voice sensor processing is human voice recognition information, and

the voice sensor processing includes voice recognition processing for interpreting a voice of the detection target based on voice recognition results and recognizing a command inputted through a voice.

11. A human interface processing method using a plurality of sensors according to claim 9, wherein the detection target is a human being, and the sensor processings include at least image sensor processing and auxiliary sensor processing,

a detection signal of the image sensor processing is human picture recognition information,

the image sensor processing includes action recognition processing for interpreting an action of the detection target based on picture recognition results and recognizing a command inputted through a gesture, and

a detection signal of the auxiliary sensor processing is information useful for detecting human position information.

13. A human interface processing method using a plurality of sensors according to claim 9, wherein the detection target is a human being, and the sensor processings include at least voice sensor processing and auxiliary sensor processing,

a detection signal of the voice sensor processing is human voice recognition information,

the voice sensor processing includes voice recognition processing for interpreting a voice of the detection target based on voice recognition results and recognizing a command inputted through a voice, and

a detection signal of the auxiliary sensor processing is information useful for detecting human position information.

15. A human interface processing method using a plurality of sensors according to claim 9, wherein the detection target is a human being, and the sensor processings include at least image sensor processing, voice sensor processing, and auxiliary sensor processing,

a detection signal of the image sensor processing is human picture recognition information,

the image sensor processing includes action recognition processing for interpreting an action of the detection target based on picture recognition results and recognizing a command inputted through a gesture, and

a detection signal of the voice sensor processing is human voice recognition information,

the voice sensor processing includes voice recognition processing for interpreting a voice of the detection target based on voice recognition results and recognizing a command input by a voice input, and

a detection signal of the auxiliary sensor processing is information useful for detecting human position information.

12. A human interface processing method using a plurality of sensors according 11, wherein either of or a combination of ultrasonic sensor processing, infrared sensor processing, and range sensor processing is used as the auxiliary sensor processing.

14. A human interface processing method using a plurality of sensors according 13, wherein either of or a combination of ultrasonic sensor processing, infrared sensor processing, and range sensor processing is used as the auxiliary sensor processing.

16. A human interface processing method using a plurality of sensors according 15, wherein either of or a combination of ultrasonic sensor processing, infrared sensor processing, and range sensor processing is used as the auxiliary sensor processing.