Method and apparatus for a verbo-manual gesture interface
First Claim
1. A verbo-manual gesture interface system, comprising:
- a manipulandum adapted to be movable according to a multi-stroke manual gesture imparted by a user, the multi-stroke manual gesture comprising a plurality of characteristic manual strokes;
a sensor adapted to detect each of the plurality of characteristic manual strokes within the multi-stroke manual gesture imparted to the manipulandum and to generate a sensor signal representing the timing of each of the plurality of characteristic manual strokes within the multi stroke manual gesture;
a microphone adapted to detect a spoken phrase uttered by the user, the spoken phrase including a plurality of uttered words spoken by the user and to generate an utterance signal representing the timing of each of the plurality of uttered words within the spoken phrase; and
a control system adapted to receive the generated sensor signal and the generated utterance signal and to transmit a command signal to at least one electronic device, the command signal being dependent, at least in part, upon a predetermined temporal relationship between a particular one of the plurality of characteristic manual strokes within the multi-stroke manual gesture and a particular one of the plurality of uttered words within the spoken phrase,wherein the predetermined temporal relationship requires that the particular one of the plurality of characteristic manual strokes within the multi-stroke manual gesture is imparted with substantial simultaneity with the particular one of the plurality of uttered word within the spoken phrase.
3 Assignments
0 Petitions
Accused Products
Abstract
An interface system including a manipulandum adapted to be moveable according to a manual gesture imparted by the user; a sensor adapted to detect a characteristic of the manual gesture imparted to the manipulandum and to generate a sensor signal representing the detected characteristic of the manual gesture; a microphone adapted to detect a characteristic of an utterance spoken by the user and to generate an audio signal representing the detected characteristic of the spoken utterance; and a control system adapted receive the generated sensor and audio signals and to transmit a command signal to an electronic device via a communication link, the command signal being based on the generated sensor and audio signals and the time synchronization between them.
-
Citations
12 Claims
-
1. A verbo-manual gesture interface system, comprising:
-
a manipulandum adapted to be movable according to a multi-stroke manual gesture imparted by a user, the multi-stroke manual gesture comprising a plurality of characteristic manual strokes; a sensor adapted to detect each of the plurality of characteristic manual strokes within the multi-stroke manual gesture imparted to the manipulandum and to generate a sensor signal representing the timing of each of the plurality of characteristic manual strokes within the multi stroke manual gesture; a microphone adapted to detect a spoken phrase uttered by the user, the spoken phrase including a plurality of uttered words spoken by the user and to generate an utterance signal representing the timing of each of the plurality of uttered words within the spoken phrase; and a control system adapted to receive the generated sensor signal and the generated utterance signal and to transmit a command signal to at least one electronic device, the command signal being dependent, at least in part, upon a predetermined temporal relationship between a particular one of the plurality of characteristic manual strokes within the multi-stroke manual gesture and a particular one of the plurality of uttered words within the spoken phrase, wherein the predetermined temporal relationship requires that the particular one of the plurality of characteristic manual strokes within the multi-stroke manual gesture is imparted with substantial simultaneity with the particular one of the plurality of uttered word within the spoken phrase. - View Dependent Claims (2, 3, 4)
-
-
5. A verbo-manual interface method, comprising the steps of:
-
detecting each of a plurality of characteristic manual strokes within a multi-stroke manual gesture imparted to a manipulandum by a user; generating a sensor signal representing the timing of each of the plurality of characteristic manual strokes within the multi-stroke manual gesture; detecting a spoken phrase uttered by a user, the spoken phrase including a plurality of uttered words spoken by the user; generating an utterance signal representing the timing of each of the plurality of uttered words within the spoken phrase; and controlling at least one electronic device in dependence upon a predetermined temporal relationship between a particular one of the plurality of characteristic manual strokes within the multi-stroke manual gesture and a particular one of the plurality of uttered words within the spoken phrase, wherein the controlling step comprises the predetermined temporal relationship requiring that the first manual stroke is imparted with substantial simultaneity with the first spoken word and that the second manual stroke is imparted with a substantial simultaneity with the second spoken word. - View Dependent Claims (6, 7, 8, 9, 10, 11, 12)
-
Specification