Gestural motion and speech interface control method for 3d audio-video-data navigation on handheld devices
First Claim
1. A system for navigation on a handheld device comprising:
- a gesture module that can receive gesture commands from a physical interface on the device or a remote rendering computer server by executing stored first computer instructions on a processor contained in the devicea motion module that produces motion commands by detecting motion of the device in 3-dimensional space by executing second stored computer instructions on said processor that read and process data from motion sensors contained in the device;
a speech module that can decode voice commands spoken through a microphone contained in the device by executing third stored computer instructions on said processor;
a graphics driver adapted to display an image on a display screen attached to the device;
data from each of said gesture module, said motion module and said speech module combined in a prevalence module that executes fourth stored computer instructions on said processor to prioritize a gesture command over a motion command, and prioritize a motion command over a speech command, said prevalence module adapted to issue an action command based on either a gesture, a motion of the device, or a speech command to said graphics driver to cause said graphics driver to modify the image on the display according to said gesture, motion or speech command; and
,wherein said prevalence module transitions smoothly between gesture, motion and speech commands.
3 Assignments
0 Petitions
Accused Products
Abstract
A cognizant and adaptive method of informing a multi-modal navigation interface or a user'"'"'s intent. This provides the user with the experience of exploring an immersive representation of the processed multimedia (audio-video-data) sources available that automatically adapts to her/his fruition preference. These results are obtained by first reconciling and aligning the User and the Device'"'"'s frames of reference in tri-dimensional space and then dynamically and adaptively Smoothly Switching and/or combining both Gesture, Motion and Speech modalities. The direct consequence is a user experience that naturally adapts to the user choice of interaction and movement.
-
Citations
19 Claims
-
1. A system for navigation on a handheld device comprising:
-
a gesture module that can receive gesture commands from a physical interface on the device or a remote rendering computer server by executing stored first computer instructions on a processor contained in the device a motion module that produces motion commands by detecting motion of the device in 3-dimensional space by executing second stored computer instructions on said processor that read and process data from motion sensors contained in the device; a speech module that can decode voice commands spoken through a microphone contained in the device by executing third stored computer instructions on said processor; a graphics driver adapted to display an image on a display screen attached to the device; data from each of said gesture module, said motion module and said speech module combined in a prevalence module that executes fourth stored computer instructions on said processor to prioritize a gesture command over a motion command, and prioritize a motion command over a speech command, said prevalence module adapted to issue an action command based on either a gesture, a motion of the device, or a speech command to said graphics driver to cause said graphics driver to modify the image on the display according to said gesture, motion or speech command; and
,wherein said prevalence module transitions smoothly between gesture, motion and speech commands. - View Dependent Claims (2, 3, 4, 5, 6, 7, 8, 9, 10)
-
-
11. A method for image navigation on a handheld computer device having a touch screen, camera or joystick, a set of motion or geo-location sensors and a microphone driving a speech recognition sub-system, said device also having a 3D graphic engine on board or connected to a remote rendering device and a current device window relating to a displayed image, comprising:
-
receiving a gesture command from said touch screen, camera or joystick, and/or; receiving a motion command from said motion sensors, and/or; receiving a speech command from said speech recognition sub-system; computing static class data from a static gesture command, if present; else computing static data from a static motion command, if present;
else computing static data from a static speech command if present;computing displaced class data from a displaced gesture command, if present;
else computing displaced data from a displaced motion command, if present;
else computing displaced data from a displaced speech command if present;determining locally or remotely an update for said current device window based on said static class data and said displaced class data; commanding the 3D graphic engine locally or remotely to perform said update to the current device window for said displayed image. - View Dependent Claims (12)
-
-
13. A system for navigation on a handheld device comprising:
-
a gesture module configured to receive gesture commands from a physical interface on the device; a motion module that produces motion commands by detecting motion of the device in 3-dimensional space configured to read and process data from motion sensors contained in the device; a speech module configured to decode voice commands spoken through a microphone contained in the device; a graphics driver adapted to display an image on a display screen attached to the device; data from each of said gesture module, said motion module and said speech module combined in a prevalence module that executes stored computer instructions on said processor to prioritize a gesture command over a motion command, and prioritize a motion command over a speech command, said prevalence module adapted to issue an action command based on either a gesture, a motion of the device, or a speech command to said graphics driver to cause said graphics driver to modify the image on the display according to said gesture, motion or speech command; and
,wherein said prevalence module transitions smoothly between gesture, motion and speech commands. - View Dependent Claims (14, 15, 16, 17, 18, 19)
-
Specification